sharang/compliance-scanner-agent
1
0
Fork 0
Code Issues 79 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

STORY · Nightly agent regression against the demo targets #188

New Issue
Open
opened 2026-07-16 22:16:33 +00:00 by sharang · 0 comments
sharang commented 2026-07-16 22:16:33 +00:00
Owner
Copy Link
Copy Source

As a Certifai maintainer I want a nightly run of the agent against the
demo targets with assertions on the expected output so that pipeline
regressions are caught automatically.

Part of #186. Depends on the demo-target fixtures story.

Acceptance criteria

  • Scheduled job (CI cron or agent SCAN_SCHEDULE) runs all demo targets nightly.
  • Golden baselines assert expected finding rule_ids, SBOM component counts,
    and CVE presence per target.
  • Failures raise an alert via the existing notification path.
**As a** Certifai maintainer **I want** a nightly run of the agent against the demo targets with assertions on the expected output **so that** pipeline regressions are caught automatically. Part of #186. Depends on the demo-target fixtures story. ## Acceptance criteria - [ ] Scheduled job (CI cron or agent `SCAN_SCHEDULE`) runs all demo targets nightly. - [ ] Golden baselines assert expected finding `rule_id`s, SBOM component counts, and CVE presence per target. - [ ] Failures raise an alert via the existing notification path.
sharang added the enhancementonboarding labels 2026-07-16 22:16:33 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P0-demo
CERTifAI P0 demo build tickets (CS-*)
 bug
critical
enhancement
epic
Tracking issue spanning multiple sub-issues
 high
infrastructure
onboarding
Multi-target onboarding wizard redesign
 performance
security
v0.3.0
No labels enhancement onboarding
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
Benjamin_Boenisch sharang (Sharang Parnerkar)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sharang/compliance-scanner-agent#188