feat(controls): B2 — grounded surface checks for absence-based CRA controls #219

2026-07-21T12:41:36Z

sharang commented

2026-07-21 12:41:36 +00:00

Second slice of B (hybrid coverage): the controls violated by an absence — no rate limiting on login (cra-ai-11), no security logging (cra-ai-24), no update-signature check (cra-ai-6/29/30), etc. — have no syntactic pattern for semgrep. Instead we deterministically retrieve the code surface each governs, then let the grounded judge decide whether the control holds.

Mechanism

controls/surface.rs — bounded, deterministic retrieval (keyword + window, merged + capped per control) for cra-ai-6,11,24,27,28,29,30.
grounded_surface_findings() — retrieve surfaces → GroundedControlChecker → net-new findings, each already tagged with its control and grounded to a verbatim snippet (ground() drops anything that doesn't quote real code).
orchestrator Stage 5d, gated on breakpilot.grounded_control_checks (BREAKPILOT_GROUNDED_CHECKS, default off) — absence detection is the least deterministic path; kept off until tuned against live scans.
LUT: the 7 controls' notes now point to the gated grounded mechanism (kept needs_tooling so the coverage count stays honest until live-validated).

Local validation (real Qwen, temp 0) — correct positive and negative discrimination, with grounding + correct CWEs:

Control	Violation case	Clean case
cra-ai-11 brute-force	unprotected login → `violates`, CWE-307, grounded	flask-limiter → `false`
cra-ai-24 logging	admin delete, no log → `violates`, CWE-778, grounded	with `audit_log` → `false`

This proves the v1 judge prompt handles absence detection (it quotes the unprotected surface as the offending code). Green locally (clippy -D warnings, fmt, full suite; +4 surface tests).

Next: B3 (mark cra-ai-2,3,4,5 pure-architectural as not_code_checkable).

🤖 Generated with Claude Code

Second slice of **B** (hybrid coverage): the controls violated by an *absence* — no rate limiting on login (cra-ai-11), no security logging (cra-ai-24), no update-signature check (cra-ai-6/29/30), etc. — have no syntactic pattern for semgrep. Instead we deterministically retrieve the code **surface** each governs, then let the grounded judge decide whether the control holds. **Mechanism** - `controls/surface.rs` — bounded, deterministic retrieval (keyword + window, merged + capped per control) for cra-ai-6,11,24,27,28,29,30. - `grounded_surface_findings()` — retrieve surfaces → `GroundedControlChecker` → net-new findings, each already tagged with its control and grounded to a verbatim snippet (`ground()` drops anything that doesn't quote real code). - orchestrator **Stage 5d**, gated on `breakpilot.grounded_control_checks` (`BREAKPILOT_GROUNDED_CHECKS`, **default off**) — absence detection is the least deterministic path; kept off until tuned against live scans. - LUT: the 7 controls' notes now point to the gated grounded mechanism (kept `needs_tooling` so the coverage count stays honest until live-validated). **Local validation** (real Qwen, temp 0) — correct positive **and** negative discrimination, with grounding + correct CWEs: | Control | Violation case | Clean case | |---|---|---| | cra-ai-11 brute-force | unprotected login → `violates`, CWE-307, grounded | flask-limiter → `false` | | cra-ai-24 logging | admin delete, no log → `violates`, CWE-778, grounded | with `audit_log` → `false` | This proves the v1 judge prompt handles absence detection (it quotes the unprotected surface as the offending code). Green locally (clippy -D warnings, fmt, full suite; +4 surface tests). Next: B3 (mark cra-ai-2,3,4,5 pure-architectural as not_code_checkable). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

sharang added 1 commit 2026-07-21 12:41:37 +00:00

feat(controls): B2 — grounded surface checks for absence-based CRA controls

CI / Check (push) Skipped

CI / Check (pull_request) Successful in 5m44s

Details

CI / Detect Changes (pull_request) Skipped

Details

CI / Deploy Agent (pull_request) Skipped

Details

CI / Deploy Dashboard (pull_request) Skipped

Details

CI / Deploy Docs (pull_request) Skipped

Details

CI / Deploy MCP (pull_request) Skipped

Details

82ed4afd10

Second slice of B (hybrid coverage): the controls violated by an *absence*
(no rate limiting, no security logging, no update-signature check) have no
syntactic pattern for semgrep, so we retrieve the code surface each governs and
let the grounded judge decide whether the control holds.

- controls/surface.rs: deterministic, bounded surface retrieval (keyword +
  window, capped per control) for cra-ai-6,11,24,27,28,29,30.
- grounded_surface_findings(): retrieve surfaces -> GroundedControlChecker ->
  net-new findings, each already tagged with its control and grounded to a real
  snippet (ground() drops anything not quoting verbatim code).
- orchestrator Stage 5d, gated on breakpilot.grounded_control_checks
  (BREAKPILOT_GROUNDED_CHECKS, default off) — absence detection is the least
  deterministic path, kept off until tuned against live scans.
- LUT: the 7 controls' notes now point to the gated grounded mechanism (kept
  needs_tooling; coverage stays honest until live-validated).

Local validation (real Qwen, temp 0), correct positive+negative discrimination:
  cra-ai-11 unprotected login  -> violates, CWE-307, grounded; protected -> false
  cra-ai-24 unlogged admin del -> violates, CWE-778, grounded; logged   -> false

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

sharang merged commit 4ef257bfe2 into main

2026-07-21 12:48:13 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sharang/compliance-scanner-agent#219