CI / Check (push) Skipped
CI / Check (pull_request) Successful in 5m59s
CI / Detect Changes (pull_request) Skipped
CI / Deploy Agent (pull_request) Skipped
CI / Deploy Dashboard (pull_request) Skipped
CI / Deploy Docs (pull_request) Skipped
CI / Deploy MCP (pull_request) Skipped
After the deterministic tools run, the orchestrator's new control_triage stage stamps each finding with the compliance control(s) it is evidence for and flags control false positives. triage_repo_findings builds control specs from the ingested OSCAL catalog, reads a code window per finding, and runs ControlTriage (control-map LUT -> grounded judge). Adds Finding.control_refs (serde default); the ground gate stamps it. Opt-in via BREAKPILOT_BASE_URL. 2 region tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
115 lines
3.7 KiB
Rust
115 lines
3.7 KiB
Rust
//! The grounded control checker: judge each candidate region for a control, then
|
|
//! keep only the verdicts that survive the grounding gate.
|
|
//!
|
|
//! Generic over [`ControlJudge`] so tests drive it with a deterministic stub —
|
|
//! the whole recognize → ground path is then exercised without an LLM. With the
|
|
//! real judge, determinism comes from temperature 0 plus the gate.
|
|
|
|
use compliance_core::control_check::{ground, CandidateRegion, ControlCheckSpec};
|
|
use compliance_core::models::Finding;
|
|
|
|
use super::judge::ControlJudge;
|
|
|
|
/// Runs a [`ControlJudge`] over candidate regions and grounds the results.
|
|
pub struct GroundedControlChecker<J> {
|
|
judge: J,
|
|
}
|
|
|
|
impl<J: ControlJudge> GroundedControlChecker<J> {
|
|
pub fn new(judge: J) -> Self {
|
|
Self { judge }
|
|
}
|
|
|
|
/// Judge every candidate region for `spec` and return the grounded findings.
|
|
/// A verdict that doesn't quote real code in its region is dropped by
|
|
/// [`ground`], so nothing fabricated reaches the caller.
|
|
pub async fn check(
|
|
&self,
|
|
spec: &ControlCheckSpec,
|
|
regions: &[CandidateRegion],
|
|
repo_id: &str,
|
|
) -> Vec<Finding> {
|
|
let mut findings = Vec::new();
|
|
for region in regions {
|
|
let verdict = self.judge.judge(spec, region).await;
|
|
if let Some(finding) = ground(spec, region, &verdict, repo_id) {
|
|
findings.push(finding);
|
|
}
|
|
}
|
|
findings
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use compliance_core::control_check::LlmVerdict;
|
|
use compliance_core::models::finding::Severity;
|
|
|
|
/// Deterministic stub: returns a fixed verdict for every region, so the
|
|
/// recognize → ground composition is tested without an LLM.
|
|
struct StubJudge {
|
|
verdict: LlmVerdict,
|
|
}
|
|
|
|
impl ControlJudge for StubJudge {
|
|
async fn judge(&self, _spec: &ControlCheckSpec, _region: &CandidateRegion) -> LlmVerdict {
|
|
self.verdict.clone()
|
|
}
|
|
}
|
|
|
|
fn spec() -> ControlCheckSpec {
|
|
ControlCheckSpec {
|
|
control_id: "cra-ai-8".into(),
|
|
title: "No default passwords".into(),
|
|
requirement: "No default credentials".into(),
|
|
default_cwe: Some("CWE-798".into()),
|
|
severity: Severity::High,
|
|
}
|
|
}
|
|
|
|
fn region(content: &str) -> CandidateRegion {
|
|
CandidateRegion {
|
|
file: "src/auth.py".into(),
|
|
start_line: 1,
|
|
content: content.into(),
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn keeps_grounded_and_drops_ungrounded() {
|
|
let checker = GroundedControlChecker::new(StubJudge {
|
|
verdict: LlmVerdict {
|
|
violates: true,
|
|
snippet: "PASSWORD = \"admin\"".into(),
|
|
cwe: None,
|
|
confidence: 0.9,
|
|
},
|
|
});
|
|
let regions = vec![
|
|
region("x = 1\nPASSWORD = \"admin\"\n"), // quotes real code → grounded
|
|
region("totally unrelated code\n"), // snippet absent → dropped
|
|
];
|
|
let findings = checker.check(&spec(), ®ions, "repo").await;
|
|
assert_eq!(findings.len(), 1);
|
|
assert_eq!(findings[0].control_refs, vec!["cra-ai-8".to_string()]);
|
|
assert_eq!(findings[0].line_number, Some(2));
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn non_violation_yields_nothing() {
|
|
let checker = GroundedControlChecker::new(StubJudge {
|
|
verdict: LlmVerdict {
|
|
violates: false,
|
|
snippet: String::new(),
|
|
cwe: None,
|
|
confidence: 0.0,
|
|
},
|
|
});
|
|
let findings = checker
|
|
.check(&spec(), &[region("PASSWORD = \"admin\"\n")], "repo")
|
|
.await;
|
|
assert!(findings.is_empty());
|
|
}
|
|
}
|