Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
563d8afb2a |
+8
-27
@@ -7,13 +7,6 @@ on:
|
|||||||
pull_request:
|
pull_request:
|
||||||
|
|
||||||
env:
|
env:
|
||||||
# registry + cosign creds via env, NOT inline ${{ }}: the Harbor robot
|
|
||||||
# username contains '$', which sh expands when interpolated into the
|
|
||||||
# script (robot$ci-push -> robot-push) => docker login unauthorized.
|
|
||||||
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
|
||||||
COSIGN_KEY: ${{ secrets.COSIGN_KEY }}
|
|
||||||
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
|
|
||||||
CARGO_TERM_COLOR: always
|
CARGO_TERM_COLOR: always
|
||||||
RUSTFLAGS: "-D warnings"
|
RUSTFLAGS: "-D warnings"
|
||||||
# Compile cache: sccache -> Hetzner S3 (breakpilot-sccache), runner-independent
|
# Compile cache: sccache -> Hetzner S3 (breakpilot-sccache), runner-independent
|
||||||
@@ -213,14 +206,11 @@ jobs:
|
|||||||
apk add --no-cache git curl openssl
|
apk add --no-cache git curl openssl
|
||||||
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
||||||
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
||||||
IMAGE=repo.meghsakha.com/certifai/compliance-agent
|
IMAGE=registry.meghsakha.com/compliance-agent
|
||||||
echo "$REGISTRY_PASSWORD" | docker login repo.meghsakha.com -u "$REGISTRY_USERNAME" --password-stdin
|
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
|
||||||
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
|
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
|
||||||
-f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
-f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
||||||
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
||||||
{ command -v cosign >/dev/null 2>&1 || curl -sSfLo /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64 || wget -qO /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64; } || echo "::warning::cosign fetch failed"
|
|
||||||
chmod +x /usr/local/bin/cosign 2>/dev/null || true
|
|
||||||
cosign sign --yes --key env://COSIGN_KEY "$IMAGE:latest" || echo "::warning::cosign failed"
|
|
||||||
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy agent"}}' "${GITHUB_SHA}")
|
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy agent"}}' "${GITHUB_SHA}")
|
||||||
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
||||||
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
||||||
@@ -240,14 +230,11 @@ jobs:
|
|||||||
apk add --no-cache git curl openssl
|
apk add --no-cache git curl openssl
|
||||||
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
||||||
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
||||||
IMAGE=repo.meghsakha.com/certifai/compliance-dashboard
|
IMAGE=registry.meghsakha.com/compliance-dashboard
|
||||||
echo "$REGISTRY_PASSWORD" | docker login repo.meghsakha.com -u "$REGISTRY_USERNAME" --password-stdin
|
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
|
||||||
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
|
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
|
||||||
-f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
-f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
||||||
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
||||||
{ command -v cosign >/dev/null 2>&1 || curl -sSfLo /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64 || wget -qO /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64; } || echo "::warning::cosign fetch failed"
|
|
||||||
chmod +x /usr/local/bin/cosign 2>/dev/null || true
|
|
||||||
cosign sign --yes --key env://COSIGN_KEY "$IMAGE:latest" || echo "::warning::cosign failed"
|
|
||||||
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy dashboard"}}' "${GITHUB_SHA}")
|
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy dashboard"}}' "${GITHUB_SHA}")
|
||||||
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
||||||
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
||||||
@@ -265,13 +252,10 @@ jobs:
|
|||||||
apk add --no-cache git curl openssl
|
apk add --no-cache git curl openssl
|
||||||
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
||||||
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
||||||
IMAGE=repo.meghsakha.com/certifai/compliance-docs
|
IMAGE=registry.meghsakha.com/compliance-docs
|
||||||
echo "$REGISTRY_PASSWORD" | docker login repo.meghsakha.com -u "$REGISTRY_USERNAME" --password-stdin
|
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
|
||||||
docker build -f Dockerfile.docs -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
docker build -f Dockerfile.docs -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
||||||
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
||||||
{ command -v cosign >/dev/null 2>&1 || curl -sSfLo /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64 || wget -qO /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64; } || echo "::warning::cosign fetch failed"
|
|
||||||
chmod +x /usr/local/bin/cosign 2>/dev/null || true
|
|
||||||
cosign sign --yes --key env://COSIGN_KEY "$IMAGE:latest" || echo "::warning::cosign failed"
|
|
||||||
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy docs"}}' "${GITHUB_SHA}")
|
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy docs"}}' "${GITHUB_SHA}")
|
||||||
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
||||||
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
||||||
@@ -291,14 +275,11 @@ jobs:
|
|||||||
apk add --no-cache git curl openssl
|
apk add --no-cache git curl openssl
|
||||||
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
||||||
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
|
||||||
IMAGE=repo.meghsakha.com/certifai/compliance-mcp
|
IMAGE=registry.meghsakha.com/compliance-mcp
|
||||||
echo "$REGISTRY_PASSWORD" | docker login repo.meghsakha.com -u "$REGISTRY_USERNAME" --password-stdin
|
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
|
||||||
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
|
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
|
||||||
-f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
-f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
|
||||||
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
|
||||||
{ command -v cosign >/dev/null 2>&1 || curl -sSfLo /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64 || wget -qO /usr/local/bin/cosign https://github.com/sigstore/cosign/releases/download/v2.4.3/cosign-linux-amd64; } || echo "::warning::cosign fetch failed"
|
|
||||||
chmod +x /usr/local/bin/cosign 2>/dev/null || true
|
|
||||||
cosign sign --yes --key env://COSIGN_KEY "$IMAGE:latest" || echo "::warning::cosign failed"
|
|
||||||
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy mcp"}}' "${GITHUB_SHA}")
|
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy mcp"}}' "${GITHUB_SHA}")
|
||||||
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
|
||||||
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
|
||||||
|
|||||||
@@ -103,8 +103,5 @@ fn load_breakpilot_config() -> BreakpilotConfig {
|
|||||||
semantic_mapping: env_var_opt("BREAKPILOT_SEMANTIC_MAPPING")
|
semantic_mapping: env_var_opt("BREAKPILOT_SEMANTIC_MAPPING")
|
||||||
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
|
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
|
||||||
.unwrap_or(d.semantic_mapping),
|
.unwrap_or(d.semantic_mapping),
|
||||||
grounded_control_checks: env_var_opt("BREAKPILOT_GROUNDED_CHECKS")
|
|
||||||
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
|
|
||||||
.unwrap_or(d.grounded_control_checks),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,13 +11,12 @@ mod judge;
|
|||||||
mod oscal_provider;
|
mod oscal_provider;
|
||||||
mod scan_triage;
|
mod scan_triage;
|
||||||
mod semantic;
|
mod semantic;
|
||||||
mod surface;
|
|
||||||
mod triage;
|
mod triage;
|
||||||
|
|
||||||
pub use checker::GroundedControlChecker;
|
pub use checker::GroundedControlChecker;
|
||||||
pub use index::ControlIndex;
|
pub use index::ControlIndex;
|
||||||
pub use judge::{ControlJudge, LlmControlJudge, PROMPT_VERSION};
|
pub use judge::{ControlJudge, LlmControlJudge, PROMPT_VERSION};
|
||||||
pub use oscal_provider::OscalControlsProvider;
|
pub use oscal_provider::OscalControlsProvider;
|
||||||
pub use scan_triage::{grounded_surface_findings, semantic_stamp_findings, triage_repo_findings};
|
pub use scan_triage::{semantic_stamp_findings, triage_repo_findings};
|
||||||
pub use semantic::SemanticControlChecker;
|
pub use semantic::SemanticControlChecker;
|
||||||
pub use triage::{ControlTriage, TriageOutcome};
|
pub use triage::{ControlTriage, TriageOutcome};
|
||||||
|
|||||||
@@ -16,10 +16,9 @@ use compliance_core::models::onboarding::ComplianceFramework;
|
|||||||
use compliance_core::AgentConfig;
|
use compliance_core::AgentConfig;
|
||||||
use control_map::ControlMap;
|
use control_map::ControlMap;
|
||||||
|
|
||||||
use super::surface;
|
|
||||||
use super::{
|
use super::{
|
||||||
ControlIndex, ControlTriage, GroundedControlChecker, LlmControlJudge, OscalControlsProvider,
|
ControlIndex, ControlTriage, LlmControlJudge, OscalControlsProvider, SemanticControlChecker,
|
||||||
SemanticControlChecker, TriageOutcome,
|
TriageOutcome,
|
||||||
};
|
};
|
||||||
use crate::llm::LlmClient;
|
use crate::llm::LlmClient;
|
||||||
|
|
||||||
@@ -106,50 +105,6 @@ async fn build_specs(provider: &OscalControlsProvider) -> HashMap<String, Contro
|
|||||||
specs
|
specs
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Absence-based control pass (the grounded half of the hybrid coverage): for each
|
|
||||||
/// control with a [`surface`] definition, deterministically retrieve the code
|
|
||||||
/// surfaces it governs (login routes, logging setup, update/download code) and have
|
|
||||||
/// the grounded judge decide whether the control holds there. Returns net-new
|
|
||||||
/// findings, each already tagged with its control and grounded to a real snippet.
|
|
||||||
///
|
|
||||||
/// The orchestrator runs this when `breakpilot.grounded_control_checks` is set
|
|
||||||
/// (on by default). Validated live; it covers the 8 absence-based CRA controls
|
|
||||||
/// (the judge decides presence/absence, grounded to a real snippet).
|
|
||||||
pub async fn grounded_surface_findings(
|
|
||||||
config: &AgentConfig,
|
|
||||||
llm: Arc<LlmClient>,
|
|
||||||
repo_path: &Path,
|
|
||||||
repo_id: &str,
|
|
||||||
) -> Vec<Finding> {
|
|
||||||
let Some(base_url) = config.breakpilot.base_url.clone() else {
|
|
||||||
return Vec::new();
|
|
||||||
};
|
|
||||||
let provider = OscalControlsProvider::new(
|
|
||||||
reqwest::Client::new(),
|
|
||||||
base_url,
|
|
||||||
config.breakpilot.token.clone(),
|
|
||||||
&config.breakpilot.snapshot_dir,
|
|
||||||
);
|
|
||||||
let specs = build_specs(&provider).await;
|
|
||||||
if specs.is_empty() {
|
|
||||||
return Vec::new();
|
|
||||||
}
|
|
||||||
let checker = GroundedControlChecker::new(LlmControlJudge::new(llm));
|
|
||||||
|
|
||||||
let mut out = Vec::new();
|
|
||||||
for surf in surface::SURFACES {
|
|
||||||
let Some(spec) = specs.get(surf.control_id) else {
|
|
||||||
continue; // catalog doesn't carry this control
|
|
||||||
};
|
|
||||||
let regions = surface::retrieve(repo_path, surf.terms);
|
|
||||||
if regions.is_empty() {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
out.extend(checker.check(spec, ®ions, repo_id).await);
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Read a window of lines around `line` (1-based) from `repo_path/file`.
|
/// Read a window of lines around `line` (1-based) from `repo_path/file`.
|
||||||
fn fetch_region(repo_path: &Path, file: &str, line: u32) -> Option<CandidateRegion> {
|
fn fetch_region(repo_path: &Path, file: &str, line: u32) -> Option<CandidateRegion> {
|
||||||
let content = std::fs::read_to_string(repo_path.join(file)).ok()?;
|
let content = std::fs::read_to_string(repo_path.join(file)).ok()?;
|
||||||
@@ -173,10 +128,11 @@ fn fetch_region(repo_path: &Path, file: &str, line: u32) -> Option<CandidateRegi
|
|||||||
/// ~13.6k master-control corpus (which has no CWE to LUT on). Returns the number
|
/// ~13.6k master-control corpus (which has no CWE to LUT on). Returns the number
|
||||||
/// of findings that gained a master-control ref.
|
/// of findings that gained a master-control ref.
|
||||||
///
|
///
|
||||||
/// The orchestrator runs this when `breakpilot.semantic_mapping` is set (on by
|
/// Gated: the orchestrator runs this only when `breakpilot.semantic_mapping` is
|
||||||
/// default). The control embedding index is built once and cached to
|
/// set (default off, flipped on once the master-controls catalog is live). The
|
||||||
/// `snapshot_dir` keyed by corpus hash ([`ControlIndex::load_or_build`]), so only
|
/// control embedding index is built once and cached to `snapshot_dir` keyed by
|
||||||
/// the first scan after a catalog change pays the embedding cost.
|
/// corpus hash ([`ControlIndex::load_or_build`]), so only the first scan after a
|
||||||
|
/// catalog change pays the embedding cost.
|
||||||
pub async fn semantic_stamp_findings(
|
pub async fn semantic_stamp_findings(
|
||||||
config: &AgentConfig,
|
config: &AgentConfig,
|
||||||
llm: Arc<LlmClient>,
|
llm: Arc<LlmClient>,
|
||||||
@@ -233,22 +189,13 @@ pub async fn semantic_stamp_findings(
|
|||||||
let Some(region) = fetch_region(repo_path, &file, line) else {
|
let Some(region) = fetch_region(repo_path, &file, line) else {
|
||||||
continue;
|
continue;
|
||||||
};
|
};
|
||||||
// Retrieve on the finding's intent + the code, not the region alone: two
|
let region_emb = match llm.embed(vec![region.content.clone()]).await {
|
||||||
// findings in one file share overlapping windows and otherwise embed alike,
|
|
||||||
// collapsing onto the same controls. The finding's title/description carry
|
|
||||||
// the discriminating signal (e.g. "brute-force protection" vs "weak hash").
|
|
||||||
// The raw `region` still goes to the judge for snippet grounding.
|
|
||||||
let query = format!(
|
|
||||||
"{}\n{}\n\n{}",
|
|
||||||
finding.title, finding.description, region.content
|
|
||||||
);
|
|
||||||
let query_emb = match llm.embed(vec![query]).await {
|
|
||||||
Ok(mut embs) => match embs.pop() {
|
Ok(mut embs) => match embs.pop() {
|
||||||
Some(v) => v,
|
Some(v) => v,
|
||||||
None => continue,
|
None => continue,
|
||||||
},
|
},
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
tracing::warn!(error = %e, "query embed failed; skipping finding");
|
tracing::warn!(error = %e, "region embed failed; skipping finding");
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
@@ -256,7 +203,7 @@ pub async fn semantic_stamp_findings(
|
|||||||
.check(
|
.check(
|
||||||
&index,
|
&index,
|
||||||
®ion,
|
®ion,
|
||||||
&query_emb,
|
®ion_emb,
|
||||||
SEMANTIC_TOP_K,
|
SEMANTIC_TOP_K,
|
||||||
&finding.repo_id,
|
&finding.repo_id,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -22,20 +22,18 @@ impl<J: ControlJudge> SemanticControlChecker<J> {
|
|||||||
Self { judge }
|
Self { judge }
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Map a code region to the controls it violates. `query_embedding` is the
|
/// Map a code region to the controls it violates. `region_embedding` is the
|
||||||
/// caller-supplied retrieval embedding — typically the finding's intent
|
/// region's embedding (the caller computes it via the LLM); the top-`k`
|
||||||
/// (title/description) plus the region, so retrieval keys on what the finding
|
/// nearest controls in `index` are judged and grounded.
|
||||||
/// is *about*, not just the ambient code. The top-`k` nearest controls in
|
|
||||||
/// `index` are then judged against the raw `region` and grounded.
|
|
||||||
pub async fn check(
|
pub async fn check(
|
||||||
&self,
|
&self,
|
||||||
index: &ControlIndex,
|
index: &ControlIndex,
|
||||||
region: &CandidateRegion,
|
region: &CandidateRegion,
|
||||||
query_embedding: &[f64],
|
region_embedding: &[f64],
|
||||||
k: usize,
|
k: usize,
|
||||||
repo_id: &str,
|
repo_id: &str,
|
||||||
) -> Vec<Finding> {
|
) -> Vec<Finding> {
|
||||||
let candidates = index.nearest(query_embedding, k);
|
let candidates = index.nearest(region_embedding, k);
|
||||||
let mut findings = Vec::new();
|
let mut findings = Vec::new();
|
||||||
for spec in &candidates {
|
for spec in &candidates {
|
||||||
let verdict = self.judge.judge(spec, region).await;
|
let verdict = self.judge.judge(spec, region).await;
|
||||||
|
|||||||
@@ -1,258 +0,0 @@
|
|||||||
//! Surface retrieval for absence-based controls.
|
|
||||||
//!
|
|
||||||
//! Some CRA controls are violated by an *absence* — no rate limiting on login, no
|
|
||||||
//! security logging, no signature check on an update — so there's no offending
|
|
||||||
//! pattern for semgrep to match. Instead we deterministically locate the code
|
|
||||||
//! *surface* the control governs (a login route, a logging setup, update/download
|
|
||||||
//! code) by identifier/route terms, then hand each surface region to the grounded
|
|
||||||
//! judge, which decides whether the control is satisfied there. The resulting
|
|
||||||
//! finding grounds to the surface snippet, so nothing fabricated survives.
|
|
||||||
//!
|
|
||||||
//! Retrieval is intentionally cheap and bounded: keyword match + a fixed window,
|
|
||||||
//! capped per control to keep the downstream LLM cost predictable.
|
|
||||||
|
|
||||||
use std::path::Path;
|
|
||||||
|
|
||||||
use compliance_core::control_check::CandidateRegion;
|
|
||||||
|
|
||||||
/// An absence-based control and the case-insensitive terms that mark the code
|
|
||||||
/// surface it governs.
|
|
||||||
pub struct Surface {
|
|
||||||
pub control_id: &'static str,
|
|
||||||
pub terms: &'static [&'static str],
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The absence-based CRA controls we retrieve surfaces for — the grounded half of
|
|
||||||
/// the hybrid coverage (the pattern-expressible half is custom semgrep rules).
|
|
||||||
pub const SURFACES: &[Surface] = &[
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-6", // Integritaetspruefung
|
|
||||||
terms: &[
|
|
||||||
"checksum",
|
|
||||||
"sha256",
|
|
||||||
"signature",
|
|
||||||
"hmac",
|
|
||||||
"integrity",
|
|
||||||
"verify",
|
|
||||||
],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-11", // Brute-Force-Schutz
|
|
||||||
terms: &[
|
|
||||||
"login",
|
|
||||||
"signin",
|
|
||||||
"authenticate",
|
|
||||||
"/auth",
|
|
||||||
"password",
|
|
||||||
"ratelimit",
|
|
||||||
],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-12", // Rollenbasierte Autorisierung (RBAC)
|
|
||||||
terms: &[
|
|
||||||
"authorize",
|
|
||||||
"permission",
|
|
||||||
"role",
|
|
||||||
"rbac",
|
|
||||||
"require_role",
|
|
||||||
"has_role",
|
|
||||||
],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-24", // Security-Logging
|
|
||||||
terms: &["login", "authorize", "permission", "role", "admin", "audit"],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-27", // Log-Integritaet und -Aufbewahrung
|
|
||||||
terms: &["logging", "logger", "getlogger", "audit_log"],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-28", // Sichere Update-Mechanismen
|
|
||||||
terms: &["update", "upgrade", "download", "firmware"],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-29", // Update-Authentizitaet
|
|
||||||
terms: &["update", "signature", "verify", "pubkey", "certificate"],
|
|
||||||
},
|
|
||||||
Surface {
|
|
||||||
control_id: "cra-ai-30", // Update-Integritaet
|
|
||||||
terms: &["update", "checksum", "digest", "integrity", "verify"],
|
|
||||||
},
|
|
||||||
];
|
|
||||||
|
|
||||||
/// Source file extensions worth reading (skip binaries/assets/lockfiles).
|
|
||||||
const CODE_EXTS: &[&str] = &[
|
|
||||||
"py", "js", "ts", "tsx", "jsx", "go", "java", "rb", "php", "rs", "cs", "kt",
|
|
||||||
];
|
|
||||||
/// Directories never worth walking.
|
|
||||||
const SKIP_DIRS: &[&str] = &[
|
|
||||||
".git",
|
|
||||||
"node_modules",
|
|
||||||
"target",
|
|
||||||
"vendor",
|
|
||||||
".venv",
|
|
||||||
"__pycache__",
|
|
||||||
"dist",
|
|
||||||
"build",
|
|
||||||
];
|
|
||||||
/// Lines of context on each side of a hit.
|
|
||||||
const WINDOW: usize = 6;
|
|
||||||
/// Cap on regions per control, to bound downstream LLM calls.
|
|
||||||
const MAX_REGIONS_PER_CONTROL: usize = 8;
|
|
||||||
/// Skip files larger than this (generated/minified).
|
|
||||||
const MAX_FILE_BYTES: u64 = 512 * 1024;
|
|
||||||
|
|
||||||
/// Deterministically retrieve up to [`MAX_REGIONS_PER_CONTROL`] code regions in
|
|
||||||
/// `repo_path` whose lines mention any of `terms`. Hits close together within a
|
|
||||||
/// file are merged into one region; results are capped to bound LLM cost.
|
|
||||||
pub fn retrieve(repo_path: &Path, terms: &[&str]) -> Vec<CandidateRegion> {
|
|
||||||
let lowered: Vec<String> = terms.iter().map(|t| t.to_lowercase()).collect();
|
|
||||||
let mut regions = Vec::new();
|
|
||||||
for entry in walk(repo_path) {
|
|
||||||
if regions.len() >= MAX_REGIONS_PER_CONTROL {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
let path = entry.path();
|
|
||||||
if !has_code_ext(path) {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
let Ok(meta) = entry.metadata() else { continue };
|
|
||||||
if !meta.is_file() || meta.len() > MAX_FILE_BYTES {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
let Ok(content) = std::fs::read_to_string(path) else {
|
|
||||||
continue;
|
|
||||||
};
|
|
||||||
let rel = path
|
|
||||||
.strip_prefix(repo_path)
|
|
||||||
.unwrap_or(path)
|
|
||||||
.to_string_lossy()
|
|
||||||
.to_string();
|
|
||||||
let lines: Vec<&str> = content.lines().collect();
|
|
||||||
let hits: Vec<usize> = lines
|
|
||||||
.iter()
|
|
||||||
.enumerate()
|
|
||||||
.filter(|(_, line)| {
|
|
||||||
let ll = line.to_lowercase();
|
|
||||||
lowered.iter().any(|t| ll.contains(t.as_str()))
|
|
||||||
})
|
|
||||||
.map(|(i, _)| i)
|
|
||||||
.collect();
|
|
||||||
for center in merge_centers(&hits) {
|
|
||||||
if regions.len() >= MAX_REGIONS_PER_CONTROL {
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
let start = center.saturating_sub(WINDOW);
|
|
||||||
let end = (center + WINDOW + 1).min(lines.len());
|
|
||||||
regions.push(CandidateRegion {
|
|
||||||
file: rel.clone(),
|
|
||||||
start_line: (start as u32) + 1,
|
|
||||||
content: lines[start..end].join("\n"),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
regions
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Collapse ascending hit indices that fall within one window into a single
|
|
||||||
/// representative center, so overlapping regions aren't judged repeatedly.
|
|
||||||
fn merge_centers(hits: &[usize]) -> Vec<usize> {
|
|
||||||
let mut out: Vec<usize> = Vec::new();
|
|
||||||
for &h in hits {
|
|
||||||
match out.last() {
|
|
||||||
Some(&last) if h.saturating_sub(last) <= WINDOW => {}
|
|
||||||
_ => out.push(h),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
fn has_code_ext(path: &Path) -> bool {
|
|
||||||
path.extension()
|
|
||||||
.and_then(|e| e.to_str())
|
|
||||||
.is_some_and(|e| CODE_EXTS.contains(&e))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn walk(root: &Path) -> Vec<walkdir::DirEntry> {
|
|
||||||
walkdir::WalkDir::new(root)
|
|
||||||
.into_iter()
|
|
||||||
.filter_entry(|e| {
|
|
||||||
let name = e.file_name().to_string_lossy();
|
|
||||||
!SKIP_DIRS.contains(&name.as_ref())
|
|
||||||
})
|
|
||||||
.filter_map(|e| e.ok())
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
#[allow(clippy::unwrap_used)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
fn write(dir: &Path, rel: &str, body: &str) {
|
|
||||||
let p = dir.join(rel);
|
|
||||||
if let Some(parent) = p.parent() {
|
|
||||||
std::fs::create_dir_all(parent).unwrap();
|
|
||||||
}
|
|
||||||
std::fs::write(p, body).unwrap();
|
|
||||||
}
|
|
||||||
|
|
||||||
fn terms_for(control_id: &str) -> &'static [&'static str] {
|
|
||||||
SURFACES
|
|
||||||
.iter()
|
|
||||||
.find(|s| s.control_id == control_id)
|
|
||||||
.unwrap()
|
|
||||||
.terms
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn surfaces_cover_the_absence_based_controls() {
|
|
||||||
assert_eq!(SURFACES.len(), 8);
|
|
||||||
for id in [
|
|
||||||
"cra-ai-6",
|
|
||||||
"cra-ai-11",
|
|
||||||
"cra-ai-12",
|
|
||||||
"cra-ai-24",
|
|
||||||
"cra-ai-27",
|
|
||||||
"cra-ai-28",
|
|
||||||
"cra-ai-29",
|
|
||||||
"cra-ai-30",
|
|
||||||
] {
|
|
||||||
assert!(SURFACES.iter().any(|s| s.control_id == id), "{id} missing");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn retrieves_matching_region_with_context() {
|
|
||||||
let dir = std::env::temp_dir().join(format!("surface-{}", uuid::Uuid::new_v4()));
|
|
||||||
std::fs::create_dir_all(&dir).unwrap();
|
|
||||||
write(
|
|
||||||
&dir,
|
|
||||||
"app/auth.py",
|
|
||||||
"import x\n\n\n\n\n\n\ndef login(u, p):\n return check(u, p)\n",
|
|
||||||
);
|
|
||||||
let regions = retrieve(&dir, terms_for("cra-ai-11"));
|
|
||||||
assert_eq!(regions.len(), 1);
|
|
||||||
assert!(regions[0].content.contains("def login"));
|
|
||||||
assert_eq!(regions[0].file, "app/auth.py");
|
|
||||||
let _ = std::fs::remove_dir_all(&dir);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn skips_non_code_and_vendored() {
|
|
||||||
let dir = std::env::temp_dir().join(format!("surface-{}", uuid::Uuid::new_v4()));
|
|
||||||
std::fs::create_dir_all(&dir).unwrap();
|
|
||||||
write(&dir, "README.md", "login and password and audit\n"); // not code ext
|
|
||||||
write(&dir, "node_modules/pkg/index.js", "function login() {}\n"); // vendored
|
|
||||||
assert!(retrieve(&dir, terms_for("cra-ai-11")).is_empty());
|
|
||||||
let _ = std::fs::remove_dir_all(&dir);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn merges_adjacent_hits_into_one_region() {
|
|
||||||
// Two hits one line apart collapse to a single center/region.
|
|
||||||
assert_eq!(merge_centers(&[10, 11, 30]), vec![10, 30]);
|
|
||||||
assert_eq!(merge_centers(&[]), Vec::<usize>::new());
|
|
||||||
assert_eq!(merge_centers(&[5]), vec![5]);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -22,11 +22,6 @@ struct EmbeddingData {
|
|||||||
index: usize,
|
index: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Max inputs per embedding request. The bge/OpenAI-like backends cap the input
|
|
||||||
/// array (bge-multilingual-gemma2 rejects >25 with "batch size overflow"), so we
|
|
||||||
/// chunk larger corpora — a whole control catalog (~1.8k) would otherwise 500.
|
|
||||||
const EMBED_BATCH_SIZE: usize = 16;
|
|
||||||
|
|
||||||
// ── Embedding implementation ───────────────────────────────────
|
// ── Embedding implementation ───────────────────────────────────
|
||||||
|
|
||||||
impl LlmClient {
|
impl LlmClient {
|
||||||
@@ -34,21 +29,8 @@ impl LlmClient {
|
|||||||
&self.embed_model
|
&self.embed_model
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Generate embeddings for a batch of texts, chunking into backend-sized
|
/// Generate embeddings for a batch of texts
|
||||||
/// requests and preserving input order across chunks.
|
|
||||||
pub async fn embed(&self, texts: Vec<String>) -> Result<Vec<Vec<f64>>, AgentError> {
|
pub async fn embed(&self, texts: Vec<String>) -> Result<Vec<Vec<f64>>, AgentError> {
|
||||||
if texts.is_empty() {
|
|
||||||
return Ok(Vec::new());
|
|
||||||
}
|
|
||||||
let mut out = Vec::with_capacity(texts.len());
|
|
||||||
for chunk in texts.chunks(EMBED_BATCH_SIZE) {
|
|
||||||
out.extend(self.embed_batch(chunk.to_vec()).await?);
|
|
||||||
}
|
|
||||||
Ok(out)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Embed one backend-sized batch (≤ [`EMBED_BATCH_SIZE`]) in a single request.
|
|
||||||
async fn embed_batch(&self, texts: Vec<String>) -> Result<Vec<Vec<f64>>, AgentError> {
|
|
||||||
let url = format!("{}/v1/embeddings", self.base_url.trim_end_matches('/'));
|
let url = format!("{}/v1/embeddings", self.base_url.trim_end_matches('/'));
|
||||||
|
|
||||||
let request_body = EmbeddingRequest {
|
let request_body = EmbeddingRequest {
|
||||||
@@ -90,33 +72,3 @@ impl LlmClient {
|
|||||||
Ok(data.into_iter().map(|d| d.embedding).collect())
|
Ok(data.into_iter().map(|d| d.embedding).collect())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use secrecy::SecretString;
|
|
||||||
|
|
||||||
fn client() -> LlmClient {
|
|
||||||
LlmClient::new(
|
|
||||||
"http://unused".into(),
|
|
||||||
SecretString::from(String::new()),
|
|
||||||
"m".into(),
|
|
||||||
"e".into(),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn empty_input_makes_no_request() {
|
|
||||||
// Must short-circuit before any HTTP call (base_url is unroutable).
|
|
||||||
let out = client().embed(Vec::new()).await.unwrap();
|
|
||||||
assert!(out.is_empty());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn batch_size_is_within_backend_cap() {
|
|
||||||
assert!(
|
|
||||||
EMBED_BATCH_SIZE <= 25,
|
|
||||||
"must stay under the bge 25-input cap"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -232,9 +232,9 @@ impl PipelineOrchestrator {
|
|||||||
|
|
||||||
// Stage 5c: semantic control mapping — scale path for the master-controls
|
// Stage 5c: semantic control mapping — scale path for the master-controls
|
||||||
// corpus (no CWE to LUT on): embed each finding's region, retrieve the
|
// corpus (no CWE to LUT on): embed each finding's region, retrieve the
|
||||||
// nearest master controls, grounded-judge, and stamp confirmed refs. On by
|
// nearest master controls, grounded-judge, and stamp confirmed refs. Gated
|
||||||
// default (validated live); the corpus embedding is cached so only the
|
// (default off) as the corpus embedding + per-finding judging is the heavy
|
||||||
// first scan after a catalog change pays it.
|
// path; enabled once verified live against a deployed master-controls catalog.
|
||||||
if self.config.breakpilot.semantic_mapping {
|
if self.config.breakpilot.semantic_mapping {
|
||||||
self.update_phase(scan_run_id, "semantic_control_mapping")
|
self.update_phase(scan_run_id, "semantic_control_mapping")
|
||||||
.await;
|
.await;
|
||||||
@@ -252,31 +252,6 @@ impl PipelineOrchestrator {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Stage 5d: grounded surface checks — the absence-based controls (no
|
|
||||||
// rate limiting, no security logging, no update-signature check) have no
|
|
||||||
// syntactic pattern to match, so we retrieve the code surface each governs
|
|
||||||
// and let the grounded judge decide whether it holds, producing net-new
|
|
||||||
// findings already tagged + grounded. On by default (validated live); it
|
|
||||||
// covers the 8 absence-based CRA controls.
|
|
||||||
if self.config.breakpilot.grounded_control_checks {
|
|
||||||
self.update_phase(scan_run_id, "grounded_control_checks")
|
|
||||||
.await;
|
|
||||||
let grounded = crate::controls::grounded_surface_findings(
|
|
||||||
&self.config,
|
|
||||||
self.llm.clone(),
|
|
||||||
&repo_path,
|
|
||||||
&repo_id,
|
|
||||||
)
|
|
||||||
.await;
|
|
||||||
if !grounded.is_empty() {
|
|
||||||
tracing::info!(
|
|
||||||
"[{repo_id}] Grounded surface checks raised {} control findings",
|
|
||||||
grounded.len()
|
|
||||||
);
|
|
||||||
all_findings.extend(grounded);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Dedup against existing findings and insert new ones
|
// Dedup against existing findings and insert new ones
|
||||||
let mut new_count = 0u32;
|
let mut new_count = 0u32;
|
||||||
let mut new_findings: Vec<Finding> = Vec::new();
|
let mut new_findings: Vec<Finding> = Vec::new();
|
||||||
|
|||||||
@@ -1,125 +0,0 @@
|
|||||||
//! C5 example 2 — exploratory (not a committed regression test). Four topically
|
|
||||||
//! distinct findings, to see whether tuned semantic retrieval maps each to the
|
|
||||||
//! right master-control family. Run:
|
|
||||||
//! export ... (LITELLM_* + BREAKPILOT_BASE_URL)
|
|
||||||
//! cargo test -p compliance-agent --test c5_example2 -- --ignored --nocapture
|
|
||||||
|
|
||||||
mod common;
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use compliance_agent::llm::LlmClient;
|
|
||||||
use compliance_core::config::BreakpilotConfig;
|
|
||||||
use compliance_core::models::finding::{Finding, Severity};
|
|
||||||
use compliance_core::models::scan::ScanType;
|
|
||||||
use secrecy::SecretString;
|
|
||||||
|
|
||||||
fn env(k: &str) -> String {
|
|
||||||
std::env::var(k).unwrap_or_else(|_| panic!("env {k} must be set"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn mk(file: &str, line: u32, title: &str, desc: &str) -> Finding {
|
|
||||||
let mut f = Finding::new(
|
|
||||||
"repo-c5b".into(),
|
|
||||||
format!("{file}:{line}"),
|
|
||||||
"semgrep".into(),
|
|
||||||
ScanType::Sast,
|
|
||||||
title.into(),
|
|
||||||
desc.into(),
|
|
||||||
Severity::High,
|
|
||||||
);
|
|
||||||
f.file_path = Some(file.into());
|
|
||||||
f.line_number = Some(line);
|
|
||||||
f
|
|
||||||
}
|
|
||||||
|
|
||||||
fn write(repo: &std::path::Path, rel: &str, body: &str) {
|
|
||||||
let p = repo.join(rel);
|
|
||||||
if let Some(parent) = p.parent() {
|
|
||||||
std::fs::create_dir_all(parent).unwrap();
|
|
||||||
}
|
|
||||||
std::fs::write(p, body).unwrap();
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
#[ignore = "live: api-dev + LiteLLM"]
|
|
||||||
async fn c5b_varied_findings() {
|
|
||||||
let llm = Arc::new(LlmClient::new(
|
|
||||||
env("LITELLM_URL"),
|
|
||||||
SecretString::from(env("LITELLM_API_KEY")),
|
|
||||||
env("LITELLM_MODEL"),
|
|
||||||
env("LITELLM_EMBED_MODEL"),
|
|
||||||
));
|
|
||||||
let mut config = common::dev_config("mongodb://unused".into(), "c5b".into());
|
|
||||||
config.breakpilot = BreakpilotConfig {
|
|
||||||
base_url: Some(env("BREAKPILOT_BASE_URL")),
|
|
||||||
token: None,
|
|
||||||
snapshot_dir: std::env::temp_dir()
|
|
||||||
.join("c5-oscal-snap")
|
|
||||||
.to_string_lossy()
|
|
||||||
.into_owned(),
|
|
||||||
semantic_mapping: true,
|
|
||||||
grounded_control_checks: false,
|
|
||||||
};
|
|
||||||
|
|
||||||
let repo = std::env::temp_dir().join("c5b-fixture-repo");
|
|
||||||
let _ = std::fs::remove_dir_all(&repo);
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/db.py",
|
|
||||||
"import sqlite3\n\ndef get_user(username):\n q = \"SELECT * FROM users WHERE name = '\" + username + \"'\"\n return conn.execute(q)\n",
|
|
||||||
);
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/config.py",
|
|
||||||
"# service config\nAPI_KEY = \"sk_live_51H8xYz3kQ9v2bNmR7wT4uSpQ\"\nDB_HOST = \"db.internal\"\n",
|
|
||||||
);
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/net.py",
|
|
||||||
"import requests\n\ndef fetch(url):\n return requests.get(url, verify=False, timeout=5)\n",
|
|
||||||
);
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/ser.py",
|
|
||||||
"import pickle\n\ndef load_state(blob):\n return pickle.loads(blob)\n",
|
|
||||||
);
|
|
||||||
|
|
||||||
let mut findings = vec![
|
|
||||||
mk(
|
|
||||||
"app/db.py",
|
|
||||||
4,
|
|
||||||
"SQL injection via string-concatenated query",
|
|
||||||
"User input is concatenated directly into a SQL statement, allowing SQL injection.",
|
|
||||||
),
|
|
||||||
mk(
|
|
||||||
"app/config.py",
|
|
||||||
2,
|
|
||||||
"Hardcoded API credential in source",
|
|
||||||
"A live API key is hardcoded in source code instead of a secret store.",
|
|
||||||
),
|
|
||||||
mk(
|
|
||||||
"app/net.py",
|
|
||||||
4,
|
|
||||||
"TLS certificate verification disabled",
|
|
||||||
"requests is called with verify=False, disabling TLS certificate validation.",
|
|
||||||
),
|
|
||||||
mk(
|
|
||||||
"app/ser.py",
|
|
||||||
3,
|
|
||||||
"Insecure deserialization with pickle.loads",
|
|
||||||
"Untrusted data is deserialized with pickle.loads, allowing remote code execution.",
|
|
||||||
),
|
|
||||||
];
|
|
||||||
|
|
||||||
let tagged =
|
|
||||||
compliance_agent::controls::semantic_stamp_findings(&config, llm, &repo, &mut findings)
|
|
||||||
.await;
|
|
||||||
println!("\n=== C5 example 2: varied findings ===");
|
|
||||||
for f in &findings {
|
|
||||||
println!(" {:52} -> {:?}", f.title, f.control_refs);
|
|
||||||
}
|
|
||||||
println!("tagged: {tagged}/4");
|
|
||||||
let _ = std::fs::remove_dir_all(&repo);
|
|
||||||
assert!(tagged >= 1);
|
|
||||||
}
|
|
||||||
@@ -1,145 +0,0 @@
|
|||||||
//! C5 live verification — the semantic master-controls path end to end against the
|
|
||||||
//! deployed api-dev catalog. Ignored (hits api-dev + LiteLLM). Run explicitly:
|
|
||||||
//!
|
|
||||||
//! set -a; . ./.env; set +a
|
|
||||||
//! BREAKPILOT_BASE_URL=https://api-dev.breakpilot.ai \
|
|
||||||
//! cargo test -p compliance-agent --test c5_semantic_live -- --ignored --nocapture
|
|
||||||
//!
|
|
||||||
//! Pulls the live master-controls catalog, embeds the corpus (chunked), then for a
|
|
||||||
//! couple of real vulnerable findings retrieves the nearest master controls and
|
|
||||||
//! grounded-judges them, stamping master-control refs.
|
|
||||||
|
|
||||||
mod common;
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use compliance_agent::llm::LlmClient;
|
|
||||||
use compliance_core::config::BreakpilotConfig;
|
|
||||||
use compliance_core::models::finding::{Finding, Severity};
|
|
||||||
use compliance_core::models::scan::ScanType;
|
|
||||||
use secrecy::SecretString;
|
|
||||||
|
|
||||||
fn env(k: &str) -> String {
|
|
||||||
std::env::var(k).unwrap_or_else(|_| panic!("env {k} must be set for the live C5 test"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn mk_finding(file: &str, line: u32, title: &str) -> Finding {
|
|
||||||
let mut f = Finding::new(
|
|
||||||
"repo-c5".into(),
|
|
||||||
format!("{file}:{line}"),
|
|
||||||
"semgrep".into(),
|
|
||||||
ScanType::Sast,
|
|
||||||
title.into(),
|
|
||||||
title.into(),
|
|
||||||
Severity::High,
|
|
||||||
);
|
|
||||||
f.file_path = Some(file.into());
|
|
||||||
f.line_number = Some(line);
|
|
||||||
f
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
#[ignore = "live: requires deployed api-dev master-controls (fetch+parse only, no LLM)"]
|
|
||||||
async fn c5_ingest_master_controls_catalog() {
|
|
||||||
use compliance_agent::controls::OscalControlsProvider;
|
|
||||||
|
|
||||||
let provider = OscalControlsProvider::new(
|
|
||||||
reqwest::Client::new(),
|
|
||||||
env("BREAKPILOT_BASE_URL"),
|
|
||||||
None,
|
|
||||||
std::env::temp_dir().join("c5-ingest-snap"),
|
|
||||||
);
|
|
||||||
let doc = provider
|
|
||||||
.load_master_controls()
|
|
||||||
.await
|
|
||||||
.expect("pull + parse master-controls catalog");
|
|
||||||
let controls = doc.to_controls();
|
|
||||||
println!(
|
|
||||||
"\n=== C5 ingest: {} master controls parsed ===",
|
|
||||||
controls.len()
|
|
||||||
);
|
|
||||||
for c in controls.iter().take(4) {
|
|
||||||
let text: String = c.text.chars().take(90).collect();
|
|
||||||
println!(" {} | {} | {}", c.id, c.title, text);
|
|
||||||
}
|
|
||||||
assert!(
|
|
||||||
!controls.is_empty(),
|
|
||||||
"expected a non-empty master-control corpus"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
#[ignore = "live: requires deployed api-dev master-controls + LiteLLM"]
|
|
||||||
async fn c5_semantic_stamps_master_control_refs() {
|
|
||||||
let llm = Arc::new(LlmClient::new(
|
|
||||||
env("LITELLM_URL"),
|
|
||||||
SecretString::from(env("LITELLM_API_KEY")),
|
|
||||||
env("LITELLM_MODEL"),
|
|
||||||
env("LITELLM_EMBED_MODEL"),
|
|
||||||
));
|
|
||||||
|
|
||||||
let mut config = common::dev_config("mongodb://unused".into(), "c5".into());
|
|
||||||
let snapshot = std::env::temp_dir().join("c5-oscal-snap");
|
|
||||||
config.breakpilot = BreakpilotConfig {
|
|
||||||
base_url: Some(env("BREAKPILOT_BASE_URL")),
|
|
||||||
token: None,
|
|
||||||
snapshot_dir: snapshot.to_string_lossy().into_owned(),
|
|
||||||
semantic_mapping: true,
|
|
||||||
grounded_control_checks: false,
|
|
||||||
};
|
|
||||||
|
|
||||||
// Fixture repo with recognizable code-checkable surfaces.
|
|
||||||
let repo = std::env::temp_dir().join("c5-fixture-repo");
|
|
||||||
let _ = std::fs::remove_dir_all(&repo);
|
|
||||||
std::fs::create_dir_all(repo.join("app")).expect("mkdir");
|
|
||||||
std::fs::write(
|
|
||||||
repo.join("app/auth.py"),
|
|
||||||
concat!(
|
|
||||||
"import hashlib\n",
|
|
||||||
"\n",
|
|
||||||
"def store_password(user, password):\n",
|
|
||||||
" # weak, unsalted password hashing\n",
|
|
||||||
" digest = hashlib.md5(password.encode()).hexdigest()\n",
|
|
||||||
" db.save(user, digest)\n",
|
|
||||||
"\n",
|
|
||||||
"@app.route('/login', methods=['POST'])\n",
|
|
||||||
"def login():\n",
|
|
||||||
" u = request.form['username']\n",
|
|
||||||
" p = request.form['password']\n",
|
|
||||||
" return 'ok' if check(u, p) else ('bad', 401)\n",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
.expect("write fixture");
|
|
||||||
|
|
||||||
let mut findings = vec![
|
|
||||||
mk_finding("app/auth.py", 5, "Weak password hash (md5, unsalted)"),
|
|
||||||
mk_finding(
|
|
||||||
"app/auth.py",
|
|
||||||
9,
|
|
||||||
"Login endpoint without brute-force protection",
|
|
||||||
),
|
|
||||||
];
|
|
||||||
|
|
||||||
let tagged =
|
|
||||||
compliance_agent::controls::semantic_stamp_findings(&config, llm, &repo, &mut findings)
|
|
||||||
.await;
|
|
||||||
|
|
||||||
println!("\n=== C5 semantic master-controls stamping ===");
|
|
||||||
for f in &findings {
|
|
||||||
println!(
|
|
||||||
" {:50} {}:{:?} -> {:?}",
|
|
||||||
f.title,
|
|
||||||
f.file_path.as_deref().unwrap_or(""),
|
|
||||||
f.line_number,
|
|
||||||
f.control_refs
|
|
||||||
);
|
|
||||||
}
|
|
||||||
println!("findings that gained >=1 master-control ref: {tagged}");
|
|
||||||
let _ = std::fs::remove_dir_all(&repo);
|
|
||||||
|
|
||||||
// Live corpus — assert only that the path runs and stamps at least one ref.
|
|
||||||
assert!(
|
|
||||||
tagged >= 1,
|
|
||||||
"expected at least one finding to gain a master-control ref"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
//! Live validation of the grounded surface path (Stage 5d) for absence-based CRA
|
|
||||||
//! controls. Ignored (hits api-dev CRA catalog + LiteLLM). Run:
|
|
||||||
//! export ... (LITELLM_* + BREAKPILOT_BASE_URL)
|
|
||||||
//! cargo test -p compliance-agent --test grounded_surface_live -- --ignored --nocapture
|
|
||||||
//!
|
|
||||||
//! Builds a fixture whose code surfaces trigger several absence-based controls
|
|
||||||
//! (no rate limiting, no security logging, unverified update) and checks that the
|
|
||||||
//! grounded checker produces control-tagged findings.
|
|
||||||
|
|
||||||
mod common;
|
|
||||||
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use compliance_agent::llm::LlmClient;
|
|
||||||
use compliance_core::config::BreakpilotConfig;
|
|
||||||
use secrecy::SecretString;
|
|
||||||
|
|
||||||
fn env(k: &str) -> String {
|
|
||||||
std::env::var(k).unwrap_or_else(|_| panic!("env {k} must be set"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn write(repo: &std::path::Path, rel: &str, body: &str) {
|
|
||||||
let p = repo.join(rel);
|
|
||||||
if let Some(parent) = p.parent() {
|
|
||||||
std::fs::create_dir_all(parent).unwrap();
|
|
||||||
}
|
|
||||||
std::fs::write(p, body).unwrap();
|
|
||||||
}
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
#[ignore = "live: api-dev CRA catalog + LiteLLM"]
|
|
||||||
async fn grounded_surface_flags_absence_controls() {
|
|
||||||
let llm = Arc::new(LlmClient::new(
|
|
||||||
env("LITELLM_URL"),
|
|
||||||
SecretString::from(env("LITELLM_API_KEY")),
|
|
||||||
env("LITELLM_MODEL"),
|
|
||||||
env("LITELLM_EMBED_MODEL"),
|
|
||||||
));
|
|
||||||
let mut config = common::dev_config("mongodb://unused".into(), "grounded".into());
|
|
||||||
config.breakpilot = BreakpilotConfig {
|
|
||||||
base_url: Some(env("BREAKPILOT_BASE_URL")),
|
|
||||||
token: None,
|
|
||||||
snapshot_dir: std::env::temp_dir()
|
|
||||||
.join("grounded-snap")
|
|
||||||
.to_string_lossy()
|
|
||||||
.into_owned(),
|
|
||||||
semantic_mapping: false,
|
|
||||||
grounded_control_checks: true,
|
|
||||||
};
|
|
||||||
|
|
||||||
let repo = std::env::temp_dir().join("grounded-fixture-repo");
|
|
||||||
let _ = std::fs::remove_dir_all(&repo);
|
|
||||||
// cra-ai-11: login endpoint with no rate limiting / lockout
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/auth.py",
|
|
||||||
"@app.route('/login', methods=['POST'])\ndef login():\n u = request.form['username']\n p = request.form['password']\n if authenticate(u, p):\n return redirect('/')\n return 'bad credentials', 401\n",
|
|
||||||
);
|
|
||||||
// cra-ai-24: privileged admin action with no security/audit logging
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/admin.py",
|
|
||||||
"@app.route('/admin/delete_user', methods=['POST'])\ndef admin_delete_user():\n uid = request.form['uid']\n db.users.delete_one({'_id': uid})\n return 'ok', 200\n",
|
|
||||||
);
|
|
||||||
// cra-ai-28/29/30: firmware update applied without signature / checksum verification
|
|
||||||
write(
|
|
||||||
&repo,
|
|
||||||
"app/updater.py",
|
|
||||||
"def apply_firmware_update(url):\n blob = download(url)\n install_firmware(blob)\n reboot_device()\n",
|
|
||||||
);
|
|
||||||
|
|
||||||
let findings =
|
|
||||||
compliance_agent::controls::grounded_surface_findings(&config, llm, &repo, "repo-grounded")
|
|
||||||
.await;
|
|
||||||
|
|
||||||
println!("\n=== Grounded surface findings ({}) ===", findings.len());
|
|
||||||
for f in &findings {
|
|
||||||
println!(
|
|
||||||
" {:24} {}:{:?} {}",
|
|
||||||
f.control_refs.join(","),
|
|
||||||
f.file_path.as_deref().unwrap_or(""),
|
|
||||||
f.line_number,
|
|
||||||
f.title
|
|
||||||
);
|
|
||||||
}
|
|
||||||
let _ = std::fs::remove_dir_all(&repo);
|
|
||||||
|
|
||||||
assert!(
|
|
||||||
!findings.is_empty(),
|
|
||||||
"expected the grounded pass to flag at least one absence-based control"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -76,16 +76,10 @@ pub struct BreakpilotConfig {
|
|||||||
/// Directory for catalog snapshots.
|
/// Directory for catalog snapshots.
|
||||||
pub snapshot_dir: String,
|
pub snapshot_dir: String,
|
||||||
/// Enable the master-controls **semantic** mapping pass (embed regions,
|
/// Enable the master-controls **semantic** mapping pass (embed regions,
|
||||||
/// Enable the master-controls **semantic** mapping pass (embed regions,
|
/// retrieve nearest controls, grounded-judge). Off by default: it is the
|
||||||
/// retrieve nearest controls, grounded-judge). On by default — validated live
|
/// scale path and stays gated until verified live against a deployed
|
||||||
/// against the deployed master-controls catalog. Still a no-op unless
|
/// master-controls catalog.
|
||||||
/// `base_url` is set and the catalog is reachable.
|
|
||||||
pub semantic_mapping: bool,
|
pub semantic_mapping: bool,
|
||||||
/// Enable the **grounded surface** pass for absence-based controls (retrieve
|
|
||||||
/// the code surface a control governs, judge whether it holds). On by default
|
|
||||||
/// — validated live; it covers the 8 absence-based CRA controls that no
|
|
||||||
/// syntactic rule can.
|
|
||||||
pub grounded_control_checks: bool,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Default for BreakpilotConfig {
|
impl Default for BreakpilotConfig {
|
||||||
@@ -94,8 +88,7 @@ impl Default for BreakpilotConfig {
|
|||||||
base_url: None,
|
base_url: None,
|
||||||
token: None,
|
token: None,
|
||||||
snapshot_dir: "/data/compliance-scanner/oscal".to_string(),
|
snapshot_dir: "/data/compliance-scanner/oscal".to_string(),
|
||||||
semantic_mapping: true,
|
semantic_mapping: false,
|
||||||
grounded_control_checks: true,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -42,19 +42,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
|
|
||||||
let pool_for_factory = pool.clone();
|
let pool_for_factory = pool.clone();
|
||||||
let service = StreamableHttpService::new(
|
let service = StreamableHttpService::new(
|
||||||
move || {
|
move || Ok(ComplianceMcpServer::new(pool_for_factory.clone())),
|
||||||
// The factory runs in the request task, still inside the bearer
|
|
||||||
// middleware's `TENANT_ID` scope, and BEFORE rmcp spawns the
|
|
||||||
// session task (which would lose the task_local). So bind the
|
|
||||||
// tenant into the session's server instance here, once.
|
|
||||||
let tenant_id = auth::current_tenant_id().ok_or_else(|| {
|
|
||||||
std::io::Error::other("no tenant context when creating MCP session")
|
|
||||||
})?;
|
|
||||||
Ok(ComplianceMcpServer::new(
|
|
||||||
pool_for_factory.clone(),
|
|
||||||
tenant_id,
|
|
||||||
))
|
|
||||||
},
|
|
||||||
Arc::new(LocalSessionManager::default()),
|
Arc::new(LocalSessionManager::default()),
|
||||||
StreamableHttpServerConfig::default(),
|
StreamableHttpServerConfig::default(),
|
||||||
);
|
);
|
||||||
@@ -81,11 +69,16 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
tenant_id = %synth_tenant,
|
tenant_id = %synth_tenant,
|
||||||
"stdio transport — using synthetic tenant id; DO NOT use in production"
|
"stdio transport — using synthetic tenant id; DO NOT use in production"
|
||||||
);
|
);
|
||||||
let server = ComplianceMcpServer::new(pool, synth_tenant);
|
let server = ComplianceMcpServer::new(pool);
|
||||||
let transport = rmcp::transport::stdio();
|
let transport = rmcp::transport::stdio();
|
||||||
use rmcp::ServiceExt;
|
use rmcp::ServiceExt;
|
||||||
let handle = server.serve(transport).await?;
|
auth::TENANT_ID
|
||||||
handle.waiting().await?;
|
.scope(synth_tenant, async {
|
||||||
|
let handle = server.serve(transport).await?;
|
||||||
|
handle.waiting().await?;
|
||||||
|
Ok::<_, Box<dyn std::error::Error>>(())
|
||||||
|
})
|
||||||
|
.await?;
|
||||||
}
|
}
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
|
|||||||
@@ -2,33 +2,37 @@ use rmcp::{
|
|||||||
handler::server::wrapper::Parameters, model::*, tool, tool_handler, tool_router, ServerHandler,
|
handler::server::wrapper::Parameters, model::*, tool, tool_handler, tool_router, ServerHandler,
|
||||||
};
|
};
|
||||||
|
|
||||||
|
use crate::auth::current_tenant_id;
|
||||||
use crate::database::{Database, DatabasePool};
|
use crate::database::{Database, DatabasePool};
|
||||||
use crate::tools::{dast, findings, oscal, pentest, sbom};
|
use crate::tools::{dast, findings, oscal, pentest, sbom};
|
||||||
|
|
||||||
pub struct ComplianceMcpServer {
|
pub struct ComplianceMcpServer {
|
||||||
pool: DatabasePool,
|
pool: DatabasePool,
|
||||||
/// Tenant this session serves. Bound once at session creation (the HTTP
|
|
||||||
/// factory reads the bearer-set tenant while still in the request scope;
|
|
||||||
/// stdio passes a synthetic id) — NOT a per-request `task_local`, which is
|
|
||||||
/// lost across the `tokio::spawn` that runs the Streamable-HTTP session.
|
|
||||||
tenant_id: String,
|
|
||||||
#[allow(dead_code)]
|
#[allow(dead_code)]
|
||||||
tool_router: rmcp::handler::server::router::tool::ToolRouter<Self>,
|
tool_router: rmcp::handler::server::router::tool::ToolRouter<Self>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl ComplianceMcpServer {
|
impl ComplianceMcpServer {
|
||||||
/// The per-tenant `Database` for this session.
|
/// Resolve the per-tenant `Database` from the bearer-set
|
||||||
|
/// `task_local`. Every tool handler calls this; missing context
|
||||||
|
/// surfaces as `internal_error` because it means the auth
|
||||||
|
/// middleware was misconfigured (handler ran without scope).
|
||||||
fn tenant_db(&self) -> Result<Database, rmcp::ErrorData> {
|
fn tenant_db(&self) -> Result<Database, rmcp::ErrorData> {
|
||||||
Ok(self.pool.for_tenant_id(&self.tenant_id))
|
let tenant_id = current_tenant_id().ok_or_else(|| {
|
||||||
|
rmcp::ErrorData::internal_error(
|
||||||
|
"no tenant context — bearer middleware not in chain".to_string(),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
Ok(self.pool.for_tenant_id(&tenant_id))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tool_router]
|
#[tool_router]
|
||||||
impl ComplianceMcpServer {
|
impl ComplianceMcpServer {
|
||||||
pub fn new(pool: DatabasePool, tenant_id: String) -> Self {
|
pub fn new(pool: DatabasePool) -> Self {
|
||||||
Self {
|
Self {
|
||||||
pool,
|
pool,
|
||||||
tenant_id,
|
|
||||||
tool_router: Self::tool_router(),
|
tool_router: Self::tool_router(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -25,43 +25,36 @@
|
|||||||
"control": "cra-ai-2",
|
"control": "cra-ai-2",
|
||||||
"title": "Minimale Angriffsflaeche",
|
"title": "Minimale Angriffsflaeche",
|
||||||
"scans": [],
|
"scans": [],
|
||||||
"note": "design property (minimal attack surface) — not derivable from local code patterns; architecture/threat-model review",
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"status": "not_code_checkable"
|
"status": "needs_tooling"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-3",
|
"control": "cra-ai-3",
|
||||||
"title": "Sichere Systemarchitektur",
|
"title": "Sichere Systemarchitektur",
|
||||||
"scans": [],
|
"scans": [],
|
||||||
"note": "design property (secure system architecture) — architecture review, not statically code-checkable",
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"status": "not_code_checkable"
|
"status": "needs_tooling"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-4",
|
"control": "cra-ai-4",
|
||||||
"title": "Least-Privilege-Prinzip",
|
"title": "Least-Privilege-Prinzip",
|
||||||
"scans": [],
|
"scans": [],
|
||||||
"note": "design property (least-privilege) — deployment/IAM & architecture review, not a local code pattern",
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"status": "not_code_checkable"
|
"status": "needs_tooling"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-5",
|
"control": "cra-ai-5",
|
||||||
"title": "Manipulationsschutz",
|
"title": "Manipulationsschutz",
|
||||||
"scans": [],
|
"scans": [],
|
||||||
"note": "design property (tamper protection) — hardware/runtime & operational control, not statically code-checkable",
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"status": "not_code_checkable"
|
"status": "needs_tooling"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-6",
|
"control": "cra-ai-6",
|
||||||
"title": "Integritaetspruefung",
|
"title": "Integritaetspruefung",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-7",
|
"control": "cra-ai-7",
|
||||||
@@ -145,30 +138,16 @@
|
|||||||
{
|
{
|
||||||
"control": "cra-ai-11",
|
"control": "cra-ai-11",
|
||||||
"title": "Brute-Force-Schutz",
|
"title": "Brute-Force-Schutz",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-12",
|
"control": "cra-ai-12",
|
||||||
"title": "Rollenbasierte Autorisierung",
|
"title": "Rollenbasierte Autorisierung",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-13",
|
"control": "cra-ai-13",
|
||||||
@@ -328,16 +307,9 @@
|
|||||||
{
|
{
|
||||||
"control": "cra-ai-24",
|
"control": "cra-ai-24",
|
||||||
"title": "Security-Logging",
|
"title": "Security-Logging",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-25",
|
"control": "cra-ai-25",
|
||||||
@@ -356,58 +328,30 @@
|
|||||||
{
|
{
|
||||||
"control": "cra-ai-27",
|
"control": "cra-ai-27",
|
||||||
"title": "Log-Integritaet und -Aufbewahrung",
|
"title": "Log-Integritaet und -Aufbewahrung",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-28",
|
"control": "cra-ai-28",
|
||||||
"title": "Sichere Update-Mechanismen",
|
"title": "Sichere Update-Mechanismen",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-29",
|
"control": "cra-ai-29",
|
||||||
"title": "Update-Authentizitaet",
|
"title": "Update-Authentizitaet",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-30",
|
"control": "cra-ai-30",
|
||||||
"title": "Update-Integritaet",
|
"title": "Update-Integritaet",
|
||||||
"scans": [
|
"scans": [],
|
||||||
{
|
"note": "code-checkable but no off-the-shelf tool digs it out — author a detector (custom semgrep rule / check)",
|
||||||
"tool": "grounded-control-check",
|
"status": "needs_tooling"
|
||||||
"scan_type": "code_review",
|
|
||||||
"cwe": [],
|
|
||||||
"rules": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"note": "covered by the grounded surface check (retrieve code surface + grounded LLM judge decides presence/absence); validated live",
|
|
||||||
"status": "covered"
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"control": "cra-ai-31",
|
"control": "cra-ai-31",
|
||||||
|
|||||||
+2
-27
@@ -178,13 +178,11 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn covered_and_not_checkable_are_populated() {
|
fn every_bucket_is_represented() {
|
||||||
let s = ControlMap::cra().unwrap().summary();
|
let s = ControlMap::cra().unwrap().summary();
|
||||||
assert!(s.covered > 0);
|
assert!(s.covered > 0);
|
||||||
|
assert!(s.needs_tooling > 0);
|
||||||
assert!(s.not_code_checkable > 0);
|
assert!(s.not_code_checkable > 0);
|
||||||
// needs_tooling is now empty: every code-checkable control is either
|
|
||||||
// tool-covered or covered by the grounded surface pass.
|
|
||||||
assert_eq!(s.needs_tooling, 0);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -216,29 +214,6 @@ mod tests {
|
|||||||
assert!(hits.iter().any(|c| c.control == "cra-ai-1"));
|
assert!(hits.iter().any(|c| c.control == "cra-ai-1"));
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn coverage_after_grounded_promotion() {
|
|
||||||
let s = ControlMap::cra().unwrap().summary();
|
|
||||||
// 9 off-the-shelf + 4 custom-semgrep + 8 grounded surface controls (promoted
|
|
||||||
// after the grounded path was validated live).
|
|
||||||
assert_eq!(s.covered, 21);
|
|
||||||
// Nothing left as needs_tooling — every code-checkable control is covered.
|
|
||||||
assert_eq!(s.needs_tooling, 0);
|
|
||||||
// The 4 pure-architectural controls remain not code-checkable.
|
|
||||||
assert_eq!(s.not_code_checkable, 19);
|
|
||||||
assert_eq!(s.total(), 40);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn architectural_controls_are_not_code_checkable() {
|
|
||||||
let map = ControlMap::cra().unwrap();
|
|
||||||
for id in ["cra-ai-2", "cra-ai-3", "cra-ai-4", "cra-ai-5"] {
|
|
||||||
let c = map.coverage(id).unwrap();
|
|
||||||
assert_eq!(c.status, Coverage::NotCodeCheckable, "{id}");
|
|
||||||
assert!(c.scans.is_empty(), "{id} should carry no scan bindings");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn custom_rule_controls_do_not_bind_by_broad_cwe() {
|
fn custom_rule_controls_do_not_bind_by_broad_cwe() {
|
||||||
let map = ControlMap::cra().unwrap();
|
let map = ControlMap::cra().unwrap();
|
||||||
|
|||||||
@@ -36,7 +36,6 @@ export default withMermaid(defineConfig({
|
|||||||
{ text: 'Pentest Architecture', link: '/features/pentest-architecture' },
|
{ text: 'Pentest Architecture', link: '/features/pentest-architecture' },
|
||||||
{ text: 'AI Chat', link: '/features/ai-chat' },
|
{ text: 'AI Chat', link: '/features/ai-chat' },
|
||||||
{ text: 'Code Knowledge Graph', link: '/features/graph' },
|
{ text: 'Code Knowledge Graph', link: '/features/graph' },
|
||||||
{ text: 'Compliance Control Mapping', link: '/features/control-mapping' },
|
|
||||||
{ text: 'MCP Integration', link: '/features/mcp-server' },
|
{ text: 'MCP Integration', link: '/features/mcp-server' },
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1,127 +0,0 @@
|
|||||||
# Compliance Control Mapping
|
|
||||||
|
|
||||||
Control mapping connects the scanner's raw output — deterministic tool findings and the code itself — to the **compliance controls** each piece of evidence supports. A hardcoded credential stops being just "CWE-798 from semgrep" and becomes evidence for *"cra-ai-8: no default passwords"* and, at scale, master control *`mc-31761` hardcoded_secrets_detection*. Findings carry those references (`control_refs`) into the dashboard and out over the MCP server as OSCAL, so the compliance report is built from real, grounded findings rather than a questionnaire.
|
|
||||||
|
|
||||||
## The core principle: tools detect, the LLM judges
|
|
||||||
|
|
||||||
The design has one rule, borrowed from the ZeroFalse / IRIS line of research: **deterministic tools are the detectors; the LLM is only ever a grounded false-positive filter, never the thing that finds the issue.**
|
|
||||||
|
|
||||||
- A tool (semgrep, gitleaks, syft/osv, ZAP, nuclei) detects deterministically.
|
|
||||||
- An **authored, human-reviewed lookup table** (`control-map`) maps that detection to the control(s) it's evidence for.
|
|
||||||
- The LLM enters last, to *confirm or refute* the mapping against the actual code — and every surviving verdict is anchored to a verbatim snippet by the grounding gate.
|
|
||||||
|
|
||||||
This keeps hallucination out of detection. The LLM supplies cross-language, cross-stack pattern *recognition*; the surrounding machinery supplies determinism.
|
|
||||||
|
|
||||||
## Coverage model
|
|
||||||
|
|
||||||
Every control lands in one of three buckets, recorded in the `control-map` LUT (`control-map/data/cra_control_map.json`) and never decided by an LLM:
|
|
||||||
|
|
||||||
| Bucket | Meaning |
|
|
||||||
| --- | --- |
|
|
||||||
| `covered` | An existing tool's scan surfaces findings for this control |
|
|
||||||
| `needs_tooling` | Code-checkable, but no off-the-shelf tool digs it out — we author a detector or use the grounded surface check |
|
|
||||||
| `not_code_checkable` | A design/process property — out of static-scan scope |
|
|
||||||
|
|
||||||
For the **CRA** framework (40 controls) the split is **13 covered · 8 needs_tooling · 19 not_code_checkable**. The 16 originally-uncovered controls were resolved as a hybrid:
|
|
||||||
|
|
||||||
- **4 custom semgrep detectors** (`cra-ai-1`, `7`, `10`, `14`) — secure-by-default, weak password hashing, insecure session cookies, weak data-at-rest ciphers. Shipped in the binary and matched back to controls **by rule id** so a broad CWE can't over-attribute.
|
|
||||||
- **8 grounded surface checks** (`cra-ai-6`, `11`, `12`, `24`, `27`, `28`, `29`, `30`) — the absence-based controls (no rate limiting, no security logging, no update-signature check…) that have no syntactic pattern.
|
|
||||||
- **4 marked not_code_checkable** (`cra-ai-2`, `3`, `4`, `5`) — minimal attack surface, secure architecture, least privilege, tamper protection.
|
|
||||||
|
|
||||||
At scale, the **master-controls** corpus (breakpilot's deduped clusters, exported as OSCAL) currently provides **~2,882 code-checkable controls** (2,143 `network` + 739 `source_code`), matched semantically.
|
|
||||||
|
|
||||||
## The three mapping paths
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
T[Deterministic tools\nsemgrep · gitleaks · syft/osv · ZAP] --> F[Findings]
|
|
||||||
F --> B["Stage 5b — LUT triage\ncontrols_for(tool, cwe / rule_id)"]
|
|
||||||
F --> C["Stage 5c — Semantic\nembed region+intent → top-K master controls"]
|
|
||||||
R[Repo source] --> D["Stage 5d — Grounded surface\nretrieve surface for absence-based controls"]
|
|
||||||
B --> J{{Grounded LLM judge\ntemp 0 · verbatim snippet}}
|
|
||||||
C --> J
|
|
||||||
D --> J
|
|
||||||
J -->|snippet grounds in region| S[Stamp control_refs]
|
|
||||||
J -->|refuted / ungrounded| X[Dropped]
|
|
||||||
```
|
|
||||||
|
|
||||||
All three paths converge on the same **grounded judge** and the same **grounding gate**. They differ only in how candidate (finding/region, control) pairs are produced.
|
|
||||||
|
|
||||||
### Stage 5b — deterministic LUT triage
|
|
||||||
|
|
||||||
The default path. A tool finding is matched to controls via `control_map.controls_for_finding(tool, cwe, rule_id)`; the judge then confirms each mapped control against the code region. Outcomes: `Confirmed([ids])` (stamp them), `FalsePositive` (drop the finding), or `Unmapped` (keep it untagged). Runs whenever `BREAKPILOT_BASE_URL` is set.
|
|
||||||
|
|
||||||
### Stage 5c — semantic retrieval (master-controls scale)
|
|
||||||
|
|
||||||
Master controls carry no CWE, so they can't be LUT-mapped. Instead we map by *similarity*: embed every control's requirement text once (cached), then for each finding retrieve the top-K nearest controls and hand them to the judge. Gated behind `BREAKPILOT_SEMANTIC_MAPPING` (default off). See [Semantic retrieval](#semantic-retrieval-in-detail).
|
|
||||||
|
|
||||||
### Stage 5d — grounded surface checks (absence-based controls)
|
|
||||||
|
|
||||||
Some controls are violated by an *absence* — no rate limiting on login, no security logging, no signature check on an update. There's no pattern for semgrep to match, so we deterministically retrieve the code **surface** the control governs (a login route, a logging setup, update/download code) by identifier/route terms, and let the judge decide whether the control holds there. Produces net-new, already-grounded findings. Gated behind `BREAKPILOT_GROUNDED_CHECKS` (default off).
|
|
||||||
|
|
||||||
## The grounding gate
|
|
||||||
|
|
||||||
No matter the path, a verdict becomes a finding only if it survives `compliance_core::control_check::ground`:
|
|
||||||
|
|
||||||
1. The judge runs at **temperature 0** with a closed prompt and must quote the offending code **verbatim** into `snippet`.
|
|
||||||
2. That snippet must appear **literally** in the retrieved region — otherwise the verdict is dropped.
|
|
||||||
3. The finding's line is **recomputed from the match**; the model's own line number is never trusted.
|
|
||||||
4. Verdicts are cached by content hash, so re-scans reproduce.
|
|
||||||
|
|
||||||
The model is allowed to be smart; it is never trusted.
|
|
||||||
|
|
||||||
## Semantic retrieval in detail
|
|
||||||
|
|
||||||
1. **Embed the corpus once.** Each control's requirement text is embedded with `bge-multilingual-gemma2` (3584-dim — multilingual matters, the master controls are in German while code is English). The embedding backend caps input arrays at 25 per request, so `embed()` chunks at 16; the whole `ControlIndex` is persisted to `snapshot_dir` keyed by a **corpus hash**, so only the first scan after a catalog change pays the embedding cost.
|
|
||||||
2. **Build the query from the finding's intent, not just the code.** The retrieval query is `finding.title + finding.description + region`, not the raw region. This is the single most important tuning: two findings in one file share overlapping windows and, on the code alone, embed alike and collapse onto the same controls. The finding's own words ("brute-force protection" vs "weak hash") carry the discriminating signal. The raw region still goes to the judge for grounding.
|
|
||||||
3. **Retrieve → judge → ground.** Top-K nearest by cosine, each judged against the region, each grounded.
|
|
||||||
|
|
||||||
## Worked examples
|
|
||||||
|
|
||||||
Both examples are from the live end-to-end verification (`c5_semantic_live.rs`) against the real ~2,882-control corpus.
|
|
||||||
|
|
||||||
### Example 1 — a small auth file (the tuning story)
|
|
||||||
|
|
||||||
Two findings in one `auth.py`: a weak `hashlib.md5(password)` hash and a login endpoint with no brute-force protection.
|
|
||||||
|
|
||||||
| Finding | Region-only retrieval | Intent-enriched retrieval |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Weak md5 hash | 19874, 20683, 23149, 29985 | **`mc-23149`** (eliminate weak unsalted hashes) at rank 1, + `mc-21634` salted hashing |
|
|
||||||
| Login w/o brute-force protection | *identical 4, reordered* | newly surfaces **`mc-19984`** brute_force_protection + **`mc-23186`** account_lockout |
|
|
||||||
|
|
||||||
Region-only retrieval gave both findings the *same* four password-hashing controls — the brute-force finding never found its real controls because its window is saturated with `password` tokens. Enriching the query with the finding's intent fixed it: the brute-force finding now pulls the correct rate-limiting / lockout controls out of the 2,882.
|
|
||||||
|
|
||||||
### Example 2 — four topically distinct vulnerabilities
|
|
||||||
|
|
||||||
| Finding | Top matched controls | Family |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| SQL injection (string-concat query) | `sql_injection_prevention`, `sql_injection`, `parameterized_queries`, input_sanitization | input-validation ✓ |
|
|
||||||
| Hardcoded API credential | `hardcoded_secrets_detection`, credential_scanning, secrets_detection | credentials ✓ |
|
|
||||||
| TLS verification disabled (`verify=False`) | `https_enforcement`, `configuration_verification`, transport config | transport-encryption ✓ |
|
|
||||||
| Insecure deserialization (`pickle.loads`) | `deserialization`, `deserialization_testing`, `deserialization_security` | deserialization ✓ |
|
|
||||||
|
|
||||||
Every finding maps to its exact control family, with the most specific control often at the top, and the four sets are distinct.
|
|
||||||
|
|
||||||
## Known limitations
|
|
||||||
|
|
||||||
- **Absence findings are weak for semantic retrieval.** Similarity matches what code *is about*, not what it *lacks*; a "missing rate limiting" finding embeds like login code. This is exactly why the grounded surface path (Stage 5d) exists — it decides presence/absence at a retrieved surface rather than by embedding distance.
|
|
||||||
- **Generic catch-all controls co-occur.** `mc-20890 secure_development_security_code_review` appears in the top-K for many code-security findings because it is semantically near almost all of them. It's harmless (the judge grounds it, and it never crowds out the specific controls — the SQLi example didn't get it) but is a candidate for future down-weighting.
|
|
||||||
- **Corpus classification noise.** The master-controls `verification_method` classification is imperfect — e.g. a documentation control (`eu_declaration_accuracy`) is currently tagged `source_code`. That's a corpus-side data-quality issue, separate from the mapping engine.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
| Variable | Effect |
|
|
||||||
| --- | --- |
|
|
||||||
| `BREAKPILOT_BASE_URL` | breakpilot-compliance root; enables control ingest + Stage 5b. Unset disables all control mapping. |
|
|
||||||
| `BREAKPILOT_SEMANTIC_MAPPING` | Enables Stage 5c (semantic master-controls mapping). Default off. |
|
|
||||||
| `BREAKPILOT_GROUNDED_CHECKS` | Enables Stage 5d (grounded surface checks). Default off. |
|
|
||||||
| `BREAKPILOT_SNAPSHOT_DIR` | Where OSCAL catalog snapshots and the cached control-embedding index live. |
|
|
||||||
|
|
||||||
The semantic and grounded passes are gated because they are the heavier, less deterministic paths; they stay off until verified live against a deployed catalog. The live verification lives in `compliance-agent/tests/c5_semantic_live.rs` (ignored; run with `--ignored`).
|
|
||||||
|
|
||||||
## Appendix — the master-controls data pipeline
|
|
||||||
|
|
||||||
The master-controls corpus is produced by breakpilot-compliance and pulled as an OSCAL catalog from `GET /api/compliance/v1/oscal/catalog?framework=master-controls`. Two operational lessons are worth recording, because they cost real time to diagnose:
|
|
||||||
|
|
||||||
- **The catalog is served from `breakpilot_db`, not `postgres`.** Diagnostics run against the wrong database will look clean while the app serves something else entirely. Confirm the app's datname (`pg_stat_activity`) before trusting any count or `EXPLAIN`.
|
|
||||||
- **A constraint-less dump triplicated the master-control tables.** Restored without their PK/unique constraints, `master_controls` / `mc_verification` / `master_control_members` accumulated identical rows 3× (the same artifact migration `158` fixed for `doc_check_controls`). That inflated the catalog to ~26k dup'd controls and, with the indexes also missing, drove the export query to a >120s / 502. The fix (breakpilot migration `160`) ctid-dedups each table by its natural key and restores the constraints + indexes so it can't recur; the export query was also rewritten set-based (a single windowed pass instead of a per-row correlated subquery). After dedup: 41,850 → 13,950 master controls, catalog **25,938 → 2,882** code-checkable, endpoint **502 → 200 in ~3s**.
|
|
||||||
Reference in New Issue
Block a user