Benjamin Admin
c39787ad96
Semantic correction of the knowledge base BEFORE the empirical loop (#59) is built — otherwise the Observation Store would learn from already-misclassified signals. The Silent Pass conflated two kinds of signal into one: an OBSERVATION ("I saw an SBOM in the repo") and a REQUIREMENT ("a tender DEMANDS an SBOM"). They were aliased to the same canonical id, so a tender clause read as "SBOM already present" and suppressed the very question that should have been asked. Fix — make the kind explicit and authoritative (no new architecture, data + thin wiring): - `kind` ∈ {observation, requirement} on ProducedSignal (producer may declare) and on the canonical SignalVocabularyEntry (AUTHORITATIVE — a mislabelled producer cannot collapse the two). - Vocabulary split: sbom_file_found → sbom_present (obs) + sbom_required (req); security_txt_or_cvd_policy → cvd_policy_present (obs) + psirt_required (req); add signed_updates_required. requirement signals are intentionally UNMAPPED in intake_signal_map (they describe a target, not state). - silent_intake() consumes ONLY kind==observation; requirement signals are preserved in `requirements_seen` (visible/auditable) but NEVER become a detected capability. - normalize_signals() stamps the vocabulary's kind onto every IntakeSignal; unknown ids still pass through. This is the same Observation-vs-Requirement split the Requirements Verification Platform rests on: observations are reality, requirements are targets, and their comparison is the delta. A tender / OEM spec / law now produces requirement signals; scanners / repos / documents produce observation signals. Tests: rewrote the two test_signal_producer cases that previously ASSERTED the bug (tender == repo) to pin the correct split; regression — `requires_sbom` yields no capability + stays in requirements_seen while `cyclonedx_found` still detects sbom_creation; endpoint-level regression that a tender requirement does not auto-detect and the gap stays asked; vocabulary-kind-overrides-mislabelled-producer. 25 onboarding tests pass, mypy --strict clean, demo runs, check-loc 0. Runtime effect → deploy + smoke. (Fix A; partial-vs- detected decoupling follows as Fix B before #59.)
68 lines
2.9 KiB
Python
68 lines
2.9 KiB
Python
"""POST /onboarding/advisor-start — the runtime endpoint that exposes the existing Advisor.
|
|
|
|
Exercises the router in isolation (no DB, no full app): scanner findings (ProducedSignal) -> normalize
|
|
-> Silent Pass -> Advisor -> the advisory payload. No new reasoning logic — just the wiring.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from fastapi import FastAPI
|
|
from fastapi.testclient import TestClient
|
|
|
|
from compliance.api.onboarding_routes import router
|
|
|
|
_app = FastAPI()
|
|
_app.include_router(router)
|
|
_client = TestClient(_app)
|
|
|
|
_BODY = {
|
|
"company": "synthetic", "industry": "machine_builder", "products": ["parking payment system"],
|
|
"markets": ["EU"], "certifications": ["ISO27001", "ISO9001"], "known_evidence": ["CE process"],
|
|
"target": "CRA",
|
|
"scanner_findings": [
|
|
{"signal_id": "cyclonedx_found", "source_type": "repository", "evidence": "sbom", "provenance": "sbom.cdx.json"},
|
|
{"signal_id": "vdp_found", "source_type": "website", "provenance": "/.well-known/security.txt"},
|
|
{"signal_id": "risk_assessment_pdf", "source_type": "document", "provenance": "risk.pdf"},
|
|
{"signal_id": "cloud_hosted", "source_type": "product"},
|
|
],
|
|
}
|
|
|
|
|
|
def test_targets_endpoint_lists_supported():
|
|
r = _client.get("/onboarding/targets")
|
|
assert r.status_code == 200
|
|
assert "CRA" in r.json()["targets"]
|
|
|
|
|
|
def test_advisor_start_returns_full_payload():
|
|
r = _client.post("/onboarding/advisor-start", json=_BODY)
|
|
assert r.status_code == 200, r.text
|
|
d = r.json()
|
|
for field in ["silent_intake_summary", "inferred_assumptions", "rejected_assumptions",
|
|
"top_5_questions", "capability_delta", "top_measures", "evidence_requests",
|
|
"completeness_summary"]:
|
|
assert field in d
|
|
assert len(d["top_5_questions"]) <= 5
|
|
assert d["auto_detected"] # Silent Pass recognised things from the scanners
|
|
assert "sbom_creation" not in {q["capability_id"] for q in d["top_5_questions"]} # detected -> not asked
|
|
|
|
|
|
def test_requirement_signal_does_not_auto_detect_capability():
|
|
# a tender that DEMANDS an SBOM (requirement) must NOT be read as "SBOM present": sbom_creation stays
|
|
# open (asked / in the delta), unlike a real cyclonedx_found observation.
|
|
body = dict(_BODY, scanner_findings=[
|
|
{"signal_id": "requires_sbom", "source_type": "tender", "provenance": "tender §4.2"},
|
|
])
|
|
r = _client.post("/onboarding/advisor-start", json=body)
|
|
assert r.status_code == 200, r.text
|
|
d = r.json()
|
|
assert "sbom_creation" not in d["auto_detected"] # demanded != present
|
|
asked = {q["capability_id"] for q in d["top_5_questions"]}
|
|
assert "sbom_creation" in asked or "sbom_creation" in d["capability_delta"] # still an open gap
|
|
|
|
|
|
def test_unknown_target_is_404():
|
|
body = dict(_BODY, target="NOPE")
|
|
r = _client.post("/onboarding/advisor-start", json=body)
|
|
assert r.status_code == 404
|