Per the user's decision (2026-06-28): observations are CALIBRATION data for the knowledge base, NOT
business data and NOT product-DB data. So they live with the other versioned knowledge artifacts as an
append-only JSONL log under knowledge/observations/ — NO migration, NO DB. (A real persistence layer is
only warranted once thousands of onboardings exist; not before.)
- ObservationRecord = Observation + log metadata (observation_id, timestamp [caller-stamped, no hidden
clock], customer_archetype [anonymised — NEVER a real name], evidence, provenance, knowledge_version).
- append_observation() writes one JSON line; append-only, lines are never rewritten. A later review is a
NEW line with the same observation_id; load_observations(reconcile=True) keeps the latest per id.
- load_observations() reads a single .jsonl or a directory of monthly .jsonl files.
- aggregate_by_hypothesis() (59c) -> per-hypothesis distribution + confidence, COMPUTED from the log
(computed-not-stored); the review gate (reviewed-only) is enforced in empirical_distribution/confidence.
- review_queue() -> the unreviewed worklist. Observation -> Review -> Accepted -> recompute, never
Observation -> confidence++. Nothing is ever written back to a hypothesis.
You can `rm` the log and recompute, `git diff` it over months, or rebuild confidence under a new policy —
fully consistent with computed-not-stored and the product/knowledge data separation.
Non-runtime (module + tests only, no endpoint) -> origin/main, NO dev deploy. 5 new tests (append-only,
review supersession, review-gate statistics, queue, monthly-file load); 27 onboarding tests pass, mypy
--strict clean (9 modules), check-loc 0. 59d (surface computed confidence at runtime) stays a later step.
Two reviewed knowledge decisions (2026-06-28) + the deferred cosmetic counter, before #59.
1. ISO13485 removed from the incident_management hypothesis. ISO 13485 CAPA / quality-safety incident
handling is NOT security incident management — the mapping was too broad and would seed false
hypotheses for the empirical loop. A dedicated manage_quality_and_safety_incidents capability can come
later IF a target needs it; not forced now. (ISO27001/TISAX/IEC62443 keep incident_management.)
2. patch_policy_doc -> secure_signed_update_distribution stays `partial`, but the curated rationale is
sharpened: "indicates update governance, does not evidence signed distribution" (a patch policy is not
proof of SIGNED distribution). New optional SignalMapping.rationale field carries the curated note.
(github_actions_ci -> SDL and dependency_scanning -> vuln-mgmt reviewed and APPROVED as-is.)
3. Cosmetic (folded in since we touched the file): the silent-intake summary now counts detected and
indications SEPARATELY ("N automatisch erkannt, M Indikation(en)") instead of lumping partial signals
into "automatisch erkannt" — consistent with the three-state model just shipped.
Tests: ISO13485 no longer resolves to incident_management; summary counts split correctly. 29 onboarding
tests pass, mypy --strict clean, demo runs, check-loc 0. Runtime-visible (hypothesis resolution + summary
text) -> deploy + smoke.
Fix B of the pre-#59 semantic correction. The Silent Pass had only TWO effective states though the data
carries three: a `detected` mapping (a concrete artifact) AND a `partial` mapping (an indicative signal,
e.g. a CI pipeline -> secure-development-lifecycle) both flowed through capability_ids() and were fed to
the Advisor as already-present — so a weak indication silently removed a question, exactly the Welt-1/
Welt-2 transparency we want to keep.
Now three distinct states:
- detected -> reduces the delta immediately (auto_detected, not asked). [unchanged]
- partial -> raises assumption strength but does NOT replace the question (surfaced as `indications`,
the capability stays in the delta and is still asked).
- requirement-> describes a target, never the present state (already handled by Fix A's kind split).
Changes (data + thin wiring, no new architecture):
- SilentIntakeResult.capability_ids() returns only relationship==detected; new indicative_capability_ids()
returns the partial ones.
- advisor_start() gains indicative_capabilities (NOT fed into the profile) and surfaces result.indications
= indicative ∩ required − auto_detected.
- AdvisorResult / AdvisorResponse gain `indications` (additive, contract-safe); the service passes the
indicative ids through.
Tests: a partial CI signal is indicative-not-detected and does NOT shrink the delta; end-to-end it appears
in `indications`, not `auto_detected`, and the gap is still asked. 28 onboarding tests pass, mypy --strict
clean on the onboarding modules, demo runs, check-loc 0. Runtime effect -> deploy + smoke.
Semantic correction of the knowledge base BEFORE the empirical loop (#59) is built — otherwise the
Observation Store would learn from already-misclassified signals. The Silent Pass conflated two kinds of
signal into one: an OBSERVATION ("I saw an SBOM in the repo") and a REQUIREMENT ("a tender DEMANDS an
SBOM"). They were aliased to the same canonical id, so a tender clause read as "SBOM already present" and
suppressed the very question that should have been asked.
Fix — make the kind explicit and authoritative (no new architecture, data + thin wiring):
- `kind` ∈ {observation, requirement} on ProducedSignal (producer may declare) and on the canonical
SignalVocabularyEntry (AUTHORITATIVE — a mislabelled producer cannot collapse the two).
- Vocabulary split: sbom_file_found → sbom_present (obs) + sbom_required (req);
security_txt_or_cvd_policy → cvd_policy_present (obs) + psirt_required (req); add signed_updates_required.
requirement signals are intentionally UNMAPPED in intake_signal_map (they describe a target, not state).
- silent_intake() consumes ONLY kind==observation; requirement signals are preserved in
`requirements_seen` (visible/auditable) but NEVER become a detected capability.
- normalize_signals() stamps the vocabulary's kind onto every IntakeSignal; unknown ids still pass through.
This is the same Observation-vs-Requirement split the Requirements Verification Platform rests on:
observations are reality, requirements are targets, and their comparison is the delta. A tender / OEM spec /
law now produces requirement signals; scanners / repos / documents produce observation signals.
Tests: rewrote the two test_signal_producer cases that previously ASSERTED the bug (tender == repo) to pin
the correct split; regression — `requires_sbom` yields no capability + stays in requirements_seen while
`cyclonedx_found` still detects sbom_creation; endpoint-level regression that a tender requirement does not
auto-detect and the gap stays asked; vocabulary-kind-overrides-mislabelled-producer. 25 onboarding tests
pass, mypy --strict clean, demo runs, check-loc 0. Runtime effect → deploy + smoke. (Fix A; partial-vs-
detected decoupling follows as Fix B before #59.)
Not scanner stubs — the scanners exist. The Silent Pass needs only their UNIFIED output. This adds the
small common DATA FORMAT (not a new module/framework) the user asked for, exactly the Requirement-
Source / MCAP / regulation-alias pattern: many inputs, one language.
Producer A / B / C -> normalize_signals (vocabulary: id + aliases) -> canonical IntakeSignal -> Silent Pass
- ProducedSignal {signal_id, source_type, confidence, evidence, provenance} = what ANY source emits
(website scanner, repo scanner, PDF parser, tender parser, API, the user).
- knowledge/onboarding/signal_vocabulary.yaml reduces producer dialects to a canonical signal: "SBOM
present" arrives as cyclonedx_found / spdx_found / sbom_uploaded / requires_sbom (tender) — all become
`sbom_file_found`. The Silent Pass cannot tell where it came from -> no per-scanner special logic, ever.
- Unknown signals pass through (a new producer stays visible). confidence/evidence/provenance flow to
the detected capability for the audit trail.
A tender that "requires SBOM" now produces the same effect as a repo that HAS one — fits Vision V2
(Requirement Source over Regulation). Endpoint (#58) then has its final shape: POST -> Producers ->
Normalizer -> Silent Pass -> Profile -> Delta -> Questions -> Roadmap. Non-runtime -> no deploy. mypy
--strict clean, 14 onboarding tests pass, check-loc 0.
Not the endpoint yet — the bigger knowledge lever first. The Advisor can say "I need 5 answers" but
does not yet decide what it can find out by ITSELF. The Silent Knowledge Pass runs in front of the
Advisor and, from signals existing scanners/parsers already produce (website, repository, documents,
product data), deterministically derives capabilities the company demonstrably HAS + product facts
that drive scope — so every recognised item shrinks the delta and removes a question.
compliance/onboarding/silent_intake.py: silent_intake(signals, signal_map) -> detected_capabilities
(+ evidence already in hand) + product_facts. The signal->conclusion map is curated DATA
(knowledge/onboarding/intake_signal_map.yaml), signals are injected (scanners are upstream). Pure,
deterministic, no LLM. advisor_start gains detected_capabilities (folded into the profile at HIGH
confidence -> covered, not asked) and an auto_detected result + headline.
The experience flips from a question wall to "we already recognised 4 capabilities, 2 product facts
and have 4 pieces of evidence in hand — only these few remain". Order now: Silent Pass -> #58
endpoint/frontend -> #59 empirical loop. NOT new architecture, just an orchestration step in front.
Non-runtime (no app caller) -> no deploy. 15 onboarding tests pass, mypy --strict clean, check-loc 0.
The learning point is not the hypothesis, it is the QUESTION — and confirmed/refuted is too coarse.
"partial, only critical suppliers" or "certified but not lived" are not "wrong", they are valuable
knowledge. So the chain is Hypothesis -> Question -> Observation -> (Review) -> Hypothesis, and the
observation model must be defined cleanly before any store/API (else thousands of too-coarse
observations get migrated later).
compliance/onboarding/observations.py:
- ObservationType: confirmed / partial / refuted / not_applicable / unknown (richer than binary).
- Observation: {hypothesis_id, capability, question, answer (free text), observation_type,
scope_note ("only critical suppliers"), evidence_uploaded, reviewed, reviewed_by}.
- empirical_distribution() -> a DISTRIBUTION (confirmed 61 / partial 31 / refuted 8), not one %.
- empirical_confidence() -> (confirmed + 0.5*partial) / (confirmed+partial+refuted); n.a./unknown
excluded; None until calibrated.
- REVIEW GATE: only reviewed observations calibrate — a raw answer never changes a hypothesis (no
learning from outliers).
Refactor: the hypothesis is now PURE curated knowledge — the binary observations counter and any
confidence are removed from CapabilityHypothesis and the YAML; confidence is COMPUTED from the separate
reviewed observation stream. Pure, mypy --strict clean. Persistence/aggregation/calibration are 59b/c/d.
Non-runtime -> no deploy. 12 tests pass, check-loc 0.
The bottleneck is knowledge, not the endpoint. This builds the knowledge the Onboarding Advisor needs,
restructured per the user's key insight: NOT "ISO27001 -> 30 capabilities" but each hypothesis as its
own object "capability -> supported_by: [certs]". A capability is written ONCE with all supporting
certs, so the shared management-system core (document control, incident, supplier, audit, access,
asset, monitoring, training, crypto, release, risk) covers most certifications with ~18 hypotheses
instead of ~300 — and multi-certification merges AUTOMATICALLY (a company's inferred caps = every
hypothesis whose supported_by intersects its certs).
Welt-1 throughout: "IF cert present, EXPECT capability (verification required)", never "erfüllt".
Capabilities NO cert suggests (SBOM, signed updates, CVD, support period) have no hypothesis -> they
stay in the delta and get asked. confidence is EMPIRICAL: computed from real-onboarding observations
(confirmed/(confirmed+refuted)), None until calibrated — never an LLM/expert score (record_observation
+ empirical_confidence). The long-term moat: knowledge that learns from reality, not from a norm.
compliance/onboarding/hypotheses.py (resolve_for_certifications / inferred_hypotheses / empirical_
confidence / record_observation) feeds the existing advisor_start unchanged; the demo now runs on the
curated library. Pure, mypy --strict clean, library is DATA (no norm text, no real names). Non-runtime
-> no deploy. 12 tests pass, check-loc 0.
The user-named "right next runtime step": stop building knowledge, start using it automatically in
onboarding — no sales training, no regulation picking. compliance/onboarding/ is an ORCHESTRATOR (not
a new engine) wiring Company 2A -> RS-005 -> optimization -> completeness:
advisor_start(input, cert_hypotheses, target_requirements, ...) -> AdvisorResult
From (company + products + certifications + target) it returns inferred_assumptions, rejected_
assumptions, next_best_questions (<=5, ranked by information_gain + leverage + unknown_high_risk +
evidence_missing, each self-explaining), capability_delta, top_measures, evidence_requests,
unsupported_domains, completeness_summary. apply_answer() updates the profile (delta shrinks).
Welt-1 throughout: certificates REDUCE questions but satisfy nothing automatically (verification_
required); relevance(evidence,target) keeps ISO 14001 out of the CRA result. Certificate->capability
hypotheses + target requirements are INJECTED (curated knowledge, outsourced; not in code).
All 7 acceptance criteria pass; mypy --strict clean. First app-caller wiring the engines into a
product flow — still no endpoint/persistence, so 0 runtime effect -> no deploy yet (deploys when
POST /onboarding/advisor-start + frontend are wired). check-loc 0.
Benjamin Admin