Benjamin Admin
98f67e75d9
Turn the architecture inside-out: instead of refining classes/registries/journeys, force the whole
platform to behave as ONE expert system and run a real consulting project end-to-end — measuring how
often the consultant has to "jump" (special-case glue instead of a clean engine-to-engine handoff). A
Reference Scenario asks "is the knowledge correct?"; a Customer Mission asks "can a customer WORK with
it?". This is the last big architecture test before broad corpus expansion.
- reference_scenarios/mission_machine_builder.py: a synthetic machine builder (ISO9001 + ISMS + CE +
PLC + remote maintenance + cloud + 80 devs + EU; no real names) asks "what must I do in the next 6
months?". Runs the REAL engines: Regulatory Map -> Journey selection -> Capability Delta (RS-005) ->
Roadmap (leverage) -> Playbooks -> Evidence -> Verification -> Completeness, and produces the 6-month
consulting answer ("the top-5 measures close 9/16 = 56%, starting with the ones that satisfy CRA AND
MaschinenVO at once").
- Flow-Continuity audit (the actual test): 5 CLEAN, 2 JUMPS, 2 deliberate DEPENDENCIES. The two real
seams: (1) Scope -> Journey (no `certs x targets -> journeys` selector engine; the data exists in
transitions.yaml, only the selection is glue); (2) Evidence -> Verification (parked, Vision V2). The
two dependencies (cert->capability map @Execution, corpus_status curation) are intended ownership
boundaries, not architecture breaks.
- Finding: the platform carries the WHOLE consulting flow end-to-end. Once the Scope->Journey selector
exists, the foundation is essentially done — from there the work is knowledge, not architecture.
4 end-to-end tests (mission runs, exactly two known jumps, full flow present, no real company names).
check-loc 0. Non-runtime harness -> no deploy (ADR-001).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
51 lines
1.9 KiB
Python
51 lines
1.9 KiB
Python
"""End-to-end test for Customer Mission #1 — the whole platform as ONE connected expert system.
|
|
|
|
This is NOT a knowledge-correctness test (that is the Reference Scenarios). It runs the FULL consulting
|
|
flow with the real engines and asserts the flow-continuity audit: the platform must carry a synthetic
|
|
machine builder from „what applies?" to a prioritised 6-month plan, with exactly the two known jumps
|
|
(Scope→Journey selector missing; Evidence→Verification parked) and no others creeping in.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import subprocess
|
|
import sys
|
|
|
|
|
|
def _run_mission():
|
|
root = os.path.join(os.path.dirname(__file__), "..")
|
|
r = subprocess.run(
|
|
[sys.executable, "reference_scenarios/mission_machine_builder.py"],
|
|
cwd=root, env={**os.environ, "PYTHONPATH": "."}, capture_output=True, text=True,
|
|
)
|
|
assert r.returncode == 0, r.stderr
|
|
return r.stdout
|
|
|
|
|
|
def test_mission_runs_end_to_end():
|
|
out = _run_mission()
|
|
assert "Customer Mission #1" in out and "Flow-Continuity-Audit" in out
|
|
# the consulting answer must be produced (top-5 leverage closes 9/16 = 56%)
|
|
assert "56%" in out and "6-Monats-Antwort" in out
|
|
|
|
|
|
def test_exactly_two_real_jumps_no_regressions():
|
|
out = _run_mission()
|
|
# the flow must stay continuous: exactly the two KNOWN seams, no new ones
|
|
assert "2 Sprünge" in out
|
|
assert "Scope → Journey" in out and "Evidence → Verification" in out
|
|
|
|
|
|
def test_full_consulting_flow_present():
|
|
out = _run_mission()
|
|
for stage in ["1. Scope", "2. Journey", "3. Capability Delta", "4. Roadmap",
|
|
"5. Playbooks", "6. Nachweise", "7. Verification", "8. Completeness"]:
|
|
assert stage in out
|
|
|
|
|
|
def test_no_real_company_names():
|
|
out = _run_mission().lower()
|
|
for name in ["eto", "owis", "winterhalter"]:
|
|
assert name not in out # synthetic archetype only
|