Benjamin Admin
cefacb87af
The focus has shifted: no more architecture epics (the Journey Matcher was the last building block). The question is no longer "can the architecture do this?" but "where does it fail under real domain knowledge?". This operationalises the two KPIs almost nobody measures, as a non- runtime, auditable ledger: - Architecture Stability : per integrated Requirement Source — new runtime classes? new pipeline? - Knowledge Velocity : can a domain EXPERT integrate a source data-only, without a developer? A new domain is a ROW in knowledge/architecture_stability/integration_ledger.yaml (data), never a code change — so the KPI improves by adding data, which IS the proof. Current state: 6 sources across 5 target types (CRA, MaschinenVO, TISAX, Tender, OEM, Environmental) = 6/6 = 100% stability and 100% data-only. The pipeline functions are listed honestly as one-time, domain-agnostic infrastructure (now frozen), so the KPI cannot be gamed. The test is a LIVING GUARDRAIL: it fails the day a source needs runtime code, surfacing the exact moment generality breaks. Non-runtime -> no deploy. 5 tests pass, check-loc 0.
67 lines
2.3 KiB
Python
67 lines
2.3 KiB
Python
"""Architecture Stability + Knowledge Velocity KPI — Phase Ω guardrail.
|
|
|
|
This is not a feature test; it is a LIVING GUARDRAIL. The day someone integrates a Requirement Source
|
|
that needs a new runtime class or a new pipeline (i.e. NOT data-only), `test_every_source_is_data_only`
|
|
fails — surfacing the exact moment the architecture stopped being general. That is the whole point of
|
|
Phase Ω: measure where it breaks under real domain knowledge.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import subprocess
|
|
import sys
|
|
|
|
import yaml
|
|
|
|
|
|
def _ledger():
|
|
p = os.path.join(os.path.dirname(__file__), "..", "knowledge", "architecture_stability", "integration_ledger.yaml")
|
|
return yaml.safe_load(open(p, encoding="utf-8"))
|
|
|
|
|
|
def _run():
|
|
root = os.path.join(os.path.dirname(__file__), "..")
|
|
r = subprocess.run(
|
|
[sys.executable, "reference_scenarios/architecture_stability_kpi.py"],
|
|
cwd=root, env={**os.environ, "PYTHONPATH": "."}, capture_output=True, text=True,
|
|
)
|
|
assert r.returncode == 0, r.stderr
|
|
return r.stdout
|
|
|
|
|
|
def test_every_source_is_data_only_and_zero_runtime():
|
|
# GUARDRAIL: every integrated source must cost 0 runtime classes, no new pipeline, data-only.
|
|
# If a future domain breaks this, fix the architecture or record the break honestly — do not
|
|
# weaken this assertion to make it pass.
|
|
for s in _ledger()["sources"]:
|
|
assert s["new_runtime_classes"] == 0, s["source"]
|
|
assert s["new_pipeline"] is False, s["source"]
|
|
assert s["integration_kind"] == "data_only", s["source"]
|
|
|
|
|
|
def test_kpis_reported_at_full_stability():
|
|
out = _run()
|
|
n = len(_ledger()["sources"])
|
|
assert "Architecture Stability: %d/%d = 100%%" % (n, n) in out
|
|
assert "Knowledge Velocity: %d/%d = 100%%" % (n, n) in out
|
|
|
|
|
|
def test_pipeline_functions_are_one_time_infrastructure():
|
|
out = _run()
|
|
assert "EINMALIG (jetzt eingefroren)" in out
|
|
assert "journey_matcher" in out and "letzte architektonische Baustein" in out
|
|
|
|
|
|
def test_three_knowledge_layers_present():
|
|
out = _run()
|
|
for layer in ["Beschreibend", "Transformation", "Produktion"]:
|
|
assert layer in out
|
|
|
|
|
|
def test_three_success_questions_present():
|
|
out = _run()
|
|
assert "Musste für eine neue Domäne Runtime-Code geändert werden?" in out
|
|
assert "Knowledge Velocity" in out
|
|
assert "Architecture Stability" in out
|