Merge pull request 'Architecture Stability + Knowledge Velocity KPI (Phase Omega)' (#35) from feat/architecture-stability-kpi into main
This commit is contained in:
pilotadmin
@@ -0,0 +1,80 @@
|
||||
# Architecture Stability + Knowledge Velocity ledger — Phase Ω (Evidence of Generality).
|
||||
#
|
||||
# The question is no longer "can the architecture do this?" but "where does it fail under real domain
|
||||
# knowledge?". Two KPIs almost nobody measures:
|
||||
# - Architecture Stability : per integrated Requirement Source — new runtime classes? new pipeline?
|
||||
# (target: 0 / 0)
|
||||
# - Knowledge Velocity : can a DOMAIN EXPERT integrate a new source WITHOUT a software developer?
|
||||
# (target: every source = data_only)
|
||||
#
|
||||
# HOW TO INTEGRATE A NEW SOURCE: add a ROW under `sources`. That is the whole point — a new domain is a
|
||||
# DATA change here, never a code change. If you ever have to add a row under `pipeline_functions`, the
|
||||
# stability claim broke and Phase Ω failed; record it honestly.
|
||||
|
||||
# --- Integrated Requirement Sources: each is DATA (a pattern / a Required set), run by the shared pipeline ---
|
||||
sources:
|
||||
- source: "Cyber Resilience Act (CRA)"
|
||||
domain: industrial_automation
|
||||
target_type: regulation
|
||||
integrated_as: transition_pattern_data
|
||||
new_runtime_classes: 0
|
||||
new_pipeline: false
|
||||
integration_kind: data_only
|
||||
exercised_by: "customer_mission_1/2/3, journey_matcher_demo"
|
||||
- source: "Maschinenverordnung (MaschinenVO)"
|
||||
domain: industrial_automation
|
||||
target_type: regulation
|
||||
integrated_as: transition_pattern_data
|
||||
new_runtime_classes: 0
|
||||
new_pipeline: false
|
||||
integration_kind: data_only
|
||||
exercised_by: "customer_mission_1/3, journey_matcher_demo"
|
||||
- source: "TISAX"
|
||||
domain: automotive
|
||||
target_type: certification
|
||||
integrated_as: transition_pattern_data
|
||||
new_runtime_classes: 0
|
||||
new_pipeline: false
|
||||
integration_kind: data_only
|
||||
exercised_by: "customer_mission_3/5, journey_matcher_demo"
|
||||
- source: "Public Tender (öffentliche Ausschreibung)"
|
||||
domain: cross_industry
|
||||
target_type: contract
|
||||
integrated_as: injected_required_set
|
||||
new_runtime_classes: 0
|
||||
new_pipeline: false
|
||||
integration_kind: data_only
|
||||
exercised_by: "customer_mission_3/4"
|
||||
- source: "OEM Specification (Lastenheft)"
|
||||
domain: automotive
|
||||
target_type: contract
|
||||
integrated_as: injected_required_set
|
||||
new_runtime_classes: 0
|
||||
new_pipeline: false
|
||||
integration_kind: data_only
|
||||
exercised_by: "customer_mission_4"
|
||||
- source: "Environmental / Material evidence target"
|
||||
domain: environmental
|
||||
target_type: environmental
|
||||
integrated_as: injected_required_set
|
||||
new_runtime_classes: 0
|
||||
new_pipeline: false
|
||||
integration_kind: data_only
|
||||
exercised_by: "customer_mission_5"
|
||||
|
||||
# --- One-time, domain-AGNOSTIC pipeline functions (built once, now FROZEN per Phase Ω). ---
|
||||
# Listed for honesty so the stability KPI cannot be gamed: these are NOT per-domain costs. The last
|
||||
# one (journey_matcher) was the final architectural building block.
|
||||
pipeline_functions:
|
||||
- { fn: "transition_reasoning (RS-005)", maps: "Capability -> Delta", layer: transformation }
|
||||
- { fn: "optimization", maps: "Delta -> Roadmap", layer: transformation }
|
||||
- { fn: "journey_matcher (ADR-011)", maps: "Delta -> Journey", layer: transformation }
|
||||
- { fn: "playbook", maps: "Capability -> Playbook", layer: production }
|
||||
- { fn: "completeness", maps: "coverage audit", layer: production }
|
||||
- { fn: "company (2A)", maps: "Evidence -> Capability", layer: descriptive }
|
||||
|
||||
# --- The architecture has settled into three non-overlapping knowledge layers (a good sign). ---
|
||||
knowledge_layers:
|
||||
descriptive: ["Requirements", "Capabilities", "Evidence"] # what IS
|
||||
transformation: ["Delta", "Journey", "Roadmap"] # how to MOVE
|
||||
production: ["Playbooks", "Verification", "Reference Scenarios"] # how to DO + PROVE
|
||||
@@ -0,0 +1,36 @@
|
||||
# Architecture Stability + Knowledge Velocity — Phase Ω (Evidence of Generality)
|
||||
|
||||
_Der Fokus hat sich verschoben: nicht mehr „kann die Architektur das?", sondern „wo versagt sie bei echtem Fachwissen?". Diese zwei KPIs erhebt kaum jemand. Eine neue Domäne ist eine ZEILE im Ledger (Daten), nie eine Codeänderung — genau das macht den KPI auditierbar._
|
||||
|
||||
## Architecture Stability — pro integrierter Anforderungsquelle: neue Runtime-Klassen? neue Pipeline?
|
||||
|
||||
| Quelle | Zieltyp | als | neue Runtime-Klassen | neue Pipeline | Ergebnis |
|
||||
|---|---|---|---:|---:|---|
|
||||
| Cyber Resilience Act (CRA) | regulation | transition_pattern_data | 0 | 0 | ✅ |
|
||||
| Maschinenverordnung (MaschinenVO) | regulation | transition_pattern_data | 0 | 0 | ✅ |
|
||||
| TISAX | certification | transition_pattern_data | 0 | 0 | ✅ |
|
||||
| Public Tender (öffentliche Ausschreibung) | contract | injected_required_set | 0 | 0 | ✅ |
|
||||
| OEM Specification (Lastenheft) | contract | injected_required_set | 0 | 0 | ✅ |
|
||||
| Environmental / Material evidence target | environmental | injected_required_set | 0 | 0 | ✅ |
|
||||
|
||||
- **Architecture Stability: 6/6 = 100%** der Quellen ohne neue Runtime-Klasse und ohne neue Pipeline.
|
||||
- **Knowledge Velocity: 6/6 = 100%** der Quellen **data-only** integriert (kein Entwickler nötig).
|
||||
|
||||
## Ehrlichkeit: die Pipeline-Funktionen sind EINMALIG (jetzt eingefroren)
|
||||
- 6 domänen-AGNOSTISCHE Funktionen, einmal gebaut, nicht je Domäne: `transition_reasoning (RS-005)`, `optimization`, `journey_matcher (ADR-011)`, `playbook`, `completeness`, `company (2A)`.
|
||||
- Die letzte (`journey_matcher`) war der **letzte architektonische Baustein** (ADR-011). Ab hier: Wissensarbeit, nicht Architektur.
|
||||
|
||||
## Drei saubere Wissensebenen (greifen ineinander, vermischen sich nicht)
|
||||
| Ebene | Inhalt |
|
||||
|---|---|
|
||||
| **Beschreibend** (was IST) | Requirements, Capabilities, Evidence |
|
||||
| **Transformation** (wie BEWEGEN) | Delta, Journey, Roadmap |
|
||||
| **Produktion** (wie TUN/BEWEISEN) | Playbooks, Verification, Reference Scenarios |
|
||||
|
||||
## Die drei Erfolgsfragen ab jetzt (statt Coverage)
|
||||
1. **Musste für eine neue Domäne Runtime-Code geändert werden?** → bisher: **nein** (6/6).
|
||||
2. **Knowledge Velocity** — neues Wissen ohne Entwickler aufnehmbar? → bisher: **ja** (6/6 data-only).
|
||||
3. **Architecture Stability** — bestehende Capability/Journey strukturell ändern oder nur Daten ergänzen? → bisher: **nur Daten**.
|
||||
|
||||
> **Befund:** Über fünf Zielarten und sechs Quellen blieb `Reality → Evidence → Capability → Required → Delta → Journey → Roadmap → Playbooks → Verification` unverändert. Das ist der eigentliche Nachweis: keine Compliance-Architektur, sondern eine allgemeine Requirements-Verifikationsarchitektur, die ihre Generalität UNTER realer fachlicher Belastung behält. Der nächste Test ist nicht ein Feature, sondern die nächste echte Domäne (Umwelt-Cluster · Automotive · Medizintechnik · Payment) — jede als neue Ledger-Zeile, bei stabilem KPI.
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# ruff: noqa
|
||||
# mypy: ignore-errors
|
||||
"""Architecture Stability + Knowledge Velocity KPI — Phase Ω (Evidence of Generality).
|
||||
|
||||
Not "can the architecture do this?" but "where does it fail under real domain knowledge?". This reads
|
||||
the integration ledger and computes two KPIs almost nobody measures:
|
||||
- Architecture Stability : share of integrated Requirement Sources that needed 0 new runtime classes
|
||||
AND no new pipeline.
|
||||
- Knowledge Velocity : share of sources a DOMAIN EXPERT could integrate data-only (no developer).
|
||||
|
||||
A new domain is a ROW in the ledger (data), never a code change — so this KPI literally improves by
|
||||
adding data, which is the proof. Non-runtime -> no deploy.
|
||||
Run: cd backend-compliance && PYTHONPATH=. python3 reference_scenarios/architecture_stability_kpi.py
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import yaml
|
||||
|
||||
OUT = []
|
||||
|
||||
|
||||
def w(s=""):
|
||||
OUT.append(s)
|
||||
|
||||
|
||||
_LEDGER = os.path.join(os.path.dirname(__file__), "..", "knowledge", "architecture_stability", "integration_ledger.yaml")
|
||||
L = yaml.safe_load(open(_LEDGER, encoding="utf-8"))
|
||||
sources = L["sources"]
|
||||
n = len(sources)
|
||||
|
||||
stable = [s for s in sources if s["new_runtime_classes"] == 0 and not s["new_pipeline"]]
|
||||
data_only = [s for s in sources if s["integration_kind"] == "data_only"]
|
||||
arch_stability = len(stable) / n if n else 0.0
|
||||
knowledge_velocity = len(data_only) / n if n else 0.0
|
||||
|
||||
w("# Architecture Stability + Knowledge Velocity — Phase Ω (Evidence of Generality)")
|
||||
w("")
|
||||
w('_Der Fokus hat sich verschoben: nicht mehr „kann die Architektur das?", sondern „wo versagt sie bei echtem Fachwissen?". Diese zwei KPIs erhebt kaum jemand. Eine neue Domäne ist eine ZEILE im Ledger (Daten), nie eine Codeänderung — genau das macht den KPI auditierbar._')
|
||||
w("")
|
||||
w("## Architecture Stability — pro integrierter Anforderungsquelle: neue Runtime-Klassen? neue Pipeline?")
|
||||
w("")
|
||||
w("| Quelle | Zieltyp | als | neue Runtime-Klassen | neue Pipeline | Ergebnis |")
|
||||
w("|---|---|---|---:|---:|---|")
|
||||
for s in sources:
|
||||
ok = "✅" if (s["new_runtime_classes"] == 0 and not s["new_pipeline"]) else "❌"
|
||||
w("| %s | %s | %s | %d | %s | %s |" % (
|
||||
s["source"], s["target_type"], s["integrated_as"], s["new_runtime_classes"],
|
||||
"ja" if s["new_pipeline"] else "0", ok))
|
||||
w("")
|
||||
w("- **Architecture Stability: %d/%d = %d%%** der Quellen ohne neue Runtime-Klasse und ohne neue Pipeline." % (
|
||||
len(stable), n, round(arch_stability * 100)))
|
||||
w("- **Knowledge Velocity: %d/%d = %d%%** der Quellen **data-only** integriert (kein Entwickler nötig)." % (
|
||||
len(data_only), n, round(knowledge_velocity * 100)))
|
||||
w("")
|
||||
|
||||
# pipeline functions = one-time, domain-agnostic infrastructure (honesty: not per-domain costs)
|
||||
pf = L["pipeline_functions"]
|
||||
w("## Ehrlichkeit: die Pipeline-Funktionen sind EINMALIG (jetzt eingefroren)")
|
||||
w("- %d domänen-AGNOSTISCHE Funktionen, einmal gebaut, nicht je Domäne: %s." % (
|
||||
len(pf), ", ".join("`%s`" % f["fn"] for f in pf)))
|
||||
w("- Die letzte (`journey_matcher`) war der **letzte architektonische Baustein** (ADR-011). Ab hier: Wissensarbeit, nicht Architektur.")
|
||||
w("")
|
||||
|
||||
# three knowledge layers (the architecture has settled symmetrically)
|
||||
kl = L["knowledge_layers"]
|
||||
w("## Drei saubere Wissensebenen (greifen ineinander, vermischen sich nicht)")
|
||||
w("| Ebene | Inhalt |")
|
||||
w("|---|---|")
|
||||
w("| **Beschreibend** (was IST) | %s |" % ", ".join(kl["descriptive"]))
|
||||
w("| **Transformation** (wie BEWEGEN) | %s |" % ", ".join(kl["transformation"]))
|
||||
w("| **Produktion** (wie TUN/BEWEISEN) | %s |" % ", ".join(kl["production"]))
|
||||
w("")
|
||||
|
||||
w("## Die drei Erfolgsfragen ab jetzt (statt Coverage)")
|
||||
w("1. **Musste für eine neue Domäne Runtime-Code geändert werden?** → bisher: **nein** (%d/%d)." % (len(stable), n))
|
||||
w("2. **Knowledge Velocity** — neues Wissen ohne Entwickler aufnehmbar? → bisher: **ja** (%d/%d data-only)." % (len(data_only), n))
|
||||
w("3. **Architecture Stability** — bestehende Capability/Journey strukturell ändern oder nur Daten ergänzen? → bisher: **nur Daten**.")
|
||||
w("")
|
||||
w('> **Befund:** Über fünf Zielarten und sechs Quellen blieb `Reality → Evidence → Capability → Required → Delta → Journey → Roadmap → Playbooks → Verification` unverändert. Das ist der eigentliche Nachweis: keine Compliance-Architektur, sondern eine allgemeine Requirements-Verifikationsarchitektur, die ihre Generalität UNTER realer fachlicher Belastung behält. Der nächste Test ist nicht ein Feature, sondern die nächste echte Domäne (Umwelt-Cluster · Automotive · Medizintechnik · Payment) — jede als neue Ledger-Zeile, bei stabilem KPI.')
|
||||
w("")
|
||||
|
||||
print("\n".join(OUT))
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Architecture Stability + Knowledge Velocity KPI — Phase Ω guardrail.
|
||||
|
||||
This is not a feature test; it is a LIVING GUARDRAIL. The day someone integrates a Requirement Source
|
||||
that needs a new runtime class or a new pipeline (i.e. NOT data-only), `test_every_source_is_data_only`
|
||||
fails — surfacing the exact moment the architecture stopped being general. That is the whole point of
|
||||
Phase Ω: measure where it breaks under real domain knowledge.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
def _ledger():
|
||||
p = os.path.join(os.path.dirname(__file__), "..", "knowledge", "architecture_stability", "integration_ledger.yaml")
|
||||
return yaml.safe_load(open(p, encoding="utf-8"))
|
||||
|
||||
|
||||
def _run():
|
||||
root = os.path.join(os.path.dirname(__file__), "..")
|
||||
r = subprocess.run(
|
||||
[sys.executable, "reference_scenarios/architecture_stability_kpi.py"],
|
||||
cwd=root, env={**os.environ, "PYTHONPATH": "."}, capture_output=True, text=True,
|
||||
)
|
||||
assert r.returncode == 0, r.stderr
|
||||
return r.stdout
|
||||
|
||||
|
||||
def test_every_source_is_data_only_and_zero_runtime():
|
||||
# GUARDRAIL: every integrated source must cost 0 runtime classes, no new pipeline, data-only.
|
||||
# If a future domain breaks this, fix the architecture or record the break honestly — do not
|
||||
# weaken this assertion to make it pass.
|
||||
for s in _ledger()["sources"]:
|
||||
assert s["new_runtime_classes"] == 0, s["source"]
|
||||
assert s["new_pipeline"] is False, s["source"]
|
||||
assert s["integration_kind"] == "data_only", s["source"]
|
||||
|
||||
|
||||
def test_kpis_reported_at_full_stability():
|
||||
out = _run()
|
||||
n = len(_ledger()["sources"])
|
||||
assert "Architecture Stability: %d/%d = 100%%" % (n, n) in out
|
||||
assert "Knowledge Velocity: %d/%d = 100%%" % (n, n) in out
|
||||
|
||||
|
||||
def test_pipeline_functions_are_one_time_infrastructure():
|
||||
out = _run()
|
||||
assert "EINMALIG (jetzt eingefroren)" in out
|
||||
assert "journey_matcher" in out and "letzte architektonische Baustein" in out
|
||||
|
||||
|
||||
def test_three_knowledge_layers_present():
|
||||
out = _run()
|
||||
for layer in ["Beschreibend", "Transformation", "Produktion"]:
|
||||
assert layer in out
|
||||
|
||||
|
||||
def test_three_success_questions_present():
|
||||
out = _run()
|
||||
assert "Musste für eine neue Domäne Runtime-Code geändert werden?" in out
|
||||
assert "Knowledge Velocity" in out
|
||||
assert "Architecture Stability" in out
|
||||
Reference in New Issue
Block a user