feat: Control Library UI, dedup migration, QA tooling, docs
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 31s
CI/CD / test-python-backend-compliance (push) Successful in 1m35s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Has been skipped
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 31s
CI/CD / test-python-backend-compliance (push) Successful in 1m35s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Has been skipped
- Control Library: parent control display, ObligationTypeBadge, GenerationStrategyBadge variants, evidence string fallback - API: expose parent_control_uuid/id/title in canonical controls - Fix: DSFA SQLAlchemy 2.0 Row._mapping compatibility - Migration 074: control_parent_links + control_dedup_reviews tables - QA scripts: benchmark, gap analysis, OSCAL import, OWASP cleanup, phase5 normalize, phase74 gap fill, sync_db, run_job - Docs: dedup engine, RAG benchmark, lessons learned, pipeline docs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -567,7 +567,86 @@ curl -X POST https://api-dev.breakpilot.ai/api/compliance/v1/canonical/generate/
|
||||
|
||||
---
|
||||
|
||||
## Pass 0a/0b: Atomare Control-Zerlegung
|
||||
|
||||
Die Pipeline v3 erweitert die 7-Stufen-Pipeline um einen Vor-Pass, der Rich Controls in atomare Controls zerlegt.
|
||||
|
||||
### Pass 0a: Obligation Extraction
|
||||
|
||||
Extrahiert individuelle normative Pflichten aus Rich Controls via LLM.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Rich Control] -->|LLM| B[Obligations]
|
||||
B --> C{Quality Gate}
|
||||
C -->|Pass| D[validated]
|
||||
C -->|Fail| E[rejected]
|
||||
```
|
||||
|
||||
**3-Tier Klassifikation:**
|
||||
|
||||
| Typ | Erkennungsmuster | Beispiel |
|
||||
|-----|-----------------|---------|
|
||||
| **Pflicht** | muss, ist verpflichtet, hat sicherzustellen | "Der Verantwortliche MUSS ein Verzeichnis fuehren" |
|
||||
| **Empfehlung** | soll, sollte, wird empfohlen | "Es SOLLTE eine Risikobewertung durchgefuehrt werden" |
|
||||
| **Kann** | kann, darf, ist berechtigt | "Die Aufsichtsbehoerde KANN Geldbussen verhaengen" |
|
||||
|
||||
**Quality Gate (6 Regeln):**
|
||||
|
||||
1. Nur normative Aussagen (muss, sicherzustellen, verpflichtet)
|
||||
2. Ein Hauptverb pro Obligation
|
||||
3. Test-Obligations separat von operativen
|
||||
4. Reporting-Obligations separat
|
||||
5. Nicht auf Evidence-Ebene splitten
|
||||
6. Parent-Link immer erhalten
|
||||
|
||||
### Pass 0b: Atomic Control Composition
|
||||
|
||||
Verwandelt jede validierte Obligation in ein eigenstaendiges atomares Control.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Obligation] -->|LLM| B[Atomic Control]
|
||||
B -->|Dedup Check| C{4-Stage Dedup}
|
||||
C -->|NEW| D[Insert + Index]
|
||||
C -->|LINK| E[Parent-Link]
|
||||
C -->|REVIEW| F[Review-Queue]
|
||||
```
|
||||
|
||||
**Konfiguration:**
|
||||
|
||||
| Variable | Default | Beschreibung |
|
||||
|----------|---------|-------------|
|
||||
| `DECOMPOSITION_LLM_MODEL` | `claude-sonnet-4-6` | LLM fuer Pass 0a/0b |
|
||||
| `DECOMPOSITION_BATCH_SIZE` | `5` | Obligations pro LLM-Call |
|
||||
| `DECOMPOSITION_LLM_TIMEOUT` | `120` | Timeout in Sekunden |
|
||||
|
||||
**Ergebnisse (Stand 2026-03-21):**
|
||||
|
||||
| Metrik | Wert |
|
||||
|--------|------|
|
||||
| Rich Controls (technisch) | ~6.800 |
|
||||
| Atomare Controls (bisher) | 30 (PoC: 10x CRYP, AUTH, SEC) |
|
||||
| Ziel nach Full Run | ~18.000 unique Master Controls |
|
||||
| Obligations pro Rich Control | ~10 |
|
||||
| Dedup-Reduktion erwartet | ~70% |
|
||||
|
||||
### Quelldateien (Pass 0a/0b)
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|-------------|
|
||||
| `compliance/services/decomposition_pass.py` | Pass 0a + 0b Logik |
|
||||
| `compliance/services/control_dedup.py` | 4-Stufen Dedup-Engine |
|
||||
| `migrations/061_obligation_candidates.sql` | Obligation-Tabelle |
|
||||
| `migrations/074_control_dedup.sql` | Dedup-Tabellen (Parent-Links, Review-Queue) |
|
||||
| `tests/test_decomposition_pass.py` | 90 Tests |
|
||||
| `tests/test_control_dedup.py` | 56 Tests |
|
||||
|
||||
---
|
||||
|
||||
## Verwandte Dokumentation
|
||||
|
||||
- [Canonical Control Library (CP-CLIB)](canonical-control-library.md) — Domains, Datenmodell, Too-Close-Detektor, CI/CD Validation
|
||||
- [Deduplizierungs-Engine](dedup-engine.md) — 4-Stufen Dedup, Multi-Parent-Linking, Review-Queue
|
||||
- [RAG Pipeline Benchmark](../../development/rag-pipeline-benchmark.md) — State-of-the-Art Vergleich, Optimierungsempfehlungen
|
||||
- [Multi-Layer Control Architecture](canonical-control-library.md#multi-layer-control-architecture) — 10-Stage Pipeline-Erweiterung mit Obligations, Patterns, Crosswalk
|
||||
|
||||
Reference in New Issue
Block a user