feat(decomposition): add merge pass, enrichment, and Pass 0b refinements
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 51s
CI/CD / test-python-backend-compliance (push) Successful in 34s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 20s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Has been skipped

Add obligation refinement pipeline between Pass 0a and 0b:
- Merge pass: rule-based dedup of implementation-level duplicate obligations
  within the same parent control (Jaccard similarity on action+object)
- Enrich pass: classify trigger_type (event/periodic/continuous) and detect
  is_implementation_specific from obligation text (regex-based, no LLM)
- Pass 0b: skip merged obligations, cap severity for impl-specific, override
  category to 'testing' for test obligations
- Migration 075: merged_into_id, trigger_type, is_implementation_specific
- Two new API endpoints: merge-obligations, enrich-obligations
- 30+ new tests (122 total, all passing)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-03-21 22:27:09 +01:00
parent 71b8c33270
commit a14e2f3a00
4 changed files with 804 additions and 12 deletions

View File

@@ -13,6 +13,8 @@ Endpoints:
GET /v1/canonical/crosswalk/stats — Coverage statistics
POST /v1/canonical/migrate/decompose — Pass 0a: Obligation extraction
POST /v1/canonical/migrate/merge-obligations — Merge implementation-level dupes
POST /v1/canonical/migrate/enrich-obligations — Add trigger_type, impl metadata
POST /v1/canonical/migrate/compose-atomic — Pass 0b: Atomic control composition
POST /v1/canonical/migrate/link-obligations — Pass 1: Obligation linkage
POST /v1/canonical/migrate/classify-patterns — Pass 2: Pattern classification
@@ -157,6 +159,9 @@ class DecompositionStatusResponse(BaseModel):
rejected: int = 0
composed: int = 0
atomic_controls: int = 0
merged: int = 0
enriched: int = 0
ready_for_pass0b: int = 0
decomposition_pct: float = 0.0
composition_pct: float = 0.0
@@ -488,6 +493,50 @@ async def migrate_decompose(req: MigrationRequest):
db.close()
@router.post("/migrate/merge-obligations", response_model=MigrationResponse)
async def migrate_merge_obligations():
"""Merge implementation-level duplicate obligations within each parent.
Run AFTER Pass 0a, BEFORE Pass 0b. No LLM calls — rule-based.
Merges obligations that share similar action+object into the more
abstract survivor, marking the concrete duplicate as 'merged'.
"""
from compliance.services.decomposition_pass import DecompositionPass
db = SessionLocal()
try:
decomp = DecompositionPass(db=db)
stats = decomp.run_merge_pass()
return MigrationResponse(status="completed", stats=stats)
except Exception as e:
logger.error("Merge pass failed: %s", e)
raise HTTPException(status_code=500, detail=str(e))
finally:
db.close()
@router.post("/migrate/enrich-obligations", response_model=MigrationResponse)
async def migrate_enrich_obligations():
"""Add trigger_type and is_implementation_specific metadata.
Run AFTER merge pass, BEFORE Pass 0b. No LLM calls — rule-based.
Classifies trigger_type (event/periodic/continuous) from obligation text
and detects implementation-specific obligations (concrete tools/protocols).
"""
from compliance.services.decomposition_pass import DecompositionPass
db = SessionLocal()
try:
decomp = DecompositionPass(db=db)
stats = decomp.enrich_obligations()
return MigrationResponse(status="completed", stats=stats)
except Exception as e:
logger.error("Enrich pass failed: %s", e)
raise HTTPException(status_code=500, detail=str(e))
finally:
db.close()
@router.post("/migrate/compose-atomic", response_model=MigrationResponse)
async def migrate_compose_atomic(req: MigrationRequest):
"""Pass 0b: Compose atomic controls from obligation candidates.