Adds scoped mypy disable-error-code headers to all 15 agent-created
service files covering the ORM Column[T] + raw-SQL result type issues.
Updates mypy.ini to flip 14 personally-refactored route files to strict;
defers 4 agent-refactored routes (dsr, vendor, notfallplan, isms) until
return type annotations are added.
mypy compliance/ -> Success: no issues found in 162 source files
173/173 pytest pass
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous commit (32e121f) left isms_assessment_service.py at 639 LOC,
exceeding the 500-line hard cap. This follow-up extracts ReadinessCheckService
and OverviewService into a new isms_readiness_service.py (400 LOC), leaving
isms_assessment_service.py at 257 LOC (Management Reviews, Internal Audits,
Audit Trail only).
Updated isms_routes.py imports to reference the new service file.
File sizes after split:
- isms_routes.py: 446 LOC (thin handlers)
- isms_governance_service.py: 416 LOC (scope, context, policy, objectives, SoA)
- isms_findings_service.py: 276 LOC (findings, CAPA)
- isms_assessment_service.py: 257 LOC (mgmt reviews, internal audits, audit trail)
- isms_readiness_service.py: 400 LOC (readiness check, ISO 27001 overview)
All 58 integration tests + 173 unit/contract tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/isms_routes.py (1676 LOC) -> 445 LOC thin routes +
three service files:
- isms_governance_service.py (416) — scope, context, policy, objectives, SoA
- isms_findings_service.py (276) — findings, CAPA, audit trail
- isms_assessment_service.py (639) — management reviews, internal audits,
readiness checks, ISO 27001 overview
NOTE: isms_assessment_service.py exceeds the 500-line hard cap at 639 LOC.
This needs a follow-up split (management_review_service vs
internal_audit_service). Flagged for next session.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split vendor_compliance_routes.py (1107 LOC) into thin route handlers
plus three service modules: VendorService (vendors CRUD/stats/status),
ContractService (contracts CRUD), and FindingService + ControlInstanceService
+ ControlsLibraryService (findings, control instances, controls library).
All files under 500 lines. 215 tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add [mypy-compliance.api.routes] to mypy.ini strict scope
- Fix bare `dict` type annotation in routes.py update_requirement handler
- Fix Column[str] return type in control_export_service.download_file
- Fix unused type:ignore in legal_document_service.upload_word
- Add union-attr ignore for optional requirement null access in routes.py
mypy compliance/ -> Success on 149 source files
173/173 pytest pass
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract consent, audit log, cookie category, and consent stats endpoints
from legal_document_routes into LegalDocumentConsentService. The route
file is now a thin handler layer delegating to LegalDocumentService and
LegalDocumentConsentService with translate_domain_errors(). Legacy
helpers (_doc_to_response, _version_to_response, _transition,
_log_approval) and schemas are re-exported for existing tests. Two
transition tests updated to expect domain errors instead of HTTPException.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/evidence_routes.py (641 LOC) -> 240 LOC thin routes + 460-line
EvidenceService. Manages evidence CRUD, file upload, CI/CD evidence
collection (SAST/dependency/SBOM/container scans), and CI status dashboard.
Service injection pattern: EvidenceService takes the EvidenceRepository,
ControlRepository, and AutoRiskUpdater classes as constructor parameters.
The route's get_evidence_service factory reads these class references from
its own module namespace so tests that
``patch("compliance.api.evidence_routes.EvidenceRepository", ...)`` still
take effect through the factory.
The `_store_evidence` and `_update_risks` helpers stay as module-level
callables in evidence_service and are re-exported from the route module.
The collect_ci_evidence handler remains inline (not delegated to a service
method) so tests can patch
`compliance.api.evidence_routes._store_evidence` and have the patch take
effect at the handler's call site.
Legacy re-exports via __all__: SOURCE_CONTROL_MAP, EvidenceRepository,
ControlRepository, AutoRiskUpdater, _parse_ci_evidence,
_extract_findings_detail, _store_evidence, _update_risks.
Verified:
- 208/208 pytest (core + 35 evidence tests) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 135 source files
- evidence_routes.py 641 -> 240 LOC
- Hard-cap violations: 10 -> 9
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/screening_routes.py (597 LOC) -> 233 LOC thin routes +
353-line ScreeningService + 60-line schemas file. Manages SBOM generation
(CycloneDX 1.5) and OSV.dev vulnerability scanning.
Pure helpers (parse_package_lock, parse_requirements_txt, parse_yarn_lock,
detect_and_parse, generate_sbom, query_osv, map_osv_severity,
extract_fix_version, scan_vulnerabilities) moved to the service module.
The two lookup endpoints (get_screening, list_screenings) delegate to
the new ScreeningService class.
Test-mock compatibility: tests/test_screening_routes.py uses
`patch("compliance.api.screening_routes.SessionLocal", ...)` and
`patch("compliance.api.screening_routes.scan_vulnerabilities", ...)`.
Both names are re-imported and re-exported from the route module so the
patches still take effect. The scan handler keeps direct
`SessionLocal()` usage; the lookup handlers also use SessionLocal so the
test mocks intercept them.
Latent bug fixed: the original scan handler had
text = content.decode("utf-8")
on line 339, shadowing the imported `sqlalchemy.text` so that the
subsequent `text("INSERT ...")` calls would have raised at runtime.
The variable is now named `file_text`. Allowed under "minor behavior
fixes" — the bug was unreachable in tests because they always patched
SessionLocal.
Verified:
- 240/240 pytest pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 134 source files
- screening_routes.py 597 -> 233 LOC
- Hard-cap violations: 11 -> 10
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/canonical_control_routes.py (514 LOC) -> 192 LOC thin
routes + 316-line CanonicalControlService + 105-line schemas file.
Canonical Control Library manages OWASP/NIST/ENISA-anchored security
control frameworks and controls. Like company_profile_routes, this file
uses raw SQL via sqlalchemy.text() because there are no SQLAlchemy
models for canonical_control_frameworks or canonical_controls.
Single-service split. Session management moved from bespoke
`with SessionLocal() as db:` blocks to Depends(get_db) for consistency.
Legacy test imports preserved via re-export (FrameworkResponse,
ControlResponse, SimilarityCheckRequest, SimilarityCheckResponse,
_control_row).
Validation extracted to a module-level `_validate_control_input` helper
so both create and update share the same checks. ValidationError (from
compliance.domain) replaces raw HTTPException(400) raises.
Verified:
- 187/187 pytest (173 core + 14 canonical) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 130 source files
- canonical_control_routes.py 514 -> 192 LOC
- Hard-cap violations: 13 -> 12
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/vvt_routes.py (550 LOC) -> 225 LOC thin routes + 475-line
VVTService. Covers the organization header, processing activities CRUD,
audit log, JSON/CSV export, stats, and version lookups for the Art. 30
DSGVO Verzeichnis.
Single-service split: organization + activities + audit + stats all
revolve around the same tenant's VVT document, and the existing test
suite (tests/test_vvt_routes.py — 768 LOC, tests/test_vvt_tenant_isolation.py
— 205 LOC) exercises them together.
Module-level helpers (_activity_to_response, _log_audit, _export_csv)
stay module-level in compliance.services.vvt_service and are re-exported
from compliance.api.vvt_routes so the two test files keep importing
from the old path.
Pydantic schemas already live in compliance.schemas.vvt from Step 3 —
no new schema file needed this round.
mypy.ini flips compliance.api.vvt_routes from ignore_errors=True to
False. Two SQLAlchemy Column[str] vs str dict-index errors fixed with
explicit str() casts on status/business_function in the stats loop.
Verified:
- 242/242 pytest (173 core + 69 VVT integration) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 128 source files
- vvt_routes.py 550 -> 225 LOC
- vvt_service.py 475 LOC (under 500 hard cap)
- Hard-cap violations: 14 -> 13
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/company_profile_routes.py (640 LOC) -> 154 LOC thin routes.
Unusual for this repo: persistence uses raw SQL via sqlalchemy.text()
because the underlying compliance_company_profiles table has ~45 columns
with complex jsonb coercion and there is no SQLAlchemy model for it.
New files:
compliance/schemas/company_profile.py (127) — 4 request/response models
compliance/services/company_profile_service.py (340) — Service class + row_to_response + log_audit
compliance/services/_company_profile_sql.py (139) — 70-line INSERT/UPDATE statements
separated for readability
Minor behavioral improvement: the handlers now use Depends(get_db) for
session management instead of the bespoke `db = SessionLocal(); try: ...
finally: db.close()` pattern. This makes the routes consistent with
every other refactored service, fixes the broken-ness under test
dependency_overrides, and removes 6 duplicate try/finally blocks.
Legacy exports preserved: CompanyProfileRequest, CompanyProfileResponse,
AuditEntryResponse, AuditListResponse, row_to_response, and log_audit are
re-exported from compliance.api.company_profile_routes so that the two
existing test files
(tests/test_company_profile_routes.py, tests/test_company_profile_extend.py)
keep importing from the same path.
Pre-existing broken tests noted: 6 tests in those files feed a 40-tuple
row into row_to_response, but _BASE_COLUMNS_LIST has 46 columns (has had
since the Phase 2 Stammdaten extension). These tests fail on main too
(verified via `git stash` round-trip). Not fixed in this commit — they
require a rewrite of the test's _make_row helper, which is out of scope
for a pure structural refactor. Flagged for follow-up.
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 127 source files
- company_profile_routes.py 640 -> 154 LOC
- All new files under soft 300 target except service (340, under hard 500)
- Hard-cap violations: 15 -> 14
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/tom_routes.py (609 LOC) -> 215 LOC thin routes +
434-line TOMService. Request bodies (TOMStateBody, TOMMeasureCreate,
TOMMeasureUpdate, TOMMeasureBulkItem, TOMMeasureBulkBody) moved to
compliance/schemas/tom.py (joining the existing response models from
the Step 3 split).
Single-service split (not two like banner): state, measures CRUD + bulk
upsert, stats, export, and version lookups are all tightly coupled
around the TOMMeasureDB aggregate, so splitting would create artificial
boundaries. TOMService is 434 LOC — comfortably under the 500 hard cap.
Domain error mapping:
- ConflictError -> 409 (version conflict on state save; duplicate control_id on create)
- NotFoundError -> 404 (missing measure on update; missing version)
- ValidationError -> 400 (missing tenant_id on DELETE /state)
Legacy test compat: the existing tests/test_tom_routes.py imports
TOMMeasureBulkItem, _parse_dt, _measure_to_dict, and DEFAULT_TENANT_ID
directly from compliance.api.tom_routes. All re-exported via __all__ so
the 44-test file runs unchanged.
mypy.ini flips compliance.api.tom_routes from ignore_errors=True to
False. TOMService carries the scoped Column[T] header.
Verified:
- 217/217 pytest (173 baseline + 44 TOM) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 124 source files
- tom_routes.py 609 -> 215 LOC
- Hard-cap violations: 16 -> 15
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 4, file 2 of 18. Same cookbook as audit_routes (4a91814 +
883ef70) applied to banner_routes.py.
compliance/api/banner_routes.py (653 LOC) is decomposed into:
compliance/api/banner_routes.py (255) — thin handlers
compliance/services/banner_consent_service.py (298) — public SDK surface
compliance/services/banner_admin_service.py (238) — site/category/vendor CRUD
compliance/services/_banner_serializers.py ( 81) — ORM-to-dict helpers
shared between the
two services
compliance/schemas/banner.py ( 85) — Pydantic request models
Split rationale: the SDK-facing endpoints (consent CRUD, config
retrieval, export, stats) and the admin CRUD endpoints (sites +
categories + vendors) have distinct audiences and different auth stories,
and combined they would push the service file over the 500 hard cap.
Two focused services is cleaner than one ~540-line god class.
The shared ORM-to-dict helpers live in a private sibling module
(_banner_serializers) rather than a static method on either service, so
both services can import without a cycle.
Handlers follow the established pattern:
- Depends(get_consent_service) or Depends(get_admin_service)
- `with translate_domain_errors():` wrapping the service call
- Explicit return type annotations
- ~3-5 lines per handler
Services raise NotFoundError / ConflictError / ValidationError from
compliance.domain; no HTTPException in the service layer.
mypy.ini flips compliance.api.banner_routes from ignore_errors=True to
False, joining audit_routes in the strict scope. The services carry the
same scoped `# mypy: disable-error-code="arg-type,assignment"` header
used by the audit services for the ORM Column[T] issue.
Pydantic schemas moved to compliance.schemas.banner (mirroring the Step 3
schemas split). They were previously defined inline in banner_routes.py
and not referenced by anything outside it, so no backwards-compat shim
is needed.
Verified:
- 224/224 pytest (173 baseline + 26 audit integration + 25 banner
integration) pass
- tests/contracts/test_openapi_baseline.py green (360/484 unchanged)
- mypy compliance/ -> Success: no issues found in 123 source files
- All new files under the 300 soft target (largest: 298)
- banner_routes.py drops from 653 -> 255 LOC (below hard cap)
Hard-cap violations remaining: 16 (was 17).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 4 follow-up addressing the debt flagged in the worked-example
commit (4a91814).
## mypy --strict policy
Adds backend-compliance/mypy.ini declaring the strict-mode scope:
Fully strict (enforced today):
- compliance/domain/
- compliance/schemas/
- compliance/api/_http_errors.py
- compliance/api/audit_routes.py (refactored in Step 4)
- compliance/services/audit_session_service.py
- compliance/services/audit_signoff_service.py
Loose (ignore_errors=True) with a migration path:
- compliance/db/* — SQLAlchemy 1.x Column[] vs
runtime T; unblocks Phase 1
until a Mapped[T] migration.
- compliance/api/<route>.py — each route file flips to
strict as its own Step 4
refactor lands.
- compliance/services/<legacy util> — 14 utility services
(llm_provider, pdf_extractor,
seeder, ...) that predate the
clean-arch refactor.
- compliance/tests/ — excluded (legacy placeholder
style). The new TestClient-
based integration suite is
type-annotated.
The two new service files carry a scoped `# mypy: disable-error-code="arg-type,assignment"`
header for the ORM Column[T] issue — same underlying SQLAlchemy limitation,
narrowly scoped rather than wholesale ignore_errors.
Flow: `cd backend-compliance && mypy compliance/` -> clean on 119 files.
CI yaml updated to use the config instead of ad-hoc package lists.
## Bugs fixed while enabling strict
mypy --strict surfaced two latent bugs in the pre-refactor code. Both
were invisible because the old `compliance/tests/test_audit_routes.py`
is a placeholder suite that asserts on request-data shape and never
calls the handlers:
- AuditSessionResponse.updated_at is a required field in the schema,
but the original handler didn't pass it. Fixed in
AuditSessionService._to_response.
- PaginationMeta requires has_next + has_prev. The original audit
checklist handler didn't compute them. Fixed in
AuditSignOffService.get_checklist.
Both are behavior-preserving at the HTTP level because the old code
would have raised Pydantic ValidationError at response serialization
had the endpoint actually been exercised.
## Integration test suite
Adds backend-compliance/tests/test_audit_routes_integration.py — 26
real TestClient tests against an in-memory sqlite backend (StaticPool).
Replaces the coverage gap left by the placeholder suite.
Covers:
- Session CRUD + lifecycle transitions (draft -> in_progress -> completed
-> archived), including the 409 paths for illegal transitions
- Checklist pagination, filtering, search
- Sign-off create / update / auto-start-session / count-flipping
- Sign-off 400 (invalid result), 404 (missing requirement), 409 (completed session)
- Get-signoff 404 / 200 round-trip
Uses a module-scoped schema fixture + per-test DELETE-sweep so the
suite runs in ~2.3s despite the ~50-table ORM surface.
Verified:
- 199/199 pytest (173 original + 26 new audit integration) pass
- tests/contracts/test_openapi_baseline.py green, OpenAPI 360/484 unchanged
- mypy compliance/ -> Success: no issues found in 119 source files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 4 of PHASE1_RUNBOOK.md, first worked example. Demonstrates
the router -> service delegation pattern for all 18 oversized route
files still above the 500 LOC hard cap.
compliance/api/audit_routes.py (637 LOC) is decomposed into:
compliance/api/audit_routes.py (198) — thin handlers
compliance/services/audit_session_service.py (259) — session lifecycle
compliance/services/audit_signoff_service.py (319) — checklist + sign-off
compliance/api/_http_errors.py ( 43) — reusable error translator
Handlers shrink to 3-6 lines each:
@router.post("/sessions", response_model=AuditSessionResponse)
async def create_audit_session(
request: CreateAuditSessionRequest,
service: AuditSessionService = Depends(get_audit_session_service),
):
with translate_domain_errors():
return service.create(request)
Services are HTTP-agnostic: they raise NotFoundError / ConflictError /
ValidationError from compliance.domain, and the route layer translates
those to HTTPException(404/409/400) via the translate_domain_errors()
context manager in compliance.api._http_errors. The error translator is
reusable by every future Step 4 refactor.
Services take a sqlalchemy Session in the constructor and are wired via
Depends factories (get_audit_session_service / get_audit_signoff_service).
No globals, no module-level state.
Behavior is byte-identical at the HTTP boundary:
- Same paths, methods, status codes, response models
- Same error messages (domain error __str__ preserved)
- Same auto-start-on-first-signoff, same statistics calculation,
same signature hash format, same PDF streaming response
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360 paths / 484 operations unchanged
- audit_routes.py under soft 300 target
- Both new service files under soft 300 / hard 500
Note: compliance/tests/test_audit_routes.py contains placeholder tests
that do not actually import or call the handler functions — they only
assert on request-data shape. Real behavioral coverage relies on the
contract test. A follow-up commit should add TestClient-based
integration tests for the audit endpoints. Flagged in PHASE1_RUNBOOK.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 5 of PHASE1_RUNBOOK.md.
compliance/db/repository.py (1547 LOC) decomposed into seven sibling
per-aggregate repository modules:
regulation_repository.py (268) — Regulation + Requirement
control_repository.py (291) — Control + ControlMapping
evidence_repository.py (143)
risk_repository.py (148)
audit_export_repository.py (110)
service_module_repository.py (247)
audit_session_repository.py (478) — AuditSession + AuditSignOff
compliance/db/isms_repository.py (838 LOC) decomposed into two
sub-aggregate modules mirroring the models split:
isms_governance_repository.py (354) — Scope, Policy, Objective, SoA
isms_audit_repository.py (499) — Finding, CAPA, Review, Internal Audit,
Trail, Readiness
Both original files become thin re-export shims (37 and 25 LOC
respectively) so every existing import continues to work unchanged.
New code SHOULD import from the aggregate module directly.
All new sibling files under the 500-line hard cap; largest is
isms_audit_repository.py at 499 (on the edge; when Phase 1 Step 4
router->service extraction lands, the audit_session repo may split
further if growth exceeds 500).
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360 paths / 484 operations unchanged
- All repo files under 500 LOC
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 3 of PHASE1_RUNBOOK.md. compliance/api/schemas.py is
decomposed into 16 per-domain Pydantic schema modules under
compliance/schemas/:
common.py ( 79) — 6 API enums + PaginationMeta
regulation.py ( 52)
requirement.py ( 80)
control.py (119) — Control + Mapping
evidence.py ( 66)
risk.py ( 79)
ai_system.py ( 63)
dashboard.py (195) — Dashboard, Export, Executive Dashboard
service_module.py (121)
bsi.py ( 58) — BSI + PDF extraction
audit_session.py (172)
report.py ( 53)
isms_governance.py (343) — Scope, Context, Policy, Objective, SoA
isms_audit.py (431) — Finding, CAPA, Review, Internal Audit, Readiness, Trail, ISO27001
vvt.py (168)
tom.py ( 71)
compliance/api/schemas.py becomes a 39-line re-export shim so existing
imports (from compliance.api.schemas import RegulationResponse) keep
working unchanged. New code should import from the domain module
directly (from compliance.schemas.regulation import RegulationResponse).
Deferred-from-sweep: all 28 class Config blocks in the original file
were converted to model_config = ConfigDict(...) during the split.
schemas.py-sourced PydanticDeprecatedSince20 warnings are now gone.
Cross-domain references handled via targeted imports (e.g. dashboard.py
imports EvidenceResponse from evidence, RiskResponse from risk). common
API enums + PaginationMeta are imported by every domain module.
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360 paths / 484 operations unchanged (contract test green)
- All new files under the 500-line hard cap (largest: isms_audit.py
at 431, isms_governance.py at 343, dashboard.py at 195)
- No file in compliance/schemas/ or compliance/api/schemas.py
exceeds the hard cap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Squash of branch refactor/phase0-guardrails-and-models-split — 4 commits,
81 files, 173/173 pytest green, OpenAPI contract preserved (360 paths /
484 operations).
## Phase 0 — Architecture guardrails
Three defense-in-depth layers to keep the architecture rules enforced
regardless of who opens Claude Code in this repo:
1. .claude/settings.json PreToolUse hook on Write/Edit blocks any file
that would exceed the 500-line hard cap. Auto-loads in every Claude
session in this repo.
2. scripts/githooks/pre-commit (install via scripts/install-hooks.sh)
enforces the LOC cap locally, freezes migrations/ without
[migration-approved], and protects guardrail files without
[guardrail-change].
3. .gitea/workflows/ci.yaml gains loc-budget + guardrail-integrity +
sbom-scan (syft+grype) jobs, adds mypy --strict for the new Python
packages (compliance/{services,repositories,domain,schemas}), and
tsc --noEmit for admin-compliance + developer-portal.
Per-language conventions documented in AGENTS.python.md, AGENTS.go.md,
AGENTS.typescript.md at the repo root — layering, tooling, and explicit
"what you may NOT do" lists. Root CLAUDE.md is prepended with the six
non-negotiable rules. Each of the 10 services gets a README.md.
scripts/check-loc.sh enforces soft 300 / hard 500 and surfaces the
current baseline of 205 hard + 161 soft violations so Phases 1-4 can
drain it incrementally. CI gates only CHANGED files in PRs so the
legacy baseline does not block unrelated work.
## Deprecation sweep
47 files. Pydantic V1 regex= -> pattern= (2 sites), class Config ->
ConfigDict in source_policy_router.py (schemas.py intentionally skipped;
it is the Phase 1 Step 3 split target). datetime.utcnow() ->
datetime.now(timezone.utc) everywhere including SQLAlchemy default=
callables. All DB columns already declare timezone=True, so this is a
latent-bug fix at the Python side, not a schema change.
DeprecationWarning count dropped from 158 to 35.
## Phase 1 Step 1 — Contract test harness
tests/contracts/test_openapi_baseline.py diffs the live FastAPI /openapi.json
against tests/contracts/openapi.baseline.json on every test run. Fails on
removed paths, removed status codes, or new required request body fields.
Regenerate only via tests/contracts/regenerate_baseline.py after a
consumer-updated contract change. This is the safety harness for all
subsequent refactor commits.
## Phase 1 Step 2 — models.py split (1466 -> 85 LOC shim)
compliance/db/models.py is decomposed into seven sibling aggregate modules
following the existing repo pattern (dsr_models.py, vvt_models.py, ...):
regulation_models.py (134) — Regulation, Requirement
control_models.py (279) — Control, Mapping, Evidence, Risk
ai_system_models.py (141) — AISystem, AuditExport
service_module_models.py (176) — ServiceModule, ModuleRegulation, ModuleRisk
audit_session_models.py (177) — AuditSession, AuditSignOff
isms_governance_models.py (323) — ISMSScope, Context, Policy, Objective, SoA
isms_audit_models.py (468) — Finding, CAPA, MgmtReview, InternalAudit,
AuditTrail, Readiness
models.py becomes an 85-line re-export shim in dependency order so
existing imports continue to work unchanged. Schema is byte-identical:
__tablename__, column definitions, relationship strings, back_populates,
cascade directives all preserved.
All new sibling files are under the 500-line hard cap; largest is
isms_audit_models.py at 468. No file in compliance/db/ now exceeds
the hard cap.
## Phase 1 Step 3 — infrastructure only
backend-compliance/compliance/{schemas,domain,repositories}/ packages
are created as landing zones with docstrings. compliance/domain/
exports DomainError / NotFoundError / ConflictError / ValidationError /
PermissionError — the base classes services will use to raise
domain-level errors instead of HTTPException.
PHASE1_RUNBOOK.md at backend-compliance/PHASE1_RUNBOOK.md documents
the nine-step execution plan for Phase 1: snapshot baseline,
characterization tests, split models.py (this commit), split schemas.py
(next), extract services, extract repositories, mypy --strict, coverage.
## Verification
backend-compliance/.venv-phase1: uv python install 3.12 + pip -r requirements.txt
PYTHONPATH=. pytest compliance/tests/ tests/contracts/
-> 173 passed, 0 failed, 35 warnings, OpenAPI 360/484 unchanged
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- source_article/source_regulation VARCHAR(100) → TEXT for long NIST refs
- Pass 0b NOT EXISTS queries now skip deprecated/duplicate controls
- Duplicate Guard excludes deprecated/duplicate from existence check
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The NOT EXISTS check and Duplicate Guard now exclude deprecated and
duplicate controls, enabling clean re-runs after invalidation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Truncate object keys to 40 chars (was 80) at underscore boundary
- Strip German qualifying prepositional phrases (bei/für/gemäß/von/zur/...)
- Add 65 new synonym mappings for near-duplicate patterns found in analysis
- Strip trailing noise tokens (articles/prepositions)
- Add _truncate_at_boundary() helper and _QUALIFYING_PHRASE_RE regex
- 11 new tests for normalization improvements (227 total pass)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Duplicate Guard: merge_hint-Lookup vor INSERT in _write_atomic_control()
verhindert semantisch identische Controls unter demselben Parent.
2. Severity-Kalibrierung: action_type-basiert statt blind vom Parent.
define/review/test → max medium, implement/monitor → max high.
3. Title-Truncation: Schnitt am Wortende statt mitten im Wort.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Neue Endpunkte POST /obligations/dedup und GET /obligations/dedup-stats.
Pro candidate_id wird der aelteste Eintrag behalten, alle weiteren erhalten
release_state='duplicate' mit merged_into_id + quality_flags fuer Traceability.
Detail-View filtert Duplikate aus. MKDocs aktualisiert.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend: Facets zaehlen jetzt Controls OHNE Wert (z.B. "Ohne Nachweis")
als __none__. Filter unterstuetzen __none__ fuer verification_method,
category, evidence_type. Counts addieren sich immer zum Total.
Frontend: "Ohne X" Optionen in Dropdowns. AbortController verhindert
dass aeltere API-Antworten neuere ueberschreiben.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend: controls-meta akzeptiert alle Filter-Parameter und berechnet
Faceted Counts (jede Dimension zaehlt mit allen ANDEREN Filtern).
Neue Facets: severity, verification_method, category, evidence_type,
release_state — zusaetzlich zu domains, sources, type_counts.
Frontend: loadMeta laedt bei jeder Filteraenderung neu, alle Dropdowns
zeigen kontextsensitive Zahlen. Proxy leitet Filter an controls-meta weiter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend: control_type=eigenentwicklung in list_controls + count_controls,
type_counts (rich/atomic/eigenentwicklung) in controls-meta Endpoint.
Frontend: Typ-Dropdown zeigt Eigenentwicklung mit Anzahl.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Die atomic_controls_dedup Collection (51k Punkte) enthaelt nur atomare
Controls ohne source_citation. Jetzt wird der Parent-Control aufgeloest,
der die Rechtsgrundlage traegt. Deduplizierung nach Parent-UUID verhindert
mehrfache Eintraege fuer die gleiche Regulation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verhindert 'invalid transaction' Fehler wenn ein LLM-Call fehlschlaegt
und nachfolgende DB-Operationen blockiert.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
qwen3.5 gibt Antworten im 'thinking'-Feld statt 'response' zurueck.
Mit think:false wird der Thinking-Mode deaktiviert und die Antwort
korrekt im response-Feld geliefert.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ollama als eigener Enum-Wert neben self_hosted, damit die
docker-compose-Konfiguration (ollama) korrekt aufgeloest wird.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
POST /controls/backfill-rationale — ersetzt Placeholder "Aus Obligation
abgeleitet." durch LLM-generierte Begruendungen (Ollama/qwen3.5).
Optimierung: gruppiert ~86k Controls nach ~7k Parents, ein LLM-Call pro Parent.
Paginierung via batch_size/offset fuer kontrollierte Ausfuehrung.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DB-Constraint erlaubt nur must/should/may. 'can' gibt es nicht.
Alle Referenzen auf 'can' durch 'may' ersetzt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Die Obligation kennt ihren Parent-Rich-Control direkt. Dessen
source_citation->>'source' gibt die Quell-Regulierung zuverlaessiger
als der Umweg ueber control_parent_links (M:N-Inflation).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend: provenance endpoint (obligations, doc refs, merged duplicates,
regulations summary) + atomic-stats aggregation endpoint.
Frontend: ControlDetail mit Provenance-Sektionen, klickbare Navigation,
neue /sdk/atomic-controls Seite mit Stats-Bar und gefilterer Liste.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SQLAlchemy's text() parser doesn't properly handle :param::type
syntax — it fails to recognize :dd as a bind parameter when followed
by ::jsonb. Using CAST(:dd AS jsonb) instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SQLAlchemy sessions enter a failed state after SQL errors.
Without rollback(), all subsequent queries on the same session
fail with InFailedSqlTransaction. Added try/except with rollback
in _mark_duplicate, _mark_duplicate_to, _write_review, cross-group
pass, and the main phase1 loop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All Pass 0b controls have pattern_id=NULL. Rewritten to:
- Phase 1: Group by merge_group_hint (action:object:trigger), 52k groups
- Phase 2: Cross-group embedding search for semantically similar masters
- Qdrant search uses unfiltered cross-regulation endpoint
- API param changed: pattern_id → hint_filter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
