These files existed on the feature branch but were never cherry-picked
to main, causing ModuleNotFoundError on import:
- dse_parser.py — parses DSE HTML into structured sections
- dse_matcher.py — matches detected services against DSE sections
- mandatory_content_checker.py — checks Art. 13 DSGVO mandatory fields
- legal_basis_validator.py — validates legal basis (lit. a-f)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
NameError: name 're' is not defined at line 146 — the import was
accidentally removed when extracting helper functions to agent_scan_helpers.py.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- CookieBannerOverlay: shows vendors per category with expandable tables
(Verarbeiter, Cookies, Dauer, Land) for full transparency
- Demo vendors: 4 necessary, 3 statistics, 3 marketing, 3 functional
- cookie_table_generator.py: renders {{COOKIE_TABLE}} Markdown tables
from vendor configs (DB) or service registry (fallback)
- SERVICE_COOKIES: 16 known vendor-to-cookie mappings with provider + country
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Email now lists all scanned URLs with checkmark/cross status.
Frontend shows collapsible "X Seiten gescannt — Details anzeigen".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SDK LLM chat returns empty content due to Qwen think-mode. Direct Ollama
/api/generate call with stream:false gets the full response including
think tags which we strip.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Backend: mode field in request, adapts summary tone and email subject
- Pre-launch: "Implementieren Sie X vor Veroeffentlichung"
- Post-launch: "ACHTUNG: Maengel sind oeffentlich sichtbar, sofortige Nachbesserung"
- Frontend: Mode toggle (internes Dokument vs. Live-Website)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Scan public website for cancellation button, imprint, privacy link, cookie consent
- Generate follow-up questions when checks can't be verified without login
- User answers "no" → finding with legal basis is added to results
- Frontend: FollowUpQuestions component with Ja/Nein buttons
- Sidebar: "Compliance Agent" entry added under KI-Compliance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SessionLocal: 5x verwendet fuer DB-Sessions ausserhalb Depends()
HTTPException: verwendet in Framework-Validation
text: 55x verwendet fuer raw SQL queries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Python: add missing 'import enum' to compliance/db/models.py shim.
TypeScript: remove duplicate export of useVendorCompliance from
vendor-compliance/context.tsx (already exported from ./hooks).
Docs: add mandatory pre-push checklist (lint + test + build) to
AGENTS.python.md and AGENTS.go.md. [guardrail-change]
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- loc-budget CI job: remove if/else PR-only guard; now runs scripts/check-loc.sh
(no || true) on every push and PR, scanning the full repo
- sbom-scan: remove || true from grype command — high+ CVEs now block PRs
- scripts/check-loc.sh: add test_*.py / */test_*.py and *.html exclusions so
Python test files and Jinja/HTML templates are not counted against the budget
- .claude/rules/loc-exceptions.txt: grandfather 40 remaining oversized files
into the exceptions list (one-off scripts, docs copies, platform SDKs,
and Phase 1 backend-compliance refactor backlog)
- ai-compliance-sdk/.golangci.yml: add strict golangci-lint config (errcheck,
govet, staticcheck, gosec, gocyclo, gocritic, revive, goimports)
- delete stray routes.py.backup (2512 LOC)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds scoped mypy disable-error-code headers to all 15 agent-created
service files covering the ORM Column[T] + raw-SQL result type issues.
Updates mypy.ini to flip 14 personally-refactored route files to strict;
defers 4 agent-refactored routes (dsr, vendor, notfallplan, isms) until
return type annotations are added.
mypy compliance/ -> Success: no issues found in 162 source files
173/173 pytest pass
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous commit (32e121f) left isms_assessment_service.py at 639 LOC,
exceeding the 500-line hard cap. This follow-up extracts ReadinessCheckService
and OverviewService into a new isms_readiness_service.py (400 LOC), leaving
isms_assessment_service.py at 257 LOC (Management Reviews, Internal Audits,
Audit Trail only).
Updated isms_routes.py imports to reference the new service file.
File sizes after split:
- isms_routes.py: 446 LOC (thin handlers)
- isms_governance_service.py: 416 LOC (scope, context, policy, objectives, SoA)
- isms_findings_service.py: 276 LOC (findings, CAPA)
- isms_assessment_service.py: 257 LOC (mgmt reviews, internal audits, audit trail)
- isms_readiness_service.py: 400 LOC (readiness check, ISO 27001 overview)
All 58 integration tests + 173 unit/contract tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/isms_routes.py (1676 LOC) -> 445 LOC thin routes +
three service files:
- isms_governance_service.py (416) — scope, context, policy, objectives, SoA
- isms_findings_service.py (276) — findings, CAPA, audit trail
- isms_assessment_service.py (639) — management reviews, internal audits,
readiness checks, ISO 27001 overview
NOTE: isms_assessment_service.py exceeds the 500-line hard cap at 639 LOC.
This needs a follow-up split (management_review_service vs
internal_audit_service). Flagged for next session.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split vendor_compliance_routes.py (1107 LOC) into thin route handlers
plus three service modules: VendorService (vendors CRUD/stats/status),
ContractService (contracts CRUD), and FindingService + ControlInstanceService
+ ControlsLibraryService (findings, control instances, controls library).
All files under 500 lines. 215 tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add [mypy-compliance.api.routes] to mypy.ini strict scope
- Fix bare `dict` type annotation in routes.py update_requirement handler
- Fix Column[str] return type in control_export_service.download_file
- Fix unused type:ignore in legal_document_service.upload_word
- Add union-attr ignore for optional requirement null access in routes.py
mypy compliance/ -> Success on 149 source files
173/173 pytest pass
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract consent, audit log, cookie category, and consent stats endpoints
from legal_document_routes into LegalDocumentConsentService. The route
file is now a thin handler layer delegating to LegalDocumentService and
LegalDocumentConsentService with translate_domain_errors(). Legacy
helpers (_doc_to_response, _version_to_response, _transition,
_log_approval) and schemas are re-exported for existing tests. Two
transition tests updated to expect domain errors instead of HTTPException.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/evidence_routes.py (641 LOC) -> 240 LOC thin routes + 460-line
EvidenceService. Manages evidence CRUD, file upload, CI/CD evidence
collection (SAST/dependency/SBOM/container scans), and CI status dashboard.
Service injection pattern: EvidenceService takes the EvidenceRepository,
ControlRepository, and AutoRiskUpdater classes as constructor parameters.
The route's get_evidence_service factory reads these class references from
its own module namespace so tests that
``patch("compliance.api.evidence_routes.EvidenceRepository", ...)`` still
take effect through the factory.
The `_store_evidence` and `_update_risks` helpers stay as module-level
callables in evidence_service and are re-exported from the route module.
The collect_ci_evidence handler remains inline (not delegated to a service
method) so tests can patch
`compliance.api.evidence_routes._store_evidence` and have the patch take
effect at the handler's call site.
Legacy re-exports via __all__: SOURCE_CONTROL_MAP, EvidenceRepository,
ControlRepository, AutoRiskUpdater, _parse_ci_evidence,
_extract_findings_detail, _store_evidence, _update_risks.
Verified:
- 208/208 pytest (core + 35 evidence tests) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 135 source files
- evidence_routes.py 641 -> 240 LOC
- Hard-cap violations: 10 -> 9
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/screening_routes.py (597 LOC) -> 233 LOC thin routes +
353-line ScreeningService + 60-line schemas file. Manages SBOM generation
(CycloneDX 1.5) and OSV.dev vulnerability scanning.
Pure helpers (parse_package_lock, parse_requirements_txt, parse_yarn_lock,
detect_and_parse, generate_sbom, query_osv, map_osv_severity,
extract_fix_version, scan_vulnerabilities) moved to the service module.
The two lookup endpoints (get_screening, list_screenings) delegate to
the new ScreeningService class.
Test-mock compatibility: tests/test_screening_routes.py uses
`patch("compliance.api.screening_routes.SessionLocal", ...)` and
`patch("compliance.api.screening_routes.scan_vulnerabilities", ...)`.
Both names are re-imported and re-exported from the route module so the
patches still take effect. The scan handler keeps direct
`SessionLocal()` usage; the lookup handlers also use SessionLocal so the
test mocks intercept them.
Latent bug fixed: the original scan handler had
text = content.decode("utf-8")
on line 339, shadowing the imported `sqlalchemy.text` so that the
subsequent `text("INSERT ...")` calls would have raised at runtime.
The variable is now named `file_text`. Allowed under "minor behavior
fixes" — the bug was unreachable in tests because they always patched
SessionLocal.
Verified:
- 240/240 pytest pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 134 source files
- screening_routes.py 597 -> 233 LOC
- Hard-cap violations: 11 -> 10
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/canonical_control_routes.py (514 LOC) -> 192 LOC thin
routes + 316-line CanonicalControlService + 105-line schemas file.
Canonical Control Library manages OWASP/NIST/ENISA-anchored security
control frameworks and controls. Like company_profile_routes, this file
uses raw SQL via sqlalchemy.text() because there are no SQLAlchemy
models for canonical_control_frameworks or canonical_controls.
Single-service split. Session management moved from bespoke
`with SessionLocal() as db:` blocks to Depends(get_db) for consistency.
Legacy test imports preserved via re-export (FrameworkResponse,
ControlResponse, SimilarityCheckRequest, SimilarityCheckResponse,
_control_row).
Validation extracted to a module-level `_validate_control_input` helper
so both create and update share the same checks. ValidationError (from
compliance.domain) replaces raw HTTPException(400) raises.
Verified:
- 187/187 pytest (173 core + 14 canonical) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 130 source files
- canonical_control_routes.py 514 -> 192 LOC
- Hard-cap violations: 13 -> 12
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/vvt_routes.py (550 LOC) -> 225 LOC thin routes + 475-line
VVTService. Covers the organization header, processing activities CRUD,
audit log, JSON/CSV export, stats, and version lookups for the Art. 30
DSGVO Verzeichnis.
Single-service split: organization + activities + audit + stats all
revolve around the same tenant's VVT document, and the existing test
suite (tests/test_vvt_routes.py — 768 LOC, tests/test_vvt_tenant_isolation.py
— 205 LOC) exercises them together.
Module-level helpers (_activity_to_response, _log_audit, _export_csv)
stay module-level in compliance.services.vvt_service and are re-exported
from compliance.api.vvt_routes so the two test files keep importing
from the old path.
Pydantic schemas already live in compliance.schemas.vvt from Step 3 —
no new schema file needed this round.
mypy.ini flips compliance.api.vvt_routes from ignore_errors=True to
False. Two SQLAlchemy Column[str] vs str dict-index errors fixed with
explicit str() casts on status/business_function in the stats loop.
Verified:
- 242/242 pytest (173 core + 69 VVT integration) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 128 source files
- vvt_routes.py 550 -> 225 LOC
- vvt_service.py 475 LOC (under 500 hard cap)
- Hard-cap violations: 14 -> 13
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/company_profile_routes.py (640 LOC) -> 154 LOC thin routes.
Unusual for this repo: persistence uses raw SQL via sqlalchemy.text()
because the underlying compliance_company_profiles table has ~45 columns
with complex jsonb coercion and there is no SQLAlchemy model for it.
New files:
compliance/schemas/company_profile.py (127) — 4 request/response models
compliance/services/company_profile_service.py (340) — Service class + row_to_response + log_audit
compliance/services/_company_profile_sql.py (139) — 70-line INSERT/UPDATE statements
separated for readability
Minor behavioral improvement: the handlers now use Depends(get_db) for
session management instead of the bespoke `db = SessionLocal(); try: ...
finally: db.close()` pattern. This makes the routes consistent with
every other refactored service, fixes the broken-ness under test
dependency_overrides, and removes 6 duplicate try/finally blocks.
Legacy exports preserved: CompanyProfileRequest, CompanyProfileResponse,
AuditEntryResponse, AuditListResponse, row_to_response, and log_audit are
re-exported from compliance.api.company_profile_routes so that the two
existing test files
(tests/test_company_profile_routes.py, tests/test_company_profile_extend.py)
keep importing from the same path.
Pre-existing broken tests noted: 6 tests in those files feed a 40-tuple
row into row_to_response, but _BASE_COLUMNS_LIST has 46 columns (has had
since the Phase 2 Stammdaten extension). These tests fail on main too
(verified via `git stash` round-trip). Not fixed in this commit — they
require a rewrite of the test's _make_row helper, which is out of scope
for a pure structural refactor. Flagged for follow-up.
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 127 source files
- company_profile_routes.py 640 -> 154 LOC
- All new files under soft 300 target except service (340, under hard 500)
- Hard-cap violations: 15 -> 14
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compliance/api/tom_routes.py (609 LOC) -> 215 LOC thin routes +
434-line TOMService. Request bodies (TOMStateBody, TOMMeasureCreate,
TOMMeasureUpdate, TOMMeasureBulkItem, TOMMeasureBulkBody) moved to
compliance/schemas/tom.py (joining the existing response models from
the Step 3 split).
Single-service split (not two like banner): state, measures CRUD + bulk
upsert, stats, export, and version lookups are all tightly coupled
around the TOMMeasureDB aggregate, so splitting would create artificial
boundaries. TOMService is 434 LOC — comfortably under the 500 hard cap.
Domain error mapping:
- ConflictError -> 409 (version conflict on state save; duplicate control_id on create)
- NotFoundError -> 404 (missing measure on update; missing version)
- ValidationError -> 400 (missing tenant_id on DELETE /state)
Legacy test compat: the existing tests/test_tom_routes.py imports
TOMMeasureBulkItem, _parse_dt, _measure_to_dict, and DEFAULT_TENANT_ID
directly from compliance.api.tom_routes. All re-exported via __all__ so
the 44-test file runs unchanged.
mypy.ini flips compliance.api.tom_routes from ignore_errors=True to
False. TOMService carries the scoped Column[T] header.
Verified:
- 217/217 pytest (173 baseline + 44 TOM) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 124 source files
- tom_routes.py 609 -> 215 LOC
- Hard-cap violations: 16 -> 15
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 4, file 2 of 18. Same cookbook as audit_routes (4a91814 +
883ef70) applied to banner_routes.py.
compliance/api/banner_routes.py (653 LOC) is decomposed into:
compliance/api/banner_routes.py (255) — thin handlers
compliance/services/banner_consent_service.py (298) — public SDK surface
compliance/services/banner_admin_service.py (238) — site/category/vendor CRUD
compliance/services/_banner_serializers.py ( 81) — ORM-to-dict helpers
shared between the
two services
compliance/schemas/banner.py ( 85) — Pydantic request models
Split rationale: the SDK-facing endpoints (consent CRUD, config
retrieval, export, stats) and the admin CRUD endpoints (sites +
categories + vendors) have distinct audiences and different auth stories,
and combined they would push the service file over the 500 hard cap.
Two focused services is cleaner than one ~540-line god class.
The shared ORM-to-dict helpers live in a private sibling module
(_banner_serializers) rather than a static method on either service, so
both services can import without a cycle.
Handlers follow the established pattern:
- Depends(get_consent_service) or Depends(get_admin_service)
- `with translate_domain_errors():` wrapping the service call
- Explicit return type annotations
- ~3-5 lines per handler
Services raise NotFoundError / ConflictError / ValidationError from
compliance.domain; no HTTPException in the service layer.
mypy.ini flips compliance.api.banner_routes from ignore_errors=True to
False, joining audit_routes in the strict scope. The services carry the
same scoped `# mypy: disable-error-code="arg-type,assignment"` header
used by the audit services for the ORM Column[T] issue.
Pydantic schemas moved to compliance.schemas.banner (mirroring the Step 3
schemas split). They were previously defined inline in banner_routes.py
and not referenced by anything outside it, so no backwards-compat shim
is needed.
Verified:
- 224/224 pytest (173 baseline + 26 audit integration + 25 banner
integration) pass
- tests/contracts/test_openapi_baseline.py green (360/484 unchanged)
- mypy compliance/ -> Success: no issues found in 123 source files
- All new files under the 300 soft target (largest: 298)
- banner_routes.py drops from 653 -> 255 LOC (below hard cap)
Hard-cap violations remaining: 16 (was 17).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 4 follow-up addressing the debt flagged in the worked-example
commit (4a91814).
## mypy --strict policy
Adds backend-compliance/mypy.ini declaring the strict-mode scope:
Fully strict (enforced today):
- compliance/domain/
- compliance/schemas/
- compliance/api/_http_errors.py
- compliance/api/audit_routes.py (refactored in Step 4)
- compliance/services/audit_session_service.py
- compliance/services/audit_signoff_service.py
Loose (ignore_errors=True) with a migration path:
- compliance/db/* — SQLAlchemy 1.x Column[] vs
runtime T; unblocks Phase 1
until a Mapped[T] migration.
- compliance/api/<route>.py — each route file flips to
strict as its own Step 4
refactor lands.
- compliance/services/<legacy util> — 14 utility services
(llm_provider, pdf_extractor,
seeder, ...) that predate the
clean-arch refactor.
- compliance/tests/ — excluded (legacy placeholder
style). The new TestClient-
based integration suite is
type-annotated.
The two new service files carry a scoped `# mypy: disable-error-code="arg-type,assignment"`
header for the ORM Column[T] issue — same underlying SQLAlchemy limitation,
narrowly scoped rather than wholesale ignore_errors.
Flow: `cd backend-compliance && mypy compliance/` -> clean on 119 files.
CI yaml updated to use the config instead of ad-hoc package lists.
## Bugs fixed while enabling strict
mypy --strict surfaced two latent bugs in the pre-refactor code. Both
were invisible because the old `compliance/tests/test_audit_routes.py`
is a placeholder suite that asserts on request-data shape and never
calls the handlers:
- AuditSessionResponse.updated_at is a required field in the schema,
but the original handler didn't pass it. Fixed in
AuditSessionService._to_response.
- PaginationMeta requires has_next + has_prev. The original audit
checklist handler didn't compute them. Fixed in
AuditSignOffService.get_checklist.
Both are behavior-preserving at the HTTP level because the old code
would have raised Pydantic ValidationError at response serialization
had the endpoint actually been exercised.
## Integration test suite
Adds backend-compliance/tests/test_audit_routes_integration.py — 26
real TestClient tests against an in-memory sqlite backend (StaticPool).
Replaces the coverage gap left by the placeholder suite.
Covers:
- Session CRUD + lifecycle transitions (draft -> in_progress -> completed
-> archived), including the 409 paths for illegal transitions
- Checklist pagination, filtering, search
- Sign-off create / update / auto-start-session / count-flipping
- Sign-off 400 (invalid result), 404 (missing requirement), 409 (completed session)
- Get-signoff 404 / 200 round-trip
Uses a module-scoped schema fixture + per-test DELETE-sweep so the
suite runs in ~2.3s despite the ~50-table ORM surface.
Verified:
- 199/199 pytest (173 original + 26 new audit integration) pass
- tests/contracts/test_openapi_baseline.py green, OpenAPI 360/484 unchanged
- mypy compliance/ -> Success: no issues found in 119 source files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Benjamin Admin
sharang