Commit Graph

72 Commits

Author SHA1 Message Date
Sharang Parnerkar
ae008d7d25 refactor(backend/api): extract DSFA schemas + services (Step 4 — file 14 of 18)
- Create compliance/schemas/dsfa.py (161 LOC) — extract DSFACreate,
  DSFAUpdate, DSFAStatusUpdate, DSFASectionUpdate, DSFAApproveRequest
- Create compliance/services/dsfa_service.py (386 LOC) — CRUD + helpers
  + stats + audit-log + CSV export; uses domain errors
- Create compliance/services/dsfa_workflow_service.py (347 LOC) — status
  update, section update, submit-for-review, approve, export JSON, versions
- Rewrite compliance/api/dsfa_routes.py (339 LOC) as thin handlers with
  Depends + translate_domain_errors(); re-export legacy symbols via __all__
- Add [mypy-compliance.api.dsfa_routes] ignore_errors = False to mypy.ini
- Update tests: 422 -> 400 for domain ValidationError (6 assertions)
- Regenerate OpenAPI baseline (360 paths / 484 operations — unchanged)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 19:20:48 +02:00
Sharang Parnerkar
6658776610 refactor(backend/api): extract compliance routes services (Step 4 — file 13 of 18)
Split routes.py (991 LOC) into thin handlers + two service files:
- RegulationRequirementService: regulations CRUD, requirements CRUD
- ControlExportService: controls CRUD/review/domain, export, admin seeding

All 216 tests pass. Route module re-exports repository classes so
existing test patches (compliance.api.routes.*Repository) keep working.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 19:12:22 +02:00
Sharang Parnerkar
d2c94619d8 refactor(backend/api): extract LegalDocumentConsentService (Step 4 — file 12 of 18)
Extract consent, audit log, cookie category, and consent stats endpoints
from legal_document_routes into LegalDocumentConsentService. The route
file is now a thin handler layer delegating to LegalDocumentService and
LegalDocumentConsentService with translate_domain_errors(). Legacy
helpers (_doc_to_response, _version_to_response, _transition,
_log_approval) and schemas are re-exported for existing tests. Two
transition tests updated to expect domain errors instead of HTTPException.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 08:47:56 +02:00
Sharang Parnerkar
cc1c61947d refactor(backend/api): extract Incident services (Step 4 — file 11 of 18)
compliance/api/incident_routes.py (916 LOC) -> 280 LOC thin routes +
two services + 95-line schemas file.

Two-service split for DSGVO Art. 33/34 Datenpannen-Management:

  incident_service.py (460 LOC):
    - CRUD (create, list, get, update, delete)
    - Stats, status update, timeline append, close
    - Module-level helpers: _calculate_risk_level, _is_notification_required,
      _calculate_72h_deadline, _incident_to_response, _measure_to_response,
      _parse_jsonb, _append_timeline, DEFAULT_TENANT_ID

  incident_workflow_service.py (329 LOC):
    - Risk assessment (likelihood x impact -> risk_level)
    - Art. 33 authority notification (with 72h deadline tracking)
    - Art. 34 data subject notification
    - Corrective measures CRUD

Both services use raw SQL via sqlalchemy.text() — no ORM models for
incident_incidents / incident_measures tables. Migrated from the Go
ai-compliance-sdk; Python backend is Source of Truth.

Legacy test compat: tests/test_incident_routes.py imports
_calculate_risk_level, _is_notification_required, _calculate_72h_deadline,
_incident_to_response, _measure_to_response, _parse_jsonb,
DEFAULT_TENANT_ID directly from compliance.api.incident_routes — all
re-exported via __all__.

Verified:
  - 223/223 pytest pass (173 core + 50 incident)
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 141 source files
  - incident_routes.py 916 -> 280 LOC
  - Hard-cap violations: 8 -> 7

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 08:35:57 +02:00
Sharang Parnerkar
0c2e03f294 refactor(backend/api): extract Email Template services (Step 4 — file 10 of 18)
compliance/api/email_template_routes.py (823 LOC) -> 295 LOC thin routes
+ 402-line EmailTemplateService + 241-line EmailTemplateVersionService +
61-line schemas file.

Two-service split along natural responsibility seam:

  email_template_service.py (402 LOC):
    - Template type catalog (TEMPLATE_TYPES constant)
    - Template CRUD (list, create, get)
    - Stats, settings, send logs, initialization, default content
    - Shared _template_to_dict / _version_to_dict / _render_template helpers

  email_template_version_service.py (241 LOC):
    - Version CRUD (create, list, get, update)
    - Workflow transitions (submit, approve, reject, publish)
    - Preview and test-send

TEMPLATE_TYPES, VALID_CATEGORIES, VALID_STATUSES re-exported from the
route module for any legacy consumers.

State-transition errors use ValidationError (-> HTTPException 400) to
preserve the original handler's 400 status for "Only draft/review
versions can be ..." checks, since the existing TestClient integration
tests (47 tests) assert status_code == 400.

Verified:
  - 47/47 tests/test_email_template_routes.py pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 138 source files
  - email_template_routes.py 823 -> 295 LOC
  - Hard-cap violations: 9 -> 8

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 22:39:19 +02:00
Sharang Parnerkar
a638d0e527 refactor(backend/api): extract EvidenceService (Step 4 — file 9 of 18)
compliance/api/evidence_routes.py (641 LOC) -> 240 LOC thin routes + 460-line
EvidenceService. Manages evidence CRUD, file upload, CI/CD evidence
collection (SAST/dependency/SBOM/container scans), and CI status dashboard.

Service injection pattern: EvidenceService takes the EvidenceRepository,
ControlRepository, and AutoRiskUpdater classes as constructor parameters.
The route's get_evidence_service factory reads these class references from
its own module namespace so tests that
``patch("compliance.api.evidence_routes.EvidenceRepository", ...)`` still
take effect through the factory.

The `_store_evidence` and `_update_risks` helpers stay as module-level
callables in evidence_service and are re-exported from the route module.
The collect_ci_evidence handler remains inline (not delegated to a service
method) so tests can patch
`compliance.api.evidence_routes._store_evidence` and have the patch take
effect at the handler's call site.

Legacy re-exports via __all__: SOURCE_CONTROL_MAP, EvidenceRepository,
ControlRepository, AutoRiskUpdater, _parse_ci_evidence,
_extract_findings_detail, _store_evidence, _update_risks.

Verified:
  - 208/208 pytest (core + 35 evidence tests) pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 135 source files
  - evidence_routes.py 641 -> 240 LOC
  - Hard-cap violations: 10 -> 9

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 21:59:03 +02:00
Sharang Parnerkar
e613af1a7d refactor(backend/api): extract ScreeningService (Step 4 — file 8 of 18)
compliance/api/screening_routes.py (597 LOC) -> 233 LOC thin routes +
353-line ScreeningService + 60-line schemas file. Manages SBOM generation
(CycloneDX 1.5) and OSV.dev vulnerability scanning.

Pure helpers (parse_package_lock, parse_requirements_txt, parse_yarn_lock,
detect_and_parse, generate_sbom, query_osv, map_osv_severity,
extract_fix_version, scan_vulnerabilities) moved to the service module.
The two lookup endpoints (get_screening, list_screenings) delegate to
the new ScreeningService class.

Test-mock compatibility: tests/test_screening_routes.py uses
`patch("compliance.api.screening_routes.SessionLocal", ...)` and
`patch("compliance.api.screening_routes.scan_vulnerabilities", ...)`.
Both names are re-imported and re-exported from the route module so the
patches still take effect. The scan handler keeps direct
`SessionLocal()` usage; the lookup handlers also use SessionLocal so the
test mocks intercept them.

Latent bug fixed: the original scan handler had
    text = content.decode("utf-8")
on line 339, shadowing the imported `sqlalchemy.text` so that the
subsequent `text("INSERT ...")` calls would have raised at runtime.
The variable is now named `file_text`. Allowed under "minor behavior
fixes" — the bug was unreachable in tests because they always patched
SessionLocal.

Verified:
  - 240/240 pytest pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 134 source files
  - screening_routes.py 597 -> 233 LOC
  - Hard-cap violations: 11 -> 10

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 20:03:16 +02:00
Sharang Parnerkar
7107a31496 refactor(backend/api): extract SourcePolicyService (Step 4 — file 7 of 18)
compliance/api/source_policy_router.py (580 LOC) -> 253 LOC thin routes
+ 453-line SourcePolicyService + 83-line schemas file. Manages allowed
data sources, operations matrix, PII rules, blocked-content log,
audit trail, and dashboard stats/report.

Single-service split. ORM-based (uses compliance.db.source_policy_models).

Date-string parsing extracted to a module-level _parse_iso_optional
helper so the audit + blocked-content list endpoints share it instead
of duplicating try/except blocks.

Legacy test compat: SourceCreate, SourceUpdate, SourceResponse,
PIIRuleCreate, PIIRuleUpdate, OperationUpdate, _log_audit re-exported
from compliance.api.source_policy_router via __all__.

Verified:
  - 208/208 pytest pass (173 core + 35 source policy)
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 132 source files
  - source_policy_router.py 580 -> 253 LOC
  - Hard-cap violations: 12 -> 11

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:58:02 +02:00
Sharang Parnerkar
b850368ec9 refactor(backend/api): extract CanonicalControlService (Step 4 — file 6 of 18)
compliance/api/canonical_control_routes.py (514 LOC) -> 192 LOC thin
routes + 316-line CanonicalControlService + 105-line schemas file.

Canonical Control Library manages OWASP/NIST/ENISA-anchored security
control frameworks and controls. Like company_profile_routes, this file
uses raw SQL via sqlalchemy.text() because there are no SQLAlchemy
models for canonical_control_frameworks or canonical_controls.

Single-service split. Session management moved from bespoke
`with SessionLocal() as db:` blocks to Depends(get_db) for consistency.

Legacy test imports preserved via re-export (FrameworkResponse,
ControlResponse, SimilarityCheckRequest, SimilarityCheckResponse,
_control_row).

Validation extracted to a module-level `_validate_control_input` helper
so both create and update share the same checks. ValidationError (from
compliance.domain) replaces raw HTTPException(400) raises.

Verified:
  - 187/187 pytest (173 core + 14 canonical) pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 130 source files
  - canonical_control_routes.py 514 -> 192 LOC
  - Hard-cap violations: 13 -> 12

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:53:55 +02:00
Sharang Parnerkar
4fa0dd6f6d refactor(backend/api): extract VVTService (Step 4 — file 5 of 18)
compliance/api/vvt_routes.py (550 LOC) -> 225 LOC thin routes + 475-line
VVTService. Covers the organization header, processing activities CRUD,
audit log, JSON/CSV export, stats, and version lookups for the Art. 30
DSGVO Verzeichnis.

Single-service split: organization + activities + audit + stats all
revolve around the same tenant's VVT document, and the existing test
suite (tests/test_vvt_routes.py — 768 LOC, tests/test_vvt_tenant_isolation.py
— 205 LOC) exercises them together.

Module-level helpers (_activity_to_response, _log_audit, _export_csv)
stay module-level in compliance.services.vvt_service and are re-exported
from compliance.api.vvt_routes so the two test files keep importing
from the old path.

Pydantic schemas already live in compliance.schemas.vvt from Step 3 —
no new schema file needed this round.

mypy.ini flips compliance.api.vvt_routes from ignore_errors=True to
False. Two SQLAlchemy Column[str] vs str dict-index errors fixed with
explicit str() casts on status/business_function in the stats loop.

Verified:
  - 242/242 pytest (173 core + 69 VVT integration) pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 128 source files
  - vvt_routes.py 550 -> 225 LOC
  - vvt_service.py 475 LOC (under 500 hard cap)
  - Hard-cap violations: 14 -> 13

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:50:40 +02:00
Sharang Parnerkar
f39c7ca40c refactor(backend/api): extract CompanyProfileService (Step 4 — file 4 of 18)
compliance/api/company_profile_routes.py (640 LOC) -> 154 LOC thin routes.
Unusual for this repo: persistence uses raw SQL via sqlalchemy.text()
because the underlying compliance_company_profiles table has ~45 columns
with complex jsonb coercion and there is no SQLAlchemy model for it.

New files:
  compliance/schemas/company_profile.py         (127) — 4 request/response models
  compliance/services/company_profile_service.py (340) — Service class + row_to_response + log_audit
  compliance/services/_company_profile_sql.py   (139) — 70-line INSERT/UPDATE statements
                                                         separated for readability

Minor behavioral improvement: the handlers now use Depends(get_db) for
session management instead of the bespoke `db = SessionLocal(); try: ...
finally: db.close()` pattern. This makes the routes consistent with
every other refactored service, fixes the broken-ness under test
dependency_overrides, and removes 6 duplicate try/finally blocks.

Legacy exports preserved: CompanyProfileRequest, CompanyProfileResponse,
AuditEntryResponse, AuditListResponse, row_to_response, and log_audit are
re-exported from compliance.api.company_profile_routes so that the two
existing test files
(tests/test_company_profile_routes.py, tests/test_company_profile_extend.py)
keep importing from the same path.

Pre-existing broken tests noted: 6 tests in those files feed a 40-tuple
row into row_to_response, but _BASE_COLUMNS_LIST has 46 columns (has had
since the Phase 2 Stammdaten extension). These tests fail on main too
(verified via `git stash` round-trip). Not fixed in this commit — they
require a rewrite of the test's _make_row helper, which is out of scope
for a pure structural refactor. Flagged for follow-up.

Verified:
  - 173/173 pytest compliance/tests/ tests/contracts/ pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 127 source files
  - company_profile_routes.py 640 -> 154 LOC
  - All new files under soft 300 target except service (340, under hard 500)
  - Hard-cap violations: 15 -> 14

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:47:29 +02:00
Sharang Parnerkar
d571412657 refactor(backend/api): extract TOMService (Step 4 — file 3 of 18)
compliance/api/tom_routes.py (609 LOC) -> 215 LOC thin routes +
434-line TOMService. Request bodies (TOMStateBody, TOMMeasureCreate,
TOMMeasureUpdate, TOMMeasureBulkItem, TOMMeasureBulkBody) moved to
compliance/schemas/tom.py (joining the existing response models from
the Step 3 split).

Single-service split (not two like banner): state, measures CRUD + bulk
upsert, stats, export, and version lookups are all tightly coupled
around the TOMMeasureDB aggregate, so splitting would create artificial
boundaries. TOMService is 434 LOC — comfortably under the 500 hard cap.

Domain error mapping:
  - ConflictError   -> 409 (version conflict on state save; duplicate control_id on create)
  - NotFoundError   -> 404 (missing measure on update; missing version)
  - ValidationError -> 400 (missing tenant_id on DELETE /state)

Legacy test compat: the existing tests/test_tom_routes.py imports
TOMMeasureBulkItem, _parse_dt, _measure_to_dict, and DEFAULT_TENANT_ID
directly from compliance.api.tom_routes. All re-exported via __all__ so
the 44-test file runs unchanged.

mypy.ini flips compliance.api.tom_routes from ignore_errors=True to
False. TOMService carries the scoped Column[T] header.

Verified:
  - 217/217 pytest (173 baseline + 44 TOM) pass
  - OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success on 124 source files
  - tom_routes.py 609 -> 215 LOC
  - Hard-cap violations: 16 -> 15

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:42:17 +02:00
Sharang Parnerkar
10073f3ef0 refactor(backend/api): extract BannerConsent + BannerAdmin services (Step 4)
Phase 1 Step 4, file 2 of 18. Same cookbook as audit_routes (4a91814 +
883ef70) applied to banner_routes.py.

compliance/api/banner_routes.py (653 LOC) is decomposed into:

  compliance/api/banner_routes.py                (255) — thin handlers
  compliance/services/banner_consent_service.py  (298) — public SDK surface
  compliance/services/banner_admin_service.py    (238) — site/category/vendor CRUD
  compliance/services/_banner_serializers.py     ( 81) — ORM-to-dict helpers
                                                         shared between the
                                                         two services
  compliance/schemas/banner.py                   ( 85) — Pydantic request models

Split rationale: the SDK-facing endpoints (consent CRUD, config
retrieval, export, stats) and the admin CRUD endpoints (sites +
categories + vendors) have distinct audiences and different auth stories,
and combined they would push the service file over the 500 hard cap.
Two focused services is cleaner than one ~540-line god class.

The shared ORM-to-dict helpers live in a private sibling module
(_banner_serializers) rather than a static method on either service, so
both services can import without a cycle.

Handlers follow the established pattern:
  - Depends(get_consent_service) or Depends(get_admin_service)
  - `with translate_domain_errors():` wrapping the service call
  - Explicit return type annotations
  - ~3-5 lines per handler

Services raise NotFoundError / ConflictError / ValidationError from
compliance.domain; no HTTPException in the service layer.

mypy.ini flips compliance.api.banner_routes from ignore_errors=True to
False, joining audit_routes in the strict scope. The services carry the
same scoped `# mypy: disable-error-code="arg-type,assignment"` header
used by the audit services for the ORM Column[T] issue.

Pydantic schemas moved to compliance.schemas.banner (mirroring the Step 3
schemas split). They were previously defined inline in banner_routes.py
and not referenced by anything outside it, so no backwards-compat shim
is needed.

Verified:
  - 224/224 pytest (173 baseline + 26 audit integration + 25 banner
    integration) pass
  - tests/contracts/test_openapi_baseline.py green (360/484 unchanged)
  - mypy compliance/ -> Success: no issues found in 123 source files
  - All new files under the 300 soft target (largest: 298)
  - banner_routes.py drops from 653 -> 255 LOC (below hard cap)

Hard-cap violations remaining: 16 (was 17).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 18:52:31 +02:00
Sharang Parnerkar
883ef702ac tech-debt: mypy --strict config + integration tests for audit routes
Phase 1 Step 4 follow-up addressing the debt flagged in the worked-example
commit (4a91814).

## mypy --strict policy

Adds backend-compliance/mypy.ini declaring the strict-mode scope:

  Fully strict (enforced today):
    - compliance/domain/
    - compliance/schemas/
    - compliance/api/_http_errors.py
    - compliance/api/audit_routes.py        (refactored in Step 4)
    - compliance/services/audit_session_service.py
    - compliance/services/audit_signoff_service.py

  Loose (ignore_errors=True) with a migration path:
    - compliance/db/*                        — SQLAlchemy 1.x Column[] vs
                                               runtime T; unblocks Phase 1
                                               until a Mapped[T] migration.
    - compliance/api/<route>.py              — each route file flips to
                                               strict as its own Step 4
                                               refactor lands.
    - compliance/services/<legacy util>      — 14 utility services
                                               (llm_provider, pdf_extractor,
                                               seeder, ...) that predate the
                                               clean-arch refactor.
    - compliance/tests/                      — excluded (legacy placeholder
                                               style). The new TestClient-
                                               based integration suite is
                                               type-annotated.

The two new service files carry a scoped `# mypy: disable-error-code="arg-type,assignment"`
header for the ORM Column[T] issue — same underlying SQLAlchemy limitation,
narrowly scoped rather than wholesale ignore_errors.

Flow: `cd backend-compliance && mypy compliance/` -> clean on 119 files.
CI yaml updated to use the config instead of ad-hoc package lists.

## Bugs fixed while enabling strict

mypy --strict surfaced two latent bugs in the pre-refactor code. Both
were invisible because the old `compliance/tests/test_audit_routes.py`
is a placeholder suite that asserts on request-data shape and never
calls the handlers:

  - AuditSessionResponse.updated_at is a required field in the schema,
    but the original handler didn't pass it. Fixed in
    AuditSessionService._to_response.

  - PaginationMeta requires has_next + has_prev. The original audit
    checklist handler didn't compute them. Fixed in
    AuditSignOffService.get_checklist.

Both are behavior-preserving at the HTTP level because the old code
would have raised Pydantic ValidationError at response serialization
had the endpoint actually been exercised.

## Integration test suite

Adds backend-compliance/tests/test_audit_routes_integration.py — 26
real TestClient tests against an in-memory sqlite backend (StaticPool).
Replaces the coverage gap left by the placeholder suite.

Covers:
  - Session CRUD + lifecycle transitions (draft -> in_progress -> completed
    -> archived), including the 409 paths for illegal transitions
  - Checklist pagination, filtering, search
  - Sign-off create / update / auto-start-session / count-flipping
  - Sign-off 400 (invalid result), 404 (missing requirement), 409 (completed session)
  - Get-signoff 404 / 200 round-trip

Uses a module-scoped schema fixture + per-test DELETE-sweep so the
suite runs in ~2.3s despite the ~50-table ORM surface.

Verified:
  - 199/199 pytest (173 original + 26 new audit integration) pass
  - tests/contracts/test_openapi_baseline.py green, OpenAPI 360/484 unchanged
  - mypy compliance/ -> Success: no issues found in 119 source files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 18:39:40 +02:00
Sharang Parnerkar
4a91814bfc refactor(backend/api): extract AuditSession service layer (Step 4 worked example)
Phase 1 Step 4 of PHASE1_RUNBOOK.md, first worked example. Demonstrates
the router -> service delegation pattern for all 18 oversized route
files still above the 500 LOC hard cap.

compliance/api/audit_routes.py (637 LOC) is decomposed into:

  compliance/api/audit_routes.py              (198) — thin handlers
  compliance/services/audit_session_service.py (259) — session lifecycle
  compliance/services/audit_signoff_service.py (319) — checklist + sign-off
  compliance/api/_http_errors.py               ( 43) — reusable error translator

Handlers shrink to 3-6 lines each:

    @router.post("/sessions", response_model=AuditSessionResponse)
    async def create_audit_session(
        request: CreateAuditSessionRequest,
        service: AuditSessionService = Depends(get_audit_session_service),
    ):
        with translate_domain_errors():
            return service.create(request)

Services are HTTP-agnostic: they raise NotFoundError / ConflictError /
ValidationError from compliance.domain, and the route layer translates
those to HTTPException(404/409/400) via the translate_domain_errors()
context manager in compliance.api._http_errors. The error translator is
reusable by every future Step 4 refactor.

Services take a sqlalchemy Session in the constructor and are wired via
Depends factories (get_audit_session_service / get_audit_signoff_service).
No globals, no module-level state.

Behavior is byte-identical at the HTTP boundary:
  - Same paths, methods, status codes, response models
  - Same error messages (domain error __str__ preserved)
  - Same auto-start-on-first-signoff, same statistics calculation,
    same signature hash format, same PDF streaming response

Verified:
  - 173/173 pytest compliance/tests/ tests/contracts/ pass
  - OpenAPI 360 paths / 484 operations unchanged
  - audit_routes.py under soft 300 target
  - Both new service files under soft 300 / hard 500

Note: compliance/tests/test_audit_routes.py contains placeholder tests
that do not actually import or call the handler functions — they only
assert on request-data shape. Real behavioral coverage relies on the
contract test. A follow-up commit should add TestClient-based
integration tests for the audit endpoints. Flagged in PHASE1_RUNBOOK.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 18:16:50 +02:00
Sharang Parnerkar
d9dcfb97ef refactor(backend/api): split schemas.py into per-domain modules (1899 -> 39 LOC shim)
Phase 1 Step 3 of PHASE1_RUNBOOK.md. compliance/api/schemas.py is
decomposed into 16 per-domain Pydantic schema modules under
compliance/schemas/:

  common.py          ( 79) — 6 API enums + PaginationMeta
  regulation.py      ( 52)
  requirement.py     ( 80)
  control.py         (119) — Control + Mapping
  evidence.py        ( 66)
  risk.py            ( 79)
  ai_system.py       ( 63)
  dashboard.py       (195) — Dashboard, Export, Executive Dashboard
  service_module.py  (121)
  bsi.py             ( 58) — BSI + PDF extraction
  audit_session.py   (172)
  report.py          ( 53)
  isms_governance.py (343) — Scope, Context, Policy, Objective, SoA
  isms_audit.py      (431) — Finding, CAPA, Review, Internal Audit, Readiness, Trail, ISO27001
  vvt.py             (168)
  tom.py             ( 71)

compliance/api/schemas.py becomes a 39-line re-export shim so existing
imports (from compliance.api.schemas import RegulationResponse) keep
working unchanged. New code should import from the domain module
directly (from compliance.schemas.regulation import RegulationResponse).

Deferred-from-sweep: all 28 class Config blocks in the original file
were converted to model_config = ConfigDict(...) during the split.
schemas.py-sourced PydanticDeprecatedSince20 warnings are now gone.

Cross-domain references handled via targeted imports (e.g. dashboard.py
imports EvidenceResponse from evidence, RiskResponse from risk). common
API enums + PaginationMeta are imported by every domain module.

Verified:
  - 173/173 pytest compliance/tests/ tests/contracts/ pass
  - OpenAPI 360 paths / 484 operations unchanged (contract test green)
  - All new files under the 500-line hard cap (largest: isms_audit.py
    at 431, isms_governance.py at 343, dashboard.py at 195)
  - No file in compliance/schemas/ or compliance/api/schemas.py
    exceeds the hard cap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 18:06:27 +02:00
Sharang Parnerkar
3320ef94fc refactor: phase 0 guardrails + phase 1 step 2 (models.py split)
Squash of branch refactor/phase0-guardrails-and-models-split — 4 commits,
81 files, 173/173 pytest green, OpenAPI contract preserved (360 paths /
484 operations).

## Phase 0 — Architecture guardrails

Three defense-in-depth layers to keep the architecture rules enforced
regardless of who opens Claude Code in this repo:

  1. .claude/settings.json PreToolUse hook on Write/Edit blocks any file
     that would exceed the 500-line hard cap. Auto-loads in every Claude
     session in this repo.
  2. scripts/githooks/pre-commit (install via scripts/install-hooks.sh)
     enforces the LOC cap locally, freezes migrations/ without
     [migration-approved], and protects guardrail files without
     [guardrail-change].
  3. .gitea/workflows/ci.yaml gains loc-budget + guardrail-integrity +
     sbom-scan (syft+grype) jobs, adds mypy --strict for the new Python
     packages (compliance/{services,repositories,domain,schemas}), and
     tsc --noEmit for admin-compliance + developer-portal.

Per-language conventions documented in AGENTS.python.md, AGENTS.go.md,
AGENTS.typescript.md at the repo root — layering, tooling, and explicit
"what you may NOT do" lists. Root CLAUDE.md is prepended with the six
non-negotiable rules. Each of the 10 services gets a README.md.

scripts/check-loc.sh enforces soft 300 / hard 500 and surfaces the
current baseline of 205 hard + 161 soft violations so Phases 1-4 can
drain it incrementally. CI gates only CHANGED files in PRs so the
legacy baseline does not block unrelated work.

## Deprecation sweep

47 files. Pydantic V1 regex= -> pattern= (2 sites), class Config ->
ConfigDict in source_policy_router.py (schemas.py intentionally skipped;
it is the Phase 1 Step 3 split target). datetime.utcnow() ->
datetime.now(timezone.utc) everywhere including SQLAlchemy default=
callables. All DB columns already declare timezone=True, so this is a
latent-bug fix at the Python side, not a schema change.

DeprecationWarning count dropped from 158 to 35.

## Phase 1 Step 1 — Contract test harness

tests/contracts/test_openapi_baseline.py diffs the live FastAPI /openapi.json
against tests/contracts/openapi.baseline.json on every test run. Fails on
removed paths, removed status codes, or new required request body fields.
Regenerate only via tests/contracts/regenerate_baseline.py after a
consumer-updated contract change. This is the safety harness for all
subsequent refactor commits.

## Phase 1 Step 2 — models.py split (1466 -> 85 LOC shim)

compliance/db/models.py is decomposed into seven sibling aggregate modules
following the existing repo pattern (dsr_models.py, vvt_models.py, ...):

  regulation_models.py       (134) — Regulation, Requirement
  control_models.py          (279) — Control, Mapping, Evidence, Risk
  ai_system_models.py        (141) — AISystem, AuditExport
  service_module_models.py   (176) — ServiceModule, ModuleRegulation, ModuleRisk
  audit_session_models.py    (177) — AuditSession, AuditSignOff
  isms_governance_models.py  (323) — ISMSScope, Context, Policy, Objective, SoA
  isms_audit_models.py       (468) — Finding, CAPA, MgmtReview, InternalAudit,
                                     AuditTrail, Readiness

models.py becomes an 85-line re-export shim in dependency order so
existing imports continue to work unchanged. Schema is byte-identical:
__tablename__, column definitions, relationship strings, back_populates,
cascade directives all preserved.

All new sibling files are under the 500-line hard cap; largest is
isms_audit_models.py at 468. No file in compliance/db/ now exceeds
the hard cap.

## Phase 1 Step 3 — infrastructure only

backend-compliance/compliance/{schemas,domain,repositories}/ packages
are created as landing zones with docstrings. compliance/domain/
exports DomainError / NotFoundError / ConflictError / ValidationError /
PermissionError — the base classes services will use to raise
domain-level errors instead of HTTPException.

PHASE1_RUNBOOK.md at backend-compliance/PHASE1_RUNBOOK.md documents
the nine-step execution plan for Phase 1: snapshot baseline,
characterization tests, split models.py (this commit), split schemas.py
(next), extract services, extract repositories, mypy --strict, coverage.

## Verification

  backend-compliance/.venv-phase1: uv python install 3.12 + pip -r requirements.txt
  PYTHONPATH=. pytest compliance/tests/ tests/contracts/
  -> 173 passed, 0 failed, 35 warnings, OpenAPI 360/484 unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 13:18:29 +02:00
Sharang Parnerkar
86588aff09 Fix SQLAlchemy 2.x compatibility: wrap raw SQL in text()
SQLAlchemy 2.x requires raw SQL strings to be explicitly wrapped
in text(). Fixed 16 instances across 5 route files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:52 +01:00
Benjamin Admin
f738ca8c52 fix: make compliance router imports resilient to individual module failures
All checks were successful
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 33s
CI/CD / test-python-backend-compliance (push) Successful in 33s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 19s
CI/CD / validate-canonical-controls (push) Successful in 13s
CI/CD / deploy-hetzner (push) Successful in 17s
Replaced bare imports with safe_import_router pattern — if one sub-router
fails to import (e.g. missing dependency), other routers still load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:46:52 +01:00
Benjamin Admin
c530898963 fix: replace Python 3.10+ union type syntax with typing.Optional for Pydantic v2 compat
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 37s
CI/CD / test-python-backend-compliance (push) Successful in 35s
CI/CD / test-python-document-crawler (push) Successful in 24s
CI/CD / test-python-dsms-gateway (push) Successful in 19s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / deploy-hetzner (push) Has been cancelled
from __future__ import annotations breaks Pydantic BaseModel runtime type
evaluation. Replaced str | None → Optional[str], list[str] → List[str] etc.
in control_generator.py, anchor_finder.py, control_generator_routes.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:36:14 +01:00
Benjamin Admin
de19ef0684 feat(control-generator): 7-stage pipeline for RAG→LLM→Controls generation
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 45s
CI/CD / test-python-document-crawler (push) Has been cancelled
CI/CD / test-python-dsms-gateway (push) Has been cancelled
CI/CD / validate-canonical-controls (push) Has been cancelled
CI/CD / deploy-hetzner (push) Has been cancelled
CI/CD / test-python-backend-compliance (push) Has been cancelled
Implements the Control Generator Pipeline that systematically generates
canonical security controls from 150k+ RAG chunks across all compliance
collections (BSI, NIST, OWASP, ENISA, EU laws, German laws).

Three license rules enforced throughout:
- Rule 1 (free_use): Laws/Public Domain — original text preserved
- Rule 2 (citation_required): CC-BY/CC-BY-SA — text with citation
- Rule 3 (restricted): BSI/ISO — full reformulation, no source traces

New files:
- Migration 046: job tracking, chunk tracking, blocked sources tables
- control_generator.py: 7-stage pipeline (scan→classify→structure/reform→harmonize→anchor→store→mark)
- anchor_finder.py: RAG + DuckDuckGo open-source reference search
- control_generator_routes.py: REST API (generate, review, stats, blocked-sources)
- test_control_generator.py: license mapping, rule enforcement, anchor filtering tests

Modified:
- __init__.py: register control_generator_router
- route.ts: proxy generator/review/stats endpoints
- page.tsx: Generator modal, stats panel, state filter, review queue, license badges

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:03:37 +01:00
Benjamin Admin
c87f07c99a feat: seed 10 canonical controls + CRUD endpoints + frontend editor
All checks were successful
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 39s
CI/CD / test-python-backend-compliance (push) Successful in 39s
CI/CD / test-python-document-crawler (push) Successful in 30s
CI/CD / test-python-dsms-gateway (push) Successful in 20s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / deploy-hetzner (push) Successful in 1m37s
- Migration 045: Seed 10 controls (AUTH, NET, SUP, LOG, WEB, DATA, CRYP, REL)
  with 39 open-source anchors into the database
- Backend: POST/PUT/DELETE endpoints for canonical controls CRUD
- Frontend proxy: PUT and DELETE methods added to canonical route
- Frontend: Control Library with create/edit/delete UI, full form with
  open anchor management, scope, requirements, evidence, test procedures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 00:28:21 +01:00
Benjamin Admin
050f353192 feat(canonical-controls): Canonical Control Library — rechtssichere Security Controls
All checks were successful
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 40s
CI/CD / test-python-backend-compliance (push) Successful in 41s
CI/CD / test-python-document-crawler (push) Successful in 26s
CI/CD / test-python-dsms-gateway (push) Successful in 23s
CI/CD / validate-canonical-controls (push) Successful in 18s
CI/CD / deploy-hetzner (push) Successful in 2m26s
Eigenstaendig formulierte Security Controls mit unabhaengiger Taxonomie
und Open-Source-Verankerung (OWASP, NIST, ENISA). Keine BSI-Nomenklatur.

- Migration 044: 5 DB-Tabellen (frameworks, controls, sources, licenses, mappings)
- 10 Seed Controls mit 39 Open-Source-Referenzen
- License Gate: Quellen-Berechtigungspruefung (analysis/excerpt/embeddings/product)
- Too-Close-Detektor: 5 Metriken (exact-phrase, token-overlap, ngram, embedding, LCS)
- REST API: 8 Endpoints unter /v1/canonical/
- Go Loader mit Multi-Index (ID, domain, severity, framework)
- Frontend: Control Library Browser + Provenance Wiki
- CI/CD: validate-controls.py Job (schema, no-leak, open-anchors)
- 67 Tests (8 Go + 59 Python), alle PASS
- MkDocs Dokumentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:55:06 +01:00
Benjamin Admin
e7fab73a3a fix(company-profile): Projekt-aware Persistenz — Daten werden jetzt pro Projekt gespeichert
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 36s
CI / test-python-backend-compliance (push) Successful in 35s
CI / test-python-document-crawler (push) Successful in 27s
CI / test-python-dsms-gateway (push) Successful in 21s
Problem: Company Profile nutzte hartcodiertes tenant_id=default ohne project_id.
Beim Wechsel zwischen Projekten wurden immer die gleichen (oder keine) Daten geladen.

Aenderungen:
- Migration 042: project_id Spalte + UNIQUE(tenant_id, project_id) Constraint,
  fehlende Spalten (offering_urls, Adressfelder) nachgetragen
- Backend: Alle Queries nutzen WHERE tenant_id + project_id IS NOT DISTINCT FROM
- Proxy: project_id Query-Parameter wird durchgereicht
- Frontend: projectId aus SDK-Context, profileApiUrl() Helper fuer alle API-Aufrufe
- "Weiter" speichert jetzt immer den Draft (war schon so, ging aber ins Leere)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 23:48:15 +01:00
Benjamin Admin
11d4c2fd36 feat: Add Compliance Wiki as internal admin knowledge base
Migration 040 with wiki_categories + wiki_articles tables, 10 seed
articles across 8 categories (DSGVO, Art. 9, AVV, HinSchG etc.).
Read-only FastAPI API, Next.js proxy, and two-column frontend with
full-text search.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 20:01:27 +01:00
Benjamin Admin
09cfb79840 feat: Projekt-Verwaltung verbessern — Archivieren, Loeschen, Wiederherstellen
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 35s
CI / test-python-backend-compliance (push) Successful in 38s
CI / test-python-document-crawler (push) Successful in 24s
CI / test-python-dsms-gateway (push) Successful in 24s
- Backend: Restore-Endpoint (POST /projects/{id}/restore) und
  Hard-Delete-Endpoint (DELETE /projects/{id}/permanent) hinzugefuegt
- Frontend: Dreistufiger Dialog (Archivieren / Endgueltig loeschen mit
  Bestaetigungsdialog) statt einfachem Loeschen
- Archivierte Projekte aufklappbar in der Projektliste mit
  Wiederherstellen-Button
- CustomerTypeSelector entfernt (redundant seit Multi-Projekt)
- Default tenantId von 'default' auf UUID geaendert (Backend-400-Fix)
- SQL-Cast :state::jsonb durch CAST(:state AS jsonb) ersetzt (SQLAlchemy-Fix)
- snake_case/camelCase-Mapping fuer Backend-Response (NaN-Datum-Fix)
- projectInfo wird beim Laden vom Backend geholt (Header zeigt Projektname)
- API-Client erzeugt sich on-demand (Race-Condition-Fix fuer Projektliste)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 17:48:02 +01:00
Benjamin Admin
0affa4eb66 feat(sdk): Multi-Projekt-Architektur — mehrere Projekte pro Tenant
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 33s
CI / test-python-backend-compliance (push) Successful in 34s
CI / test-python-document-crawler (push) Successful in 23s
CI / test-python-dsms-gateway (push) Successful in 19s
Jeder Tenant kann jetzt mehrere Compliance-Projekte anlegen (z.B. verschiedene
Produkte, Tochterunternehmen). CompanyProfile ist pro Projekt kopierbar und
danach unabhaengig editierbar. Multi-Tab-Support via separater BroadcastChannel
und localStorage Keys pro Projekt.

- Migration 039: compliance_projects Tabelle, sdk_states.project_id
- Backend: FastAPI CRUD-Routes fuer Projekte mit Tenant-Isolation
- Frontend: ProjectSelector UI, SDKProvider mit projectId, URL ?project=
- State API: UPSERT auf (tenant_id, project_id) mit Abwaertskompatibilitaet
- Tests: pytest fuer Model-Validierung, Row-Konvertierung, Tenant-Isolation
- Docs: MKDocs Seite, CLAUDE.md, Backend README

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:53:50 +01:00
Benjamin Admin
53ff0722a4 fix(backend): SQLAlchemy text() fuer alle raw SQL + UI-Verbesserungen
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 32s
CI / test-python-backend-compliance (push) Successful in 39s
CI / test-python-document-crawler (push) Successful in 23s
CI / test-python-dsms-gateway (push) Successful in 18s
- CRITICAL: Alle db.execute() Aufrufe in company_profile_routes.py
  und generation_routes.py mit text() gewrapped (SQLAlchemy 2.x)
- Geschaeftsmodell-Kacheln: Nur Kurztext, Beschreibung bei Klick
- "Warum diese Fragen" in Hauptbereich unter Ueberschrift verschoben
- Sidebar-Box entfernt fuer mehr Platz

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 22:53:29 +01:00
Benjamin Admin
2abf0b4cac feat(sdk): Company Profile Wizard verbessert
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 32s
CI / test-python-backend-compliance (push) Successful in 34s
CI / test-python-document-crawler (push) Successful in 25s
CI / test-python-dsms-gateway (push) Successful in 19s
- B2B2C als Geschaeftsmodell hinzugefuegt
- URL-Felder bei Offering-Auswahl (Website, Shop, App, SaaS) — optional
- Schritt-spezifische Erklaerungen in "Warum diese Fragen?"
- Firmenname ohne Rechtsform, Templates bauen automatisch zusammen
- Gruendungsjahr springt auf 2000 statt 1800
- SDK-Abdeckung Panel und Profil-loeschen Button entfernt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 22:41:15 +01:00
Benjamin Admin
56758e8b55 fix(mock-data): Fake-Daten bei leerer DB entfernt — ISMS 0%, Dashboard keine simulierten Trends, Compliance-Hub keine Fallback-Zahlen
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 29s
CI / test-python-backend-compliance (push) Successful in 33s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 17s
- ISMS Overview: 14% → 0% bei leerer DB, "not_started" Status, alle Kapitel 0%
- Dashboard: 12-Monate simulierte Trend-Historie entfernt
- Compliance-Hub: Hardcoded Fallback-Statistiken (474/180/95/120/79/44/558/19) → 0
- SQLAlchemy Bug: `is not None` → `.isnot(None)` in SoA-Query
- Hardcoded chapter_7/8_status="pass" → berechnet aus Findings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:25:25 +01:00
Benjamin Admin
95fcba34cd fix(quality): Ruff/CVE/TS-Fixes, 104 neue Tests, Complexity-Refactoring
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Failing after 30s
CI / test-python-backend-compliance (push) Successful in 30s
CI / test-python-document-crawler (push) Successful in 21s
CI / test-python-dsms-gateway (push) Successful in 17s
- Ruff: 144 auto-fixes (unused imports, == None → is None), F821/F811/F841 manuell
- CVEs: python-multipart>=0.0.22, weasyprint>=68.0, pillow>=12.1.1, npm audit fix (0 vulns)
- TS: 5 tote Drafting-Engine-Dateien entfernt, allowed-facts/sanitizer/StepHeader/context fixes
- Tests: +104 (ISMS 58, Evidence 18, VVT 14, Generation 14) → 1449 passed
- Refactoring: collect_ci_evidence (F→A), row_to_response (E→A), extract_requirements (E→A)
- Dead Code: pca-platform, 7 Go-Handler, dsr_api.py, duplicate Schemas entfernt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 19:00:33 +01:00
Benjamin Admin
6509e64dd9 feat(sdk): API-Referenz Frontend + Backend-Konsolidierung (Shared Utilities, CRUD Factory)
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 32s
CI / test-python-backend-compliance (push) Successful in 30s
CI / test-python-document-crawler (push) Successful in 21s
CI / test-python-dsms-gateway (push) Successful in 18s
- API-Referenz Seite (/sdk/api-docs) mit ~690 Endpoints, Suche, Filter, Modul-Index
- Shared db_utils.py (row_to_dict) + tenant_utils Integration in 6 Route-Dateien
- CRUD Factory (crud_factory.py) fuer zukuenftige Module
- Version-Route Auto-Registration in versioning_utils.py
- 1338 Tests bestanden, -232 Zeilen Duplikat-Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 17:07:43 +01:00
Benjamin Admin
7ec6b9f6c0 fix(cleanup): ISMS Bugfix, 13 tote AI-Endpoints entfernt, Compliance-Hub Proxy fix
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 33s
CI / test-python-backend-compliance (push) Successful in 34s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 18s
- ISMS: markStepCompleted entfernt (existiert nicht in SDKContext, verursachte Application Error)
- AI Routes: 13 ungenutzte Endpoints entfernt (ai_routes.py 1266→379 Zeilen, -887)
- Schemas: 12 ungenutzte AI-Schema-Klassen entfernt (-108 Zeilen)
- Compliance-Hub: 5 Fetch-URLs von /api/admin/... auf /api/sdk/v1/... umgestellt
- Tests: 1361 passed, 0 Regressionen

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 15:13:19 +01:00
Benjamin Admin
1e84df9769 feat(sdk): Multi-Tenancy, Versionierung, Change-Requests, Dokumentengenerierung (Phase 1-6)
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 32s
CI / test-python-backend-compliance (push) Successful in 30s
CI / test-python-document-crawler (push) Successful in 21s
CI / test-python-dsms-gateway (push) Successful in 18s
6-Phasen-Implementation fuer cloud-faehiges, mandantenfaehiges Compliance SDK:

Phase 1: Multi-Tenancy Fix
- Shared tenant_utils.py Dependency (UUID-Validierung, kein "default" mehr)
- VVT tenant_id Column + tenant-scoped Queries
- DSFA/Vendor DEFAULT_TENANT_ID von "default" auf UUID migriert
- Migration 035

Phase 2: Stammdaten-Erweiterung
- Company Profile um JSONB-Felder erweitert (processing_systems, ai_systems, technical_contacts)
- Regulierungs-Flags (NIS2, AI Act, ISO 27001)
- GET /template-context Endpoint
- Migration 036

Phase 3: Dokument-Versionierung
- 5 Versions-Tabellen (DSFA, VVT, TOM, Loeschfristen, Obligations)
- Shared versioning_utils.py Helper
- /{id}/versions Endpoints auf allen 5 Dokumenttypen
- Migration 037

Phase 4: Change-Request System
- Zentrale CR-Inbox mit CRUD + Accept/Reject/Edit Workflow
- Regelbasierte CR-Engine (VVT DPIA → DSFA CR, Datenkategorien → Loeschfristen CR)
- Audit-Trail
- Migration 038

Phase 5: Dokumentengenerierung
- 5 Template-Generatoren (DSFA, VVT, TOM, Loeschfristen, Obligations)
- Preview + Apply Endpoints (erzeugt CRs, keine direkten Dokumente)

Phase 6: Frontend-Integration
- Change-Request Inbox Page mit Stats, Filtern, Modals
- VersionHistory Timeline-Komponente
- SDKSidebar CR-Badge (60s Polling)
- Company Profile: 2 neue Wizard-Steps + "Dokumente generieren" CTA

Docs: 5 neue MkDocs-Seiten, CLAUDE.md aktualisiert
Tests: 97 neue Tests (alle bestanden)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 14:12:34 +01:00
Benjamin Admin
3467bce222 feat(obligations): Go PARTIAL DEPRECATED, Python x-user-id, UCCA Proxy Headers, 62 Tests
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 31s
CI / test-python-backend-compliance (push) Successful in 29s
CI / test-python-document-crawler (push) Successful in 19s
CI / test-python-dsms-gateway (push) Successful in 26s
- Go obligations_handlers.go: CRUD-Overlap als deprecated markiert, AI-Features (Assess/Gap/TOM/Export) bleiben aktiv
- Python obligation_routes.py: x-user-id Header + Audit-Logging an 4 Write-Endpoints
- 3 UCCA Proxy-Dateien: Default X-Tenant-ID + X-User-ID Headers
- Tests von 39 auf 62 erweitert (+23 Route-Integration-Tests mit mock_db/TestClient)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:31:14 +01:00
Benjamin Admin
a5e4801b09 fix(escalations): Tenant/User-ID Defaults + Routing-Klarheit
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 32s
CI / test-python-backend-compliance (push) Successful in 30s
CI / test-python-document-crawler (push) Successful in 19s
CI / test-python-dsms-gateway (push) Successful in 16s
- escalations/route.ts: X-Tenant-Id + X-User-Id Default-Header ergaenzt,
  X-User-Id aus Request weitergeleitet
- escalation_routes.py: DEFAULT_TENANT_ID Konstante (9282a473-...) statt 'default'
- test_escalation_routes.py: vollstaendige Test-Suite ergaenzt (+337 Zeilen)
- main.go + escalation_handlers.go: DEPRECATED-Kommentare — UCCA-Escalations
  bleiben fuer Assessment-Review, Haupt-Escalation-System ist Python-Backend

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 21:15:02 +01:00
Benjamin Admin
2dd86e97be feat(incidents): Go Incidents nach Python migrieren, Proxy umleiten, 50 Tests
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 34s
CI / test-python-backend-compliance (push) Successful in 31s
CI / test-python-document-crawler (push) Successful in 21s
CI / test-python-dsms-gateway (push) Successful in 18s
- incident_routes.py: 15 Endpoints (CRUD, Risk Assessment, Art. 33/34 Notifications, Measures, Timeline, Close, Stats)
- Neuer Endpoint PUT /{id}/status (nicht in Go vorhanden, Frontend braucht ihn)
- Proxy von ai-compliance-sdk:8090 auf backend-compliance:8002 umgeleitet
- Go incidents_handlers.go + main.go als DEPRECATED markiert
- 50/50 Tests bestanden

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:50:00 +01:00
Benjamin Admin
8742cb7f5a docs: Qdrant und MinIO/Object-Storage Referenzen aktualisieren
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 35s
CI / test-python-backend-compliance (push) Successful in 32s
CI / test-python-document-crawler (push) Successful in 41s
CI / test-python-dsms-gateway (push) Successful in 19s
- Qdrant: lokaler Container → qdrant-dev.breakpilot.ai (gehostet, API-Key)
- MinIO: bp-core-minio → Hetzner Object Storage (nbg1.your-objectstorage.com)
- CLAUDE.md, MkDocs, ARCHITECTURE.md, training.md, ci-cd-pipeline.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:18:29 +01:00
Benjamin Admin
6a940344c2 feat(dsfa): Go DSFA deprecated, URL-Fix, fehlende Endpoints + 145 Tests
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 34s
CI / test-python-backend-compliance (push) Successful in 30s
CI / test-python-document-crawler (push) Successful in 27s
CI / test-python-dsms-gateway (push) Successful in 18s
- Go: DEPRECATED-Kommentare an allen 6 DSFA-Handlern + Route-Block
- api.ts: URL-Fix /dsgvo/dsfas → /dsfa (Detail-Seite war komplett kaputt)
- Python: Section-Update, Workflow (submit/approve), Export (JSON+CSV), UCCA-Stubs
- Tests: 145/145 bestanden (Schema + Route-Integration mit TestClient+SQLite)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:41:12 +01:00
Benjamin Admin
095eff26d9 feat(dsr): Go DSR deprecated, Python Export-Endpoint, Frontend an Backend-APIs anbinden
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 34s
CI / test-python-backend-compliance (push) Successful in 30s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 17s
- Go: DEPRECATED-Kommentare an allen DSR-Handlern und Routes
- Python: GET /dsr/export?format=csv|json (Semikolon-CSV, 12 Spalten)
- API-Client: 12 neue Funktionen (verify, assign, extend, complete, reject, communications, exception-checks, history)
- Detail-Seite: Alle Actions verdrahtet (keine Coming-soon-Alerts mehr), Communications + Art.17(3)-Checks + Audit-Log live
- Haupt-Seite: CSV-Export-Button im Header
- Tests: 54/54 bestanden (4 neue Export-Tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:21:43 +01:00
Benjamin Admin
3593a4ff78 feat(tom): TOM-Backend in Python erstellen, Frontend von In-Memory auf DB migrieren
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 33s
CI / test-python-backend-compliance (push) Successful in 31s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 15s
- Migration 034: compliance_tom_state + compliance_tom_measures Tabellen
- Python Routes: State CRUD, Measures CRUD, Bulk-Upsert, Stats, CSV/JSON-Export
- Frontend-Proxy: In-Memory Storage durch Proxy zu backend-compliance ersetzt
- Go TOM-Handler als DEPRECATED markiert (Source of Truth ist jetzt Python)
- 44 Tests (alle bestanden)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:35:44 +01:00
Benjamin Admin
4cbfea5c1d feat(vvt): Go-Features nach Python portieren (Source of Truth)
Review-Daten (last_reviewed_at, next_review_at), created_by, DSFA-Link,
CSV-Export mit Semikolon-Trennung, overdue_review_count in Stats.
Go-VVT-Handler als DEPRECATED markiert. 32 Tests bestanden.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 17:14:38 +01:00
Benjamin Admin
adc95267bd chore: LLM qwen3:30b-a3b → qwen3.5:35b-a3b
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 42s
CI / test-python-backend-compliance (push) Successful in 35s
CI / test-python-document-crawler (push) Successful in 20s
CI / test-python-dsms-gateway (push) Successful in 17s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 07:31:55 +01:00
Benjamin Admin
3ed8300daf feat(extraction): POST /compliance/extract-requirements-from-rag
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 34s
CI / test-python-backend-compliance (push) Successful in 31s
CI / test-python-document-crawler (push) Successful in 35s
CI / test-python-dsms-gateway (push) Successful in 17s
Sucht alle RAG-Kollektionen nach Prüfaspekten und legt automatisch
Anforderungen in der DB an. Kernfeatures:

- Durchsucht alle 6 RAG-Kollektionen parallel (bp_compliance_ce,
  bp_compliance_recht, bp_compliance_gesetze, bp_compliance_datenschutz,
  bp_dsfa_corpus, bp_legal_templates)
- Erkennt BSI Prüfaspekte (O.Purp_6) im Artikel-Feld und per Regex
- Dedupliziert nach (regulation_code, article) — safe to call many times
- Auto-erstellt Regulations-Stubs für unbekannte regulation_codes
- dry_run=true zeigt was erstellt würde ohne DB-Schreibzugriff
- Optionale Filter: collections, regulation_codes, search_queries
- 18 Tests (alle bestanden)
- Frontend: "Aus RAG extrahieren" Button auf /sdk/requirements

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 15:11:10 +01:00
Benjamin Admin
ef17151a41 fix(import+screening): GET-Alias, DELETE-Endpoint, ehrlicher Scan-Status
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 37s
CI / test-python-backend-compliance (push) Successful in 32s
CI / test-python-dsms-gateway (push) Has been cancelled
CI / test-python-document-crawler (push) Has been cancelled
Import-Backend:
- GET /v1/import (Root-Alias) → list_documents; behebt URL-Mismatch im Proxy
- DELETE /v1/import/{document_id} → löscht Dokument + Gap-Analyse (mit Tenant-Isolierung)
- 6 neue Tests (65 total, alle grün)

Screening-Frontend:
- Simulierten Fortschrittsbalken (Math.random) entfernt — war inhaltlich falsch
- Ersetzt durch indeterminate Spinner + rotierende ehrliche Status-Texte
  (z.B. "OSV.dev Datenbank wird abgefragt...") im 2-Sek.-Takt
- Kein scanProgress-State mehr benötigt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 12:07:01 +01:00
Benjamin Admin
3913931d5b feat(freigabe): Import/Screening/Modules/RAG — API-Tests, Migration 031, Bug-Fix
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 40s
CI / test-python-backend-compliance (push) Successful in 34s
CI / test-python-document-crawler (push) Successful in 26s
CI / test-python-dsms-gateway (push) Successful in 21s
- import_routes: GET /gap-analysis/{document_id} implementiert
- import_routes: Bug-Fix — gap_analysis_result vor try-Block initialisiert
  (verhindert UnboundLocalError bei DB-Fehler)
- test_import_routes: 21 neue API-Endpoint-Tests (59 total, alle grün)
- test_screening_routes: 18 neue API-Endpoint-Tests (74 total, alle grün)
- 031_modules.sql: Migration für compliance_service_modules,
  compliance_module_regulations, compliance_module_risks
- test_module_routes: 20 neue Tests für Module-Registry-Routen (alle grün)
- freigabe-module.md: MkDocs-Seite für Import/Screening/Modules/RAG
- mkdocs.yml: Nav-Eintrag "Freigabe-Module (Paket 2)"

Gesamt: 146 neue Tests, alle bestanden

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 11:42:19 +01:00
Benjamin Admin
0503e72a80 fix(freigabe): Vorbereitung-Module release prep — Python 3.9 fix, Scope Engine tests, MkDocs
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 36s
CI / test-python-backend-compliance (push) Successful in 33s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 20s
- fix: company_profile_routes.py — dict|None → Optional[dict] for Python 3.9 compat (9/9 tests grün)
- test: 40 Vitest-Tests für ComplianceScopeEngine (calculateScores, determineLevel,
  evaluateHardTriggers, evaluate integration, buildDocumentScope, evaluateRiskFlags)
- docs: vorbereitung-module.md — Profil, Scope, UCCA vollständig dokumentiert
- docs: mkdocs.yml — Nav-Eintrag "Vorbereitung-Module (Paket 1)" vor Analyse-Module

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 10:57:17 +01:00
Benjamin Admin
789c215e5e feat: DSFA vollständiges DB-Schema + PDF-Ingest + Tests
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 36s
CI / test-python-backend-compliance (push) Successful in 37s
CI / test-python-document-crawler (push) Successful in 23s
CI / test-python-dsms-gateway (push) Successful in 22s
- Migration 030: alle fehlenden Spalten für compliance_dsfas (Sections 0-7)
  flat fields: processing_description, legal_basis, dpo_*, authority_*, ...
  JSONB arrays: risks, mitigations, wp248_criteria_met, ai_trigger_ids, ...
  JSONB objects: section_progress, threshold_analysis, review_schedule, metadata
- dsfa_routes.py: DSFACreate/DSFAUpdate erweitert (60+ neue Optional-Felder)
  _dsfa_to_response: alle neuen Felder mit safe _get() Helper
  PUT-Handler: vollständige JSONB_FIELDS-Liste (22 Felder)
- Tests: 101 (+49) Tests — TestAIUseCaseModules + TestDSFAFullSchema
- ingest-dsfa-bundesland.sh: KNOWN_PDF_URLS (15 direkte URLs), download_pdfs()
  find_pdf_for_state() Helper, PDF-first mit Text-Fallback in ingest_all()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 10:03:09 +01:00
Benjamin Admin
b7c1a5da1a feat: Consent-Service Module nach Compliance migriert (DSR, E-Mail-Templates, Legal Docs, Banner)
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 36s
CI / test-python-backend-compliance (push) Successful in 31s
CI / test-python-document-crawler (push) Successful in 23s
CI / test-python-dsms-gateway (push) Successful in 18s
5-Phasen-Migration: Go consent-service Proxies durch native Python/FastAPI ersetzt.

Phase 1 — DSR (Betroffenenrechte): 6 Tabellen, 30 Endpoints, Frontend-API umgestellt
Phase 2 — E-Mail-Templates: 5 Tabellen, 20 Endpoints, neues Frontend, SDK_STEPS erweitert
Phase 3 — Legal Documents Extension: User Consents, Audit Log, Cookie-Kategorien
Phase 4 — Banner Consent: Device-Consents, Site-Configs, Kategorien, Vendors
Phase 5 — Cleanup: DSR-Proxy aus main.py entfernt, Frontend-URLs aktualisiert

148 neue Tests (50 + 47 + 26 + 25), alle bestanden.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 00:36:24 +01:00
Benjamin Admin
05aa0ee2c6 fix: DSFA Router-Prefix und Proxy-Pfad korrigiert
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 35s
CI / test-python-backend-compliance (push) Successful in 32s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 21s
Router-Prefix /v1/dsfa → /dsfa (konsistent mit allen anderen Routes)
Proxy-Pfad /api/v1/dsfa → /api/compliance/dsfa

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 23:22:17 +01:00