d2c94619d8d596fd56331a2f3a3d04851652aad5
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Sharang Parnerkar
|
cc1c61947d |
refactor(backend/api): extract Incident services (Step 4 — file 11 of 18)
compliance/api/incident_routes.py (916 LOC) -> 280 LOC thin routes +
two services + 95-line schemas file.
Two-service split for DSGVO Art. 33/34 Datenpannen-Management:
incident_service.py (460 LOC):
- CRUD (create, list, get, update, delete)
- Stats, status update, timeline append, close
- Module-level helpers: _calculate_risk_level, _is_notification_required,
_calculate_72h_deadline, _incident_to_response, _measure_to_response,
_parse_jsonb, _append_timeline, DEFAULT_TENANT_ID
incident_workflow_service.py (329 LOC):
- Risk assessment (likelihood x impact -> risk_level)
- Art. 33 authority notification (with 72h deadline tracking)
- Art. 34 data subject notification
- Corrective measures CRUD
Both services use raw SQL via sqlalchemy.text() — no ORM models for
incident_incidents / incident_measures tables. Migrated from the Go
ai-compliance-sdk; Python backend is Source of Truth.
Legacy test compat: tests/test_incident_routes.py imports
_calculate_risk_level, _is_notification_required, _calculate_72h_deadline,
_incident_to_response, _measure_to_response, _parse_jsonb,
DEFAULT_TENANT_ID directly from compliance.api.incident_routes — all
re-exported via __all__.
Verified:
- 223/223 pytest pass (173 core + 50 incident)
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 141 source files
- incident_routes.py 916 -> 280 LOC
- Hard-cap violations: 8 -> 7
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Sharang Parnerkar
|
0c2e03f294 |
refactor(backend/api): extract Email Template services (Step 4 — file 10 of 18)
compliance/api/email_template_routes.py (823 LOC) -> 295 LOC thin routes
+ 402-line EmailTemplateService + 241-line EmailTemplateVersionService +
61-line schemas file.
Two-service split along natural responsibility seam:
email_template_service.py (402 LOC):
- Template type catalog (TEMPLATE_TYPES constant)
- Template CRUD (list, create, get)
- Stats, settings, send logs, initialization, default content
- Shared _template_to_dict / _version_to_dict / _render_template helpers
email_template_version_service.py (241 LOC):
- Version CRUD (create, list, get, update)
- Workflow transitions (submit, approve, reject, publish)
- Preview and test-send
TEMPLATE_TYPES, VALID_CATEGORIES, VALID_STATUSES re-exported from the
route module for any legacy consumers.
State-transition errors use ValidationError (-> HTTPException 400) to
preserve the original handler's 400 status for "Only draft/review
versions can be ..." checks, since the existing TestClient integration
tests (47 tests) assert status_code == 400.
Verified:
- 47/47 tests/test_email_template_routes.py pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 138 source files
- email_template_routes.py 823 -> 295 LOC
- Hard-cap violations: 9 -> 8
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Sharang Parnerkar
|
a638d0e527 |
refactor(backend/api): extract EvidenceService (Step 4 — file 9 of 18)
compliance/api/evidence_routes.py (641 LOC) -> 240 LOC thin routes + 460-line
EvidenceService. Manages evidence CRUD, file upload, CI/CD evidence
collection (SAST/dependency/SBOM/container scans), and CI status dashboard.
Service injection pattern: EvidenceService takes the EvidenceRepository,
ControlRepository, and AutoRiskUpdater classes as constructor parameters.
The route's get_evidence_service factory reads these class references from
its own module namespace so tests that
``patch("compliance.api.evidence_routes.EvidenceRepository", ...)`` still
take effect through the factory.
The `_store_evidence` and `_update_risks` helpers stay as module-level
callables in evidence_service and are re-exported from the route module.
The collect_ci_evidence handler remains inline (not delegated to a service
method) so tests can patch
`compliance.api.evidence_routes._store_evidence` and have the patch take
effect at the handler's call site.
Legacy re-exports via __all__: SOURCE_CONTROL_MAP, EvidenceRepository,
ControlRepository, AutoRiskUpdater, _parse_ci_evidence,
_extract_findings_detail, _store_evidence, _update_risks.
Verified:
- 208/208 pytest (core + 35 evidence tests) pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 135 source files
- evidence_routes.py 641 -> 240 LOC
- Hard-cap violations: 10 -> 9
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Sharang Parnerkar
|
7107a31496 |
refactor(backend/api): extract SourcePolicyService (Step 4 — file 7 of 18)
compliance/api/source_policy_router.py (580 LOC) -> 253 LOC thin routes + 453-line SourcePolicyService + 83-line schemas file. Manages allowed data sources, operations matrix, PII rules, blocked-content log, audit trail, and dashboard stats/report. Single-service split. ORM-based (uses compliance.db.source_policy_models). Date-string parsing extracted to a module-level _parse_iso_optional helper so the audit + blocked-content list endpoints share it instead of duplicating try/except blocks. Legacy test compat: SourceCreate, SourceUpdate, SourceResponse, PIIRuleCreate, PIIRuleUpdate, OperationUpdate, _log_audit re-exported from compliance.api.source_policy_router via __all__. Verified: - 208/208 pytest pass (173 core + 35 source policy) - OpenAPI 360/484 unchanged - mypy compliance/ -> Success on 132 source files - source_policy_router.py 580 -> 253 LOC - Hard-cap violations: 12 -> 11 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Sharang Parnerkar
|
b850368ec9 |
refactor(backend/api): extract CanonicalControlService (Step 4 — file 6 of 18)
compliance/api/canonical_control_routes.py (514 LOC) -> 192 LOC thin routes + 316-line CanonicalControlService + 105-line schemas file. Canonical Control Library manages OWASP/NIST/ENISA-anchored security control frameworks and controls. Like company_profile_routes, this file uses raw SQL via sqlalchemy.text() because there are no SQLAlchemy models for canonical_control_frameworks or canonical_controls. Single-service split. Session management moved from bespoke `with SessionLocal() as db:` blocks to Depends(get_db) for consistency. Legacy test imports preserved via re-export (FrameworkResponse, ControlResponse, SimilarityCheckRequest, SimilarityCheckResponse, _control_row). Validation extracted to a module-level `_validate_control_input` helper so both create and update share the same checks. ValidationError (from compliance.domain) replaces raw HTTPException(400) raises. Verified: - 187/187 pytest (173 core + 14 canonical) pass - OpenAPI 360/484 unchanged - mypy compliance/ -> Success on 130 source files - canonical_control_routes.py 514 -> 192 LOC - Hard-cap violations: 13 -> 12 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Sharang Parnerkar
|
4fa0dd6f6d |
refactor(backend/api): extract VVTService (Step 4 — file 5 of 18)
compliance/api/vvt_routes.py (550 LOC) -> 225 LOC thin routes + 475-line VVTService. Covers the organization header, processing activities CRUD, audit log, JSON/CSV export, stats, and version lookups for the Art. 30 DSGVO Verzeichnis. Single-service split: organization + activities + audit + stats all revolve around the same tenant's VVT document, and the existing test suite (tests/test_vvt_routes.py — 768 LOC, tests/test_vvt_tenant_isolation.py — 205 LOC) exercises them together. Module-level helpers (_activity_to_response, _log_audit, _export_csv) stay module-level in compliance.services.vvt_service and are re-exported from compliance.api.vvt_routes so the two test files keep importing from the old path. Pydantic schemas already live in compliance.schemas.vvt from Step 3 — no new schema file needed this round. mypy.ini flips compliance.api.vvt_routes from ignore_errors=True to False. Two SQLAlchemy Column[str] vs str dict-index errors fixed with explicit str() casts on status/business_function in the stats loop. Verified: - 242/242 pytest (173 core + 69 VVT integration) pass - OpenAPI 360/484 unchanged - mypy compliance/ -> Success on 128 source files - vvt_routes.py 550 -> 225 LOC - vvt_service.py 475 LOC (under 500 hard cap) - Hard-cap violations: 14 -> 13 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Sharang Parnerkar
|
f39c7ca40c |
refactor(backend/api): extract CompanyProfileService (Step 4 — file 4 of 18)
compliance/api/company_profile_routes.py (640 LOC) -> 154 LOC thin routes.
Unusual for this repo: persistence uses raw SQL via sqlalchemy.text()
because the underlying compliance_company_profiles table has ~45 columns
with complex jsonb coercion and there is no SQLAlchemy model for it.
New files:
compliance/schemas/company_profile.py (127) — 4 request/response models
compliance/services/company_profile_service.py (340) — Service class + row_to_response + log_audit
compliance/services/_company_profile_sql.py (139) — 70-line INSERT/UPDATE statements
separated for readability
Minor behavioral improvement: the handlers now use Depends(get_db) for
session management instead of the bespoke `db = SessionLocal(); try: ...
finally: db.close()` pattern. This makes the routes consistent with
every other refactored service, fixes the broken-ness under test
dependency_overrides, and removes 6 duplicate try/finally blocks.
Legacy exports preserved: CompanyProfileRequest, CompanyProfileResponse,
AuditEntryResponse, AuditListResponse, row_to_response, and log_audit are
re-exported from compliance.api.company_profile_routes so that the two
existing test files
(tests/test_company_profile_routes.py, tests/test_company_profile_extend.py)
keep importing from the same path.
Pre-existing broken tests noted: 6 tests in those files feed a 40-tuple
row into row_to_response, but _BASE_COLUMNS_LIST has 46 columns (has had
since the Phase 2 Stammdaten extension). These tests fail on main too
(verified via `git stash` round-trip). Not fixed in this commit — they
require a rewrite of the test's _make_row helper, which is out of scope
for a pure structural refactor. Flagged for follow-up.
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360/484 unchanged
- mypy compliance/ -> Success on 127 source files
- company_profile_routes.py 640 -> 154 LOC
- All new files under soft 300 target except service (340, under hard 500)
- Hard-cap violations: 15 -> 14
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Sharang Parnerkar
|
d571412657 |
refactor(backend/api): extract TOMService (Step 4 — file 3 of 18)
compliance/api/tom_routes.py (609 LOC) -> 215 LOC thin routes + 434-line TOMService. Request bodies (TOMStateBody, TOMMeasureCreate, TOMMeasureUpdate, TOMMeasureBulkItem, TOMMeasureBulkBody) moved to compliance/schemas/tom.py (joining the existing response models from the Step 3 split). Single-service split (not two like banner): state, measures CRUD + bulk upsert, stats, export, and version lookups are all tightly coupled around the TOMMeasureDB aggregate, so splitting would create artificial boundaries. TOMService is 434 LOC — comfortably under the 500 hard cap. Domain error mapping: - ConflictError -> 409 (version conflict on state save; duplicate control_id on create) - NotFoundError -> 404 (missing measure on update; missing version) - ValidationError -> 400 (missing tenant_id on DELETE /state) Legacy test compat: the existing tests/test_tom_routes.py imports TOMMeasureBulkItem, _parse_dt, _measure_to_dict, and DEFAULT_TENANT_ID directly from compliance.api.tom_routes. All re-exported via __all__ so the 44-test file runs unchanged. mypy.ini flips compliance.api.tom_routes from ignore_errors=True to False. TOMService carries the scoped Column[T] header. Verified: - 217/217 pytest (173 baseline + 44 TOM) pass - OpenAPI 360/484 unchanged - mypy compliance/ -> Success on 124 source files - tom_routes.py 609 -> 215 LOC - Hard-cap violations: 16 -> 15 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Sharang Parnerkar
|
10073f3ef0 |
refactor(backend/api): extract BannerConsent + BannerAdmin services (Step 4)
Phase 1 Step 4, file 2 of 18. Same cookbook as audit_routes ( |
||
|
Sharang Parnerkar
|
4a91814bfc |
refactor(backend/api): extract AuditSession service layer (Step 4 worked example)
Phase 1 Step 4 of PHASE1_RUNBOOK.md, first worked example. Demonstrates
the router -> service delegation pattern for all 18 oversized route
files still above the 500 LOC hard cap.
compliance/api/audit_routes.py (637 LOC) is decomposed into:
compliance/api/audit_routes.py (198) — thin handlers
compliance/services/audit_session_service.py (259) — session lifecycle
compliance/services/audit_signoff_service.py (319) — checklist + sign-off
compliance/api/_http_errors.py ( 43) — reusable error translator
Handlers shrink to 3-6 lines each:
@router.post("/sessions", response_model=AuditSessionResponse)
async def create_audit_session(
request: CreateAuditSessionRequest,
service: AuditSessionService = Depends(get_audit_session_service),
):
with translate_domain_errors():
return service.create(request)
Services are HTTP-agnostic: they raise NotFoundError / ConflictError /
ValidationError from compliance.domain, and the route layer translates
those to HTTPException(404/409/400) via the translate_domain_errors()
context manager in compliance.api._http_errors. The error translator is
reusable by every future Step 4 refactor.
Services take a sqlalchemy Session in the constructor and are wired via
Depends factories (get_audit_session_service / get_audit_signoff_service).
No globals, no module-level state.
Behavior is byte-identical at the HTTP boundary:
- Same paths, methods, status codes, response models
- Same error messages (domain error __str__ preserved)
- Same auto-start-on-first-signoff, same statistics calculation,
same signature hash format, same PDF streaming response
Verified:
- 173/173 pytest compliance/tests/ tests/contracts/ pass
- OpenAPI 360 paths / 484 operations unchanged
- audit_routes.py under soft 300 target
- Both new service files under soft 300 / hard 500
Note: compliance/tests/test_audit_routes.py contains placeholder tests
that do not actually import or call the handler functions — they only
assert on request-data shape. Real behavioral coverage relies on the
contract test. A follow-up commit should add TestClient-based
integration tests for the audit endpoints. Flagged in PHASE1_RUNBOOK.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Sharang Parnerkar
|
3320ef94fc |
refactor: phase 0 guardrails + phase 1 step 2 (models.py split)
Squash of branch refactor/phase0-guardrails-and-models-split — 4 commits,
81 files, 173/173 pytest green, OpenAPI contract preserved (360 paths /
484 operations).
## Phase 0 — Architecture guardrails
Three defense-in-depth layers to keep the architecture rules enforced
regardless of who opens Claude Code in this repo:
1. .claude/settings.json PreToolUse hook on Write/Edit blocks any file
that would exceed the 500-line hard cap. Auto-loads in every Claude
session in this repo.
2. scripts/githooks/pre-commit (install via scripts/install-hooks.sh)
enforces the LOC cap locally, freezes migrations/ without
[migration-approved], and protects guardrail files without
[guardrail-change].
3. .gitea/workflows/ci.yaml gains loc-budget + guardrail-integrity +
sbom-scan (syft+grype) jobs, adds mypy --strict for the new Python
packages (compliance/{services,repositories,domain,schemas}), and
tsc --noEmit for admin-compliance + developer-portal.
Per-language conventions documented in AGENTS.python.md, AGENTS.go.md,
AGENTS.typescript.md at the repo root — layering, tooling, and explicit
"what you may NOT do" lists. Root CLAUDE.md is prepended with the six
non-negotiable rules. Each of the 10 services gets a README.md.
scripts/check-loc.sh enforces soft 300 / hard 500 and surfaces the
current baseline of 205 hard + 161 soft violations so Phases 1-4 can
drain it incrementally. CI gates only CHANGED files in PRs so the
legacy baseline does not block unrelated work.
## Deprecation sweep
47 files. Pydantic V1 regex= -> pattern= (2 sites), class Config ->
ConfigDict in source_policy_router.py (schemas.py intentionally skipped;
it is the Phase 1 Step 3 split target). datetime.utcnow() ->
datetime.now(timezone.utc) everywhere including SQLAlchemy default=
callables. All DB columns already declare timezone=True, so this is a
latent-bug fix at the Python side, not a schema change.
DeprecationWarning count dropped from 158 to 35.
## Phase 1 Step 1 — Contract test harness
tests/contracts/test_openapi_baseline.py diffs the live FastAPI /openapi.json
against tests/contracts/openapi.baseline.json on every test run. Fails on
removed paths, removed status codes, or new required request body fields.
Regenerate only via tests/contracts/regenerate_baseline.py after a
consumer-updated contract change. This is the safety harness for all
subsequent refactor commits.
## Phase 1 Step 2 — models.py split (1466 -> 85 LOC shim)
compliance/db/models.py is decomposed into seven sibling aggregate modules
following the existing repo pattern (dsr_models.py, vvt_models.py, ...):
regulation_models.py (134) — Regulation, Requirement
control_models.py (279) — Control, Mapping, Evidence, Risk
ai_system_models.py (141) — AISystem, AuditExport
service_module_models.py (176) — ServiceModule, ModuleRegulation, ModuleRisk
audit_session_models.py (177) — AuditSession, AuditSignOff
isms_governance_models.py (323) — ISMSScope, Context, Policy, Objective, SoA
isms_audit_models.py (468) — Finding, CAPA, MgmtReview, InternalAudit,
AuditTrail, Readiness
models.py becomes an 85-line re-export shim in dependency order so
existing imports continue to work unchanged. Schema is byte-identical:
__tablename__, column definitions, relationship strings, back_populates,
cascade directives all preserved.
All new sibling files are under the 500-line hard cap; largest is
isms_audit_models.py at 468. No file in compliance/db/ now exceeds
the hard cap.
## Phase 1 Step 3 — infrastructure only
backend-compliance/compliance/{schemas,domain,repositories}/ packages
are created as landing zones with docstrings. compliance/domain/
exports DomainError / NotFoundError / ConflictError / ValidationError /
PermissionError — the base classes services will use to raise
domain-level errors instead of HTTPException.
PHASE1_RUNBOOK.md at backend-compliance/PHASE1_RUNBOOK.md documents
the nine-step execution plan for Phase 1: snapshot baseline,
characterization tests, split models.py (this commit), split schemas.py
(next), extract services, extract repositories, mypy --strict, coverage.
## Verification
backend-compliance/.venv-phase1: uv python install 3.12 + pip -r requirements.txt
PYTHONPATH=. pytest compliance/tests/ tests/contracts/
-> 173 passed, 0 failed, 35 warnings, OpenAPI 360/484 unchanged
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|