36c6101b91
Merge feat/zeroclaw-compliance-agent into main
...
Brings all compliance doc-check features:
- 162 regex checks + 1874 Master Controls
- LLM-agnostic agent with tool calling
- Banner check (46 checks, 30 CMPs, stealth, Shadow DOM)
- Impressum check (24 checks)
- Deep consent verification (DataLayer, GCM, TCF)
- CMP E2E tests (39 tests)
- HTML email reports, FAQ, persistent history
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-11 11:44:20 +02:00
289ec5f396
feat(cmp): vendor-agnostic consent data model — 13 new fields
...
Build + Deploy / build-admin-compliance (push) Successful in 2m28s
Build + Deploy / build-backend-compliance (push) Successful in 3m48s
Build + Deploy / build-ai-sdk (push) Failing after 45s
Build + Deploy / build-developer-portal (push) Successful in 1m28s
Build + Deploy / build-tts (push) Successful in 1m48s
Build + Deploy / build-document-crawler (push) Successful in 48s
Build + Deploy / build-dsms-gateway (push) Successful in 34s
Build + Deploy / build-dsms-node (push) Successful in 20s
CI / branch-name (push) Has been skipped
Build + Deploy / trigger-orca (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / loc-budget (push) Failing after 24s
CI / secret-scan (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / nodejs-build (push) Successful in 3m1s
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / test-go (push) Failing after 49s
CI / test-python-backend (push) Successful in 45s
CI / test-python-document-crawler (push) Successful in 31s
CI / test-python-dsms-gateway (push) Successful in 27s
CI / validate-canonical-controls (push) Successful in 18s
Extend banner consent records with consent_method, banner_version,
banner_config_hash, geo, page_url, referrer, device info, session_id
and consent_scope for full Art. 7 DSGVO proof with any tracking vendor.
Migration 107, backward-compatible (all fields nullable).
Admin detail modal shows tracking context, device info and technical data.
Fix pre-existing str|None → Optional[str] for Python 3.9 compat.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-10 23:12:20 +02:00
c89a68e59e
feat: Whistleblower backend + Scanner banner-check (last 2 gaps)
...
Whistleblower (HinSchG):
- Migration 118: 3 tables (reports, messages, measures) with
HinSchG deadlines (7d acknowledgment, 3mo feedback)
- whistleblower_routes.py: 14 endpoints (CRUD, acknowledge, close,
messages, measures, public submit, anonymous status check)
- Frontend api-operations.ts rewired from Go SDK to compliance proxy
- Access key format XXXX-XXXX-XXXX for anonymous reporters
Scanner banner-check (TTDSG § 25):
- CMP Dashboard: green "Kein Cookie-Banner erforderlich" when no
trackers detected + no banner configured
- Red warning "Cookie-Banner fehlt!" when trackers found but no banner
- Mandatory note: Impressum (DDG § 5) + DSE (DSGVO Art. 13) still required
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-04 00:22:18 +02:00
eb4ea8bc42
feat: EmailDeliveryService + professional DSR email templates
...
- EmailDeliveryService: load template → find published version →
render {{variables}} → send via SMTP → audit log. Fallback to
inline HTML when no published template exists.
- Migration 117: Professional HTML/text content for all 5 DSR
templates (receipt, completion, rejection, identity, extension)
with branded styling and proper Art. references
- DSRArt11Service now uses EmailDeliveryService with dsr_rejection
template instead of hardcoded HTML
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 23:38:32 +02:00
630fffc0cc
feat: Academy integration — training gap detection after document approval (F7)
...
- Migration 115: compliance_role_training_mapping table (org roles → training codes)
- TrainingLinkService: queries training_modules/matrix/assignments to find gaps
per person and role. Gracefully degrades when Go training tables don't exist yet.
- document_review_routes: 2 new endpoints (training-requirements, training-gaps)
- _notify_approval() now checks training gaps and sends emails to persons
with outstanding modules, linking to /sdk/training/learner
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 22:03:25 +02:00
965af3a34c
feat: A/B Testing + Compliance Report PDF (F5 + F8)
...
F5: A/B Testing for Consent Rate
- Migration 116: banner_variants table + variant tracking in audit log
- BannerABService: deterministic sticky bucketing via device hash,
chi-squared significance testing, variant CRUD
- banner_ab_routes: 6 endpoints (CRUD + stats + assign)
- ABTestPanel.tsx: variant creation, traffic sliders, opt-in comparison
chart with winner/significance badges
- New "A/B-Test" tab in cookie-banner page
F8: Compliance Report PDF
- CompliancePDFGenerator: reportlab-based A4 PDF covering all modules
(Company Profile, TOM, VVT, DSFA, Risks, Vendors, Incidents,
Reviews, Consents, Roles)
- compliance_report_routes: GET /compliance/report/pdf
- "Compliance-Report herunterladen" button on SDK dashboard
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 21:42:50 +02:00
c3fcfe88ee
feat: Vendor-level consent + Consent analytics (F4 + F6)
...
F4: Granular Vendor-Level Consent
- Migration 113: vendor_consents JSONB on banner_consents + audit_log
- ConsentCreate schema + BannerConsentDB model extended
- banner_consent_service stores vendor_consents alongside categories
- Audit trail includes vendor-level decisions + user_agent
F6: Consent Rate Analytics
- Migration 114: user_agent on audit_log + time-series index
- BannerAnalyticsService: time series, category breakdown, device stats
- banner_analytics_routes: 4 endpoints (overview, time-series, categories, devices)
- AnalyticsDashboard.tsx: KPIs, bar chart, category bars, device breakdown
- New "Analytik" tab in cookie-banner page
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 20:58:06 +02:00
9b4be663f7
feat: Rollenkonzept backend + SOP template (Phase 1-3)
...
- Migration 111: 3 new tables (org_roles, document_reviews, document_role_mapping)
with seed data mapping all 71 doc types to 7 compliance roles
- org_role_routes.py: CRUD for roles, seed defaults, test email, mapping API
- document_review_routes.py: Review lifecycle (create→send→approve/reject)
with approval notification to all affected roles
- Migration 112: SOP template (ISO 9001 structure, 21 placeholders)
- Added standard_operating_procedure to TemplateType, doc-labels, presets
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 13:03:38 +02:00
64700b355e
feat: Review all 12 remaining policy templates + categorize
...
Migration 110: Updated descriptions and version for 12 previously
unreviewed templates (asset_management, backup, change_management,
cloud_security, devsecops, incident_response, logging, patch_management,
secrets_management, vulnerability_management, informationspflichten,
verpflichtungserklaerung).
All templates assessed as "Very Good" quality — only incremental
updates needed (AI Act, CRA, NIS2UmsuCG references in descriptions).
informationspflichten: Kept as separate compact checklist (distinct
from the full privacy_policy DSI template).
verpflichtungserklaerung: Kept as standalone HR document (employee
signs at onboarding). Added to HR & Mitarbeiter category.
Result: 88 templates, 44 at v1.1+, 0 unreviewed remaining.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 07:19:41 +02:00
4b9cf34243
feat: Full template cleanup + categories by use case
...
Cleanup (109):
- Removed DPA duplicates (v1 DE + v1 EN, kept v2 DE)
- Removed cookie_banner duplicate (kept larger with IF-blocks)
- Removed impressum duplicate (kept larger with IF-blocks)
- Removed TOM duplicate (kept newest)
- Removed DSFA v1 (kept v2)
- Kept all 8 VVT templates (1 main + 7 industry templates)
- DB: 98 → 88 templates, 0 duplicates remaining
Categories restructured by use case:
- Website/App: DSI, Impressum, Cookie, Social Media
- Online-Shop: AGB, Widerruf, DSI, Cookie
- SaaS/Cloud: AGB, AVV, SLA, Cloud Agreement
- App/Plattform: Nutzungsbedingungen, Community Guidelines, AUP
- Vertraege (B2B): AVV, NDA, SLA, Cloud
- DSGVO-Pflichten: TOM, VVT, Loeschkonzept, DSFA
- Sicherheitskonzepte + Richtlinien (separate categories)
- HR & Mitarbeiter, Daten-Governance, Vendor, BCM
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 07:09:16 +02:00
5298467275
feat: Privacy notice cleanup + English v2
...
- 108: Remove DSI duplicate (023 + 093 both wrote privacy_policy DE),
remove outdated EN v1, create English Privacy Notice v2 with all
modular sections (data categories table, retention periods, processor
vs. controller guidance, Art. 21 right to object highlighted)
DB now has exactly 2 privacy_policy templates: DE + EN, both v2.0.0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 07:03:06 +02:00
91b4034fee
feat: AGB cleanup + English Terms v2
...
- 106: Remove AGB duplicates and obsolete templates (terms_of_service
DE/EN v1.0, liability clause) — replaced by agb v2.0
- 107: English Terms and Conditions v2 (EU-compliant, same structure
as DE version with all IF-blocks)
DB now has exactly 2 AGB templates: DE + EN, both v2.0.0
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-03 06:59:28 +02:00
a1f5d883cc
feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)
...
Phase 1: Vendor sync from service registry (82+ services → banner vendors)
Phase 2: Category-based retention (marketing=90d, statistics=790d, not hardcoded 365d)
Phase 3: DSR ↔ Banner email linking (link-email, by-email, Art.17 erasure, Art.15/20 export)
Phase 4: Consent sync (Banner → Einwilligungen bridge)
Phase 6: Consent proof (SHA256 config hash + config_version in audit log, Art. 7(1) DSGVO)
New files:
- banner_dsr_service.py — email linking + DSR integration
- vendor_banner_sync.py — service registry → vendor configs
- migration 106 — linked_email, banner_config_hash, consent_version columns
Tests: 20+ new backend tests + 2 Playwright E2E test suites (API + UI)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-02 19:52:04 +02:00
17c67b4f25
feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)
...
Phase 1: Vendor sync from service registry (82+ services → banner vendors)
Phase 2: Category-based retention (marketing=90d, statistics=790d, not hardcoded 365d)
Phase 3: DSR ↔ Banner email linking (link-email, by-email, Art.17 erasure, Art.15/20 export)
Phase 4: Consent sync (Banner → Einwilligungen bridge)
Phase 6: Consent proof (SHA256 config hash + config_version in audit log, Art. 7(1) DSGVO)
New files:
- banner_dsr_service.py — email linking + DSR integration
- vendor_banner_sync.py — service registry → vendor configs
- migration 106 — linked_email, banner_config_hash, consent_version columns
Tests: 20+ new backend tests + 2 Playwright E2E test suites (API + UI)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-02 19:41:22 +02:00
d942b21354
feat: SCC + TIA templates for third-country transfers
...
New templates for the Vendor Compliance module:
- 105: Transfer Impact Assessment (TIA) — Schrems II risk assessment
with country evaluation, government access assessment, supplementary
measures, risk matrix, and go/conditional/deny decision
- 105: SCC Companion Document — annexes to EU Decision 2021/914
(module selection C2C/C2P/P2P/P2C, party details, data description,
TOMs, sub-processor list)
Template recommendations: SCC+TIA triggered by tech_third_country answer
Generator: New "Drittlandtransfer" category
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-01 10:19:56 +02:00
3984f39329
feat: Phase 5 — Special templates (AI policy, BYOD, ISMS, consent, video DSI)
...
Phase 5 of the Document Templates Masterplan:
- 104: 5 new special templates:
- ai_usage_policy: AI usage policy (AI Act Art. 4 training obligation,
forbidden inputs, quality check, labeling, TDM opt-out)
- byod_policy: Bring Your Own Device (container solution, remote wipe,
DSFA, cost sharing options)
- consent_texts: Double-Opt-In texts, newsletter, marketing, tracking,
profiling consent, unsubscribe confirmation
- video_conference_dsi: Video conference privacy notice (Zoom/Teams/Meet,
recording consent, third-country transfer)
- isms_manual: ISMS handbook (ISO 27001, document structure map to all
other templates, PDCA cycle, management review)
Generator: 6 new categories (AI governance, ISMS, consent, special DSI,
internal policies)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-01 09:25:32 +02:00
4417938558
feat: Phase 3 — Security + HR/Vendor/BCM policies
...
Phase 3 of the Document Templates Masterplan:
- 103: 4 new security policies (information_security_policy, password_policy,
encryption_policy, access_control_policy) + updates for CRA (056) and
all 15 HR/Vendor/BCM policies (072)
New templates:
- Information Security Policy: ISMS-Leitlinie (ISO 27001, BSI, NIS2)
- Password Policy: BSI/NIST compliant (12+ chars, MFA, no forced rotation)
- Encryption Policy: BSI TR-02102, algorithms, key management, TLS config
- Access Control Policy: RBAC, Least Privilege, Zero Trust, rezertification
Updates: AI Act + NIS2UmsuCG references for CRA and all 15 HR/Vendor/BCM
Generator: 6 new categories (security, HR, data, vendor, BCM policies)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-01 09:05:03 +02:00
90c7f02b40
feat: Phase 2 — Security Concepts + DSFA + DSR updates
...
Phase 2 of the Document Templates Masterplan:
- 101: Security Concepts v2 (7 templates) — NIS2UmsuCG references,
BSI Grundschutz++ modernization, AI Act cross-references,
Zero Trust principle, ransomware-protected backups, NIS2 logging
- 102: DSFA + Pflichtenregister + DSR v2 — AI Act Art. 9 for DSFA,
NIS2UmsuCG for Pflichtenregister, tenant_id fix for DSR processes
All 16 templates reviewed — already at good product level, only
incremental updates needed (standards references, cross-doc links).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-01 08:45:04 +02:00
f591871277
feat: Phase 1 — Whistleblower + Cookie/Impressum + HR-DSI templates
...
Phase 1 of the Document Templates Masterplan:
- 098: Whistleblower-Richtlinie (HinSchG) — 10 sections, anonymous
reporting, 7-day confirmation, 3-month feedback, reprisal protection
- 099: Cookie-Banner + Impressum updates — OS-Plattform discontinued
note (July 2025), description updates
- 100: Applicant DSI + Employee DSI — two new HR privacy notices with
§ 26 BDSG, 6-month retention (applicants), modular blocks for video
interviews, talent pool, IT monitoring, company vehicles, works council
Generator: 25 new fields (whistleblower, applicant, employee categories)
Categories: whistleblower, hr_dsi added to document generator
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-01 08:29:52 +02:00
bae59e2ce0
feat: Document Templates v2 — 11 migrations + scope-based generator
...
Complete overhaul of document generator templates based on paragraph-by-paragraph
legal review of attorney-drafted templates (TOM, AVV, AGB, DSI, Community
Guidelines, Nutzungsbedingungen, Widerrufsbelehrung, Cookie-Richtlinie).
Templates (11 migrations 087-097):
- 087: TOM-Dokumentation v2 (11 categories incl. Trennungskontrolle)
- 088: AVV Art. 28 DSGVO (complete, §§ 1-11, 3 annexes)
- 089: Cross-document updates (Löschkonzept DIN 66399, VVT recipients)
- 090: AGB SaaS/Shop v2 (18 §§, B2B/B2C, IoT, physical goods, IP protection)
- 091: Community Guidelines v2 (3 tones, 11 modular categories, DSA-compliant)
- 092: Media & Content modules (MStV, AI Act Art. 50, UWG, Pressekodex)
- 093: DSI/Privacy Policy v2 (Art. 13 complete, shop+corporate modules)
- 094: Nutzungsbedingungen (Terms of Use, UGC, tipping, wallet, CC licenses)
- 095: Widerrufsbelehrung (SaaS + physical + IoT bundle + combo)
- 096: Social Media DSI (Facebook, YouTube, LinkedIn, TikTok, Meta Pixel)
- 097: Cookie-Richtlinie v2 (TDDDG § 25, consent banner, browser links)
Frontend (generator):
- scopeDefaults.ts: L1-L4 scope-based defaults from Compliance Scope Engine
- contextBridge.ts: TOMCtx + DPACtx interfaces (70+ new fields)
- contextBridge-helpers.ts: 35+ placeholder mappings for TOM/DPA/AGB
- _constants.ts: 120+ new generator fields (TOM, DPA, AGB, community,
media, social, nutzungsbedingungen, widerruf, cookie, shop, IoT)
- page.tsx: Auto-prefill TOM/DPA from scope engine decision
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-01 01:18:33 +02:00
e35db90232
feat: Phase 5 — DB persistence for scan results + Phase 10 in plan
...
- Migration 086: compliance_agent_scans table (findings, services, corrections)
- agent_history_routes.py: POST /scans (save), GET /scans (list), GET /scans/{id}
- Scan results survive page reloads and can be reviewed later
- Phase 10 (Playwright website scanner) added to product roadmap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-29 15:17:51 +02:00
b2a28eb4cd
feat: DSR Prozessbeschreibungen Art. 15-21 mit Swim-Lane-Diagrammen
...
Build + Deploy / build-admin-compliance (push) Successful in 10s
Build + Deploy / build-backend-compliance (push) Successful in 9s
Build + Deploy / build-ai-sdk (push) Successful in 8s
Build + Deploy / build-developer-portal (push) Successful in 7s
Build + Deploy / build-tts (push) Successful in 7s
Build + Deploy / build-document-crawler (push) Successful in 7s
Build + Deploy / build-dsms-gateway (push) Successful in 7s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go (push) Failing after 41s
CI / test-python-backend (push) Successful in 35s
CI / test-python-document-crawler (push) Successful in 25s
CI / test-python-dsms-gateway (push) Successful in 21s
CI / loc-budget (push) Failing after 13s
CI / secret-scan (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / nodejs-build (push) Successful in 2m29s
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / validate-canonical-controls (push) Successful in 13s
Build + Deploy / trigger-orca (push) Successful in 1m53s
7 vollstaendige Prozessbeschreibungen fuer den Document Generator:
- Art. 15: Auskunftsrecht (30 Tage, 6 Schritte, Informationskatalog)
- Art. 16: Berichtigungsrecht (14 Tage, inkl. Art. 19 Mitteilung)
- Art. 17: Loeschungsrecht (14 Tage, Art. 17(3) Ausnahmen-Checkliste)
- Art. 18: Einschraenkungsrecht (14 Tage, erlaubte Verarbeitung)
- Art. 19: Mitteilungspflicht (automatisch bei Art. 16/17/18)
- Art. 20: Datenuebertragbarkeit (30 Tage, JSON/CSV/XML Export)
- Art. 21: Widerspruchsrecht (30 Tage, Sonderfall Direktwerbung)
Jede Beschreibung enthaelt:
- Mermaid Swim-Lane-Diagramm (Betroffener/Sachbearbeitung/Fachabteilung/DSB)
- Detaillierte Schritt-Tabelle mit Verantwortlichkeiten und Fristen
- Rechtsgrundlagen-Verweise
- Firmen-Platzhalter (FIRMENNAME, VERSION, DATUM, DSB_NAME)
Integration:
- 7 neue Typen in VALID_DOCUMENT_TYPES (legal_template_routes.py)
- Neue Kategorie "DSR-Prozesse" im Document Generator Frontend
- DSR types-core.ts: templateType Feld verknuepft DSR → Document Generator
- Migration 085 seeded die Templates in die legal_templates Tabelle
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-28 19:25:38 +02:00
b39c1d5dce
feat: DSR Prozessbeschreibungen Art. 15-21 mit Swim-Lane-Diagrammen
...
Build + Deploy / build-admin-compliance (push) Successful in 1m56s
Build + Deploy / build-backend-compliance (push) Successful in 3m5s
Build + Deploy / build-ai-sdk (push) Successful in 47s
Build + Deploy / build-developer-portal (push) Successful in 1m5s
Build + Deploy / build-tts (push) Successful in 1m23s
Build + Deploy / build-document-crawler (push) Successful in 33s
Build + Deploy / build-dsms-gateway (push) Successful in 23s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / loc-budget (push) Failing after 17s
CI / secret-scan (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / nodejs-build (push) Successful in 2m40s
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / test-go (push) Successful in 42s
CI / test-python-backend (push) Successful in 47s
CI / test-python-document-crawler (push) Successful in 33s
CI / test-python-dsms-gateway (push) Successful in 22s
CI / validate-canonical-controls (push) Successful in 18s
Build + Deploy / trigger-orca (push) Successful in 2m53s
7 vollstaendige Prozessbeschreibungen fuer den Document Generator:
- Art. 15: Auskunftsrecht (30 Tage, 6 Schritte, Informationskatalog)
- Art. 16: Berichtigungsrecht (14 Tage, inkl. Art. 19 Mitteilung)
- Art. 17: Loeschungsrecht (14 Tage, Art. 17(3) Ausnahmen-Checkliste)
- Art. 18: Einschraenkungsrecht (14 Tage, erlaubte Verarbeitung)
- Art. 19: Mitteilungspflicht (automatisch bei Art. 16/17/18)
- Art. 20: Datenuebertragbarkeit (30 Tage, JSON/CSV/XML Export)
- Art. 21: Widerspruchsrecht (30 Tage, Sonderfall Direktwerbung)
Jede Beschreibung enthaelt:
- Mermaid Swim-Lane-Diagramm (Betroffener/Sachbearbeitung/Fachabteilung/DSB)
- Detaillierte Schritt-Tabelle mit Verantwortlichkeiten und Fristen
- Rechtsgrundlagen-Verweise
- Firmen-Platzhalter (FIRMENNAME, VERSION, DATUM, DSB_NAME)
Integration:
- 7 neue Typen in VALID_DOCUMENT_TYPES (legal_template_routes.py)
- Neue Kategorie "DSR-Prozesse" im Document Generator Frontend
- DSR types-core.ts: templateType Feld verknuepft DSR → Document Generator
- Migration 085 seeded die Templates in die legal_templates Tabelle
[migration-approved]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-28 17:53:44 +02:00
bc75b4455d
feat: AI Act Decision Tree — Zwei-Achsen-Klassifikation (GPAI + High-Risk)
...
Interaktiver 12-Fragen-Entscheidungsbaum für die AI Act Klassifikation
auf zwei Achsen: High-Risk (Anhang III, Q1-Q7) und GPAI (Art. 51-56, Q8-Q12).
Deterministische Auswertung ohne LLM.
Backend (Go):
- Neue Structs: GPAIClassification, DecisionTreeAnswer, DecisionTreeResult
- Decision Tree Engine mit BuildDecisionTreeDefinition() und EvaluateDecisionTree()
- Store-Methoden für CRUD der Ergebnisse
- API-Endpoints: GET/POST /decision-tree, GET/DELETE /decision-tree/results
- 12 Unit Tests (alle bestanden)
Frontend (Next.js):
- DecisionTreeWizard: Wizard-UI mit Ja/Nein-Fragen, Dual-Progress-Bar, Ergebnis-Ansicht
- AI Act Page refactored: Tabs (Übersicht | Entscheidungsbaum | Ergebnisse)
- Proxy-Route für decision-tree Endpoints
Migration 083: ai_act_decision_tree_results Tabelle
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-29 10:14:09 +02:00
447ec08509
Add migration 082: widen source_article to TEXT, fix pass0b query filters
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 40s
CI/CD / test-python-backend-compliance (push) Successful in 31s
CI/CD / test-python-document-crawler (push) Successful in 21s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Successful in 5s
- source_article/source_regulation VARCHAR(100) → TEXT for long NIST refs
- Pass 0b NOT EXISTS queries now skip deprecated/duplicate controls
- Duplicate Guard excludes deprecated/duplicate from existence check
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-28 12:47:26 +01:00
f39e5a71af
feat: Obligation-Deduplizierung — 34.617 Duplikate als 'duplicate' markiert
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 33s
CI/CD / test-python-backend-compliance (push) Successful in 35s
CI/CD / test-python-document-crawler (push) Successful in 30s
CI/CD / test-python-dsms-gateway (push) Successful in 20s
CI/CD / validate-canonical-controls (push) Successful in 13s
CI/CD / Deploy (push) Successful in 3s
Neue Endpunkte POST /obligations/dedup und GET /obligations/dedup-stats.
Pro candidate_id wird der aelteste Eintrag behalten, alle weiteren erhalten
release_state='duplicate' mit merged_into_id + quality_flags fuer Traceability.
Detail-View filtert Duplikate aus. MKDocs aktualisiert.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-26 20:13:00 +01:00
db7c207464
feat: V1 Control Enrichment — Eigenentwicklung-Label, regulatorisches Matching & Vergleichsansicht
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 39s
CI/CD / test-python-backend-compliance (push) Successful in 32s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 16s
CI/CD / validate-canonical-controls (push) Successful in 9s
CI/CD / Deploy (push) Successful in 4s
863 v1-Controls (manuell geschrieben, ohne Rechtsgrundlage) werden als
"Eigenentwicklung" gekennzeichnet und automatisch mit regulatorischen
Controls (DSGVO, NIS2, OWASP etc.) per Embedding-Similarity abgeglichen.
Backend:
- Migration 080: v1_control_matches Tabelle (Cross-Reference)
- v1_enrichment.py: Batch-Matching via BGE-M3 + Qdrant (Threshold 0.75)
- 3 neue API-Endpoints: enrich-v1-matches, v1-matches, v1-enrichment-stats
- 6 Tests (dry-run, execution, matches, pagination, detection)
Frontend:
- Orange "Eigenentwicklung"-Badge statt grauem "v1" (wenn kein Source)
- "Regulatorische Abdeckung"-Sektion im ControlDetail mit Match-Karten
- Side-by-Side V1CompareView (Eigenentwicklung vs. regulatorisch gedeckt)
- Prev/Next Navigation durch alle Matches
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-26 10:32:08 +01:00
5e9cab6ab5
feat: evidence_type Feld (code/process/hybrid) fuer Controls
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 38s
CI/CD / test-python-backend-compliance (push) Successful in 31s
CI/CD / test-python-document-crawler (push) Successful in 19s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Successful in 4s
Neues Feld auf canonical_controls klassifiziert, ob ein Control
technisch im Source Code (code), organisatorisch via Dokumente (process)
oder beides (hybrid) nachgewiesen wird. Inklusive Backfill-Endpoint,
Frontend-Badge/Filter und MkDocs-Dokumentation.
- Migration 079: evidence_type VARCHAR(20) + Index
- Backend: Filter, Backfill-Endpoint mit Domain-Heuristik, CRUD
- Frontend: EvidenceTypeBadge (sky/amber/violet), Nachweisart-Dropdown
- Proxy: evidence_type Passthrough fuer controls + controls-count
- Tests: 22 Tests fuer Klassifikations-Heuristik
- Docs: Eigenes MkDocs-Kapitel mit Mermaid-Diagramm
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-25 21:53:40 +01:00
35784c35eb
feat: Batch Dedup Runner — 85k→~18-25k Master Controls
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 32s
CI/CD / test-python-backend-compliance (push) Successful in 30s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 16s
CI/CD / validate-canonical-controls (push) Successful in 9s
CI/CD / Deploy (push) Successful in 1s
Adds batch orchestration for deduplicating ~85k Pass 0b atomic controls
into ~18-25k unique masters with M:N parent linking.
New files:
- migrations/078_batch_dedup.sql: merged_into_uuid column, perf indexes,
link_type CHECK extended for cross_regulation
- batch_dedup_runner.py: BatchDedupRunner with quality scoring, merge-hint
grouping, title-identical short-circuit, parent-link transfer, and
cross-regulation pass
- tests/test_batch_dedup_runner.py: 21 tests (all passing)
Modified:
- control_dedup.py: optional collection param on Qdrant functions
- crosswalk_routes.py: POST/GET batch-dedup endpoints
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-24 07:06:38 +01:00
e6201d5239
feat: Anti-Fake-Evidence System (Phase 1-4b)
...
Implement full evidence integrity pipeline to prevent compliance theater:
- Confidence levels (E0-E4), truth status tracking, assertion engine
- Four-Eyes approval workflow, audit trail, reject endpoint
- Evidence distribution dashboard, LLM audit routes
- Traceability matrix (backend endpoint + Compliance Hub UI tab)
- Anti-fake badges, control status machine, normative patterns
- 2 migrations, 4 test suites, MkDocs documentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-23 17:15:45 +01:00
a14e2f3a00
feat(decomposition): add merge pass, enrichment, and Pass 0b refinements
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 51s
CI/CD / test-python-backend-compliance (push) Successful in 34s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 20s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Has been skipped
Add obligation refinement pipeline between Pass 0a and 0b:
- Merge pass: rule-based dedup of implementation-level duplicate obligations
within the same parent control (Jaccard similarity on action+object)
- Enrich pass: classify trigger_type (event/periodic/continuous) and detect
is_implementation_specific from obligation text (regex-based, no LLM)
- Pass 0b: skip merged obligations, cap severity for impl-specific, override
category to 'testing' for test obligations
- Migration 075: merged_into_id, trigger_type, is_implementation_specific
- Two new API endpoints: merge-obligations, enrich-obligations
- 30+ new tests (122 total, all passing)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-21 22:27:09 +01:00
643b26618f
feat: Control Library UI, dedup migration, QA tooling, docs
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 31s
CI/CD / test-python-backend-compliance (push) Successful in 1m35s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Has been skipped
- Control Library: parent control display, ObligationTypeBadge,
GenerationStrategyBadge variants, evidence string fallback
- API: expose parent_control_uuid/id/title in canonical controls
- Fix: DSFA SQLAlchemy 2.0 Row._mapping compatibility
- Migration 074: control_parent_links + control_dedup_reviews tables
- QA scripts: benchmark, gap analysis, OSCAL import, OWASP cleanup,
phase5 normalize, phase74 gap fill, sync_db, run_job
- Docs: dedup engine, RAG benchmark, lessons learned, pipeline docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-21 11:56:08 +01:00
1cc34c23d9
feat(document-generator): 33 policy + module document templates
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 36s
CI/CD / test-python-backend-compliance (push) Successful in 34s
CI/CD / test-python-document-crawler (push) Successful in 25s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Has been skipped
- Migration 071: 14 IT-Security policy templates (ISO 27001/BSI)
information_security, access_control, password, encryption, logging,
backup, incident_response, change_management, patch_management,
asset_management, cloud_security, devsecops, secrets_management,
vulnerability_management
- Migration 072: 15 Data/HR/Vendor/BCM policy templates
data_protection, data_classification, data_retention, data_transfer,
privacy_incident, employee_security, security_awareness, remote_work,
offboarding, vendor_risk_management, third_party_security,
supplier_security, business_continuity, disaster_recovery,
crisis_management
- Migration 073: 4 module document reference templates
vvt_register, tom_documentation, loeschkonzept, pflichtenregister
- TemplateType union: 17 → 61 types with German labels
- VALID_DOCUMENT_TYPES: +6 types (cybersecurity_policy, dsfa, 4 module docs)
- CATEGORIES: new "DSGVO-Dokumente" category for module documents
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-19 23:27:25 +01:00
c3afa628ed
feat(sdk): vendor-compliance cross-module integration — VVT, obligations, TOM, loeschfristen
...
Integrate the vendor-compliance module with four DSGVO modules to eliminate
data silos and resolve the VVT processor tab's ephemeral state problem.
- Reposition vendor-compliance sidebar from seq 4200 to 2500 (after VVT)
- VVT: replace ephemeral ProcessorRecord state with Vendor-API fetch (read-only)
- Obligations: add linked_vendor_ids (JSONB) + compliance check #12 MISSING_VENDOR_LINK
- TOM: add vendor TOM-controls cross-reference table in overview tab
- Loeschfristen: add linked_vendor_ids (JSONB) + vendor picker + document section
- Migrations: 069_obligations_vendor_link.sql, 070_loeschfristen_vendor_link.sql
- Tests: 12 new backend tests (125 total pass)
- Docs: update obligations.md + vendors.md with cross-module integration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-19 13:59:43 +01:00
4b1eede45b
feat(tom): audit document, compliance checks, 25 controls, canonical control mapping
...
Phase A: TOM document HTML generator (12 sections, inline CSS, A4 print)
Phase B: TOMDocumentTab component (org-header form, revisions, print/download)
Phase C: 11 compliance checks with severity-weighted scoring
Phase D: MkDocs documentation for TOM module
Phase E: 25 new controls (63 → 88) in 13 categories
Canonical Control Mapping (three-layer architecture):
- Migration 068: tom_control_mappings + tom_control_sync_state tables
- 6 API endpoints: sync, list, by-tom, stats, manual add, delete
- Category mapping: 13 TOM categories → 17 canonical categories
- Frontend: sync button + coverage card (Overview), drill-down (Editor),
belegende Controls count (Document)
- 20 tests (unit + API with mocked DB)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-19 11:56:53 +01:00
2a70441eaa
feat(sdk): VVT master libraries, process templates, Loeschfristen profiling + document
...
VVT: Master library tables (7 catalogs), 500+ seed entries, process templates
with instantiation, library API endpoints + 18 tests.
Loeschfristen: Baseline catalog, compliance checks, profiling engine, HTML document
generator, MkDocs documentation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-19 11:56:25 +01:00
f2819b99af
feat(pipeline): v3 — scoped control applicability + source_type classification
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 36s
CI/CD / test-python-backend-compliance (push) Successful in 36s
CI/CD / test-python-document-crawler (push) Successful in 27s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Has been skipped
Phase 4: source_type (law/guideline/standard/restricted) on source_citation
- NIST/OWASP/ENISA correctly shown as "Standard" instead of "Gesetzliche Grundlage"
- Dynamic frontend labels based on source_type
- Backfill endpoint POST /v1/canonical/generate/backfill-source-type
Phase v3: Scoped Control Applicability
- 3 new fields: applicable_industries, applicable_company_size, scope_conditions
- LLM prompt extended with 39 industries, 5 company sizes, 10 scope signals
- All 5 generation paths (Rule 1/2/3, batch structure, batch reform) updated
- _build_control_from_json: parsing + validation (string→list, size validation)
- _store_control: writes 3 new JSONB columns
- API: response models, create/update requests, SELECT queries extended
- Migration 063: 3 new JSONB columns with GIN indexes
- 110 generator tests + 28 route tests = 138 total, all passing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-18 16:28:05 +01:00
a9e0869205
feat(pipeline): pipeline_version v2, migration 062, docs + 71 tests
...
- Add PIPELINE_VERSION=2 constant and pipeline_version column to
canonical_controls and canonical_processed_chunks (migration 062)
- Anthropic API decides chunk relevance via null-returns (skip_prefilter)
- Annex/appendix chunks explicitly protected in prompts
- Fix 6 failing tests (CRYP domain, _process_batch tuple return)
- Add TestPipelineVersion + TestRegulationFilter test classes (10 new tests)
- Add MkDocs page: control-generator-pipeline.md (541 lines)
- Update canonical-control-library.md with v2 pipeline diagram
- Update testing.md with 71-test breakdown table
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-17 17:31:11 +01:00
825e070ed9
feat(multi-layer): complete Multi-Layer Control Architecture (Phases 1-8 + Pass 0)
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 47s
CI/CD / test-python-backend-compliance (push) Successful in 33s
CI/CD / test-python-document-crawler (push) Successful in 24s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Has been skipped
Implements the full Multi-Layer Control Architecture for migrating ~25,000
Rich Controls into atomic, deduplicated Master Controls with full traceability.
Architecture: Legal Source → Obligation → Control Pattern → Master Control → Customer Instance
New services:
- ObligationExtractor: 3-tier extraction (exact → embedding → LLM)
- PatternMatcher: 2-tier matching (keyword + embedding + domain-bonus)
- ControlComposer: Pattern + Obligation → Master Control
- PipelineAdapter: Pipeline integration + Migration Passes 1-5
- DecompositionPass: Pass 0a/0b — Rich Control → atomic Controls
- CrosswalkRoutes: 15 API endpoints under /v1/canonical/
New DB schema:
- Migration 060: obligation_extractions, control_patterns, crosswalk_matrix
- Migration 061: obligation_candidates, parent_control_uuid tracking
Pattern Library: 50 YAML patterns (30 core + 20 IT-security)
Go SDK: Pattern loader with YAML validation and indexing
Documentation: MkDocs updated with full architecture overview
500 Python tests passing across all components.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-17 09:00:37 +01:00
4f6bc8f6f6
feat(training+controls): interactive video pipeline, training blocks, control generator, CE libraries
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 37s
CI/CD / test-python-backend-compliance (push) Successful in 39s
CI/CD / test-python-document-crawler (push) Successful in 26s
CI/CD / test-python-dsms-gateway (push) Successful in 23s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Has been skipped
Interactive Training Videos (CP-TRAIN):
- DB migration 022: training_checkpoints + checkpoint_progress tables
- NarratorScript generation via Anthropic (AI Teacher persona, German)
- TTS batch synthesis + interactive video pipeline (slides + checkpoint slides + FFmpeg)
- 4 new API endpoints: generate-interactive, interactive-manifest, checkpoint submit, checkpoint progress
- InteractiveVideoPlayer component (HTML5 Video, quiz overlay, seek protection, progress tracking)
- Learner portal integration with automatic completion on all checkpoints passed
- 30 new tests (handler validation + grading logic + manifest/progress + seek protection)
Training Blocks:
- Block generator, block store, block config CRUD + preview/generate endpoints
- Migration 021: training_blocks schema
Control Generator + Canonical Library:
- Control generator routes + service enhancements
- Canonical control library helpers, sidebar entry
- Citation backfill service + tests
- CE libraries data (hazard, protection, evidence, lifecycle, components)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-16 21:41:48 +01:00
c8fd9cc780
feat(control-library): document-grouped batching, generation strategy tracking, sort by source
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 31s
CI/CD / test-python-backend-compliance (push) Successful in 31s
CI/CD / test-python-document-crawler (push) Successful in 21s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Successful in 2s
- Group chunks by regulation_code before batching for better LLM context
- Add generation_strategy column (ungrouped=v1, document_grouped=v2)
- Add v1/v2 badge to control cards in frontend
- Add sort-by-source option with visual group headers
- Add frontend page tests (18 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-15 15:10:52 +01:00
dd09fa7a46
feat: CRA wiki, cybersecurity policy template, Phase H RAG ingestion
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 35s
CI/CD / test-python-backend-compliance (push) Successful in 33s
CI/CD / test-python-document-crawler (push) Successful in 22s
CI/CD / test-python-dsms-gateway (push) Successful in 19s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
- Wiki: add CRA category with 3 articles (Grundlagen, 35 Security Controls,
CRA+NIS2+AI Act Framework)
- Document Generator: add CRA-konforme Cybersecurity Policy template with
21 sections covering governance, SSDLC, vulnerability management,
incident response (24h/72h), SBOM, patch management
- RAG: ingest Phase H — 17 EU regulations + 2 NIST frameworks now in Qdrant
(CRA, AI Act, NIS2, DSGVO, DMA, GPSR, Batterieverordnung, etc.)
- Phase H script: add scripts/ingest-phase-h.sh for reproducible ingestion
- rag-sources.md: update status to ingestiert, add CRA entry
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-15 00:43:46 +01:00
f3e05c1bf7
feat: enhance whistleblower HinSchG content, fix control-library filter layout
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 34s
CI/CD / test-python-backend-compliance (push) Successful in 35s
CI/CD / test-python-document-crawler (push) Successful in 26s
CI/CD / test-python-dsms-gateway (push) Successful in 21s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
- Whistleblower page: expand overview tab with comprehensive HinSchG legal info
(Gesetzliche Grundlage, Fristen-Cards, Anwendungsbereich, Schutz des Hinweisgebers)
- StepHeader: enrich whistleblower tips with detailed HinSchG paragraphs and sanctions
- Wiki: add migration 054 with 5 new/updated HinSchG articles (Anwendungsbereich,
Hinweisgeberschutz, Meldestellen, Verfahrensablauf, Datenschutz-Anforderungen)
- MKDocs: rewrite whistleblower docs with full legal basis, architecture, API, DB schema
- Control library: fix filter dropdown overflow by splitting into search + filter rows
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-15 00:23:19 +01:00
637fab6fdb
fix: migration runner strips BEGIN/COMMIT and guards missing tables
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 33s
CI/CD / test-python-backend-compliance (push) Successful in 39s
CI/CD / test-python-document-crawler (push) Successful in 22s
CI/CD / test-python-dsms-gateway (push) Successful in 21s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
Root cause: migrations 046-047 used explicit BEGIN/COMMIT which
conflicts with psycopg2 implicit transactions, and ALTER TABLE
on canonical_controls fails when the table doesn't exist on
production. This blocked all subsequent migrations (048-053).
Changes:
- migration_runner.py: strip BEGIN/COMMIT from SQL before executing
- 046: wrap canonical_controls ALTER in DO $$ IF EXISTS block
- 047: wrap canonical_controls ALTER in DO $$ IF EXISTS block
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-14 21:59:10 +01:00
5f8aebf5b1
fix: make migrations 048/049 safe for environments without canonical tables
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 32s
CI/CD / test-python-backend-compliance (push) Successful in 33s
CI/CD / test-python-document-crawler (push) Successful in 21s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
Migrations 048 and 049 reference canonical_processed_chunks and
canonical_controls tables which may not exist on all environments.
Wrap ALTER TABLE statements in DO blocks that check for table
existence first. This unblocks migrations 050-053 on production.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-14 21:45:00 +01:00
49ce417428
feat: add compliance modules 2-5 (dashboard, security templates, process manager, evidence collector)
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 32s
CI/CD / test-python-backend-compliance (push) Successful in 34s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 21s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Successful in 2s
Module 2: Extended Compliance Dashboard with roadmap, module-status, next-actions, snapshots, score-history
Module 3: 7 German security document templates (IT-Sicherheitskonzept, Datenschutz, Backup, Logging, Incident-Response, Zugriff, Risikomanagement)
Module 4: Compliance Process Manager with CRUD, complete/skip/seed, ~50 seed tasks, 3-tab UI
Module 5: Evidence Collector Extended with automated checks, control-mapping, coverage report, 4-tab UI
Also includes: canonical control library enhancements (verification method, categories, dedup), control generator improvements, RAG client extensions
52 tests pass, frontend builds clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-14 21:03:04 +01:00
b6e6ffaaee
feat: add verification method, categories, and dedup UI to control library
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 44s
CI/CD / test-python-backend-compliance (push) Successful in 40s
CI/CD / test-python-document-crawler (push) Successful in 22s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Successful in 4s
- Migration 047: verification_method + category columns, 17 category lookup table
- Backend: new filters, GET /categories, GET /controls/{id}/similar (embedding-based)
- Frontend: filter dropdowns, badges, dedup UI in ControlDetail with merge workflow
- ControlForm: verification method + category selects
- Provenance: verification methods, categories, master library strategy sections
- Fix UUID cast syntax in generator routes (::uuid -> CAST)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-14 07:55:22 +01:00
de19ef0684
feat(control-generator): 7-stage pipeline for RAG→LLM→Controls generation
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 45s
CI/CD / test-python-document-crawler (push) Has been cancelled
CI/CD / test-python-dsms-gateway (push) Has been cancelled
CI/CD / validate-canonical-controls (push) Has been cancelled
CI/CD / deploy-hetzner (push) Has been cancelled
CI/CD / test-python-backend-compliance (push) Has been cancelled
Implements the Control Generator Pipeline that systematically generates
canonical security controls from 150k+ RAG chunks across all compliance
collections (BSI, NIST, OWASP, ENISA, EU laws, German laws).
Three license rules enforced throughout:
- Rule 1 (free_use): Laws/Public Domain — original text preserved
- Rule 2 (citation_required): CC-BY/CC-BY-SA — text with citation
- Rule 3 (restricted): BSI/ISO — full reformulation, no source traces
New files:
- Migration 046: job tracking, chunk tracking, blocked sources tables
- control_generator.py: 7-stage pipeline (scan→classify→structure/reform→harmonize→anchor→store→mark)
- anchor_finder.py: RAG + DuckDuckGo open-source reference search
- control_generator_routes.py: REST API (generate, review, stats, blocked-sources)
- test_control_generator.py: license mapping, rule enforcement, anchor filtering tests
Modified:
- __init__.py: register control_generator_router
- route.ts: proxy generator/review/stats endpoints
- page.tsx: Generator modal, stats panel, state filter, review queue, license badges
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-13 09:03:37 +01:00
c87f07c99a
feat: seed 10 canonical controls + CRUD endpoints + frontend editor
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 39s
CI/CD / test-python-backend-compliance (push) Successful in 39s
CI/CD / test-python-document-crawler (push) Successful in 30s
CI/CD / test-python-dsms-gateway (push) Successful in 20s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / deploy-hetzner (push) Successful in 1m37s
- Migration 045: Seed 10 controls (AUTH, NET, SUP, LOG, WEB, DATA, CRYP, REL)
with 39 open-source anchors into the database
- Backend: POST/PUT/DELETE endpoints for canonical controls CRUD
- Frontend proxy: PUT and DELETE methods added to canonical route
- Frontend: Control Library with create/edit/delete UI, full form with
open anchor management, scope, requirements, evidence, test procedures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-13 00:28:21 +01:00
050f353192
feat(canonical-controls): Canonical Control Library — rechtssichere Security Controls
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 40s
CI/CD / test-python-backend-compliance (push) Successful in 41s
CI/CD / test-python-document-crawler (push) Successful in 26s
CI/CD / test-python-dsms-gateway (push) Successful in 23s
CI/CD / validate-canonical-controls (push) Successful in 18s
CI/CD / deploy-hetzner (push) Successful in 2m26s
Eigenstaendig formulierte Security Controls mit unabhaengiger Taxonomie
und Open-Source-Verankerung (OWASP, NIST, ENISA). Keine BSI-Nomenklatur.
- Migration 044: 5 DB-Tabellen (frameworks, controls, sources, licenses, mappings)
- 10 Seed Controls mit 39 Open-Source-Referenzen
- License Gate: Quellen-Berechtigungspruefung (analysis/excerpt/embeddings/product)
- Too-Close-Detektor: 5 Metriken (exact-phrase, token-overlap, ngram, embedding, LCS)
- REST API: 8 Endpoints unter /v1/canonical/
- Go Loader mit Multi-Index (ID, domain, severity, framework)
- Frontend: Control Library Browser + Provenance Wiki
- CI/CD: validate-controls.py Job (schema, no-leak, open-anchors)
- 67 Tests (8 Go + 59 Python), alle PASS
- MkDocs Dokumentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-12 19:55:06 +01:00
Benjamin Admin