e536247c20
feat(quaidal): backend API + frontend tab for BSI QUAIDAL data-quality controls
...
Wire the 195 Clean-Room QUAIDAL controls (from breakpilot-core migration 011)
into the compliance SaaS UI.
Backend:
- GET /api/v1/quaidal/stats - counts by kind + source provenance
- GET /api/v1/quaidal/controls - list, optional kind= filter
- GET /api/v1/quaidal/controls/{id} - single derived control
- GET /api/v1/quaidal/criteria - 10 QKB criteria
- GET /api/v1/quaidal/criteria/{id} - QKB with QB/MA/QM tree
Frontend:
- /sdk/quality: new "Trainingsdaten-Qualität (BSI QUAIDAL)" tab with
10 QKB cards and a drill-down modal showing the full QB→MA→QM tree
plus original BSI source link and license note.
- /sdk/ai-act: Art. 10 tile on each high-risk/unacceptable result,
linking to /sdk/quality?category=data_quality.
Pattern matches existing IACE module DIN-reference handling:
own wording, source section + URL preserved for due diligence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-19 13:03:54 +02:00
6c223c7c9b
feat(compliance-check): exec-summary + voll-audit + TDM-respect + cookie-KB-extended + saving-scan-funnel
...
CI / detect-changes (push) Successful in 10s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / validate-canonical-controls (push) Successful in 14s
CI / loc-budget (push) Failing after 15s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / nodejs-build (push) Successful in 2m43s
CI / test-go (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 37s
CI / test-python-document-crawler (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
P1 — Exec-Summary oben im Email-Report (4 KPIs + 2 CTAs, dunkler Gradient)
P3 — no_direct_sales-Flag fuer OEM-Konfigurator-Sites; AGB/Widerruf/AGB als
"NICHT ANWENDBAR" (grau) statt "NICHT GEFUNDEN" (rot)
P5 — Voll-Audit Unification: alle Findings (MC + Pflichtangaben + Vendor +
Redundanz) in /data/compliance_audits.db.unified_findings; neuer
/api/compliance/agent/findings/<id> Endpoint + FindingsTab im Audit-UI
mit Filter + CSV-Export
P7 — Crawl-Hardening: TDM-Reservation-Check (robots.txt / ai.txt / Header /
Meta) vor jedem Run mit 24h-Cache; HeadlessChrome-UA (Firma noch nicht
gegruendet — Switch via BREAKPILOT_BRANDED_UA env); per-Domain
Rate-Limit 1 req/s + max 2 concurrent
P2 — Cookie-Knowledge-DB additiv erweitert (35 -> 74 Cookies): Adobe, Meta,
Microsoft, LinkedIn, TikTok, HubSpot, Marketo, Salesforce, Hotjar,
FullStory, Mouseflow, Intercom, Drift, Zendesk, Cloudflare, Stripe,
OneTrust/Cookiebot/Usercentrics, Matomo, Pinterest, Snapchat, X/Twitter,
YouTube, Vimeo, Klaviyo, Mailchimp, Mixpanel, Segment, Amplitude,
Optimizely, Datadog; Wire-in in cookie_function_classifier liefert
compliance_risk-Label (kritisch/hoch/mittel/gering) pro Vendor
A — k-Anonymitaets-Helper (benchmark_k_anonymity) fuer P6-Vorbereitung
B — Cross-Tenant-Domain-Assertion im /findings-Endpoint (expected_domain
Query-Param -> 403 bei Mismatch)
C — Saving-Scan-Funnel: /api/compliance/agent/saving-scan/start mit
Validierung + 24h-Rate-Limit pro Domain + Lead-Persistenz in
saving_scan_leads + Auto-Discovery via _run_compliance_check; 6 Tests
D — Risk-Badge im Email-Vendor-Row
Rechtliche Leitplanken (Memory feedback_oem_data_legal.md): nur eigene
Knapp-Bewertungen + Source-Pointer, keine 1:1-Kopien fremder CMP-Texte.
TDM-Opt-Out-Respect nach § 44b UrhG. KEINE Schema-Aenderungen — alles in
Sidecar-SQLite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-18 23:48:34 +02:00
1cf5de1d45
feat(cra): CRA Compliance module Phase 1+2+3 (intake, scope, path, requirements, backlog, sbom, checks)
...
Phase 1 — Intake + Scope + Path:
- Migration 119: compliance_cra_projects table (intake + classification + path + status state machine)
- Backend service cra_routes.py: CRUD + scope-check + path-select
- Deterministic Annex III/IV classifier (verbatim mapping from migration 059 wiki)
- Path validation per classification (CRITICAL → notified_body mandatory)
- Frontend: project list, dashboard, 3-step wizard (intake/scope/path)
- Sidebar entry under "CRA Compliance" (red)
Phase 2 — Annex I Requirements + Priorisierungs-Backlog:
- cra_annex_i_data.py: 40 Annex-I requirements (8 categories), 9 measures (M540-M548), 3 CRA deadlines
- Endpoints: /requirements (40 items), /backlog (priority-sorted with deadline pressure)
- Frontend: requirements table with filters + expandable details, backlog with deadline banner + score-ranked table
- Dashboard KPI cards (Critical count, days to CE deadline, etc.) + top-10 backlog snippet
Phase 3 — SBOM Upload + Automated Checks:
- Migration 120: compliance_cra_sboms (versioned uploads, CycloneDX + SPDX)
- SBOM endpoints: POST /sbom/upload (format detection, summary extraction), GET /sboms
- Checks reuse compliance_evidence_checks: init creates 6 default CRA checks, run executes
- Real implementations: cra_security_txt (HTTP + Contact: line) and cra_tls_cert_check (TLS handshake)
- Frontend: SBOM file upload + version list, Checks page with per-check URL input + Run button
Backend-Reuse: gap_projects (intake pre-population), compliance_evidence_checks/_check_results.
Tenant scoping via existing X-Tenant-ID header pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-18 17:56:52 +02:00
df7d83134b
feat(agent): migrate compliance-check results to banner + documents (M1-M5)
...
After a compliance-check run finishes, the user can now apply the
extracted vendor inventory directly to their own:
- CookieBanner config (admin /sdk/einwilligungen)
- Cookie-Policy / VVT-Register / Privacy-Policy templates
(admin /sdk/document-generator)
Backend:
- migration_to_banner.py: vendor list -> CookieBannerConfig with
ESSENTIAL/PERFORMANCE/PERSONALIZATION/EXTERNAL_MEDIA buckets +
review flags (broken opt-out URLs, missing expiry, no cookies listed)
- migration_to_document.py: vendor list -> pre-fills for 3 doc
templates, recipient-type aware (INTERNAL/GROUP/PROCESSOR/CONTROLLER)
- agent_migration_routes.py: GET /banner-preview, /document-preview,
/summary keyed on check_id
- compliance_audit_log: new check_payloads table persists cmp_vendors +
extracted_profile so the preview survives an app restart
- tests: 9 mapper units + 4 endpoint integration tests
Frontend:
- MigrationPanel.tsx: modal showing banner-config diff + document
pre-fills, plus links into the existing editors
- ComplianceCheckTab.tsx: replaces standalone audit link with the
panel; net -3 lines, stays at the 500-cap
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-17 14:06:28 +02:00
0326d5baab
feat(vendor-assessment): AVV/SCC/TOM/Sub-Processor checklists + assessment service
...
Phase 1-3 of the Vendor Contract Assessment:
Backend checklists (Doc-Check L1/L2 engine compatible):
- avv_checks.py: 28 checks (11 L1 + 17 L2) for Art. 28(3) DSGVO
- scc_checks.py: 7 checks for EU SCC 2021 (modules, annexes, TIA)
- tom_annex_checks.py: 12 checks for Art. 32 (8 control objectives)
- sub_processor_checks.py: 7 checks for sub-processor list completeness
Assessment service:
- POST /vendor-compliance/assessments — async contract analysis
- GET /vendor-compliance/assessments/{id} — poll status
- Cross-check engine: detects missing SCC when AVV mentions third-country,
missing TOM annex, missing sub-processor list
All checklists registered in runner.py CHECKLIST_MAP (27 doc_types total).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-12 23:14:54 +02:00
0d0e705117
feat: Unified Compliance-Check — 8 document types in one form
...
New 3-tab structure: Website-Scan, Compliance-Check, Banner-Check.
Compliance-Check Tab (replaces Dokumenten-Pruefung + Impressum-Check):
- 8 document rows: DSI, Impressum, Social Media, Cookie, AGB,
Nutzungsbedingungen, Widerruf, DSB-Kontakt
- Each row: URL input + "Text laden" + file upload + manual text
- "Text laden" extracts via consent-tester, shows in editable textarea
- User verifies/corrects text before checking
- Empty fields = "not present" → own finding
Business Profiler (business_profiler.py):
- Detects B2B/B2C/B2G from all documents together
- Recognizes regulated professions, online shops, editorial content
- Context-aware: INFO checks become PASS/FAIL based on profile
Backend: /compliance-check + /extract-text endpoints
Frontend: ComplianceCheckTab.tsx + DocumentRow.tsx
API proxies: compliance-check/route.ts + extract-text/route.ts
Also: Impressum regex fixes (Telefon, AG, Geschaeftsfuehrung)
and INFO severity for context-dependent checks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-11 20:56:10 +02:00
8336c01c5c
feat: Phase 6-8 — PDF export, recurring scans, multi-website compare
...
Phase 6: PDF export via WeasyPrint — POST /agent/scans/pdf generates
printable compliance report with findings table, service comparison,
risk badge, and legal disclaimer.
Phase 7: Recurring scans — POST /agent/monitored-urls to add URLs,
POST /agent/run-scheduled triggers all enabled scans (cron/ZeroClaw).
In-memory storage with DB upgrade path.
Phase 8: Multi-website compare — POST /agent/compare with 2-5 URLs,
parallel scanning, comparison table (risk, findings, services, compliance
features per site).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-29 15:27:51 +02:00
e35db90232
feat: Phase 5 — DB persistence for scan results + Phase 10 in plan
...
- Migration 086: compliance_agent_scans table (findings, services, corrections)
- agent_history_routes.py: POST /scans (save), GET /scans (list), GET /scans/{id}
- Scan results survive page reloads and can be reviewed later
- Phase 10 (Playwright website scanner) added to product roadmap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-29 15:17:51 +02:00
711b9b3146
feat: website scanner with SOLL/IST service comparison + corrections
...
- website_scanner.py: multi-page crawl, 20+ service patterns (tracking,
CDN, chatbots, payment, fonts, captcha, video), AI text detection
- dse_service_extractor.py: LLM extracts services from privacy policy text
- agent_scan_routes.py: POST /agent/scan — combines scan + DSE comparison,
generates findings (undocumented, outdated, third-country transfer),
auto-corrections via Qwen in pre-launch mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-28 15:35:31 +02:00
0c0dd4e3a6
feat: ZeroClaw compliance agent — document analysis + role assignment + email
...
Add autonomous compliance agent that fetches web documents (cookie banners,
privacy policies), classifies them via Qwen/Ollama, assesses DSGVO compliance,
assigns to the responsible role, and sends notification emails.
Components:
- ZeroClaw SOP (6-step workflow: fetch, classify, assess, summarize, assign, notify)
- Backend: /api/compliance/agent/analyze (combined endpoint)
- Backend: /api/compliance/agent/notify (standalone email)
- Frontend: /sdk/agent page (Manager UI with URL input + results)
- Helper scripts + E2E test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-27 23:28:21 +02:00
f2924a58ed
debug: add /debug/routers endpoint to diagnose import failures
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 32s
CI/CD / test-python-backend-compliance (push) Successful in 1m37s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Has been skipped
Crosswalk routes returning 404 on production. This adds a diagnostic
endpoint that reports which sub-routers failed to load and why.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-21 12:23:26 +01:00
49ce417428
feat: add compliance modules 2-5 (dashboard, security templates, process manager, evidence collector)
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 32s
CI/CD / test-python-backend-compliance (push) Successful in 34s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 21s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Successful in 2s
Module 2: Extended Compliance Dashboard with roadmap, module-status, next-actions, snapshots, score-history
Module 3: 7 German security document templates (IT-Sicherheitskonzept, Datenschutz, Backup, Logging, Incident-Response, Zugriff, Risikomanagement)
Module 4: Compliance Process Manager with CRUD, complete/skip/seed, ~50 seed tasks, 3-tab UI
Module 5: Evidence Collector Extended with automated checks, control-mapping, coverage report, 4-tab UI
Also includes: canonical control library enhancements (verification method, categories, dedup), control generator improvements, RAG client extensions
52 tests pass, frontend builds clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-14 21:03:04 +01:00
cdafc4d9f4
feat: auto-run SQL migrations on backend startup
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 35s
CI/CD / test-python-backend-compliance (push) Successful in 33s
CI/CD / test-python-document-crawler (push) Successful in 26s
CI/CD / test-python-dsms-gateway (push) Successful in 19s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / deploy-hetzner (push) Successful in 2m35s
Adds migration_runner.py that executes pending migrations from
migrations/ directory when backend-compliance starts. Tracks applied
migrations in _migration_history table.
Handles existing databases: detects if tables from migrations 001-045
already exist and seeds the history table accordingly, so only new
migrations (046+) are applied.
Skippable via SKIP_MIGRATIONS=true env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-13 09:14:18 +01:00
b7c1a5da1a
feat: Consent-Service Module nach Compliance migriert (DSR, E-Mail-Templates, Legal Docs, Banner)
...
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 36s
CI / test-python-backend-compliance (push) Successful in 31s
CI / test-python-document-crawler (push) Successful in 23s
CI / test-python-dsms-gateway (push) Successful in 18s
5-Phasen-Migration: Go consent-service Proxies durch native Python/FastAPI ersetzt.
Phase 1 — DSR (Betroffenenrechte): 6 Tabellen, 30 Endpoints, Frontend-API umgestellt
Phase 2 — E-Mail-Templates: 5 Tabellen, 20 Endpoints, neues Frontend, SDK_STEPS erweitert
Phase 3 — Legal Documents Extension: User Consents, Audit Log, Cookie-Kategorien
Phase 4 — Banner Consent: Device-Consents, Site-Configs, Kategorien, Vendors
Phase 5 — Cleanup: DSR-Proxy aus main.py entfernt, Frontend-URLs aktualisiert
148 neue Tests (50 + 47 + 26 + 25), alle bestanden.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-05 00:36:24 +01:00
e6d666b89b
feat: Vorbereitung-Module auf 100% — Persistenz, Backend-Services, UCCA Frontend
...
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 37s
CI / test-python-backend-compliance (push) Successful in 32s
CI / test-python-document-crawler (push) Successful in 22s
CI / test-python-dsms-gateway (push) Successful in 18s
Phase A: PostgreSQL State Store (sdk_states Tabelle, InMemory-Fallback)
Phase B: Modules dynamisch vom Backend, Scope DB-Persistenz, Source Policy State
Phase C: UCCA Frontend (3 Seiten, Wizard, RiskScoreGauge), Obligations Live-Daten
Phase D: Document Import (PDF/LLM/Gap-Analyse), System Screening (SBOM/OSV.dev)
Phase E: Company Profile CRUD mit Audit-Logging
Phase F: Tests (Python + TypeScript), flow-data.ts DB-Tabellen aktualisiert
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-02 11:04:31 +01:00
a228b3b528
feat: add RAG corpus versioning and source policy backend
...
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 34s
CI / test-python-backend-compliance (push) Successful in 32s
CI / test-python-document-crawler (push) Successful in 23s
CI / test-python-dsms-gateway (push) Successful in 18s
Part 1 — RAG Corpus Versioning:
- New DB table compliance_corpus_versions (migration 017)
- Go CorpusVersionStore with CRUD operations
- Assessment struct extended with corpus_version_id
- API endpoints: GET /rag/corpus-status, /rag/corpus-versions/:collection
- RAG routes (search, regulations) now registered in main.go
- Ingestion script registers corpus versions after each run
- Frontend staleness badge in SDK sidebar
Part 3 — Source Policy Backend:
- New FastAPI router with CRUD for allowed sources, PII rules,
operations matrix, audit trail, stats, and compliance report
- SQLAlchemy models for all source policy tables (migration 001)
- Frontend API base corrected from edu-search:8088/8089 to
backend-compliance:8002/api
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-02 07:58:08 +01:00
4435e7ea0a
Initial commit: breakpilot-compliance - Compliance SDK Platform
...
Services: Admin-Compliance, Backend-Compliance,
AI-Compliance-SDK, Consent-SDK, Developer-Portal,
PCA-Platform, DSMS
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-11 23:47:28 +01:00
Benjamin Admin