Benjamin Admin 6c5e086356 fix: DSI dedup — skip anchor links, filter noise, merge duplicates + fix false positives ...

Dedup fixes:
- Anchor links (#cookies, #betroffenenrechte) on same page are skipped entirely
- Noise titles filtered: 'drucken', 'nach oben', 'Datenschutz' (too generic)
- Documents with < 50 words filtered (navigation snippets)
- Documents with identical word_count merged (same page, different title)
- URL-only titles filtered

False positive fixes (dsi_document_checker.py):
- 'Kontaktdaten des Verantwortlichen' pattern for controller check
- 'Zweck und Rechtsgrundlage' combined heading pattern
- 'Welche Daten werden verarbeitet' question-style headings
- 'Betroffenenrechte' as standalone heading
- 'Welche Rechte hat der Betroffene' question pattern
- 'Daten werden geloescht' retention pattern
- 'Auftragsverarbeiter' as recipient indicator

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-05 11:41:07 +02:00

..

__init__.py

Initial commit: breakpilot-compliance - Compliance SDK Platform

2026-02-11 23:47:28 +01:00

_banner_serializers.py

feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)

2026-05-02 19:52:04 +02:00

_company_profile_sql.py

refactor(backend/api): extract CompanyProfileService (Step 4 — file 4 of 18)

2026-04-07 19:47:29 +02:00

ai_compliance_assistant.py

fix(quality): Ruff/CVE/TS-Fixes, 104 neue Tests, Complexity-Refactoring

2026-03-07 19:00:33 +01:00

anchor_finder.py

feat(rag): optimize RAG pipeline — JSON-Mode, CoT, Hybrid Search, Re-Ranking, Cross-Reg Dedup, chunk 1024

2026-03-21 11:49:43 +01:00

assertion_engine.py

feat: Anti-Fake-Evidence System (Phase 1-4b)

2026-03-23 17:15:45 +01:00

audit_pdf_generator.py

refactor: phase 0 guardrails + phase 1 step 2 (models.py split)

2026-04-07 13:18:29 +02:00

audit_session_service.py

tech-debt: mypy --strict config + integration tests for audit routes

2026-04-07 18:39:40 +02:00

audit_signoff_service.py

tech-debt: mypy --strict config + integration tests for audit routes

2026-04-07 18:39:40 +02:00

auto_risk_updater.py

refactor: phase 0 guardrails + phase 1 step 2 (models.py split)

2026-04-07 13:18:29 +02:00

banner_admin_service.py

refactor(backend/api): extract BannerConsent + BannerAdmin services (Step 4)

2026-04-07 18:52:31 +02:00

banner_consent_service.py

feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)

2026-05-02 19:52:04 +02:00

banner_dsr_service.py

feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)

2026-05-02 19:52:04 +02:00

batch_dedup_runner.py

fix: use CAST(:dd AS jsonb) instead of :dd::jsonb in _write_review

2026-03-24 08:48:58 +01:00

canonical_control_service.py

refactor(backend/api): extract CanonicalControlService (Step 4 — file 6 of 18)

2026-04-07 19:53:55 +02:00

citation_backfill.py

feat(rag): optimize RAG pipeline — JSON-Mode, CoT, Hybrid Search, Re-Ranking, Cross-Reg Dedup, chunk 1024

2026-03-21 11:49:43 +01:00

company_profile_service.py

refactor(backend/api): extract CompanyProfileService (Step 4 — file 4 of 18)

2026-04-07 19:47:29 +02:00

control_composer.py

feat(multi-layer): complete Multi-Layer Control Architecture (Phases 1-8 + Pass 0)

2026-03-17 09:00:37 +01:00

control_dedup.py

feat: Batch Dedup Runner — 85k→~18-25k Master Controls

2026-03-24 07:06:38 +01:00

control_export_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

control_generator.py

feat: Anti-Fake-Evidence System (Phase 1-4b)

2026-03-23 17:15:45 +01:00

control_status_machine.py

feat: Anti-Fake-Evidence System (Phase 1-4b)

2026-03-23 17:15:45 +01:00

cookie_table_generator.py

feat: Cookie banner vendors per category + {{COOKIE_TABLE}} generator

2026-05-02 20:06:57 +02:00

decomposition_pass.py

feat: Pass 0b quality — negative actions, container detection, session object classes

2026-03-28 17:24:19 +01:00

dse_matcher.py

fix: Add missing service modules required by agent_scan_routes

2026-05-04 23:22:30 +02:00

dse_parser.py

fix: Add missing service modules required by agent_scan_routes

2026-05-04 23:22:30 +02:00

dse_service_extractor.py

fix: increase LLM timeouts for scan corrections (90s) and DSE extraction (120s)

2026-04-28 16:05:35 +02:00

dsfa_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

dsfa_workflow_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

dsi_document_checker.py

fix: DSI dedup — skip anchor links, filter noise, merge duplicates + fix false positives

2026-05-05 11:41:07 +02:00

dsr_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

dsr_workflow_service.py

feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)

2026-05-02 19:52:04 +02:00

email_template_service.py

refactor(backend/api): extract Email Template services (Step 4 — file 10 of 18)

2026-04-08 22:39:19 +02:00

email_template_version_service.py

refactor(backend/api): extract Email Template services (Step 4 — file 10 of 18)

2026-04-08 22:39:19 +02:00

evidence_service.py

refactor(backend/api): extract EvidenceService (Step 4 — file 9 of 18)

2026-04-08 21:59:03 +02:00

export_generator.py

refactor: phase 0 guardrails + phase 1 step 2 (models.py split)

2026-04-07 13:18:29 +02:00

framework_decomposition.py

feat: Framework Decomposition Engine + Composite Detection for Pass 0b

2026-03-23 12:11:55 +01:00

incident_service.py

refactor(backend/api): extract Incident services (Step 4 — file 11 of 18)

2026-04-09 08:35:57 +02:00

incident_workflow_service.py

refactor(backend/api): extract Incident services (Step 4 — file 11 of 18)

2026-04-09 08:35:57 +02:00

isms_assessment_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

isms_findings_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

isms_governance_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

isms_readiness_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

legal_basis_validator.py

fix: Add missing service modules required by agent_scan_routes

2026-05-04 23:22:30 +02:00

legal_document_consent_service.py

refactor(backend/api): extract LegalDocumentConsentService (Step 4 — file 12 of 18)

2026-04-09 08:47:56 +02:00

legal_document_service.py

chore: mypy fixes for routes.py + legal_document_service + control_export_service

2026-04-09 20:04:16 +02:00

license_gate.py

feat(canonical-controls): Canonical Control Library — rechtssichere Security Controls

2026-03-12 19:55:06 +01:00

llm_provider.py

fix: Ollama think:false fuer qwen3.5 Thinking-Mode

2026-03-25 23:25:14 +01:00

mandatory_content_checker.py

fix: Add missing service modules required by agent_scan_routes

2026-05-04 23:22:30 +02:00

normative_patterns.py

feat: Anti-Fake-Evidence System (Phase 1-4b)

2026-03-23 17:15:45 +01:00

notfallplan_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

notfallplan_workflow_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

obligation_extractor.py

feat(rag): optimize RAG pipeline — JSON-Mode, CoT, Hybrid Search, Re-Ranking, Cross-Reg Dedup, chunk 1024

2026-03-21 11:49:43 +01:00

pattern_matcher.py

feat(multi-layer): complete Multi-Layer Control Architecture (Phases 1-8 + Pass 0)

2026-03-17 09:00:37 +01:00

pdf_extractor.py

Initial commit: breakpilot-compliance - Compliance SDK Platform

2026-02-11 23:47:28 +01:00

pipeline_adapter.py

feat(multi-layer): complete Multi-Layer Control Architecture (Phases 1-8 + Pass 0)

2026-03-17 09:00:37 +01:00

rag_client.py

feat(rag): optimize RAG pipeline — JSON-Mode, CoT, Hybrid Search, Re-Ranking, Cross-Reg Dedup, chunk 1024

2026-03-21 11:49:43 +01:00

regulation_requirement_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

regulation_scraper.py

refactor: phase 0 guardrails + phase 1 step 2 (models.py split)

2026-04-07 13:18:29 +02:00

report_generator.py

refactor: phase 0 guardrails + phase 1 step 2 (models.py split)

2026-04-07 13:18:29 +02:00

reranker.py

feat(rag): optimize RAG pipeline — JSON-Mode, CoT, Hybrid Search, Re-Ranking, Cross-Reg Dedup, chunk 1024

2026-03-21 11:49:43 +01:00

screening_service.py

refactor(backend/api): extract ScreeningService (Step 4 — file 8 of 18)

2026-04-07 20:03:16 +02:00

seeder.py

fix(quality): Ruff/CVE/TS-Fixes, 104 neue Tests, Complexity-Refactoring

2026-03-07 19:00:33 +01:00

service_registry.py

fix: Add missing service_registry.py to main

2026-05-04 23:34:00 +02:00

similarity_detector.py

feat(canonical-controls): Canonical Control Library — rechtssichere Security Controls

2026-03-12 19:55:06 +01:00

smtp_sender.py

feat: ZeroClaw compliance agent — document analysis + role assignment + email

2026-04-27 23:28:21 +02:00

source_policy_service.py

refactor(backend/api): extract SourcePolicyService (Step 4 — file 7 of 18)

2026-04-07 19:58:02 +02:00

tom_service.py

refactor(backend/api): extract TOMService (Step 4 — file 3 of 18)

2026-04-07 19:42:17 +02:00

v1_enrichment.py

fix: V1 Enrichment Threshold auf 0.70 gesenkt (typische Top-Scores 0.70-0.77)

2026-03-26 11:13:37 +01:00

vendor_banner_sync.py

feat: Cookie-Banner ↔ Backend Integration (DSR, Retention, Consent Proof)

2026-05-02 19:52:04 +02:00

vendor_compliance_extra_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

vendor_compliance_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

vendor_compliance_sub_service.py

chore: mypy cleanup — comprehensive disable headers for agent-created services

2026-04-10 11:23:43 +02:00

vvt_service.py

refactor(backend/api): extract VVTService (Step 4 — file 5 of 18)

2026-04-07 19:50:40 +02:00

website_scanner.py

feat: website scanner with SOLL/IST service comparison + corrections

2026-04-28 15:35:31 +02:00