291 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Benjamin Admin	825e070ed9	feat(multi-layer): complete Multi-Layer Control Architecture (Phases 1-8 + Pass 0) ... Implements the full Multi-Layer Control Architecture for migrating ~25,000 Rich Controls into atomic, deduplicated Master Controls with full traceability. Architecture: Legal Source → Obligation → Control Pattern → Master Control → Customer Instance New services: - ObligationExtractor: 3-tier extraction (exact → embedding → LLM) - PatternMatcher: 2-tier matching (keyword + embedding + domain-bonus) - ControlComposer: Pattern + Obligation → Master Control - PipelineAdapter: Pipeline integration + Migration Passes 1-5 - DecompositionPass: Pass 0a/0b — Rich Control → atomic Controls - CrosswalkRoutes: 15 API endpoints under /v1/canonical/ New DB schema: - Migration 060: obligation_extractions, control_patterns, crosswalk_matrix - Migration 061: obligation_candidates, parent_control_uuid tracking Pattern Library: 50 YAML patterns (30 core + 20 IT-security) Go SDK: Pattern loader with YAML validation and indexing Documentation: MkDocs updated with full architecture overview 500 Python tests passing across all components. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-17 09:00:37 +01:00
Benjamin Admin	4f6bc8f6f6	feat(training+controls): interactive video pipeline, training blocks, control generator, CE libraries ... Interactive Training Videos (CP-TRAIN): - DB migration 022: training_checkpoints + checkpoint_progress tables - NarratorScript generation via Anthropic (AI Teacher persona, German) - TTS batch synthesis + interactive video pipeline (slides + checkpoint slides + FFmpeg) - 4 new API endpoints: generate-interactive, interactive-manifest, checkpoint submit, checkpoint progress - InteractiveVideoPlayer component (HTML5 Video, quiz overlay, seek protection, progress tracking) - Learner portal integration with automatic completion on all checkpoints passed - 30 new tests (handler validation + grading logic + manifest/progress + seek protection) Training Blocks: - Block generator, block store, block config CRUD + preview/generate endpoints - Migration 021: training_blocks schema Control Generator + Canonical Library: - Control generator routes + service enhancements - Canonical control library helpers, sidebar entry - Citation backfill service + tests - CE libraries data (hazard, protection, evidence, lifecycle, components) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 21:41:48 +01:00
Benjamin Admin	d2133dbfa2	test+docs(iace): add handler tests, error-handling tests, JSON export tests, TipTap docs ... - Create iace_handler_test.go (22 tests): input validation for InitFromProfile, GenerateSingleSection, ExportTechFile, CheckCompleteness, getTenantID, CreateProject, ListProjects, Component CRUD handlers - Add error-handling tests to tech_file_generator_test.go: nil context, nil project, empty components/hazards/classifications/evidence, unknown section type, all 19 getSystemPrompt types, AI-specific section prompts - Add JSON export tests to document_export_test.go: valid output, empty project, nil project error, special character handling (German text, XML escapes) - Add iace-hazard-library.md to mkdocs.yml navigation - Add TipTap Rich-Text-Editor section to iace.md documentation Total: 181 tests passing (was 165), 0 failures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 13:15:31 +01:00
Benjamin Admin	6d2de9b897	feat(iace): complete CE risk assessment — LLM tech-file generation, multi-format export, TipTap editor ... Phase 1: Fix completeness gates G23 (require verified/rejected mitigations) and G09 (audit trail check) Phase 2: LLM-based tech-file section generation with 19 German prompts and RAG enrichment Phase 3: Multi-format document export (PDF/Excel/DOCX/Markdown/JSON) Phase 4: Company profile → IACE data flow with auto component/classification creation Phase 5: TipTap WYSIWYG editor replacing textarea for tech-file sections Phase 6: User journey tests, developer portal API reference, updated documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 12:50:53 +01:00
Benjamin Admin	5adb1c5f16	feat(iace): integrate Rule Library as 58 extended hazard patterns (HP045-HP102) ... Parsed 171 explicit rules from 4 Rule Library Word documents (R051-R1550), deduplicated into 58 unique (component, energy_source) patterns, and mapped to existing IACE IDs (component tags, M-IDs, E-IDs). Changes: - hazard_patterns_extended.go: 58 new patterns derived from Rule Library - pattern_engine.go: combines builtin (44) + extended (58) = 102 total patterns - iace_handler.go: ListHazardPatterns returns all 102 patterns - iace.md: updated documentation for 102 patterns - scripts/generate-rule-patterns.py: mapping + Go code generator - scripts/parsed-rule-library.json: extracted rule data Tests: 132 passing (9 new extended pattern tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 11:24:07 +01:00
Benjamin Admin	9c1355c05f	feat(iace): Phase 5+6 — frontend integration, RAG library search, comprehensive tests ... Phase 5 — Frontend Integration: - components/page.tsx: ComponentLibraryModal with 120 components + 20 energy sources - hazards/page.tsx: AutoSuggestPanel with 3-column pattern matching review - mitigations/page.tsx: SuggestMeasuresModal per hazard with 3-level grouping - verification/page.tsx: SuggestEvidenceModal per mitigation with evidence types Phase 6 — RAG Library Search: - Added bp_iace_libraries to AllowedCollections whitelist in rag_handlers.go - SearchLibrary endpoint: POST /iace/library-search (semantic search across libraries) - EnrichTechFileSection endpoint: POST /projects/:id/tech-file/:section/enrich - Created ingest-iace-libraries.sh ingestion script for Qdrant collection Tests (123 passing): - tag_taxonomy_test.go: 8 tests for taxonomy entries, domains, essential tags - controls_library_test.go: 7 tests for measures, reduction types, subtypes - integration_test.go: 7 integration tests for full match flow and library consistency - Extended tag_resolver_test.go: 9 new tests for FindByTags and cross-category resolution Documentation: - Updated iace.md with Hazard-Matching-Engine, RAG enrichment, and new DB tables Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 10:22:49 +01:00
Benjamin Admin	3b2006ebce	feat(iace): add hazard-matching-engine with component library, tag system, and pattern engine ... Implements Phases 1-4 of the IACE Hazard-Matching-Engine: - 120 machine components (C001-C120) in 11 categories - 20 energy sources (EN01-EN20) - ~85 tag taxonomy across 5 domains - 44 hazard patterns with AND/NOT matching logic - Pattern engine with tag resolution and confidence scoring - 8 new API endpoints (component-library, energy-sources, tags, patterns, match/apply) - Completeness gate G09 for pattern matching - 320 tests passing (36 new) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 08:50:11 +01:00
Benjamin Admin	c7651796c9	feat(iace): integrate ISO 12100 machine risk model with 4-factor assessment ... Add dual-mode risk engine: legacy S×E×P (avoidance=0) and ISO mode S×F×P×A (avoidance>=1) with new thresholds (low/medium/high/very_high/not_acceptable). - 150+ hazard library entries across 28 categories incl. physical hazards (mechanical, electrical, thermal, pneumatic/hydraulic, noise/vibration, ergonomic, material/environmental) - 160-entry protective measures library with 3-step hierarchy validation (design → protective → information) - 25 lifecycle phases, 20 affected person roles, 50 evidence types - 10 verification methods (expanded from 7) - New API endpoints: lifecycle-phases, roles, evidence-types, protective-measures-library, validate-mitigation-hierarchy - DB migrations 018+019 for extended schema - Frontend: 4-slider risk assessment, hierarchy warnings, measures library modal - MkDocs wiki updated with ISO mode docs and legal notice (no norm text) All content uses original wording — norms referenced as methodology only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 23:13:41 +01:00
Benjamin Admin	c8fd9cc780	feat(control-library): document-grouped batching, generation strategy tracking, sort by source ... - Group chunks by regulation_code before batching for better LLM context - Add generation_strategy column (ungrouped=v1, document_grouped=v2) - Add v1/v2 badge to control cards in frontend - Add sort-by-source option with visual group headers - Add frontend page tests (18 tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 15:10:52 +01:00
Benjamin Admin	0d95c3bb44	feat(control-provenance): add filter explanations, badges, and updated taxonomy ... - Add new "Filter in der Control Library" section explaining all 7 dropdowns - Add "Badges & Lizenzregeln" section explaining Rule 1/2/3 and all badges - Update taxonomy with actual top-10 domains and counts (~3100+ controls) - Update Master Library strategy with current numbers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 10:45:21 +01:00
Benjamin Admin	f066cf1a03	feat(control-library): add document source dropdown filter ... Add "Dokumentenursprung" filter dropdown to the control library page. Extracts unique source_citation.source values from controls, sorted by frequency. Includes "Ohne Quelle" option for controls without source info. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 09:03:21 +01:00
Benjamin Admin	dd09fa7a46	feat: CRA wiki, cybersecurity policy template, Phase H RAG ingestion ... - Wiki: add CRA category with 3 articles (Grundlagen, 35 Security Controls, CRA+NIS2+AI Act Framework) - Document Generator: add CRA-konforme Cybersecurity Policy template with 21 sections covering governance, SSDLC, vulnerability management, incident response (24h/72h), SBOM, patch management - RAG: ingest Phase H — 17 EU regulations + 2 NIST frameworks now in Qdrant (CRA, AI Act, NIS2, DSGVO, DMA, GPSR, Batterieverordnung, etc.) - Phase H script: add scripts/ingest-phase-h.sh for reproducible ingestion - rag-sources.md: update status to ingestiert, add CRA entry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 00:43:46 +01:00
Benjamin Admin	f3e05c1bf7	feat: enhance whistleblower HinSchG content, fix control-library filter layout ... - Whistleblower page: expand overview tab with comprehensive HinSchG legal info (Gesetzliche Grundlage, Fristen-Cards, Anwendungsbereich, Schutz des Hinweisgebers) - StepHeader: enrich whistleblower tips with detailed HinSchG paragraphs and sanctions - Wiki: add migration 054 with 5 new/updated HinSchG articles (Anwendungsbereich, Hinweisgeberschutz, Meldestellen, Verfahrensablauf, Datenschutz-Anforderungen) - MKDocs: rewrite whistleblower docs with full legal basis, architecture, API, DB schema - Control library: fix filter dropdown overflow by splitting into search + filter rows Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 00:23:19 +01:00
Benjamin Admin	2ed1c08acf	feat: enhance legal basis display, add batch processing tests and docs ... - Backfill 81 controls with empty source_citation.source from generation_metadata - Add fallback to generation_metadata.source_regulation in ControlDetail blue box - Improve Rule 3 amber box text for reformulated controls - Add 30 new tests for batch processing (TestParseJsonArray, TestBatchSizeConfig, TestBatchProcessingLoop) — all 61 control generator tests passing - Fix stale test_config_defaults assertion (max_controls 50→0) - Update canonical-control-library.md with batch processing pipeline docs, processed chunks tracking, migration guide, and stats endpoint - Update testing.md with canonical control generator test section Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 23:51:52 +01:00
Benjamin Admin	4018b9af9b	chore: add coverage.out to .gitignore ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 22:55:05 +01:00
Benjamin Admin	a9f291ff49	test+docs: add policy library tests (67 tests) and MKDocs documentation ... - New test_policy_templates.py: 67 tests covering all 29 policy types, API creation, filtering, placeholders, seed script validation - Updated test_legal_template_routes.py: fix type count 16→52 - New MKDocs page policy-bibliothek.md with full template reference - Updated dokumentengenerierung.md and rechtliche-texte.md with cross-refs - Added policy-bibliothek to mkdocs.yml navigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 22:50:50 +01:00
Benjamin Admin	0171d611f6	feat: add policy library with 29 German policy templates ... Add 29 new document types (IT security, data, personnel, vendor, BCM policies) to VALID_DOCUMENT_TYPES and 5 category pills to the document generator UI. Include seed script for production DB population. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 22:37:33 +01:00
Benjamin Admin	637fab6fdb	fix: migration runner strips BEGIN/COMMIT and guards missing tables ... Root cause: migrations 046-047 used explicit BEGIN/COMMIT which conflicts with psycopg2 implicit transactions, and ALTER TABLE on canonical_controls fails when the table doesn't exist on production. This blocked all subsequent migrations (048-053). Changes: - migration_runner.py: strip BEGIN/COMMIT from SQL before executing - 046: wrap canonical_controls ALTER in DO $$ IF EXISTS block - 047: wrap canonical_controls ALTER in DO $$ IF EXISTS block Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:59:10 +01:00
Benjamin Admin	d462141ccd	fix: migration runner continues on failure instead of aborting ... Previously, a single failed migration would abort all subsequent migrations via raise RuntimeError. Now the runner logs the failure and continues with remaining migrations, so independent schema changes (e.g. 050-053) are not blocked by an unrelated failure in an earlier migration (e.g. 048). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:54:08 +01:00
Benjamin Admin	5f8aebf5b1	fix: make migrations 048/049 safe for environments without canonical tables ... Migrations 048 and 049 reference canonical_processed_chunks and canonical_controls tables which may not exist on all environments. Wrap ALTER TABLE statements in DO blocks that check for table existence first. This unblocks migrations 050-053 on production. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:45:00 +01:00
Benjamin Admin	c74f506415	fix: add API proxy routes for process-tasks and evidence-checks ... The frontend pages were calling /api/sdk/v1/compliance/process-tasks/* and /api/sdk/v1/compliance/evidence-checks/* but no Next.js proxy routes existed for these paths, causing 404s and empty data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:40:57 +01:00
Benjamin Admin	49ce417428	feat: add compliance modules 2-5 (dashboard, security templates, process manager, evidence collector) ... Module 2: Extended Compliance Dashboard with roadmap, module-status, next-actions, snapshots, score-history Module 3: 7 German security document templates (IT-Sicherheitskonzept, Datenschutz, Backup, Logging, Incident-Response, Zugriff, Risikomanagement) Module 4: Compliance Process Manager with CRUD, complete/skip/seed, ~50 seed tasks, 3-tab UI Module 5: Evidence Collector Extended with automated checks, control-mapping, coverage report, 4-tab UI Also includes: canonical control library enhancements (verification method, categories, dedup), control generator improvements, RAG client extensions 52 tests pass, frontend builds clean. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:03:04 +01:00
Benjamin Admin	13d13c8226	fix: add all RAG regulation codes to license mapping ... Many regulation codes (nist_sp800_53r5, eucsa, owasp_top10_2021, EDPB guidelines, EU laws, AT/FR/ES/NL/IT/HU laws) were defaulting to Rule 3 (restricted) because they weren't in REGULATION_LICENSE_MAP. Now all ~100 regulation codes from RAG are properly mapped to Rule 1 or 2. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 08:38:31 +01:00
Benjamin Admin	b6e6ffaaee	feat: add verification method, categories, and dedup UI to control library ... - Migration 047: verification_method + category columns, 17 category lookup table - Backend: new filters, GET /categories, GET /controls/{id}/similar (embedding-based) - Frontend: filter dropdowns, badges, dedup UI in ControlDetail with merge workflow - ControlForm: verification method + category selects - Provenance: verification methods, categories, master library strategy sections - Fix UUID cast syntax in generator routes (::uuid -> CAST) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 07:55:22 +01:00
Benjamin Admin	8a05fcc2f0	refactor: split control library into components, add generator UI ... - Extract ControlForm, ControlDetail, GeneratorModal, helpers into separate component files (max ~470 lines each, was 1210) - Add Collection selector in Generator modal - Add Job History view in Generator modal - Add Review Queue button with counter badge - Add review mode navigation (prev/next through review items) - Add vitest tests for helpers (getDomain, constants, options) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 18:52:42 +01:00
Benjamin Admin	9812ff46f3	feat: add 7-stage control generator pipeline with 3 license rules ... - control_generator.py: RAG→License→Structure/Reform→Harmonize→Anchor→Store→Mark pipeline with Anthropic Claude API (primary) + Ollama fallback for LLM reformulation - anchor_finder.py: RAG-based + DuckDuckGo anchor search for open references - control_generator_routes.py: REST API for generate, job status, review queue, processed stats - 046_control_generator.sql: job tracking, chunk tracking, blocked sources tables; extends canonical_controls with license_rule, source_original_text, source_citation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 18:42:40 +01:00
Benjamin Admin	30236c0001	docs: add post-push deploy monitoring to CLAUDE.md ... After every push to gitea, Claude now automatically polls health endpoints and notifies the user when the deployment is ready for testing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 13:39:12 +01:00
Benjamin Admin	b4d2be83eb	Merge gitea/main: resolve ci.yaml conflict, keep Coolify deploy ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 13:26:17 +01:00
Benjamin Admin	38c7cf0a00	Merge branch 'main' of ssh://gitea.meghsakha.com:22222/Benjamin_Boenisch/breakpilot-compliance	2026-03-13 13:23:30 +01:00
Benjamin Admin	399fa62267	docs: update all docs to reflect Coolify deployment model ... Replace Hetzner references with Coolify. Deployment is now: - Core + Compliance: Push gitea → Coolify auto-deploys - Lehrer: stays local on Mac Mini Updated: CLAUDE.md, MkDocs CI/CD pipeline, MkDocs index. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 12:09:51 +01:00
sharang	f1710fdb9e	fix: migrate deployment from Hetzner to Coolify (#1) ... ## Summary - Add Coolify deployment configuration (docker-compose, healthchecks, network setup) - Replace deploy-hetzner CI job with Coolify webhook deploy - Externalize postgres, qdrant, S3 for Coolify environment ## All changes since branch creation - Coolify docker-compose with Traefik labels and healthchecks - CI pipeline: deploy-hetzner → deploy-coolify (simple webhook curl) - SQLAlchemy 2.x text() compatibility fixes - Alpine-compatible Dockerfile fixes Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com> Reviewed-on: #1	2026-03-13 10:45:35 +00:00
Benjamin Admin	499ddc04d5	chore: trigger redeploy via Gitea Actions CI/CD ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:54:23 +01:00
Benjamin Admin	f738ca8c52	fix: make compliance router imports resilient to individual module failures ... Replaced bare imports with safe_import_router pattern — if one sub-router fails to import (e.g. missing dependency), other routers still load. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:46:52 +01:00
Benjamin Admin	c530898963	fix: replace Python 3.10+ union type syntax with typing.Optional for Pydantic v2 compat ... from __future__ import annotations breaks Pydantic BaseModel runtime type evaluation. Replaced str | None → Optional[str], list[str] → List[str] etc. in control_generator.py, anchor_finder.py, control_generator_routes.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:36:14 +01:00
Benjamin Admin	cdafc4d9f4	feat: auto-run SQL migrations on backend startup ... Adds migration_runner.py that executes pending migrations from migrations/ directory when backend-compliance starts. Tracks applied migrations in _migration_history table. Handles existing databases: detects if tables from migrations 001-045 already exist and seeds the history table accordingly, so only new migrations (046+) are applied. Skippable via SKIP_MIGRATIONS=true env var. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:14:18 +01:00
Benjamin Admin	de19ef0684	feat(control-generator): 7-stage pipeline for RAG→LLM→Controls generation ... Implements the Control Generator Pipeline that systematically generates canonical security controls from 150k+ RAG chunks across all compliance collections (BSI, NIST, OWASP, ENISA, EU laws, German laws). Three license rules enforced throughout: - Rule 1 (free_use): Laws/Public Domain — original text preserved - Rule 2 (citation_required): CC-BY/CC-BY-SA — text with citation - Rule 3 (restricted): BSI/ISO — full reformulation, no source traces New files: - Migration 046: job tracking, chunk tracking, blocked sources tables - control_generator.py: 7-stage pipeline (scan→classify→structure/reform→harmonize→anchor→store→mark) - anchor_finder.py: RAG + DuckDuckGo open-source reference search - control_generator_routes.py: REST API (generate, review, stats, blocked-sources) - test_control_generator.py: license mapping, rule enforcement, anchor filtering tests Modified: - __init__.py: register control_generator_router - route.ts: proxy generator/review/stats endpoints - page.tsx: Generator modal, stats panel, state filter, review queue, license badges Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:03:37 +01:00
Benjamin Admin	c87f07c99a	feat: seed 10 canonical controls + CRUD endpoints + frontend editor ... - Migration 045: Seed 10 controls (AUTH, NET, SUP, LOG, WEB, DATA, CRYP, REL) with 39 open-source anchors into the database - Backend: POST/PUT/DELETE endpoints for canonical controls CRUD - Frontend proxy: PUT and DELETE methods added to canonical route - Frontend: Control Library with create/edit/delete UI, full form with open anchor management, scope, requirements, evidence, test procedures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 00:28:21 +01:00
Benjamin Admin	453eec9ed8	fix: correct canonical control proxy paths to include /compliance prefix ... The backend mounts the compliance router at /api/compliance, so canonical control endpoints are at /api/compliance/v1/canonical/*, not /api/v1/canonical/*. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 20:49:06 +01:00
Benjamin Admin	050f353192	feat(canonical-controls): Canonical Control Library — rechtssichere Security Controls ... Eigenstaendig formulierte Security Controls mit unabhaengiger Taxonomie und Open-Source-Verankerung (OWASP, NIST, ENISA). Keine BSI-Nomenklatur. - Migration 044: 5 DB-Tabellen (frameworks, controls, sources, licenses, mappings) - 10 Seed Controls mit 39 Open-Source-Referenzen - License Gate: Quellen-Berechtigungspruefung (analysis/excerpt/embeddings/product) - Too-Close-Detektor: 5 Metriken (exact-phrase, token-overlap, ngram, embedding, LCS) - REST API: 8 Endpoints unter /v1/canonical/ - Go Loader mit Multi-Index (ID, domain, severity, framework) - Frontend: Control Library Browser + Provenance Wiki - CI/CD: validate-controls.py Job (schema, no-leak, open-anchors) - 67 Tests (8 Go + 59 Python), alle PASS - MkDocs Dokumentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 19:55:06 +01:00
Benjamin Admin	8442115e7c	fix(rag): Fix bash compatibility + missing mkdir in phase functions ... - Replace ${var,,} (bash 4+) with $(echo | tr) for macOS bash 3.2 compat - Add mkdir -p to phase_gesetze, phase_eu, phase_templates, phase_datenschutz, phase_dach — prevents download failures when running phases individually Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 16:44:15 +01:00
Benjamin Admin	999cc81c78	feat(rag): Phase J — Security Guidelines & Standards (NIST, OWASP, ENISA) ... Add phase_security() with 15 documents across 3 sub-phases: - J1: 7 NIST standards (SP 800-53, 800-218, 800-63, 800-207, 8259A/B, AI RMF) - J2: 6 OWASP projects (Top 10, API Security, ASVS, MASVS, SAMM, Mobile Top 10) - J3: 2 ENISA guides (Procurement Hospitals, Cloud Security SMEs) All documents are commercially licensed (Public Domain / CC BY / CC BY-SA). Wire up 'security' phase in dispatcher and workflow yaml. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 16:14:44 +01:00
Benjamin Admin	ff66612beb	fix(rag): Make download failures non-fatal — prevent set -e from aborting entire ingestion ... download_pdf() and extract_gesetz_html() now return 0 on failure and clean up partial files. This prevents set -euo pipefail from aborting the entire script when a single download fails (e.g. EUR-Lex timeout, BSI redirect). Root cause of H2 EU loop only processing 1 document in Run #724: first failed download_pdf returned 1, triggering set -e script abort. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 15:56:23 +01:00
Benjamin Admin	42ec3cad6d	feat(rag): Phase I DACH-Erweiterung — Gesetze, Templates, Urteile ... New ingestion phase 'dach' adds missing documents from DACH catalog: I1: UStG (Retention), MStV (Impressum) I2: DSK Muster-VVT, DSK KP5 DSFA, BfDI Beispiel-VVT (DL-DE/BY-2.0) I3: BSI IT-Grundschutz Kompendium 2024 (CC BY-SA 4.0) I4: 7 Gerichtsentscheidungen as Praxisanker: - DE: LG Bonn 1&1, BGH Planet49, BGH Art.82 (2x) - AT: OGH Schutzzweck, OGH Art.15+82 EuGH-Vorlage - CH: BVGer DSG-Auskunft, BGer Datensperre Trigger: workflow_dispatch phase=dach Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 14:36:59 +01:00
Benjamin Admin	9945a62a50	fix(rag): docker cp into /workspace_scripts, then copy at runtime ... docker cp fails when target dir doesn't exist in a created container. Copy scripts to /workspace_scripts, then cp them at container start. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 14:24:36 +01:00
Benjamin Admin	eef1c2e7d3	fix(rag): Use docker cp to inject checked-out scripts ... The runner container can't access host paths directly, so the deploy dir scripts were always stale. Now uses docker create + docker cp + docker start to copy the freshly checked-out scripts into the ingestion container before starting it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 13:37:57 +01:00
Benjamin Admin	a0e2a35e66	fix(rag): Git pull deploy dir before ingestion ... The RAG workflow mounts scripts from /opt/breakpilot-compliance/scripts (deploy dir) but this may not have the latest fixes if CI hasn't deployed yet. Add explicit git pull before running ingestion. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 13:13:33 +01:00
Benjamin Admin	57f390190d	fix(rag): Arithmetic error, dedup auth, EGBGB timeout ... - collection_count() returns 0 (not ?) on failure — fixes arithmetic error - Pass QDRANT_API_KEY to ingestion container for dedup checks - Include api-key header in collection_count() and dedup scroll queries - Lower large-file threshold to 256KB (EGBGB 310KB was timing out) - More targeted EGBGB XML extraction (Art. 246a + Anlage only) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 12:05:07 +01:00
Benjamin Admin	cf60c39658	fix(scope-engine): Normalize UPPERCASE trigger docs to lowercase ScopeDocumentType ... Critical bug fix: mandatoryDocuments in Hard-Trigger-Rules used UPPERCASE names (VVT, TOM, DSE) that never matched lowercase ScopeDocumentType keys (vvt, tom, dsi). This meant no trigger documents were ever recognized as mandatory in buildDocumentScope(). - Add normalizeDocType() mapping function with alias support (DSE→dsi, LOESCHKONZEPT→lf, DSR_PROZESS→betroffenenrechte, etc.) - Fix buildDocumentScope() to use normalized doc types - Fix estimateEffort() to use lowercase keys matching ScopeDocumentType - Add 2 tests for UPPERCASE normalization and alias resolution Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 09:39:31 +01:00
Benjamin Admin	c88653b221	fix(rag): Dedup check, BGB split, GewO timeout, arithmetic fix ... - Add Qdrant dedup check in upload_file() — skip if regulation_id already exists - Split BGB (2.7MB) into 5 targeted parts via XML extraction: AGB §§305-310, Fernabsatz §§312-312k, Kaufrecht §§433-480, Widerruf §§355-361, Digitale Produkte §§327-327u - Lower large-file threshold 512KB→384KB (fixes GewO 432KB timeout) - Fix arithmetic syntax error when collection_count returns "?" - Replace EGBGB PDF (was empty) with XML extraction - Add unzip to Alpine container for XML archives Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 09:39:09 +01:00
Benjamin Admin	87d06c8b20	fix(rag): Handle large file uploads + don't abort on individual failures ... - Extended timeout (15 min) for files > 500KB (BGB is 1.5MB) - upload_file returns 0 even on failure so set -e doesn't kill script - Failed uploads are still counted and reported in summary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-11 23:33:28 +01:00