290 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Benjamin Admin	4f6bc8f6f6	feat(training+controls): interactive video pipeline, training blocks, control generator, CE libraries ... Interactive Training Videos (CP-TRAIN): - DB migration 022: training_checkpoints + checkpoint_progress tables - NarratorScript generation via Anthropic (AI Teacher persona, German) - TTS batch synthesis + interactive video pipeline (slides + checkpoint slides + FFmpeg) - 4 new API endpoints: generate-interactive, interactive-manifest, checkpoint submit, checkpoint progress - InteractiveVideoPlayer component (HTML5 Video, quiz overlay, seek protection, progress tracking) - Learner portal integration with automatic completion on all checkpoints passed - 30 new tests (handler validation + grading logic + manifest/progress + seek protection) Training Blocks: - Block generator, block store, block config CRUD + preview/generate endpoints - Migration 021: training_blocks schema Control Generator + Canonical Library: - Control generator routes + service enhancements - Canonical control library helpers, sidebar entry - Citation backfill service + tests - CE libraries data (hazard, protection, evidence, lifecycle, components) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 21:41:48 +01:00
Benjamin Admin	d2133dbfa2	test+docs(iace): add handler tests, error-handling tests, JSON export tests, TipTap docs ... - Create iace_handler_test.go (22 tests): input validation for InitFromProfile, GenerateSingleSection, ExportTechFile, CheckCompleteness, getTenantID, CreateProject, ListProjects, Component CRUD handlers - Add error-handling tests to tech_file_generator_test.go: nil context, nil project, empty components/hazards/classifications/evidence, unknown section type, all 19 getSystemPrompt types, AI-specific section prompts - Add JSON export tests to document_export_test.go: valid output, empty project, nil project error, special character handling (German text, XML escapes) - Add iace-hazard-library.md to mkdocs.yml navigation - Add TipTap Rich-Text-Editor section to iace.md documentation Total: 181 tests passing (was 165), 0 failures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 13:15:31 +01:00
Benjamin Admin	6d2de9b897	feat(iace): complete CE risk assessment — LLM tech-file generation, multi-format export, TipTap editor ... Phase 1: Fix completeness gates G23 (require verified/rejected mitigations) and G09 (audit trail check) Phase 2: LLM-based tech-file section generation with 19 German prompts and RAG enrichment Phase 3: Multi-format document export (PDF/Excel/DOCX/Markdown/JSON) Phase 4: Company profile → IACE data flow with auto component/classification creation Phase 5: TipTap WYSIWYG editor replacing textarea for tech-file sections Phase 6: User journey tests, developer portal API reference, updated documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 12:50:53 +01:00
Benjamin Admin	5adb1c5f16	feat(iace): integrate Rule Library as 58 extended hazard patterns (HP045-HP102) ... Parsed 171 explicit rules from 4 Rule Library Word documents (R051-R1550), deduplicated into 58 unique (component, energy_source) patterns, and mapped to existing IACE IDs (component tags, M-IDs, E-IDs). Changes: - hazard_patterns_extended.go: 58 new patterns derived from Rule Library - pattern_engine.go: combines builtin (44) + extended (58) = 102 total patterns - iace_handler.go: ListHazardPatterns returns all 102 patterns - iace.md: updated documentation for 102 patterns - scripts/generate-rule-patterns.py: mapping + Go code generator - scripts/parsed-rule-library.json: extracted rule data Tests: 132 passing (9 new extended pattern tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 11:24:07 +01:00
Benjamin Admin	9c1355c05f	feat(iace): Phase 5+6 — frontend integration, RAG library search, comprehensive tests ... Phase 5 — Frontend Integration: - components/page.tsx: ComponentLibraryModal with 120 components + 20 energy sources - hazards/page.tsx: AutoSuggestPanel with 3-column pattern matching review - mitigations/page.tsx: SuggestMeasuresModal per hazard with 3-level grouping - verification/page.tsx: SuggestEvidenceModal per mitigation with evidence types Phase 6 — RAG Library Search: - Added bp_iace_libraries to AllowedCollections whitelist in rag_handlers.go - SearchLibrary endpoint: POST /iace/library-search (semantic search across libraries) - EnrichTechFileSection endpoint: POST /projects/:id/tech-file/:section/enrich - Created ingest-iace-libraries.sh ingestion script for Qdrant collection Tests (123 passing): - tag_taxonomy_test.go: 8 tests for taxonomy entries, domains, essential tags - controls_library_test.go: 7 tests for measures, reduction types, subtypes - integration_test.go: 7 integration tests for full match flow and library consistency - Extended tag_resolver_test.go: 9 new tests for FindByTags and cross-category resolution Documentation: - Updated iace.md with Hazard-Matching-Engine, RAG enrichment, and new DB tables Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 10:22:49 +01:00
Benjamin Admin	3b2006ebce	feat(iace): add hazard-matching-engine with component library, tag system, and pattern engine ... Implements Phases 1-4 of the IACE Hazard-Matching-Engine: - 120 machine components (C001-C120) in 11 categories - 20 energy sources (EN01-EN20) - ~85 tag taxonomy across 5 domains - 44 hazard patterns with AND/NOT matching logic - Pattern engine with tag resolution and confidence scoring - 8 new API endpoints (component-library, energy-sources, tags, patterns, match/apply) - Completeness gate G09 for pattern matching - 320 tests passing (36 new) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 08:50:11 +01:00
Benjamin Admin	c7651796c9	feat(iace): integrate ISO 12100 machine risk model with 4-factor assessment ... Add dual-mode risk engine: legacy S×E×P (avoidance=0) and ISO mode S×F×P×A (avoidance>=1) with new thresholds (low/medium/high/very_high/not_acceptable). - 150+ hazard library entries across 28 categories incl. physical hazards (mechanical, electrical, thermal, pneumatic/hydraulic, noise/vibration, ergonomic, material/environmental) - 160-entry protective measures library with 3-step hierarchy validation (design → protective → information) - 25 lifecycle phases, 20 affected person roles, 50 evidence types - 10 verification methods (expanded from 7) - New API endpoints: lifecycle-phases, roles, evidence-types, protective-measures-library, validate-mitigation-hierarchy - DB migrations 018+019 for extended schema - Frontend: 4-slider risk assessment, hierarchy warnings, measures library modal - MkDocs wiki updated with ISO mode docs and legal notice (no norm text) All content uses original wording — norms referenced as methodology only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 23:13:41 +01:00
Benjamin Admin	c8fd9cc780	feat(control-library): document-grouped batching, generation strategy tracking, sort by source ... - Group chunks by regulation_code before batching for better LLM context - Add generation_strategy column (ungrouped=v1, document_grouped=v2) - Add v1/v2 badge to control cards in frontend - Add sort-by-source option with visual group headers - Add frontend page tests (18 tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 15:10:52 +01:00
Benjamin Admin	0d95c3bb44	feat(control-provenance): add filter explanations, badges, and updated taxonomy ... - Add new "Filter in der Control Library" section explaining all 7 dropdowns - Add "Badges & Lizenzregeln" section explaining Rule 1/2/3 and all badges - Update taxonomy with actual top-10 domains and counts (~3100+ controls) - Update Master Library strategy with current numbers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 10:45:21 +01:00
Benjamin Admin	f066cf1a03	feat(control-library): add document source dropdown filter ... Add "Dokumentenursprung" filter dropdown to the control library page. Extracts unique source_citation.source values from controls, sorted by frequency. Includes "Ohne Quelle" option for controls without source info. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 09:03:21 +01:00
Benjamin Admin	dd09fa7a46	feat: CRA wiki, cybersecurity policy template, Phase H RAG ingestion ... - Wiki: add CRA category with 3 articles (Grundlagen, 35 Security Controls, CRA+NIS2+AI Act Framework) - Document Generator: add CRA-konforme Cybersecurity Policy template with 21 sections covering governance, SSDLC, vulnerability management, incident response (24h/72h), SBOM, patch management - RAG: ingest Phase H — 17 EU regulations + 2 NIST frameworks now in Qdrant (CRA, AI Act, NIS2, DSGVO, DMA, GPSR, Batterieverordnung, etc.) - Phase H script: add scripts/ingest-phase-h.sh for reproducible ingestion - rag-sources.md: update status to ingestiert, add CRA entry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 00:43:46 +01:00
Benjamin Admin	f3e05c1bf7	feat: enhance whistleblower HinSchG content, fix control-library filter layout ... - Whistleblower page: expand overview tab with comprehensive HinSchG legal info (Gesetzliche Grundlage, Fristen-Cards, Anwendungsbereich, Schutz des Hinweisgebers) - StepHeader: enrich whistleblower tips with detailed HinSchG paragraphs and sanctions - Wiki: add migration 054 with 5 new/updated HinSchG articles (Anwendungsbereich, Hinweisgeberschutz, Meldestellen, Verfahrensablauf, Datenschutz-Anforderungen) - MKDocs: rewrite whistleblower docs with full legal basis, architecture, API, DB schema - Control library: fix filter dropdown overflow by splitting into search + filter rows Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-15 00:23:19 +01:00
Benjamin Admin	2ed1c08acf	feat: enhance legal basis display, add batch processing tests and docs ... - Backfill 81 controls with empty source_citation.source from generation_metadata - Add fallback to generation_metadata.source_regulation in ControlDetail blue box - Improve Rule 3 amber box text for reformulated controls - Add 30 new tests for batch processing (TestParseJsonArray, TestBatchSizeConfig, TestBatchProcessingLoop) — all 61 control generator tests passing - Fix stale test_config_defaults assertion (max_controls 50→0) - Update canonical-control-library.md with batch processing pipeline docs, processed chunks tracking, migration guide, and stats endpoint - Update testing.md with canonical control generator test section Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 23:51:52 +01:00
Benjamin Admin	4018b9af9b	chore: add coverage.out to .gitignore ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 22:55:05 +01:00
Benjamin Admin	a9f291ff49	test+docs: add policy library tests (67 tests) and MKDocs documentation ... - New test_policy_templates.py: 67 tests covering all 29 policy types, API creation, filtering, placeholders, seed script validation - Updated test_legal_template_routes.py: fix type count 16→52 - New MKDocs page policy-bibliothek.md with full template reference - Updated dokumentengenerierung.md and rechtliche-texte.md with cross-refs - Added policy-bibliothek to mkdocs.yml navigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 22:50:50 +01:00
Benjamin Admin	0171d611f6	feat: add policy library with 29 German policy templates ... Add 29 new document types (IT security, data, personnel, vendor, BCM policies) to VALID_DOCUMENT_TYPES and 5 category pills to the document generator UI. Include seed script for production DB population. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 22:37:33 +01:00
Benjamin Admin	637fab6fdb	fix: migration runner strips BEGIN/COMMIT and guards missing tables ... Root cause: migrations 046-047 used explicit BEGIN/COMMIT which conflicts with psycopg2 implicit transactions, and ALTER TABLE on canonical_controls fails when the table doesn't exist on production. This blocked all subsequent migrations (048-053). Changes: - migration_runner.py: strip BEGIN/COMMIT from SQL before executing - 046: wrap canonical_controls ALTER in DO $$ IF EXISTS block - 047: wrap canonical_controls ALTER in DO $$ IF EXISTS block Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:59:10 +01:00
Benjamin Admin	d462141ccd	fix: migration runner continues on failure instead of aborting ... Previously, a single failed migration would abort all subsequent migrations via raise RuntimeError. Now the runner logs the failure and continues with remaining migrations, so independent schema changes (e.g. 050-053) are not blocked by an unrelated failure in an earlier migration (e.g. 048). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:54:08 +01:00
Benjamin Admin	5f8aebf5b1	fix: make migrations 048/049 safe for environments without canonical tables ... Migrations 048 and 049 reference canonical_processed_chunks and canonical_controls tables which may not exist on all environments. Wrap ALTER TABLE statements in DO blocks that check for table existence first. This unblocks migrations 050-053 on production. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:45:00 +01:00
Benjamin Admin	c74f506415	fix: add API proxy routes for process-tasks and evidence-checks ... The frontend pages were calling /api/sdk/v1/compliance/process-tasks/* and /api/sdk/v1/compliance/evidence-checks/* but no Next.js proxy routes existed for these paths, causing 404s and empty data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:40:57 +01:00
Benjamin Admin	49ce417428	feat: add compliance modules 2-5 (dashboard, security templates, process manager, evidence collector) ... Module 2: Extended Compliance Dashboard with roadmap, module-status, next-actions, snapshots, score-history Module 3: 7 German security document templates (IT-Sicherheitskonzept, Datenschutz, Backup, Logging, Incident-Response, Zugriff, Risikomanagement) Module 4: Compliance Process Manager with CRUD, complete/skip/seed, ~50 seed tasks, 3-tab UI Module 5: Evidence Collector Extended with automated checks, control-mapping, coverage report, 4-tab UI Also includes: canonical control library enhancements (verification method, categories, dedup), control generator improvements, RAG client extensions 52 tests pass, frontend builds clean. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 21:03:04 +01:00
Benjamin Admin	13d13c8226	fix: add all RAG regulation codes to license mapping ... Many regulation codes (nist_sp800_53r5, eucsa, owasp_top10_2021, EDPB guidelines, EU laws, AT/FR/ES/NL/IT/HU laws) were defaulting to Rule 3 (restricted) because they weren't in REGULATION_LICENSE_MAP. Now all ~100 regulation codes from RAG are properly mapped to Rule 1 or 2. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 08:38:31 +01:00
Benjamin Admin	b6e6ffaaee	feat: add verification method, categories, and dedup UI to control library ... - Migration 047: verification_method + category columns, 17 category lookup table - Backend: new filters, GET /categories, GET /controls/{id}/similar (embedding-based) - Frontend: filter dropdowns, badges, dedup UI in ControlDetail with merge workflow - ControlForm: verification method + category selects - Provenance: verification methods, categories, master library strategy sections - Fix UUID cast syntax in generator routes (::uuid -> CAST) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-14 07:55:22 +01:00
Benjamin Admin	8a05fcc2f0	refactor: split control library into components, add generator UI ... - Extract ControlForm, ControlDetail, GeneratorModal, helpers into separate component files (max ~470 lines each, was 1210) - Add Collection selector in Generator modal - Add Job History view in Generator modal - Add Review Queue button with counter badge - Add review mode navigation (prev/next through review items) - Add vitest tests for helpers (getDomain, constants, options) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 18:52:42 +01:00
Benjamin Admin	9812ff46f3	feat: add 7-stage control generator pipeline with 3 license rules ... - control_generator.py: RAG→License→Structure/Reform→Harmonize→Anchor→Store→Mark pipeline with Anthropic Claude API (primary) + Ollama fallback for LLM reformulation - anchor_finder.py: RAG-based + DuckDuckGo anchor search for open references - control_generator_routes.py: REST API for generate, job status, review queue, processed stats - 046_control_generator.sql: job tracking, chunk tracking, blocked sources tables; extends canonical_controls with license_rule, source_original_text, source_citation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 18:42:40 +01:00
Benjamin Admin	30236c0001	docs: add post-push deploy monitoring to CLAUDE.md ... After every push to gitea, Claude now automatically polls health endpoints and notifies the user when the deployment is ready for testing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 13:39:12 +01:00
Benjamin Admin	b4d2be83eb	Merge gitea/main: resolve ci.yaml conflict, keep Coolify deploy ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 13:26:17 +01:00
Benjamin Admin	38c7cf0a00	Merge branch 'main' of ssh://gitea.meghsakha.com:22222/Benjamin_Boenisch/breakpilot-compliance	2026-03-13 13:23:30 +01:00
Benjamin Admin	399fa62267	docs: update all docs to reflect Coolify deployment model ... Replace Hetzner references with Coolify. Deployment is now: - Core + Compliance: Push gitea → Coolify auto-deploys - Lehrer: stays local on Mac Mini Updated: CLAUDE.md, MkDocs CI/CD pipeline, MkDocs index. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 12:09:51 +01:00
Sharang Parnerkar	f1710fdb9e	fix: migrate deployment from Hetzner to Coolify (#1) ... ## Summary - Add Coolify deployment configuration (docker-compose, healthchecks, network setup) - Replace deploy-hetzner CI job with Coolify webhook deploy - Externalize postgres, qdrant, S3 for Coolify environment ## All changes since branch creation - Coolify docker-compose with Traefik labels and healthchecks - CI pipeline: deploy-hetzner → deploy-coolify (simple webhook curl) - SQLAlchemy 2.x text() compatibility fixes - Alpine-compatible Dockerfile fixes Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com> Reviewed-on: #1	2026-03-13 10:45:35 +00:00
Benjamin Admin	499ddc04d5	chore: trigger redeploy via Gitea Actions CI/CD ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:54:23 +01:00
Benjamin Admin	f738ca8c52	fix: make compliance router imports resilient to individual module failures ... Replaced bare imports with safe_import_router pattern — if one sub-router fails to import (e.g. missing dependency), other routers still load. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:46:52 +01:00
Benjamin Admin	c530898963	fix: replace Python 3.10+ union type syntax with typing.Optional for Pydantic v2 compat ... from __future__ import annotations breaks Pydantic BaseModel runtime type evaluation. Replaced str | None → Optional[str], list[str] → List[str] etc. in control_generator.py, anchor_finder.py, control_generator_routes.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:36:14 +01:00
Benjamin Admin	cdafc4d9f4	feat: auto-run SQL migrations on backend startup ... Adds migration_runner.py that executes pending migrations from migrations/ directory when backend-compliance starts. Tracks applied migrations in _migration_history table. Handles existing databases: detects if tables from migrations 001-045 already exist and seeds the history table accordingly, so only new migrations (046+) are applied. Skippable via SKIP_MIGRATIONS=true env var. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:14:18 +01:00
Benjamin Admin	de19ef0684	feat(control-generator): 7-stage pipeline for RAG→LLM→Controls generation ... Implements the Control Generator Pipeline that systematically generates canonical security controls from 150k+ RAG chunks across all compliance collections (BSI, NIST, OWASP, ENISA, EU laws, German laws). Three license rules enforced throughout: - Rule 1 (free_use): Laws/Public Domain — original text preserved - Rule 2 (citation_required): CC-BY/CC-BY-SA — text with citation - Rule 3 (restricted): BSI/ISO — full reformulation, no source traces New files: - Migration 046: job tracking, chunk tracking, blocked sources tables - control_generator.py: 7-stage pipeline (scan→classify→structure/reform→harmonize→anchor→store→mark) - anchor_finder.py: RAG + DuckDuckGo open-source reference search - control_generator_routes.py: REST API (generate, review, stats, blocked-sources) - test_control_generator.py: license mapping, rule enforcement, anchor filtering tests Modified: - __init__.py: register control_generator_router - route.ts: proxy generator/review/stats endpoints - page.tsx: Generator modal, stats panel, state filter, review queue, license badges Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 09:03:37 +01:00
Benjamin Admin	c87f07c99a	feat: seed 10 canonical controls + CRUD endpoints + frontend editor ... - Migration 045: Seed 10 controls (AUTH, NET, SUP, LOG, WEB, DATA, CRYP, REL) with 39 open-source anchors into the database - Backend: POST/PUT/DELETE endpoints for canonical controls CRUD - Frontend proxy: PUT and DELETE methods added to canonical route - Frontend: Control Library with create/edit/delete UI, full form with open anchor management, scope, requirements, evidence, test procedures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 00:28:21 +01:00
Benjamin Admin	453eec9ed8	fix: correct canonical control proxy paths to include /compliance prefix ... The backend mounts the compliance router at /api/compliance, so canonical control endpoints are at /api/compliance/v1/canonical/*, not /api/v1/canonical/*. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 20:49:06 +01:00
Benjamin Admin	050f353192	feat(canonical-controls): Canonical Control Library — rechtssichere Security Controls ... Eigenstaendig formulierte Security Controls mit unabhaengiger Taxonomie und Open-Source-Verankerung (OWASP, NIST, ENISA). Keine BSI-Nomenklatur. - Migration 044: 5 DB-Tabellen (frameworks, controls, sources, licenses, mappings) - 10 Seed Controls mit 39 Open-Source-Referenzen - License Gate: Quellen-Berechtigungspruefung (analysis/excerpt/embeddings/product) - Too-Close-Detektor: 5 Metriken (exact-phrase, token-overlap, ngram, embedding, LCS) - REST API: 8 Endpoints unter /v1/canonical/ - Go Loader mit Multi-Index (ID, domain, severity, framework) - Frontend: Control Library Browser + Provenance Wiki - CI/CD: validate-controls.py Job (schema, no-leak, open-anchors) - 67 Tests (8 Go + 59 Python), alle PASS - MkDocs Dokumentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 19:55:06 +01:00
Benjamin Admin	8442115e7c	fix(rag): Fix bash compatibility + missing mkdir in phase functions ... - Replace ${var,,} (bash 4+) with $(echo | tr) for macOS bash 3.2 compat - Add mkdir -p to phase_gesetze, phase_eu, phase_templates, phase_datenschutz, phase_dach — prevents download failures when running phases individually Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 16:44:15 +01:00
Benjamin Admin	999cc81c78	feat(rag): Phase J — Security Guidelines & Standards (NIST, OWASP, ENISA) ... Add phase_security() with 15 documents across 3 sub-phases: - J1: 7 NIST standards (SP 800-53, 800-218, 800-63, 800-207, 8259A/B, AI RMF) - J2: 6 OWASP projects (Top 10, API Security, ASVS, MASVS, SAMM, Mobile Top 10) - J3: 2 ENISA guides (Procurement Hospitals, Cloud Security SMEs) All documents are commercially licensed (Public Domain / CC BY / CC BY-SA). Wire up 'security' phase in dispatcher and workflow yaml. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 16:14:44 +01:00
Benjamin Admin	ff66612beb	fix(rag): Make download failures non-fatal — prevent set -e from aborting entire ingestion ... download_pdf() and extract_gesetz_html() now return 0 on failure and clean up partial files. This prevents set -euo pipefail from aborting the entire script when a single download fails (e.g. EUR-Lex timeout, BSI redirect). Root cause of H2 EU loop only processing 1 document in Run #724: first failed download_pdf returned 1, triggering set -e script abort. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 15:56:23 +01:00
Benjamin Admin	42ec3cad6d	feat(rag): Phase I DACH-Erweiterung — Gesetze, Templates, Urteile ... New ingestion phase 'dach' adds missing documents from DACH catalog: I1: UStG (Retention), MStV (Impressum) I2: DSK Muster-VVT, DSK KP5 DSFA, BfDI Beispiel-VVT (DL-DE/BY-2.0) I3: BSI IT-Grundschutz Kompendium 2024 (CC BY-SA 4.0) I4: 7 Gerichtsentscheidungen as Praxisanker: - DE: LG Bonn 1&1, BGH Planet49, BGH Art.82 (2x) - AT: OGH Schutzzweck, OGH Art.15+82 EuGH-Vorlage - CH: BVGer DSG-Auskunft, BGer Datensperre Trigger: workflow_dispatch phase=dach Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 14:36:59 +01:00
Benjamin Admin	9945a62a50	fix(rag): docker cp into /workspace_scripts, then copy at runtime ... docker cp fails when target dir doesn't exist in a created container. Copy scripts to /workspace_scripts, then cp them at container start. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 14:24:36 +01:00
Benjamin Admin	eef1c2e7d3	fix(rag): Use docker cp to inject checked-out scripts ... The runner container can't access host paths directly, so the deploy dir scripts were always stale. Now uses docker create + docker cp + docker start to copy the freshly checked-out scripts into the ingestion container before starting it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 13:37:57 +01:00
Benjamin Admin	a0e2a35e66	fix(rag): Git pull deploy dir before ingestion ... The RAG workflow mounts scripts from /opt/breakpilot-compliance/scripts (deploy dir) but this may not have the latest fixes if CI hasn't deployed yet. Add explicit git pull before running ingestion. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 13:13:33 +01:00
Benjamin Admin	57f390190d	fix(rag): Arithmetic error, dedup auth, EGBGB timeout ... - collection_count() returns 0 (not ?) on failure — fixes arithmetic error - Pass QDRANT_API_KEY to ingestion container for dedup checks - Include api-key header in collection_count() and dedup scroll queries - Lower large-file threshold to 256KB (EGBGB 310KB was timing out) - More targeted EGBGB XML extraction (Art. 246a + Anlage only) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 12:05:07 +01:00
Benjamin Admin	cf60c39658	fix(scope-engine): Normalize UPPERCASE trigger docs to lowercase ScopeDocumentType ... Critical bug fix: mandatoryDocuments in Hard-Trigger-Rules used UPPERCASE names (VVT, TOM, DSE) that never matched lowercase ScopeDocumentType keys (vvt, tom, dsi). This meant no trigger documents were ever recognized as mandatory in buildDocumentScope(). - Add normalizeDocType() mapping function with alias support (DSE→dsi, LOESCHKONZEPT→lf, DSR_PROZESS→betroffenenrechte, etc.) - Fix buildDocumentScope() to use normalized doc types - Fix estimateEffort() to use lowercase keys matching ScopeDocumentType - Add 2 tests for UPPERCASE normalization and alias resolution Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 09:39:31 +01:00
Benjamin Admin	c88653b221	fix(rag): Dedup check, BGB split, GewO timeout, arithmetic fix ... - Add Qdrant dedup check in upload_file() — skip if regulation_id already exists - Split BGB (2.7MB) into 5 targeted parts via XML extraction: AGB §§305-310, Fernabsatz §§312-312k, Kaufrecht §§433-480, Widerruf §§355-361, Digitale Produkte §§327-327u - Lower large-file threshold 512KB→384KB (fixes GewO 432KB timeout) - Fix arithmetic syntax error when collection_count returns "?" - Replace EGBGB PDF (was empty) with XML extraction - Add unzip to Alpine container for XML archives Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 09:39:09 +01:00
Benjamin Admin	87d06c8b20	fix(rag): Handle large file uploads + don't abort on individual failures ... - Extended timeout (15 min) for files > 500KB (BGB is 1.5MB) - upload_file returns 0 even on failure so set -e doesn't kill script - Failed uploads are still counted and reported in summary Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-11 23:33:28 +01:00
Benjamin Admin	0b47612272	fix(rag): Always run download phase before ingestion phases ... The gesetze phase failed because it expects text files created by the download phase. Now the workflow automatically runs download first for any phase that depends on it. Also adds git and python3 to the alpine container for repo cloning and text extraction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-11 23:13:33 +01:00