Commit Graph

9 Commits

Author SHA1 Message Date
Benjamin Admin
643b26618f feat: Control Library UI, dedup migration, QA tooling, docs
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 31s
CI/CD / test-python-backend-compliance (push) Successful in 1m35s
CI/CD / test-python-document-crawler (push) Successful in 20s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Has been skipped
- Control Library: parent control display, ObligationTypeBadge,
  GenerationStrategyBadge variants, evidence string fallback
- API: expose parent_control_uuid/id/title in canonical controls
- Fix: DSFA SQLAlchemy 2.0 Row._mapping compatibility
- Migration 074: control_parent_links + control_dedup_reviews tables
- QA scripts: benchmark, gap analysis, OSCAL import, OWASP cleanup,
  phase5 normalize, phase74 gap fill, sync_db, run_job
- Docs: dedup engine, RAG benchmark, lessons learned, pipeline docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 11:56:08 +01:00
Benjamin Admin
2a0449c9b7 docs(qa): add Control Quality Pipeline documentation
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 33s
CI/CD / test-python-backend-compliance (push) Successful in 32s
CI/CD / test-python-document-crawler (push) Successful in 21s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Has been skipped
QA process, article types, match rates, preamble dedup rules,
and next steps documented in MkDocs under Entwicklung.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 08:16:07 +01:00
Benjamin Admin
f2819b99af feat(pipeline): v3 — scoped control applicability + source_type classification
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 36s
CI/CD / test-python-backend-compliance (push) Successful in 36s
CI/CD / test-python-document-crawler (push) Successful in 27s
CI/CD / test-python-dsms-gateway (push) Successful in 18s
CI/CD / validate-canonical-controls (push) Successful in 11s
CI/CD / Deploy (push) Has been skipped
Phase 4: source_type (law/guideline/standard/restricted) on source_citation
- NIST/OWASP/ENISA correctly shown as "Standard" instead of "Gesetzliche Grundlage"
- Dynamic frontend labels based on source_type
- Backfill endpoint POST /v1/canonical/generate/backfill-source-type

Phase v3: Scoped Control Applicability
- 3 new fields: applicable_industries, applicable_company_size, scope_conditions
- LLM prompt extended with 39 industries, 5 company sizes, 10 scope signals
- All 5 generation paths (Rule 1/2/3, batch structure, batch reform) updated
- _build_control_from_json: parsing + validation (string→list, size validation)
- _store_control: writes 3 new JSONB columns
- API: response models, create/update requests, SELECT queries extended
- Migration 063: 3 new JSONB columns with GIN indexes
- 110 generator tests + 28 route tests = 138 total, all passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 16:28:05 +01:00
Benjamin Admin
148c7ba3af feat(qa): recital detection, review split, duplicate comparison
Some checks failed
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Failing after 42s
CI/CD / test-python-backend-compliance (push) Successful in 34s
CI/CD / test-python-document-crawler (push) Successful in 21s
CI/CD / test-python-dsms-gateway (push) Successful in 20s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Has been skipped
Add _detect_recital() to QA pipeline — flags controls where
source_original_text contains Erwägungsgrund markers instead of
article text (28% of controls with source text affected).

- Recital detection via regex + phrase matching in QA validation
- 10 new tests (TestRecitalDetection), 81 total
- ReviewCompare component for side-by-side duplicate comparison
- Review mode split: Duplikat-Verdacht vs Rule-3-ohne-Anchor tabs
- MkDocs: recital detection documentation
- Detection script for bulk analysis (scripts/find_recital_controls.py)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 08:20:02 +01:00
Benjamin Admin
a9e0869205 feat(pipeline): pipeline_version v2, migration 062, docs + 71 tests
- Add PIPELINE_VERSION=2 constant and pipeline_version column to
  canonical_controls and canonical_processed_chunks (migration 062)
- Anthropic API decides chunk relevance via null-returns (skip_prefilter)
- Annex/appendix chunks explicitly protected in prompts
- Fix 6 failing tests (CRYP domain, _process_batch tuple return)
- Add TestPipelineVersion + TestRegulationFilter test classes (10 new tests)
- Add MkDocs page: control-generator-pipeline.md (541 lines)
- Update canonical-control-library.md with v2 pipeline diagram
- Update testing.md with 71-test breakdown table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:31:11 +01:00
Benjamin Admin
2ed1c08acf feat: enhance legal basis display, add batch processing tests and docs
All checks were successful
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 32s
CI/CD / test-python-backend-compliance (push) Successful in 31s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
- Backfill 81 controls with empty source_citation.source from generation_metadata
- Add fallback to generation_metadata.source_regulation in ControlDetail blue box
- Improve Rule 3 amber box text for reformulated controls
- Add 30 new tests for batch processing (TestParseJsonArray, TestBatchSizeConfig,
  TestBatchProcessingLoop) — all 61 control generator tests passing
- Fix stale test_config_defaults assertion (max_controls 50→0)
- Update canonical-control-library.md with batch processing pipeline docs,
  processed chunks tracking, migration guide, and stats endpoint
- Update testing.md with canonical control generator test section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 23:51:52 +01:00
Benjamin Admin
399fa62267 docs: update all docs to reflect Coolify deployment model
Replace Hetzner references with Coolify. Deployment is now:
- Core + Compliance: Push gitea → Coolify auto-deploys
- Lehrer: stays local on Mac Mini

Updated: CLAUDE.md, MkDocs CI/CD pipeline, MkDocs index.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:09:51 +01:00
Benjamin Admin
8742cb7f5a docs: Qdrant und MinIO/Object-Storage Referenzen aktualisieren
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-ai-compliance (push) Successful in 35s
CI / test-python-backend-compliance (push) Successful in 32s
CI / test-python-document-crawler (push) Successful in 41s
CI / test-python-dsms-gateway (push) Successful in 19s
- Qdrant: lokaler Container → qdrant-dev.breakpilot.ai (gehostet, API-Key)
- MinIO: bp-core-minio → Hetzner Object Storage (nbg1.your-objectstorage.com)
- CLAUDE.md, MkDocs, ARCHITECTURE.md, training.md, ci-cd-pipeline.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:18:29 +01:00
Benjamin Boenisch
c11270f8e0 Add CLAUDE.md, MkDocs docs, docs page in admin, .claude/rules
- CLAUDE.md: Comprehensive documentation for Compliance SDK platform
- docs-src: AI-Compliance-SDK docs (architecture, developer, auditor, SBOM)
- mkdocs.yml: Compliance-specific nav with purple theme
- docker-compose: Added docs service (port 8011, profile: docs)
- admin-compliance: New /development/docs page with iframe + quick links
- navigation.ts: Added development category with docs module
- .claude/rules: testing, docs, open-source, compliance-checklist

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 00:49:28 +01:00