Benjamin Admin
2ed1c08acf
feat: enhance legal basis display, add batch processing tests and docs
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 32s
CI/CD / test-python-backend-compliance (push) Successful in 31s
CI/CD / test-python-document-crawler (push) Successful in 23s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
- Backfill 81 controls with empty source_citation.source from generation_metadata
- Add fallback to generation_metadata.source_regulation in ControlDetail blue box
- Improve Rule 3 amber box text for reformulated controls
- Add 30 new tests for batch processing (TestParseJsonArray, TestBatchSizeConfig,
TestBatchProcessingLoop) — all 61 control generator tests passing
- Fix stale test_config_defaults assertion (max_controls 50→0)
- Update canonical-control-library.md with batch processing pipeline docs,
processed chunks tracking, migration guide, and stats endpoint
- Update testing.md with canonical control generator test section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-14 23:51:52 +01:00
Benjamin Admin
de19ef0684
feat(control-generator): 7-stage pipeline for RAG→LLM→Controls generation
...
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 45s
CI/CD / test-python-document-crawler (push) Has been cancelled
CI/CD / test-python-dsms-gateway (push) Has been cancelled
CI/CD / validate-canonical-controls (push) Has been cancelled
CI/CD / deploy-hetzner (push) Has been cancelled
CI/CD / test-python-backend-compliance (push) Has been cancelled
Implements the Control Generator Pipeline that systematically generates
canonical security controls from 150k+ RAG chunks across all compliance
collections (BSI, NIST, OWASP, ENISA, EU laws, German laws).
Three license rules enforced throughout:
- Rule 1 (free_use): Laws/Public Domain — original text preserved
- Rule 2 (citation_required): CC-BY/CC-BY-SA — text with citation
- Rule 3 (restricted): BSI/ISO — full reformulation, no source traces
New files:
- Migration 046: job tracking, chunk tracking, blocked sources tables
- control_generator.py: 7-stage pipeline (scan→classify→structure/reform→harmonize→anchor→store→mark)
- anchor_finder.py: RAG + DuckDuckGo open-source reference search
- control_generator_routes.py: REST API (generate, review, stats, blocked-sources)
- test_control_generator.py: license mapping, rule enforcement, anchor filtering tests
Modified:
- __init__.py: register control_generator_router
- route.ts: proxy generator/review/stats endpoints
- page.tsx: Generator modal, stats panel, state filter, review queue, license badges
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-13 09:03:37 +01:00
