8510af46eb
feat(pipeline): MC Quality Overhaul — 74.5% → 92.8% accuracy, 5.3K → 13.6K MCs
...
Phase 0: Quality Audit script (Claude Sonnet, 1750 samples)
Phase 1: Object ontology expanded 31 → 74 tokens with descriptions + boundaries
Phase 2: 174K controls re-classified via Haiku (10 batches, $50)
- Generic tokens removed (documentation, procedure, process)
- L2 sub-topics added (108K + 64K controls)
- Bad subtopics fixed (stakeholder_*, escalation fragments)
Phase 3: Re-clustering K=18704 (37K objects → 16.7K groups)
Phase 4: Direct MC generation from canonical tokens (gpre2_direct_mc.py)
Phase 5: Regulation-source split (gpre3, dry-run tested)
New features:
- Tenant-isolated document upload API (rag-service)
- BAuA crawler (Playwright, 131 PDFs downloaded)
- OSHA Technical Manual crawler (23 chapters)
- CE obligation extractor (6141 obligations from Qdrant)
RAG ingestion:
- 126 BAuA PDFs (TRBS/TRGS/ASR): 27,664 chunks
- OSHA Technical Manual: 7,241 chunks
- OSHA 1910 Subpart O (full): 745 chunks
- EuGH C-588/21 P: 216 chunks
- EU 2018/1725: 842 chunks
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-10 15:08:15 +02:00
82c28a2b6e
Add QDRANT_API_KEY support to rag-service
...
- Add QDRANT_API_KEY to config.py (empty string = no auth)
- Pass api_key to QdrantClient constructor (None when empty)
- Add QDRANT_API_KEY to coolify compose and env example
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-13 10:16:59 +01:00
be45adb975
fix(rag): Auto-create Qdrant collection on first index
...
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 33s
CI / test-python-voice (push) Successful in 36s
CI / test-bqas (push) Successful in 31s
CI / deploy-hetzner (push) Successful in 38s
Collections may not exist if init_collections() failed at startup
(e.g. Qdrant not ready). Now index_documents() ensures the
collection exists before upserting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-11 21:02:05 +01:00
7c932c441f
feat(rag): Add bp_compliance_gesetze + bp_compliance_ce collections
...
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 35s
CI / test-python-voice (push) Successful in 50s
CI / test-bqas (push) Successful in 33s
CI / deploy-hetzner (push) Successful in 39s
Required for Verbraucherschutz + EU law ingestion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-11 20:41:26 +01:00
5c8307f58a
fix(rag): use query_points instead of deprecated search method
...
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 38s
CI / test-python-voice (push) Successful in 36s
CI / test-bqas (push) Successful in 28s
qdrant-client 1.17.0 removed the search() method in favor of
query_points(). Update the wrapper to use the new API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-27 07:51:12 +01:00
ad111d5e69
Initial commit: breakpilot-core - Shared Infrastructure
...
Docker Compose with 24+ services:
- PostgreSQL (PostGIS), Valkey, MinIO, Qdrant
- Vault (PKI/TLS), Nginx (Reverse Proxy)
- Backend Core API, Consent Service, Billing Service
- RAG Service, Embedding Service
- Gitea, Woodpecker CI/CD
- Night Scheduler, Health Aggregator
- Jitsi (Web/XMPP/JVB/Jicofo), Mailpit
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-11 23:47:13 +01:00
Benjamin Admin