New table: compliance_commits (commit hash, affected controls, risk level)
New API:
POST /v1/compliance-commits (SDK registers commit + impact)
GET /v1/compliance-commits (list with filters)
GET /v1/compliance-commits/by-control/{id} (all commits for a control)
GET /v1/compliance-commits/stats (dashboard)
GET /v1/compliance-commits/{id} (detail)
GIN index on affected_control_ids for fast @> containment queries.
454 tests pass, 0 regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New table: decision_traces (status, reason, evidence, fix plan per control)
New API:
POST/GET/PUT /v1/decision-traces (CRUD for decisions)
GET /v1/decision-traces/stats (compliance dashboard)
GET /v1/controls/{id}/full-trace (Regulation→Obligation→Control→Decision→Evidence)
454 tests pass, 0 regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
G-pre1: 144k objects clustered into 7,466 groups via Mini-Batch K-Means
on bge-m3 embeddings. Two-stage: k=5000 base + sub-cluster groups >50.
G-pre2: 5,114 Master Controls from lifecycle phase chains
(define→implement→test→monitor), linking 172,504 atomic controls.
G-pre3: REST API for Master Controls
GET /v1/master-controls (list, search, filter)
GET /v1/master-controls/stats
GET /v1/master-controls/{mc_id} (detail with phase-controls)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Registers /generate/run-pass0a and /generate/pass0a-status/{job_id}
on the core control-pipeline (port 8098). Previously Pass 0a was only
available on the compliance backend which connects to Production DB,
causing a split-brain when controls are generated locally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sends 67k review candidates to Haiku Batch API in pairs.
Each pair gets a DUPLIKAT/VERSCHIEDEN decision with reasoning.
Results stored in control_dedup_reviews.review_status.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents duplicate batch submissions that caused ~$170 in extra costs.
Refuses new submit if a batch was submitted in the last 10 minutes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prompt v4 adds 6 new fields to Pass 0b output:
- applicability: condition rules (same format as dependency engine)
- check_type: expanded to 10 granular types
- scanner_hint: search_terms + negative_indicators for MCP
- manual_review_required_if: escalation conditions
- evidence_type: code/process/hybrid
- provides_context: context variables this control creates
New endpoint POST /generate/backfill-extended:
- Backfills existing 9k controls via Haiku Batch API (~$1.50)
- Adds all 6 new fields to generation_metadata
- Supports dry_run mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix truncated title detection: only flag near-200-char titles or mid-word cutoffs
- Fix evidence leak detection: check title start patterns, not keyword substring
("nachweisen" verb is valid action, "Nachweis vorliegen" is evidence)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GET /generate/batch-api-status/{batch_id} — check Anthropic batch status
- POST /generate/process-batch — process completed batch results (background)
- GET /generate/process-batch-status/{job_id} — poll processing progress
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous version searched against atomic_controls_dedup collection which
only contains Pass 0b atomic controls. Now creates a temporary collection
with ALL draft controls as reference, then checks targets against it.
Two phases:
1. Index ~53k reference drafts into temp Qdrant collection (batch 32)
2. Search each of 14k target controls, Embedding + LLM for borderline
3. Cleanup temp collection when done
Status updates every 50 controls (fixed counter bug).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /generate/harmonization-recheck verifies promoted controls
against Qdrant dedup collection via Embedding + LLM. Runs as stable
asyncio background task inside the container (no docker exec issues).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /v1/canonical/generate/backfill-applicability enriches controls
with applicable_industries, applicable_company_size, scope_conditions
via Anthropic API. Targets ~26k controls from pipeline version < 3.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ollama API supports "think": false to disable extended thinking mode
on qwen3.5. Reduces response time from 95s to ~3s per comparison.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
qwen3.5 uses extended thinking by default which causes 95s+ responses
and 30s timeouts. Add /no_think to system prompt and num_predict=200
to keep responses short.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /v1/canonical/generate/llm-dedup uses local Ollama (qwen3.5:35b-a3b)
to verify borderline duplicate matches (score 0.85-0.91). More accurate
than embedding similarity for compliance controls with subtle scope
differences (e.g. "documented" vs "implemented").
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /v1/canonical/generate/backfill-repair uses Anthropic API to
generate missing fields from available context (source text, other
fields). Handles 1,470 controls with incomplete data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Go SDK RAG proxy returns 401 (Qdrant API key mismatch). Switch
AnchorFinder to use direct Qdrant vector search + embedding service,
same approach as the main pipeline. No dependency on Go SDK anymore.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add POST /v1/canonical/generate/backfill-anchors endpoint for batch
populating open_anchors on controls generated with skip_web_search=true
- Uses AnchorFinder Stage A (RAG search) to find OWASP/NIST/ENISA refs
- Background job with progress tracking (same pattern as other backfills)
- Promotes needs_review controls that gain anchors to draft state
- Target audience normalization (enterprise/authority/provider → JSON arrays)
already applied via SQL
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Anchor-Search (DuckDuckGo + RAG via SDK) verlangsamt Pipeline von
~50 Chunks/min auf ~10 Chunks/min. Anchors (OWASP/NIST-Referenzen)
koennen nachtraeglich in einem Batch-Job befuellt werden.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Control-Pipeline (Pass 0a/0b, BatchDedup, Generator) als eigenstaendiger
Service in Core, damit Compliance-Repo unabhaengig refakturiert werden kann.
Schreibt weiterhin ins compliance-Schema der shared PostgreSQL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Benjamin Admin