- Fix truncated title detection: only flag near-200-char titles or mid-word cutoffs
- Fix evidence leak detection: check title start patterns, not keyword substring
("nachweisen" verb is valid action, "Nachweis vorliegen" is evidence)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GET /generate/batch-api-status/{batch_id} — check Anthropic batch status
- POST /generate/process-batch — process completed batch results (background)
- GET /generate/process-batch-status/{job_id} — poll processing progress
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous version searched against atomic_controls_dedup collection which
only contains Pass 0b atomic controls. Now creates a temporary collection
with ALL draft controls as reference, then checks targets against it.
Two phases:
1. Index ~53k reference drafts into temp Qdrant collection (batch 32)
2. Search each of 14k target controls, Embedding + LLM for borderline
3. Cleanup temp collection when done
Status updates every 50 controls (fixed counter bug).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /generate/harmonization-recheck verifies promoted controls
against Qdrant dedup collection via Embedding + LLM. Runs as stable
asyncio background task inside the container (no docker exec issues).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /v1/canonical/generate/backfill-applicability enriches controls
with applicable_industries, applicable_company_size, scope_conditions
via Anthropic API. Targets ~26k controls from pipeline version < 3.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ollama API supports "think": false to disable extended thinking mode
on qwen3.5. Reduces response time from 95s to ~3s per comparison.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
qwen3.5 uses extended thinking by default which causes 95s+ responses
and 30s timeouts. Add /no_think to system prompt and num_predict=200
to keep responses short.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /v1/canonical/generate/llm-dedup uses local Ollama (qwen3.5:35b-a3b)
to verify borderline duplicate matches (score 0.85-0.91). More accurate
than embedding similarity for compliance controls with subtle scope
differences (e.g. "documented" vs "implemented").
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
POST /v1/canonical/generate/backfill-repair uses Anthropic API to
generate missing fields from available context (source text, other
fields). Handles 1,470 controls with incomplete data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Go SDK RAG proxy returns 401 (Qdrant API key mismatch). Switch
AnchorFinder to use direct Qdrant vector search + embedding service,
same approach as the main pipeline. No dependency on Go SDK anymore.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add POST /v1/canonical/generate/backfill-anchors endpoint for batch
populating open_anchors on controls generated with skip_web_search=true
- Uses AnchorFinder Stage A (RAG search) to find OWASP/NIST/ENISA refs
- Background job with progress tracking (same pattern as other backfills)
- Promotes needs_review controls that gain anchors to draft state
- Target audience normalization (enterprise/authority/provider → JSON arrays)
already applied via SQL
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Anchor-Search (DuckDuckGo + RAG via SDK) verlangsamt Pipeline von
~50 Chunks/min auf ~10 Chunks/min. Anchors (OWASP/NIST-Referenzen)
koennen nachtraeglich in einem Batch-Job befuellt werden.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Control-Pipeline (Pass 0a/0b, BatchDedup, Generator) als eigenstaendiger
Service in Core, damit Compliance-Repo unabhaengig refakturiert werden kann.
Schreibt weiterhin ins compliance-Schema der shared PostgreSQL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
