fix(drafting): Drafting-Engine auf prod reparieren — RAG via ai-sdk + OVH-LLM-Kaskade
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Has been skipped
CI / test-python-document-crawler (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
CI / detect-changes (push) Successful in 7s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / build-sha-integrity (push) Successful in 5s
CI / validate-canonical-controls (push) Successful in 4s
CI / loc-budget (push) Successful in 17s
CI / go-lint (push) Has been skipped
CI / nodejs-build (push) Successful in 3m2s
CI / test-go (push) Has been skipped

Die Drafting-Engine (Dokument-Entwurf, v2-Pipeline, Validierung, Drafting-Chat,
Vendor-Vertragspruefung) war auf prod doppelt tot:
- RAG ueber bp-core-rag-service:8097 (existiert auf prod nicht)
- LLM ueber OLLAMA_URL/api/chat mit qwen2.5vl (prod = ollama-embed, kein Chat-Modell)

Fix (analog zum Compliance-Advisor):
- rag-query.ts -> ai-compliance-sdk /sdk/v1/rag/search (bge-m3, prod-erreichbar).
- Neue lib/sdk/drafting-engine/llm-cascade.ts: OVH/LiteLLM (gpt-oss-120b) zuerst,
  Ollama als Dev-Fallback; cascadeComplete (JSON) + cascadeStream. Das Backend nutzt
  OVH+JSON bereits erfolgreich auf prod (extract-datasheet).
- 5 Aufrufstellen (draft-helpers, draft-helpers-v2, validate, chat, vendor-review)
  auf die Kaskade umgestellt; keine direkten Ollama-Calls mehr.
- Tests: llm-cascade + rag-query aktualisiert.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-06-19 10:02:06 +02:00
parent cd3e0b15ad
commit 90a70c8404
9 changed files with 398 additions and 203 deletions
@@ -11,9 +11,7 @@ import { queryRAG } from '@/lib/sdk/drafting-engine/rag-query'
import { DOCUMENT_RAG_CONFIG } from '@/lib/sdk/drafting-engine/rag-config'
import { readSoulFile } from '@/lib/sdk/agents/soul-reader'
import type { ScopeDocumentType } from '@/lib/sdk/compliance-scope-types'
const OLLAMA_URL = process.env.OLLAMA_URL || 'http://host.docker.internal:11434'
const LLM_MODEL = process.env.COMPLIANCE_LLM_MODEL || 'qwen2.5vl:32b'
import { cascadeStream } from '@/lib/sdk/drafting-engine/llm-cascade'
// Fallback SOUL prompt (used when .soul.md file is unavailable)
const FALLBACK_DRAFTING_PROMPT = `# Drafting Agent - Compliance-Dokumententwurf
@@ -81,66 +79,20 @@ export async function POST(request: NextRequest) {
]
// 4. Call LLM with streaming
const ollamaResponse = await fetch(`${OLLAMA_URL}/api/chat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: LLM_MODEL,
messages,
stream: true,
think: false,
options: {
temperature: mode === 'draft' ? 0.2 : 0.3,
num_predict: mode === 'draft' ? 16384 : 8192,
num_ctx: 8192,
},
}),
signal: AbortSignal.timeout(120000),
// 4. LLM-Kaskade (OVH -> Ollama) -> Plain-Text-Stream
const stream = await cascadeStream(messages, {
temperature: mode === 'draft' ? 0.2 : 0.3,
maxTokens: mode === 'draft' ? 16384 : 8192,
timeoutMs: 120000,
})
if (!ollamaResponse.ok) {
const errorText = await ollamaResponse.text()
console.error('LLM error:', ollamaResponse.status, errorText)
if (!stream) {
return NextResponse.json(
{ error: `LLM nicht erreichbar (Status ${ollamaResponse.status})` },
{ error: 'LLM nicht erreichbar (weder OVH noch Ollama)' },
{ status: 502 }
)
}
// 5. Stream response back
const encoder = new TextEncoder()
const stream = new ReadableStream({
async start(controller) {
const reader = ollamaResponse.body!.getReader()
const decoder = new TextDecoder()
try {
while (true) {
const { done, value } = await reader.read()
if (done) break
const chunk = decoder.decode(value, { stream: true })
const lines = chunk.split('\n').filter((l) => l.trim())
for (const line of lines) {
try {
const json = JSON.parse(line)
if (json.message?.content) {
controller.enqueue(encoder.encode(json.message.content))
}
} catch {
// Partial JSON, skip
}
}
}
} catch (error) {
console.error('Stream error:', error)
} finally {
controller.close()
}
},
})
return new NextResponse(stream, {
headers: {
'Content-Type': 'text/plain; charset=utf-8',