fix(scan): TCF SQL column + cascade diagnose-logs
VW-Scan-Befunde aus 0a8aa16e: 1. TCF lookup failed 5x mit: column 'source' does not exist. Korrekt: 'source_name' (siehe DELETE-Query in derselben Datei). Mit dem Fix funktioniert das TCF-Cross-Reference fuer alle Vendors statt 0. 2. Cascade tier-1 fail loggte leere message — jetzt mit type+model+base. 3. Cascade collapse (tier 2+3 unconfigured) wird beim ersten Aufruf geloggt damit der Operator den ENV-Mangel sofort sieht. 4. vendor_llm_extractor loggt jetzt START + 0-vendor-Return (vorher silent skip — sah aus als waere er nie aufgerufen worden). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -59,9 +59,17 @@ async def extract_vendors_via_llm(
|
||||
und passt in Qwen3-30b-a3b (128k Context) sowie OVH 120B.
|
||||
"""
|
||||
if not cookie_text or len(cookie_text) < 500:
|
||||
logger.info(
|
||||
"LLM vendor extraction SKIP: cookie_text too short (%d chars, need >=500)",
|
||||
len(cookie_text or ""),
|
||||
)
|
||||
return []
|
||||
excerpt = cookie_text[:max_text_chars]
|
||||
user_prompt = f"Cookie-Richtlinie-Text:\n\n{excerpt}"
|
||||
logger.info(
|
||||
"LLM vendor extraction START: input=%d chars (excerpt %d), calling cascade",
|
||||
len(cookie_text), len(excerpt),
|
||||
)
|
||||
|
||||
# P31: nutze tiered LLM-Cascade mit Cache (Qwen → OVH → Anthropic).
|
||||
# Re-Runs derselben Cookie-Doc landen im Valkey-Cache (7d TTL) und
|
||||
@@ -76,13 +84,24 @@ async def extract_vendors_via_llm(
|
||||
vendors = _parse_vendor_list(res.get("text", ""))
|
||||
if vendors:
|
||||
logger.info(
|
||||
"LLM vendor extraction (cascade %s, conf=%.2f, cached=%s): %d vendors",
|
||||
"LLM vendor extraction OK (cascade %s, conf=%.2f, cached=%s): %d vendors",
|
||||
res.get("source"), res.get("confidence", 0),
|
||||
res.get("cached"), len(vendors),
|
||||
)
|
||||
return vendors
|
||||
return vendors
|
||||
# Silent failure ist unbrauchbar fuer Diagnose: log explicit dass
|
||||
# cascade returnte aber 0 Vendors (parse failed oder LLM gab nix).
|
||||
logger.warning(
|
||||
"LLM vendor extraction returned 0 vendors (cascade source=%s "
|
||||
"text_len=%d below_threshold=%s) — fallback to direct calls",
|
||||
res.get("source"), len(res.get("text", "") or ""),
|
||||
res.get("below_threshold"),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("Cascade extract failed, fallback to direct Qwen: %s", e)
|
||||
logger.warning(
|
||||
"Cascade extract failed, fallback to direct Qwen: %s (%s)",
|
||||
str(e) or "(no message)", type(e).__name__,
|
||||
)
|
||||
|
||||
# Fallback: alte direkte Logik
|
||||
content = await _call_ollama(user_prompt)
|
||||
|
||||
Reference in New Issue
Block a user