feat(compliance-check): MC-Classification + Embedding + Vendor-Redundanz + Action-Recipes + Borlabs-Features

Massiv-Update auf Basis BMW-Test-Iterationen (v1→v9):

Core Compliance-Check
- Sonnet check_type Klassifikation: text/process/review fuer alle 1874 MCs
  in compliance.doc_check_controls (script + Sidecar /data/mc_classification.db).
  rag_document_checker filtert auf check_type='text' fuer doc_check.
  Plus fits_doc_type-Audit (v2) + ui_only-Audit fuer DSA/E-Commerce-MCs in
  falscher doc_type-Schublade.
- scope_requires-Filter: biometric/ai_decision/child_targeting MCs werden
  per business_profile gefiltert (FRT skipped fuer BMW etc.).
- Embedding-Match (BGE-M3) als Phase-3 nach Regex-Match:
  Per-doc_type-Threshold-Override (impressum 0.50, dse/cookie 0.60),
  Short-Field-Rescue (15-Wort-Chunks) fuer Pflichtfelder im Impressum.
  Title+check_question als Embedding-Input fuer mehr Kontext.
- Cookie-Text-Routing: consent-tester gibt cmp_cookie_text aus dem
  CMP-Reconstruct zurueck, Backend bevorzugt das gegen DOM-Extraction
  wenn richer (BMW 1824 vs 600 Worte).

Vendor-Redundanz + EU-Alternativen + Cost-Saving
- vendor_redundancy.analyze() — funktionale Kategorisierung der CMP-Vendors,
  Detektion von Mehrfach-Anbietern pro Kategorie, EU-Alternative-Lookup
  (Matomo, IONOS, HERE, Friendly Captcha, Smart AdServer, ...).
- vendor_cost_estimator: Tier-Inferenz aus Cookie-Footprint (Cookie-Anzahl
  + Premium-Feature-Cookies + Third-Party-Quote → starter/professional/
  enterprise/premier).
- Self-Service-Werbung (Google/Meta/Pinterest/...) = 0 Lizenz-Kosten
  (nur Media-Spend, separat). DSP-Plattformen behalten enge Range.
- Tier-aware Saving-Range: bei Enterprise/Premier nutzen wir den
  oberen 40-100%-Band der Listpreise, nicht starter→premier.
- Multi-Function-Tools (Matomo Pro, SAP CX, IONOS Cloud, Userlike, Smart
  AdServer, HERE Maps, Vimeo Pro, LamaPoll) — ein Tool ersetzt mehrere
  Kategorien gleichzeitig.

Cookie-Wissens-DB + Funktionale Klassifikation
- cookie_knowledge_db: 50 kuratierte Top-Cookies (Google/Meta/Adobe/MS/...)
  mit vendor, exact_purpose, data_collected, IAB-TCF-IDs, reid_risk,
  schrems_ii_status, EuGH-Urteile, EU-Alternative.
- cookie_function_classifier: pro Cookie funktionale Rolle (tracking_id,
  ad_pixel, session_id, ab_test, csrf, ...) + blocking_impact.

Country-Inferenz aus Rechtsform
- cookie_link_validator: Country-Field wird aus Vendor-Name abgeleitet
  (A/S=DK, GmbH=DE, Inc=US, B.V.=NL, ...) plus Vendor-Lookup-Table.
  Reduziert false-positive no_country-Flags bei eindeutig-EU-Vendors
  (Adform DK, Pinterest IE).

Action-Recipes + Doc-Anchor-Locator
- finding_action_recipes: pro Finding-Typ (no_cookies_listed, no_country,
  broken_opt_out, "Auftragsverarbeiter erwaehnen", "Art. 22 Profiling",
  ...) eine strukturierte Anweisung mit what/why/fix_text/where/example.
  Zum 1:1-Einfuegen in Kunden-Dokumente.
- doc_anchor_locator: Embedding-basiert (BGE-M3 cosine) — sucht den
  passenden Absatz im existierenden Kundendokument fuer jeden Finding.
  Per-Run Thread-Local-Cache. Fallback: keyword-Match.
- Email-Rendering integriert Recipe + Anchor pro Doc-Pruefungs-Fail
  + Vendor-Flag-Liste mit aufklappbarer Action-Liste.
- Score-Erklaerung pro Vendor-Zeile (3/5-Untertitel + Tooltip).

Migration-Pipeline (Compliance-Check -> Customer Banner/Documents)
- migration_to_banner.py: Vendor-Liste -> CookieBannerConfig mit
  4 Kategorien + Review-Flags.
- migration_to_document.py: Vendor-Liste -> Cookie-Policy + VVT-Register
  + Privacy-Policy-Pre-Fills.
- agent_migration_routes: 3 Preview-Endpoints (banner-preview,
  document-preview, summary). Persistierung der cmp_vendors in
  /data/compliance_audits.db check_payloads-Tabelle.

Borlabs-Parity Cookie-Banner-Features
- Consent-Historie im Banner: window.bpShowConsentHistory() + localStorage.
- Content-Blocker: cookie-banner-content-blocker.ts — YouTube/Maps/Video
  Placeholder bis Einwilligung.
- Google Consent Mode v2 erweitert: wait_for_update + region=EEA/CH/GB.
- Consent-Log Export (CSV/JSON) per einwilligungen_export_routes.

Bug-Fixes
- canonical_control_routes: _jsonish-Helper fuer string-typed jsonb,
  similar-controls-Endpoint mit _has_embedding_col()-Cache (kein 500 mehr).
- Control-Library Frontend: defensive .map-Coercer in 2 Detail-Views.
- Embedding-Service-Batching (32er Batches statt 165 in einem Call).
- KeyError 'control_id' in MC-Result-Aggregation (defensive .get).
- Master-Controls-Klick-Through von /sdk/master-controls auf
  /sdk/control-library?control=<id> mit URL-Param-Auto-Open.
- Dockerfile: /data pre-chowned auf appuser (Audit-DB-Schreibrecht).
- Cookie-Text-Routing-Bug (cmp_reconstructed > DOM-extraction).
- doc_type-aware MC-Filter (statt all-text-MCs).
- Master-Contract-Dedup (60 BMW-Internal-Eintraege = 1 Adobe-Vertrag).
- A3-v2-Audit hat 24 UI-Sprache-MCs als 'process' reklassifiziert.

Tests
- test_migration_mappers.py (9 Tests)
- test_migration_endpoints.py (4 Tests)

Skripte (one-shot)
- classify_mc_check_type.py (v1) + _v2 (PK=control_id,doc_type)
- audit_mc_doctype_fit.py (v1 fits) + _v2 (ui_only + scope_requires)

BMW-Run-Bilanz v1 (broken) -> v9 (alle Fixes):
  DSE     7,5% -> 81-83%
  Impressum 4%   -> 100% (6 echte MCs alle erfuellt)
  Cookie  0%    -> 79-83% (CMP-Text-Routing + Embedding)
  Plus: 10 Konsolidierungs-Kategorien, geschaetzte Saving 200k-3M / Jahr
  Plus: Action-Recipes + Doc-Anchors fuer jeden Fail

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

backend-compliance/compliance/api/agent_doc_check_report.py

@@ -7,8 +7,12 @@ including L1/L2 check hierarchy, progress bars, and actionable hints.
 
 from __future__ import annotations
 
+import logging
+import re
 from typing import TYPE_CHECKING
 
+logger = logging.getLogger(__name__)
+
 if TYPE_CHECKING:
     from .agent_doc_check_routes import CheckItem, DocCheckResult
 
@@ -32,12 +36,93 @@ def _icon(passed: bool, skipped: bool = False) -> str:
     return '<span style="color:#ef4444;font-weight:bold">&#10007;</span>'
 
 
-def _hint_box(hint: str) -> str:
-    return (
+def _first_sentence(text: str, max_chars: int = 300) -> str:
+    """Erster vollstaendiger Satz statt erste Zeile — robust gegen
+    mehrzeilige Fix-Texte die mit Bullet-Listen anfangen."""
+    if not text:
+        return ""
+    # Suche Satz-Endezeichen vor max_chars
+    snippet = text[:max_chars]
+    m = re.search(r"^(.+?[\.\?\!])(?:\s|$)", snippet, re.DOTALL)
+    if m:
+        first = m.group(1).strip()
+        # Wenn der "Satz" eine Variant-Header wie "Variante A:" ist, nimm
+        # weiter — der echte Inhalt kommt erst danach
+        if re.fullmatch(r"(Variante [A-Z]\s*\([^\)]+\):?|Beispiel\s*\d*:?)",
+                        first, re.IGNORECASE):
+            rest = text[m.end():].lstrip()
+            return _first_sentence(rest, max_chars)
+        return first
+    # Kein Satz-Endezeichen — nimm bis max_chars
+    line = (text.splitlines() or [""])[0]
+    return line[:max_chars] + ("…" if len(line) > max_chars else "")
+
+
+def _hint_box(hint: str, check_label: str = "", doc_text: str = "",
+              doc_id: str | None = None) -> str:
+    """Hint-Block mit angereichertem Recipe + Doc-Anchor wenn moeglich."""
+    base = (
         f'<div style="font-size:11px;color:#dc2626;margin:2px 0 4px 20px;'
         f'padding:4px 8px;background:#fef2f2;border-radius:4px;'
-        f'border-left:3px solid #fca5a5">{hint}</div>'
+        f'border-left:3px solid #fca5a5">{hint}'
     )
+    # Recipe + Anker hinzufuegen wenn check_label bekannt
+    if check_label:
+        try:
+            from compliance.services.finding_action_recipes import recipe_for
+            from compliance.services.doc_anchor_locator import locate_anchor
+            rec = recipe_for(check_label)
+            if rec and rec.get("fix_text"):
+                first_sentence = _first_sentence(rec["fix_text"], 300)
+                full = rec["fix_text"]
+                # Statt <details> ein einfaches Inline-Block-Layout —
+                # robuster bei Plain-Text-Mail-Render
+                more = ""
+                if len(full) > len(first_sentence) + 10:
+                    more = (
+                        f'<div style="margin-top:4px;padding:6px 8px;background:#fff;'
+                        f'border:1px solid #fcd5d5;border-radius:4px;font-size:10px;'
+                        f'white-space:pre-wrap;color:#1e293b">'
+                        f'<strong style="display:block;margin-bottom:3px;color:#475569">'
+                        f'Vollstaendiger Textbaustein zum Einfuegen:</strong>'
+                        f'{full}</div>'
+                    )
+                base += (
+                    f'<div style="margin-top:6px;padding-top:6px;border-top:1px solid #fecaca">'
+                    f'<strong style="color:#7c3aed;font-size:10px">Konkrete Massnahme:</strong> '
+                    f'<span style="color:#1e293b">{first_sentence}</span>'
+                    f'{more}'
+                )
+                # Anker via Embedding-Locator (mit doc_id-Cache)
+                if doc_text:
+                    anchor = locate_anchor(check_label, doc_text, doc_id)
+                    if anchor and anchor.get("anchor_phrase") and anchor.get("confidence") != "low":
+                        conf_label = anchor.get("confidence", "")
+                        conf_badge = (
+                            f' <span style="color:#94a3b8;font-size:9px">'
+                            f'(Match-Konfidenz {conf_label}, '
+                            f'Score {anchor.get("score", "—")})</span>'
+                        )
+                        base += (
+                            f'<div style="margin-top:4px;color:#475569;font-size:10px">'
+                            f'<strong>Einfuegen:</strong> {anchor["position_hint"]}'
+                            f'{conf_badge}</div>'
+                        )
+                    elif rec.get("where"):
+                        # Kein guter Anchor-Match — zeige generischen Fallback
+                        base += (
+                            f'<div style="margin-top:4px;color:#475569;font-size:10px">'
+                            f'<strong>Einfuegen:</strong> {rec["where"]} '
+                            f'<span style="color:#94a3b8;font-size:9px">'
+                            f'(kein eindeutiger Absatz im Dokument gefunden — '
+                            f'Anweisung allgemein)</span></div>'
+                        )
+                base += '</div>'
+        except Exception as e:
+            logger.debug("Hint-box enrichment failed: %s", e)
+            pass  # Recipes optional — Hint-Box muss nie crashen
+    base += '</div>'
+    return base
 
 
 def build_management_summary(results: list[DocCheckResult]) -> str:
@@ -158,8 +243,14 @@ def _check_to_action(doc_label: str, check_label: str, hint: str) -> str:
 def build_html_report(
     results: list[DocCheckResult],
     cookie_result: dict | None,
+    doc_texts: dict[str, str] | None = None,
 ) -> str:
-    """Build HTML email report styled like the frontend."""
+    """Build HTML email report styled like the frontend.
+
+    `doc_texts` is the doc_type→text dict so hint-boxes can locate the
+    relevant Absatz in the original document for the Einfuege-Empfehlung.
+    """
+    doc_texts = doc_texts or {}
     ok_count = sum(1 for r in results if r.completeness_pct == 100)
     html = [
         '<div style="font-family:-apple-system,BlinkMacSystemFont,sans-serif;'
@@ -170,7 +261,7 @@ def build_html_report(
     ]
 
     for r in results:
-        _render_document(html, r)
+        _render_document(html, r, doc_texts.get(r.doc_type, ""))
 
     if cookie_result:
         _render_cookie_banner(html, cookie_result)
@@ -179,7 +270,7 @@ def build_html_report(
     return "\n".join(html)
 
 
-def _render_document(html: list[str], r: DocCheckResult) -> None:
+def _render_document(html: list[str], r: DocCheckResult, doc_text: str = "") -> None:
     pct = r.completeness_pct
     cpct = r.correctness_pct
     bar_color = "green" if pct >= 80 else "yellow" if pct >= 50 else "red"
@@ -244,7 +335,7 @@ def _render_document(html: list[str], r: DocCheckResult) -> None:
     else:
         html.append('<div style="padding:8px 16px 12px">')
         for c in l1_checks:
-            _render_l1_check(html, c, l2_by_parent.get(c.id, []))
+            _render_l1_check(html, c, l2_by_parent.get(c.id, []), doc_text)
 
         # Master-Control aggregation: with 1874 MCs evaluated per run,
         # rendering every L2 check inline produces ~600 rows per doc and
@@ -289,6 +380,7 @@ def _render_document(html: list[str], r: DocCheckResult) -> None:
 
 def _render_l1_check(
     html: list[str], c: CheckItem, children: list[CheckItem],
+    doc_text: str = "",
 ) -> None:
     l2_sub = [ch for ch in children if not ch.skipped]
     l2_passed = sum(1 for ch in l2_sub if ch.passed)
@@ -301,16 +393,16 @@ def _render_l1_check(
     if l2_sub:
         html.append(f' <span style="color:#9ca3af;font-size:11px">({l2_passed}/{len(l2_sub)})</span>')
     if not c.passed and c.hint:
-        html.append(_hint_box(c.hint))
+        html.append(_hint_box(c.hint, c.label, doc_text))
     html.append('</div>')
 
     for ch in children:
         if ch.skipped:
             continue
-        _render_l2_check(html, ch)
+        _render_l2_check(html, ch, doc_text)
 
 
-def _render_l2_check(html: list[str], ch: CheckItem) -> None:
+def _render_l2_check(html: list[str], ch: CheckItem, doc_text: str = "") -> None:
     style = "color:#dc2626;font-weight:500" if not ch.passed else "color:#6b7280"
     html.append(
         f'<div style="padding:2px 0 2px 24px;border-left:2px solid #e5e7eb;margin-left:8px">'
@@ -324,7 +416,7 @@ def _render_l2_check(html: list[str], ch: CheckItem) -> None:
             f'white-space:nowrap">"...{ch.matched_text[:80]}..."</div>'
         )
     if not ch.passed and ch.hint:
-        html.append(_hint_box(ch.hint))
+        html.append(_hint_box(ch.hint, ch.label, doc_text))
     html.append('</div>')