Files
breakpilot-compliance/backend-compliance/compliance/services/mail_render_v2/_blocks.py
T
Benjamin Admin d0e3621192 feat(audit): V2 mail render + 5 new findings (B4/B5/B6/B7/B8) + LLM-Plausibility-Phase
Mail Render V2 (compliance/services/mail_render_v2/) — 11-Modul-Subpackage
das einen einheitlichen Audit-Mail-Output erzeugt mit:
  - Header + KPI-Kacheln (Score / Findings / Docs / Vendors)
  - TOC + Sprung-Links
  - 3-Bucket-Trennung: Kritische Befunde / Manuelle Prüfung / Interne Reminder
  - Cookie-Inventar (Name·Vendor·Kategorie·Speicherdauer·Löschfrist·Sitzland·Quelle·Status)
  - Sofortmaßnahmen-Aggregator ("Sitzland ergänzen für 11 Cookies")
  - 24 Legacy-Wrappers — alle alten build_*_html in V2-Sections
  - Scope-Filter: FIN/GOV/MED/INS/EDU/LEG aus Berichten wenn nicht relevant
  - Hint/Action-Dedup: keine doppelten Sätze pro Card mehr
Aktiviert via env MAIL_RENDER_V2=true (Default: legacy renderer).

5 neue deterministische Findings als Phase D-2b/B4/B5/B6/B7/B8:

  B4 vendor_consistency_check — Cross-Doc-Provider-Widerspruch
     (Elli: DSE nennt Vertex AI für Chatbot, /de/cookies nennt Iadvize → HIGH).
     6 Service-Types: chatbot/analytics/tag_manager/pixel/cdn/cmp.

  B5 ai_act_transparency_check — AI Act Art. 50 Transparenzpflicht
     (Elli: Vertex AI vorhanden ohne Pre-Chat-Disclosure → HIGH).
     Plus B5-Erweiterung: Rechtsgrundlage Art-6-Abs-1-lit-f bei AI → MED
     (Einwilligung empfehlen).

  B6 cross_doc_dpo_check — DPO in DSE genannt, nicht im Impressum (LOW).

  B7 doc_staleness_check — Datum-Extraktion aus DSE/AGB/Nutzungsbedingungen.
     Cap: AGB/NB 3y, DSE 2y. Älter → MEDIUM (Elli NB Stand 2018 → HIGH).

  B8 cmp_fingerprint_check — Banner detected, aber CMP-Provider generic
     (kein Usercentrics/OneTrust/Cookiebot/etc → MED).

  B3-Erweiterung detect_intra_doc_contradictions — Widersprüchliche
     Speicherdauer im SELBEN Doc (Elli: Logfile 7d vs 30d → HIGH).

LLM-Plausibility-Phase (Phase D-2b, finding_plausibility_check.py):
  - Läuft AFTER MC pipeline, BEFORE D3 render
  - Prompt mit Beispiel-IDs + 3-Phase-Mapping: exact-ID / position-fallback /
    fuzzy-tail-match
  - Stempelt llm_title / llm_severity / llm_recommendation / llm_drop auf
    jeden FAIL CheckItem
  - V2-Render zeigt "🤖 LLM-Plausibility:" Box pro Finding wenn gestempelt
  - KNOWN ISSUE: qwen3:30b-a3b liefert oft empty content auf format='json' +
    8000-char-excerpt prompts. Pipeline läuft mit stamped=0 weiter. Task #16.

Coverage gegen Elli Ground Truth (zeroclaw/docs/ground-truth/elli_eco_2026-06-06.json,
13 expected findings via WebFetch-Agent-Crawl):
  - 4/4 HIGH-Findings ✓ (COOKIE-CONSENT-UX-001 + WIDERRUFSBELEHRUNG-001 +
    VENDOR-CONSISTENCY-001 + AI-ACT-TRANSPARENCY-001)
  - 4/6 MEDIUM ✓
  - 2/3 LOW ✓
  - Total: 10/13 = 77% (Sprung von 4/13 = 31%)

Restliche 3 Gaps als Task #17: IMPRESSUM-001 (multi-entity USt-IdNr),
TRANSFER-001 (Vendor-Mechanismus DPF/SCC), TH-RETENTION-002 (AI-Retention
pro Datenkategorie).

V2-Mail-Preview in Mailpit: 'v2all@local.test' Subject '[V2 ALL] ELLI'.
Backend healthy, B1+B3+B4+B5+B6+B7+B8 alle live im Orchestrator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-06 21:19:49 +02:00

368 lines
14 KiB
Python

"""Mail-V2 section renderers — one function per top-level block.
Each renderer takes a slice of `state` and returns ready-to-concatenate
HTML using the helpers from `_style`. Every block is full-width, has
the same card shell, and uses the same color palette.
Finding-bucket renderers (critical / manual / internal) live in
`_blocks_findings.py` to keep this file under the LOC cap.
"""
from __future__ import annotations
from html import escape as h
from ._aggregator import group_by_action
from ._blocks_findings import count_critical, count_internal, count_manual
from ._cookie_inventory import (
build_cookie_inventory,
inventory_headers,
render_inventory_rows,
)
from ._style import (
SZ_H3,
SZ_SMALL,
TEXT,
TEXT_MUTED,
card,
chip,
kpi_row,
section,
table,
)
# ── Helpers ──────────────────────────────────────────────────────
def _score_sev(pct: int | None) -> str:
if pct is None:
return "info"
if pct >= 90:
return "pass"
if pct >= 70:
return "info"
if pct >= 40:
return "warn"
return "fail"
# ── 1. Header + KPI row ──────────────────────────────────────────
def render_header(state: dict) -> str:
site = h(state.get("site_name") or "")
dom = h(state.get("domain") or "")
scorecard = state.get("scorecard") or {}
score_pct = (scorecard.get("totals") or {}).get("pct")
doc_count = state.get("doc_count") or 0
docs_total = len(state.get("results") or [])
findings = state.get("total_findings") or 0
vendors = len(state.get("cmp_vendors") or [])
title_html = (
f'<h1 style="font-size:24px;margin:0 0 4px;color:{TEXT};'
f'font-weight:700;">{site}</h1>'
f'<div style="font-size:13px;color:{TEXT_MUTED};margin-bottom:8px;">'
f'{dom} · Compliance-Audit</div>'
)
kpis = [
{"label": "Compliance-Score",
"value": f"{score_pct}%" if score_pct is not None else "",
"sev": _score_sev(score_pct)},
{"label": "Findings", "value": str(findings),
"sev": "fail" if findings > 5 else "warn" if findings > 0 else "pass"},
{"label": "Dokumente",
"value": f"{doc_count}/{docs_total}", "sev": "info"},
{"label": "Vendors", "value": str(vendors),
"sev": "warn" if vendors > 20 else "info"},
]
return title_html + kpi_row(kpis)
# ── 2. Table of contents ────────────────────────────────────────
def render_toc(state: dict) -> str:
rows = [
("#critical", f"Kritische Befunde ({count_critical(state)})"),
("#manual", f"Manuelle Prüfung ({count_manual(state)})"),
("#internal", f"Interne Reminder ({count_internal(state)})"),
("#sofortmassnahmen", "Sofortmaßnahmen"),
("#per-doc",
f"Pro Dokument ({len(state.get('results') or [])})"),
("#per-theme", "Pro Thema"),
("#caveats",
f"Audit-Vorbehalte ({len(state.get('audit_quality_findings') or [])})"),
("#attach",
f"Anhänge ({1 if state.get('cookie_evidence_slices') else 0})"),
]
items = "".join(
f'<li style="margin:6px 0;"><a href="{href}" style="color:#1e40af;'
f'text-decoration:none;">{h(label)}</a></li>'
for href, label in rows
)
return section(
"📋 Inhalt",
f'<ol style="margin:0;padding-left:18px;font-size:14px;">{items}</ol>',
)
# ── 4. Per-document blocks ──────────────────────────────────────
def render_per_doc(state: dict) -> str:
results = state.get("results") or []
if not results:
return ""
cards = []
for r in results:
label = h(getattr(r, "label", "") or "")
url = getattr(r, "url", "") or ""
url_html = (f'<a href="{h(url)}" style="color:#1e40af;font-size:'
f'{SZ_SMALL};">{h(url)}</a>') if url else ""
corr = getattr(r, "correctness_pct", 0) or 0
err = getattr(r, "error", "") or ""
checks = getattr(r, "checks", []) or []
n_total = len(checks)
n_pass = sum(1 for c in checks if c.passed and not c.skipped)
n_fail = sum(1 for c in checks if not c.passed and not c.skipped)
n_skip = sum(1 for c in checks if c.skipped)
score_sev = _score_sev(corr)
head = (
f'<div style="display:flex;justify-content:space-between;'
f'align-items:flex-start;">'
f'<div><span style="font-size:{SZ_H3};font-weight:600;">{label}</span>'
f'<div>{url_html}</div></div>'
f'<div style="text-align:right;">'
f'{chip(f"{corr}%", score_sev)}</div></div>'
)
if err:
body = (f'<p style="margin:8px 0 0;color:{TEXT_MUTED};">'
f'{h(err)}</p>')
else:
counts = (
f'<div style="margin:8px 0;font-size:{SZ_SMALL};'
f'color:{TEXT_MUTED};">'
f'{n_total} MCs · {n_pass} ✓ · {n_fail} ✗ · {n_skip} ?</div>'
)
top = [c for c in checks
if not c.passed and not c.skipped][:3]
top_list = ""
if top:
lis = "".join(
f'<li style="margin:4px 0;">'
f'{h(getattr(c, "label", "")[:120])}</li>'
for c in top
)
top_list = (
f'<ul style="margin:6px 0 0 16px;padding:0;'
f'font-size:13px;color:{TEXT};">{lis}</ul>'
)
body = counts + top_list
cards.append(card(head + body,
sev=score_sev if not err else "info"))
return section(f"📄 4. Pro Dokument ({len(results)})",
"".join(cards), anchor="per-doc")
# ── 5. Per-theme blocks ─────────────────────────────────────────
def render_theme_cookie_banner(state: dict) -> str:
br = state.get("banner_result") or {}
if not br:
return ""
detected = br.get("detected") or br.get("banner_detected")
provider = br.get("provider") or br.get("banner_provider") or ""
violations = br.get("violations") or len(
(br.get("banner_checks") or {}).get("violations") or [])
body = (
f'<div><strong>Provider:</strong> {h(str(provider))} · '
f'<strong>Detected:</strong> '
f'{chip("Ja" if detected else "Nein", "pass" if detected else "fail")} · '
f'<strong>Violations:</strong> {violations}</div>'
)
return card(
f'<h3 style="margin:0 0 6px;font-size:{SZ_H3};">▶ Cookie-Banner</h3>'
+ body,
sev="warn" if violations else "pass",
)
def render_theme_cookie_inventory(state: dict) -> str:
rows, summary = build_cookie_inventory(state)
if summary["total"] == 0:
return ""
head = (
f'<h3 style="margin:0 0 6px;font-size:{SZ_H3};">'
f'▶ Cookie-Inventar ({summary["total"]})</h3>'
f'<div style="font-size:{SZ_SMALL};color:{TEXT_MUTED};'
f'margin-bottom:6px;">'
f'{summary["declared"]} deklariert · '
f'{summary["in_browser"]} im Browser · '
f'<span style="color:#dc2626;">{summary["undoc"]} UNDOC</span> · '
f'<span style="color:#92400e;">{summary["orph"]} ORPH</span> · '
f'<span style="color:#15803d;">{summary["ok"]} OK</span>'
f' · {summary["third_country"]} Drittland'
f'</div>'
f'<div style="font-size:{SZ_SMALL};color:{TEXT_MUTED};'
f'margin-bottom:6px;">'
f'Fehlende Pflichtangaben — Sitzland: {summary["missing_country"]}'
f' · Speicherdauer: {summary["missing_duration"]}'
f'</div>'
)
show_rows = render_inventory_rows(rows[:50])
body = table(inventory_headers(), show_rows)
if len(rows) > 50:
body += (
f'<p style="margin:6px 0 0;font-size:{SZ_SMALL};'
f'color:{TEXT_MUTED};">'
f'… und {len(rows) - 50} weitere</p>'
)
sev = "fail" if summary["undoc"] else "warn" if summary["orph"] else "pass"
return card(head + body, sev=sev)
def render_sofortmassnahmen(state: dict) -> str:
"""Aggregated bulk-recommendations: '1 Aktion fixt N Items'."""
groups = group_by_action(state)
if not groups:
return ""
rows = []
for g in groups:
items = g["items"]
sample = ", ".join(items[:5])
more = f" + {len(items) - 5} weitere" if len(items) > 5 else ""
eff_sev = ("pass" if g["effort"] == "low"
else "warn" if g["effort"] == "med" else "fail")
rows.append([
f'{g.get("icon") or ""} <strong>{h(g["label"])}</strong>'
f'<div style="font-size:11px;color:{TEXT_MUTED};margin-top:2px;">'
f'{h(g.get("norm") or "")}</div>',
f'<strong>{g["count"]}</strong>',
f'<div style="font-size:12px;color:{TEXT};">'
f'{h(sample)}{h(more)}</div>',
chip(g["effort"].upper(), eff_sev),
])
body = table(["Maßnahme", "Anz.", "Betrifft", "Aufwand"], rows)
return section(
f"🛠 Sofortmaßnahmen ({len(groups)})",
'<p style="margin:0 0 8px;color:' + TEXT_MUTED + ';font-size:13px;">'
'Eine Aktion behebt mehrere Findings auf einmal — nach Aufwand sortiert.'
'</p>' + body,
sev="warn",
anchor="sofortmassnahmen",
)
def render_theme_retention(state: dict) -> str:
s = state.get("retention_theme_summary") or {}
findings = state.get("retention_findings") or []
if not s.get("total"):
return ""
head = (
f'<h3 style="margin:0 0 6px;font-size:{SZ_H3};">'
f'▶ Speicherdauer-Konsistenz (TH-RETENTION)</h3>'
f'<div style="font-size:{SZ_SMALL};color:{TEXT_MUTED};'
f'margin-bottom:6px;">'
f'{s["total"]} Cookies · '
f'<span style="color:#15803d;">{s["passed"]} ✓</span> · '
f'<span style="color:#dc2626;">{s["failed"]} ✗</span> · '
f'<span style="color:#64748b;">{s["incomplete"]} ?</span>'
f'</div>'
)
fails = [f for f in findings
if not f.get("matches")
and f.get("severity_reason") != "incomplete"][:5]
if not fails:
return card(head, sev="pass")
rows = []
for f in fails:
sev = (f.get("severity") or "").upper()
sev_key = "fail" if sev == "HIGH" else "warn"
rows.append([
f'<code>{h(f.get("cookie_name") or "")}</code>',
h(f.get("vendor_name") or ""),
h(f.get("mismatch_type") or ""),
chip(sev, sev_key),
])
body = table(["Cookie", "Vendor", "Mismatch", "Sev"], rows)
sev = "fail" if s.get("failed", 0) else "warn"
return card(head + body, sev=sev)
def render_theme_reachability(state: dict) -> str:
f = state.get("reachability_finding") or {}
if not f:
return ""
passed = f.get("passed")
sev_key = "pass" if passed else (
"fail" if (f.get("severity") or "").upper() == "HIGH" else "warn")
notes_html = "".join(
f'<li style="margin:3px 0;">{h(n)}</li>'
for n in (f.get("notes") or [])
)
sub = (
f'<ul style="margin:6px 0 0 16px;font-size:13px;color:{TEXT};">'
f'{notes_html}</ul>' if notes_html else ""
)
head = (
f'<h3 style="margin:0 0 6px;font-size:{SZ_H3};">'
f'▶ Mobile Reachability (COOKIE-CONSENT-UX-001)</h3>'
f'<div>{chip((f.get("severity") or "PASS").upper(), sev_key)} '
f'<span style="margin-left:6px;font-size:{SZ_SMALL};'
f'color:{TEXT_MUTED};">{h(f.get("severity_reason") or "ok")}</span>'
f'</div>'
)
return card(head + sub, sev=sev_key)
def render_per_theme(state: dict) -> str:
parts = [
render_theme_cookie_banner(state),
render_theme_cookie_inventory(state),
render_theme_retention(state),
render_theme_reachability(state),
]
parts = [p for p in parts if p]
if not parts:
return ""
return section("🎯 5. Pro Thema", "".join(parts), anchor="per-theme")
# ── 6. Audit caveats ────────────────────────────────────────────
def render_caveats(state: dict) -> str:
fs = state.get("audit_quality_findings") or []
if not fs:
return ""
items = []
for f in fs:
sev = (f.get("severity") or "INFO").upper()
sev_key = ("fail" if sev == "HIGH"
else "warn" if sev == "MEDIUM" else "info")
title = h(f.get("title") or f.get("label") or "Vorbehalt")
msg = h(f.get("message") or f.get("hint") or "")
items.append(card(
f'<strong>{chip(sev, sev_key)} {title}</strong>'
f'<div style="margin-top:6px;">{msg}</div>',
sev=sev_key,
))
return section(f"⚠️ 6. Audit-Vorbehalte ({len(fs)})",
"".join(items), sev="warn", anchor="caveats")
# ── 7. Attachments ──────────────────────────────────────────────
def render_attachments(state: dict) -> str:
slices = state.get("cookie_evidence_slices") or []
if not slices:
return ""
meta = state.get("cookie_evidence_meta") or {}
n = len(slices)
body = (
f'<p style="margin:0;">'
f'Beweis-ZIP <code>evidence-{h(state.get("check_id", "")[:8])}.zip</code> '
f'mit <strong>{n}</strong> Slice(s), '
f'manifest.json + audit_metadata.json (SHA256 pro Slice).</p>'
f'<p style="margin:6px 0 0;font-size:{SZ_SMALL};color:{TEXT_MUTED};">'
f'Quelle: {h(meta.get("url") or "")}'
f'</p>'
)
return section("📎 7. Anhänge", body, sev="info", anchor="attach")