feat(audit-mail): P58/P59c/P60b/P61/P62 — Mercedes-Cycle Phase 1 abgeschlossen
CI / nodejs-build (push) Has been skipped
CI / test-go (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 38s
CI / test-python-document-crawler (push) Has been skipped
CI / detect-changes (push) Successful in 12s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / validate-canonical-controls (push) Successful in 14s
CI / loc-budget (push) Failing after 15s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
CI / nodejs-build (push) Has been skipped
CI / test-go (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 38s
CI / test-python-document-crawler (push) Has been skipped
CI / detect-changes (push) Successful in 12s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / validate-canonical-controls (push) Successful in 14s
CI / loc-budget (push) Failing after 15s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
P58 Anti-Audit-Detection robuster (script-domain + settings-spezifisch —
war bereits im Code, jetzt sauber als completed dokumentiert).
P59c DACH-Custom-Cookies in compliance.cookie_library: Borlabs,
etracker, Matomo/Piwik, Userlike, Cookiebot/Cookieyes/Usercentrics,
Akamai/Cloudflare/Datadome Bot-Manager + HubSpot. 21 neue Eintraege
(3 von 24 schon via Open-Cookie-Database vorhanden).
Script: backend-compliance/scripts/seed_dach_cookies.py.
P60b Vendor-Pattern-Dedupe mit Fuzzy-Match (Jaccard >= 0.7) statt exakter
Tuple-Equality. Vendors mit teilweise befuellten Feldern (z.B.
Sitzland eingetragen) fallen nicht mehr aus der globalen Notice —
Bug: Amazon/Psyma/Qualtrics hatten zuvor wiederholte per-row Actions.
P61 "Untergeschobene Cookies"-Erkennung — wenn ein deklarierter Vendor
(z.B. Google Tag Manager) automatisch weitere mitbringt (GA + GCL_AU
+ DoubleClick), werden diese als separater Mail-Block (gelb) mit
COOKIE/VENDOR-Badges + Quellen-Doku ausgewiesen. Neuer Service:
compliance.services.vendor_package_cookies (8 Primary-Vendors mit
je 2-4 implicit Cookies/Vendors).
P62 Marketing-Manager-Disclaimer "Was wir sehen / nicht sehen" als
blauer Box-Block direkt unter dem Critical-Findings-Block. Erklaert
Grenzen unseres Audits (Server-Side-Tracking, Vendor-interne
Datenweitergabe, Cross-Page-Banner) und Risiko des Falschvertrauens
in einen 100%-Score. Neuer Renderer: compliance.api.scope_disclaimer.
Architektur: VVT-Tabellen-Renderer aus agent_doc_check_extras.py (552
LOC -> 242 LOC) in compliance.api.vvt_table_renderer ausgelagert, um den
500-LOC-Hardcap einzuhalten.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -234,302 +234,9 @@ def _category_label(kat: str) -> str:
|
||||
}.get(kat, kat or "—")
|
||||
|
||||
|
||||
def build_vvt_table_html(vendors: list[dict]) -> str:
|
||||
"""Render the per-vendor VVT-style table for the email report.
|
||||
# VVT-Tabelle (gruppiert + P60/P60b Pattern-Notice) wurde in
|
||||
# vvt_table_renderer.py ausgelagert, damit dieses File unter dem
|
||||
# 500-LOC-Hardcap bleibt. Re-export, damit bestehende Aufrufer (z.B.
|
||||
# agent_compliance_check_routes) unveraendert weiter funktionieren.
|
||||
from compliance.api.vvt_table_renderer import build_vvt_table_html # noqa: E402,F401
|
||||
|
||||
Splits vendors into 3-4 sections by recipient_type (Art. 30(1)(d)
|
||||
DSGVO):
|
||||
|
||||
1. INTERNAL — own departments / own systems
|
||||
2. GROUP_COMPANY — parent/subsidiary (if any)
|
||||
3. PROCESSOR — Auftragsverarbeiter (AVV-pflichtig)
|
||||
4. CONTROLLER — joint/independent controllers (Meta, Google,
|
||||
LinkedIn — they build own profiles)
|
||||
5. AUTHORITY / OTHER — rest
|
||||
|
||||
Within each section: rows sorted by compliance_score ascending so
|
||||
the weakest entries surface first.
|
||||
"""
|
||||
if not vendors:
|
||||
return ""
|
||||
|
||||
# Import here to avoid pulling backend service deps at module load
|
||||
from compliance.services.vendor_classifier import RECIPIENT_TYPE_SECTIONS
|
||||
|
||||
# Bucket vendors by recipient_type
|
||||
by_type: dict[str, list[dict]] = {}
|
||||
for v in vendors:
|
||||
rt = (v.get("recipient_type") or "OTHER").upper()
|
||||
by_type.setdefault(rt, []).append(v)
|
||||
|
||||
# Top summary
|
||||
n_total = len(vendors)
|
||||
n_internal = sum(1 for v in vendors
|
||||
if (v.get("recipient_type") or "").upper()
|
||||
in ("INTERNAL", "GROUP_COMPANY"))
|
||||
n_external = n_total - n_internal
|
||||
n_critical = sum(1 for v in vendors if v.get("compliance_score", 0) < 50)
|
||||
|
||||
summary_parts = [f"{n_total} Verarbeitungen erfasst"]
|
||||
if n_internal and n_external:
|
||||
summary_parts.append(
|
||||
f"— {n_internal} eigene + {n_external} externe Empfaenger"
|
||||
)
|
||||
if n_critical:
|
||||
summary_parts.append(
|
||||
f', <strong style="color:#dc2626">{n_critical} unter 50%</strong>'
|
||||
)
|
||||
else:
|
||||
summary_parts.append("— alle ueber 50%")
|
||||
summary = " ".join(summary_parts)
|
||||
|
||||
# P60: Wenn viele Vendors die GLEICHEN Flag-Sets haben, einmal
|
||||
# global hinweisen statt 42x pro Vendor wiederholen.
|
||||
from collections import Counter
|
||||
flag_sets = Counter()
|
||||
for v in vendors:
|
||||
flags = v.get("compliance_flags") or []
|
||||
if flags:
|
||||
flag_sets[tuple(sorted(flags))] += 1
|
||||
pattern_notice = ""
|
||||
if flag_sets:
|
||||
most_common, n_match = flag_sets.most_common(1)[0]
|
||||
share = n_match / max(1, len(vendors))
|
||||
if n_match >= 8 and share >= 0.5:
|
||||
from compliance.services.finding_action_recipes import recipe_for
|
||||
labels = [_flag_short(f) for f in most_common]
|
||||
shared_actions = []
|
||||
for f in most_common:
|
||||
rec = recipe_for(f)
|
||||
if rec:
|
||||
shared_actions.append(
|
||||
f'<li><strong>{_flag_short(f)}:</strong> '
|
||||
f'{rec.get("fix_text", "").splitlines()[0][:180]}</li>'
|
||||
)
|
||||
pattern_notice = (
|
||||
f'<div style="margin:8px 0 12px;padding:10px 14px;'
|
||||
f'background:#fef3c7;border-left:3px solid #d97706;'
|
||||
f'border-radius:4px;font-size:11px;color:#92400e">'
|
||||
f'<strong>Wiederkehrendes Muster ({n_match} von {len(vendors)} '
|
||||
f'Anbietern, {int(share*100)}%):</strong> '
|
||||
f'Bei diesen Anbietern fehlen jeweils: '
|
||||
f'<em>{", ".join(labels)}</em>. '
|
||||
f'Vermutlich systembedingt (z.B. Settings-Export liefert '
|
||||
f'nur Namen, oder Banner-API blockiert Detail-Extraktion). '
|
||||
f'Die globalen Empfehlungen unten gelten fuer all diese Eintraege; '
|
||||
f'in der Tabelle werden sie nicht pro Zeile wiederholt.'
|
||||
+ (f'<ul style="margin:8px 0 0 0;padding-left:20px">{"".join(shared_actions)}</ul>'
|
||||
if shared_actions else '')
|
||||
+ '</div>'
|
||||
)
|
||||
# Mark vendors so _render_vendor_row can suppress redundant actions
|
||||
for v in vendors:
|
||||
if tuple(sorted(v.get("compliance_flags") or [])) == most_common:
|
||||
v["_actions_in_global_notice"] = True
|
||||
|
||||
out: list[str] = [
|
||||
'<div style="font-family:-apple-system,BlinkMacSystemFont,sans-serif;'
|
||||
'max-width:760px;margin:0 auto 16px;padding:12px 16px;'
|
||||
'background:#fafafa;border:1px solid #e5e7eb;border-radius:8px">',
|
||||
'<h3 style="margin:0 0 4px;font-size:14px;color:#334155">'
|
||||
'VVT-Vorschlag: Verarbeitungstaetigkeiten und Empfaenger aus der '
|
||||
'Cookie-Richtlinie</h3>',
|
||||
f'<p style="margin:0 0 10px;font-size:11px;color:#6b7280">{summary}. '
|
||||
'Gruppiert nach Empfaengerkategorie (Art. 30(1)(d) DSGVO). Innerhalb '
|
||||
'jeder Gruppe nach Compliance-Score sortiert. Bei eigenen '
|
||||
'Verarbeitungen (INTERNAL/GROUP) werden Opt-Out und Privacy-Link '
|
||||
'NICHT als Pflicht gewertet — der Widerruf erfolgt ueber das '
|
||||
'Cookie-Banner, Privacy ist in der Haupt-DSI dokumentiert.</p>',
|
||||
pattern_notice,
|
||||
]
|
||||
|
||||
for rtype, section_label in RECIPIENT_TYPE_SECTIONS:
|
||||
rows = by_type.get(rtype) or []
|
||||
if not rows:
|
||||
continue
|
||||
rows = sorted(rows, key=lambda v: v.get("compliance_score", 0))
|
||||
n = len(rows)
|
||||
n_bad = sum(1 for v in rows if v.get("compliance_score", 0) < 50)
|
||||
bad_hint = (f' <span style="color:#dc2626">({n_bad} unter 50%)</span>'
|
||||
if n_bad else "")
|
||||
out.append(
|
||||
f'<h4 style="margin:14px 0 4px;font-size:12px;color:#1e293b;'
|
||||
f'border-top:1px solid #e2e8f0;padding-top:8px">'
|
||||
f'{section_label} <span style="color:#94a3b8;font-weight:400">'
|
||||
f'({n}){bad_hint}</span></h4>'
|
||||
)
|
||||
out.append(_render_vendor_section(rows))
|
||||
|
||||
out.append('</div>')
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _render_vendor_section(rows: list[dict]) -> str:
|
||||
body: list[str] = [
|
||||
'<table style="width:100%;border-collapse:collapse;font-size:11px">'
|
||||
'<thead><tr style="background:#f1f5f9;color:#475569;text-align:left">'
|
||||
'<th style="padding:5px 8px">Name</th>'
|
||||
'<th style="padding:5px 8px">Kategorie</th>'
|
||||
'<th style="padding:5px 8px">Sitz</th>'
|
||||
'<th style="padding:5px 8px;text-align:center">Cookies</th>'
|
||||
'<th style="padding:5px 8px;text-align:center">Opt-Out</th>'
|
||||
'<th style="padding:5px 8px;text-align:center">Privacy</th>'
|
||||
'<th style="padding:5px 8px;text-align:right">Score</th>'
|
||||
'</tr></thead><tbody>',
|
||||
]
|
||||
for v in rows:
|
||||
body.append(_render_vendor_row_full(v))
|
||||
body.append('</tbody></table>')
|
||||
return "".join(body)
|
||||
|
||||
|
||||
def _render_vendor_row_full(v: dict) -> str:
|
||||
rtype = (v.get("recipient_type") or "OTHER").upper()
|
||||
is_own = rtype in ("INTERNAL", "GROUP_COMPANY")
|
||||
cat = (v.get("category") or "").lower()
|
||||
is_necessary = cat in ("necessary", "strictlynecessary")
|
||||
|
||||
name = v.get("name") or "Unbekannt"
|
||||
category = _category_label(v.get("category", ""))
|
||||
country = v.get("country") or ("—" if is_own else "—")
|
||||
cookies = v.get("cookies") or []
|
||||
n_cookies = len(cookies)
|
||||
score = int(v.get("compliance_score", 0))
|
||||
flags = v.get("compliance_flags") or []
|
||||
|
||||
# Opt-Out: nicht erforderlich fuer eigene Verarbeitung oder
|
||||
# technisch notwendige Cookies (§25 Abs. 2 TDDDG).
|
||||
opt_na_reason = ("Nicht erforderlich (eigene Verarbeitung — "
|
||||
"Widerruf ueber Cookie-Banner)") if is_own else (
|
||||
"Nicht erforderlich (§25 Abs. 2 TDDDG — technisch notwendig)"
|
||||
if is_necessary else None
|
||||
)
|
||||
opt_status = _link_status_badge(
|
||||
v.get("opt_out_url"), v.get("opt_out_ok"), v.get("opt_out_status"),
|
||||
na_label=opt_na_reason,
|
||||
)
|
||||
# Privacy: nicht erforderlich fuer eigene Verarbeitung (Haupt-DSI).
|
||||
privacy_na_reason = (
|
||||
"Nicht erforderlich (eigene Verarbeitung — durch Haupt-DSI abgedeckt)"
|
||||
if is_own else None
|
||||
)
|
||||
privacy_status = _link_status_badge(
|
||||
v.get("privacy_policy_url"), v.get("privacy_ok"),
|
||||
v.get("privacy_status"), na_label=privacy_na_reason,
|
||||
)
|
||||
score_color = ("#16a34a" if score >= 80 else
|
||||
"#d97706" if score >= 50 else "#dc2626")
|
||||
|
||||
# Score-Erklaerung: was wurde gewertet, was fehlt
|
||||
# Annahme: Score = bestandene Kriterien / Gesamtkriterien * 100.
|
||||
# Typisch 5 Kriterien fuer EXT: country, cookies, opt_out, privacy, scoring.
|
||||
# Bei INTERNAL/GROUP: opt_out + privacy nicht gewertet (3 Kriterien).
|
||||
n_criteria = 3 if is_own else 5
|
||||
n_failed = len(flags) if flags else 0
|
||||
score_tooltip = (
|
||||
f"{n_criteria - n_failed} von {n_criteria} Kriterien erfuellt"
|
||||
+ (f" — fehlt: {', '.join(_flag_short(f) for f in flags[:3])}"
|
||||
if flags else "")
|
||||
)
|
||||
|
||||
# Inline-Aktions-Anweisungen pro Flag
|
||||
actions_html = ""
|
||||
# P60: skip per-row actions when already covered by global pattern notice
|
||||
skip_actions = bool(v.get("_actions_in_global_notice"))
|
||||
if flags and not skip_actions:
|
||||
from compliance.services.finding_action_recipes import recipe_for
|
||||
action_items = []
|
||||
for f in flags:
|
||||
rec = recipe_for(f)
|
||||
if not rec:
|
||||
continue
|
||||
action_items.append(
|
||||
f'<li style="margin-bottom:6px"><strong>{_flag_short(f)}:</strong> '
|
||||
f'{rec.get("what", "")}<br/>'
|
||||
f'<span style="color:#475569"><strong>Was tun:</strong> '
|
||||
f'{rec.get("fix_text", "").splitlines()[0][:200]}</span><br/>'
|
||||
f'<span style="color:#94a3b8;font-size:9px">Quelle: '
|
||||
f'{rec.get("why", "")[:160]}</span></li>'
|
||||
)
|
||||
if action_items:
|
||||
actions_html = (
|
||||
f'<details style="margin-top:4px"><summary style="cursor:pointer;'
|
||||
f'color:#dc2626;font-size:10px">Was muss ich tun? '
|
||||
f'({len(action_items)} Action{"s" if len(action_items) != 1 else ""})</summary>'
|
||||
f'<ul style="margin:4px 0 0 14px;padding:0;font-size:10px;color:#1e293b">'
|
||||
+ "".join(action_items)
|
||||
+ '</ul></details>'
|
||||
)
|
||||
|
||||
flag_str = ""
|
||||
if flags:
|
||||
flag_str = (
|
||||
f'<div style="font-size:10px;color:#94a3b8;margin-top:2px">'
|
||||
f'{", ".join(flags[:4])}</div>'
|
||||
f'{actions_html}'
|
||||
)
|
||||
risk = v.get("compliance_risk") or {}
|
||||
risk_label = risk.get("label") or ""
|
||||
risk_badge = ""
|
||||
if risk_label and risk_label != "unklar":
|
||||
rc = {"kritisch": ("#dc2626", "#fff"), "hoch": ("#fecaca", "#991b1b"),
|
||||
"mittel": ("#fde68a", "#92400e"), "gering": ("#d1fae5", "#065f46")}.get(risk_label, ("#e5e7eb", "#475569"))
|
||||
risk_badge = (f'<span style="margin-left:6px;padding:1px 5px;border-radius:3px;font-size:9px;'
|
||||
f'background:{rc[0]};color:{rc[1]}">Risk: {risk_label}</span>')
|
||||
return (
|
||||
f'<tr style="border-top:1px solid #e2e8f0">'
|
||||
f'<td style="padding:6px 8px;color:#1e293b;font-size:11px">'
|
||||
f'{name}{risk_badge}{flag_str}</td>'
|
||||
f'<td style="padding:6px 8px;color:#475569;font-size:11px">{category}</td>'
|
||||
f'<td style="padding:6px 8px;color:#475569;font-size:11px">{country}</td>'
|
||||
f'<td style="padding:6px 8px;text-align:center;color:#475569;font-size:11px">'
|
||||
f'{n_cookies}</td>'
|
||||
f'<td style="padding:6px 8px;text-align:center">{opt_status}</td>'
|
||||
f'<td style="padding:6px 8px;text-align:center">{privacy_status}</td>'
|
||||
f'<td style="padding:6px 8px;text-align:right;font-weight:600;'
|
||||
f'color:{score_color};font-size:11px" title="{score_tooltip}">'
|
||||
f'{score}%<div style="font-size:9px;font-weight:400;color:#94a3b8">'
|
||||
f'{n_criteria - n_failed}/{n_criteria}</div></td>'
|
||||
f'</tr>'
|
||||
)
|
||||
|
||||
|
||||
def _flag_short(f: str) -> str:
|
||||
"""Lesbare deutsche Form fuer einen Flag-Token."""
|
||||
labels = {
|
||||
"no_cookies_listed": "Cookies fehlen",
|
||||
"no_country": "Sitzland fehlt",
|
||||
"no_privacy_url": "Privacy-Link fehlt",
|
||||
"broken_privacy_url": "Privacy-Link broken",
|
||||
"no_opt_out_url": "Opt-Out fehlt",
|
||||
"broken_opt_out": "Opt-Out broken",
|
||||
}
|
||||
return labels.get(f, f)
|
||||
|
||||
|
||||
def _link_status_badge(
|
||||
url: str | None,
|
||||
ok: bool | None,
|
||||
status: int | None,
|
||||
na_label: str | None = None,
|
||||
) -> str:
|
||||
"""Render the link-status cell.
|
||||
|
||||
- url + ok -> green check
|
||||
- url + broken -> red cross with status
|
||||
- no url + na_label -> neutral em-dash with explanation tooltip
|
||||
(used for INTERNAL/necessary rows where the field isn't required)
|
||||
- no url + no na_label -> red cross (real gap)
|
||||
"""
|
||||
if not url:
|
||||
if na_label:
|
||||
return ('<span style="color:#94a3b8;font-size:11px" '
|
||||
f'title="{na_label}">—</span>')
|
||||
return ('<span style="color:#dc2626;font-size:11px" '
|
||||
'title="Kein Link">✗</span>')
|
||||
if ok:
|
||||
return ('<span style="color:#16a34a;font-size:11px" '
|
||||
f'title="HTTP {status}">✓</span>')
|
||||
status_str = str(status) if status else "?"
|
||||
return ('<span style="color:#dc2626;font-size:11px" '
|
||||
f'title="HTTP {status_str}">✗ ({status_str})</span>')
|
||||
|
||||
Reference in New Issue
Block a user