feat(cookie-check): enhanced patterns + active opt-out link validator

cookie_checks.py:
- cookie_names_listed: now also matches CMP placeholder notation
  (BMW: 'Adfpc###', 'CT###') and 'Diese Datenverarbeitung verwendet die
  folgenden Cookies oder ähnliche Technologien' as list-shape signal.
  Cryptic vendor names like 'audience', 'adformfrpid' are accepted via
  the surrounding markup, not by hard-coding each one.
- cookie_providers_named: new pattern 'Gesetzt von: <Firma>' (BMW/ePaaS
  per-cookie vendor naming) + recognition of full legal-form names
  (Adform A/S, BMW AG, Adobe Systems Software Ireland Limited).
- cookie_duration_values: now matches 'Ablauf: 1 Jahr' / 'Speicherdauer:
  30 Tage' (BMW format) in addition to the legacy '<n> <unit>'.

New L1 + L2 checks for controller in cookie-policy:
- cookie_controller (L1): the cookie policy must name Verantwortlich(er)
- cookie_controller_address (L2): PLZ + Ort or address keywords
- cookie_controller_contact_or_link (L2): email/phone OR link back to
  Datenschutzerklärung (the practical equivalent — BMW does this)

New L2 checks (parented under opt_out):
- cookie_optout_links: detects per-provider opt-out URLs in the text
- cookie_privacy_policy_links: per-provider privacy-policy URLs

New service: cookie_link_validator.py
- extract_links(text): pulls all https?://… URLs that follow 'Opt-Out
  Link:' / 'Link zur Privacy Policy:' (deduped)
- validate_links(links): probes every URL concurrently (HEAD first, GET
  fallback for 405/403). 10 parallel, 8s per request, 60s batch cap.
  Returns reachable=True/False + status + final_url.
- build_check_items(): renders 2 CheckItems (opt-out + privacy-policy),
  each pass if ALL links 2xx/3xx, fail with up-to-5 broken-link examples.

Hook in _check_single: doc_type=='cookie' triggers the validator after
regex+MC checks. Recomputes correctness with the new L2 items.

This addresses two concrete BMW observations:
1. BMW's per-cookie structure (Name + Zweck + Ablauf, Gesetzt von: …,
   Opt-Out Link: …) now recognised → 'Konkrete Cookie-Namen aufgelistet'
   and 'Konkrete Speicherdauern' should pass.
2. Defective opt-out URLs surface as compliance findings rather than
   silently passing — Art. 7(3) DSGVO requires a working withdrawal
   path per provider.
This commit is contained in:
Benjamin Admin
2026-05-17 09:38:32 +02:00
parent 4a5924b8c4
commit c9c0fb5965
3 changed files with 280 additions and 4 deletions
@@ -701,6 +701,29 @@ async def _check_single(
except Exception as e:
logger.warning("LLM verification skipped: %s", e)
# Cookie-policy only: actively HTTP-probe the Opt-Out + Privacy-Policy
# URLs the document advertises. Broken links make individual provider
# entries non-compliant under Art. 7(3) DSGVO.
if doc_type == "cookie":
try:
from compliance.services.cookie_link_validator import (
extract_links, validate_links, build_check_items,
)
links = extract_links(text)
if links:
logger.info("Cookie-link validator: %d urls extracted from %s",
len(links), label)
validated = await validate_links(links)
for item in build_check_items(validated):
all_checks.append(CheckItem(**item))
# Re-compute correctness with the new L2 items
l2_active = [c for c in all_checks if c.level == 2 and not c.skipped]
l2_passed = sum(1 for c in l2_active if c.passed)
if l2_active:
correctness = round(l2_passed / len(l2_active) * 100)
except Exception as e:
logger.warning("Cookie-link validation skipped for %s: %s", label, e)
non_score = [f for f in findings if "SCORE" not in f.get("code", "")]
return DocCheckResult(
label=label, url=url, doc_type=doc_type,