Files
breakpilot-compliance/admin-compliance/app/sdk/master-controls/page.tsx
T
Benjamin Admin 662327e8b4
CI / nodejs-build (push) Successful in 2m47s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / detect-changes (push) Successful in 10s
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / validate-canonical-controls (push) Successful in 16s
CI / loc-budget (push) Failing after 17s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-python-backend (push) Successful in 42s
CI / test-python-document-crawler (push) Has been skipped
CI / test-go (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
feat(compliance-check): MC-Classification + Embedding + Vendor-Redundanz + Action-Recipes + Borlabs-Features
Massiv-Update auf Basis BMW-Test-Iterationen (v1→v9):

Core Compliance-Check
- Sonnet check_type Klassifikation: text/process/review fuer alle 1874 MCs
  in compliance.doc_check_controls (script + Sidecar /data/mc_classification.db).
  rag_document_checker filtert auf check_type='text' fuer doc_check.
  Plus fits_doc_type-Audit (v2) + ui_only-Audit fuer DSA/E-Commerce-MCs in
  falscher doc_type-Schublade.
- scope_requires-Filter: biometric/ai_decision/child_targeting MCs werden
  per business_profile gefiltert (FRT skipped fuer BMW etc.).
- Embedding-Match (BGE-M3) als Phase-3 nach Regex-Match:
  Per-doc_type-Threshold-Override (impressum 0.50, dse/cookie 0.60),
  Short-Field-Rescue (15-Wort-Chunks) fuer Pflichtfelder im Impressum.
  Title+check_question als Embedding-Input fuer mehr Kontext.
- Cookie-Text-Routing: consent-tester gibt cmp_cookie_text aus dem
  CMP-Reconstruct zurueck, Backend bevorzugt das gegen DOM-Extraction
  wenn richer (BMW 1824 vs 600 Worte).

Vendor-Redundanz + EU-Alternativen + Cost-Saving
- vendor_redundancy.analyze() — funktionale Kategorisierung der CMP-Vendors,
  Detektion von Mehrfach-Anbietern pro Kategorie, EU-Alternative-Lookup
  (Matomo, IONOS, HERE, Friendly Captcha, Smart AdServer, ...).
- vendor_cost_estimator: Tier-Inferenz aus Cookie-Footprint (Cookie-Anzahl
  + Premium-Feature-Cookies + Third-Party-Quote → starter/professional/
  enterprise/premier).
- Self-Service-Werbung (Google/Meta/Pinterest/...) = 0 Lizenz-Kosten
  (nur Media-Spend, separat). DSP-Plattformen behalten enge Range.
- Tier-aware Saving-Range: bei Enterprise/Premier nutzen wir den
  oberen 40-100%-Band der Listpreise, nicht starter→premier.
- Multi-Function-Tools (Matomo Pro, SAP CX, IONOS Cloud, Userlike, Smart
  AdServer, HERE Maps, Vimeo Pro, LamaPoll) — ein Tool ersetzt mehrere
  Kategorien gleichzeitig.

Cookie-Wissens-DB + Funktionale Klassifikation
- cookie_knowledge_db: 50 kuratierte Top-Cookies (Google/Meta/Adobe/MS/...)
  mit vendor, exact_purpose, data_collected, IAB-TCF-IDs, reid_risk,
  schrems_ii_status, EuGH-Urteile, EU-Alternative.
- cookie_function_classifier: pro Cookie funktionale Rolle (tracking_id,
  ad_pixel, session_id, ab_test, csrf, ...) + blocking_impact.

Country-Inferenz aus Rechtsform
- cookie_link_validator: Country-Field wird aus Vendor-Name abgeleitet
  (A/S=DK, GmbH=DE, Inc=US, B.V.=NL, ...) plus Vendor-Lookup-Table.
  Reduziert false-positive no_country-Flags bei eindeutig-EU-Vendors
  (Adform DK, Pinterest IE).

Action-Recipes + Doc-Anchor-Locator
- finding_action_recipes: pro Finding-Typ (no_cookies_listed, no_country,
  broken_opt_out, "Auftragsverarbeiter erwaehnen", "Art. 22 Profiling",
  ...) eine strukturierte Anweisung mit what/why/fix_text/where/example.
  Zum 1:1-Einfuegen in Kunden-Dokumente.
- doc_anchor_locator: Embedding-basiert (BGE-M3 cosine) — sucht den
  passenden Absatz im existierenden Kundendokument fuer jeden Finding.
  Per-Run Thread-Local-Cache. Fallback: keyword-Match.
- Email-Rendering integriert Recipe + Anchor pro Doc-Pruefungs-Fail
  + Vendor-Flag-Liste mit aufklappbarer Action-Liste.
- Score-Erklaerung pro Vendor-Zeile (3/5-Untertitel + Tooltip).

Migration-Pipeline (Compliance-Check -> Customer Banner/Documents)
- migration_to_banner.py: Vendor-Liste -> CookieBannerConfig mit
  4 Kategorien + Review-Flags.
- migration_to_document.py: Vendor-Liste -> Cookie-Policy + VVT-Register
  + Privacy-Policy-Pre-Fills.
- agent_migration_routes: 3 Preview-Endpoints (banner-preview,
  document-preview, summary). Persistierung der cmp_vendors in
  /data/compliance_audits.db check_payloads-Tabelle.

Borlabs-Parity Cookie-Banner-Features
- Consent-Historie im Banner: window.bpShowConsentHistory() + localStorage.
- Content-Blocker: cookie-banner-content-blocker.ts — YouTube/Maps/Video
  Placeholder bis Einwilligung.
- Google Consent Mode v2 erweitert: wait_for_update + region=EEA/CH/GB.
- Consent-Log Export (CSV/JSON) per einwilligungen_export_routes.

Bug-Fixes
- canonical_control_routes: _jsonish-Helper fuer string-typed jsonb,
  similar-controls-Endpoint mit _has_embedding_col()-Cache (kein 500 mehr).
- Control-Library Frontend: defensive .map-Coercer in 2 Detail-Views.
- Embedding-Service-Batching (32er Batches statt 165 in einem Call).
- KeyError 'control_id' in MC-Result-Aggregation (defensive .get).
- Master-Controls-Klick-Through von /sdk/master-controls auf
  /sdk/control-library?control=<id> mit URL-Param-Auto-Open.
- Dockerfile: /data pre-chowned auf appuser (Audit-DB-Schreibrecht).
- Cookie-Text-Routing-Bug (cmp_reconstructed > DOM-extraction).
- doc_type-aware MC-Filter (statt all-text-MCs).
- Master-Contract-Dedup (60 BMW-Internal-Eintraege = 1 Adobe-Vertrag).
- A3-v2-Audit hat 24 UI-Sprache-MCs als 'process' reklassifiziert.

Tests
- test_migration_mappers.py (9 Tests)
- test_migration_endpoints.py (4 Tests)

Skripte (one-shot)
- classify_mc_check_type.py (v1) + _v2 (PK=control_id,doc_type)
- audit_mc_doctype_fit.py (v1 fits) + _v2 (ui_only + scope_requires)

BMW-Run-Bilanz v1 (broken) -> v9 (alle Fixes):
  DSE     7,5% -> 81-83%
  Impressum 4%   -> 100% (6 echte MCs alle erfuellt)
  Cookie  0%    -> 79-83% (CMP-Text-Routing + Embedding)
  Plus: 10 Konsolidierungs-Kategorien, geschaetzte Saving 200k-3M / Jahr
  Plus: Action-Recipes + Doc-Anchors fuer jeden Fail

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:30:08 +02:00

248 lines
9.3 KiB
TypeScript

'use client'
import React, { useState, useEffect } from 'react'
import { ControlListView } from '../control-library/components/ControlListView'
import { useControlLibraryState } from '../control-library/components/useControlLibraryState'
/**
* Master Controls page — reuses the Control Library UI exactly,
* but shows Master Controls (13.5K grouped controls) instead of
* individual atomic controls (272K).
*
* The MC API route (/api/sdk/v1/master-controls) returns data in
* the same format as the canonical controls endpoint.
*/
export default function MasterControlsPage() {
// Reuse the exact same state hook — it fetches from BACKEND_URL
// We override BACKEND_URL via a wrapper, but for now we reuse as-is
// since both endpoints speak the same format.
const state = useControlLibraryState('/api/sdk/v1/master-controls')
if (state.loading && state.controls.length === 0) {
return (
<div className="flex items-center justify-center h-96">
<div className="animate-spin rounded-full h-8 w-8 border-2 border-purple-600 border-t-transparent" />
</div>
)
}
if (state.error) {
return (
<div className="flex items-center justify-center h-96">
<p className="text-red-600">{state.error}</p>
</div>
)
}
// DETAIL mode — show MC members
if (state.mode === 'detail' && state.selectedControl) {
return (
<MCDetail
mc={state.selectedControl}
onBack={() => { state.setMode('list'); state.setSelectedControl(null) }}
/>
)
}
// LIST mode — exact same UI as Control Library
return (
<ControlListView
frameworks={state.frameworks}
controls={state.controls}
totalCount={state.totalCount}
meta={state.meta}
loading={state.loading}
reviewCount={0}
bulkProcessing={false}
showStats={state.showStats}
processedStats={state.processedStats}
showGenerator={false}
currentPage={state.currentPage}
totalPages={state.totalPages}
sortBy={state.sortBy}
searchQuery={state.searchQuery}
severityFilter={state.severityFilter}
domainFilter={state.domainFilter}
stateFilter={state.stateFilter}
verificationFilter={state.verificationFilter}
categoryFilter={state.categoryFilter}
evidenceTypeFilter={state.evidenceTypeFilter}
audienceFilter={state.audienceFilter}
sourceFilter={state.sourceFilter}
typeFilter={state.typeFilter}
hideDuplicates={state.hideDuplicates}
setSearchQuery={state.setSearchQuery}
setSeverityFilter={state.setSeverityFilter}
setDomainFilter={state.setDomainFilter}
setStateFilter={state.setStateFilter}
setVerificationFilter={state.setVerificationFilter}
setCategoryFilter={state.setCategoryFilter}
setEvidenceTypeFilter={state.setEvidenceTypeFilter}
setAudienceFilter={state.setAudienceFilter}
setSourceFilter={state.setSourceFilter}
setTypeFilter={state.setTypeFilter}
setHideDuplicates={state.setHideDuplicates}
setSortBy={state.setSortBy}
setShowStats={state.setShowStats}
setShowGenerator={() => {}}
setCurrentPage={state.setCurrentPage}
onSelectControl={(ctrl) => { state.setSelectedControl(ctrl); state.setMode('detail') }}
onCreateMode={() => {}}
onEnterReview={() => {}}
onBulkReject={async () => {}}
onRefresh={() => { state.loadControls(); state.loadMeta() }}
onLoadStats={state.loadProcessedStats}
onFullReload={state.fullReload}
/>
)
}
// ── MC Detail Panel ─────────────────────────────────────────────
interface Member {
control_id: string
title: string
severity: string
phase: string
action: string
regulation_source?: string
regulation_article?: string
}
const SEV = {
critical: 'bg-red-100 text-red-800',
high: 'bg-orange-100 text-orange-800',
medium: 'bg-yellow-100 text-yellow-800',
low: 'bg-blue-100 text-blue-800',
} as Record<string, string>
function MCDetail({ mc, onBack }: { mc: Record<string, unknown>; onBack: () => void }) {
const [members, setMembers] = useState<Member[]>([])
const [loading, setLoading] = useState(true)
const [phaseFilter, setPhaseFilter] = useState('')
const mcId = (mc.control_id || mc.master_control_id || '') as string
const mcName = (mc.title || mc.canonical_name || '') as string
const totalControls = (mc.total_controls || 0) as number
const phases = (mc.phases_covered || []) as string[]
useEffect(() => {
setLoading(true)
fetch(`/api/sdk/v1/master-controls?endpoint=control&id=${mcId}`)
.then(r => r.ok ? r.json() : null)
.then(data => {
if (data?.members) setMembers(data.members)
else if (data?.requirements) {
// Fallback: parse requirements strings
setMembers((data.requirements as string[]).map((req: string) => {
const match = req.match(/^\[(\w+)\]\s+(\S+):\s+(.+)$/)
return match
? { control_id: match[2], title: match[3], phase: match[1], action: '', severity: '' }
: { control_id: '', title: req, phase: '', action: '', severity: '' }
}))
}
})
.catch(() => {})
.finally(() => setLoading(false))
}, [mcId])
const filtered = phaseFilter ? members.filter(m => m.phase === phaseFilter) : members
const uniquePhases = [...new Set(members.map(m => m.phase).filter(Boolean))]
const phaseGroups = uniquePhases.reduce((acc, p) => {
acc[p] = members.filter(m => m.phase === p).length
return acc
}, {} as Record<string, number>)
return (
<div className="max-w-5xl mx-auto">
{/* Header */}
<button onClick={onBack} className="mb-4 text-purple-600 hover:text-purple-800 text-sm flex items-center gap-1">
Zurueck zur Liste
</button>
<div className="bg-white rounded-xl shadow-sm border border-gray-200 p-6 mb-4">
<h1 className="text-2xl font-bold text-gray-900">{mcName}</h1>
<p className="text-gray-500 mt-1">{mcId} {totalControls} Atomic Controls</p>
{/* Phase badges */}
<div className="flex flex-wrap gap-2 mt-4">
{uniquePhases.map(p => (
<button
key={p}
onClick={() => setPhaseFilter(phaseFilter === p ? '' : p)}
className={`px-3 py-1 rounded-full text-xs font-medium border transition-colors ${
phaseFilter === p
? 'bg-purple-100 border-purple-400 text-purple-800'
: 'border-gray-200 text-gray-600 hover:bg-gray-50'
}`}
>
{p} ({phaseGroups[p]})
</button>
))}
{phaseFilter && (
<button onClick={() => setPhaseFilter('')} className="text-xs text-gray-400 hover:text-gray-600 ml-2">
Filter aufheben
</button>
)}
</div>
</div>
{/* Members */}
<div className="bg-white rounded-xl shadow-sm border border-gray-200 overflow-hidden">
<div className="px-4 py-3 bg-gray-50 border-b text-sm text-gray-500">
{filtered.length} von {members.length} Controls{phaseFilter ? ` (Phase: ${phaseFilter})` : ''}
</div>
{loading ? (
<div className="p-8 text-center">
<div className="animate-spin rounded-full h-6 w-6 border-2 border-purple-600 border-t-transparent mx-auto" />
</div>
) : (
<div className="divide-y divide-gray-50">
{filtered.map((m, i) => {
const inner = (
<>
<div className="flex items-center gap-2 mb-1">
<span className="text-xs font-mono text-gray-400">{m.control_id}</span>
{m.severity && (
<span className={`px-1.5 py-0.5 rounded text-[10px] font-bold ${SEV[m.severity] || 'bg-gray-100 text-gray-600'}`}>
{m.severity}
</span>
)}
{m.phase && (
<span className="text-[10px] text-purple-600 bg-purple-50 px-1.5 py-0.5 rounded">
{m.phase}
</span>
)}
{m.action && (
<span className="text-[10px] text-gray-400">{m.action}</span>
)}
</div>
<p className="text-sm text-gray-900">{m.title}</p>
{m.regulation_source && (
<p className="text-xs text-blue-600 mt-1">
{m.regulation_source} {m.regulation_article}
</p>
)}
</>
)
return m.control_id ? (
<a key={i}
href={`/sdk/control-library?control=${encodeURIComponent(m.control_id)}`}
className="block px-4 py-3 hover:bg-purple-50/40 transition-colors">
{inner}
</a>
) : (
<div key={i} className="px-4 py-3 hover:bg-gray-50">{inner}</div>
)
})}
{filtered.length === 0 && !loading && (
<div className="p-8 text-center text-gray-400">Keine Controls gefunden</div>
)}
</div>
)}
</div>
</div>
)
}