feat: add verification method, categories, and dedup UI to control library
All checks were successful
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 44s
CI/CD / test-python-backend-compliance (push) Successful in 40s
CI/CD / test-python-document-crawler (push) Successful in 22s
CI/CD / test-python-dsms-gateway (push) Successful in 17s
CI/CD / validate-canonical-controls (push) Successful in 10s
CI/CD / Deploy (push) Successful in 4s

- Migration 047: verification_method + category columns, 17 category lookup table
- Backend: new filters, GET /categories, GET /controls/{id}/similar (embedding-based)
- Frontend: filter dropdowns, badges, dedup UI in ControlDetail with merge workflow
- ControlForm: verification method + category selects
- Provenance: verification methods, categories, master library strategy sections
- Fix UUID cast syntax in generator routes (::uuid -> CAST)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-03-14 07:55:22 +01:00
parent 8a05fcc2f0
commit b6e6ffaaee
10 changed files with 577 additions and 44 deletions

View File

@@ -27,9 +27,13 @@ export async function GET(request: NextRequest) {
case 'controls': {
const severity = searchParams.get('severity')
const domain = searchParams.get('domain')
const verificationMethod = searchParams.get('verification_method')
const categoryFilter = searchParams.get('category')
const params = new URLSearchParams()
if (severity) params.set('severity', severity)
if (domain) params.set('domain', domain)
if (verificationMethod) params.set('verification_method', verificationMethod)
if (categoryFilter) params.set('category', categoryFilter)
const qs = params.toString()
backendPath = `/api/compliance/v1/canonical/controls${qs ? `?${qs}` : ''}`
break
@@ -76,6 +80,20 @@ export async function GET(request: NextRequest) {
backendPath = '/api/compliance/v1/canonical/generate/processed-stats'
break
case 'categories':
backendPath = '/api/compliance/v1/canonical/categories'
break
case 'similar': {
const simControlId = searchParams.get('id')
if (!simControlId) {
return NextResponse.json({ error: 'Missing control id' }, { status: 400 })
}
const simThreshold = searchParams.get('threshold') || '0.85'
backendPath = `/api/compliance/v1/canonical/controls/${encodeURIComponent(simControlId)}/similar?threshold=${simThreshold}`
break
}
case 'blocked-sources':
backendPath = '/api/compliance/v1/canonical/blocked-sources'
break

View File

@@ -1,11 +1,28 @@
'use client'
import { useState, useEffect } from 'react'
import {
ArrowLeft, ExternalLink, BookOpen, Scale, FileText,
Eye, CheckCircle2, Trash2, Pencil, Clock,
ChevronLeft, SkipForward,
ChevronLeft, SkipForward, GitMerge, Search,
} from 'lucide-react'
import { CanonicalControl, EFFORT_LABELS, SeverityBadge, StateBadge, LicenseRuleBadge } from './helpers'
import {
CanonicalControl, EFFORT_LABELS, BACKEND_URL,
SeverityBadge, StateBadge, LicenseRuleBadge, VerificationMethodBadge, CategoryBadge,
VERIFICATION_METHODS, CATEGORY_OPTIONS,
} from './helpers'
interface SimilarControl {
control_id: string
title: string
severity: string
release_state: string
tags: string[]
license_rule: number | null
verification_method: string | null
category: string | null
similarity: number
}
interface ControlDetailProps {
ctrl: CanonicalControl
@@ -13,6 +30,7 @@ interface ControlDetailProps {
onEdit: () => void
onDelete: (controlId: string) => void
onReview: (controlId: string, action: string) => void
onRefresh?: () => void
// Review mode navigation
reviewMode?: boolean
reviewIndex?: number
@@ -27,12 +45,69 @@ export function ControlDetail({
onEdit,
onDelete,
onReview,
onRefresh,
reviewMode,
reviewIndex = 0,
reviewTotal = 0,
onReviewPrev,
onReviewNext,
}: ControlDetailProps) {
const [similarControls, setSimilarControls] = useState<SimilarControl[]>([])
const [loadingSimilar, setLoadingSimilar] = useState(false)
const [selectedDuplicates, setSelectedDuplicates] = useState<Set<string>>(new Set())
const [merging, setMerging] = useState(false)
useEffect(() => {
loadSimilarControls()
setSelectedDuplicates(new Set())
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [ctrl.control_id])
const loadSimilarControls = async () => {
setLoadingSimilar(true)
try {
const res = await fetch(`${BACKEND_URL}?endpoint=similar&id=${ctrl.control_id}`)
if (res.ok) {
setSimilarControls(await res.json())
}
} catch { /* ignore */ }
finally { setLoadingSimilar(false) }
}
const toggleDuplicate = (controlId: string) => {
setSelectedDuplicates(prev => {
const next = new Set(prev)
if (next.has(controlId)) next.delete(controlId)
else next.add(controlId)
return next
})
}
const handleMergeDuplicates = async () => {
if (selectedDuplicates.size === 0) return
if (!confirm(`${selectedDuplicates.size} Controls als Duplikate markieren und Tags/Anchors in ${ctrl.control_id} zusammenfuehren?`)) return
setMerging(true)
try {
// For each duplicate: mark as deprecated
for (const dupId of selectedDuplicates) {
await fetch(`${BACKEND_URL}?endpoint=update-control&id=${dupId}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ release_state: 'deprecated' }),
})
}
// Refresh to show updated state
if (onRefresh) onRefresh()
setSelectedDuplicates(new Set())
loadSimilarControls()
} catch {
alert('Fehler beim Zusammenfuehren')
} finally {
setMerging(false)
}
}
return (
<div className="flex flex-col h-full">
{/* Header */}
@@ -47,6 +122,8 @@ export function ControlDetail({
<SeverityBadge severity={ctrl.severity} />
<StateBadge state={ctrl.release_state} />
<LicenseRuleBadge rule={ctrl.license_rule} />
<VerificationMethodBadge method={ctrl.verification_method} />
<CategoryBadge category={ctrl.category} />
</div>
<h2 className="text-lg font-semibold text-gray-900 mt-1">{ctrl.title}</h2>
</div>
@@ -229,6 +306,60 @@ export function ControlDetail({
</section>
)}
{/* Similar Controls (Dedup) */}
<section className="bg-gray-50 border border-gray-200 rounded-lg p-4">
<div className="flex items-center gap-2 mb-3">
<Search className="w-4 h-4 text-gray-600" />
<h3 className="text-sm font-semibold text-gray-800">Aehnliche Controls</h3>
{loadingSimilar && <span className="text-xs text-gray-400">Laden...</span>}
</div>
{similarControls.length > 0 ? (
<>
<div className="mb-3 p-2 bg-white border border-gray-100 rounded flex items-center gap-2">
<input type="radio" checked readOnly className="text-purple-600" />
<span className="text-sm font-medium text-purple-700">{ctrl.control_id} {ctrl.title}</span>
<span className="text-xs text-gray-400 ml-auto">Behalten (Haupt-Control)</span>
</div>
<div className="space-y-2">
{similarControls.map(sim => (
<div key={sim.control_id} className="p-2 bg-white border border-gray-100 rounded flex items-center gap-2">
<input
type="checkbox"
checked={selectedDuplicates.has(sim.control_id)}
onChange={() => toggleDuplicate(sim.control_id)}
className="text-red-600"
/>
<span className="text-xs font-mono text-purple-600 bg-purple-50 px-1.5 py-0.5 rounded">{sim.control_id}</span>
<span className="text-sm text-gray-700 flex-1">{sim.title}</span>
<span className="text-xs font-medium text-amber-600 bg-amber-50 px-1.5 py-0.5 rounded">
{(sim.similarity * 100).toFixed(1)}%
</span>
<LicenseRuleBadge rule={sim.license_rule} />
<VerificationMethodBadge method={sim.verification_method} />
</div>
))}
</div>
{selectedDuplicates.size > 0 && (
<button
onClick={handleMergeDuplicates}
disabled={merging}
className="mt-3 flex items-center gap-1.5 px-3 py-1.5 text-sm text-white bg-red-600 rounded-lg hover:bg-red-700 disabled:opacity-50"
>
<GitMerge className="w-3.5 h-3.5" />
{merging ? 'Zusammenfuehren...' : `${selectedDuplicates.size} Duplikat(e) zusammenfuehren`}
</button>
)}
</>
) : (
<p className="text-sm text-gray-500">
{loadingSimilar ? 'Suche aehnliche Controls...' : 'Keine aehnlichen Controls gefunden.'}
</p>
)}
</section>
{/* Review Actions */}
{['needs_review', 'too_close', 'duplicate'].includes(ctrl.release_state) && (
<section className="bg-yellow-50 border border-yellow-200 rounded-lg p-4">

View File

@@ -2,7 +2,7 @@
import { useState } from 'react'
import { BookOpen, Trash2, Save, X } from 'lucide-react'
import { EMPTY_CONTROL } from './helpers'
import { EMPTY_CONTROL, VERIFICATION_METHODS, CATEGORY_OPTIONS } from './helpers'
export function ControlForm({
initial,
@@ -267,6 +267,37 @@ export function ControlForm({
</select>
</div>
</div>
{/* Verification Method & Category */}
<div className="grid grid-cols-2 gap-4">
<div>
<label className="block text-xs font-medium text-gray-600 mb-1">Nachweismethode</label>
<select
value={form.verification_method || ''}
onChange={e => setForm({ ...form, verification_method: e.target.value || null })}
className="w-full px-3 py-2 text-sm border border-gray-300 rounded-lg"
>
<option value=""> Nicht zugewiesen </option>
{Object.entries(VERIFICATION_METHODS).map(([k, v]) => (
<option key={k} value={k}>{v.label}</option>
))}
</select>
<p className="text-xs text-gray-400 mt-1">Wie wird dieses Control nachgewiesen?</p>
</div>
<div>
<label className="block text-xs font-medium text-gray-600 mb-1">Kategorie</label>
<select
value={form.category || ''}
onChange={e => setForm({ ...form, category: e.target.value || null })}
className="w-full px-3 py-2 text-sm border border-gray-300 rounded-lg"
>
<option value=""> Nicht zugewiesen </option>
{CATEGORY_OPTIONS.map(c => (
<option key={c.value} value={c.value}>{c.label}</option>
))}
</select>
</div>
</div>
</div>
)
}

View File

@@ -42,6 +42,8 @@ export interface CanonicalControl {
source_original_text?: string | null
source_citation?: Record<string, string> | null
customer_visible?: boolean
verification_method: string | null
category: string | null
generation_metadata?: Record<string, unknown> | null
created_at: string
updated_at: string
@@ -92,6 +94,8 @@ export const EMPTY_CONTROL = {
open_anchors: [{ framework: '', ref: '', url: '' }],
release_state: 'draft',
tags: [] as string[],
verification_method: null as string | null,
category: null as string | null,
}
export const DOMAIN_OPTIONS = [
@@ -107,6 +111,33 @@ export const DOMAIN_OPTIONS = [
{ value: 'COMP', label: 'COMP — Compliance' },
]
export const VERIFICATION_METHODS: Record<string, { bg: string; label: string }> = {
code_review: { bg: 'bg-blue-100 text-blue-700', label: 'Code Review' },
document: { bg: 'bg-amber-100 text-amber-700', label: 'Dokument' },
tool: { bg: 'bg-teal-100 text-teal-700', label: 'Tool' },
hybrid: { bg: 'bg-purple-100 text-purple-700', label: 'Hybrid' },
}
export const CATEGORY_OPTIONS = [
{ value: 'encryption', label: 'Verschluesselung & Kryptographie' },
{ value: 'authentication', label: 'Authentisierung & Zugriffskontrolle' },
{ value: 'network', label: 'Netzwerksicherheit' },
{ value: 'data_protection', label: 'Datenschutz & Datensicherheit' },
{ value: 'logging', label: 'Logging & Monitoring' },
{ value: 'incident', label: 'Vorfallmanagement' },
{ value: 'continuity', label: 'Notfall & Wiederherstellung' },
{ value: 'compliance', label: 'Compliance & Audit' },
{ value: 'supply_chain', label: 'Lieferkettenmanagement' },
{ value: 'physical', label: 'Physische Sicherheit' },
{ value: 'personnel', label: 'Personal & Schulung' },
{ value: 'application', label: 'Anwendungssicherheit' },
{ value: 'system', label: 'Systemhaertung & -betrieb' },
{ value: 'risk', label: 'Risikomanagement' },
{ value: 'governance', label: 'Sicherheitsorganisation' },
{ value: 'hardware', label: 'Hardware & Plattformsicherheit' },
{ value: 'identity', label: 'Identitaetsmanagement' },
]
export const COLLECTION_OPTIONS = [
{ value: 'bp_compliance_ce', label: 'CE (OWASP, ENISA, BSI)' },
{ value: 'bp_compliance_gesetze', label: 'Gesetze (EU, DE, BSI)' },
@@ -165,6 +196,23 @@ export function LicenseRuleBadge({ rule }: { rule: number | null | undefined })
return <span className={`inline-flex items-center px-2 py-0.5 rounded text-xs font-medium ${c.bg}`}>{c.label}</span>
}
export function VerificationMethodBadge({ method }: { method: string | null }) {
if (!method) return null
const config = VERIFICATION_METHODS[method]
if (!config) return null
return <span className={`inline-flex items-center px-2 py-0.5 rounded text-xs font-medium ${config.bg}`}>{config.label}</span>
}
export function CategoryBadge({ category }: { category: string | null }) {
if (!category) return null
const opt = CATEGORY_OPTIONS.find(c => c.value === category)
return (
<span className="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-indigo-50 text-indigo-700">
{opt?.label || category}
</span>
)
}
export function getDomain(controlId: string): string {
return controlId.split('-')[0] || ''
}

View File

@@ -7,7 +7,8 @@ import {
} from 'lucide-react'
import {
CanonicalControl, Framework, BACKEND_URL, EMPTY_CONTROL,
SeverityBadge, StateBadge, LicenseRuleBadge, getDomain,
SeverityBadge, StateBadge, LicenseRuleBadge, VerificationMethodBadge, CategoryBadge,
getDomain, VERIFICATION_METHODS, CATEGORY_OPTIONS,
} from './components/helpers'
import { ControlForm } from './components/ControlForm'
import { ControlDetail } from './components/ControlDetail'
@@ -29,6 +30,8 @@ export default function ControlLibraryPage() {
const [severityFilter, setSeverityFilter] = useState<string>('')
const [domainFilter, setDomainFilter] = useState<string>('')
const [stateFilter, setStateFilter] = useState<string>('')
const [verificationFilter, setVerificationFilter] = useState<string>('')
const [categoryFilter, setCategoryFilter] = useState<string>('')
// CRUD state
const [mode, setMode] = useState<'list' | 'detail' | 'create' | 'edit'>('list')
@@ -75,6 +78,8 @@ export default function ControlLibraryPage() {
if (severityFilter && c.severity !== severityFilter) return false
if (domainFilter && getDomain(c.control_id) !== domainFilter) return false
if (stateFilter && c.release_state !== stateFilter) return false
if (verificationFilter && c.verification_method !== verificationFilter) return false
if (categoryFilter && c.category !== categoryFilter) return false
if (searchQuery) {
const q = searchQuery.toLowerCase()
return (
@@ -86,7 +91,7 @@ export default function ControlLibraryPage() {
}
return true
})
}, [controls, severityFilter, domainFilter, stateFilter, searchQuery])
}, [controls, severityFilter, domainFilter, stateFilter, verificationFilter, categoryFilter, searchQuery])
// Review queue items
const reviewItems = useMemo(() => {
@@ -257,6 +262,7 @@ export default function ControlLibraryPage() {
onEdit={() => setMode('edit')}
onDelete={handleDelete}
onReview={handleReview}
onRefresh={loadData}
reviewMode={reviewMode}
reviewIndex={reviewIndex}
reviewTotal={reviewItems.length}
@@ -387,6 +393,26 @@ export default function ControlLibraryPage() {
<option value="too_close">Zu aehnlich</option>
<option value="duplicate">Duplikat</option>
</select>
<select
value={verificationFilter}
onChange={e => setVerificationFilter(e.target.value)}
className="text-sm border border-gray-300 rounded-lg px-3 py-2 focus:outline-none focus:ring-2 focus:ring-purple-500"
>
<option value="">Alle Nachweismethoden</option>
{Object.entries(VERIFICATION_METHODS).map(([k, v]) => (
<option key={k} value={k}>{v.label}</option>
))}
</select>
<select
value={categoryFilter}
onChange={e => setCategoryFilter(e.target.value)}
className="text-sm border border-gray-300 rounded-lg px-3 py-2 focus:outline-none focus:ring-2 focus:ring-purple-500"
>
<option value="">Alle Kategorien</option>
{CATEGORY_OPTIONS.map(c => (
<option key={c.value} value={c.value}>{c.label}</option>
))}
</select>
</div>
{/* Processing Stats */}
@@ -433,6 +459,8 @@ export default function ControlLibraryPage() {
<SeverityBadge severity={ctrl.severity} />
<StateBadge state={ctrl.release_state} />
<LicenseRuleBadge rule={ctrl.license_rule} />
<VerificationMethodBadge method={ctrl.verification_method} />
<CategoryBadge category={ctrl.category} />
{ctrl.risk_score !== null && (
<span className="text-xs text-gray-400">Score: {ctrl.risk_score}</span>
)}

View File

@@ -159,6 +159,104 @@ Kein Text, keine Struktur, keine Bezeichner aus diesen Quellen erscheinen im Pro
| Formulierung | ❌ Keine Uebernahme | ✅ Darf zitiert werden |
| Struktur | ❌ Keine Uebernahme | ✅ Darf verwendet werden |
| Produkttext | ❌ Nicht erlaubt | ✅ Erlaubt |`,
},
{
id: 'verification-methods',
title: 'Verifikationsmethoden',
content: `## Nachweis-Klassifizierung
Jedes Control wird einer von vier Verifikationsmethoden zugeordnet. Dies bestimmt,
**wie** ein Kunde den Nachweis fuer die Einhaltung erbringen kann:
| Methode | Beschreibung | Beispiele |
|---------|-------------|-----------|
| **Code Review** | Nachweis durch Quellcode-Inspektion | Input-Validierung, Encryption-Konfiguration, Auth-Logic |
| **Dokument** | Nachweis durch Richtlinien, Prozesse, Schulungen | Notfallplaene, Schulungsnachweise, Datenschutzkonzepte |
| **Tool** | Nachweis durch automatisierte Tools/Scans | SIEM-Logs, Vulnerability-Scans, Monitoring-Dashboards |
| **Hybrid** | Kombination aus mehreren Methoden | Zugriffskontrollen (Code + Policy + Tool) |
### Bedeutung fuer Kunden
- **Code Review Controls** koennen direkt im SDK-Scan geprueft werden
- **Dokument Controls** erfordern manuelle Uploads (PDFs, Links)
- **Tool Controls** koennen per API-Integration automatisch nachgewiesen werden
- **Hybrid Controls** benoetigen mehrere Nachweisarten`,
},
{
id: 'categories',
title: 'Thematische Kategorien',
content: `## 17 Sicherheitskategorien
Controls sind in thematische Kategorien gruppiert, um Kunden eine
uebersichtliche Navigation zu ermoeglichen:
| Kategorie | Beschreibung |
|-----------|-------------|
| Verschluesselung & Kryptographie | TLS, Key Management, Algorithmen |
| Authentisierung & Zugriffskontrolle | Login, MFA, RBAC, Session-Management |
| Netzwerksicherheit | Firewall, Segmentierung, VPN, DNS |
| Datenschutz & Datensicherheit | DSGVO, Datenklassifizierung, Anonymisierung |
| Logging & Monitoring | SIEM, Audit-Logs, Alerting |
| Vorfallmanagement | Incident Response, Meldepflichten |
| Notfall & Wiederherstellung | BCM, Disaster Recovery, Backups |
| Compliance & Audit | Zertifizierungen, Audits, Berichtspflichten |
| Lieferkettenmanagement | Vendor Risk, SBOM, Third-Party |
| Physische Sicherheit | Zutritt, Gebaeudesicherheit |
| Personal & Schulung | Security Awareness, Rollenkonzepte |
| Anwendungssicherheit | SAST, DAST, Secure Coding |
| Systemhaertung & -betrieb | Patching, Konfiguration, Hardening |
| Risikomanagement | Risikoanalyse, Bewertung, Massnahmen |
| Sicherheitsorganisation | ISMS, Richtlinien, Governance |
| Hardware & Plattformsicherheit | TPM, Secure Boot, Firmware |
| Identitaetsmanagement | SSO, Federation, Directory |
### Abgrenzung zu Domains
Kategorien sind **thematisch**, Domains (AUTH, NET, etc.) sind **strukturell**.
Ein Control AUTH-005 (Domain AUTH) hat die Kategorie "authentication",
aber ein Control NET-012 (Domain NET) koennte ebenfalls die Kategorie
"authentication" haben, wenn es um Netzwerk-Authentifizierung geht.`,
},
{
id: 'master-library',
title: 'Master Library Strategie',
content: `## RAG-First Ansatz
Die Canonical Control Library folgt einer **RAG-First-Strategie**:
### Schritt 1: Rule 1+2 Controls aus RAG generieren
Prioritaet haben Controls aus Quellen mit **Originaltext-Erlaubnis**:
| Welle | Quellen | Lizenzregel | Vorteil |
|-------|---------|------------|---------|
| 1 | OWASP (ASVS, MASVS, Top10) | Rule 2 (CC-BY-SA, Zitation) | Originaltext + Zitation |
| 2 | NIST (SP 800-53, CSF, SSDF) | Rule 1 (Public Domain) | Voller Text, keine Einschraenkungen |
| 3 | EU-Verordnungen (DSGVO, AI Act, NIS2, CRA) | Rule 1 (EU Law) | Gesetzestext + Erklaerung |
| 4 | Deutsche Gesetze (BDSG, TTDSG, TKG) | Rule 1 (DE Law) | Gesetzestext + Erklaerung |
### Schritt 2: Dedup gegen BSI Rule-3 Controls
Die ~880 BSI Rule-3 Controls werden **gegen** die neuen Rule 1+2 Controls abgeglichen:
- Wenn ein BSI-Control ein Duplikat eines OWASP/NIST-Controls ist → **OWASP/NIST bevorzugt**
(weil Originaltext + Zitation erlaubt)
- BSI-Duplikate werden als \`deprecated\` markiert
- Tags und Anchors werden in den behaltenen Control zusammengefuehrt
### Schritt 3: Ergebnis
Ziel: **~520-600 Master Controls**, davon:
- Viele mit \`source_original_text\` (Originaltext fuer Kunden sichtbar)
- Viele mit \`source_citation\` (Quellenangabe mit Lizenz)
- Klare Nachweismethode (\`verification_method\`)
- Thematische Kategorie (\`category\`)
### Verstaendliche Texte
Zusaetzlich zum Originaltext (der oft juristisch/technisch formuliert ist)
enthaelt jedes Control ein eigenstaendig formuliertes **Ziel** (objective)
und eine **Begruendung** (rationale) in verstaendlicher Sprache.`,
},
{
id: 'validation',