feat(agents): Semantic-Validator + Auto-Learning-Pattern-Library
CI / detect-changes (push) Successful in 5s
CI / branch-name (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / build-sha-integrity (push) Failing after 4s
CI / validate-canonical-controls (push) Successful in 11s
CI / loc-budget (push) Successful in 14s
CI / go-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go (push) Has been skipped
CI / nodejs-build (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 29s
CI / test-python-document-crawler (push) Has been skipped

Sprint 1.10 — Semantic-Validator (User-Vorgabe 2026-06-09):
  - Statt unendlich Regex-Pattern fuer jede Schreibweise zu pflegen
    (Tel/Telefon/Telefonnr/Phone/Fon/Funkanschluss/…), nutzen wir
    bei MC-MISS einen LLM-Call: 'Ist die Pflichtangabe semantisch
    doch da, nur unter abweichendem Label?'
  - Bei LLM-Treffer: HIGH/MEDIUM-Finding wird zu LOW demoted,
    Empfehlung wird zu 'Best-Practice Umbenennung: Management ->
    Geschaeftsfuehrer' (mit STANDARD_LABELS-Mapping).
  - 1 LLM-Call pro Slot statt N: cost-effizient.

Sprint 1.11 — Auto-Learning-Pattern-Library:
  - Jedes Label das SVL findet wird in JSON persistiert:
    /tmp/breakpilot/agent_learned_patterns.json
  - Beim naechsten Run prueft der Agent zuerst gelernte Patterns
    BEVOR er das HIGH-Finding emittiert -> kein LLM-Call mehr.
  - Asymptotisch 0 LLM-Calls fuer haeufige Edge-Cases.
  - Halluzinations-Schutz: prune_low_confidence() loescht Patterns
    mit <0.5 Avg-Confidence nach 100 Beobachtungen.
  - Idempotent: gleicher (field_id, label, agent) -> Counter +1.

Tests: 40/40 gruen (10 Pattern-Library + 7 SVL + 13 GT + 11 v2).

STANDARD_LABELS-Map deckt Impressum + Cookie-Policy. Spaeter
erweiterbar fuer DSE, AGB, Widerrufs-Agenten.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-06-09 08:16:21 +02:00
parent 882e4f9798
commit ca8c388f37
5 changed files with 721 additions and 0 deletions
@@ -0,0 +1,119 @@
"""Tests für den Semantic-Validator-Layer."""
from __future__ import annotations
import asyncio
import pytest
from compliance.services.specialist_agents import AgentInput, ImpressumAgent
from compliance.services.specialist_agents._semantic_validator import (
STANDARD_LABELS,
build_rename_action,
standard_label,
validate_present,
)
def _run(coro):
return asyncio.get_event_loop().run_until_complete(coro)
def test_standard_labels_cover_impressum_fields():
"""Alle Impressum-Pflichtangaben müssen ein Standard-Label haben."""
for fid in (
"kontakt_telefon", "kontakt_email", "vertretungsberechtigte",
"handelsregister", "ust_id", "name_anbieter",
):
assert fid in STANDARD_LABELS, f"missing standard label: {fid}"
def test_build_rename_action_includes_old_and_new():
a = build_rename_action("kontakt_telefon", "Telefonnr.")
assert "Telefonnr." in a
assert "Telefon" in a
assert "Best-Practice" in a or "Umbenennung" in a
def test_standard_label_falls_back_to_field_id():
assert standard_label("kontakt_telefon") == "Telefon"
assert standard_label("ghost_field") == "ghost_field"
def test_validate_present_short_text_returns_empty():
out = _run(validate_present(
"x", [("kontakt_telefon", "Telefon")],
))
assert out == {}
def test_validate_present_no_fields_returns_empty():
out = _run(validate_present("Long impressum text" * 100, []))
assert out == {}
def test_semantic_demotion_high_to_low(monkeypatch):
"""Wenn LLM bestätigt dass Pflichtangabe da ist: HIGH→LOW.
Test-Setup: Impressum-Text OHNE jegliche Telefon-Markierung
(Pattern matched nicht). LLM-Mock behauptet aber 'Funkanschluss'
wäre ein abweichendes Label für die Telefonnummer.
"""
from compliance.services.specialist_agents._escalation import (
EscalationResult, SourceType,
)
from compliance.services.specialist_agents._base import EscalationLog
async def _fake_cascade(sys_prompt, user_prompt,
expect_json=True, skip_ovh=False):
# Nur auf den SVL-Prompt reagieren
if "FEHLENDE PFLICHTANGABEN" not in user_prompt:
return None, []
log = EscalationLog(
stage=SourceType.LLM_LOCAL, model="qwen2.5:7b",
duration_ms=42, success=True,
)
res = EscalationResult(
content='{"results":[]}',
stage=SourceType.LLM_LOCAL,
model="qwen2.5:7b",
log=log,
parsed={"results": [{
"field_id": "kontakt_telefon",
"found": True,
"label_used": "Funkanschluss",
"evidence": "Funkanschluss 0761/123456",
"confidence": 0.9,
}]},
)
return res, [log]
monkeypatch.setattr(
"compliance.services.specialist_agents._semantic_validator.cascade",
_fake_cascade,
)
monkeypatch.setattr(
"compliance.services.specialist_agents.impressum.agent.cascade",
_fake_cascade,
)
# Text OHNE Telefon-Label → MC matched nicht → HIGH-Finding
text = (
"Beispiel GmbH\nMusterstr. 1\n12345 Berlin\n"
"E-Mail: x@y.de\nFunkanschluss 0761/123456\n"
"Geschäftsführer: Max Mustermann\n"
"Handelsregister Berlin HRB 12345\n"
"USt-IdNr: DE123456789"
)
agent = ImpressumAgent()
out = _run(agent.evaluate(AgentInput(doc_type="impressum", text=text)))
telefon_findings = [f for f in out.findings
if f.field_id == "kontakt_telefon"]
assert telefon_findings, "expected MC-miss → finding"
f = telefon_findings[0]
# Erwartet: SVL hat demoted zu LOW
assert f.severity == "LOW", (
f"Erwartet: LOW nach semantic-demote, got: {f.severity}. "
f"Finding: {f}"
)
assert f.severity_reason == "label_mismatch"
assert "Funkanschluss" in f.action
assert "Telefon" in f.action