feat(agents): Semantic-Validator + Auto-Learning-Pattern-Library
CI / detect-changes (push) Successful in 5s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / build-sha-integrity (push) Failing after 4s
CI / validate-canonical-controls (push) Successful in 11s
CI / loc-budget (push) Successful in 14s
CI / go-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go (push) Has been skipped
CI / nodejs-build (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 29s
CI / test-python-document-crawler (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped

Sprint 1.10 — Semantic-Validator (User-Vorgabe 2026-06-09):
  - Statt unendlich Regex-Pattern fuer jede Schreibweise zu pflegen
    (Tel/Telefon/Telefonnr/Phone/Fon/Funkanschluss/…), nutzen wir
    bei MC-MISS einen LLM-Call: 'Ist die Pflichtangabe semantisch
    doch da, nur unter abweichendem Label?'
  - Bei LLM-Treffer: HIGH/MEDIUM-Finding wird zu LOW demoted,
    Empfehlung wird zu 'Best-Practice Umbenennung: Management ->
    Geschaeftsfuehrer' (mit STANDARD_LABELS-Mapping).
  - 1 LLM-Call pro Slot statt N: cost-effizient.

Sprint 1.11 — Auto-Learning-Pattern-Library:
  - Jedes Label das SVL findet wird in JSON persistiert:
    /tmp/breakpilot/agent_learned_patterns.json
  - Beim naechsten Run prueft der Agent zuerst gelernte Patterns
    BEVOR er das HIGH-Finding emittiert -> kein LLM-Call mehr.
  - Asymptotisch 0 LLM-Calls fuer haeufige Edge-Cases.
  - Halluzinations-Schutz: prune_low_confidence() loescht Patterns
    mit <0.5 Avg-Confidence nach 100 Beobachtungen.
  - Idempotent: gleicher (field_id, label, agent) -> Counter +1.

Tests: 40/40 gruen (10 Pattern-Library + 7 SVL + 13 GT + 11 v2).

STANDARD_LABELS-Map deckt Impressum + Cookie-Policy. Spaeter
erweiterbar fuer DSE, AGB, Widerrufs-Agenten.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-06-09 08:16:21 +02:00
parent 882e4f9798
commit ca8c388f37
5 changed files with 721 additions and 0 deletions
@@ -28,7 +28,12 @@ from .._base import (
lint_output,
)
from .._escalation import cascade
from .._pattern_library import load_patterns_for, record as record_pattern
from .._rollup import rollup
from .._semantic_validator import (
build_rename_action,
validate_present,
)
from .mcs import MC_IDS, MCS, detect_automotive, scope_matches
logger = logging.getLogger(__name__)
@@ -90,6 +95,18 @@ class ImpressumAgent(BaseSpecialistAgent):
))
continue
found = any(p.search(text) for p in mc.patterns)
if not found:
# 1.11: Auto-Learning — gelernte Labels probieren.
# Wenn ein gelerntes Pattern matcht: als OK werten +
# Coverage-Reason markiert das.
learned = load_patterns_for(mc.field_id, self.agent_id)
if any(lp.search(text) for lp in learned):
coverage.append(McCoverage(
mc_id=mc.mc_id, status="ok",
reason=f"learned-pattern matched "
f"({len(learned)} gelernt)",
))
continue
if found:
coverage.append(McCoverage(
mc_id=mc.mc_id, status="ok",
@@ -122,6 +139,11 @@ class ImpressumAgent(BaseSpecialistAgent):
reason="missing",
))
# Semantic-Validator: prüft per LLM ob HIGH-Missings doch
# vorhanden sind (unter abweichendem Label). Demoted HIGH→LOW
# mit Rename-Empfehlung wenn ja. User-Vorgabe 2026-06-09.
await self._semantic_demote(text, mc_findings, coverage)
# Eskalation: für die identifizierten Lücken kann ein LLM
# zusätzliche Tiefen-Findings liefern (z.B. "Geschäftsführer
# genannt, aber ohne Nachname"). Confidence der MC-Findings
@@ -147,6 +169,87 @@ class ImpressumAgent(BaseSpecialistAgent):
start, mc_findings, esc_logs, coverage, confidence=overall,
)
async def _semantic_demote(
self,
text: str,
findings: list[Finding],
coverage: list[McCoverage],
) -> None:
"""LLM-Layer für HIGH/MEDIUM-missings — demote zu LOW wenn da."""
candidates: list[tuple[str, str, Finding]] = []
for f in findings:
# Demote-Kandidaten: HIGH oder MEDIUM-Pattern-Misses.
# LOW/INFO bleiben unverändert (sind selbst schon Best-
# Practice-Empfehlungen).
if f.severity not in (Severity.HIGH.value,
Severity.MEDIUM.value):
continue
if f.severity_reason != "missing":
continue
# Suche zugehöriges MC für die Beschreibung
mc = next((m for m in MCS if m.field_id == f.field_id), None)
label = mc.label if mc else f.field_id
candidates.append((f.field_id, label, f))
if not candidates:
return
result = await validate_present(
text, [(c[0], c[1]) for c in candidates],
)
if not result:
return
for field_id, label, finding in candidates:
row = result.get(field_id)
if not row or not row.get("found"):
continue
if row.get("confidence", 0) < 0.6:
continue
label_used = row.get("label_used") or "abweichendes Label"
# Demote in-place
finding.severity = Severity.LOW.value
finding.severity_reason = "label_mismatch"
finding.title = (
f"Label '{label_used}' weicht von Standard-"
f"Bezeichnung ab"
)
finding.evidence = row.get("evidence", "")[:200]
finding.action = build_rename_action(field_id, label_used)
conf = float(row.get("confidence") or 0.8)
finding.confidence = conf
finding.sources.append(EvidenceSource(
source_type=SourceType.LLM_LOCAL,
source_id="semantic_validator",
detail=f"LLM-confirmed: '{label_used}'",
confidence=conf,
))
# 1.11: Auto-Learning — Label-Match in der Library
# persistieren. Beim nächsten Run wird das gelernte
# Pattern bereits beim MC-Pass berücksichtigt, ohne
# erneuten LLM-Call.
try:
record_pattern(
field_id=field_id,
label_used=label_used,
confidence=conf,
agent_id=self.agent_id,
)
except Exception as e:
import logging
logging.getLogger(__name__).warning(
"pattern-library record failed: %s", e,
)
# Update coverage status
for c in coverage:
if c.mc_id and c.mc_id.endswith(field_id.upper()):
continue
# Robuster: nach mc_id über MCS
mc = next((m for m in MCS if m.field_id == field_id), None)
if mc:
cov = next((c for c in coverage
if c.mc_id == mc.mc_id), None)
if cov:
cov.status = "low"
cov.reason = f"label_mismatch: '{label_used}'"
async def _maybe_escalate(
self, text: str, scope: set[str],
) -> tuple[list[Finding], list[EscalationLog]]: