feat(agents): Semantic-Validator + Auto-Learning-Pattern-Library
CI / detect-changes (push) Successful in 5s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / build-sha-integrity (push) Failing after 4s
CI / validate-canonical-controls (push) Successful in 11s
CI / loc-budget (push) Successful in 14s
CI / go-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go (push) Has been skipped
CI / nodejs-build (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 29s
CI / test-python-document-crawler (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
CI / detect-changes (push) Successful in 5s
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / secret-scan (push) Has been skipped
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / build-sha-integrity (push) Failing after 4s
CI / validate-canonical-controls (push) Successful in 11s
CI / loc-budget (push) Successful in 14s
CI / go-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go (push) Has been skipped
CI / nodejs-build (push) Has been skipped
CI / iace-gt-coverage (push) Has been skipped
CI / test-python-backend (push) Successful in 29s
CI / test-python-document-crawler (push) Has been skipped
CI / test-python-dsms-gateway (push) Has been skipped
Sprint 1.10 — Semantic-Validator (User-Vorgabe 2026-06-09):
- Statt unendlich Regex-Pattern fuer jede Schreibweise zu pflegen
(Tel/Telefon/Telefonnr/Phone/Fon/Funkanschluss/…), nutzen wir
bei MC-MISS einen LLM-Call: 'Ist die Pflichtangabe semantisch
doch da, nur unter abweichendem Label?'
- Bei LLM-Treffer: HIGH/MEDIUM-Finding wird zu LOW demoted,
Empfehlung wird zu 'Best-Practice Umbenennung: Management ->
Geschaeftsfuehrer' (mit STANDARD_LABELS-Mapping).
- 1 LLM-Call pro Slot statt N: cost-effizient.
Sprint 1.11 — Auto-Learning-Pattern-Library:
- Jedes Label das SVL findet wird in JSON persistiert:
/tmp/breakpilot/agent_learned_patterns.json
- Beim naechsten Run prueft der Agent zuerst gelernte Patterns
BEVOR er das HIGH-Finding emittiert -> kein LLM-Call mehr.
- Asymptotisch 0 LLM-Calls fuer haeufige Edge-Cases.
- Halluzinations-Schutz: prune_low_confidence() loescht Patterns
mit <0.5 Avg-Confidence nach 100 Beobachtungen.
- Idempotent: gleicher (field_id, label, agent) -> Counter +1.
Tests: 40/40 gruen (10 Pattern-Library + 7 SVL + 13 GT + 11 v2).
STANDARD_LABELS-Map deckt Impressum + Cookie-Policy. Spaeter
erweiterbar fuer DSE, AGB, Widerrufs-Agenten.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -28,7 +28,12 @@ from .._base import (
|
||||
lint_output,
|
||||
)
|
||||
from .._escalation import cascade
|
||||
from .._pattern_library import load_patterns_for, record as record_pattern
|
||||
from .._rollup import rollup
|
||||
from .._semantic_validator import (
|
||||
build_rename_action,
|
||||
validate_present,
|
||||
)
|
||||
from .mcs import MC_IDS, MCS, detect_automotive, scope_matches
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -90,6 +95,18 @@ class ImpressumAgent(BaseSpecialistAgent):
|
||||
))
|
||||
continue
|
||||
found = any(p.search(text) for p in mc.patterns)
|
||||
if not found:
|
||||
# 1.11: Auto-Learning — gelernte Labels probieren.
|
||||
# Wenn ein gelerntes Pattern matcht: als OK werten +
|
||||
# Coverage-Reason markiert das.
|
||||
learned = load_patterns_for(mc.field_id, self.agent_id)
|
||||
if any(lp.search(text) for lp in learned):
|
||||
coverage.append(McCoverage(
|
||||
mc_id=mc.mc_id, status="ok",
|
||||
reason=f"learned-pattern matched "
|
||||
f"({len(learned)} gelernt)",
|
||||
))
|
||||
continue
|
||||
if found:
|
||||
coverage.append(McCoverage(
|
||||
mc_id=mc.mc_id, status="ok",
|
||||
@@ -122,6 +139,11 @@ class ImpressumAgent(BaseSpecialistAgent):
|
||||
reason="missing",
|
||||
))
|
||||
|
||||
# Semantic-Validator: prüft per LLM ob HIGH-Missings doch
|
||||
# vorhanden sind (unter abweichendem Label). Demoted HIGH→LOW
|
||||
# mit Rename-Empfehlung wenn ja. User-Vorgabe 2026-06-09.
|
||||
await self._semantic_demote(text, mc_findings, coverage)
|
||||
|
||||
# Eskalation: für die identifizierten Lücken kann ein LLM
|
||||
# zusätzliche Tiefen-Findings liefern (z.B. "Geschäftsführer
|
||||
# genannt, aber ohne Nachname"). Confidence der MC-Findings
|
||||
@@ -147,6 +169,87 @@ class ImpressumAgent(BaseSpecialistAgent):
|
||||
start, mc_findings, esc_logs, coverage, confidence=overall,
|
||||
)
|
||||
|
||||
async def _semantic_demote(
|
||||
self,
|
||||
text: str,
|
||||
findings: list[Finding],
|
||||
coverage: list[McCoverage],
|
||||
) -> None:
|
||||
"""LLM-Layer für HIGH/MEDIUM-missings — demote zu LOW wenn da."""
|
||||
candidates: list[tuple[str, str, Finding]] = []
|
||||
for f in findings:
|
||||
# Demote-Kandidaten: HIGH oder MEDIUM-Pattern-Misses.
|
||||
# LOW/INFO bleiben unverändert (sind selbst schon Best-
|
||||
# Practice-Empfehlungen).
|
||||
if f.severity not in (Severity.HIGH.value,
|
||||
Severity.MEDIUM.value):
|
||||
continue
|
||||
if f.severity_reason != "missing":
|
||||
continue
|
||||
# Suche zugehöriges MC für die Beschreibung
|
||||
mc = next((m for m in MCS if m.field_id == f.field_id), None)
|
||||
label = mc.label if mc else f.field_id
|
||||
candidates.append((f.field_id, label, f))
|
||||
if not candidates:
|
||||
return
|
||||
result = await validate_present(
|
||||
text, [(c[0], c[1]) for c in candidates],
|
||||
)
|
||||
if not result:
|
||||
return
|
||||
for field_id, label, finding in candidates:
|
||||
row = result.get(field_id)
|
||||
if not row or not row.get("found"):
|
||||
continue
|
||||
if row.get("confidence", 0) < 0.6:
|
||||
continue
|
||||
label_used = row.get("label_used") or "abweichendes Label"
|
||||
# Demote in-place
|
||||
finding.severity = Severity.LOW.value
|
||||
finding.severity_reason = "label_mismatch"
|
||||
finding.title = (
|
||||
f"Label '{label_used}' weicht von Standard-"
|
||||
f"Bezeichnung ab"
|
||||
)
|
||||
finding.evidence = row.get("evidence", "")[:200]
|
||||
finding.action = build_rename_action(field_id, label_used)
|
||||
conf = float(row.get("confidence") or 0.8)
|
||||
finding.confidence = conf
|
||||
finding.sources.append(EvidenceSource(
|
||||
source_type=SourceType.LLM_LOCAL,
|
||||
source_id="semantic_validator",
|
||||
detail=f"LLM-confirmed: '{label_used}'",
|
||||
confidence=conf,
|
||||
))
|
||||
# 1.11: Auto-Learning — Label-Match in der Library
|
||||
# persistieren. Beim nächsten Run wird das gelernte
|
||||
# Pattern bereits beim MC-Pass berücksichtigt, ohne
|
||||
# erneuten LLM-Call.
|
||||
try:
|
||||
record_pattern(
|
||||
field_id=field_id,
|
||||
label_used=label_used,
|
||||
confidence=conf,
|
||||
agent_id=self.agent_id,
|
||||
)
|
||||
except Exception as e:
|
||||
import logging
|
||||
logging.getLogger(__name__).warning(
|
||||
"pattern-library record failed: %s", e,
|
||||
)
|
||||
# Update coverage status
|
||||
for c in coverage:
|
||||
if c.mc_id and c.mc_id.endswith(field_id.upper()):
|
||||
continue
|
||||
# Robuster: nach mc_id über MCS
|
||||
mc = next((m for m in MCS if m.field_id == field_id), None)
|
||||
if mc:
|
||||
cov = next((c for c in coverage
|
||||
if c.mc_id == mc.mc_id), None)
|
||||
if cov:
|
||||
cov.status = "low"
|
||||
cov.reason = f"label_mismatch: '{label_used}'"
|
||||
|
||||
async def _maybe_escalate(
|
||||
self, text: str, scope: set[str],
|
||||
) -> tuple[list[Finding], list[EscalationLog]]:
|
||||
|
||||
Reference in New Issue
Block a user