fix(b9)+test: real-world false-positives + multi-site GT-bench
Real-World-Smoke gegen Westfield Hamburg (englische DSE) deckte
B9-Bug auf: Pattern matched "If mfi Immobilien Marketing GmbH",
"Discover our Se", "Centre Se" usw. als angebliche Entitäten —
englische Connector-Worte + abgeschnittene "Services"-Strings.
B9 Fix:
- _name_is_blocked() strenger: min 2 Worte, mind. einer ≥4 Chars
UND capitalized (vor Legal-Form-Suffix). Filtert "Se", "ag",
"If ...", "Centre Se" zuverlässig.
- _clean_entity_name() strippt jetzt führende Lowercase-
Connector-Worte (kontextuelle Verben wie "by", "If",
"according to").
- _dedup_substring() collapses
"mfi Immobilien Marketing GmbH" + "Marketing GmbH" zum längeren.
- Anwendung sowohl im HRB-Pfad als auch im Fallback-Pfad.
Multi-Site-Bench (2 neue GTs, 2 Engine-Runs):
- zeroclaw/docs/ground-truth/westfield_hamburg_2026-06-07.json:
iAdvize-Chatbot bekannt, Unibail-Management-Verantwortlicher.
- zeroclaw/docs/ground-truth/allianz_reise_chatbot_2026-06-07.json:
Twilio-Infrastruktur (US-Transfer), lit. f + 2-Mo-Retention.
- zeroclaw/docs/audits/2026-06-07-multi-site-walk-results.md:
Sprint-Briefing mit Detektor × Site Matrix, Audit-Walk-DSMS-
CIDs, identifizierte Real-World-Bugs + Backlog.
Audit-Walk-Endstand (B17 Stufen 1-3):
- Westfield: 400 KB Video, CID Qm…WJYfYDt…BXgwt
- Allianz: 1 MB Video, CID Qm…XFuiC4z…9mSMM
Beide DSMS-persistiert, Reviewer kann jederzeit verifizieren.
Tests: 21/21 grün (test_impressum/test_elli_gt_coverage).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -96,12 +96,58 @@ def _clean_entity_name(raw: str) -> str:
|
||||
if new == name:
|
||||
break
|
||||
name = new
|
||||
# Strip leading lowercase prose words (English connectors leaking
|
||||
# into the match: "If ...", "by ...", "according to ...").
|
||||
# Drop tokens until we hit the first Capitalized one.
|
||||
tokens = name.split()
|
||||
while tokens and not tokens[0][:1].isupper():
|
||||
tokens = tokens[1:]
|
||||
name = " ".join(tokens)
|
||||
return re.sub(r"\s+", " ", name).strip()
|
||||
|
||||
|
||||
def _dedup_substring(slices: list[tuple[str, str]]) -> list[tuple[str, str]]:
|
||||
"""Collapse entities whose names are substrings of each other.
|
||||
|
||||
'mfi Immobilien Marketing GmbH' and 'Marketing GmbH' both refer to
|
||||
the same legal person — keep only the longest unique name.
|
||||
"""
|
||||
sorted_by_len = sorted(slices, key=lambda x: -len(x[0]))
|
||||
kept: list[tuple[str, str]] = []
|
||||
kept_names_lc: list[str] = []
|
||||
for name, slc in sorted_by_len:
|
||||
nl = name.lower()
|
||||
if any(nl in k or k in nl for k in kept_names_lc):
|
||||
continue
|
||||
kept.append((name, slc))
|
||||
kept_names_lc.append(nl)
|
||||
return kept
|
||||
|
||||
|
||||
def _name_is_blocked(name: str) -> bool:
|
||||
nl = name.lower()
|
||||
return any(b in nl for b in _NAME_BLOCKLIST)
|
||||
if any(b in nl for b in _NAME_BLOCKLIST):
|
||||
return True
|
||||
# Minimum-name-quality: must have ≥ 2 words, at least one ≥ 4 chars
|
||||
# before the legal-form suffix. Filters out "Se", "As a se" frags.
|
||||
parts = name.strip().split()
|
||||
if len(parts) < 2:
|
||||
return True
|
||||
# Strip legal-form from the end if present
|
||||
legal_suffixes = {
|
||||
"gmbh", "ag", "ug", "kg", "se", "e.v.", "gbr", "ohg",
|
||||
"limited", "ltd", "llc",
|
||||
}
|
||||
if parts[-1].lower() in legal_suffixes:
|
||||
non_suffix = parts[:-1]
|
||||
else:
|
||||
non_suffix = parts
|
||||
if not non_suffix or len(non_suffix) < 1:
|
||||
return True
|
||||
# At least one company-name token ≥ 4 chars and capitalized
|
||||
if not any(p[0].isupper() and len(p) >= 4 for p in non_suffix):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _slice_entities(text: str) -> list[tuple[str, str]]:
|
||||
@@ -142,6 +188,7 @@ def _slice_entities(text: str) -> list[tuple[str, str]]:
|
||||
slice_end = (hrb_matches[i + 1].start()
|
||||
if i + 1 < len(hrb_matches) else len(text))
|
||||
slices.append((name, text[slice_start:slice_end]))
|
||||
slices = _dedup_substring(slices)
|
||||
if len(slices) >= 2:
|
||||
return slices
|
||||
|
||||
@@ -160,6 +207,7 @@ def _slice_entities(text: str) -> list[tuple[str, str]]:
|
||||
start = m.start()
|
||||
end = matches[i + 1].start() if i + 1 < len(matches) else len(text)
|
||||
slices.append((name, text[start:end]))
|
||||
slices = _dedup_substring(slices)
|
||||
return slices if len(slices) >= 2 else []
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user