Fix page refs deleted as artifacts + IPA spacing for DE mode
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-school (push) Successful in 54s
CI / test-go-edu-search (push) Successful in 41s
CI / test-nodejs-website (push) Has been cancelled
CI / test-python-agent-core (push) Has been cancelled
CI / test-python-klausur (push) Has started running

1. Step 5j-pre wrongly classified "p.43", "p.50" etc as artifacts
   (mixed digits+letters, <=5 chars). Added exception for page
   reference patterns (p.XX, S.XX).

2. IPA spacing regex was too narrow (only matched Unicode IPA chars).
   Now matches any [bracket] content >=2 chars directly after a letter,
   fixing German IPA like "Opa[oːpa]" → "Opa [oːpa]".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
2026-04-11 22:01:25 +02:00
parent 7f4b8757ff
commit 5eff4cf877

View File

@@ -1507,8 +1507,10 @@ async def _build_grid_core(
is_artifact = True
elif len(core) <= 3 and core.isupper() and core.lower() not in _COMMON_SHORT_WORDS:
is_artifact = True
elif len(core) <= 5 and re.search(r'\d', core) and re.search(r'[A-Za-z]', core):
elif (len(core) <= 5 and re.search(r'\d', core) and re.search(r'[A-Za-z]', core)
and not re.match(r'^[pPsS]\.?\d+$', core)):
# Mixed digits + letters in short text (e.g. "7 EN", "a=3")
# but NOT page references like "p.43", "p50", "S.12"
is_artifact = True
if is_artifact:
kept.append(None) # placeholder
@@ -1717,8 +1719,10 @@ async def _build_grid_core(
except ImportError:
pass
# --- Ensure space before IPA brackets: "word[ipa]" → "word [ipa]" ---
_IPA_NOSPACE_RE = re.compile(r'([a-zA-ZäöüÄÖÜß])(\[[^\]]*[ˈˌːɑɒæɛəɜɪɔʊʌðŋθʃʒɹɡɾ][^\]]*\])')
# --- Ensure space before IPA/phonetic brackets: "word[ipa]" → "word [ipa]" ---
# Matches any [bracket] directly after a letter, as long as the bracket
# content doesn't look like a normal text annotation (e.g. "[adj]", "[noun]").
_IPA_NOSPACE_RE = re.compile(r'([a-zA-ZäöüÄÖÜß])(\[[^\]]{2,}\])')
for z in zones_data:
for cell in z.get("cells", []):
text = cell.get("text", "")