Benjamin Admin a459636bc4 fix(rag): HTML charset detection + opening block tag newlines ...

Two bugs fixed:
1. Opening block tags (<h3>, <div>) now also create newlines, not just
   closing tags. Fixes: gesetze-im-internet.de puts § inside <h3> which
   followed inline <a> text — § ended up mid-line, not at line start.

2. HTML charset detection from meta tag (charset=iso-8859-1). Files from
   gesetze-im-internet.de use ISO-8859-1, not UTF-8. The § byte (0xA7)
   was destroyed by UTF-8 decode. Now: try UTF-8 → check meta charset →
   fallback ISO-8859-1.

32 rag-service tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-02 08:35:47 +02:00

..

__init__.py

feat(pipeline): structural metadata end-to-end (Blocks D2-D4)

2026-05-01 20:34:00 +02:00

conftest.py

feat(pipeline): structural metadata end-to-end (Blocks D2-D4)

2026-05-01 20:34:00 +02:00

test_document_upload.py

feat(pipeline): structural metadata end-to-end (Blocks D2-D4)

2026-05-01 20:34:00 +02:00

test_embedding_client.py

feat(pipeline): structural metadata end-to-end (Blocks D2-D4)

2026-05-01 20:34:00 +02:00

test_html_stripping.py

fix(rag): HTML charset detection + opening block tag newlines

2026-05-02 08:35:47 +02:00