Benjamin Admin 118be3540d feat(pipeline): D6 citation backfill + E2/E3 law ingestion scripts ...

- d6_citation_backfill.py: 3-tier matching (hash/prefix/overlap),
  archives old citations, updated 3.651 controls (93.6% coverage)
- ingest_de_laws.py: 8 German laws ingested (ArbZG, MuSchG, NachwG,
  MiLoG, GmbHG, AktG, InsO, BUrlG — 1.629 chunks)
- ingest_eu_regulations.py: EUR-Lex ingestion (needs manual HTML due
  to AWS WAF). CSRD, CSDDD, EU Taxonomy, eIDAS 2.0, Pay Transparency
  manually ingested (1.057 chunks)
- Updated session handover with current state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-03 13:19:27 +02:00

..

d6_citation_backfill.py

feat(pipeline): D6 citation backfill + E2/E3 law ingestion scripts

2026-05-03 13:19:27 +02:00

extract_and_upload_nist.py

fix(embedding): add NIST control IDs to _SECTION_NUMBER_RE

2026-05-03 07:42:06 +02:00

import_backup.py

…

ingest_bag_urteile.py

…

ingest_de_laws.py

feat(pipeline): D6 citation backfill + E2/E3 law ingestion scripts

2026-05-03 13:19:27 +02:00

ingest_eu_regulations.py

feat(pipeline): D6 citation backfill + E2/E3 law ingestion scripts

2026-05-03 13:19:27 +02:00

migrate_jsonb.py

…

quality_report.py

docs: session handover — D2-D5 complete, quality report, NIST plan

2026-05-02 22:22:55 +02:00

reingest_d5_config.py

fix(rag): strip HTML tags before chunking + D5 re-ingestion scripts

2026-05-02 08:18:25 +02:00

reingest_d5.py

feat(embedding): NIST PDF text normalization + safe re-ingest script

2026-05-03 06:42:46 +02:00

reingest_nist.py

feat(embedding): NIST PDF text normalization + safe re-ingest script

2026-05-03 06:42:46 +02:00

replace_eu_pdfs_with_html.py

feat(embedding): add NIST/ENISA/standard section numbering to chunker

2026-05-02 19:24:10 +02:00

reupload_legal_strategy.py

fix(embedding): add NIST control IDs to _SECTION_NUMBER_RE

2026-05-03 07:42:06 +02:00

test_d4_integration.py

feat(pipeline): structural metadata end-to-end (Blocks D2-D4)

2026-05-01 20:34:00 +02:00

test_qwen_backfill.py

perf(pipeline): switch to v3 prompt for generation, v4 fields via Haiku backfill

2026-04-27 00:14:47 +02:00