Benjamin Admin 5fa5767c9a Fix box column detection: use low gap_threshold for small zones ...

PaddleOCR returns multi-word blocks (whole phrases), so ALL inter-word
gaps in small zones (boxes, ≤60 words) are column boundaries. Previous
3x-median approach produced thresholds too high to detect real columns.

New approach for small zones: gap_threshold = max(median_h * 1.0, 25).
This correctly detects 4 columns in "Pounds and euros" box where gaps
range from 50-297px and word height is ~31px.

Also includes SmartSpellChecker fixes from previous commits:
- Frequency-based scoring, IPA protection, slash→l, rare-word threshold

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-13 07:55:29 +02:00

..

data

Add German IPA support (wiki-pronunciation-dict + epitran)

2026-03-25 22:18:20 +01:00

mail

…

migrations

Add OCR Kombi Pipeline: modular 11-step architecture with multi-page support

2026-03-26 15:55:28 +01:00

models

…

ocr_kombi

Add OCR Kombi Pipeline: modular 11-step architecture with multi-page support

2026-03-26 15:55:28 +01:00

policies

…

routes

…

services

feat: Sprint 2 — TrOCR ONNX, PP-DocLayout, Model Management

2026-03-23 09:53:02 +01:00

tests

SmartSpellChecker: frequency-based boundary repair for valid word pairs

2026-04-13 07:00:22 +02:00

admin_api.py

…

config.py

…

country_metadata.py

…

cv_box_detect.py

feat: run shading-based box detection alongside line detection

2026-03-16 08:12:52 +01:00

cv_box_layout.py

Add Box-Grid-Review step (Step 11) to OCR pipeline

2026-04-12 17:26:06 +02:00

cv_cell_grid.py

fix: word_boxes auch fuer breite Spalten (Full-Page OCR) speichern

2026-03-11 20:41:29 +01:00

cv_color_detect.py

Fix bullet overlap disambiguation + raise red threshold to 90

2026-03-20 18:21:00 +01:00

cv_doclayout_detect.py

feat: Sprint 2 — TrOCR ONNX, PP-DocLayout, Model Management

2026-03-23 09:53:02 +01:00

cv_graphic_detect.py

feat: Sprint 2 — TrOCR ONNX, PP-DocLayout, Model Management

2026-03-23 09:53:02 +01:00

cv_gutter_repair.py

Skip valid line-break hyphenations in gutter repair

2026-04-11 00:14:21 +02:00

cv_ipa_german.py

Add German IPA support (wiki-pronunciation-dict + epitran)

2026-03-25 22:18:20 +01:00

cv_layout.py

feat: ImageLayoutEditor, arrow-key nav, multi-select bold, wider columns

2026-03-24 07:45:39 +01:00

cv_ocr_engines.py

Fix IPA stripping digits after headwords (Theme 1 → Theme)

2026-04-11 22:13:45 +02:00

cv_preprocessing.py

fix: swap 90°/270° rotation direction in orientation detection

2026-03-17 16:39:15 +01:00

cv_review.py

Add SmartSpellChecker + refactor vocab-worksheet page.tsx

2026-04-12 12:25:01 +02:00

cv_syllable_detect.py

Use spellchecker instead of pyphen for pipe autocorrect validation

2026-03-27 16:47:42 +01:00

cv_vocab_pipeline.py

feat: Words-First Grid Builder (bottom-up alternative zu cell_grid_v2)

2026-03-12 06:46:05 +01:00

cv_vocab_types.py

Add German IPA support (wiki-pronunciation-dict + epitran)

2026-03-25 22:18:20 +01:00

cv_words_first.py

fix(ocr-pipeline): improve page crop spine detection and cell assignment

2026-03-24 09:23:30 +01:00

dsfa_corpus_ingestion.py

…

dsfa_rag_api.py

…

eh_pipeline.py

…

eh_templates.py

…

embedding_client.py

…

full_compliance_pipeline.py

…

full_reingestion.py

…

github_crawler.py

…

grid_editor_api.py

Fix build-box-grids: use structure_result boxes + raw OCR words

2026-04-12 21:50:28 +02:00

grid_editor_helpers.py

Fix box column detection: use low gap_threshold for small zones

2026-04-13 07:55:29 +02:00

handwriting_htr_api.py

…

hybrid_search.py

…

hybrid_vocab_extractor.py

…

hyde.py

…

legal_corpus_api.py

…

legal_corpus_ingestion.py

…

legal_corpus_robust.py

…

legal_templates_ingestion.py

…

main.py

Add OCR Kombi Pipeline: modular 11-step architecture with multi-page support

2026-03-26 15:55:28 +01:00

metrics_db.py

…

migrate_rag_chunks.py

…

minio_storage.py

…

nibis_ingestion.py

…

nru_worksheet_generator.py

…

ocr_labeling_api.py

…

ocr_pipeline_api.py

Add Ground Truth regression test system for OCR pipeline

2026-03-18 13:46:48 +01:00

ocr_pipeline_auto.py

refactor: split ocr_pipeline_api.py (5426 lines) into 8 modules

2026-03-18 08:42:00 +01:00

ocr_pipeline_common.py

feat: add Woerterbuch category + column add/delete in grid editor

2026-03-23 16:27:12 +01:00

ocr_pipeline_geometry.py

Invalidate grid_editor_result when exclude regions change

2026-03-19 09:19:09 +01:00

ocr_pipeline_ocr_merge.py

refactor: split ocr_pipeline_api.py (5426 lines) into 8 modules

2026-03-18 08:42:00 +01:00

ocr_pipeline_overlays.py

Preserve alphabetic marker columns, broaden junk filter, enable IPA in grid

2026-03-18 11:08:23 +01:00

ocr_pipeline_postprocess.py

refactor: split ocr_pipeline_api.py (5426 lines) into 8 modules

2026-03-18 08:42:00 +01:00

ocr_pipeline_regression.py

feat: save automatic grid snapshot before manual edits for GT comparison

2026-03-24 13:16:44 +01:00

ocr_pipeline_rows.py

refactor: split ocr_pipeline_api.py (5426 lines) into 8 modules

2026-03-18 08:42:00 +01:00

ocr_pipeline_session_store.py

Show full Grid-Review in Ground Truth step + GT badge in session list

2026-04-10 19:34:32 +02:00

ocr_pipeline_sessions.py

Fix multi-page PDF upload: include session_id for first page

2026-04-11 16:26:25 +02:00

ocr_pipeline_words.py

refactor: split ocr_pipeline_api.py (5426 lines) into 8 modules

2026-03-18 08:42:00 +01:00

orientation_crop_api.py

refactor: independent sessions for page-split + URL-based pipeline navigation

2026-03-24 17:05:33 +01:00

page_crop.py

Fix left-side gutter detection: find peak instead of scanning from edge

2026-04-11 16:52:23 +02:00

pdf_export.py

…

pdf_extraction.py

…

pipeline_checkpoints.py

…

pyproject.toml

…

qdrant_service.py

…

rag_evaluation.py

…

rbac.py

…

requirements.txt

Add German IPA support (wiki-pronunciation-dict + epitran)

2026-03-25 22:18:20 +01:00

reranker.py

…

self_rag.py

…

smart_spell.py

SmartSpellChecker: frequency scoring, IPA protection, slash→l fix

2026-04-13 07:36:39 +02:00

storage.py

…

template_sources.py

…

tesseract_vocab_extractor.py

…

training_api.py

…

training_export_service.py

…

trocr_api.py

…

upload_api.py

…

vocab_session_store.py

…

vocab_worksheet_api.py

Use grid-build zones for vocab extraction (4-column detection)

2026-04-11 01:17:40 +02:00

worksheet_cleanup_api.py

…

worksheet_editor_api.py

…

zeugnis_api.py

…

zeugnis_crawler.py

…

zeugnis_models.py

…

zeugnis_seed_data.py

…