Benjamin Admin 29d3c1caf5 fix: deduplicate overlapping words after Paddle+Tesseract merge ...

PaddleOCR can return overlapping phrases (e.g. "von jm." and "jm. =")
that produce duplicate words after splitting. Added _deduplicate_words()
post-merge pass that removes words with same text at overlapping positions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-17 10:47:42 +01:00

..

data

feat(ocr-pipeline): British/American IPA pronunciation choice

2026-03-01 11:08:52 +01:00

mail

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

migrations

feat: add Excel-like grid editor for OCR overlay (Kombi mode step 6)

2026-03-14 23:41:03 +01:00

models

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

policies

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

routes

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

services

fix: increase PaddleOCR remote timeout to 120s for large scans

2026-03-12 13:41:39 +01:00

tests

feat: box-aware column detection — exclude box content from global columns

2026-03-16 18:42:46 +01:00

admin_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

config.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

country_metadata.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

cv_box_detect.py

feat: run shading-based box detection alongside line detection

2026-03-16 08:12:52 +01:00

cv_cell_grid.py

fix: word_boxes auch fuer breite Spalten (Full-Page OCR) speichern

2026-03-11 20:41:29 +01:00

cv_color_detect.py

fix: increase color recovery occupancy padding to prevent gap artifacts

2026-03-17 10:28:56 +01:00

cv_graphic_detect.py

feat: add border ghost filter + graphic detection tests + structure overlay

2026-03-16 18:28:53 +01:00

cv_layout.py

feat: box-aware column detection — exclude box content from global columns

2026-03-16 18:42:46 +01:00

cv_ocr_engines.py

feat: use local RapidOCR as default in ocr_region_paddle(), remote as fallback

2026-03-14 08:26:04 +01:00

cv_preprocessing.py

refactor: cv_vocab_pipeline.py in 6 Module aufteilen (8163 → 6 + Fassade)

2026-03-08 23:46:47 +01:00

cv_review.py

fix: _group_words_into_lines nach cv_ocr_engines.py verschieben

2026-03-09 15:24:56 +01:00

cv_vocab_pipeline.py

feat: Words-First Grid Builder (bottom-up alternative zu cell_grid_v2)

2026-03-12 06:46:05 +01:00

cv_vocab_types.py

feat: generische Box-Erkennung fuer zonenbasierte Spaltenerkennung

2026-03-09 15:06:23 +01:00

cv_words_first.py

fix: sort word_boxes in reading order (Y-grouped, then X-sorted)

2026-03-17 10:41:30 +01:00

dsfa_corpus_ingestion.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

dsfa_rag_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

eh_pipeline.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

eh_templates.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

embedding_client.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

full_compliance_pipeline.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

full_reingestion.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

github_crawler.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

grid_editor_api.py

fix: merge inline marker columns + improve ghost edge detection

2026-03-17 10:10:07 +01:00

handwriting_htr_api.py

feat(klausur): Handschrift entfernen + Klausur-HTR implementiert

2026-03-03 12:04:26 +01:00

hybrid_search.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

hybrid_vocab_extractor.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

hyde.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

legal_corpus_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

legal_corpus_ingestion.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

legal_corpus_robust.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

legal_templates_ingestion.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

main.py

feat: add Excel-like grid editor for OCR overlay (Kombi mode step 6)

2026-03-14 23:41:03 +01:00

metrics_db.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

migrate_rag_chunks.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

minio_storage.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

nibis_ingestion.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

nru_worksheet_generator.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

ocr_labeling_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

ocr_pipeline_api.py

fix: deduplicate overlapping words after Paddle+Tesseract merge

2026-03-17 10:47:42 +01:00

ocr_pipeline_session_store.py

feat: add Structure Detection step to OCR pipeline

2026-03-16 12:31:09 +01:00

orientation_crop_api.py

refactor: Crop nach Deskew/Dewarp verschieben + content-basierter Buchscan-Crop

2026-03-09 08:52:11 +01:00

page_crop.py

refactor: Crop nach Deskew/Dewarp verschieben + content-basierter Buchscan-Crop

2026-03-09 08:52:11 +01:00

pdf_export.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

pdf_extraction.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

pipeline_checkpoints.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

pyproject.toml

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

qdrant_service.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

rag_evaluation.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

rbac.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

requirements.txt

feat: OCR pipeline v2.1 – narrow column OCR, dewarp automation, Fabric.js editor

2026-03-03 22:44:14 +01:00

reranker.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

self_rag.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

storage.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

template_sources.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

tesseract_vocab_extractor.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

training_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

training_export_service.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

trocr_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

upload_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

vocab_session_store.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

vocab_worksheet_api.py

fix: Edge-Gaps in _split_broad_columns ignorieren + return-Tuple bei leerem Ergebnis

2026-03-07 22:16:29 +01:00

worksheet_cleanup_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

worksheet_editor_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

zeugnis_api.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

zeugnis_crawler.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

zeugnis_models.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00

zeugnis_seed_data.py

Initial commit: breakpilot-lehrer - Lehrer KI Platform

2026-02-11 23:47:26 +01:00