feat: Sprint 1 — IPA hardening, regression framework, ground-truth review
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-school (push) Successful in 28s
CI / test-go-edu-search (push) Successful in 27s
CI / test-python-klausur (push) Failing after 1m55s
CI / test-python-agent-core (push) Successful in 16s
CI / test-nodejs-website (push) Successful in 19s
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-school (push) Successful in 28s
CI / test-go-edu-search (push) Successful in 27s
CI / test-python-klausur (push) Failing after 1m55s
CI / test-python-agent-core (push) Successful in 16s
CI / test-nodejs-website (push) Successful in 19s
Track A (Backend): - Compound word IPA decomposition (schoolbag→school+bag) - Trailing garbled IPA fragment removal after brackets (R21 fix) - Regression runner with DB persistence, history endpoints - Page crop determinism verified with tests Track B (Frontend): - OCR Regression dashboard (/ai/ocr-regression) - Ground Truth Review workflow (/ai/ocr-ground-truth) with split-view, confidence highlighting, inline edit, batch mark, progress tracking Track C (Docs): - OCR-Pipeline.md v5.0 (Steps 5e-5h) - Regression testing guide - mkdocs.yml nav update Track D (Infra): - TrOCR baseline benchmark script - run-regression.sh shell script - Migration 008: regression_runs table Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
166
docs-src/development/regression-testing.md
Normal file
166
docs-src/development/regression-testing.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# OCR Pipeline Regression Testing
|
||||
|
||||
**Stand:** 2026-03-23
|
||||
|
||||
---
|
||||
|
||||
## Uebersicht
|
||||
|
||||
Das Regression Framework stellt sicher, dass Aenderungen an der OCR-Pipeline keine bestehenden
|
||||
Ergebnisse verschlechtern. Ground-Truth Sessions dienen als Referenz — nach jeder Code-Aenderung
|
||||
wird die Pipeline neu ausgefuehrt und das Ergebnis mit der Referenz verglichen.
|
||||
|
||||
---
|
||||
|
||||
## Ground Truth markieren
|
||||
|
||||
### Via Admin-UI (empfohlen)
|
||||
|
||||
1. Oeffne die OCR Pipeline: [/ai/ocr-pipeline](https://macmini:3002/ai/ocr-pipeline)
|
||||
2. Lade eine Session und fuehre alle Pipeline-Schritte aus
|
||||
3. Pruefe das Ergebnis im Grid Editor (Schritt 10)
|
||||
4. Korrigiere Fehler falls noetig (Inline-Edit)
|
||||
5. Klicke **"Als Ground Truth markieren"**
|
||||
|
||||
### Via API
|
||||
|
||||
```bash
|
||||
# Bestehende Session als Ground Truth markieren
|
||||
curl -X POST "http://macmini:8086/api/v1/ocr-pipeline/sessions/{session_id}/mark-ground-truth"
|
||||
|
||||
# Ground Truth entfernen
|
||||
curl -X DELETE "http://macmini:8086/api/v1/ocr-pipeline/sessions/{session_id}/mark-ground-truth"
|
||||
|
||||
# Alle Ground-Truth Sessions auflisten
|
||||
curl "http://macmini:8086/api/v1/ocr-pipeline/ground-truth-sessions"
|
||||
```
|
||||
|
||||
### Via Ground-Truth Review UI
|
||||
|
||||
Fuer die Massenpruefung von 50-100 Sessions:
|
||||
|
||||
1. Oeffne [/ai/ocr-ground-truth](https://macmini:3002/ai/ocr-ground-truth)
|
||||
2. Filter auf "Offen" (ungeprueft)
|
||||
3. Split-View: Bild links, Grid rechts pruefen
|
||||
4. Korrekte Zeilen mit Haekchen bestaetigen
|
||||
5. Fehler inline korrigieren
|
||||
6. "Markieren & Weiter" fuer naechste Session
|
||||
|
||||
---
|
||||
|
||||
## Regression ausfuehren
|
||||
|
||||
### Via Shell-Script (CI/CD)
|
||||
|
||||
```bash
|
||||
# Standard: macmini:8086
|
||||
./scripts/run-regression.sh
|
||||
|
||||
# Custom URL
|
||||
./scripts/run-regression.sh http://localhost:8086
|
||||
|
||||
# Exit-Codes:
|
||||
# 0 = alle bestanden
|
||||
# 1 = Fehler gefunden
|
||||
# 2 = Verbindungsfehler
|
||||
```
|
||||
|
||||
### Via Admin-UI
|
||||
|
||||
1. Oeffne [/ai/ocr-regression](https://macmini:3002/ai/ocr-regression)
|
||||
2. Klicke **"Alle Tests starten"**
|
||||
3. Ergebnis: Pass/Fail pro Session mit Diff-Details
|
||||
|
||||
### Via API
|
||||
|
||||
```bash
|
||||
# Alle Ground-Truth Sessions testen
|
||||
curl -X POST "http://macmini:8086/api/v1/ocr-pipeline/regression/run?triggered_by=script"
|
||||
|
||||
# Einzelne Session testen
|
||||
curl -X POST "http://macmini:8086/api/v1/ocr-pipeline/sessions/{session_id}/regression/run"
|
||||
|
||||
# Verlauf abrufen
|
||||
curl "http://macmini:8086/api/v1/ocr-pipeline/regression/history?limit=20"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ergebnisse lesen
|
||||
|
||||
### Diff-Typen
|
||||
|
||||
| Typ | Beschreibung |
|
||||
|-----|-------------|
|
||||
| `structural_changes` | Anzahl Zonen, Spalten oder Zeilen hat sich geaendert |
|
||||
| `text_change` | Text einer Zelle hat sich geaendert |
|
||||
| `cell_missing` | Zelle war in der Referenz, fehlt jetzt |
|
||||
| `cell_added` | Neue Zelle die in der Referenz nicht existierte |
|
||||
| `col_type_change` | Spaltentyp einer Zelle hat sich geaendert |
|
||||
|
||||
### Status-Bewertung
|
||||
|
||||
- **pass**: Keine Diffs → Code-Aenderung hat keine Auswirkung
|
||||
- **fail**: Diffs gefunden → pruefen ob gewollt (Feature) oder ungewollt (Regression)
|
||||
- **error**: Pipeline-Fehler → Build oder Config-Problem
|
||||
|
||||
### Verlauf
|
||||
|
||||
Alle Laeufe werden in der Tabelle `regression_runs` persistiert:
|
||||
|
||||
```sql
|
||||
SELECT id, run_at, status, total, passed, failed, errors, duration_ms, triggered_by
|
||||
FROM regression_runs
|
||||
ORDER BY run_at DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Ground-Truth Sessions waehlen
|
||||
|
||||
Decke verschiedene Seitentypen ab:
|
||||
|
||||
- Woerterbuchseiten (2-3 Spalten, IPA-Klammern)
|
||||
- Uebungsseiten (Tabellen, Checkboxen)
|
||||
- Seiten mit Illustrationen
|
||||
- Seiten ohne IPA (reines Deutsch-Vokabular)
|
||||
- Verschiedene Verlage und Layouts
|
||||
|
||||
### Workflow vor jedem Commit
|
||||
|
||||
```bash
|
||||
# 1. Regression laufen lassen
|
||||
./scripts/run-regression.sh
|
||||
|
||||
# 2. Bei Failure: Diff pruefen
|
||||
# - Gewollte Aenderung? → Ground Truth aktualisieren
|
||||
# - Ungewollte Regression? → Code fixen
|
||||
|
||||
# 3. Bei Pass: Commit
|
||||
git add . && git commit -m "fix: ..."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Datenbank-Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE regression_runs (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
run_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
status VARCHAR(20) NOT NULL, -- pass, fail, error
|
||||
total INT NOT NULL DEFAULT 0,
|
||||
passed INT NOT NULL DEFAULT 0,
|
||||
failed INT NOT NULL DEFAULT 0,
|
||||
errors INT NOT NULL DEFAULT 0,
|
||||
duration_ms INT,
|
||||
results JSONB NOT NULL DEFAULT '[]', -- Detail-Ergebnisse pro Session
|
||||
triggered_by VARCHAR(50) DEFAULT 'manual'
|
||||
);
|
||||
```
|
||||
|
||||
Ground-Truth Referenzen werden im `ground_truth` JSONB-Feld der
|
||||
`ocr_pipeline_sessions` Tabelle gespeichert.
|
||||
Reference in New Issue
Block a user