Add CLAUDE.md, MkDocs docs, .claude/rules
- CLAUDE.md: Comprehensive documentation for Lehrer KI platform - docs-src: Klausur, Voice, Agent-Core, KI-Pipeline docs - mkdocs.yml: Lehrer-specific nav with blue theme - docker-compose: Added docs service (port 8010, profile: docs) - .claude/rules: testing, docs, open-source, abiturkorrektur, vocab-worksheet, multi-agent, experimental-dashboard Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Benjamin Boenisch
445
docs-src/services/klausur-service/OCR-Labeling-Spec.md
Normal file
445
docs-src/services/klausur-service/OCR-Labeling-Spec.md
Normal file
@@ -0,0 +1,445 @@
|
||||
# OCR-Labeling System Spezifikation
|
||||
|
||||
**Version:** 1.1.0
|
||||
**Status:** In Produktion (Mac Mini)
|
||||
|
||||
## Übersicht
|
||||
|
||||
Das OCR-Labeling System ermöglicht das Erstellen von Trainingsdaten für Handschrift-OCR-Modelle aus eingescannten Klausuren. Es unterstützt folgende OCR-Modelle:
|
||||
|
||||
| Modell | Beschreibung | Geschwindigkeit | Empfohlen für |
|
||||
|--------|--------------|-----------------|---------------|
|
||||
| **llama3.2-vision:11b** | Vision-LLM (Standard) | Langsam | Handschrift, beste Qualität |
|
||||
| **TrOCR** | Microsoft Transformer | Schnell | Gedruckter Text |
|
||||
| **PaddleOCR + LLM** | Hybrid-Ansatz (NEU) | Sehr schnell (4x) | Gemischte Dokumente |
|
||||
| **Donut** | Document Understanding (NEU) | Mittel | Tabellen, Formulare |
|
||||
| **qwen2.5:14b** | Korrektur-LLM | - | Klausurbewertung |
|
||||
|
||||
### Neue OCR-Optionen (v1.1.0)
|
||||
|
||||
#### PaddleOCR + LLM (Empfohlen für Geschwindigkeit)
|
||||
|
||||
PaddleOCR ist ein zweistufiger Ansatz:
|
||||
1. **PaddleOCR** - Schnelle, präzise Texterkennung mit Bounding-Boxes
|
||||
2. **qwen2.5:14b** - Semantische Strukturierung des erkannten Texts
|
||||
|
||||
**Vorteile:**
|
||||
- 4x schneller als Vision-LLM (~7-15 Sek vs 30-60 Sek pro Seite)
|
||||
- Höhere Genauigkeit bei gedrucktem Text (95-99%)
|
||||
- Weniger Halluzinationen (LLM korrigiert nur, erfindet nicht)
|
||||
- Position-basierte Spaltenerkennung möglich
|
||||
|
||||
**Dateien:**
|
||||
- `/klausur-service/backend/hybrid_vocab_extractor.py` - PaddleOCR Integration
|
||||
|
||||
#### Donut (Document Understanding Transformer)
|
||||
|
||||
Donut ist speziell für strukturierte Dokumente optimiert:
|
||||
- Tabellen und Formulare
|
||||
- Rechnungen und Quittungen
|
||||
- Multi-Spalten-Layouts
|
||||
|
||||
**Dateien:**
|
||||
- `/klausur-service/backend/services/donut_ocr_service.py` - Donut Service
|
||||
|
||||
## Architektur
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ OCR-Labeling System │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────────┐ ┌────────────────────────┐ │
|
||||
│ │ Frontend │◄──►│ Klausur-Service │◄──►│ PostgreSQL │ │
|
||||
│ │ (Next.js) │ │ (FastAPI) │ │ - ocr_labeling_sessions│ │
|
||||
│ │ Port 3000 │ │ Port 8086 │ │ - ocr_labeling_items │ │
|
||||
│ └─────────────┘ └────────┬─────────┘ │ - ocr_training_samples │ │
|
||||
│ │ └────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────┼──────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌───────────┐ ┌─────────┐ ┌───────────────┐ │
|
||||
│ │ MinIO │ │ Ollama │ │ Export Service │ │
|
||||
│ │ (Images) │ │ (OCR) │ │ (Training) │ │
|
||||
│ │ Port 9000 │ │ :11434 │ │ │ │
|
||||
│ └───────────┘ └─────────┘ └───────────────┘ │
|
||||
│ │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Datenmodell
|
||||
|
||||
### PostgreSQL Tabellen
|
||||
|
||||
```sql
|
||||
-- Labeling Sessions (gruppiert zusammengehörige Bilder)
|
||||
CREATE TABLE ocr_labeling_sessions (
|
||||
id VARCHAR(36) PRIMARY KEY,
|
||||
name VARCHAR(255) NOT NULL,
|
||||
source_type VARCHAR(50) NOT NULL, -- 'klausur', 'handwriting_sample', 'scan'
|
||||
description TEXT,
|
||||
ocr_model VARCHAR(100), -- z.B. 'llama3.2-vision:11b'
|
||||
total_items INTEGER DEFAULT 0,
|
||||
labeled_items INTEGER DEFAULT 0,
|
||||
confirmed_items INTEGER DEFAULT 0,
|
||||
corrected_items INTEGER DEFAULT 0,
|
||||
skipped_items INTEGER DEFAULT 0,
|
||||
teacher_id VARCHAR(100),
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Einzelne Labeling Items (Bild + OCR + Ground Truth)
|
||||
CREATE TABLE ocr_labeling_items (
|
||||
id VARCHAR(36) PRIMARY KEY,
|
||||
session_id VARCHAR(36) REFERENCES ocr_labeling_sessions(id),
|
||||
image_path TEXT NOT NULL, -- MinIO Pfad oder lokaler Pfad
|
||||
image_hash VARCHAR(64), -- SHA256 für Deduplizierung
|
||||
ocr_text TEXT, -- Von LLM erkannter Text
|
||||
ocr_confidence FLOAT, -- Konfidenz (0-1)
|
||||
ocr_model VARCHAR(100),
|
||||
ground_truth TEXT, -- Korrigierter/bestätigter Text
|
||||
status VARCHAR(20) DEFAULT 'pending', -- pending/confirmed/corrected/skipped
|
||||
labeled_by VARCHAR(100),
|
||||
labeled_at TIMESTAMP,
|
||||
label_time_seconds INTEGER,
|
||||
metadata JSONB,
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Exportierte Training Samples
|
||||
CREATE TABLE ocr_training_samples (
|
||||
id VARCHAR(36) PRIMARY KEY,
|
||||
item_id VARCHAR(36) REFERENCES ocr_labeling_items(id),
|
||||
image_path TEXT NOT NULL,
|
||||
ground_truth TEXT NOT NULL,
|
||||
export_format VARCHAR(50) NOT NULL, -- 'generic', 'trocr', 'llama_vision'
|
||||
exported_at TIMESTAMP DEFAULT NOW(),
|
||||
training_batch VARCHAR(100),
|
||||
used_in_training BOOLEAN DEFAULT FALSE
|
||||
);
|
||||
```
|
||||
|
||||
## API Referenz
|
||||
|
||||
Base URL: `http://macmini:8086/api/v1/ocr-label`
|
||||
|
||||
### Sessions
|
||||
|
||||
#### POST /sessions
|
||||
Neue Labeling-Session erstellen.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"name": "Klausur Deutsch 12a Q1",
|
||||
"source_type": "klausur",
|
||||
"description": "Gedichtanalyse Expressionismus",
|
||||
"ocr_model": "llama3.2-vision:11b"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"id": "abc-123-def",
|
||||
"name": "Klausur Deutsch 12a Q1",
|
||||
"source_type": "klausur",
|
||||
"total_items": 0,
|
||||
"labeled_items": 0,
|
||||
"created_at": "2026-01-21T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### GET /sessions
|
||||
Sessions auflisten.
|
||||
|
||||
**Query Parameter:**
|
||||
- `limit` (int, default: 50) - Maximale Anzahl
|
||||
|
||||
#### GET /sessions/{session_id}
|
||||
Einzelne Session abrufen.
|
||||
|
||||
### Upload
|
||||
|
||||
#### POST /sessions/{session_id}/upload
|
||||
Bilder zu einer Session hochladen.
|
||||
|
||||
**Request:** Multipart Form Data
|
||||
- `files` (File[]) - PNG/JPG/PDF Dateien
|
||||
- `run_ocr` (bool, default: true) - OCR direkt ausführen
|
||||
- `metadata` (JSON string) - Optional: Metadaten
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"session_id": "abc-123-def",
|
||||
"uploaded_count": 5,
|
||||
"items": [
|
||||
{
|
||||
"id": "item-1",
|
||||
"filename": "scan_001.png",
|
||||
"image_path": "ocr-labeling/abc-123/item-1.png",
|
||||
"ocr_text": "Die Lösung der Aufgabe...",
|
||||
"ocr_confidence": 0.87,
|
||||
"status": "pending"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Labeling Queue
|
||||
|
||||
#### GET /queue
|
||||
Nächste zu labelnde Items abrufen.
|
||||
|
||||
**Query Parameter:**
|
||||
- `session_id` (str, optional) - Nach Session filtern
|
||||
- `status` (str, default: "pending") - Status-Filter
|
||||
- `limit` (int, default: 10) - Maximale Anzahl
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": "item-456",
|
||||
"session_id": "abc-123",
|
||||
"session_name": "Klausur Deutsch",
|
||||
"image_path": "/app/ocr-labeling/abc-123/item-456.png",
|
||||
"image_url": "/api/v1/ocr-label/images/abc-123/item-456.png",
|
||||
"ocr_text": "Erkannter Text...",
|
||||
"ocr_confidence": 0.87,
|
||||
"ground_truth": null,
|
||||
"status": "pending",
|
||||
"metadata": {"page": 1}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Labeling Actions
|
||||
|
||||
#### POST /confirm
|
||||
OCR-Text als korrekt bestätigen.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"item_id": "item-456",
|
||||
"label_time_seconds": 5
|
||||
}
|
||||
```
|
||||
|
||||
**Effect:** `ground_truth = ocr_text`, `status = 'confirmed'`
|
||||
|
||||
#### POST /correct
|
||||
Ground Truth korrigieren.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"item_id": "item-456",
|
||||
"ground_truth": "Korrigierter Text hier",
|
||||
"label_time_seconds": 15
|
||||
}
|
||||
```
|
||||
|
||||
**Effect:** `ground_truth = <input>`, `status = 'corrected'`
|
||||
|
||||
#### POST /skip
|
||||
Item überspringen (unbrauchbar).
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"item_id": "item-456"
|
||||
}
|
||||
```
|
||||
|
||||
**Effect:** `status = 'skipped'` (wird nicht exportiert)
|
||||
|
||||
### Statistiken
|
||||
|
||||
#### GET /stats
|
||||
Labeling-Statistiken abrufen.
|
||||
|
||||
**Query Parameter:**
|
||||
- `session_id` (str, optional) - Für Session-spezifische Stats
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"total_items": 100,
|
||||
"labeled_items": 75,
|
||||
"confirmed_items": 60,
|
||||
"corrected_items": 15,
|
||||
"pending_items": 25,
|
||||
"accuracy_rate": 0.80,
|
||||
"avg_label_time_seconds": 8.5
|
||||
}
|
||||
```
|
||||
|
||||
### Training Export
|
||||
|
||||
#### POST /export
|
||||
Trainingsdaten exportieren.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"export_format": "trocr",
|
||||
"session_id": "abc-123",
|
||||
"batch_id": "batch_20260121"
|
||||
}
|
||||
```
|
||||
|
||||
**Export Formate:**
|
||||
|
||||
| Format | Beschreibung | Output |
|
||||
|--------|--------------|--------|
|
||||
| `generic` | Allgemeines JSONL | `{"id", "image_path", "ground_truth", ...}` |
|
||||
| `trocr` | Microsoft TrOCR | `{"file_name", "text", "id"}` |
|
||||
| `llama_vision` | Llama 3.2 Vision | OpenAI-style Messages mit image_url |
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"export_format": "trocr",
|
||||
"batch_id": "batch_20260121",
|
||||
"exported_count": 75,
|
||||
"export_path": "/app/ocr-exports/trocr/batch_20260121",
|
||||
"manifest_path": "/app/ocr-exports/trocr/batch_20260121/manifest.json",
|
||||
"samples": [...]
|
||||
}
|
||||
```
|
||||
|
||||
#### GET /exports
|
||||
Verfügbare Exports auflisten.
|
||||
|
||||
**Query Parameter:**
|
||||
- `export_format` (str, optional) - Nach Format filtern
|
||||
|
||||
## Export Formate im Detail
|
||||
|
||||
### TrOCR Format
|
||||
|
||||
```
|
||||
batch_20260121/
|
||||
├── manifest.json
|
||||
├── train.jsonl
|
||||
└── images/
|
||||
├── item-1.png
|
||||
└── item-2.png
|
||||
```
|
||||
|
||||
**train.jsonl:**
|
||||
```jsonl
|
||||
{"file_name": "images/item-1.png", "text": "Ground truth text", "id": "item-1"}
|
||||
{"file_name": "images/item-2.png", "text": "Another text", "id": "item-2"}
|
||||
```
|
||||
|
||||
### Llama Vision Format
|
||||
|
||||
```jsonl
|
||||
{
|
||||
"id": "item-1",
|
||||
"messages": [
|
||||
{"role": "system", "content": "Du bist ein OCR-Experte für deutsche Handschrift..."},
|
||||
{"role": "user", "content": [
|
||||
{"type": "image_url", "image_url": {"url": "images/item-1.png"}},
|
||||
{"type": "text", "text": "Lies den handgeschriebenen Text in diesem Bild."}
|
||||
]},
|
||||
{"role": "assistant", "content": "Ground truth text"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Generic Format
|
||||
|
||||
```jsonl
|
||||
{
|
||||
"id": "item-1",
|
||||
"image_path": "images/item-1.png",
|
||||
"ground_truth": "Ground truth text",
|
||||
"ocr_text": "OCR recognized text",
|
||||
"ocr_confidence": 0.87,
|
||||
"metadata": {"page": 1, "session": "Deutsch 12a"}
|
||||
}
|
||||
```
|
||||
|
||||
## Frontend Integration
|
||||
|
||||
Die OCR-Labeling UI ist unter `/admin/ocr-labeling` verfügbar.
|
||||
|
||||
### Keyboard Shortcuts
|
||||
|
||||
| Taste | Aktion |
|
||||
|-------|--------|
|
||||
| `Enter` | Bestätigen (OCR korrekt) |
|
||||
| `Tab` | Ins Korrekturfeld springen |
|
||||
| `Escape` | Überspringen |
|
||||
| `←` / `→` | Navigation (Prev/Next) |
|
||||
|
||||
### Workflow
|
||||
|
||||
1. **Session erstellen** - Name, Typ, OCR-Modell wählen
|
||||
2. **Bilder hochladen** - Drag & Drop oder File-Browser
|
||||
3. **Labeling durchführen** - Bild + OCR-Text vergleichen
|
||||
- Korrekt → Bestätigen (Enter)
|
||||
- Falsch → Korrigieren + Speichern
|
||||
- Unbrauchbar → Überspringen
|
||||
4. **Export** - Format wählen (TrOCR, Llama Vision, Generic)
|
||||
5. **Training starten** - Export-Ordner für Fine-Tuning nutzen
|
||||
|
||||
## Umgebungsvariablen
|
||||
|
||||
```bash
|
||||
# PostgreSQL
|
||||
DATABASE_URL=postgres://user:pass@postgres:5432/breakpilot_db
|
||||
|
||||
# MinIO (S3-kompatibel)
|
||||
MINIO_ENDPOINT=minio:9000
|
||||
MINIO_ACCESS_KEY=breakpilot
|
||||
MINIO_SECRET_KEY=breakpilot123
|
||||
MINIO_BUCKET=breakpilot-rag
|
||||
MINIO_SECURE=false
|
||||
|
||||
# Ollama (Vision-LLM)
|
||||
OLLAMA_BASE_URL=http://host.docker.internal:11434
|
||||
OLLAMA_VISION_MODEL=llama3.2-vision:11b
|
||||
OLLAMA_CORRECTION_MODEL=qwen2.5:14b
|
||||
|
||||
# Export
|
||||
OCR_EXPORT_PATH=/app/ocr-exports
|
||||
OCR_STORAGE_PATH=/app/ocr-labeling
|
||||
```
|
||||
|
||||
## Sicherheit & Datenschutz
|
||||
|
||||
- **100% Lokale Verarbeitung** - Alle Daten bleiben auf dem Mac Mini
|
||||
- **Keine Cloud-Uploads** - Ollama läuft vollständig offline
|
||||
- **DSGVO-konform** - Keine Schülerdaten verlassen das Schulnetzwerk
|
||||
- **Deduplizierung** - SHA256-Hash verhindert doppelte Bilder
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `klausur-service/backend/ocr_labeling_api.py` | FastAPI Router mit OCR Model Dispatcher |
|
||||
| `klausur-service/backend/training_export_service.py` | Export-Service für TrOCR/Llama |
|
||||
| `klausur-service/backend/metrics_db.py` | PostgreSQL CRUD Funktionen |
|
||||
| `klausur-service/backend/minio_storage.py` | MinIO OCR-Image Storage |
|
||||
| `klausur-service/backend/hybrid_vocab_extractor.py` | PaddleOCR Integration |
|
||||
| `klausur-service/backend/services/donut_ocr_service.py` | Donut OCR Service (NEU) |
|
||||
| `klausur-service/backend/services/trocr_service.py` | TrOCR Service (NEU) |
|
||||
| `website/app/admin/ocr-labeling/page.tsx` | Frontend UI mit Model-Auswahl |
|
||||
| `website/app/admin/ocr-labeling/types.ts` | TypeScript Interfaces inkl. OCRModel Type |
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
# Backend-Tests ausführen
|
||||
cd klausur-service/backend
|
||||
pytest tests/test_ocr_labeling.py -v
|
||||
|
||||
# Mit Coverage
|
||||
pytest tests/test_ocr_labeling.py --cov=. --cov-report=html
|
||||
```
|
||||
Reference in New Issue
Block a user