fix: Restore all files lost during destructive rebase
A previous `git pull --rebase origin main` dropped 177 local commits,
losing 3400+ files across admin-v2, backend, studio-v2, website,
klausur-service, and many other services. The partial restore attempt
(660295e2) only recovered some files.
This commit restores all missing files from pre-rebase ref 98933f5e
while preserving post-rebase additions (night-scheduler, night-mode UI,
NightModeWidget dashboard integration).
Restored features include:
- AI Module Sidebar (FAB), OCR Labeling, OCR Compare
- GPU Dashboard, RAG Pipeline, Magic Help
- Klausur-Korrektur (8 files), Abitur-Archiv (5+ files)
- Companion, Zeugnisse-Crawler, Screen Flow
- Full backend, studio-v2, website, klausur-service
- All compliance SDKs, agent-core, voice-service
- CI/CD configs, documentation, scripts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Benjamin Admin
1010
docs/ai-content-generator.md
Normal file
1010
docs/ai-content-generator.md
Normal file
File diff suppressed because it is too large
Load Diff
1361
docs/api/backend-api.md
Normal file
1361
docs/api/backend-api.md
Normal file
File diff suppressed because it is too large
Load Diff
294
docs/architecture/auth-system.md
Normal file
294
docs/architecture/auth-system.md
Normal file
@@ -0,0 +1,294 @@
|
||||
# BreakPilot Authentifizierung & Autorisierung
|
||||
|
||||
## Uebersicht
|
||||
|
||||
BreakPilot verwendet einen **Hybrid-Ansatz** fuer Authentifizierung und Autorisierung:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ AUTHENTIFIZIERUNG │
|
||||
│ "Wer bist du?" │
|
||||
│ ┌────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ HybridAuthenticator │ │
|
||||
│ │ ┌─────────────────────┐ ┌─────────────────────────────────┐ │ │
|
||||
│ │ │ Keycloak │ │ Lokales JWT │ │ │
|
||||
│ │ │ (Produktion) │ OR │ (Entwicklung) │ │ │
|
||||
│ │ │ RS256 + JWKS │ │ HS256 + Secret │ │ │
|
||||
│ │ └─────────────────────┘ └─────────────────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ AUTORISIERUNG │
|
||||
│ "Was darfst du?" │
|
||||
│ ┌────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ rbac.py (Eigenentwicklung) │ │
|
||||
│ │ ┌─────────────────┐ ┌─────────────────┐ ┌───────────────────┐ │ │
|
||||
│ │ │ Rollen-Hierarchie│ │ PolicySet │ │ DEFAULT_PERMISSIONS│ │ │
|
||||
│ │ │ 15+ Rollen │ │ Bundesland- │ │ Matrix │ │ │
|
||||
│ │ │ - Erstkorrektor │ │ spezifisch │ │ Rolle→Ressource→ │ │ │
|
||||
│ │ │ - Klassenlehrer │ │ - Niedersachsen │ │ Aktion │ │ │
|
||||
│ │ │ - Schulleitung │ │ - Bayern │ │ │ │ │
|
||||
│ │ └─────────────────┘ └─────────────────┘ └───────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Warum dieser Ansatz?
|
||||
|
||||
### Alternative Loesungen (verworfen)
|
||||
|
||||
| Tool | Problem fuer BreakPilot |
|
||||
|------|-------------------------|
|
||||
| **Casbin** | Zu generisch fuer Bundesland-spezifische Policies |
|
||||
| **Cerbos** | Overhead: Externer PDP-Service fuer ~15 Rollen ueberdimensioniert |
|
||||
| **OpenFGA** | Zanzibar-Modell optimiert fuer Graph-Beziehungen, nicht Hierarchien |
|
||||
| **Keycloak RBAC** | Kann keine ressourcen-spezifischen Zuweisungen (User X ist Erstkorrektor fuer Package Y) |
|
||||
|
||||
### Vorteile des Hybrid-Ansatzes
|
||||
|
||||
1. **Keycloak fuer Authentifizierung:**
|
||||
- Bewährtes IAM-System
|
||||
- SSO, Federation, MFA
|
||||
- Apache-2.0 Lizenz
|
||||
|
||||
2. **Eigenes rbac.py fuer Autorisierung:**
|
||||
- Domaenenspezifische Logik (Korrekturkette, Zeugnis-Workflow)
|
||||
- Bundesland-spezifische Regeln
|
||||
- Zeitlich begrenzte Zuweisungen
|
||||
- Key-Sharing fuer verschluesselte Klausuren
|
||||
|
||||
---
|
||||
|
||||
## Authentifizierung (auth/keycloak_auth.py)
|
||||
|
||||
### Konfiguration
|
||||
|
||||
```python
|
||||
# Entwicklung: Lokales JWT (Standard)
|
||||
JWT_SECRET=your-secret-key
|
||||
|
||||
# Produktion: Keycloak
|
||||
KEYCLOAK_SERVER_URL=https://keycloak.breakpilot.app
|
||||
KEYCLOAK_REALM=breakpilot
|
||||
KEYCLOAK_CLIENT_ID=breakpilot-backend
|
||||
KEYCLOAK_CLIENT_SECRET=your-client-secret
|
||||
```
|
||||
|
||||
### Token-Erkennung
|
||||
|
||||
Der `HybridAuthenticator` erkennt automatisch den Token-Typ:
|
||||
|
||||
```python
|
||||
# Keycloak-Token (RS256)
|
||||
{
|
||||
"iss": "https://keycloak.breakpilot.app/realms/breakpilot",
|
||||
"sub": "user-uuid",
|
||||
"realm_access": {"roles": ["teacher", "admin"]},
|
||||
...
|
||||
}
|
||||
|
||||
# Lokales JWT (HS256)
|
||||
{
|
||||
"iss": "breakpilot",
|
||||
"user_id": "user-uuid",
|
||||
"role": "admin",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
### FastAPI Integration
|
||||
|
||||
```python
|
||||
from auth import get_current_user
|
||||
|
||||
@app.get("/api/protected")
|
||||
async def protected_endpoint(user: dict = Depends(get_current_user)):
|
||||
# user enthält: user_id, email, role, realm_roles, tenant_id
|
||||
return {"user_id": user["user_id"]}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Autorisierung (klausur-service/backend/rbac.py)
|
||||
|
||||
### Rollen (15+)
|
||||
|
||||
| Rolle | Beschreibung | Bereich |
|
||||
|-------|--------------|---------|
|
||||
| `erstkorrektor` | Erster Prüfer | Klausur |
|
||||
| `zweitkorrektor` | Zweiter Prüfer | Klausur |
|
||||
| `drittkorrektor` | Dritter Prüfer | Klausur |
|
||||
| `klassenlehrer` | Klassenleitung | Zeugnis |
|
||||
| `fachlehrer` | Fachlehrkraft | Noten |
|
||||
| `fachvorsitz` | Fachkonferenz-Leitung | Fachschaft |
|
||||
| `schulleitung` | Schulleiter/in | Schule |
|
||||
| `zeugnisbeauftragter` | Zeugnis-Koordination | Zeugnis |
|
||||
| `sekretariat` | Verwaltung | Schule |
|
||||
| `data_protection_officer` | DSB | DSGVO |
|
||||
| ... | | |
|
||||
|
||||
### Ressourcentypen (25+)
|
||||
|
||||
```python
|
||||
class ResourceType(str, Enum):
|
||||
EXAM_PACKAGE = "exam_package" # Klausurpaket
|
||||
STUDENT_SUBMISSION = "student_submission"
|
||||
CORRECTION = "correction"
|
||||
ZEUGNIS = "zeugnis"
|
||||
FACHNOTE = "fachnote"
|
||||
KOPFNOTE = "kopfnote"
|
||||
BEMERKUNG = "bemerkung"
|
||||
...
|
||||
```
|
||||
|
||||
### Aktionen (17)
|
||||
|
||||
```python
|
||||
class Action(str, Enum):
|
||||
CREATE = "create"
|
||||
READ = "read"
|
||||
UPDATE = "update"
|
||||
DELETE = "delete"
|
||||
SIGN_OFF = "sign_off" # Freigabe
|
||||
BREAK_GLASS = "break_glass" # Notfall-Zugriff
|
||||
SHARE_KEY = "share_key" # Schlüssel teilen
|
||||
...
|
||||
```
|
||||
|
||||
### Permission-Pruefung
|
||||
|
||||
```python
|
||||
from klausur_service.backend.rbac import PolicyEngine
|
||||
|
||||
engine = PolicyEngine()
|
||||
|
||||
# Pruefe ob User X Klausur Y korrigieren darf
|
||||
allowed = engine.check_permission(
|
||||
user_id="user-uuid",
|
||||
action=Action.UPDATE,
|
||||
resource_type=ResourceType.CORRECTION,
|
||||
resource_id="klausur-uuid"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bundesland-spezifische Policies
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class PolicySet:
|
||||
bundesland: str
|
||||
abitur_type: str # "landesabitur" | "zentralabitur"
|
||||
|
||||
# Korrekturkette
|
||||
korrektoren_anzahl: int # 2 oder 3
|
||||
anonyme_erstkorrektur: bool
|
||||
|
||||
# Sichtbarkeit
|
||||
zk_visibility_mode: ZKVisibilityMode # BLIND | SEMI | FULL
|
||||
eh_visibility_mode: EHVisibilityMode
|
||||
|
||||
# Zeugnis
|
||||
kopfnoten_enabled: bool
|
||||
...
|
||||
```
|
||||
|
||||
### Beispiel: Niedersachsen
|
||||
|
||||
```python
|
||||
NIEDERSACHSEN_POLICY = PolicySet(
|
||||
bundesland="niedersachsen",
|
||||
abitur_type="landesabitur",
|
||||
korrektoren_anzahl=2,
|
||||
anonyme_erstkorrektur=True,
|
||||
zk_visibility_mode=ZKVisibilityMode.BLIND,
|
||||
eh_visibility_mode=EHVisibilityMode.SUMMARY_ONLY,
|
||||
kopfnoten_enabled=True,
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow-Beispiele
|
||||
|
||||
### Klausurkorrektur-Workflow
|
||||
|
||||
```
|
||||
1. Lehrer laedt Klausuren hoch
|
||||
└── Rolle: "lehrer" + Action.CREATE auf EXAM_PACKAGE
|
||||
|
||||
2. Erstkorrektor korrigiert
|
||||
└── Rolle: "erstkorrektor" (ressourcen-spezifisch) + Action.UPDATE auf CORRECTION
|
||||
|
||||
3. Zweitkorrektor ueberprueft
|
||||
└── Rolle: "zweitkorrektor" + Action.READ auf CORRECTION
|
||||
└── Policy: zk_visibility_mode bestimmt Sichtbarkeit
|
||||
|
||||
4. Drittkorrektor (bei Abweichung)
|
||||
└── Rolle: "drittkorrektor" + Action.SIGN_OFF
|
||||
```
|
||||
|
||||
### Zeugnis-Workflow
|
||||
|
||||
```
|
||||
1. Fachlehrer traegt Noten ein
|
||||
└── Rolle: "fachlehrer" + Action.CREATE auf FACHNOTE
|
||||
|
||||
2. Klassenlehrer prueft
|
||||
└── Rolle: "klassenlehrer" + Action.READ auf ZEUGNIS
|
||||
└── Action.SIGN_OFF freigeben
|
||||
|
||||
3. Zeugnisbeauftragter final
|
||||
└── Rolle: "zeugnisbeauftragter" + Action.SIGN_OFF
|
||||
|
||||
4. Schulleitung unterzeichnet
|
||||
└── Rolle: "schulleitung" + Action.SIGN_OFF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `backend/auth/__init__.py` | Auth-Modul Exports |
|
||||
| `backend/auth/keycloak_auth.py` | Hybrid-Authentifizierung |
|
||||
| `klausur-service/backend/rbac.py` | Autorisierungs-Engine |
|
||||
| `backend/rbac_api.py` | REST API fuer Rollenverwaltung |
|
||||
|
||||
---
|
||||
|
||||
## Konfiguration
|
||||
|
||||
### Entwicklung (ohne Keycloak)
|
||||
|
||||
```bash
|
||||
# .env
|
||||
ENVIRONMENT=development
|
||||
JWT_SECRET=dev-secret-32-chars-minimum-here
|
||||
```
|
||||
|
||||
### Produktion (mit Keycloak)
|
||||
|
||||
```bash
|
||||
# .env
|
||||
ENVIRONMENT=production
|
||||
JWT_SECRET=<openssl rand -hex 32>
|
||||
KEYCLOAK_SERVER_URL=https://keycloak.breakpilot.app
|
||||
KEYCLOAK_REALM=breakpilot
|
||||
KEYCLOAK_CLIENT_ID=breakpilot-backend
|
||||
KEYCLOAK_CLIENT_SECRET=<from keycloak admin console>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sicherheitshinweise
|
||||
|
||||
1. **Secrets niemals im Code** - Immer Umgebungsvariablen verwenden
|
||||
2. **JWT_SECRET in Produktion** - Mindestens 32 Bytes, generiert mit `openssl rand -hex 32`
|
||||
3. **Keycloak HTTPS** - KEYCLOAK_VERIFY_SSL=true in Produktion
|
||||
4. **Token-Expiration** - Keycloak-Tokens kurz halten (5-15 Minuten)
|
||||
5. **Audit-Trail** - Alle Berechtigungspruefungen werden geloggt
|
||||
314
docs/architecture/devsecops.md
Normal file
314
docs/architecture/devsecops.md
Normal file
@@ -0,0 +1,314 @@
|
||||
# BreakPilot DevSecOps Architecture
|
||||
|
||||
## Uebersicht
|
||||
|
||||
BreakPilot implementiert einen umfassenden DevSecOps-Ansatz mit Security-by-Design fuer die Entwicklung und den Betrieb der Bildungsplattform.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ DEVSECOPS PIPELINE │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Pre-Commit │───►│ CI/CD │───►│ Build │───►│ Deploy │ │
|
||||
│ │ Hooks │ │ Pipeline │ │ & Scan │ │ & Monitor │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||
│ │ │ │ │ │
|
||||
│ ▼ ▼ ▼ ▼ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Gitleaks │ │ Semgrep │ │ Trivy │ │ Falco │ │
|
||||
│ │ Bandit │ │ OWASP DC │ │ Grype │ │ (optional) │ │
|
||||
│ │ Secrets │ │ SAST/SCA │ │ SBOM │ │ Runtime │ │
|
||||
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Security Tools Stack
|
||||
|
||||
### 1. Secrets Detection
|
||||
|
||||
| Tool | Version | Lizenz | Verwendung |
|
||||
|------|---------|--------|------------|
|
||||
| **Gitleaks** | 8.18.x | MIT | Pre-commit Hook, CI/CD |
|
||||
| **detect-secrets** | 1.4.x | Apache-2.0 | Zusaetzliche Baseline-Pruefung |
|
||||
|
||||
**Konfiguration:** `.gitleaks.toml`
|
||||
|
||||
```bash
|
||||
# Lokal ausfuehren
|
||||
gitleaks detect --source . -v
|
||||
|
||||
# Pre-commit (automatisch)
|
||||
gitleaks protect --staged -v
|
||||
```
|
||||
|
||||
### 2. Static Application Security Testing (SAST)
|
||||
|
||||
| Tool | Version | Lizenz | Sprachen |
|
||||
|------|---------|--------|----------|
|
||||
| **Semgrep** | 1.52.x | LGPL-2.1 | Python, Go, JavaScript, TypeScript |
|
||||
| **Bandit** | 1.7.x | Apache-2.0 | Python (spezialisiert) |
|
||||
|
||||
**Konfiguration:** `.semgrep.yml`
|
||||
|
||||
```bash
|
||||
# Semgrep ausfuehren
|
||||
semgrep scan --config auto --config .semgrep.yml
|
||||
|
||||
# Bandit ausfuehren
|
||||
bandit -r backend/ -ll
|
||||
```
|
||||
|
||||
### 3. Software Composition Analysis (SCA)
|
||||
|
||||
| Tool | Version | Lizenz | Verwendung |
|
||||
|------|---------|--------|------------|
|
||||
| **Trivy** | 0.48.x | Apache-2.0 | Filesystem, Container, IaC |
|
||||
| **Grype** | 0.74.x | Apache-2.0 | Vulnerability Scanning |
|
||||
| **OWASP Dependency-Check** | 9.x | Apache-2.0 | CVE/NVD Abgleich |
|
||||
|
||||
**Konfiguration:** `.trivy.yaml`
|
||||
|
||||
```bash
|
||||
# Filesystem-Scan
|
||||
trivy fs . --severity HIGH,CRITICAL
|
||||
|
||||
# Container-Scan
|
||||
trivy image breakpilot-pwa-backend:latest
|
||||
```
|
||||
|
||||
### 4. SBOM (Software Bill of Materials)
|
||||
|
||||
| Tool | Version | Lizenz | Formate |
|
||||
|------|---------|--------|---------|
|
||||
| **Syft** | 0.100.x | Apache-2.0 | CycloneDX, SPDX |
|
||||
|
||||
```bash
|
||||
# SBOM generieren
|
||||
syft dir:. -o cyclonedx-json=sbom.json
|
||||
syft dir:. -o spdx-json=sbom-spdx.json
|
||||
```
|
||||
|
||||
### 5. Dynamic Application Security Testing (DAST)
|
||||
|
||||
| Tool | Version | Lizenz | Verwendung |
|
||||
|------|---------|--------|------------|
|
||||
| **OWASP ZAP** | 2.14.x | Apache-2.0 | Staging-Scans (nightly) |
|
||||
|
||||
```bash
|
||||
# ZAP Scan gegen Staging
|
||||
docker run -t owasp/zap2docker-stable zap-baseline.py \
|
||||
-t http://staging.breakpilot.app -r zap-report.html
|
||||
```
|
||||
|
||||
## Pre-Commit Hooks
|
||||
|
||||
Die Pre-Commit-Konfiguration (`.pre-commit-config.yaml`) fuehrt automatisch bei jedem Commit aus:
|
||||
|
||||
1. **Schnelle Checks** (< 10 Sekunden):
|
||||
- Gitleaks (Secrets)
|
||||
- Trailing Whitespace
|
||||
- YAML/JSON Validierung
|
||||
|
||||
2. **Code Quality** (< 30 Sekunden):
|
||||
- Black/Ruff (Python Formatting)
|
||||
- Go fmt/vet
|
||||
- ESLint (JavaScript)
|
||||
|
||||
3. **Security Checks** (< 60 Sekunden):
|
||||
- Bandit (Python Security)
|
||||
- Semgrep (Error-Severity)
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# Pre-commit installieren
|
||||
pip install pre-commit
|
||||
|
||||
# Hooks aktivieren
|
||||
pre-commit install
|
||||
|
||||
# Alle Checks manuell ausfuehren
|
||||
pre-commit run --all-files
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### GitHub Actions Pipeline
|
||||
|
||||
```yaml
|
||||
# .github/workflows/security.yml
|
||||
name: Security Scan
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
secrets:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: gitleaks/gitleaks-action@v2
|
||||
|
||||
sast:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: returntocorp/semgrep-action@v1
|
||||
with:
|
||||
config: >-
|
||||
auto
|
||||
.semgrep.yml
|
||||
|
||||
sca:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: aquasecurity/trivy-action@master
|
||||
with:
|
||||
scan-type: 'fs'
|
||||
severity: 'HIGH,CRITICAL'
|
||||
|
||||
sbom:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: anchore/sbom-action@v0
|
||||
with:
|
||||
format: cyclonedx-json
|
||||
```
|
||||
|
||||
## Security Reports
|
||||
|
||||
Alle Security-Reports werden in `security-reports/` gespeichert:
|
||||
|
||||
| Report | Format | Tool |
|
||||
|--------|--------|------|
|
||||
| `gitleaks-*.json` | JSON | Gitleaks |
|
||||
| `semgrep-*.json` | SARIF/JSON | Semgrep |
|
||||
| `bandit-*.json` | JSON | Bandit |
|
||||
| `trivy-fs-*.json` | JSON | Trivy |
|
||||
| `trivy-image-*.json` | JSON | Trivy |
|
||||
| `grype-*.json` | JSON | Grype |
|
||||
| `sbom-*.json` | CycloneDX | Syft |
|
||||
|
||||
### Security-Scan Script
|
||||
|
||||
```bash
|
||||
# Alle Scans ausfuehren
|
||||
./scripts/security-scan.sh --all
|
||||
|
||||
# Nur Secrets-Scan
|
||||
./scripts/security-scan.sh --secrets
|
||||
|
||||
# CI-Modus (Exit bei Critical Findings)
|
||||
./scripts/security-scan.sh --all --ci
|
||||
```
|
||||
|
||||
## Severity-Gates
|
||||
|
||||
| Phase | Severity | Aktion |
|
||||
|-------|----------|--------|
|
||||
| Pre-Commit | ERROR | Commit blockiert |
|
||||
| PR/CI | CRITICAL, HIGH | Pipeline blockiert |
|
||||
| Nightly Scan | MEDIUM+ | Report generiert |
|
||||
| Production Deploy | CRITICAL | Deploy blockiert |
|
||||
|
||||
## Compliance
|
||||
|
||||
Die DevSecOps-Pipeline unterstuetzt folgende Compliance-Anforderungen:
|
||||
|
||||
- **DSGVO/GDPR**: Automatische Erkennung von PII-Leaks
|
||||
- **OWASP Top 10**: SAST/DAST-Scans gegen bekannte Schwachstellen
|
||||
- **Supply Chain Security**: SBOM-Generierung fuer Audit-Trails
|
||||
- **CVE Tracking**: Automatischer Abgleich mit NVD/CVE-Datenbanken
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `.gitleaks.toml` | Gitleaks Konfiguration |
|
||||
| `.semgrep.yml` | Semgrep Custom Rules |
|
||||
| `.trivy.yaml` | Trivy Konfiguration |
|
||||
| `.trivyignore` | Trivy Ignore-Liste |
|
||||
| `.pre-commit-config.yaml` | Pre-Commit Hooks |
|
||||
| `scripts/security-scan.sh` | Security-Scan Script |
|
||||
|
||||
## Tool-Installation
|
||||
|
||||
### macOS (Homebrew)
|
||||
|
||||
```bash
|
||||
# Security Tools
|
||||
brew install gitleaks
|
||||
brew install trivy
|
||||
brew install syft
|
||||
brew install grype
|
||||
|
||||
# Python Tools
|
||||
pip install semgrep bandit pre-commit
|
||||
```
|
||||
|
||||
### Linux (apt/snap)
|
||||
|
||||
```bash
|
||||
# Gitleaks
|
||||
sudo snap install gitleaks
|
||||
|
||||
# Trivy
|
||||
sudo apt-get install trivy
|
||||
|
||||
# Python Tools
|
||||
pip install semgrep bandit pre-commit
|
||||
```
|
||||
|
||||
## Security Dashboard
|
||||
|
||||
Das BreakPilot Admin Panel enthaelt ein integriertes Security Dashboard unter **Verwaltung > Security**.
|
||||
|
||||
### Features
|
||||
|
||||
**Fuer Entwickler:**
|
||||
- Scan-Ergebnisse auf einen Blick
|
||||
- Pre-commit Hook Status
|
||||
- Quick-Fix Suggestions
|
||||
- SBOM Viewer mit Suchfunktion
|
||||
|
||||
**Fuer Security-Experten:**
|
||||
- Vulnerability Severity Distribution (Critical/High/Medium/Low)
|
||||
- CVE-Tracking mit Fix-Verfuegbarkeit
|
||||
- Compliance-Status (OWASP Top 10, DSGVO)
|
||||
- Secrets Detection History
|
||||
|
||||
**Fuer Ops:**
|
||||
- Container Image Scan Results
|
||||
- Dependency Update Status
|
||||
- Security Scan Scheduling
|
||||
- Auto-Refresh alle 30 Sekunden
|
||||
|
||||
### API Endpoints
|
||||
|
||||
```
|
||||
GET /api/v1/security/tools - Tool-Status
|
||||
GET /api/v1/security/findings - Alle Findings
|
||||
GET /api/v1/security/summary - Severity-Zusammenfassung
|
||||
GET /api/v1/security/sbom - SBOM-Daten
|
||||
GET /api/v1/security/history - Scan-Historie
|
||||
GET /api/v1/security/reports/{tool} - Tool-spezifischer Report
|
||||
POST /api/v1/security/scan/{type} - Scan starten (secrets/sast/deps/containers/sbom/all)
|
||||
GET /api/v1/security/health - Health-Check
|
||||
```
|
||||
|
||||
### Frontend-Integration
|
||||
|
||||
Das Security-Modul ist unter `backend/frontend/modules/security.py` implementiert und folgt der modularen Studio-Architektur mit:
|
||||
- `SecurityModule.get_css()` - Dashboard-Styles
|
||||
- `SecurityModule.get_html()` - Panel-Struktur
|
||||
- `SecurityModule.get_js()` - Dashboard-Logik
|
||||
|
||||
## Weiterentwicklung
|
||||
|
||||
Geplante Erweiterungen:
|
||||
|
||||
1. **OPA/Conftest**: Policy-as-Code fuer Terraform/Kubernetes
|
||||
2. **Falco**: Runtime-Security fuer Kubernetes
|
||||
3. **OWASP ZAP**: Automatisierte DAST-Scans
|
||||
4. **Dependency-Track**: SBOM-basiertes Vulnerability Management
|
||||
197
docs/architecture/environments.md
Normal file
197
docs/architecture/environments.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# Umgebungs-Architektur
|
||||
|
||||
## Übersicht
|
||||
|
||||
BreakPilot verwendet eine 3-Umgebungs-Strategie für sichere Entwicklung und Deployment:
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Development │────▶│ Staging │────▶│ Production │
|
||||
│ (develop) │ │ (staging) │ │ (main) │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
Tägliche Getesteter Code Produktionsreif
|
||||
Entwicklung
|
||||
```
|
||||
|
||||
## Umgebungen
|
||||
|
||||
### Development (Dev)
|
||||
|
||||
**Zweck:** Tägliche Entwicklungsarbeit
|
||||
|
||||
| Eigenschaft | Wert |
|
||||
|-------------|------|
|
||||
| Git Branch | `develop` |
|
||||
| Compose File | `docker-compose.yml` + `docker-compose.override.yml` (auto) |
|
||||
| Env File | `.env.dev` |
|
||||
| Database | `breakpilot_dev` |
|
||||
| Debug | Aktiviert |
|
||||
| Hot-Reload | Aktiviert |
|
||||
|
||||
**Start:**
|
||||
```bash
|
||||
./scripts/start.sh dev
|
||||
# oder einfach:
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Staging
|
||||
|
||||
**Zweck:** Getesteter, freigegebener Code vor Produktion
|
||||
|
||||
| Eigenschaft | Wert |
|
||||
|-------------|------|
|
||||
| Git Branch | `staging` |
|
||||
| Compose File | `docker-compose.yml` + `docker-compose.staging.yml` |
|
||||
| Env File | `.env.staging` |
|
||||
| Database | `breakpilot_staging` (separates Volume) |
|
||||
| Debug | Deaktiviert |
|
||||
| Hot-Reload | Deaktiviert |
|
||||
|
||||
**Start:**
|
||||
```bash
|
||||
./scripts/start.sh staging
|
||||
# oder:
|
||||
docker compose -f docker-compose.yml -f docker-compose.staging.yml up -d
|
||||
```
|
||||
|
||||
### Production (Prod)
|
||||
|
||||
**Zweck:** Live-System für Endbenutzer (ab Launch)
|
||||
|
||||
| Eigenschaft | Wert |
|
||||
|-------------|------|
|
||||
| Git Branch | `main` |
|
||||
| Compose File | `docker-compose.yml` + `docker-compose.prod.yml` |
|
||||
| Env File | `.env.prod` (NICHT im Repository!) |
|
||||
| Database | `breakpilot_prod` (separates Volume) |
|
||||
| Debug | Deaktiviert |
|
||||
| Vault | Pflicht (keine Env-Fallbacks) |
|
||||
|
||||
## Datenbank-Trennung
|
||||
|
||||
Jede Umgebung verwendet separate Docker Volumes für vollständige Datenisolierung:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PostgreSQL Volumes │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ breakpilot-dev_postgres_data │ Development Database │
|
||||
│ breakpilot_staging_postgres │ Staging Database │
|
||||
│ breakpilot_prod_postgres │ Production Database │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Port-Mapping
|
||||
|
||||
Um mehrere Umgebungen gleichzeitig laufen zu lassen, verwenden sie unterschiedliche Ports:
|
||||
|
||||
| Service | Dev Port | Staging Port | Prod Port |
|
||||
|---------|----------|--------------|-----------|
|
||||
| Backend | 8000 | 8001 | 8000 |
|
||||
| PostgreSQL | 5432 | 5433 | - (intern) |
|
||||
| MinIO | 9000/9001 | 9002/9003 | - (intern) |
|
||||
| Qdrant | 6333/6334 | 6335/6336 | - (intern) |
|
||||
| Mailpit | 8025/1025 | 8026/1026 | - (deaktiviert) |
|
||||
|
||||
## Git Branching Strategie
|
||||
|
||||
```
|
||||
main (Prod) ← Nur Release-Merges, geschützt
|
||||
│
|
||||
▼
|
||||
staging ← Getesteter Code, Review erforderlich
|
||||
│
|
||||
▼
|
||||
develop (Dev) ← Tägliche Arbeit, Default-Branch
|
||||
│
|
||||
▼
|
||||
feature/* ← Feature-Branches (optional)
|
||||
```
|
||||
|
||||
### Workflow
|
||||
|
||||
1. **Entwicklung:** Arbeite auf `develop`
|
||||
2. **Code-Review:** Erstelle PR von Feature-Branch → `develop`
|
||||
3. **Staging:** Promote `develop` → `staging` mit Tests
|
||||
4. **Release:** Promote `staging` → `main` nach Freigabe
|
||||
|
||||
### Promotion-Befehle
|
||||
|
||||
```bash
|
||||
# develop → staging
|
||||
./scripts/promote.sh dev-to-staging
|
||||
|
||||
# staging → main (Production)
|
||||
./scripts/promote.sh staging-to-prod
|
||||
```
|
||||
|
||||
## Secrets Management
|
||||
|
||||
### Development
|
||||
- `.env.dev` enthält Entwicklungs-Credentials
|
||||
- Vault optional (Dev-Token)
|
||||
- Mailpit für E-Mail-Tests
|
||||
|
||||
### Staging
|
||||
- `.env.staging` enthält Test-Credentials
|
||||
- Vault empfohlen
|
||||
- Mailpit für E-Mail-Sicherheit
|
||||
|
||||
### Production
|
||||
- `.env.prod` NICHT im Repository
|
||||
- Vault PFLICHT
|
||||
- Echte SMTP-Konfiguration
|
||||
|
||||
Siehe auch: [secrets-management.md](./secrets-management.md)
|
||||
|
||||
## Docker Compose Architektur
|
||||
|
||||
```
|
||||
docker-compose.yml ← Basis-Konfiguration
|
||||
│
|
||||
├── docker-compose.override.yml ← Dev (auto-geladen)
|
||||
│
|
||||
├── docker-compose.staging.yml ← Staging (explizit)
|
||||
│
|
||||
└── docker-compose.prod.yml ← Production (explizit)
|
||||
```
|
||||
|
||||
### Automatisches Laden
|
||||
|
||||
Docker Compose lädt automatisch:
|
||||
1. `docker-compose.yml`
|
||||
2. `docker-compose.override.yml` (falls vorhanden)
|
||||
|
||||
Daher startet `docker compose up` automatisch die Dev-Umgebung.
|
||||
|
||||
## Helper Scripts
|
||||
|
||||
| Script | Beschreibung |
|
||||
|--------|--------------|
|
||||
| `scripts/env-switch.sh` | Wechselt zwischen Umgebungen |
|
||||
| `scripts/start.sh` | Startet Services für Umgebung |
|
||||
| `scripts/stop.sh` | Stoppt Services |
|
||||
| `scripts/promote.sh` | Promotet Code zwischen Branches |
|
||||
| `scripts/status.sh` | Zeigt aktuellen Status |
|
||||
|
||||
## Verifikation
|
||||
|
||||
Nach Setup prüfen:
|
||||
|
||||
```bash
|
||||
# Status anzeigen
|
||||
./scripts/status.sh
|
||||
|
||||
# Branches prüfen
|
||||
git branch -v
|
||||
|
||||
# Volumes prüfen
|
||||
docker volume ls | grep breakpilot
|
||||
```
|
||||
|
||||
## Verwandte Dokumentation
|
||||
|
||||
- [secrets-management.md](./secrets-management.md) - Vault & Secrets
|
||||
- [devsecops.md](./devsecops.md) - CI/CD & Security
|
||||
- [system-architecture.md](./system-architecture.md) - Gesamtarchitektur
|
||||
722
docs/architecture/mail-rbac-architecture.md
Normal file
722
docs/architecture/mail-rbac-architecture.md
Normal file
@@ -0,0 +1,722 @@
|
||||
# Mail-RBAC Architektur mit Mitarbeiter-Anonymisierung
|
||||
|
||||
**Version:** 1.0.0
|
||||
**Datum:** 2026-01-10
|
||||
**Status:** Architekturplanung
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Dieses Dokument beschreibt eine neuartige Architektur, die E-Mail, Kalender und Videokonferenzen mit rollenbasierter Zugriffskontrolle (RBAC) verbindet. Das Kernkonzept ermöglicht die **vollständige Anonymisierung von Mitarbeiterdaten** bei Verlassen des Unternehmens, während geschäftliche Kommunikationshistorie erhalten bleibt.
|
||||
|
||||
**Wichtig:** Dieses Konzept existiert in dieser Form noch nicht als fertige Lösung auf dem Markt. Es handelt sich um eine innovative Architektur, die entwickelt werden muss.
|
||||
|
||||
---
|
||||
|
||||
## 1. Das Problem
|
||||
|
||||
### Traditionelle E-Mail-Systeme
|
||||
```
|
||||
max.mustermann@firma.de → Person gebunden
|
||||
→ DSGVO: Daten müssen gelöscht werden
|
||||
→ Geschäftshistorie geht verloren
|
||||
```
|
||||
|
||||
### BreakPilot-Lösung: Rollenbasierte E-Mail
|
||||
```
|
||||
klassenlehrer.5a@schule.breakpilot.app → Rolle gebunden
|
||||
→ Person kann anonymisiert werden
|
||||
→ Kommunikationshistorie bleibt erhalten
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Architektur-Übersicht
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ BreakPilot Groupware │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Webmail │ │ Kalender │ │ Jitsi │ │
|
||||
│ │ (SOGo) │ │ (SOGo) │ │ Meeting │ │
|
||||
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
|
||||
│ │ │ │ │
|
||||
│ └────────────────┼────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────┴───────────┐ │
|
||||
│ │ RBAC-Mail-Bridge │ ◄─── Neue Komponente │
|
||||
│ │ (Python/Go) │ │
|
||||
│ └───────────┬───────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────────┼─────────────────────┐ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌──────────┐ ┌──────────────┐ ┌────────────┐ │
|
||||
│ │PostgreSQL│ │ Mail Server │ │ MinIO │ │
|
||||
│ │(RBAC DB) │ │ (Stalwart) │ │ (Backups) │ │
|
||||
│ └──────────┘ └──────────────┘ └────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Komponenten-Auswahl
|
||||
|
||||
### 3.1 E-Mail Server: Stalwart Mail Server
|
||||
|
||||
**Empfehlung:** [Stalwart Mail Server](https://stalw.art/)
|
||||
|
||||
| Kriterium | Bewertung |
|
||||
|-----------|-----------|
|
||||
| Lizenz | AGPL-3.0 (Open Source) |
|
||||
| Sprache | Rust (performant, sicher) |
|
||||
| Features | IMAP, SMTP, JMAP, WebSocket |
|
||||
| Kalender | CalDAV integriert |
|
||||
| Kontakte | CardDAV integriert |
|
||||
| Spam/Virus | Integriert |
|
||||
| API | REST API für Administration |
|
||||
|
||||
**Lizenz-Implikation für kommerzielle Nutzung:**
|
||||
- AGPL-3.0 erfordert Veröffentlichung von Änderungen am Stalwart-Code selbst
|
||||
- Nutzung als Service ohne Code-Änderungen: keine Veröffentlichungspflicht
|
||||
- Unsere RBAC-Bridge ist separater Code: kann proprietär bleiben
|
||||
|
||||
### 3.2 Webmail-Client: SOGo oder Roundcube
|
||||
|
||||
**Option A: SOGo** (empfohlen)
|
||||
- Lizenz: GPL-2.0 / LGPL-2.1
|
||||
- Kalender, Kontakte, Mail in einem
|
||||
- ActiveSync Support
|
||||
- Outlook-ähnliche Oberfläche
|
||||
|
||||
**Option B: Roundcube**
|
||||
- Lizenz: GPL-3.0
|
||||
- Nur Webmail
|
||||
- Benötigt separaten Kalender
|
||||
|
||||
### 3.3 Kalender-Integration
|
||||
|
||||
Stalwart bietet CalDAV, kann mit:
|
||||
- SOGo Webinterface
|
||||
- Thunderbird
|
||||
- iOS/Android Kalender
|
||||
- Outlook (via CalDAV Plugin)
|
||||
|
||||
### 3.4 Jitsi-Integration
|
||||
|
||||
Bereits vorhanden in BreakPilot:
|
||||
- `backend/jitsi_api.py` - Meeting-Erstellung
|
||||
- `email_service.py` - Jitsi-Einladungen per E-Mail
|
||||
- Kalender-Events können Jitsi-Links enthalten
|
||||
|
||||
---
|
||||
|
||||
## 4. Datenmodell für Rollen-E-Mail
|
||||
|
||||
### 4.1 Neue Datenbank-Tabellen
|
||||
|
||||
```sql
|
||||
-- Funktionale E-Mail-Adressen (rollengebunden)
|
||||
CREATE TABLE functional_mailboxes (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
|
||||
-- Rolle und Mailbox
|
||||
role_key VARCHAR(100) NOT NULL, -- z.B. "klassenlehrer_5a"
|
||||
email_address VARCHAR(255) UNIQUE NOT NULL, -- z.B. "klassenlehrer.5a@schule.bp.app"
|
||||
display_name VARCHAR(255) NOT NULL, -- z.B. "Klassenlehrer 5a"
|
||||
|
||||
-- Zuordnung
|
||||
tenant_id UUID NOT NULL REFERENCES tenants(id),
|
||||
resource_type VARCHAR(50) DEFAULT 'class', -- class, department, function
|
||||
resource_id VARCHAR(100), -- z.B. "5a" oder "mathematik"
|
||||
|
||||
-- Status
|
||||
is_active BOOLEAN DEFAULT true,
|
||||
created_at TIMESTAMP DEFAULT NOW(),
|
||||
|
||||
CONSTRAINT fk_role FOREIGN KEY (role_key)
|
||||
REFERENCES roles(role_key) ON DELETE RESTRICT
|
||||
);
|
||||
|
||||
-- Zuordnung: Welche Person hat welche funktionale Mailbox
|
||||
CREATE TABLE mailbox_assignments (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
|
||||
mailbox_id UUID NOT NULL REFERENCES functional_mailboxes(id),
|
||||
user_id UUID NOT NULL REFERENCES users(id),
|
||||
|
||||
-- Zeitraum
|
||||
valid_from TIMESTAMP DEFAULT NOW(),
|
||||
valid_to TIMESTAMP,
|
||||
|
||||
-- Audit
|
||||
assigned_by UUID REFERENCES users(id),
|
||||
assigned_at TIMESTAMP DEFAULT NOW(),
|
||||
revoked_by UUID,
|
||||
revoked_at TIMESTAMP,
|
||||
|
||||
-- Constraints
|
||||
CONSTRAINT unique_active_assignment
|
||||
UNIQUE (mailbox_id, user_id)
|
||||
WHERE revoked_at IS NULL
|
||||
);
|
||||
|
||||
-- Persönliche E-Mail-Adressen (für Anonymisierung)
|
||||
CREATE TABLE personal_email_accounts (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id UUID NOT NULL REFERENCES users(id),
|
||||
|
||||
-- E-Mail
|
||||
email_address VARCHAR(255) UNIQUE NOT NULL,
|
||||
|
||||
-- Anonymisierungsstatus
|
||||
is_anonymized BOOLEAN DEFAULT false,
|
||||
anonymized_at TIMESTAMP,
|
||||
anonymized_by UUID,
|
||||
|
||||
-- Original-Daten (verschlüsselt, für DSGVO-Auskunft)
|
||||
original_name_encrypted BYTEA,
|
||||
original_email_encrypted BYTEA,
|
||||
encryption_key_id VARCHAR(100),
|
||||
|
||||
-- Audit
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Audit-Trail für E-Mail-Kommunikation (ohne personenbezogene Daten)
|
||||
CREATE TABLE email_audit_trail (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
|
||||
-- Kommunikation
|
||||
mailbox_id UUID REFERENCES functional_mailboxes(id),
|
||||
direction VARCHAR(10) NOT NULL, -- 'inbound', 'outbound'
|
||||
|
||||
-- Metadaten (keine Inhalte!)
|
||||
subject_hash VARCHAR(64), -- SHA-256 für Deduplizierung
|
||||
timestamp TIMESTAMP NOT NULL,
|
||||
external_party_domain VARCHAR(255), -- z.B. "eltern.de" (nicht volle Adresse)
|
||||
|
||||
-- Rolle zum Zeitpunkt
|
||||
role_key VARCHAR(100) NOT NULL,
|
||||
|
||||
-- Person NICHT gespeichert - nur über Assignment nachvollziehbar
|
||||
-- Bei Anonymisierung: Assignment-User wird anonymisiert
|
||||
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Anonymisierungsprotokoll (DSGVO-Nachweis)
|
||||
CREATE TABLE anonymization_log (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
|
||||
-- Was wurde anonymisiert
|
||||
entity_type VARCHAR(50) NOT NULL, -- 'user', 'email_account'
|
||||
entity_id UUID NOT NULL,
|
||||
|
||||
-- Wie
|
||||
anonymization_type VARCHAR(50) NOT NULL, -- 'pseudonymization', 'deletion'
|
||||
fields_affected JSONB NOT NULL,
|
||||
|
||||
-- Warum
|
||||
reason VARCHAR(100) NOT NULL, -- 'employee_departure', 'dsgvo_request'
|
||||
|
||||
-- Audit
|
||||
performed_by UUID NOT NULL,
|
||||
performed_at TIMESTAMP DEFAULT NOW(),
|
||||
|
||||
-- Bestätigung
|
||||
legal_basis VARCHAR(255), -- z.B. "Art. 17 DSGVO"
|
||||
retention_period_days INTEGER
|
||||
);
|
||||
```
|
||||
|
||||
### 4.2 Anonymisierungs-Workflow
|
||||
|
||||
```
|
||||
Mitarbeiter kündigt
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ 1. Functional Mailboxes │
|
||||
│ → Neu zuweisen oder │
|
||||
│ → Deaktivieren │
|
||||
└───────────┬───────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ 2. Personal Email Account │
|
||||
│ → Anonymisieren: │
|
||||
│ max.mustermann@... │
|
||||
│ → mitarbeiter_a7x2@... │
|
||||
└───────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ 3. Users-Tabelle │
|
||||
│ → Pseudonymisieren: │
|
||||
│ name: "Max Mustermann" │
|
||||
│ → "Ehem. Mitarbeiter" │
|
||||
└───────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ 4. Mailbox Assignments │
|
||||
│ → Bleiben für Audit │
|
||||
│ → User-Referenz zeigt │
|
||||
│ auf anonymisierte │
|
||||
│ Daten │
|
||||
└───────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ 5. E-Mail-Archiv │
|
||||
│ → Header anonymisieren │
|
||||
│ → Inhalte optional │
|
||||
│ löschen │
|
||||
└───────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. RBAC-Mail-Bridge Implementierung
|
||||
|
||||
### 5.1 Python-Komponente
|
||||
|
||||
```python
|
||||
# rbac_mail_bridge/anonymizer.py
|
||||
|
||||
class EmployeeAnonymizer:
|
||||
"""
|
||||
Anonymisiert Mitarbeiterdaten bei Ausscheiden.
|
||||
Erhält Audit-Trail und funktionale Zuordnungen.
|
||||
"""
|
||||
|
||||
async def anonymize_employee(
|
||||
self,
|
||||
user_id: str,
|
||||
reason: str,
|
||||
performed_by: str
|
||||
) -> AnonymizationResult:
|
||||
"""
|
||||
Vollständige Anonymisierung eines Mitarbeiters.
|
||||
|
||||
Schritte:
|
||||
1. Functional Mailboxes neu zuweisen
|
||||
2. Personal Email anonymisieren
|
||||
3. User-Daten pseudonymisieren
|
||||
4. E-Mail-Archive bereinigen
|
||||
5. Audit-Log erstellen
|
||||
"""
|
||||
|
||||
async with self.db.transaction():
|
||||
# 1. Functional Mailboxes
|
||||
assignments = await self.get_active_assignments(user_id)
|
||||
for assignment in assignments:
|
||||
await self.revoke_assignment(
|
||||
assignment.id,
|
||||
revoked_by=performed_by,
|
||||
reason="employee_departure"
|
||||
)
|
||||
|
||||
# 2. Personal Email
|
||||
personal_email = await self.get_personal_email(user_id)
|
||||
if personal_email:
|
||||
# Verschlüssele Original für DSGVO-Auskunft
|
||||
encrypted = await self.encrypt_for_retention(
|
||||
personal_email.email_address,
|
||||
personal_email.display_name
|
||||
)
|
||||
|
||||
# Anonymisiere
|
||||
anon_email = f"user_{generate_random_id()}@anon.local"
|
||||
await self.db.execute("""
|
||||
UPDATE personal_email_accounts
|
||||
SET
|
||||
email_address = $1,
|
||||
is_anonymized = true,
|
||||
anonymized_at = NOW(),
|
||||
original_email_encrypted = $2
|
||||
WHERE user_id = $3
|
||||
""", anon_email, encrypted, user_id)
|
||||
|
||||
# 3. User-Daten
|
||||
await self.db.execute("""
|
||||
UPDATE users SET
|
||||
name = 'Ehemaliger Mitarbeiter',
|
||||
email = $1,
|
||||
is_active = false,
|
||||
anonymized_at = NOW()
|
||||
WHERE id = $2
|
||||
""", f"anon_{user_id[:8]}@deleted.local", user_id)
|
||||
|
||||
# 4. E-Mail-Archive
|
||||
await self.mail_server.anonymize_mailbox(user_id)
|
||||
|
||||
# 5. Audit-Log
|
||||
await self.create_anonymization_log(
|
||||
entity_type="user",
|
||||
entity_id=user_id,
|
||||
reason=reason,
|
||||
performed_by=performed_by
|
||||
)
|
||||
|
||||
return AnonymizationResult(success=True)
|
||||
```
|
||||
|
||||
### 5.2 API-Endpunkte
|
||||
|
||||
```python
|
||||
# rbac_mail_bridge/api.py
|
||||
|
||||
router = APIRouter(prefix="/api/v1/mail-rbac", tags=["mail-rbac"])
|
||||
|
||||
@router.get("/mailboxes")
|
||||
async def list_functional_mailboxes(
|
||||
user: Dict = Depends(get_current_admin)
|
||||
) -> List[FunctionalMailbox]:
|
||||
"""Liste aller funktionalen Mailboxen"""
|
||||
pass
|
||||
|
||||
@router.post("/mailboxes")
|
||||
async def create_functional_mailbox(
|
||||
mailbox: FunctionalMailboxCreate,
|
||||
user: Dict = Depends(get_current_admin)
|
||||
) -> FunctionalMailbox:
|
||||
"""Erstellt eine rollengebundene Mailbox"""
|
||||
pass
|
||||
|
||||
@router.post("/mailboxes/{mailbox_id}/assign")
|
||||
async def assign_mailbox_to_user(
|
||||
mailbox_id: str,
|
||||
assignment: MailboxAssignment,
|
||||
user: Dict = Depends(get_current_admin)
|
||||
) -> Assignment:
|
||||
"""Weist eine Mailbox einem Benutzer zu"""
|
||||
pass
|
||||
|
||||
@router.post("/users/{user_id}/anonymize")
|
||||
async def anonymize_user(
|
||||
user_id: str,
|
||||
request: AnonymizationRequest,
|
||||
user: Dict = Depends(get_current_admin)
|
||||
) -> AnonymizationResult:
|
||||
"""Anonymisiert einen ausgeschiedenen Mitarbeiter"""
|
||||
pass
|
||||
|
||||
@router.get("/audit/anonymizations")
|
||||
async def list_anonymizations(
|
||||
user: Dict = Depends(get_current_admin)
|
||||
) -> List[AnonymizationLog]:
|
||||
"""Liste aller Anonymisierungen (DSGVO-Nachweis)"""
|
||||
pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Docker-Compose Integration
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml Erweiterung
|
||||
|
||||
services:
|
||||
# ... bestehende Services ...
|
||||
|
||||
# Stalwart Mail Server
|
||||
stalwart:
|
||||
image: stalwartlabs/mail-server:latest
|
||||
container_name: breakpilot-mail
|
||||
hostname: mail.breakpilot.local
|
||||
ports:
|
||||
- "25:25" # SMTP
|
||||
- "143:143" # IMAP
|
||||
- "465:465" # SMTPS
|
||||
- "993:993" # IMAPS
|
||||
- "4190:4190" # ManageSieve
|
||||
- "8080:8080" # Web Admin
|
||||
volumes:
|
||||
- stalwart-data:/opt/stalwart-mail/data
|
||||
- ./config/stalwart:/opt/stalwart-mail/etc
|
||||
environment:
|
||||
- STALWART_HOSTNAME=mail.breakpilot.local
|
||||
networks:
|
||||
- breakpilot-pwa-network
|
||||
depends_on:
|
||||
- postgres
|
||||
profiles:
|
||||
- mail # Nur mit --profile mail starten
|
||||
|
||||
# SOGo Groupware (Webmail + Kalender)
|
||||
sogo:
|
||||
image: sogo/sogo:latest
|
||||
container_name: breakpilot-sogo
|
||||
ports:
|
||||
- "20000:20000"
|
||||
volumes:
|
||||
- ./config/sogo:/etc/sogo
|
||||
environment:
|
||||
- SOGO_HOSTNAME=groupware.breakpilot.local
|
||||
depends_on:
|
||||
- stalwart
|
||||
- postgres
|
||||
networks:
|
||||
- breakpilot-pwa-network
|
||||
profiles:
|
||||
- mail
|
||||
|
||||
# RBAC-Mail-Bridge
|
||||
rbac-mail-bridge:
|
||||
build:
|
||||
context: ./rbac-mail-bridge
|
||||
dockerfile: Dockerfile
|
||||
container_name: breakpilot-rbac-mail
|
||||
environment:
|
||||
- DATABASE_URL=${DATABASE_URL}
|
||||
- STALWART_API_URL=http://stalwart:8080
|
||||
- STALWART_API_KEY=${STALWART_API_KEY}
|
||||
depends_on:
|
||||
- postgres
|
||||
- stalwart
|
||||
networks:
|
||||
- breakpilot-pwa-network
|
||||
profiles:
|
||||
- mail
|
||||
|
||||
volumes:
|
||||
stalwart-data:
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Admin-UI im Frontend
|
||||
|
||||
### 7.1 Neue Admin-Seite: `/admin/mail-management`
|
||||
|
||||
```tsx
|
||||
// website/app/admin/mail-management/page.tsx
|
||||
|
||||
// Tabs:
|
||||
// 1. Funktionale Mailboxen
|
||||
// - Liste aller rollengebundenen Adressen
|
||||
// - Zuweisungen verwalten
|
||||
//
|
||||
// 2. Mitarbeiter E-Mail
|
||||
// - Persönliche Accounts
|
||||
// - Anonymisierungs-Status
|
||||
//
|
||||
// 3. Anonymisierung
|
||||
// - Mitarbeiter-Offboarding
|
||||
// - Anonymisierungs-Wizard
|
||||
//
|
||||
// 4. Audit-Log
|
||||
// - Alle Anonymisierungen
|
||||
// - DSGVO-Export
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Lizenz-Übersicht
|
||||
|
||||
| Komponente | Lizenz | Kommerzielle Nutzung | Veröffentlichungspflicht |
|
||||
|------------|--------|---------------------|-------------------------|
|
||||
| Stalwart Mail | AGPL-3.0 | Ja | Nur bei Code-Änderungen |
|
||||
| SOGo | GPL-2.0/LGPL | Ja | Nur bei Code-Änderungen |
|
||||
| Roundcube | GPL-3.0 | Ja | Nur bei Code-Änderungen |
|
||||
| RBAC-Mail-Bridge | Eigene | N/A | Kann proprietär bleiben |
|
||||
| BreakPilot Backend | Eigene | N/A | Proprietär |
|
||||
|
||||
**Empfehlung:** Bei reiner Nutzung der Open-Source-Komponenten ohne Code-Änderungen besteht keine Veröffentlichungspflicht. Die RBAC-Mail-Bridge ist unser eigener Code.
|
||||
|
||||
---
|
||||
|
||||
## 9. Implementierungsreihenfolge
|
||||
|
||||
### Phase 1: Grundlagen (2-3 Wochen Entwicklung)
|
||||
1. Datenbank-Schema erstellen
|
||||
2. RBAC-Mail-Bridge Backend
|
||||
3. Basic API-Endpunkte
|
||||
|
||||
### Phase 2: Mail-Server Integration
|
||||
4. Stalwart konfigurieren
|
||||
5. SOGo als Webclient
|
||||
6. LDAP/OAuth Bridge
|
||||
|
||||
### Phase 3: Anonymisierung
|
||||
7. Anonymisierungs-Service
|
||||
8. E-Mail-Archive-Bereinigung
|
||||
9. DSGVO-Export-Funktion
|
||||
|
||||
### Phase 4: Frontend
|
||||
10. Admin-UI erstellen
|
||||
11. Offboarding-Wizard
|
||||
12. Audit-Dashboard
|
||||
|
||||
### Phase 5: Kalender & Jitsi
|
||||
13. CalDAV Integration
|
||||
14. Jitsi-Meeting-Einladungen aus Kalender
|
||||
15. Mobile Sync (ActiveSync/CardDAV)
|
||||
|
||||
---
|
||||
|
||||
## 10. Existiert das schon?
|
||||
|
||||
**Kurze Antwort: Nein, nicht in dieser Form.**
|
||||
|
||||
### Ähnliche Konzepte:
|
||||
|
||||
1. **Funktionale Mailboxen** (existiert)
|
||||
- Shared Mailboxes in Exchange/Microsoft 365
|
||||
- Group Addresses in Google Workspace
|
||||
- → Aber: Keine RBAC-Integration, keine Anonymisierung
|
||||
|
||||
2. **DSGVO-Löschung** (existiert)
|
||||
- Standard: Konto komplett löschen
|
||||
- → Aber: Verlust der Geschäftshistorie
|
||||
|
||||
3. **Pseudonymisierung** (existiert als Konzept)
|
||||
- In Forschungsdatenbanken üblich
|
||||
- → Aber: Nicht für E-Mail-Systeme implementiert
|
||||
|
||||
### Was BreakPilot anders macht:
|
||||
|
||||
```
|
||||
Traditionell:
|
||||
max.mustermann@firma.de → Kündigung → Löschen → Historie weg
|
||||
|
||||
BreakPilot:
|
||||
klassenlehrer.5a@schule.bp.app ← Max Mustermann zugeordnet
|
||||
← Maria Müller übernimmt
|
||||
← Historie bleibt, Person anonymisiert
|
||||
```
|
||||
|
||||
**Fazit:** Das Konzept ist innovativ und müsste entwickelt werden. Es gibt keine fertige Open-Source-Lösung, die all diese Features kombiniert.
|
||||
|
||||
---
|
||||
|
||||
## 11. Alternative: Minimale Implementation
|
||||
|
||||
Falls das vollständige System zu komplex ist, gibt es eine minimale Variante:
|
||||
|
||||
### 11.1 Nur Functional Mailboxes
|
||||
|
||||
```sql
|
||||
-- Minimales Schema
|
||||
CREATE TABLE role_email_aliases (
|
||||
id UUID PRIMARY KEY,
|
||||
role_key VARCHAR(100) UNIQUE NOT NULL,
|
||||
email_alias VARCHAR(255) UNIQUE NOT NULL, -- klassenlehrer.5a@...
|
||||
forward_to UUID REFERENCES users(id), -- Aktuelle Person
|
||||
is_active BOOLEAN DEFAULT true
|
||||
);
|
||||
```
|
||||
|
||||
- Keine eigene Mail-Infrastruktur
|
||||
- Aliases werden an persönliche Adressen weitergeleitet
|
||||
- Bei Kündigung: Alias umleiten
|
||||
- Limitation: Keine echte Anonymisierung der Historie
|
||||
|
||||
### 11.2 Mailpit als Development-Only
|
||||
|
||||
Bestehendes Mailpit bleibt für Entwicklung:
|
||||
- Kein Produktions-Mailserver
|
||||
- Externe Mails über Gmail/Microsoft
|
||||
- Functional Mailboxes als Konzept in DB
|
||||
|
||||
---
|
||||
|
||||
## 12. Nächste Schritte
|
||||
|
||||
1. **Entscheidung:** Vollständiges System oder minimale Variante?
|
||||
2. **Proof of Concept:** Functional Mailboxes mit bestehendem RBAC
|
||||
3. **Evaluierung:** Stalwart Mail Server in Testumgebung
|
||||
4. **Architektur-Review:** Mit Datenschutzbeauftragtem abstimmen
|
||||
|
||||
---
|
||||
|
||||
## Anhang A: Referenzen
|
||||
|
||||
- [Stalwart Mail Server](https://stalw.art/)
|
||||
- [SOGo Groupware](https://www.sogo.nu/)
|
||||
- [Roundcube Webmail](https://roundcube.net/)
|
||||
- [CalDAV Standard](https://tools.ietf.org/html/rfc4791)
|
||||
- [DSGVO Art. 17 - Recht auf Löschung](https://dsgvo-gesetz.de/art-17-dsgvo/)
|
||||
|
||||
## Anhang B: Bestehende BreakPilot-Integration
|
||||
|
||||
- RBAC-System: `/backend/rbac_api.py`
|
||||
- E-Mail-Service: `/backend/email_service.py`
|
||||
- Jitsi-Integration: `/backend/jitsi_api.py`
|
||||
- Mailpit-Config: `docker-compose.yml:193-202`
|
||||
|
||||
---
|
||||
|
||||
## Anhang C: Unified Inbox Implementation (2026-01)
|
||||
|
||||
### Implementierte Komponenten
|
||||
|
||||
Die Unified Inbox wurde als Teil des klausur-service implementiert:
|
||||
|
||||
| Komponente | Pfad | Beschreibung |
|
||||
|------------|------|--------------|
|
||||
| **Models** | `klausur-service/backend/mail/models.py` | Pydantic Models für Accounts, E-Mails, Tasks |
|
||||
| **Database** | `klausur-service/backend/mail/mail_db.py` | PostgreSQL-Operationen mit asyncpg |
|
||||
| **Credentials** | `klausur-service/backend/mail/credentials.py` | Vault-Integration für IMAP/SMTP-Passwörter |
|
||||
| **Aggregator** | `klausur-service/backend/mail/aggregator.py` | Multi-Account IMAP Sync |
|
||||
| **AI Service** | `klausur-service/backend/mail/ai_service.py` | KI-Analyse (Absender, Fristen, Kategorien) |
|
||||
| **Task Service** | `klausur-service/backend/mail/task_service.py` | Arbeitsvorrat-Management |
|
||||
| **API** | `klausur-service/backend/mail/api.py` | FastAPI Router mit 30+ Endpoints |
|
||||
|
||||
### Frontend-Komponenten
|
||||
|
||||
| Komponente | Pfad | Beschreibung |
|
||||
|------------|------|--------------|
|
||||
| **Admin UI** | `website/app/admin/mail/page.tsx` | Kontenverwaltung, KI-Einstellungen |
|
||||
| **User Inbox** | `website/app/mail/page.tsx` | Unified Inbox mit KI-Panel |
|
||||
| **Arbeitsvorrat** | `website/app/mail/tasks/page.tsx` | Task-Management Dashboard |
|
||||
|
||||
### API-Endpoints (Port 8086)
|
||||
|
||||
```
|
||||
# Account Management
|
||||
POST /api/v1/mail/accounts - Neues Konto hinzufügen
|
||||
GET /api/v1/mail/accounts - Alle Konten auflisten
|
||||
DELETE /api/v1/mail/accounts/{id} - Konto entfernen
|
||||
POST /api/v1/mail/accounts/{id}/test - Verbindung testen
|
||||
|
||||
# Unified Inbox
|
||||
GET /api/v1/mail/inbox - Aggregierte Inbox
|
||||
GET /api/v1/mail/inbox/{id} - Einzelne E-Mail
|
||||
POST /api/v1/mail/send - E-Mail senden
|
||||
|
||||
# KI-Features
|
||||
POST /api/v1/mail/analyze/{id} - E-Mail analysieren
|
||||
GET /api/v1/mail/suggestions/{id} - Antwortvorschläge
|
||||
|
||||
# Arbeitsvorrat
|
||||
GET /api/v1/mail/tasks - Alle Tasks
|
||||
POST /api/v1/mail/tasks - Manuelle Task erstellen
|
||||
PATCH /api/v1/mail/tasks/{id} - Task aktualisieren
|
||||
GET /api/v1/mail/tasks/dashboard - Dashboard-Statistiken
|
||||
```
|
||||
|
||||
### Niedersachsen-spezifische Absendererkennung
|
||||
|
||||
```python
|
||||
KNOWN_AUTHORITIES_NI = {
|
||||
"@mk.niedersachsen.de": "Kultusministerium Niedersachsen",
|
||||
"@rlsb.de": "Regionales Landesamt für Schule und Bildung",
|
||||
"@landesschulbehoerde-nds.de": "Landesschulbehörde",
|
||||
"@nibis.de": "NiBiS",
|
||||
}
|
||||
```
|
||||
|
||||
### LLM-Playbook für E-Mail-Analyse
|
||||
|
||||
Das Playbook `mail_analysis` in `/backend/llm_gateway/services/playbook_service.py` enthält:
|
||||
- Absender-Klassifikation (12 Typen)
|
||||
- Fristenerkennung mit Datumsextraktion
|
||||
- Kategorisierung (11 Kategorien)
|
||||
- Prioritätsvorschlag
|
||||
277
docs/architecture/secrets-management.md
Normal file
277
docs/architecture/secrets-management.md
Normal file
@@ -0,0 +1,277 @@
|
||||
# BreakPilot Secrets Management
|
||||
|
||||
## Uebersicht
|
||||
|
||||
BreakPilot verwendet **HashiCorp Vault** als zentrales Secrets-Management-System.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ SECRETS MANAGEMENT │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ HashiCorp Vault │ │
|
||||
│ │ Port 8200 │ │
|
||||
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │ │
|
||||
│ │ │ KV v2 Engine │ │ AppRole Auth │ │ Audit Logging │ │ │
|
||||
│ │ │ secret/ │ │ Token Auth │ │ Verschluesselung │ │ │
|
||||
│ │ └──────────────┘ └──────────────┘ └──────────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────────┼─────────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Python Backend │ │ Go Services │ │ Frontend │ │
|
||||
│ │ (hvac client) │ │ (vault-client) │ │ (via Backend) │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Warum Vault?
|
||||
|
||||
| Alternative | Nachteil |
|
||||
|-------------|----------|
|
||||
| Environment Variables | Keine Audit-Logs, keine Verschluesselung, keine Rotation |
|
||||
| Docker Secrets | Nur fuer Docker Swarm, keine zentrale Verwaltung |
|
||||
| AWS Secrets Manager | Cloud Lock-in, Kosten |
|
||||
| Kubernetes Secrets | Keine Verschluesselung by default, nur K8s |
|
||||
| **HashiCorp Vault** | Open Source (BSL 1.1), Self-Hosted, Enterprise Features |
|
||||
|
||||
## Architektur
|
||||
|
||||
### Secret-Hierarchie
|
||||
|
||||
```
|
||||
secret/breakpilot/
|
||||
├── api_keys/
|
||||
│ ├── anthropic # Anthropic Claude API Key
|
||||
│ ├── vast # vast.ai GPU API Key
|
||||
│ ├── stripe # Stripe Payment Key
|
||||
│ ├── stripe_webhook
|
||||
│ └── tavily # Tavily Search API Key
|
||||
├── database/
|
||||
│ ├── postgres # username, password, url
|
||||
│ └── synapse # Matrix Synapse DB
|
||||
├── auth/
|
||||
│ ├── jwt # secret, refresh_secret
|
||||
│ └── keycloak # client_secret
|
||||
├── communication/
|
||||
│ ├── matrix # access_token, db_password
|
||||
│ └── jitsi # app_secret, jicofo, jvb passwords
|
||||
├── storage/
|
||||
│ └── minio # access_key, secret_key
|
||||
└── infra/
|
||||
└── vast # api_key, instance_id, control_key
|
||||
```
|
||||
|
||||
### Python Integration
|
||||
|
||||
```python
|
||||
from secrets import get_secret
|
||||
|
||||
# Einzelnes Secret abrufen
|
||||
api_key = get_secret("ANTHROPIC_API_KEY")
|
||||
|
||||
# Mit Default-Wert
|
||||
debug = get_secret("DEBUG", default="false")
|
||||
|
||||
# Als Pflicht-Secret
|
||||
db_url = get_secret("DATABASE_URL", required=True)
|
||||
```
|
||||
|
||||
### Fallback-Reihenfolge
|
||||
|
||||
```
|
||||
1. HashiCorp Vault (wenn VAULT_ADDR gesetzt)
|
||||
↓ falls nicht verfuegbar
|
||||
2. Environment Variables
|
||||
↓ falls nicht gesetzt
|
||||
3. Docker Secrets (/run/secrets/)
|
||||
↓ falls nicht vorhanden
|
||||
4. Default-Wert (wenn angegeben)
|
||||
↓ sonst
|
||||
5. SecretNotFoundError (wenn required=True)
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
### Entwicklung (Dev Mode)
|
||||
|
||||
```bash
|
||||
# Vault starten (Dev Mode - NICHT fuer Produktion!)
|
||||
docker-compose -f docker-compose.vault.yml up -d vault
|
||||
|
||||
# Warten bis healthy
|
||||
docker-compose -f docker-compose.vault.yml up vault-init
|
||||
|
||||
# Environment setzen
|
||||
export VAULT_ADDR=http://localhost:8200
|
||||
export VAULT_TOKEN=breakpilot-dev-token
|
||||
```
|
||||
|
||||
### Secrets setzen
|
||||
|
||||
```bash
|
||||
# Anthropic API Key
|
||||
vault kv put secret/breakpilot/api_keys/anthropic value='sk-ant-api03-...'
|
||||
|
||||
# vast.ai Credentials
|
||||
vault kv put secret/breakpilot/infra/vast \
|
||||
api_key='xxx' \
|
||||
instance_id='123' \
|
||||
control_key='yyy'
|
||||
|
||||
# Database
|
||||
vault kv put secret/breakpilot/database/postgres \
|
||||
username='breakpilot' \
|
||||
password='supersecret' \
|
||||
url='postgres://breakpilot:supersecret@localhost:5432/breakpilot_db'
|
||||
```
|
||||
|
||||
### Secrets lesen
|
||||
|
||||
```bash
|
||||
# Liste aller Secrets
|
||||
vault kv list secret/breakpilot/
|
||||
|
||||
# Secret anzeigen
|
||||
vault kv get secret/breakpilot/api_keys/anthropic
|
||||
|
||||
# Nur den Wert
|
||||
vault kv get -field=value secret/breakpilot/api_keys/anthropic
|
||||
```
|
||||
|
||||
## Produktion
|
||||
|
||||
### AppRole Authentication
|
||||
|
||||
In Produktion verwenden Services AppRole statt Token-Auth:
|
||||
|
||||
```bash
|
||||
# 1. AppRole aktivieren (einmalig)
|
||||
vault auth enable approle
|
||||
|
||||
# 2. Policy erstellen
|
||||
vault policy write breakpilot-backend - <<EOF
|
||||
path "secret/data/breakpilot/*" {
|
||||
capabilities = ["read", "list"]
|
||||
}
|
||||
EOF
|
||||
|
||||
# 3. Role erstellen
|
||||
vault write auth/approle/role/breakpilot-backend \
|
||||
token_policies="breakpilot-backend" \
|
||||
token_ttl=1h \
|
||||
token_max_ttl=4h
|
||||
|
||||
# 4. Role-ID holen (fix)
|
||||
vault read -field=role_id auth/approle/role/breakpilot-backend/role-id
|
||||
|
||||
# 5. Secret-ID generieren (bei jedem Deploy neu)
|
||||
vault write -f auth/approle/role/breakpilot-backend/secret-id
|
||||
```
|
||||
|
||||
### Environment fuer Services
|
||||
|
||||
```bash
|
||||
# Docker-Compose / Kubernetes
|
||||
VAULT_ADDR=https://vault.breakpilot.app:8200
|
||||
VAULT_AUTH_METHOD=approle
|
||||
VAULT_ROLE_ID=<role-id>
|
||||
VAULT_SECRET_ID=<secret-id>
|
||||
VAULT_SECRETS_PATH=breakpilot
|
||||
```
|
||||
|
||||
## Sicherheits-Checkliste
|
||||
|
||||
### Muss erfuellt sein
|
||||
|
||||
- [ ] Keine echten Secrets in `.env` Dateien
|
||||
- [ ] `.env` in `.gitignore`
|
||||
- [ ] Vault im Sealed-State wenn nicht in Verwendung
|
||||
- [ ] TLS fuer Vault in Produktion
|
||||
- [ ] AppRole statt Token-Auth in Produktion
|
||||
- [ ] Audit-Logging aktiviert
|
||||
- [ ] Minimale Policies (Least Privilege)
|
||||
|
||||
### Sollte erfuellt sein
|
||||
|
||||
- [ ] Automatische Secret-Rotation
|
||||
- [ ] Separate Vault-Instanz fuer Produktion
|
||||
- [ ] HSM-basiertes Auto-Unseal
|
||||
- [ ] Disaster Recovery Plan
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `backend/secrets/__init__.py` | Secrets-Modul Exports |
|
||||
| `backend/secrets/vault_client.py` | Vault Client Implementation |
|
||||
| `docker-compose.vault.yml` | Vault Docker Configuration |
|
||||
| `vault/init-secrets.sh` | Entwicklungs-Secrets Initialisierung |
|
||||
| `vault/policies/` | Vault Policy Files |
|
||||
|
||||
## Fehlerbehebung
|
||||
|
||||
### Vault nicht erreichbar
|
||||
|
||||
```bash
|
||||
# Status pruefen
|
||||
vault status
|
||||
|
||||
# Falls sealed
|
||||
vault operator unseal <unseal-key>
|
||||
```
|
||||
|
||||
### Secret nicht gefunden
|
||||
|
||||
```bash
|
||||
# Pfad pruefen
|
||||
vault kv list secret/breakpilot/
|
||||
|
||||
# Cache leeren (Python)
|
||||
from secrets import get_secrets_manager
|
||||
get_secrets_manager().clear_cache()
|
||||
```
|
||||
|
||||
### Token abgelaufen
|
||||
|
||||
```bash
|
||||
# Neuen Token holen (AppRole)
|
||||
vault write auth/approle/login \
|
||||
role_id=$VAULT_ROLE_ID \
|
||||
secret_id=$VAULT_SECRET_ID
|
||||
```
|
||||
|
||||
## Migration von .env
|
||||
|
||||
Wenn Sie bestehende Secrets in .env haben:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# migrate-secrets.sh
|
||||
|
||||
# Lese .env und schreibe nach Vault
|
||||
while IFS='=' read -r key value; do
|
||||
# Skip Kommentare und leere Zeilen
|
||||
[[ $key =~ ^#.*$ ]] && continue
|
||||
[[ -z "$key" ]] && continue
|
||||
|
||||
# Sensitive Keys nach Vault
|
||||
case $key in
|
||||
*_API_KEY|*_SECRET|*_PASSWORD|*_TOKEN)
|
||||
echo "Migrating $key to Vault..."
|
||||
vault kv put secret/breakpilot/migrated/$key value="$value"
|
||||
;;
|
||||
esac
|
||||
done < .env
|
||||
|
||||
echo "Migration complete. Remember to remove secrets from .env!"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Referenzen
|
||||
|
||||
- [HashiCorp Vault Documentation](https://developer.hashicorp.com/vault/docs)
|
||||
- [hvac Python Client](https://hvac.readthedocs.io/)
|
||||
- [Vault Best Practices](https://developer.hashicorp.com/vault/tutorials/recommended-patterns)
|
||||
399
docs/architecture/system-architecture.md
Normal file
399
docs/architecture/system-architecture.md
Normal file
@@ -0,0 +1,399 @@
|
||||
# BreakPilot PWA - System-Architektur
|
||||
|
||||
## Übersicht
|
||||
|
||||
BreakPilot ist eine modulare Bildungsplattform für Lehrkräfte mit folgenden Hauptkomponenten:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Browser │
|
||||
│ ┌───────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Frontend (Studio UI) │ │
|
||||
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │
|
||||
│ │ │Dashboard │ │Worksheets│ │Correction│ │Letters/Companion │ │ │
|
||||
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────────────┘ │ │
|
||||
│ └───────────────────────────────────────────────────────────────┘ │
|
||||
└───────────────────────────┬─────────────────────────────────────────┘
|
||||
│ HTTP/REST
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Python Backend (FastAPI) │
|
||||
│ Port 8000 │
|
||||
│ ┌────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ API Layer │ │
|
||||
│ │ /api/worksheets /api/corrections /api/letters /api/state │ │
|
||||
│ │ /api/school /api/certificates /api/messenger /api/jitsi │ │
|
||||
│ └────────────────────────────────────────────────────────────────┘ │
|
||||
│ ┌────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Service Layer │ │
|
||||
│ │ FileProcessor │ PDFService │ ContentGenerators │ StateEngine │ │
|
||||
│ └────────────────────────────────────────────────────────────────┘ │
|
||||
└───────────────────────────┬─────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌─────────────────┐ ┌───────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Go Consent │ │ PostgreSQL │ │ LLM Gateway │ │ HashiCorp │
|
||||
│ Service │ │ Database │ │ (optional) │ │ Vault │
|
||||
│ Port 8081 │ │ Port 5432 │ │ │ │ Port 8200 │
|
||||
└─────────────────┘ └───────────────┘ └──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
## Komponenten
|
||||
|
||||
### 1. Admin Frontend (Next.js Website)
|
||||
|
||||
Das **Admin Frontend** ist eine vollständige Next.js 15 Anwendung für Developer und Administratoren:
|
||||
|
||||
**Technologie:** Next.js 15, React 18, TypeScript, Tailwind CSS
|
||||
|
||||
**Container:** `breakpilot-pwa-website` auf **Port 3000**
|
||||
|
||||
**Verzeichnis:** `/website`
|
||||
|
||||
| Modul | Route | Beschreibung |
|
||||
|-------|-------|--------------|
|
||||
| Dashboard | `/admin` | Übersicht & Statistiken |
|
||||
| GPU Infrastruktur | `/admin/gpu` | vast.ai GPU Management |
|
||||
| Consent Verwaltung | `/admin/consent` | Rechtliche Dokumente & Versionen |
|
||||
| Datenschutzanfragen | `/admin/dsr` | DSGVO Art. 15-21 Anfragen |
|
||||
| DSMS | `/admin/dsms` | Datenschutz-Management-System |
|
||||
| Education Search | `/admin/edu-search` | Bildungsquellen & Crawler |
|
||||
| Personensuche | `/admin/staff-search` | Uni-Mitarbeiter & Publikationen |
|
||||
| Uni-Crawler | `/admin/uni-crawler` | Universitäts-Crawling Orchestrator |
|
||||
| LLM Vergleich | `/admin/llm-compare` | KI-Provider Vergleich |
|
||||
| PCA Platform | `/admin/pca-platform` | Bot-Erkennung & Monetarisierung |
|
||||
| Production Backlog | `/admin/backlog` | Go-Live Checkliste |
|
||||
| Developer Docs | `/admin/docs` | API & Architektur Dokumentation |
|
||||
| Kommunikation | `/admin/communication` | Matrix & Jitsi Monitoring |
|
||||
| **Security** | `/admin/security` | DevSecOps Dashboard, Scans, Findings |
|
||||
| **SBOM** | `/admin/sbom` | Software Bill of Materials |
|
||||
|
||||
### 2. Lehrer Frontend (Studio UI)
|
||||
|
||||
Das **Lehrer Frontend** ist ein Single-Page-Application-ähnliches System für Lehrkräfte, das in Python-Modulen organisiert ist:
|
||||
|
||||
| Modul | Datei | Beschreibung |
|
||||
|-------|-------|--------------|
|
||||
| Base | `frontend/modules/base.py` | TopBar, Sidebar, Theme, Login |
|
||||
| Dashboard | `frontend/modules/dashboard.py` | Übersichtsseite |
|
||||
| Worksheets | `frontend/modules/worksheets.py` | Lerneinheiten-Generator |
|
||||
| Correction | `frontend/modules/correction.py` | OCR-Klausurkorrektur |
|
||||
| Letters | `frontend/modules/letters.py` | Elternkommunikation |
|
||||
| Companion | `frontend/modules/companion.py` | Begleiter-Modus mit State Engine |
|
||||
| School | `frontend/modules/school.py` | Schulverwaltung |
|
||||
| Gradebook | `frontend/modules/gradebook.py` | Notenbuch |
|
||||
| ContentCreator | `frontend/modules/content_creator.py` | H5P Content Creator |
|
||||
| ContentFeed | `frontend/modules/content_feed.py` | Content Discovery |
|
||||
| Messenger | `frontend/modules/messenger.py` | Matrix Messenger |
|
||||
| Jitsi | `frontend/modules/jitsi.py` | Videokonferenzen |
|
||||
| **KlausurKorrektur** | `frontend/modules/klausur_korrektur.py` | **Abitur-Klausurkorrektur (15-Punkte-System)** |
|
||||
| **AbiturDocsAdmin** | `frontend/modules/abitur_docs_admin.py` | **Admin für Abitur-Dokumente (NiBiS)** |
|
||||
|
||||
Jedes Modul exportiert:
|
||||
- `get_css()` - CSS-Styles
|
||||
- `get_html()` - HTML-Template
|
||||
- `get_js()` - JavaScript-Logik
|
||||
|
||||
### 2. Python Backend (FastAPI)
|
||||
|
||||
#### API-Router
|
||||
|
||||
| Router | Präfix | Beschreibung |
|
||||
|--------|--------|--------------|
|
||||
| `worksheets_api` | `/api/worksheets` | Content-Generatoren (MC, Cloze, Mindmap, Quiz) |
|
||||
| `correction_api` | `/api/corrections` | OCR-Pipeline für Klausurkorrektur |
|
||||
| `letters_api` | `/api/letters` | Elternbriefe mit GFK-Integration |
|
||||
| `state_engine_api` | `/api/state` | Begleiter-Modus Phasen & Vorschläge |
|
||||
| `school_api` | `/api/school` | Schulverwaltung (Proxy zu school-service) |
|
||||
| `certificates_api` | `/api/certificates` | Zeugniserstellung |
|
||||
| `messenger_api` | `/api/messenger` | Matrix Messenger Integration |
|
||||
| `jitsi_api` | `/api/jitsi` | Jitsi Meeting-Einladungen |
|
||||
| `consent_api` | `/api/consent` | DSGVO Consent-Verwaltung |
|
||||
| `gdpr_api` | `/api/gdpr` | GDPR-Export |
|
||||
| **`klausur_korrektur_api`** | `/api/klausur-korrektur` | **Abitur-Klausuren (15-Punkte, Gutachten, Fairness)** |
|
||||
| **`abitur_docs_api`** | `/api/abitur-docs` | **NiBiS-Dokumentenverwaltung für RAG** |
|
||||
|
||||
#### Services
|
||||
|
||||
| Service | Datei | Beschreibung |
|
||||
|---------|-------|--------------|
|
||||
| FileProcessor | `services/file_processor.py` | OCR mit PaddleOCR |
|
||||
| PDFService | `services/pdf_service.py` | PDF-Generierung |
|
||||
| ContentGenerators | `services/content_generators/` | MC, Cloze, Mindmap, Quiz |
|
||||
| StateEngine | `state_engine/` | Phasen-Management & Antizipation |
|
||||
|
||||
### 3. Klausur-Korrektur System (Abitur)
|
||||
|
||||
Das Klausur-Korrektur-System implementiert die vollständige Abitur-Bewertungspipeline:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Klausur-Korrektur Modul │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌──────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Modus-Wahl │───►│ Text-Quellen & │───►│ Erwartungs- │ │
|
||||
│ │ LandesAbi/ │ │ Rights-Gate │ │ horizont │ │
|
||||
│ │ Vorabitur │ └──────────────────┘ └─────────────────┘ │
|
||||
│ └─────────────┘ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Schülerarbeiten-Pipeline │ │
|
||||
│ │ Upload → OCR → KI-Bewertung → Gutachten → 15-Punkte-Note │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────┐ ┌──────────────────────────────────┐ │
|
||||
│ │ Erst-/Zweitprüfer │───►│ Fairness-Analyse & PDF-Export │ │
|
||||
│ └────────────────────┘ └──────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 15-Punkte-Notensystem
|
||||
|
||||
Das System verwendet den deutschen Abitur-Notenschlüssel:
|
||||
|
||||
| Punkte | Prozent | Note |
|
||||
|--------|---------|------|
|
||||
| 15-13 | 95-85% | 1+/1/1- |
|
||||
| 12-10 | 80-70% | 2+/2/2- |
|
||||
| 9-7 | 65-55% | 3+/3/3- |
|
||||
| 6-4 | 50-40% | 4+/4/4- |
|
||||
| 3-1 | 33-20% | 5+/5/5- |
|
||||
| 0 | <20% | 6 |
|
||||
|
||||
#### Bewertungskriterien
|
||||
|
||||
| Kriterium | Gewicht | Beschreibung |
|
||||
|-----------|---------|--------------|
|
||||
| Rechtschreibung | 15% | Orthografie |
|
||||
| Grammatik | 15% | Grammatik & Syntax |
|
||||
| **Inhalt** | **40%** | Inhaltliche Qualität (höchste Gewichtung) |
|
||||
| Struktur | 15% | Aufbau & Gliederung |
|
||||
| Stil | 15% | Ausdruck & Stil |
|
||||
|
||||
#### Abitur-Docs RAG-System
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Abitur-Docs Admin │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌──────────────────┐ ┌─────────────────┐ │
|
||||
│ │ ZIP-Import │───►│ KI-Datei- │───►│ Metadaten- │ │
|
||||
│ │ NiBiS PDFs │ │ erkennung │ │ Bestätigung │ │
|
||||
│ └─────────────┘ └──────────────────┘ └─────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Vector Store (ChromaDB) │ │
|
||||
│ │ Indexierung für RAG-Suche nach Fach, Jahr, Niveau │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
NiBiS-Dateinamen werden automatisch erkannt:
|
||||
- `2025_Deutsch_eA_I.pdf` → Deutsch, erhöhtes Niveau, Aufgabe I
|
||||
- `2025_Deutsch_eA_I_EWH.pdf` → Erwartungshorizont
|
||||
|
||||
### 4. State Engine (Begleiter-Modus)
|
||||
|
||||
Das State Engine Modul implementiert proaktive Unterstützung für Lehrkräfte:
|
||||
|
||||
```
|
||||
state_engine/
|
||||
├── __init__.py # Exports
|
||||
├── models.py # SchoolYearPhase, TeacherContext, Suggestion
|
||||
├── rules.py # 15+ Antizipations-Regeln
|
||||
└── engine.py # AnticipationEngine, PhaseService
|
||||
```
|
||||
|
||||
#### Schuljahr-Phasen
|
||||
|
||||
1. `ONBOARDING` - Ersteinrichtung
|
||||
2. `SCHOOL_YEAR_START` - Schuljahresbeginn
|
||||
3. `TEACHING_SETUP` - Unterrichtsvorbereitung
|
||||
4. `PERFORMANCE_1` - 1. Leistungsphase
|
||||
5. `SEMESTER_END` - Halbjahreszeugnisse
|
||||
6. `PERFORMANCE_2` - 2. Leistungsphase
|
||||
7. `EXAM_PHASE` - Prüfungsphase
|
||||
8. `YEAR_END` - Schuljahresende
|
||||
9. `ARCHIVED` - Archiviert
|
||||
|
||||
#### Antizipations-Regeln
|
||||
|
||||
Regeln evaluieren den `TeacherContext` und generieren `Suggestion`-Objekte:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class Rule:
|
||||
id: str
|
||||
name: str
|
||||
phases: List[SchoolYearPhase]
|
||||
condition: Callable[[TeacherContext], bool]
|
||||
suggestion_builder: Callable[[TeacherContext], Suggestion]
|
||||
```
|
||||
|
||||
### 4. Go Consent Service
|
||||
|
||||
Verwaltet DSGVO-Einwilligungen:
|
||||
|
||||
```
|
||||
consent-service/
|
||||
├── cmd/server/ # Main entry point
|
||||
├── internal/
|
||||
│ ├── handlers/ # HTTP Handler
|
||||
│ ├── services/ # Business Logic
|
||||
│ ├── models/ # Data Models
|
||||
│ └── middleware/ # Auth Middleware
|
||||
└── migrations/ # SQL Migrations
|
||||
```
|
||||
|
||||
### 5. LLM Gateway (Optional)
|
||||
|
||||
Wenn `LLM_GATEWAY_ENABLED=true`:
|
||||
|
||||
```
|
||||
llm_gateway/
|
||||
├── routes/
|
||||
│ ├── chat.py # Chat-Completion API
|
||||
│ ├── communication.py # GFK-Validierung
|
||||
│ ├── edu_search_seeds.py # Bildungssuche
|
||||
│ └── legal_crawler.py # Schulgesetz-Crawler
|
||||
└── services/
|
||||
└── communication_service.py
|
||||
```
|
||||
|
||||
## Datenfluss
|
||||
|
||||
### Worksheet-Generierung
|
||||
|
||||
```
|
||||
User Input → Frontend (worksheets.py)
|
||||
↓
|
||||
POST /api/worksheets/generate/multiple-choice
|
||||
↓
|
||||
worksheets_api.py → MCGenerator (services/content_generators/)
|
||||
↓
|
||||
Optional: LLM für erweiterte Generierung
|
||||
↓
|
||||
Response: WorksheetContent → Frontend rendert Ergebnis
|
||||
```
|
||||
|
||||
### Klausurkorrektur
|
||||
|
||||
```
|
||||
File Upload → Frontend (correction.py)
|
||||
↓
|
||||
POST /api/corrections/ (erstellen)
|
||||
POST /api/corrections/{id}/upload (Datei)
|
||||
↓
|
||||
Background Task: OCR via FileProcessor
|
||||
↓
|
||||
Poll GET /api/corrections/{id} bis status="ocr_complete"
|
||||
↓
|
||||
POST /api/corrections/{id}/analyze
|
||||
↓
|
||||
Review Interface → PUT /api/corrections/{id} (Anpassungen)
|
||||
↓
|
||||
GET /api/corrections/{id}/export-pdf
|
||||
```
|
||||
|
||||
### State Engine Flow
|
||||
|
||||
```
|
||||
Frontend Companion-Mode aktiviert
|
||||
↓
|
||||
GET /api/state/dashboard?teacher_id=xxx
|
||||
↓
|
||||
AnticipationEngine.get_suggestions(context)
|
||||
↓
|
||||
Rules evaluieren TeacherContext
|
||||
↓
|
||||
Priorisierte Vorschläge (max 5) zurück
|
||||
↓
|
||||
User führt Aktion aus
|
||||
↓
|
||||
POST /api/state/milestone (Meilenstein abschließen)
|
||||
↓
|
||||
PhaseService.check_and_transition() prüft Phasenübergang
|
||||
```
|
||||
|
||||
## Sicherheit
|
||||
|
||||
### Authentifizierung & Autorisierung
|
||||
|
||||
BreakPilot verwendet einen **Hybrid-Ansatz**:
|
||||
|
||||
| Schicht | Komponente | Beschreibung |
|
||||
|---------|------------|--------------|
|
||||
| **Authentifizierung** | Keycloak (Prod) / Lokales JWT (Dev) | Token-Validierung via JWKS oder HS256 |
|
||||
| **Autorisierung** | rbac.py (Eigenentwicklung) | Domaenenspezifische Berechtigungen |
|
||||
|
||||
Siehe: [docs/architecture/auth-system.md](auth-system.md)
|
||||
|
||||
### Basis-Rollen
|
||||
|
||||
| Rolle | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `user` | Normaler Benutzer |
|
||||
| `teacher` / `lehrer` | Lehrkraft |
|
||||
| `admin` | Administrator |
|
||||
| `data_protection_officer` | Datenschutzbeauftragter |
|
||||
|
||||
### Erweiterte Rollen (rbac.py)
|
||||
|
||||
15+ domaenenspezifische Rollen fuer Klausurkorrektur und Zeugnisse:
|
||||
- `erstkorrektor`, `zweitkorrektor`, `drittkorrektor`
|
||||
- `klassenlehrer`, `fachlehrer`, `fachvorsitz`
|
||||
- `schulleitung`, `zeugnisbeauftragter`, `sekretariat`
|
||||
|
||||
### Sicherheitsfeatures
|
||||
|
||||
- JWT-basierte Authentifizierung (RS256/HS256)
|
||||
- CORS konfiguriert für Frontend-Zugriff
|
||||
- DSGVO-konformes Consent-Management
|
||||
- **HashiCorp Vault** fuer Secrets-Management (keine hardcodierten Secrets)
|
||||
- Bundesland-spezifische Policy-Sets
|
||||
- **DevSecOps Pipeline** mit automatisierten Security-Scans (SAST, SCA, Secrets Detection)
|
||||
|
||||
Siehe:
|
||||
- [docs/architecture/secrets-management.md](secrets-management.md)
|
||||
- [docs/architecture/devsecops.md](devsecops.md)
|
||||
|
||||
## Deployment
|
||||
|
||||
```yaml
|
||||
services:
|
||||
backend:
|
||||
build: ./backend
|
||||
ports: ["8000:8000"]
|
||||
environment:
|
||||
- DATABASE_URL=postgresql://...
|
||||
- LLM_GATEWAY_ENABLED=false
|
||||
|
||||
consent-service:
|
||||
build: ./consent-service
|
||||
ports: ["8081:8081"]
|
||||
|
||||
postgres:
|
||||
image: postgres:15
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
```
|
||||
|
||||
## Erweiterung
|
||||
|
||||
Neues Frontend-Modul hinzufügen:
|
||||
|
||||
1. Modul erstellen: `frontend/modules/new_module.py`
|
||||
2. Klasse mit `get_css()`, `get_html()`, `get_js()` implementieren
|
||||
3. In `frontend/modules/__init__.py` importieren und exportieren
|
||||
4. Optional: Zugehörige API in `new_module_api.py` erstellen
|
||||
5. In `main.py` Router registrieren
|
||||
|
||||
Neue State Engine Regel:
|
||||
|
||||
1. In `state_engine/rules.py` neue `Rule` definieren
|
||||
2. `condition` und `suggestion_builder` Funktionen implementieren
|
||||
3. Zur `RULES` Liste hinzufügen
|
||||
4. Passende `phases` angeben
|
||||
411
docs/architecture/zeugnis-system.md
Normal file
411
docs/architecture/zeugnis-system.md
Normal file
@@ -0,0 +1,411 @@
|
||||
# Zeugnis-System - Architecture Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
The Zeugnis (Certificate) System enables schools to generate official school certificates with grades, attendance data, and remarks. It extends the existing School-Service with comprehensive grade management and certificate generation workflows.
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Python Backend (Port 8000) │
|
||||
│ backend/frontend/modules/school.py │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────┐ │
|
||||
│ │ panel-school-certificates │ │
|
||||
│ │ - Klassenauswahl │ │
|
||||
│ │ - Notenspiegel │ │
|
||||
│ │ - Zeugnis-Wizard (5 Steps) │ │
|
||||
│ │ - Workflow-Status │ │
|
||||
│ └─────────────────────────────────┘ │
|
||||
└──────────────────┬──────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ School-Service (Go, Port 8084) │
|
||||
├─────────────────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────────────────┐ │
|
||||
│ │ Grade Handlers │ │ Statistics Handlers │ │ Certificate Handlers │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ GetClassGrades │ │ GetClassStatistics │ │ GetCertificateTemplates │ │
|
||||
│ │ GetStudentGrades │ │ GetSubjectStatistics│ │ GetClassCertificates │ │
|
||||
│ │ UpdateOralGrade │ │ GetStudentStatistics│ │ GenerateCertificate │ │
|
||||
│ │ CalculateFinalGrades│ │ GetNotenspiegel │ │ BulkGenerateCertificates │ │
|
||||
│ │ TransferApprovedGrades│ │ │ │ FinalizeCertificate │ │
|
||||
│ │ LockFinalGrade │ │ │ │ GetCertificatePDF │ │
|
||||
│ │ UpdateGradeWeights │ │ │ │ │ │
|
||||
│ └─────────────────────┘ └─────────────────────┘ └─────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────────────────────┐│
|
||||
│ │ Grade Service Layer ││
|
||||
│ │ ││
|
||||
│ │ - ClassStatistics struct (average, pass_rate, at_risk_count, grade_distribution) ││
|
||||
│ │ - SubjectStatistics struct (class average, best/worst, exam results) ││
|
||||
│ │ - StudentStatistics struct (overall average, subject performance) ││
|
||||
│ │ - Notenspiegel struct (grade 1-6 distribution) ││
|
||||
│ └─────────────────────────────────────────────────────────────────────────────────────┘│
|
||||
└─────────────────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ PostgreSQL Database │
|
||||
│ │
|
||||
│ Tables: │
|
||||
│ - grade_overview │
|
||||
│ - exam_results │
|
||||
│ - students │
|
||||
│ - classes │
|
||||
│ - subjects │
|
||||
│ - certificates │
|
||||
│ - attendance │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Zeugnis Workflow (Role Chain)
|
||||
|
||||
The certificate workflow follows a strict approval chain from subject teachers to school principal:
|
||||
|
||||
```
|
||||
┌──────────────────┐ ┌──────────────────┐ ┌────────────────────────┐ ┌────────────────────┐ ┌──────────────────┐
|
||||
│ FACHLEHRER │───▶│ KLASSENLEHRER │───▶│ ZEUGNISBEAUFTRAGTER │───▶│ SCHULLEITUNG │───▶│ SEKRETARIAT │
|
||||
│ (Subject │ │ (Class │ │ (Certificate │ │ (Principal) │ │ (Secretary) │
|
||||
│ Teacher) │ │ Teacher) │ │ Coordinator) │ │ │ │ │
|
||||
└──────────────────┘ └──────────────────┘ └────────────────────────┘ └────────────────────┘ └──────────────────┘
|
||||
│ │ │ │ │
|
||||
▼ ▼ ▼ ▼ ▼
|
||||
Grades Entry Approve Quality Check Sign-off & Lock Print & Archive
|
||||
(Oral/Written) Grades & Review
|
||||
```
|
||||
|
||||
### Workflow States
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ DRAFT │────▶│ SUBMITTED │────▶│ REVIEWED │────▶│ SIGNED │────▶│ PRINTED │
|
||||
│ (Entwurf) │ │ (Eingereicht)│ │ (Geprueft) │ │(Unterzeichnet) │ (Gedruckt) │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
|
||||
│ │ │ │
|
||||
▼ ▼ ▼ ▼
|
||||
Fachlehrer Klassenlehrer Zeugnisbeauftragter Schulleitung
|
||||
```
|
||||
|
||||
## RBAC Integration
|
||||
|
||||
### Certificate-Related Roles
|
||||
|
||||
| Role | German | Description |
|
||||
|------|--------|-------------|
|
||||
| `FACHLEHRER` | Fachlehrer | Subject teacher - enters grades |
|
||||
| `KLASSENLEHRER` | Klassenlehrer | Class teacher - approves class grades |
|
||||
| `ZEUGNISBEAUFTRAGTER` | Zeugnisbeauftragter | Certificate coordinator - quality control |
|
||||
| `SCHULLEITUNG` | Schulleitung | Principal - final sign-off |
|
||||
| `SEKRETARIAT` | Sekretariat | Secretary - printing & archiving |
|
||||
|
||||
### Certificate Resource Types
|
||||
|
||||
| ResourceType | Description |
|
||||
|--------------|-------------|
|
||||
| `ZEUGNIS` | Final certificate document |
|
||||
| `ZEUGNIS_VORLAGE` | Certificate template (per Bundesland) |
|
||||
| `ZEUGNIS_ENTWURF` | Draft certificate (before approval) |
|
||||
| `FACHNOTE` | Subject grade |
|
||||
| `KOPFNOTE` | Head grade (Arbeits-/Sozialverhalten) |
|
||||
| `BEMERKUNG` | Certificate remarks |
|
||||
| `STATISTIK` | Class/subject statistics |
|
||||
| `NOTENSPIEGEL` | Grade distribution chart |
|
||||
|
||||
### Permission Matrix
|
||||
|
||||
| Role | ZEUGNIS | ZEUGNIS_ENTWURF | ZEUGNIS_VORLAGE | FACHNOTE | Actions |
|
||||
|------|---------|-----------------|-----------------|----------|---------|
|
||||
| FACHLEHRER | R | CRUD | R | CRUD | Create/Update grades |
|
||||
| KLASSENLEHRER | CRU | CRUD | R | CRUD | Approve class grades |
|
||||
| ZEUGNISBEAUFTRAGTER | RU | RU | RUU | RU | Quality review |
|
||||
| SCHULLEITUNG | R/SIGN/LOCK | RU | RU | R | Final approval |
|
||||
| SEKRETARIAT | RD | R | R | R | Print & archive |
|
||||
|
||||
Legend: C=Create, R=Read, U=Update, D=Delete, SIGN=Sign-off, LOCK=Lock final
|
||||
|
||||
### VerfahrenType for Certificates
|
||||
|
||||
```python
|
||||
class VerfahrenType(str, Enum):
|
||||
# Exam types
|
||||
ABITUR = "abitur"
|
||||
KLAUSUR = "klausur"
|
||||
...
|
||||
|
||||
# Certificate types
|
||||
HALBJAHRESZEUGNIS = "halbjahreszeugnis" # Mid-year certificate
|
||||
JAHRESZEUGNIS = "jahreszeugnis" # End-of-year certificate
|
||||
ABSCHLUSSZEUGNIS = "abschlusszeugnis" # Graduation certificate
|
||||
ABGANGSZEUGNIS = "abgangszeugnis" # Leaving certificate
|
||||
```
|
||||
|
||||
## Statistics API
|
||||
|
||||
### Endpoints
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/v1/school/statistics/:classId` | Class statistics |
|
||||
| GET | `/api/v1/school/statistics/:classId/subject/:subjectId` | Subject statistics |
|
||||
| GET | `/api/v1/school/statistics/student/:studentId` | Student statistics |
|
||||
| GET | `/api/v1/school/statistics/:classId/notenspiegel` | Grade distribution |
|
||||
|
||||
### Response Structures
|
||||
|
||||
**ClassStatistics:**
|
||||
```json
|
||||
{
|
||||
"class_id": "uuid",
|
||||
"class_name": "5a",
|
||||
"semester": 1,
|
||||
"average": 2.4,
|
||||
"pass_rate": 92.8,
|
||||
"at_risk_count": 2,
|
||||
"student_count": 25,
|
||||
"subjects": [
|
||||
{
|
||||
"subject_id": "uuid",
|
||||
"subject_name": "Mathematik",
|
||||
"average": 2.8
|
||||
}
|
||||
],
|
||||
"grade_distribution": {
|
||||
"1": 3,
|
||||
"2": 8,
|
||||
"3": 10,
|
||||
"4": 5,
|
||||
"5": 2,
|
||||
"6": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notenspiegel:**
|
||||
```json
|
||||
{
|
||||
"class_id": "uuid",
|
||||
"subject_id": "uuid",
|
||||
"exam_id": "uuid",
|
||||
"distribution": {
|
||||
"1": 2,
|
||||
"2": 5,
|
||||
"3": 8,
|
||||
"4": 6,
|
||||
"5": 3,
|
||||
"6": 1
|
||||
},
|
||||
"total_students": 25,
|
||||
"average": 3.1,
|
||||
"median": 3.0
|
||||
}
|
||||
```
|
||||
|
||||
## Frontend Components
|
||||
|
||||
### Certificate Panel (`panel-school-certificates`)
|
||||
|
||||
Located in: `backend/frontend/modules/school.py`
|
||||
|
||||
Features:
|
||||
- **Statistik-Karten**: Overview cards showing class average, pass rate, at-risk students
|
||||
- **Notenspiegel Chart**: Bar chart visualization of grade distribution (1-6)
|
||||
- **Workflow-Status**: Visual display of certificate approval chain
|
||||
- **Student Table**: List of students with grades, attendance, certificate status
|
||||
|
||||
### Zeugnis-Wizard (5 Steps)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Zeugnis-Wizard Modal │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Step 1: Klasse auswaehlen │
|
||||
│ ├── Dropdown: Klasse (5a, 5b, 6a, ...) │
|
||||
│ ├── Dropdown: Halbjahr (1 oder 2) │
|
||||
│ └── Vorschau: Anzahl Schueler, Durchschnitt │
|
||||
│ │
|
||||
│ Step 2: Noten pruefen │
|
||||
│ ├── Tabelle: Schueler | Fach1 | Fach2 | ... | Status │
|
||||
│ ├── Fehlende Noten markiert │
|
||||
│ └── Edit-Moeglichkeit direkt in Tabelle │
|
||||
│ │
|
||||
│ Step 3: Vorlage waehlen │
|
||||
│ ├── Dropdown: Bundesland (Niedersachsen, Bayern, ...) │
|
||||
│ ├── Dropdown: Schulform (Gymnasium, Realschule, ...) │
|
||||
│ └── Vorschau: Template-Preview │
|
||||
│ │
|
||||
│ Step 4: Bemerkungen │
|
||||
│ ├── Textarea pro Schueler │
|
||||
│ ├── Textbausteine / Vorlagen │
|
||||
│ └── KI-generierte Vorschlaege (optional) │
|
||||
│ │
|
||||
│ Step 5: Zusammenfassung & Generieren │
|
||||
│ ├── Uebersicht aller Einstellungen │
|
||||
│ ├── Checkbox: Alle Zeugnisse generieren │
|
||||
│ └── Button: Zeugnisse erstellen (mit Fortschrittsbalken) │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Seed Data Generator
|
||||
|
||||
Located in: `school-service/internal/seed/`
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Run seed data generation
|
||||
go run cmd/seed/main.go -teacher-id=<teacher-uuid>
|
||||
|
||||
# Or with default teacher ID
|
||||
go run cmd/seed/main.go
|
||||
```
|
||||
|
||||
### Generated Data
|
||||
|
||||
| Entity | Count | Details |
|
||||
|--------|-------|---------|
|
||||
| School Years | 3 | 2022/23, 2023/24, 2024/25 |
|
||||
| Classes per Year | 6 | 5a, 5b, 6a, 6b, 7a, 7b |
|
||||
| Students per Class | 15-25 | Realistic German names |
|
||||
| Subjects | 10 | Deutsch, Mathe, Englisch, ... |
|
||||
| Exams per Class/Subject | 3-5 | With grade distribution |
|
||||
| Attendance Records | Variable | 0-10 absence days |
|
||||
|
||||
### German Name Generator
|
||||
|
||||
```go
|
||||
var firstNames = []string{
|
||||
"Anna", "Emma", "Mia", "Sophie", "Marie", "Lena", "Lea", "Emily",
|
||||
"Ben", "Paul", "Leon", "Luis", "Max", "Felix", "Noah", "Elias", ...
|
||||
}
|
||||
|
||||
var lastNames = []string{
|
||||
"Mueller", "Schmidt", "Schneider", "Fischer", "Weber", "Meyer",
|
||||
"Wagner", "Becker", "Schulz", "Hoffmann", "Koch", "Richter", ...
|
||||
}
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
school-service/
|
||||
├── cmd/
|
||||
│ ├── server/
|
||||
│ │ └── main.go # Main server with all routes
|
||||
│ └── seed/
|
||||
│ └── main.go # Seed data CLI
|
||||
├── internal/
|
||||
│ ├── config/
|
||||
│ │ └── config.go
|
||||
│ ├── database/
|
||||
│ │ └── database.go
|
||||
│ ├── handlers/
|
||||
│ │ ├── handlers.go # Base handler
|
||||
│ │ ├── grade_handlers.go # Grade + Statistics endpoints
|
||||
│ │ └── certificate_handlers.go
|
||||
│ ├── services/
|
||||
│ │ └── grade_service.go # Statistics calculations
|
||||
│ ├── models/
|
||||
│ │ └── models.go
|
||||
│ └── seed/
|
||||
│ └── seed_data.go # Fake data generator
|
||||
└── go.mod
|
||||
|
||||
backend/frontend/modules/
|
||||
└── school.py # Zeugnis-UI (panel-school-certificates)
|
||||
|
||||
klausur-service/backend/
|
||||
├── rbac.py # RBAC with certificate roles
|
||||
└── tests/
|
||||
└── test_rbac.py # RBAC unit tests
|
||||
```
|
||||
|
||||
## API Routes (School-Service)
|
||||
|
||||
### Grade Management
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/v1/school/grades/:classId` | Get class grades |
|
||||
| GET | `/api/v1/school/grades/student/:studentId` | Get student grades |
|
||||
| PUT | `/api/v1/school/grades/:studentId/:subjectId/oral` | Update oral grade |
|
||||
| POST | `/api/v1/school/grades/calculate` | Calculate final grades |
|
||||
| POST | `/api/v1/school/grades/transfer` | Transfer approved grades |
|
||||
| PUT | `/api/v1/school/grades/:studentId/:subjectId/lock` | Lock final grade |
|
||||
| PUT | `/api/v1/school/grades/:studentId/:subjectId/weights` | Update grade weights |
|
||||
|
||||
### Statistics
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/v1/school/statistics/:classId` | Class statistics |
|
||||
| GET | `/api/v1/school/statistics/:classId/subject/:subjectId` | Subject statistics |
|
||||
| GET | `/api/v1/school/statistics/student/:studentId` | Student statistics |
|
||||
| GET | `/api/v1/school/statistics/:classId/notenspiegel` | Grade distribution |
|
||||
|
||||
### Certificates
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/api/v1/school/certificates/templates` | List templates |
|
||||
| GET | `/api/v1/school/certificates/class/:classId` | Class certificates |
|
||||
| POST | `/api/v1/school/certificates/generate` | Generate single |
|
||||
| POST | `/api/v1/school/certificates/generate-bulk` | Generate bulk |
|
||||
| GET | `/api/v1/school/certificates/detail/:id` | Get certificate |
|
||||
| PUT | `/api/v1/school/certificates/detail/:id` | Update certificate |
|
||||
| PUT | `/api/v1/school/certificates/detail/:id/finalize` | Finalize |
|
||||
| GET | `/api/v1/school/certificates/detail/:id/pdf` | Download PDF |
|
||||
| DELETE | `/api/v1/school/certificates/detail/:id` | Delete |
|
||||
| GET | `/api/v1/school/certificates/feedback/:studentId` | AI feedback |
|
||||
|
||||
## German Grading System
|
||||
|
||||
| Grade | Meaning | Points |
|
||||
|-------|---------|--------|
|
||||
| 1 | sehr gut (excellent) | 15-13 |
|
||||
| 2 | gut (good) | 12-10 |
|
||||
| 3 | befriedigend (satisfactory) | 9-7 |
|
||||
| 4 | ausreichend (adequate) | 6-4 |
|
||||
| 5 | mangelhaft (poor) | 3-1 |
|
||||
| 6 | ungenuegend (inadequate) | 0 |
|
||||
|
||||
### Grade Calculation
|
||||
|
||||
```
|
||||
Final Grade = (Written Weight * Written Avg) + (Oral Weight * Oral Avg)
|
||||
|
||||
Default weights:
|
||||
- Written (Klassenarbeiten): 50%
|
||||
- Oral (muendliche Note): 50%
|
||||
|
||||
Customizable per subject/student via UpdateGradeWeights endpoint.
|
||||
```
|
||||
|
||||
## Integration with BYOEH
|
||||
|
||||
The Zeugnis system can optionally integrate with BYOEH for:
|
||||
- AI-generated certificate remarks
|
||||
- Template suggestion based on student performance
|
||||
- Feedback text generation (`/certificates/feedback/:studentId`)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **RBAC Enforcement**: All certificate operations check user role permissions
|
||||
2. **Tenant Isolation**: Teachers only see their own classes/students
|
||||
3. **Audit Trail**: All grade changes and approvals logged
|
||||
4. **Lock Mechanism**: Finalized certificates cannot be modified
|
||||
5. **Workflow Enforcement**: Cannot skip approval steps
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] PDF template editor (LaTeX/HTML)
|
||||
- [ ] Bundesland-specific templates (all 16 states)
|
||||
- [ ] Kopfnoten (head grades) support
|
||||
- [ ] Digital signatures for certificates
|
||||
- [ ] Parent portal access to certificates
|
||||
- [ ] Archive/retention policies
|
||||
1081
docs/ci-cd/TEST-PIPELINE-DEVELOPER-GUIDE.md
Normal file
1081
docs/ci-cd/TEST-PIPELINE-DEVELOPER-GUIDE.md
Normal file
File diff suppressed because it is too large
Load Diff
1737
docs/consent-banner/SPECIFICATION.md
Normal file
1737
docs/consent-banner/SPECIFICATION.md
Normal file
File diff suppressed because it is too large
Load Diff
258
docs/guides/environment-setup.md
Normal file
258
docs/guides/environment-setup.md
Normal file
@@ -0,0 +1,258 @@
|
||||
# Entwickler-Guide: Umgebungs-Setup
|
||||
|
||||
Dieser Guide erklärt das tägliche Arbeiten mit den Dev/Staging/Prod-Umgebungen.
|
||||
|
||||
## Schnellstart
|
||||
|
||||
```bash
|
||||
# 1. Wechsle in das Projektverzeichnis
|
||||
cd /Users/benjaminadmin/Projekte/breakpilot-pwa
|
||||
|
||||
# 2. Starte die Entwicklungsumgebung
|
||||
./scripts/start.sh dev
|
||||
|
||||
# 3. Prüfe den Status
|
||||
./scripts/status.sh
|
||||
```
|
||||
|
||||
## Täglicher Workflow
|
||||
|
||||
### Morgens: Entwicklung starten
|
||||
|
||||
```bash
|
||||
# Auf develop-Branch wechseln
|
||||
git checkout develop
|
||||
|
||||
# Neueste Änderungen holen (falls Remote konfiguriert)
|
||||
git pull origin develop
|
||||
|
||||
# Umgebung starten
|
||||
./scripts/start.sh dev
|
||||
```
|
||||
|
||||
### Während der Arbeit
|
||||
|
||||
```bash
|
||||
# Logs eines Services anzeigen
|
||||
docker compose logs -f backend
|
||||
|
||||
# Service neustarten
|
||||
docker compose restart backend
|
||||
|
||||
# Status prüfen
|
||||
./scripts/status.sh
|
||||
```
|
||||
|
||||
### Änderungen committen
|
||||
|
||||
```bash
|
||||
# Änderungen anzeigen
|
||||
git status
|
||||
|
||||
# Dateien hinzufügen
|
||||
git add .
|
||||
|
||||
# Commit erstellen
|
||||
git commit -m "Feature: Beschreibung der Änderung"
|
||||
```
|
||||
|
||||
### Abends: Umgebung stoppen
|
||||
|
||||
```bash
|
||||
./scripts/stop.sh dev
|
||||
```
|
||||
|
||||
## Umgebung wechseln
|
||||
|
||||
### Von Dev zu Staging
|
||||
|
||||
```bash
|
||||
# Stoppe Dev
|
||||
./scripts/stop.sh dev
|
||||
|
||||
# Starte Staging
|
||||
./scripts/start.sh staging
|
||||
```
|
||||
|
||||
### Zurück zu Dev
|
||||
|
||||
```bash
|
||||
./scripts/stop.sh staging
|
||||
./scripts/start.sh dev
|
||||
```
|
||||
|
||||
## Code promoten
|
||||
|
||||
### Dev → Staging (nach erfolgreichem Test)
|
||||
|
||||
```bash
|
||||
# Stelle sicher, dass alle Änderungen committet sind
|
||||
git status
|
||||
|
||||
# Promote zu Staging
|
||||
./scripts/promote.sh dev-to-staging
|
||||
|
||||
# Push zu Remote (falls konfiguriert)
|
||||
git push origin staging
|
||||
```
|
||||
|
||||
### Staging → Production (Release)
|
||||
|
||||
```bash
|
||||
# Nur nach vollständigem Test auf Staging!
|
||||
./scripts/promote.sh staging-to-prod
|
||||
|
||||
# Push zu Remote
|
||||
git push origin main
|
||||
```
|
||||
|
||||
## Nützliche Befehle
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
# Alle Container anzeigen
|
||||
docker compose ps
|
||||
|
||||
# Logs folgen
|
||||
docker compose logs -f [service]
|
||||
|
||||
# In Container einsteigen
|
||||
docker compose exec backend bash
|
||||
docker compose exec postgres psql -U breakpilot -d breakpilot_dev
|
||||
|
||||
# Container neustarten
|
||||
docker compose restart [service]
|
||||
|
||||
# Alle Container stoppen und entfernen
|
||||
docker compose down
|
||||
|
||||
# Mit Volumes löschen (VORSICHT!)
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
### Git
|
||||
|
||||
```bash
|
||||
# Aktuellen Branch anzeigen
|
||||
git branch --show-current
|
||||
|
||||
# Alle Branches anzeigen
|
||||
git branch -v
|
||||
|
||||
# Änderungen zwischen Branches anzeigen
|
||||
git diff develop..staging
|
||||
```
|
||||
|
||||
### Datenbank
|
||||
|
||||
```bash
|
||||
# Direkt mit PostgreSQL verbinden (Dev)
|
||||
docker compose exec postgres psql -U breakpilot -d breakpilot_dev
|
||||
|
||||
# Backup erstellen
|
||||
./scripts/backup.sh
|
||||
|
||||
# Backup wiederherstellen
|
||||
./scripts/restore.sh backup-file.sql.gz
|
||||
```
|
||||
|
||||
## Häufige Probleme
|
||||
|
||||
### "Port already in use"
|
||||
|
||||
Ein anderer Prozess oder Container verwendet den Port.
|
||||
|
||||
```bash
|
||||
# Laufende Container prüfen
|
||||
docker ps
|
||||
|
||||
# Alte Container stoppen
|
||||
docker compose down
|
||||
|
||||
# Prozess auf Port finden (z.B. 8000)
|
||||
lsof -i :8000
|
||||
```
|
||||
|
||||
### Container startet nicht
|
||||
|
||||
```bash
|
||||
# Logs prüfen
|
||||
docker compose logs backend
|
||||
|
||||
# Container neu bauen
|
||||
docker compose build backend
|
||||
docker compose up -d backend
|
||||
```
|
||||
|
||||
### Datenbank-Verbindungsfehler
|
||||
|
||||
```bash
|
||||
# Prüfen ob PostgreSQL läuft
|
||||
docker compose ps postgres
|
||||
|
||||
# PostgreSQL-Logs prüfen
|
||||
docker compose logs postgres
|
||||
|
||||
# Neustart
|
||||
docker compose restart postgres
|
||||
```
|
||||
|
||||
### Falsche Umgebung aktiv
|
||||
|
||||
```bash
|
||||
# Status prüfen
|
||||
./scripts/status.sh
|
||||
|
||||
# Auf richtige Umgebung wechseln
|
||||
./scripts/env-switch.sh dev
|
||||
```
|
||||
|
||||
## Umgebungs-Dateien
|
||||
|
||||
| Datei | Beschreibung | Im Git? |
|
||||
|-------|--------------|---------|
|
||||
| `.env` | Aktive Umgebung | Nein |
|
||||
| `.env.dev` | Development Werte | Ja |
|
||||
| `.env.staging` | Staging Werte | Ja |
|
||||
| `.env.prod` | Production Werte | **NEIN** |
|
||||
| `.env.example` | Template | Ja |
|
||||
|
||||
## Ports Übersicht
|
||||
|
||||
### Development
|
||||
|
||||
| Service | Port | URL |
|
||||
|---------|------|-----|
|
||||
| Backend | 8000 | http://localhost:8000 |
|
||||
| Website | 3000 | http://localhost:3000 |
|
||||
| Consent Service | 8081 | http://localhost:8081 |
|
||||
| PostgreSQL | 5432 | localhost:5432 |
|
||||
| Mailpit UI | 8025 | http://localhost:8025 |
|
||||
| MinIO Console | 9001 | http://localhost:9001 |
|
||||
|
||||
### Staging
|
||||
|
||||
| Service | Port | URL |
|
||||
|---------|------|-----|
|
||||
| Backend | 8001 | http://localhost:8001 |
|
||||
| PostgreSQL | 5433 | localhost:5433 |
|
||||
| Mailpit UI | 8026 | http://localhost:8026 |
|
||||
| MinIO Console | 9003 | http://localhost:9003 |
|
||||
|
||||
## Hilfe
|
||||
|
||||
```bash
|
||||
# Status und Übersicht
|
||||
./scripts/status.sh
|
||||
|
||||
# Script-Hilfe
|
||||
./scripts/env-switch.sh --help
|
||||
./scripts/promote.sh --help
|
||||
```
|
||||
|
||||
## Verwandte Dokumentation
|
||||
|
||||
- [Architektur: Umgebungen](../architecture/environments.md)
|
||||
- [Secrets Management](../architecture/secrets-management.md)
|
||||
- [System-Architektur](../architecture/system-architecture.md)
|
||||
1177
docs/klausur-modul/DEVELOPER_SPECIFICATION.md
Normal file
1177
docs/klausur-modul/DEVELOPER_SPECIFICATION.md
Normal file
File diff suppressed because it is too large
Load Diff
463
docs/klausur-modul/MAIL-DEVELOPER-GUIDE.md
Normal file
463
docs/klausur-modul/MAIL-DEVELOPER-GUIDE.md
Normal file
@@ -0,0 +1,463 @@
|
||||
# Unified Inbox - Developer Guide
|
||||
|
||||
**Version:** 1.0.0
|
||||
**Datum:** 2026-01-10
|
||||
**Status:** Implementiert
|
||||
|
||||
---
|
||||
|
||||
## 1. Architektur-Übersicht
|
||||
|
||||
Das Unified Inbox System besteht aus folgenden Komponenten:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ FRONTEND │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ website/app/admin/mail/page.tsx - Admin UI │
|
||||
│ website/app/mail/page.tsx - User Inbox │
|
||||
│ website/app/mail/tasks/page.tsx - Arbeitsvorrat │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│ HTTP (Port 8086)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ KLAUSUR-SERVICE │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ klausur-service/backend/mail/ │
|
||||
│ ├── __init__.py - Module exports │
|
||||
│ ├── models.py - Pydantic Models & Enums │
|
||||
│ ├── mail_db.py - PostgreSQL operations (asyncpg) │
|
||||
│ ├── credentials.py - Vault/encrypted credential storage │
|
||||
│ ├── aggregator.py - IMAP/SMTP multi-account sync │
|
||||
│ ├── ai_service.py - LLM-powered email analysis │
|
||||
│ ├── task_service.py - Arbeitsvorrat (task management) │
|
||||
│ └── api.py - FastAPI router (30+ endpoints) │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│
|
||||
┌───────────────────┼───────────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ PostgreSQL │ │ Vault │ │ LLM Gateway │
|
||||
│ (Port 5432) │ │ (Port 8200) │ │ (Port 8000) │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Schnellstart
|
||||
|
||||
### 2.1 Service starten
|
||||
|
||||
```bash
|
||||
# Klausur-Service starten
|
||||
cd klausur-service/backend
|
||||
uvicorn main:app --host 0.0.0.0 --port 8086 --reload
|
||||
|
||||
# Frontend starten
|
||||
cd website
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### 2.2 API testen
|
||||
|
||||
```bash
|
||||
# Health Check
|
||||
curl http://localhost:8086/api/v1/mail/health
|
||||
|
||||
# Konto hinzufügen (mit Auth)
|
||||
curl -X POST http://localhost:8086/api/v1/mail/accounts \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"email": "schulleitung@example.de",
|
||||
"imap_host": "imap.example.de",
|
||||
"imap_port": 993,
|
||||
"smtp_host": "smtp.example.de",
|
||||
"smtp_port": 587,
|
||||
"username": "schulleitung",
|
||||
"password": "secret"
|
||||
}'
|
||||
|
||||
# Inbox abrufen
|
||||
curl http://localhost:8086/api/v1/mail/inbox \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
|
||||
# E-Mail analysieren
|
||||
curl -X POST http://localhost:8086/api/v1/mail/analyze/{email_id} \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Module im Detail
|
||||
|
||||
### 3.1 Models (`models.py`)
|
||||
|
||||
#### Enums
|
||||
|
||||
```python
|
||||
class SenderType(str, Enum):
|
||||
"""Klassifizierung des Absender-Typs."""
|
||||
KULTUSMINISTERIUM = "kultusministerium"
|
||||
LANDESSCHULBEHOERDE = "landesschulbehoerde"
|
||||
RLSB = "rlsb"
|
||||
SCHULAMT = "schulamt"
|
||||
NIBIS = "nibis"
|
||||
SCHULTRAEGER = "schultraeger"
|
||||
ELTERNVERTRETER = "elternvertreter"
|
||||
GEWERKSCHAFT = "gewerkschaft"
|
||||
FORTBILDUNGSINSTITUT = "fortbildungsinstitut"
|
||||
PRIVATPERSON = "privatperson"
|
||||
UNTERNEHMEN = "unternehmen"
|
||||
UNBEKANNT = "unbekannt"
|
||||
|
||||
class TaskPriority(str, Enum):
|
||||
"""Aufgaben-Priorität."""
|
||||
URGENT = "urgent" # Sofort bearbeiten
|
||||
HIGH = "high" # Zeitnah bearbeiten
|
||||
MEDIUM = "medium" # Normale Bearbeitung
|
||||
LOW = "low" # Kann warten
|
||||
```
|
||||
|
||||
#### Bekannte Behörden (Niedersachsen)
|
||||
|
||||
```python
|
||||
KNOWN_AUTHORITIES_NI = {
|
||||
"@mk.niedersachsen.de": {
|
||||
"type": SenderType.KULTUSMINISTERIUM,
|
||||
"name": "Kultusministerium Niedersachsen",
|
||||
"priority_boost": 2,
|
||||
},
|
||||
"@rlsb.de": {
|
||||
"type": SenderType.RLSB,
|
||||
"name": "Regionales Landesamt für Schule und Bildung",
|
||||
"priority_boost": 2,
|
||||
},
|
||||
"@landesschulbehoerde-nds.de": {
|
||||
"type": SenderType.LANDESSCHULBEHOERDE,
|
||||
"name": "Landesschulbehörde Niedersachsen",
|
||||
"priority_boost": 2,
|
||||
},
|
||||
"@nibis.de": {
|
||||
"type": SenderType.NIBIS,
|
||||
"name": "Niedersächsischer Bildungsserver",
|
||||
"priority_boost": 1,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 Database (`mail_db.py`)
|
||||
|
||||
#### Tabellen
|
||||
|
||||
| Tabelle | Beschreibung |
|
||||
|---------|--------------|
|
||||
| `external_email_accounts` | IMAP/SMTP-Konfiguration |
|
||||
| `aggregated_emails` | Gecachte E-Mails |
|
||||
| `inbox_tasks` | Extrahierte Aufgaben |
|
||||
| `email_templates` | Antwortvorlagen |
|
||||
| `mail_audit_log` | Audit-Trail |
|
||||
| `mail_sync_status` | Sync-Status pro Konto |
|
||||
|
||||
#### Beispiel: E-Mail speichern
|
||||
|
||||
```python
|
||||
from mail.mail_db import store_email
|
||||
|
||||
email_id = await store_email(
|
||||
account_id="acc_123",
|
||||
message_id="<msg@example.de>",
|
||||
subject="Einladung zur Fortbildung",
|
||||
sender_email="info@mk.niedersachsen.de",
|
||||
sender_name="Kultusministerium",
|
||||
recipients=["schulleitung@example.de"],
|
||||
date=datetime.now(),
|
||||
body_preview="Sehr geehrte Schulleitung...",
|
||||
body_text="Vollständiger Text...",
|
||||
has_attachments=True,
|
||||
attachment_count=2,
|
||||
user_id="user_456",
|
||||
tenant_id="tenant_789",
|
||||
)
|
||||
```
|
||||
|
||||
### 3.3 Credentials (`credentials.py`)
|
||||
|
||||
Sichere Speicherung von IMAP/SMTP-Passwörtern:
|
||||
|
||||
```python
|
||||
from mail.credentials import get_credentials_service
|
||||
|
||||
creds = get_credentials_service()
|
||||
|
||||
# Passwort speichern (verschlüsselt)
|
||||
await creds.store_credentials(
|
||||
account_id="acc_123",
|
||||
username="user@example.de",
|
||||
password="secret123"
|
||||
)
|
||||
|
||||
# Passwort abrufen
|
||||
username, password = await creds.get_credentials("acc_123")
|
||||
```
|
||||
|
||||
**Verschlüsselung:**
|
||||
- Produktion: HashiCorp Vault (KV v2)
|
||||
- Entwicklung: Fernet-Verschlüsselung mit lokalem Key
|
||||
|
||||
### 3.4 Aggregator (`aggregator.py`)
|
||||
|
||||
IMAP-Synchronisation und SMTP-Versand:
|
||||
|
||||
```python
|
||||
from mail.aggregator import get_mail_aggregator
|
||||
|
||||
aggregator = get_mail_aggregator()
|
||||
|
||||
# Verbindung testen
|
||||
result = await aggregator.test_account_connection(account_id)
|
||||
if result.success:
|
||||
print(f"Verbunden: {result.folder_count} Ordner")
|
||||
|
||||
# Konto synchronisieren
|
||||
synced = await aggregator.sync_account(
|
||||
account_id="acc_123",
|
||||
user_id="user_456",
|
||||
tenant_id="tenant_789",
|
||||
full_sync=False # Nur neue E-Mails
|
||||
)
|
||||
print(f"{synced} neue E-Mails synchronisiert")
|
||||
|
||||
# E-Mail senden
|
||||
await aggregator.send_email(
|
||||
account_id="acc_123",
|
||||
to_addresses=["empfaenger@example.de"],
|
||||
subject="Antwort",
|
||||
body_html="<p>Vielen Dank für Ihre Anfrage...</p>"
|
||||
)
|
||||
```
|
||||
|
||||
### 3.5 AI Service (`ai_service.py`)
|
||||
|
||||
KI-gestützte E-Mail-Analyse:
|
||||
|
||||
```python
|
||||
from mail.ai_service import get_ai_email_service
|
||||
|
||||
ai = get_ai_email_service()
|
||||
|
||||
# Vollständige Analyse
|
||||
result = await ai.analyze_email(email_id, user_id)
|
||||
|
||||
print(f"Absender: {result.sender.sender_type}")
|
||||
print(f"Kategorie: {result.category}")
|
||||
print(f"Priorität: {result.suggested_priority}")
|
||||
print(f"Fristen: {len(result.deadlines)}")
|
||||
for deadline in result.deadlines:
|
||||
print(f" - {deadline.deadline_date}: {deadline.description}")
|
||||
```
|
||||
|
||||
#### Absender-Klassifikation
|
||||
|
||||
1. **Domain-Matching:** Bekannte Behörden-Domains
|
||||
2. **LLM-Fallback:** Bei unbekannten Absendern
|
||||
|
||||
#### Fristen-Erkennung
|
||||
|
||||
1. **Regex-Patterns:** Deutsche Datumsformate
|
||||
2. **LLM-Verstärkung:** Kontextuelle Deadline-Erkennung
|
||||
|
||||
### 3.6 Task Service (`task_service.py`)
|
||||
|
||||
Arbeitsvorrat-Management:
|
||||
|
||||
```python
|
||||
from mail.task_service import get_task_service
|
||||
|
||||
tasks = get_task_service()
|
||||
|
||||
# Task aus E-Mail erstellen
|
||||
task_id = await tasks.create_task_from_email(
|
||||
user_id="user_456",
|
||||
tenant_id="tenant_789",
|
||||
email_id="email_123",
|
||||
deadlines=[...], # Aus AI-Analyse
|
||||
sender_type=SenderType.KULTUSMINISTERIUM,
|
||||
auto_created=True
|
||||
)
|
||||
|
||||
# Dashboard-Statistiken
|
||||
stats = await tasks.get_dashboard_stats(user_id)
|
||||
print(f"Gesamt: {stats.total_tasks}")
|
||||
print(f"Überfällig: {stats.overdue_tasks}")
|
||||
print(f"Heute fällig: {stats.due_today}")
|
||||
|
||||
# Überfällige Tasks
|
||||
overdue = await tasks.get_overdue_tasks(user_id)
|
||||
for task in overdue:
|
||||
print(f"ÜBERFÄLLIG: {task['title']}")
|
||||
```
|
||||
|
||||
#### Prioritäts-Berechnung
|
||||
|
||||
```python
|
||||
# Absender-basierte Priorität
|
||||
KULTUSMINISTERIUM → HIGH
|
||||
NIBIS → MEDIUM
|
||||
PRIVATPERSON → LOW
|
||||
|
||||
# Deadline-Anpassung
|
||||
≤ 1 Tag → URGENT
|
||||
≤ 3 Tage → min. HIGH
|
||||
≤ 7 Tage → min. MEDIUM
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. API-Endpoints
|
||||
|
||||
### 4.1 Account Management
|
||||
|
||||
| Method | Endpoint | Beschreibung |
|
||||
|--------|----------|--------------|
|
||||
| `POST` | `/api/v1/mail/accounts` | Konto hinzufügen |
|
||||
| `GET` | `/api/v1/mail/accounts` | Alle Konten |
|
||||
| `GET` | `/api/v1/mail/accounts/{id}` | Einzelnes Konto |
|
||||
| `DELETE` | `/api/v1/mail/accounts/{id}` | Konto entfernen |
|
||||
| `POST` | `/api/v1/mail/accounts/{id}/test` | Verbindung testen |
|
||||
| `POST` | `/api/v1/mail/accounts/{id}/sync` | Konto synchronisieren |
|
||||
|
||||
### 4.2 Unified Inbox
|
||||
|
||||
| Method | Endpoint | Beschreibung |
|
||||
|--------|----------|--------------|
|
||||
| `GET` | `/api/v1/mail/inbox` | Aggregierte Inbox |
|
||||
| `GET` | `/api/v1/mail/inbox/{id}` | Einzelne E-Mail |
|
||||
| `POST` | `/api/v1/mail/inbox/{id}/read` | Als gelesen markieren |
|
||||
| `POST` | `/api/v1/mail/send` | E-Mail senden |
|
||||
|
||||
### 4.3 AI Analysis
|
||||
|
||||
| Method | Endpoint | Beschreibung |
|
||||
|--------|----------|--------------|
|
||||
| `POST` | `/api/v1/mail/analyze/{id}` | E-Mail analysieren |
|
||||
| `GET` | `/api/v1/mail/suggestions/{id}` | Antwortvorschläge |
|
||||
|
||||
### 4.4 Tasks (Arbeitsvorrat)
|
||||
|
||||
| Method | Endpoint | Beschreibung |
|
||||
|--------|----------|--------------|
|
||||
| `GET` | `/api/v1/mail/tasks` | Alle Tasks |
|
||||
| `POST` | `/api/v1/mail/tasks` | Manuelle Task |
|
||||
| `GET` | `/api/v1/mail/tasks/{id}` | Einzelne Task |
|
||||
| `PATCH` | `/api/v1/mail/tasks/{id}` | Task aktualisieren |
|
||||
| `POST` | `/api/v1/mail/tasks/{id}/complete` | Als erledigt markieren |
|
||||
| `GET` | `/api/v1/mail/tasks/dashboard` | Dashboard-Stats |
|
||||
|
||||
---
|
||||
|
||||
## 5. Tests
|
||||
|
||||
### 5.1 Unit Tests ausführen
|
||||
|
||||
```bash
|
||||
cd klausur-service/backend
|
||||
pytest tests/test_mail_service.py -v
|
||||
```
|
||||
|
||||
### 5.2 Testabdeckung
|
||||
|
||||
```bash
|
||||
pytest tests/test_mail_service.py --cov=mail --cov-report=html
|
||||
```
|
||||
|
||||
### 5.3 Testfälle
|
||||
|
||||
- **Absender-Erkennung:** Behörden-Domains
|
||||
- **Prioritäts-Berechnung:** Sender-basiert, Deadline-basiert
|
||||
- **Fristen-Extraktion:** Deutsche Datumsformate
|
||||
- **Kategorie-Regeln:** Fortbildung, Personal, Finanzen
|
||||
|
||||
---
|
||||
|
||||
## 6. Konfiguration
|
||||
|
||||
### 6.1 Umgebungsvariablen
|
||||
|
||||
```env
|
||||
# Datenbank
|
||||
DATABASE_URL=postgresql://user:pass@localhost:5432/breakpilot
|
||||
|
||||
# Vault (Produktion)
|
||||
VAULT_ADDR=http://vault:8200
|
||||
VAULT_TOKEN=hvs.xxxxx
|
||||
|
||||
# Entwicklung (ohne Vault)
|
||||
MAIL_ENCRYPTION_KEY=base64-encoded-fernet-key
|
||||
|
||||
# LLM Gateway
|
||||
LLM_GATEWAY_URL=http://localhost:8000
|
||||
DEFAULT_MODEL=breakpilot-teacher-8b
|
||||
```
|
||||
|
||||
### 6.2 Playbook-Konfiguration
|
||||
|
||||
Das Mail-Analyse-Playbook ist in `/backend/llm_gateway/services/playbook_service.py` definiert:
|
||||
|
||||
```python
|
||||
"mail_analysis": Playbook(
|
||||
id="mail_analysis",
|
||||
name="E-Mail-Analyse",
|
||||
system_prompt="...", # Niedersachsen-spezifisch
|
||||
recommended_models=["breakpilot-teacher-8b"],
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Erweiterung
|
||||
|
||||
### 7.1 Neue Bundesländer hinzufügen
|
||||
|
||||
1. `KNOWN_AUTHORITIES_{KÜRZEL}` in `models.py` definieren
|
||||
2. Domain-Patterns für Behörden hinzufügen
|
||||
3. Playbook-Prompt anpassen
|
||||
|
||||
### 7.2 Neue Kategorien hinzufügen
|
||||
|
||||
1. `EmailCategory` Enum erweitern
|
||||
2. Regeln in `ai_service._apply_category_rules()` hinzufügen
|
||||
3. Frontend-Labels in `page.tsx` ergänzen
|
||||
|
||||
---
|
||||
|
||||
## 8. Troubleshooting
|
||||
|
||||
### IMAP-Verbindungsfehler
|
||||
|
||||
```
|
||||
IMAPConnectionError: Connection refused
|
||||
```
|
||||
→ Port/SSL-Einstellungen prüfen, Firewall-Regeln
|
||||
|
||||
### Vault-Authentifizierungsfehler
|
||||
|
||||
```
|
||||
VaultError: permission denied
|
||||
```
|
||||
→ Token prüfen, Policy für `secret/mail/*` erforderlich
|
||||
|
||||
### LLM-Timeout
|
||||
|
||||
```
|
||||
HTTPException: 504 Gateway Timeout
|
||||
```
|
||||
→ LLM Gateway Status prüfen, Modell-Verfügbarkeit
|
||||
|
||||
---
|
||||
|
||||
## 9. Referenzen
|
||||
|
||||
- [UNIFIED-INBOX-SPECIFICATION.md](./UNIFIED-INBOX-SPECIFICATION.md) - Vollständige Spezifikation
|
||||
- [MAIL-RBAC-DEVELOPER-SPECIFICATION.md](./MAIL-RBAC-DEVELOPER-SPECIFICATION.md) - DSGVO & RBAC Details
|
||||
- [mail-rbac-architecture.md](../architecture/mail-rbac-architecture.md) - Architektur-Dokument
|
||||
1834
docs/klausur-modul/MAIL-RBAC-DEVELOPER-SPECIFICATION.md
Normal file
1834
docs/klausur-modul/MAIL-RBAC-DEVELOPER-SPECIFICATION.md
Normal file
File diff suppressed because it is too large
Load Diff
1555
docs/klausur-modul/UNIFIED-INBOX-SPECIFICATION.md
Normal file
1555
docs/klausur-modul/UNIFIED-INBOX-SPECIFICATION.md
Normal file
File diff suppressed because it is too large
Load Diff
184
docs/testing/h5p-service-tests.md
Normal file
184
docs/testing/h5p-service-tests.md
Normal file
@@ -0,0 +1,184 @@
|
||||
# H5P Service Tests
|
||||
|
||||
## Übersicht
|
||||
|
||||
Das H5P Service Modul verfügt über umfassende Integration Tests, die alle Endpoints und Content-Typen validieren.
|
||||
|
||||
## Test-Coverage
|
||||
|
||||
### Module
|
||||
- **Server Endpoints**: Health, Info, Editor Selection
|
||||
- **8 Content Type Editors**: Quiz, Video, Presentation, Flashcards, Timeline, Drag & Drop, Fill Blanks, Memory
|
||||
- **8 Content Type Players**: Interaktive Player für alle Content-Typen
|
||||
- **Static Files**: Korrekte Bereitstellung von Editoren, Players und Assets
|
||||
- **Error Handling**: 404-Behandlung für ungültige Routen
|
||||
|
||||
## Tests ausführen
|
||||
|
||||
### Lokal
|
||||
|
||||
```bash
|
||||
cd h5p-service
|
||||
npm install
|
||||
npm test
|
||||
```
|
||||
|
||||
### Im Docker Container
|
||||
|
||||
```bash
|
||||
# Service starten
|
||||
docker compose -f docker-compose.content.yml up -d h5p-service
|
||||
|
||||
# Tests ausführen
|
||||
docker compose -f docker-compose.content.yml exec h5p-service npm test
|
||||
```
|
||||
|
||||
## Test-Dateien
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `h5p-service/tests/server.test.js` | Integration Tests für alle Endpoints |
|
||||
| `h5p-service/tests/setup.js` | Jest Test Setup & Configuration |
|
||||
| `h5p-service/jest.config.js` | Jest Configuration |
|
||||
|
||||
## Coverage Reports
|
||||
|
||||
Nach dem Ausführen der Tests:
|
||||
|
||||
```bash
|
||||
# HTML Report öffnen
|
||||
open h5p-service/coverage/lcov-report/index.html
|
||||
```
|
||||
|
||||
Coverage-Ziel: **>80%**
|
||||
|
||||
## Test-Kategorien
|
||||
|
||||
### 1. Health & Info Tests
|
||||
- Service-Verfügbarkeit
|
||||
- Health Check Endpoint
|
||||
- Service Info Page
|
||||
|
||||
### 2. Editor Selection Tests
|
||||
- Alle 8 Content-Typen sichtbar
|
||||
- Korrekte Links zu Editoren
|
||||
- UI-Elemente vorhanden
|
||||
|
||||
### 3. Content Type Tests
|
||||
|
||||
Für jeden der 8 Content-Typen:
|
||||
- Editor lädt korrekt
|
||||
- Player lädt korrekt
|
||||
- Erforderliche UI-Elemente vorhanden
|
||||
|
||||
**Content-Typen:**
|
||||
1. Quiz (Question Set)
|
||||
2. Interactive Video
|
||||
3. Course Presentation
|
||||
4. Flashcards
|
||||
5. Timeline
|
||||
6. Drag and Drop
|
||||
7. Fill in the Blanks
|
||||
8. Memory Game
|
||||
|
||||
### 4. Static File Tests
|
||||
- Core Files erreichbar
|
||||
- Editor Files erreichbar
|
||||
- Player Files erreichbar
|
||||
|
||||
### 5. Error Handling Tests
|
||||
- 404 für ungültige Routes
|
||||
- Fehlerbehandlung für fehlende Editoren/Players
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
Die Tests sind in die CI/CD Pipeline integriert:
|
||||
|
||||
```yaml
|
||||
# .github/workflows/tests.yml (Beispiel)
|
||||
name: H5P Service Tests
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Start H5P Service
|
||||
run: docker compose -f docker-compose.content.yml up -d h5p-service
|
||||
|
||||
- name: Wait for Service
|
||||
run: sleep 10
|
||||
|
||||
- name: Run Tests
|
||||
run: docker compose -f docker-compose.content.yml exec h5p-service npm run test:ci
|
||||
|
||||
- name: Upload Coverage
|
||||
uses: codecov/codecov-action@v3
|
||||
with:
|
||||
files: ./h5p-service/coverage/lcov.info
|
||||
```
|
||||
|
||||
## Test-Qualität
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Isolation**: Jeder Test ist unabhängig
|
||||
2. **Cleanup**: Keine persistenten Änderungen
|
||||
3. **Assertions**: Klare Expectations
|
||||
4. **Speed**: Tests laufen schnell (<10s gesamt)
|
||||
5. **Reliability**: Tests sind deterministisch
|
||||
|
||||
### Code Review Checklist
|
||||
|
||||
Bei neuen Features:
|
||||
- [ ] Tests für neue Endpoints hinzugefügt
|
||||
- [ ] Coverage bleibt >80%
|
||||
- [ ] Tests sind dokumentiert
|
||||
- [ ] CI/CD Tests bestehen
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Tests schlagen fehl
|
||||
|
||||
**Service nicht erreichbar:**
|
||||
```bash
|
||||
# Service Status prüfen
|
||||
docker compose -f docker-compose.content.yml ps
|
||||
|
||||
# Logs ansehen
|
||||
docker compose -f docker-compose.content.yml logs h5p-service
|
||||
```
|
||||
|
||||
**Port-Konflikte:**
|
||||
```bash
|
||||
# Prüfe, ob Port 8003 belegt ist
|
||||
lsof -i :8003
|
||||
|
||||
# Stoppe andere Services
|
||||
docker compose -f docker-compose.content.yml down
|
||||
```
|
||||
|
||||
**Veraltete Dependencies:**
|
||||
```bash
|
||||
cd h5p-service
|
||||
rm -rf node_modules package-lock.json
|
||||
npm install
|
||||
```
|
||||
|
||||
## Zukünftige Erweiterungen
|
||||
|
||||
- [ ] E2E Tests mit Playwright
|
||||
- [ ] Content Validation Tests
|
||||
- [ ] Performance Tests
|
||||
- [ ] Security Tests (XSS, CSRF)
|
||||
- [ ] Load Tests
|
||||
- [ ] Visual Regression Tests
|
||||
|
||||
## Verwandte Dokumentation
|
||||
|
||||
- [H5P Service README](../../h5p-service/tests/README.md)
|
||||
- [Content Service Tests](./content-service-tests.md)
|
||||
- [Integration Testing Guide](./integration-testing.md)
|
||||
333
docs/testing/integration-test-environment.md
Normal file
333
docs/testing/integration-test-environment.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# Integration Test Environment
|
||||
|
||||
> **Letzte Aktualisierung:** 2026-02-04
|
||||
> **Status:** Produktiv
|
||||
> **Maintainer:** DevOps Team
|
||||
|
||||
---
|
||||
|
||||
## Inhaltsverzeichnis
|
||||
|
||||
1. [Uebersicht](#1-uebersicht)
|
||||
2. [Quick Start (Lokal)](#2-quick-start-lokal)
|
||||
3. [Services](#3-services)
|
||||
4. [CI/CD Integration](#4-cicd-integration)
|
||||
5. [Konfiguration](#5-konfiguration)
|
||||
6. [Troubleshooting](#6-troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## 1. Uebersicht
|
||||
|
||||
Die Integration-Test-Umgebung ermoeglicht vollstaendige End-to-End-Tests aller Services in einer isolierten Docker-Compose-Umgebung. Sie wird sowohl lokal als auch in der CI/CD-Pipeline verwendet.
|
||||
|
||||
### Architektur
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ docker-compose.test.yml │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
||||
│ │ postgres-test│ │ valkey-test │ │ mailpit-test │ │
|
||||
│ │ Port 55432 │ │ Port 56379 │ │ Web: 58025 │ │
|
||||
│ │ PostgreSQL │ │ Redis/Valkey│ │ SMTP: 51025 │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ consent-service-test │ │
|
||||
│ │ Port 58081 │ │
|
||||
│ │ Go Authentication Service │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────┐ │
|
||||
│ │ backend-test │ │
|
||||
│ │ Port 58000 │ │
|
||||
│ │ Python FastAPI Backend │ │
|
||||
│ └─────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Vorteile
|
||||
|
||||
- **Isolation:** Keine Konflikte mit Produktions- oder Entwicklungs-Datenbanken
|
||||
- **Reproduzierbarkeit:** Identische Umgebung lokal und in CI/CD
|
||||
- **Health Checks:** Automatisches Warten auf Service-Verfuegbarkeit
|
||||
- **Cleanup:** Automatisches Entfernen aller Volumes nach Tests
|
||||
|
||||
---
|
||||
|
||||
## 2. Quick Start (Lokal)
|
||||
|
||||
### Test-Umgebung starten
|
||||
|
||||
```bash
|
||||
# 1. Test-Umgebung starten
|
||||
docker compose -f docker-compose.test.yml up -d
|
||||
|
||||
# 2. Warten bis alle Services healthy sind
|
||||
docker compose -f docker-compose.test.yml ps
|
||||
|
||||
# 3. Health-Status pruefen (alle sollten "healthy" zeigen)
|
||||
watch docker compose -f docker-compose.test.yml ps
|
||||
```
|
||||
|
||||
### Integration Tests ausfuehren
|
||||
|
||||
```bash
|
||||
# In das Backend-Verzeichnis wechseln
|
||||
cd backend
|
||||
|
||||
# Virtual Environment aktivieren (falls vorhanden)
|
||||
source venv/bin/activate
|
||||
|
||||
# Integration Tests ausfuehren
|
||||
export SKIP_INTEGRATION_TESTS=false
|
||||
pytest tests/test_integration/ -v
|
||||
```
|
||||
|
||||
### Services einzeln testen
|
||||
|
||||
```bash
|
||||
# Datenbank-Verbindung testen
|
||||
psql -h localhost -p 55432 -U breakpilot -d breakpilot_test -c "SELECT 1"
|
||||
|
||||
# Consent Service Health Check
|
||||
curl -f http://localhost:58081/health
|
||||
|
||||
# Backend Health Check
|
||||
curl -f http://localhost:58000/health
|
||||
|
||||
# Mailpit Web UI
|
||||
open http://localhost:58025
|
||||
```
|
||||
|
||||
### Aufraeumen
|
||||
|
||||
```bash
|
||||
# Alle Container stoppen und Volumes loeschen
|
||||
docker compose -f docker-compose.test.yml down -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Services
|
||||
|
||||
### Port-Strategie
|
||||
|
||||
| Service | Test-Port (extern) | Container-Port | Prod-Port |
|
||||
|---------|-------------------|----------------|-----------|
|
||||
| PostgreSQL | 55432 | 5432 | 5432 |
|
||||
| Valkey/Redis | 56379 | 6379 | 6379 |
|
||||
| Consent Service | 58081 | 8081 | 8081 |
|
||||
| Backend | 58000 | 8000 | 8000 |
|
||||
| Mailpit Web | 58025 | 8025 | - |
|
||||
| Mailpit SMTP | 51025 | 1025 | - |
|
||||
|
||||
### Service-Details
|
||||
|
||||
#### postgres-test
|
||||
|
||||
- **Image:** `postgres:16-alpine`
|
||||
- **Credentials:** `breakpilot:breakpilot_test@breakpilot_test`
|
||||
- **Health Check:** `pg_isready -U breakpilot -d breakpilot_test`
|
||||
|
||||
#### valkey-test
|
||||
|
||||
- **Image:** `valkey/valkey:7-alpine`
|
||||
- **Health Check:** `valkey-cli ping`
|
||||
- **Hinweis:** Ersetzt Redis fuer bessere ARM64-Kompatibilitaet
|
||||
|
||||
#### consent-service-test
|
||||
|
||||
- **Build:** `./consent-service/Dockerfile`
|
||||
- **Health Check:** `GET /health`
|
||||
- **Abhaengigkeiten:** postgres-test, valkey-test
|
||||
|
||||
#### backend-test
|
||||
|
||||
- **Build:** `./backend/Dockerfile`
|
||||
- **Health Check:** `GET /health`
|
||||
- **Abhaengigkeiten:** postgres-test, valkey-test, consent-service-test
|
||||
|
||||
#### mailpit-test
|
||||
|
||||
- **Image:** `axllent/mailpit:latest`
|
||||
- **Web UI:** http://localhost:58025
|
||||
- **SMTP:** localhost:51025
|
||||
- **Zweck:** E-Mail-Testing ohne echten SMTP-Server
|
||||
|
||||
---
|
||||
|
||||
## 4. CI/CD Integration
|
||||
|
||||
### Woodpecker Pipeline
|
||||
|
||||
Die Integration-Tests laufen automatisch bei jedem Push/PR zu `main` oder `develop`:
|
||||
|
||||
```yaml
|
||||
# .woodpecker/main.yml
|
||||
integration-tests:
|
||||
image: docker:27-cli
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
commands:
|
||||
- docker compose -f docker-compose.test.yml up -d
|
||||
- # Wait for services...
|
||||
- docker compose -f docker-compose.test.yml exec -T backend-test \
|
||||
pytest tests/test_integration/ -v
|
||||
- docker compose -f docker-compose.test.yml down -v
|
||||
when:
|
||||
- event: [push, pull_request]
|
||||
branch: [main, develop]
|
||||
```
|
||||
|
||||
### Pipeline-Ablauf
|
||||
|
||||
1. **Unit Tests** (test-go-*, test-python-*) laufen zuerst
|
||||
2. **Integration Tests** starten Docker Compose Umgebung
|
||||
3. **Report** sendet alle Ergebnisse ans Test Dashboard
|
||||
|
||||
### Manueller Pipeline-Trigger
|
||||
|
||||
```bash
|
||||
# Via Gitea/Woodpecker UI oder:
|
||||
curl -X POST "https://macmini:4431/api/repos/pilotadmin/breakpilot-pwa/pipelines" \
|
||||
-H "Authorization: Bearer $WOODPECKER_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"branch":"main"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Konfiguration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Die Test-Umgebung konfiguriert automatisch alle noetigen Variablen:
|
||||
|
||||
| Variable | Wert in Test-Umgebung |
|
||||
|----------|----------------------|
|
||||
| `DATABASE_URL` | `postgresql://breakpilot:breakpilot_test@postgres-test:5432/breakpilot_test` |
|
||||
| `CONSENT_SERVICE_URL` | `http://consent-service-test:8081` |
|
||||
| `VALKEY_URL` | `redis://valkey-test:6379` |
|
||||
| `REDIS_URL` | `redis://valkey-test:6379` |
|
||||
| `JWT_SECRET` | `test-jwt-secret-for-integration-tests` |
|
||||
| `ENVIRONMENT` | `test` |
|
||||
| `SMTP_HOST` | `mailpit-test` |
|
||||
| `SMTP_PORT` | `1025` |
|
||||
|
||||
### conftest.py Integration
|
||||
|
||||
Das `backend/tests/conftest.py` erkennt automatisch die Integration-Umgebung:
|
||||
|
||||
```python
|
||||
# Wird aktiviert wenn SKIP_INTEGRATION_TESTS=false
|
||||
IS_INTEGRATION_ENV = os.environ.get("SKIP_INTEGRATION_TESTS", "").lower() == "false"
|
||||
|
||||
if IS_INTEGRATION_ENV:
|
||||
os.environ.setdefault("DATABASE_URL",
|
||||
"postgresql://breakpilot:breakpilot_test@postgres-test:5432/breakpilot_test")
|
||||
# ... weitere Container-URLs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Troubleshooting
|
||||
|
||||
### Service startet nicht
|
||||
|
||||
```bash
|
||||
# Logs eines spezifischen Services anzeigen
|
||||
docker compose -f docker-compose.test.yml logs consent-service-test --tail=100
|
||||
|
||||
# Alle Logs anzeigen
|
||||
docker compose -f docker-compose.test.yml logs
|
||||
```
|
||||
|
||||
### Health Check schlaegt fehl
|
||||
|
||||
```bash
|
||||
# Container-Status pruefen
|
||||
docker compose -f docker-compose.test.yml ps
|
||||
|
||||
# In Container einloggen
|
||||
docker compose -f docker-compose.test.yml exec backend-test bash
|
||||
|
||||
# Health-Endpoint manuell pruefen
|
||||
docker compose -f docker-compose.test.yml exec backend-test curl -v http://localhost:8000/health
|
||||
```
|
||||
|
||||
### Datenbank-Verbindungsprobleme
|
||||
|
||||
```bash
|
||||
# Datenbank-Container pruefen
|
||||
docker compose -f docker-compose.test.yml exec postgres-test pg_isready -U breakpilot
|
||||
|
||||
# Datenbank-Logs
|
||||
docker compose -f docker-compose.test.yml logs postgres-test
|
||||
```
|
||||
|
||||
### Port-Konflikte
|
||||
|
||||
Falls Ports bereits belegt sind:
|
||||
|
||||
```bash
|
||||
# Pruefen welcher Prozess den Port belegt
|
||||
lsof -i :55432
|
||||
|
||||
# Container mit anderen Ports starten (manuell .yml anpassen)
|
||||
# Oder: Bestehende Container stoppen
|
||||
docker compose -f docker-compose.test.yml down
|
||||
```
|
||||
|
||||
### Tests finden keine Services
|
||||
|
||||
Stellen Sie sicher, dass:
|
||||
|
||||
1. `SKIP_INTEGRATION_TESTS=false` gesetzt ist
|
||||
2. Tests innerhalb des Docker-Netzwerks laufen
|
||||
3. Container-Namen (nicht localhost) verwendet werden
|
||||
|
||||
```bash
|
||||
# Innerhalb des Backend-Containers:
|
||||
export SKIP_INTEGRATION_TESTS=false
|
||||
pytest tests/test_integration/ -v
|
||||
```
|
||||
|
||||
### Cleanup bei Problemen
|
||||
|
||||
```bash
|
||||
# Komplettes Aufräumen
|
||||
docker compose -f docker-compose.test.yml down -v --remove-orphans
|
||||
|
||||
# Auch Netzwerke entfernen
|
||||
docker network prune -f
|
||||
|
||||
# Alle Test-Container entfernen
|
||||
docker rm -f $(docker ps -a -q --filter "name=breakpilot-*-test") 2>/dev/null || true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anhang: Test-Verzeichnisstruktur
|
||||
|
||||
```
|
||||
backend/tests/
|
||||
├── conftest.py # Pytest Konfiguration mit Integration-Detection
|
||||
├── test_consent_client.py # Unit Tests (Mock-basiert)
|
||||
├── test_gdpr_api.py # Unit Tests
|
||||
└── test_integration/ # Integration Tests (benoetigen Docker Compose)
|
||||
├── __init__.py
|
||||
├── conftest.py # Integration-spezifische Fixtures
|
||||
├── test_consent_flow.py # E2E Consent Workflow
|
||||
├── test_auth_flow.py # E2E Auth Workflow
|
||||
└── test_email_flow.py # E2E E-Mail Tests (Mailpit)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Generiert am 2026-02-04 von Claude Code*
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 32 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 9.8 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 5.6 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 10 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 8.4 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 41 KiB |
@@ -0,0 +1,3 @@
|
||||
Die EWH enthalten keine urheberrechtlich relevanten Elemente.
|
||||
|
||||
EWHs wurden ergänzt (298.05.2024)
|
||||
@@ -0,0 +1,2 @@
|
||||
in den Informatik-ZA-Aufgaben 2024 sind keine Fremdmaterialien enthalten. Also ist bzgl des Urheberrechts alles geklärt.
|
||||
|
||||
400
docs/zeugnis-system/README.md
Normal file
400
docs/zeugnis-system/README.md
Normal file
@@ -0,0 +1,400 @@
|
||||
# Zeugnis-System Dokumentation
|
||||
|
||||
## Übersicht
|
||||
|
||||
Das Zeugnis-System ist ein umfassendes Modul für die Verwaltung und Nutzung von Zeugnisverordnungen aller 16 deutschen Bundesländer. Es besteht aus drei Hauptkomponenten:
|
||||
|
||||
1. **Rights-Aware Crawler** - Automatische Sammlung von Verordnungen
|
||||
2. **Training Dashboard** - Steuerung und Überwachung von KI-Trainingsprozessen
|
||||
3. **Lehrer-Frontend** - Intuitive Benutzeroberfläche für Lehrkräfte
|
||||
|
||||
---
|
||||
|
||||
## Inhaltsverzeichnis
|
||||
|
||||
1. [Architektur](#architektur)
|
||||
2. [Rights-Aware Crawler](#rights-aware-crawler)
|
||||
3. [Training Dashboard](#training-dashboard)
|
||||
4. [Lehrer-Frontend](#lehrer-frontend)
|
||||
5. [API-Referenz](#api-referenz)
|
||||
6. [Deployment](#deployment)
|
||||
|
||||
---
|
||||
|
||||
## Architektur
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Frontend (Next.js) │
|
||||
├─────────────────┬─────────────────┬─────────────────────────────┤
|
||||
│ /admin/training │ /admin/zeugnis │ /zeugnisse (Lehrer) │
|
||||
│ Dashboard │ Crawler Admin │ Assistent │
|
||||
└────────┬────────┴────────┬────────┴────────┬────────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ API Proxy (/api/admin/...) │
|
||||
└────────────────────────────┬────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Klausur-Service (FastAPI, Port 8086) │
|
||||
├─────────────────┬─────────────────┬─────────────────────────────┤
|
||||
│ zeugnis_api.py │ zeugnis_ │ zeugnis_models.py │
|
||||
│ (Endpoints) │ crawler.py │ (Datenmodelle) │
|
||||
└────────┬────────┴────────┬────────┴────────┬────────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────────────┐ ┌───────────────┐ ┌───────────────────────────┐
|
||||
│ PostgreSQL │ │ MinIO │ │ Qdrant │
|
||||
│ (Metadata) │ │ (Dokumente) │ │ (Vektoren) │
|
||||
└────────────────┘ └───────────────┘ └───────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rights-Aware Crawler
|
||||
|
||||
### Konzept
|
||||
|
||||
Der Rights-Aware Crawler sammelt automatisch Zeugnisverordnungen und respektiert dabei die Urheberrechte:
|
||||
|
||||
- **Training erlaubt**: Amtliche Werke nach §5 UrhG
|
||||
- **Training nicht erlaubt**: Dokumente ohne explizite Lizenzierung
|
||||
|
||||
### Bundesländer-Berechtigungen
|
||||
|
||||
| Bundesland | Training | Begründung |
|
||||
|------------|----------|------------|
|
||||
| Baden-Württemberg | ✅ Ja | Amtliches Werk |
|
||||
| Bayern | ✅ Ja | Amtliches Werk |
|
||||
| Berlin | ❌ Nein | Keine Lizenz |
|
||||
| Brandenburg | ❌ Nein | Keine Lizenz |
|
||||
| Bremen | ❌ Nein | Eingeschränkt |
|
||||
| Hamburg | ❌ Nein | Keine Lizenz |
|
||||
| Hessen | ✅ Ja | Amtliches Werk |
|
||||
| Mecklenburg-Vorpommern | ❌ Nein | Eingeschränkt |
|
||||
| Niedersachsen | ✅ Ja | Amtliches Werk |
|
||||
| Nordrhein-Westfalen | ✅ Ja | Amtliches Werk |
|
||||
| Rheinland-Pfalz | ✅ Ja | Amtliches Werk |
|
||||
| Saarland | ❌ Nein | Keine Lizenz |
|
||||
| Sachsen | ✅ Ja | Amtliches Werk |
|
||||
| Sachsen-Anhalt | ❌ Nein | Eingeschränkt |
|
||||
| Schleswig-Holstein | ✅ Ja | Amtliches Werk |
|
||||
| Thüringen | ✅ Ja | Amtliches Werk |
|
||||
|
||||
### Admin-Oberfläche
|
||||
|
||||
**URL**: `/admin/zeugnisse-crawler`
|
||||
|
||||
#### Features:
|
||||
- Übersicht aller 16 Bundesländer
|
||||
- Training-Status pro Bundesland
|
||||
- Crawler starten/stoppen
|
||||
- Dokument-Browser
|
||||
- Audit-Trail
|
||||
|
||||
#### Screenshot-Bereiche:
|
||||
1. **Stats-Karten**: Gesamtdokumente, indexierte Dokumente, Training-erlaubt
|
||||
2. **Bundesland-Tabelle**: Status, Dokumentanzahl, letzter Crawl
|
||||
3. **Dokument-Liste**: Filterbar, mit Statusanzeige
|
||||
|
||||
### API-Endpunkte
|
||||
|
||||
```
|
||||
GET /api/v1/admin/zeugnis/sources # Alle Bundesländer
|
||||
POST /api/v1/admin/zeugnis/sources # Neue Quelle
|
||||
PUT /api/v1/admin/zeugnis/sources/{id}/verify # Lizenz verifizieren
|
||||
|
||||
GET /api/v1/admin/zeugnis/crawler/status # Crawler-Status
|
||||
POST /api/v1/admin/zeugnis/crawler/start # Crawler starten
|
||||
POST /api/v1/admin/zeugnis/crawler/stop # Crawler stoppen
|
||||
|
||||
GET /api/v1/admin/zeugnis/documents # Dokumente abrufen
|
||||
GET /api/v1/admin/zeugnis/stats # Statistiken
|
||||
GET /api/v1/admin/zeugnis/audit/events # Audit-Trail
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Training Dashboard
|
||||
|
||||
### Konzept
|
||||
|
||||
Das Training Dashboard ermöglicht die vollständige Kontrolle über KI-Trainingsprozesse:
|
||||
|
||||
- Echtzeit-Monitoring von Trainingsjobs
|
||||
- Konfiguration von Hyperparametern
|
||||
- Visualisierung von Metriken (Loss, Precision, Recall, F1)
|
||||
- Multi-Bundesland Training
|
||||
|
||||
**URL**: `/admin/training`
|
||||
|
||||
### Features
|
||||
|
||||
#### 1. Übersichtskarten
|
||||
- Dokumente gesamt / indexiert
|
||||
- Training-erlaubte Dokumente
|
||||
- Heute gecrawlt / Fehler
|
||||
|
||||
#### 2. Training-Job-Karten
|
||||
Jeder laufende Job zeigt:
|
||||
- **Progress Ring**: Visueller Fortschritt (0-100%)
|
||||
- **Epochen-Fortschritt**: Aktuelle / Gesamt
|
||||
- **Dokument-Fortschritt**: Verarbeitet / Gesamt
|
||||
- **Metriken**: Loss, Val Loss, Precision, F1
|
||||
- **Loss-Chart**: Verlaufsgrafik
|
||||
|
||||
#### 3. Datensatz-Übersicht
|
||||
- Verteilung nach Bundesland (Balkendiagramm)
|
||||
- Verteilung nach Dokumenttyp
|
||||
- Training-erlaubte Quote
|
||||
|
||||
#### 4. Neues Training starten (Wizard)
|
||||
|
||||
**Schritt 1 - Datenauswahl:**
|
||||
- Checkbox-Grid für Bundesländer
|
||||
- Nur Training-erlaubte wählbar
|
||||
- Deaktivierte Anzeige für nicht-erlaubte
|
||||
|
||||
**Schritt 2 - Parameter:**
|
||||
- Batch Size (Standard: 16)
|
||||
- Learning Rate (Standard: 0.00005)
|
||||
- Epochen (Standard: 10)
|
||||
- Warmup Steps (Standard: 500)
|
||||
- Mixed Precision (FP16) Toggle
|
||||
|
||||
**Schritt 3 - Bestätigung:**
|
||||
- Zusammenfassung aller Einstellungen
|
||||
- Geschätzte Trainingszeit
|
||||
- Start-Button
|
||||
|
||||
### Training-Kontrolle
|
||||
|
||||
| Aktion | Beschreibung |
|
||||
|--------|--------------|
|
||||
| Pausieren | Training temporär unterbrechen |
|
||||
| Fortsetzen | Pausiertes Training fortsetzen |
|
||||
| Abbrechen | Training beenden (Daten bleiben) |
|
||||
| Details | Erweiterte Metriken anzeigen |
|
||||
|
||||
---
|
||||
|
||||
## Lehrer-Frontend
|
||||
|
||||
### Konzept
|
||||
|
||||
Das Lehrer-Frontend bietet eine intuitive Oberfläche für Lehrkräfte zur Recherche in Zeugnisverordnungen.
|
||||
|
||||
**URL**: `/zeugnisse`
|
||||
|
||||
### Onboarding-Wizard
|
||||
|
||||
Beim ersten Besuch führt ein 4-stufiger Wizard durch die Einrichtung:
|
||||
|
||||
#### Schritt 1: Willkommen
|
||||
- Vorstellung des Assistenten
|
||||
- Feature-Übersicht (Suche, KI-Antworten, 16 Bundesländer)
|
||||
|
||||
#### Schritt 2: Bundesland
|
||||
- Grid mit allen 16 Bundesländern
|
||||
- Emoji-Icons für visuelle Zuordnung
|
||||
- Auswahl speichert bevorzugtes Bundesland
|
||||
|
||||
#### Schritt 3: Schulform
|
||||
- Auswahl der Schulform:
|
||||
- Grundschule
|
||||
- Hauptschule
|
||||
- Realschule
|
||||
- Gymnasium
|
||||
- Gesamtschule
|
||||
- Förderschule
|
||||
- Berufsschule
|
||||
|
||||
#### Schritt 4: Fertig
|
||||
- Zusammenfassung der Einstellungen
|
||||
- Option zum Ändern
|
||||
|
||||
### Hauptfunktionen
|
||||
|
||||
#### Tab 1: Assistent (Chat)
|
||||
- KI-gestützter Dialog
|
||||
- Fragen in natürlicher Sprache
|
||||
- Quellenangaben zu Antworten
|
||||
- Vorgeschlagene Fragen
|
||||
|
||||
#### Tab 2: Suche
|
||||
- Volltextsuche in Verordnungen
|
||||
- Letzte Suchen (gespeichert)
|
||||
- Häufige Fragen als Schnellauswahl
|
||||
- Relevanz-Score pro Ergebnis
|
||||
|
||||
#### Tab 3: Dokumente
|
||||
- Dokumenten-Browser
|
||||
- Filterbar nach Bundesland
|
||||
- Download-Möglichkeit
|
||||
|
||||
### Häufige Fragen (vordefiniert)
|
||||
|
||||
1. "Wie formuliere ich eine Bemerkung zur Arbeits- und Sozialverhalten?"
|
||||
2. "Welche Noten dürfen im Zeugnis stehen?"
|
||||
3. "Wann sind Zeugniskonferenzen durchzuführen?"
|
||||
4. "Wie gehe ich mit Fehlzeiten um?"
|
||||
5. "Welche Unterschriften sind erforderlich?"
|
||||
6. "Wie werden Versetzungsentscheidungen dokumentiert?"
|
||||
|
||||
### Benutzereinstellungen
|
||||
|
||||
Gespeichert im localStorage:
|
||||
- Bundesland
|
||||
- Schulform
|
||||
- Wizard-Status
|
||||
- Favoriten
|
||||
- Letzte Suchen
|
||||
|
||||
---
|
||||
|
||||
## API-Referenz
|
||||
|
||||
### Zeugnis-Crawler API
|
||||
|
||||
#### GET /api/v1/admin/zeugnis/sources
|
||||
Listet alle Bundesland-Quellen.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
[
|
||||
{
|
||||
"id": "uuid",
|
||||
"bundesland": "ni",
|
||||
"name": "Niedersachsen",
|
||||
"license_type": "gov_statute",
|
||||
"training_allowed": true,
|
||||
"verified_by": "admin@example.com",
|
||||
"verified_at": "2024-01-15T10:00:00Z"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
#### POST /api/v1/admin/zeugnis/crawler/start
|
||||
Startet den Crawler.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"bundesland": "ni", // Optional: Nur dieses Bundesland
|
||||
"priority": 5 // 1-10, Standard: 5
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Crawler started"
|
||||
}
|
||||
```
|
||||
|
||||
#### GET /api/v1/admin/zeugnis/stats
|
||||
Holt Statistiken.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"total_sources": 16,
|
||||
"total_documents": 632,
|
||||
"indexed_documents": 489,
|
||||
"training_allowed_documents": 423,
|
||||
"active_crawls": 1,
|
||||
"per_bundesland": [...]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
### Voraussetzungen
|
||||
|
||||
- Docker & Docker Compose
|
||||
- PostgreSQL 16+
|
||||
- Qdrant 1.12+
|
||||
- MinIO
|
||||
|
||||
### Umgebungsvariablen
|
||||
|
||||
```env
|
||||
# Klausur-Service
|
||||
QDRANT_URL=http://qdrant:6333
|
||||
MINIO_ENDPOINT=minio:9000
|
||||
MINIO_ACCESS_KEY=breakpilot
|
||||
MINIO_SECRET_KEY=breakpilot123
|
||||
MINIO_BUCKET=breakpilot-rag
|
||||
EMBEDDING_BACKEND=local # oder "openai"
|
||||
|
||||
# Website
|
||||
KLAUSUR_SERVICE_URL=http://klausur-service:8086
|
||||
```
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```yaml
|
||||
services:
|
||||
qdrant:
|
||||
image: qdrant/qdrant:v1.12.1
|
||||
|
||||
minio:
|
||||
image: minio/minio:latest
|
||||
|
||||
klausur-service:
|
||||
build: ./klausur-service
|
||||
depends_on:
|
||||
- qdrant
|
||||
- minio
|
||||
```
|
||||
|
||||
### Initialisierung
|
||||
|
||||
1. Container starten:
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
2. Datenbank initialisieren:
|
||||
```bash
|
||||
curl -X POST http://localhost:8086/api/v1/admin/zeugnis/init
|
||||
```
|
||||
|
||||
3. Crawler starten:
|
||||
```bash
|
||||
curl -X POST http://localhost:8086/api/v1/admin/zeugnis/crawler/start
|
||||
```
|
||||
|
||||
4. Frontend öffnen:
|
||||
- Admin: http://localhost:3000/admin/zeugnisse-crawler
|
||||
- Training: http://localhost:3000/admin/training
|
||||
- Lehrer: http://localhost:3000/zeugnisse
|
||||
|
||||
---
|
||||
|
||||
## Sicherheit
|
||||
|
||||
### Datenschutz
|
||||
|
||||
- **Audit-Trail**: Alle Zugriffe werden protokolliert
|
||||
- **DSGVO-Export**: `/api/v1/admin/zeugnis/audit/export`
|
||||
- **Verschlüsselung**: Sensible Daten in MinIO verschlüsselt
|
||||
|
||||
### Rights Management
|
||||
|
||||
- Training nur mit expliziter Erlaubnis
|
||||
- Lizenz-Verifizierung durch Admin
|
||||
- Quellen-Nachverfolgung für alle Dokumente
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
Bei Fragen oder Problemen:
|
||||
- GitHub Issues: https://github.com/...
|
||||
- E-Mail: support@breakpilot.app
|
||||
Reference in New Issue
Block a user