40 Commits

Author SHA1 Message Date
Benjamin Admin
96f94475f6 fix: downgrade to PaddleOCR 2.x — 3.x uses too much RAM on CPU
All checks were successful
CI / go-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 33s
CI / test-python-voice (push) Successful in 31s
CI / test-bqas (push) Successful in 34s
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / Deploy (push) Successful in 2s
PaddlePaddle 3.x + PP-OCRv5 requires >6GB RAM and has oneDNN
compatibility issues on CPU. PaddleOCR 2.x with PP-OCRv4 works
reliably with ~2-3GB RAM and has no MKLDNN issues.

- Pin paddlepaddle<3.0.0 and paddleocr<3.0.0
- Simplify main.py — single init strategy, direct 2.x result format
- Re-enable warmup (fits in memory with 2.x)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:13:33 +01:00
Benjamin Admin
3fd3336f6c fix: force-disable oneDNN via paddle.set_flags and enable_mkldnn=False
All checks were successful
CI / go-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 34s
CI / test-python-voice (push) Successful in 32s
CI / test-bqas (push) Successful in 32s
CI / Deploy (push) Successful in 2s
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
Previous FLAGS_use_mkldnn env var was ignored by PaddlePaddle 3.x.
Now using paddle.set_flags() API and PaddleOCR enable_mkldnn param.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:01:46 +01:00
Benjamin Admin
eaba087d11 fix: disable oneDNN/MKLDNN and support PaddleOCR 3.x result format
All checks were successful
CI / test-go-consent (push) Successful in 31s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-python-voice (push) Successful in 1m19s
CI / test-bqas (push) Successful in 32s
CI / Deploy (push) Successful in 2s
- Set FLAGS_use_mkldnn=0 before paddle import to avoid
  ConvertPirAttribute2RuntimeAttribute error
- Support both PaddleOCR 2.x (list) and 3.x (dict) result formats
- Use use_textline_orientation (3.x) instead of use_angle_cls
- Remove latin lang fallback (not supported in 3.x)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:52:31 +01:00
Benjamin Admin
ed2cc234b8 fix: add error handling and logging to OCR endpoint
All checks were successful
CI / nodejs-lint (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 31s
CI / test-python-voice (push) Successful in 32s
CI / test-bqas (push) Successful in 33s
CI / Deploy (push) Successful in 2s
Return detailed error message instead of generic 500, and handle
empty OCR results gracefully.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:37:32 +01:00
Benjamin Admin
ffd3fd1d7c fix: remove warmup OCR call — causes OOM on 6G container
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 38s
CI / test-python-voice (push) Successful in 38s
CI / test-bqas (push) Successful in 50s
CI / Deploy (push) Successful in 2s
The warmup OCR call during startup pushes memory over 6G and causes
OOM kills + restart loops. First real OCR request will be slow
(JIT compilation) but container stays stable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:24:55 +01:00
Benjamin Admin
23694b6555 fix: increase paddleocr memory limit 4G → 6G
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 34s
CI / test-python-voice (push) Successful in 32s
CI / test-bqas (push) Successful in 33s
CI / Deploy (push) Successful in 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:08:33 +01:00
Benjamin Admin
8979aa8e43 fix: add warmup OCR call to avoid timeout on first request
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 43s
CI / test-python-voice (push) Successful in 35s
CI / test-bqas (push) Successful in 34s
CI / Deploy (push) Successful in 3s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:56:08 +01:00
Benjamin Admin
c433bc021e docs: add post-push deploy monitoring to CLAUDE.md
All checks were successful
CI / Deploy (push) Successful in 2s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 32s
CI / test-python-voice (push) Successful in 33s
CI / test-bqas (push) Successful in 35s
After every push to gitea, Claude automatically polls health endpoints
and notifies the user when deployment is ready for testing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:45:09 +01:00
Benjamin Admin
f4ed1eb10c feat: add paddleocr-service to Coolify compose
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 30s
CI / test-python-voice (push) Successful in 34s
CI / test-bqas (push) Successful in 32s
CI / Deploy (push) Successful in 2s
Add PaddleOCR PP-OCRv5 service with 4G memory limit, model volume,
and health check (5min start period for model loading). Domain routing
(ocr.breakpilot.com) to be configured in Coolify UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:43:11 +01:00
Benjamin Admin
9c8663a0f1 Merge gitea/main: accept Coolify compose config
All checks were successful
CI / nodejs-lint (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 36s
CI / test-python-voice (push) Successful in 40s
CI / test-bqas (push) Successful in 32s
CI / Deploy (push) Successful in 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:27:29 +01:00
Benjamin Admin
d1632fca17 docs: update all docs to reflect Coolify deployment model
Replace Hetzner references with Coolify. Deployment is now:
- Core + Compliance: Push gitea → Coolify auto-deploys
- Lehrer: stays local on Mac Mini

Updated: CLAUDE.md, MkDocs CI/CD pipeline, MkDocs index, environments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:18:25 +01:00
fcf8aa8652 fix: migrate deployment from Hetzner to Coolify (#1)
All checks were successful
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 31s
CI / test-python-voice (push) Successful in 32s
CI / test-bqas (push) Successful in 28s
CI / Deploy (push) Successful in 2s
## Summary
- Add Coolify deployment configuration (docker-compose, healthchecks, network setup)
- Replace deploy-hetzner CI job with Coolify webhook deploy
- Externalize postgres, qdrant, S3 for Coolify environment
- Remove services not needed for SDK deployment (voice, jitsi, synapse)

## All changes since branch creation
- Coolify docker-compose with healthchecks for all services
- CI pipeline: deploy-hetzner → deploy-coolify (simple webhook curl)
- QDRANT_API_KEY support in rag-service
- Alpine-compatible Dockerfile fixes

Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com>
Reviewed-on: #1
2026-03-13 10:45:18 +00:00
Benjamin Admin
65177d3ff7 fix: robust PaddleOCR init with multiple fallback strategies
Some checks failed
CI / go-lint (pull_request) Failing after 2s
CI / python-lint (pull_request) Failing after 11s
CI / nodejs-lint (pull_request) Failing after 2s
CI / test-go-consent (pull_request) Failing after 2s
CI / test-python-voice (pull_request) Failing after 14s
CI / test-bqas (pull_request) Failing after 11s
CI / deploy-hetzner (pull_request) Has been skipped
Deploy to Coolify / deploy (push) Has been cancelled
PaddleOCR 3.x removed show_log param and lang='latin'. Try multiple
init strategies in order until one succeeds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:09:33 +01:00
Benjamin Admin
559d6a351c fix: resolve stash conflict
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
CI / go-lint (pull_request) Failing after 2s
CI / python-lint (pull_request) Failing after 14s
CI / nodejs-lint (pull_request) Failing after 3s
CI / test-go-consent (pull_request) Failing after 3s
CI / test-python-voice (pull_request) Failing after 11s
CI / test-bqas (pull_request) Failing after 10s
CI / deploy-hetzner (pull_request) Has been skipped
2026-03-13 10:59:30 +01:00
Benjamin Admin
8fd11998e4 merge: resolve docker-compose.coolify.yml conflict (accept remote) 2026-03-13 10:56:36 +01:00
Benjamin Admin
4ce649aa71 fix: upgrade PaddleOCR to 3.x for PP-OCRv5 and stability
Old paddlepaddle==2.6.2 + paddleocr==2.8.1 caused hangs on first OCR
request. Upgrading to paddlepaddle>=3.0.0 + paddleocr>=2.9.0 enables
native PP-OCRv5 support and fixes stability issues.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:53:18 +01:00
Benjamin Admin
5ee3cc0104 fix: load PaddleOCR model in background thread
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
The import and model loading can take minutes and was blocking
the startup event, causing health checks to timeout. Now loads
in a background thread — health endpoint returns 200 immediately
with status 'loading' until model is ready.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:21:59 +01:00
Benjamin Admin
b36712247b fix: add detailed logging for PaddleOCR model loading debug
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:19:10 +01:00
Benjamin Admin
86b11c7e5f fix: catch all exceptions in PaddleOCR version fallback
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
PaddleOCR 2.8.1 throws a generic Exception (not ValueError) when
ocr_version='PP-OCRv5' is used. Broadened except clause to catch
any error and fall back to lang='latin' for older versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:12:32 +01:00
Benjamin Admin
8003dcac39 fix: PaddleOCR 3.4.0 compatibility — use lang=en with PP-OCRv5
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
PaddleOCR 3.4.0 removed 'latin' language support, causing ValueError
at startup. Now uses lang='en' with ocr_version='PP-OCRv5' and falls
back to lang='latin' for older PaddleOCR versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:54:52 +01:00
Benjamin Admin
778c44226e fix: expose port 8095 directly (bypass Traefik 60s timeout)
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 14:16:04 +01:00
Benjamin Admin
79891063dd fix: pin PaddlePaddle 2.6.2 + PaddleOCR 2.8.1 (stable, no PIR bug)
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
PaddlePaddle 3.x hat oneDNN/PIR Executor Bug. Zurueck auf 2.6.2
mit bewaeherter ocr() API statt predict().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:32:54 +01:00
Benjamin Admin
2c9b0dc448 fix: disable oneDNN (FLAGS_use_mkldnn=0) for PaddlePaddle compat
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:25:36 +01:00
Benjamin Admin
3133615044 fix: add libgomp1 (OpenMP) + remove unused lang parameter
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
PaddlePaddle braucht libgomp.so.1 fuer Inferenz.
lang wird ignoriert bei explizitem model_name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:19:47 +01:00
Benjamin Admin
2bc0f87325 fix: PaddleOCR model pre-load at startup + 5min healthcheck grace
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Model wird beim Container-Start geladen (nicht erst beim ersten Request).
Health-Check start_period auf 300s erhoeht fuer initialen Download.
/health gibt "loading" zurueck bis Modell bereit ist.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:12:14 +01:00
Benjamin Admin
4ee38d6f0b fix: remove show_log (unknown in PaddleOCR v3 API)
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:52:52 +01:00
Benjamin Admin
992d4f2a6b fix: PaddleOCR v3 API — explicit model name + predict() statt ocr()
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
lang="latin" braucht text_recognition_model_name in PP-OCRv5.
Neue API nutzt predict() statt ocr(), Ergebnis-Format angepasst.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:47:07 +01:00
Benjamin Admin
8f5f9641c7 fix: libgl1-mesa-glx → libgl1 (Debian bookworm)
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 10:33:28 +01:00
Benjamin Admin
7cdb53051f feat: PaddleOCR Service (PP-OCRv5 Latin auf x86_64)
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Microservice fuer PaddleOCR auf Hetzner. FastAPI mit /ocr und /health
Endpoints, API-Key Auth, 4GB Memory Limit, Modell-Cache Volume.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 10:20:41 +01:00
Sharang Parnerkar
d834753a98 Remove services not needed by SDK from Coolify deployment
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Remove backend-core, billing-service, night-scheduler, and admin-core
as they are not used by any compliance/SDK service. Update
health-aggregator CHECK_SERVICES to reference consent-service instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 13:29:23 +01:00
Sharang Parnerkar
395011d0f4 Add healthchecks to backend-core, consent-service, billing-service, admin-core
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Coolify/Traefik requires healthchecks to route traffic to containers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 00:57:43 +01:00
Sharang Parnerkar
9e1660f954 Remove Traefik labels from coolify compose — Coolify handles routing
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 00:05:26 +01:00
Sharang Parnerkar
13ff930b5e Fix admin-core build: ensure public directory exists before build
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:44:30 +01:00
Sharang Parnerkar
5d1c837f49 Fix backend-core TARGETARCH: auto-detect instead of hardcoded arm64
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
The Dockerfile hardcoded TARGETARCH=arm64 for Mac Mini. Coolify server
is x86_64, causing exit code 126 (wrong binary arch). Now uses Docker
BuildKit's auto-detected TARGETARCH with dpkg fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:37:59 +01:00
Sharang Parnerkar
1dd9662037 Add QDRANT_API_KEY support to rag-service
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
- Add QDRANT_API_KEY to config.py (empty string = no auth)
- Pass api_key to QdrantClient constructor (None when empty)
- Add QDRANT_API_KEY to coolify compose and env example

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:35:11 +01:00
Sharang Parnerkar
4626edb232 Sync coolify compose with main: remove voice-service, update rag/embedding
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
- Remove voice-service (removed in main branch)
- Remove voice_session_data volume
- Add OLLAMA_URL and OLLAMA_EMBED_MODEL to rag-service
- Update embedding-service default model to BAAI/bge-m3, memory 4G→8G
- Update health-aggregator CHECK_SERVICES (remove voice-service)
- Update .env.coolify.example accordingly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 23:23:52 +01:00
Sharang Parnerkar
3c29b621ac Merge remote-tracking branch 'origin/main' into coolify 2026-03-07 23:10:41 +01:00
Sharang Parnerkar
755570d474 fix: use Alpine-compatible addgroup/adduser flags in Dockerfiles
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Replace --system/--gid/--uid (Debian syntax) with -S/-g/-u (BusyBox/Alpine).
Coolify ARG injection causes exit code 255 with Debian-style flags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 22:38:31 +01:00
Sharang Parnerkar
e890b1490a refactor(coolify): externalize postgres, qdrant, S3; remove jitsi/synapse
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
- Remove PostgreSQL, Qdrant, MinIO services (managed separately in Coolify)
- Remove Jitsi stack (web, xmpp, jicofo, jvb) and Synapse/synapse-db
- Add POSTGRES_HOST, QDRANT_URL, S3_ENDPOINT/S3_ACCESS_KEY/S3_SECRET_KEY env vars
- Remove Traefik labels from internal-only services
- Health aggregator no longer checks external services
- Core now has 10 services: valkey + 9 application services

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 09:23:20 +01:00
Sharang Parnerkar
d15de16c47 feat: add Coolify deployment configuration
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
Add docker-compose.coolify.yml (17 services), .env.coolify.example,
and Gitea Action workflow for Coolify API deployment. Removes nginx,
vault, gitea, woodpecker, mailpit, and dev-only services. Adds Traefik
labels for *.breakpilot.ai domain routing with Let's Encrypt SSL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 10:43:02 +01:00
9 changed files with 463 additions and 576 deletions

View File

@@ -2,28 +2,53 @@
## Entwicklungsumgebung (WICHTIG - IMMER ZUERST LESEN) ## Entwicklungsumgebung (WICHTIG - IMMER ZUERST LESEN)
### Zwei-Rechner-Setup ### Zwei-Rechner-Setup + Coolify
| Geraet | Rolle | Aufgaben | | Geraet | Rolle | Aufgaben |
|--------|-------|----------| |--------|-------|----------|
| **MacBook** | Entwicklung | Claude Terminal, Code-Entwicklung, Browser (Frontend-Tests) | | **MacBook** | Entwicklung | Claude Terminal, Code-Entwicklung, Browser (Frontend-Tests) |
| **Mac Mini** | Server | Docker, alle Services, Tests, Builds, Deployment | | **Mac Mini** | Lokaler Server | Docker fuer lokale Dev/Tests (NICHT fuer Production!) |
| **Coolify** | Production | Automatisches Build + Deploy bei Push auf gitea |
**WICHTIG:** Code wird direkt auf dem MacBook in diesem Repo bearbeitet. Docker und Services laufen auf dem Mac Mini. **WICHTIG:** Code wird direkt auf dem MacBook in diesem Repo bearbeitet. Production-Deployment laeuft automatisch ueber Coolify.
### Entwicklungsworkflow ### Entwicklungsworkflow (CI/CD — Coolify)
```bash ```bash
# 1. Code auf MacBook bearbeiten (dieses Verzeichnis) # 1. Code auf MacBook bearbeiten (dieses Verzeichnis)
# 2. Committen und pushen: # 2. Committen und zu BEIDEN Remotes pushen:
git push origin main && git push gitea main git push origin main && git push gitea main
# 3. Auf Mac Mini pullen und Container neu bauen: # 3. FERTIG! Push auf gitea triggert automatisch:
# - Gitea Actions: Tests
# - Coolify: Build → Deploy
```
**NIEMALS** manuell in Coolify auf "Redeploy" klicken — Gitea Actions triggert Coolify automatisch.
**IMMER auf `main` pushen** — sowohl origin als auch gitea.
### Post-Push Deploy-Monitoring (PFLICHT nach jedem Push auf gitea)
**IMMER wenn Claude auf gitea pusht, MUSS danach automatisch das Deploy-Monitoring laufen:**
1. Dem User sofort mitteilen: "Deploy gestartet, ich ueberwache den Status..."
2. Im Hintergrund Health-Checks pollen (alle 20 Sekunden, max 5 Minuten):
```bash
curl -sf https://api-dev.breakpilot.ai/health # Compliance Backend
curl -sf https://sdk-dev.breakpilot.ai/health # AI SDK
```
3. Sobald ALLE Endpoints healthy sind, dem User im Chat melden:
**"Deploy abgeschlossen! Du kannst jetzt testen."**
4. Falls nach 5 Minuten noch nicht healthy → Fehlermeldung mit Hinweis auf Coolify-Logs.
### Lokale Entwicklung (Mac Mini — optional, nur Dev/Tests)
```bash
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && git pull --no-rebase origin main" ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && git pull --no-rebase origin main"
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && /usr/local/bin/docker compose build --no-cache <service> && /usr/local/bin/docker compose up -d <service>" ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && /usr/local/bin/docker compose build --no-cache <service> && /usr/local/bin/docker compose up -d <service>"
``` ```
### SSH-Verbindung (fuer Docker/Tests) ### SSH-Verbindung (fuer lokale Docker/Tests)
```bash ```bash
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && <cmd>" ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && <cmd>"
@@ -51,6 +76,14 @@ networks:
name: breakpilot-network # Fixer Name, kein Auto-Prefix! name: breakpilot-network # Fixer Name, kein Auto-Prefix!
``` ```
### Deployment-Modell
| Repo | Deployment | Trigger |
|------|-----------|---------|
| **breakpilot-core** | Coolify (automatisch) | Push auf gitea main |
| **breakpilot-compliance** | Coolify (automatisch) | Push auf gitea main |
| **breakpilot-lehrer** | Mac Mini (lokal) | Manuell docker compose |
--- ---
## Haupt-URLs (via Nginx Reverse Proxy) ## Haupt-URLs (via Nginx Reverse Proxy)
@@ -161,7 +194,7 @@ networks:
| `compliance` | Compliance | compliance_*, dsr, gdpr, sdk_tenants, consent_admin | | `compliance` | Compliance | compliance_*, dsr, gdpr, sdk_tenants, consent_admin |
```bash ```bash
# DB-Zugang # DB-Zugang (lokal)
ssh macmini "docker exec bp-core-postgres psql -U breakpilot -d breakpilot_db" ssh macmini "docker exec bp-core-postgres psql -U breakpilot -d breakpilot_db"
``` ```
@@ -193,7 +226,14 @@ breakpilot-core/
## Haeufige Befehle ## Haeufige Befehle
### Docker ### Deployment (CI/CD — Standardweg)
```bash
# Committen und pushen → Coolify deployt automatisch:
git push origin main && git push gitea main
```
### Lokale Docker-Befehle (Mac Mini — nur Dev/Tests)
```bash ```bash
# Alle Core-Services starten # Alle Core-Services starten
@@ -211,31 +251,15 @@ ssh macmini "/usr/local/bin/docker ps --filter name=bp-core"
**WICHTIG:** Docker-Pfad auf Mac Mini ist `/usr/local/bin/docker` (nicht im Standard-SSH-PATH). **WICHTIG:** Docker-Pfad auf Mac Mini ist `/usr/local/bin/docker` (nicht im Standard-SSH-PATH).
### Alle 3 Projekte starten
```bash
# 1. Core (MUSS zuerst!)
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && /usr/local/bin/docker compose up -d"
# Warten auf Health:
ssh macmini "curl -sf http://127.0.0.1:8099/health"
# 2. Lehrer
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-lehrer && /usr/local/bin/docker compose up -d"
# 3. Compliance
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-compliance && /usr/local/bin/docker compose up -d"
```
### Git ### Git
```bash ```bash
# Zu BEIDEN Remotes pushen (PFLICHT!): # Zu BEIDEN Remotes pushen (PFLICHT!):
ssh macmini "cd /Users/benjaminadmin/Projekte/breakpilot-core && git push all main" git push origin main && git push gitea main
# Remotes: # Remotes:
# origin: lokale Gitea (macmini:3003) # origin: lokale Gitea (macmini:3003)
# gitea: gitea.meghsakha.com # gitea: gitea.meghsakha.com
# all: beide gleichzeitig
``` ```
--- ---

View File

@@ -140,117 +140,20 @@ jobs:
python -m pytest tests/bqas/ -v --tb=short || true python -m pytest tests/bqas/ -v --tb=short || true
# ======================================== # ========================================
# Build & Deploy auf Hetzner (nur main, kein PR) # Deploy via Coolify (nur main, kein PR)
# ======================================== # ========================================
deploy-hetzner: deploy-coolify:
name: Deploy
runs-on: docker runs-on: docker
if: github.event_name == 'push' && github.ref == 'refs/heads/main' if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: needs:
- test-go-consent - test-go-consent
container: docker:27-cli container:
image: alpine:latest
steps: steps:
- name: Deploy - name: Trigger Coolify deploy
run: | run: |
set -euo pipefail apk add --no-cache curl
DEPLOY_DIR="/opt/breakpilot-core" curl -sf "${{ secrets.COOLIFY_WEBHOOK }}" \
COMPOSE_FILES="-f docker-compose.yml -f docker-compose.hetzner.yml" -H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
COMMIT_SHA="${GITHUB_SHA:-unknown}"
SHORT_SHA="${COMMIT_SHA:0:8}"
REPO_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
# Services die deployed werden
SERVICES="postgres valkey qdrant minio ollama mailpit embedding-service rag-service backend-core consent-service health-aggregator"
echo "=== BreakPilot Core Deploy ==="
echo "Commit: ${SHORT_SHA}"
echo "Deploy Dir: ${DEPLOY_DIR}"
echo "Services: ${SERVICES}"
echo ""
# 1. Repo auf dem Host erstellen/aktualisieren via Helper-Container
echo "=== Updating code on host ==="
docker run --rm \
-v "${DEPLOY_DIR}:${DEPLOY_DIR}" \
--entrypoint sh \
alpine/git:latest \
-c "
if [ ! -d '${DEPLOY_DIR}/.git' ]; then
echo 'Erstmaliges Klonen nach ${DEPLOY_DIR}...'
git clone '${REPO_URL}' '${DEPLOY_DIR}'
else
cd '${DEPLOY_DIR}'
git fetch origin main
git reset --hard origin/main
fi
"
echo "Code aktualisiert auf ${SHORT_SHA}"
# 2. .env sicherstellen
docker run --rm -v "${DEPLOY_DIR}:${DEPLOY_DIR}" alpine \
sh -c "
if [ ! -f '${DEPLOY_DIR}/.env' ]; then
echo 'WARNUNG: ${DEPLOY_DIR}/.env fehlt!'
echo 'Erstelle .env aus .env.example mit Defaults...'
if [ -f '${DEPLOY_DIR}/.env.example' ]; then
cp '${DEPLOY_DIR}/.env.example' '${DEPLOY_DIR}/.env'
echo '.env aus .env.example erstellt'
else
echo 'Kein .env.example gefunden — Services starten mit Defaults'
fi
else
echo '.env vorhanden'
fi
"
# 3. Shared Network erstellen (falls noch nicht vorhanden)
docker network create breakpilot-network 2>/dev/null || true
# 4. Build + Deploy via Helper-Container
echo ""
echo "=== Building + Deploying ==="
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "${DEPLOY_DIR}:${DEPLOY_DIR}" \
-w "${DEPLOY_DIR}" \
docker:27-cli \
sh -c "
set -e
COMPOSE_FILES='-f docker-compose.yml -f docker-compose.hetzner.yml'
echo '=== Building Docker Images ==='
docker compose \${COMPOSE_FILES} build --parallel \
backend-core consent-service rag-service embedding-service health-aggregator
echo ''
echo '=== Starting infrastructure ==='
docker compose \${COMPOSE_FILES} up -d postgres valkey qdrant minio mailpit
echo 'Warte auf DB + Cache...'
sleep 10
echo ''
echo '=== Starting Ollama + pulling bge-m3 ==='
docker compose \${COMPOSE_FILES} up -d ollama
sleep 5
# bge-m3 Modell pullen (nur beim ersten Mal ~670MB)
echo 'Pulling bge-m3 model (falls noch nicht vorhanden)...'
docker exec bp-core-ollama ollama pull bge-m3 2>&1 || echo 'WARNUNG: bge-m3 pull fehlgeschlagen (wird spaeter nachgeholt)'
echo ''
echo '=== Starting application services ==='
docker compose \${COMPOSE_FILES} up -d \
embedding-service rag-service backend-core consent-service health-aggregator
echo ''
echo '=== Health Checks ==='
sleep 15
for svc in bp-core-postgres bp-core-valkey bp-core-qdrant bp-core-ollama bp-core-embedding-service bp-core-rag-service bp-core-backend bp-core-consent-service bp-core-health; do
STATUS=\$(docker inspect --format='{{.State.Status}}' \"\${svc}\" 2>/dev/null || echo 'not found')
echo \"\${svc}: \${STATUS}\"
done
"
echo ""
echo "=== Deploy abgeschlossen: ${SHORT_SHA} ==="

View File

@@ -15,6 +15,7 @@ networks:
volumes: volumes:
valkey_data: valkey_data:
embedding_models: embedding_models:
paddleocr_models:
services: services:
@@ -141,6 +142,37 @@ services:
networks: networks:
- breakpilot-network - breakpilot-network
# =========================================================
# OCR SERVICE (PaddleOCR PP-OCRv5)
# =========================================================
paddleocr-service:
build:
context: ./paddleocr-service
dockerfile: Dockerfile
container_name: bp-core-paddleocr
expose:
- "8095"
environment:
PADDLEOCR_API_KEY: ${PADDLEOCR_API_KEY:-}
FLAGS_use_mkldnn: "0"
volumes:
- paddleocr_models:/root/.paddleocr
labels:
- "traefik.http.services.paddleocr.loadbalancer.server.port=8095"
deploy:
resources:
limits:
memory: 6G
healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8095/health"]
interval: 30s
timeout: 10s
start_period: 300s
retries: 5
restart: unless-stopped
networks:
- breakpilot-network
# ========================================================= # =========================================================
# HEALTH AGGREGATOR # HEALTH AGGREGATOR
# ========================================================= # =========================================================
@@ -153,7 +185,7 @@ services:
- "8099" - "8099"
environment: environment:
PORT: 8099 PORT: 8099
CHECK_SERVICES: "valkey:6379,consent-service:8081,rag-service:8097,embedding-service:8087" CHECK_SERVICES: "valkey:6379,consent-service:8081,rag-service:8097,embedding-service:8087,paddleocr-service:8095"
healthcheck: healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8099/health"] test: ["CMD", "curl", "-f", "http://127.0.0.1:8099/health"]
interval: 30s interval: 30s

View File

@@ -1,194 +1,77 @@
# Umgebungs-Architektur # Umgebungs-Architektur
## Übersicht ## Uebersicht
BreakPilot verwendet eine 3-Umgebungs-Strategie für sichere Entwicklung und Deployment: BreakPilot verwendet zwei Umgebungen:
``` ```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Development │────▶│ Staging │────▶│ Production │ │ Development │───── git push ────▶│ Production │
│ (develop) (staging) (main) │ (Mac Mini) (Coolify)
└─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
Tägliche Getesteter Code Produktionsreif Lokale Automatisch
Entwicklung Entwicklung via Coolify
``` ```
## Umgebungen ## Umgebungen
### Development (Dev) ### Development (Lokal — Mac Mini)
**Zweck:** Tägliche Entwicklungsarbeit **Zweck:** Lokale Entwicklung und Tests
| Eigenschaft | Wert | | Eigenschaft | Wert |
|-------------|------| |-------------|------|
| Git Branch | `develop` | | Git Branch | `main` |
| Compose File | `docker-compose.yml` + `docker-compose.override.yml` (auto) | | Compose File | `docker-compose.yml` |
| Env File | `.env.dev` | | Database | Lokale PostgreSQL |
| Database | `breakpilot_dev` |
| Debug | Aktiviert | | Debug | Aktiviert |
| Hot-Reload | Aktiviert | | Hot-Reload | Aktiviert |
**Start:** **Start:**
```bash ```bash
./scripts/start.sh dev ssh macmini "cd ~/Projekte/breakpilot-core && /usr/local/bin/docker compose up -d"
# oder einfach:
docker compose up -d
``` ```
### Staging ### Production (Coolify)
**Zweck:** Getesteter, freigegebener Code vor Produktion **Zweck:** Live-System
| Eigenschaft | Wert |
|-------------|------|
| Git Branch | `staging` |
| Compose File | `docker-compose.yml` + `docker-compose.staging.yml` |
| Env File | `.env.staging` |
| Database | `breakpilot_staging` (separates Volume) |
| Debug | Deaktiviert |
| Hot-Reload | Deaktiviert |
**Start:**
```bash
./scripts/start.sh staging
# oder:
docker compose -f docker-compose.yml -f docker-compose.staging.yml up -d
```
### Production (Prod)
**Zweck:** Live-System für Endbenutzer (ab Launch)
| Eigenschaft | Wert | | Eigenschaft | Wert |
|-------------|------| |-------------|------|
| Git Branch | `main` | | Git Branch | `main` |
| Compose File | `docker-compose.yml` + `docker-compose.prod.yml` | | Deployment | Coolify (automatisch bei Push auf gitea) |
| Env File | `.env.prod` (NICHT im Repository!) | | Database | Externe PostgreSQL (TLS) |
| Database | `breakpilot_prod` (separates Volume) |
| Debug | Deaktiviert | | Debug | Deaktiviert |
| Vault | Pflicht (keine Env-Fallbacks) |
## Datenbank-Trennung
Jede Umgebung verwendet separate Docker Volumes für vollständige Datenisolierung:
```
┌─────────────────────────────────────────────────────────────┐
│ PostgreSQL Volumes │
├─────────────────────────────────────────────────────────────┤
│ breakpilot-dev_postgres_data │ Development Database │
│ breakpilot_staging_postgres │ Staging Database │
│ breakpilot_prod_postgres │ Production Database │
└─────────────────────────────────────────────────────────────┘
```
## Port-Mapping
Um mehrere Umgebungen gleichzeitig laufen zu lassen, verwenden sie unterschiedliche Ports:
| Service | Dev Port | Staging Port | Prod Port |
|---------|----------|--------------|-----------|
| Backend | 8000 | 8001 | 8000 |
| PostgreSQL | 5432 | 5433 | - (intern) |
| MinIO | 9000/9001 | 9002/9003 | - (intern) |
| Qdrant | 6333/6334 | 6335/6336 | - (intern) |
| Mailpit | 8025/1025 | 8026/1026 | - (deaktiviert) |
## Git Branching Strategie
```
main (Prod) ← Nur Release-Merges, geschützt
staging ← Getesteter Code, Review erforderlich
develop (Dev) ← Tägliche Arbeit, Default-Branch
feature/* ← Feature-Branches (optional)
```
### Workflow
1. **Entwicklung:** Arbeite auf `develop`
2. **Code-Review:** Erstelle PR von Feature-Branch → `develop`
3. **Staging:** Promote `develop``staging` mit Tests
4. **Release:** Promote `staging``main` nach Freigabe
### Promotion-Befehle
**Deploy:**
```bash ```bash
# develop → staging git push origin main && git push gitea main
./scripts/promote.sh dev-to-staging # Coolify baut und deployt automatisch
# staging → main (Production)
./scripts/promote.sh staging-to-prod
``` ```
## Secrets Management
### Development
- `.env.dev` enthält Entwicklungs-Credentials
- Vault optional (Dev-Token)
- Mailpit für E-Mail-Tests
### Staging
- `.env.staging` enthält Test-Credentials
- Vault empfohlen
- Mailpit für E-Mail-Sicherheit
### Production
- `.env.prod` NICHT im Repository
- Vault PFLICHT
- Echte SMTP-Konfiguration
Siehe auch: [Secrets Management](./secrets-management.md)
## Docker Compose Architektur ## Docker Compose Architektur
``` ```
docker-compose.yml ← Basis-Konfiguration docker-compose.yml ← Basis-Konfiguration (lokal, arm64)
── docker-compose.override.yml ← Dev (auto-geladen) ── docker-compose.coolify.yml Production Override (amd64)
├── docker-compose.staging.yml ← Staging (explizit)
└── docker-compose.prod.yml ← Production (explizit)
``` ```
### Automatisches Laden Coolify verwendet automatisch beide Compose-Files fuer den Production-Build.
Docker Compose lädt automatisch: ## Secrets Management
1. `docker-compose.yml`
2. `docker-compose.override.yml` (falls vorhanden)
Daher startet `docker compose up` automatisch die Dev-Umgebung. ### Development
- `.env` enthält Entwicklungs-Credentials
- Vault optional (Dev-Token)
- Mailpit für E-Mail-Tests
## Helper Scripts ### Production
- `.env` auf dem Server (nicht im Repository)
- Vault PFLICHT
- Echte SMTP-Konfiguration
| Script | Beschreibung | Siehe auch: [Secrets Management](./secrets-management.md)
|--------|--------------|
| `scripts/env-switch.sh` | Wechselt zwischen Umgebungen |
| `scripts/start.sh` | Startet Services für Umgebung |
| `scripts/stop.sh` | Stoppt Services |
| `scripts/promote.sh` | Promotet Code zwischen Branches |
| `scripts/status.sh` | Zeigt aktuellen Status |
## Verifikation
Nach Setup prüfen:
```bash
# Status anzeigen
./scripts/status.sh
# Branches prüfen
git branch -v
# Volumes prüfen
docker volume ls | grep breakpilot
```
## Verwandte Dokumentation ## Verwandte Dokumentation

View File

@@ -1,15 +1,14 @@
# CI/CD Pipeline # CI/CD Pipeline
Übersicht über den Deployment-Prozess für Breakpilot. Uebersicht ueber den Deployment-Prozess fuer BreakPilot.
## Übersicht ## Uebersicht
| Komponente | Build-Tool | Deployment | | Repo | Deployment | Trigger | Compose File |
|------------|------------|------------| |------|-----------|---------|--------------|
| Frontend (Next.js) | Docker | Mac Mini | | **breakpilot-core** | Coolify (automatisch) | Push auf `coolify` Branch | `docker-compose.coolify.yml` |
| Backend (FastAPI) | Docker | Mac Mini | | **breakpilot-compliance** | Coolify (automatisch) | Push auf `main` Branch | `docker-compose.yml` + `docker-compose.coolify.yml` |
| Go Services | Docker (Multi-stage) | Mac Mini | | **breakpilot-lehrer** | Mac Mini (lokal) | Manuell `docker compose` | `docker-compose.yml` |
| Documentation | MkDocs | Docker (Nginx) |
## Deployment-Architektur ## Deployment-Architektur
@@ -17,287 +16,146 @@
┌─────────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────────┐
│ Entwickler-MacBook │ │ Entwickler-MacBook │
│ │ │ │
│ breakpilot-core/ │ breakpilot-core/ → git push gitea coolify
├── admin-core/ (Next.js Admin, Port 3008) breakpilot-compliance/ → git push gitea main
├── backend-core/ (Python FastAPI, Port 8000) breakpilot-lehrer/ → git push + ssh macmini docker ...
│ ├── consent-service/ (Go Service, Port 8081) │
│ ├── billing-service/ (Go Service, Port 8083) │
│ └── docs-src/ (MkDocs) │
│ │ │ │
│ git push → Gitea Actions (automatisch) │
│ oder manuell: git push && ssh macmini docker compose build │
└───────────────────────────────┬─────────────────────────────────┘ └───────────────────────────────┬─────────────────────────────────┘
│ git push origin main ┌───────────┴───────────┐
┌─────────────────────────────────────────────────────────────────┐ ┌───────────────────────────┐ ┌───────────────────────────┐
Mac Mini Server (bp-core-*) Coolify (Production) Mac Mini (Lokal/Dev)
Docker Compose Gitea Actions │ breakpilot-lehrer
│ ├── admin-core (Port 3008) │ ├── Tests │ │ ├── studio-v2
── backend-core (Port 8000) ── Coolify API Deploy │ ├── klausur-service
├── consent-service (Port 8081) ├── backend-lehrer
├── billing-service (Port 8083) Core Services: └── voice-service
│ ├── gitea (Port 3003) + gitea-runner (Gitea Actions) │ ├── consent-service │ │
│ ├── docs (Port 8011) │ ├── rag-service Core Services (lokal):
│ ├── postgres, valkey, qdrant, minio │ ├── embedding-service ├── postgres
── vault, nginx, night-scheduler, health ── paddleocr-service ├── valkey, vault
└── health-aggregator │ ├── nginx, gitea
└─────────────────────────────────────────────────────────────────┘ │ │ │ └── ... │
│ Compliance Services: │ │ │
│ ├── admin-compliance │ │ │
│ ├── backend-compliance │ │ │
│ ├── ai-compliance-sdk │ │ │
│ └── developer-portal │ │ │
└───────────────────────────┘ └───────────────────────────┘
``` ```
## Sync & Deploy Workflow ## breakpilot-core → Coolify
### 1. Dateien synchronisieren ### Pipeline
```bash
# Sync aller relevanten Verzeichnisse zum Mac Mini
rsync -avz --delete \
--exclude 'node_modules' \
--exclude '.next' \
--exclude '.git' \
--exclude '__pycache__' \
--exclude 'venv' \
--exclude '.pytest_cache' \
/Users/benjaminadmin/Projekte/breakpilot-core/ \
macmini:/Users/benjaminadmin/Projekte/breakpilot-core/
```
### 2. Container bauen
```bash
# Einzelnen Service bauen
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
build --no-cache <service-name>"
# Beispiele:
# studio-v2, admin-v2, website, backend, klausur-service, docs
```
### 3. Container deployen
```bash
# Container neu starten
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
up -d <service-name>"
```
### 4. Logs prüfen
```bash
# Container-Logs anzeigen
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
logs -f <service-name>"
```
## Service-spezifische Deployments
### Next.js Frontend (studio-v2, admin-v2, website)
```bash
# 1. Sync
rsync -avz --delete \
--exclude 'node_modules' --exclude '.next' --exclude '.git' \
/Users/benjaminadmin/Projekte/breakpilot-core/studio-v2/ \
macmini:/Users/benjaminadmin/Projekte/breakpilot-core/studio-v2/
# 2. Build & Deploy
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
build --no-cache studio-v2 && \
/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
up -d studio-v2"
```
### Python Services (backend, klausur-service, voice-service)
```bash
# Build mit requirements.txt
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
build klausur-service && \
/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
up -d klausur-service"
```
### Go Services (consent-service, ai-compliance-sdk)
```bash
# Multi-stage Build (Go → Alpine)
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
build --no-cache consent-service && \
/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
up -d consent-service"
```
### MkDocs Dokumentation
```bash
# Build & Deploy
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
build --no-cache docs && \
/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
up -d docs"
# Verfügbar unter: http://macmini:8009
```
## Health Checks
### Service-Status prüfen
```bash
# Alle Container-Status
ssh macmini "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
# Health-Endpoints prüfen
curl -s http://macmini:8000/health
curl -s http://macmini:8081/health
curl -s http://macmini:8086/health
curl -s http://macmini:8090/health
```
### Logs analysieren
```bash
# Letzte 100 Zeilen
ssh macmini "docker logs --tail 100 breakpilot-core-backend-1"
# Live-Logs folgen
ssh macmini "docker logs -f breakpilot-core-backend-1"
```
## Rollback
### Container auf vorherige Version zurücksetzen
```bash
# 1. Aktuelles Image taggen
ssh macmini "docker tag breakpilot-core-backend:latest breakpilot-core-backend:backup"
# 2. Altes Image deployen
ssh macmini "/usr/local/bin/docker compose \
-f /Users/benjaminadmin/Projekte/breakpilot-core/docker-compose.yml \
up -d backend"
# 3. Bei Problemen: Backup wiederherstellen
ssh macmini "docker tag breakpilot-core-backend:backup breakpilot-core-backend:latest"
```
## Troubleshooting
### Container startet nicht
```bash
# 1. Logs prüfen
ssh macmini "docker logs breakpilot-core-<service>-1"
# 2. Container manuell starten für Debug-Output
ssh macmini "docker compose -f .../docker-compose.yml run --rm <service>"
# 3. In Container einloggen
ssh macmini "docker exec -it breakpilot-core-<service>-1 /bin/sh"
```
### Port bereits belegt
```bash
# Port-Belegung prüfen
ssh macmini "lsof -i :8000"
# Container mit dem Port finden
ssh macmini "docker ps --filter publish=8000"
```
### Build-Fehler
```bash
# Cache komplett leeren
ssh macmini "docker builder prune -a"
# Ohne Cache bauen
ssh macmini "docker compose build --no-cache <service>"
```
## Monitoring
### Resource-Nutzung
```bash
# CPU/Memory aller Container
ssh macmini "docker stats --no-stream"
# Disk-Nutzung
ssh macmini "docker system df"
```
### Cleanup
```bash
# Ungenutzte Images/Container entfernen
ssh macmini "docker system prune -a --volumes"
# Nur dangling Images
ssh macmini "docker image prune"
```
## Umgebungsvariablen
Umgebungsvariablen werden über `.env` Dateien und docker-compose.yml verwaltet:
```yaml ```yaml
# docker-compose.yml # .gitea/workflows/deploy-coolify.yml
services: on:
backend: push:
environment: branches: [coolify]
- DATABASE_URL=postgresql://...
- REDIS_URL=redis://valkey:6379 jobs:
- SECRET_KEY=${SECRET_KEY} deploy:
runs-on: ubuntu-latest
steps:
- name: Deploy via Coolify API
# Triggert Coolify Build + Deploy ueber API
# Secrets: COOLIFY_API_TOKEN, COOLIFY_RESOURCE_UUID, COOLIFY_BASE_URL
``` ```
**Wichtig**: Sensible Werte niemals in Git committen. Stattdessen: ### Workflow
- `.env` Datei auf dem Server pflegen
- Secrets über HashiCorp Vault (siehe unten) ```bash
# 1. Code auf MacBook bearbeiten
# 2. Committen und pushen:
git push origin main && git push gitea main
# 3. Fuer Production-Deploy:
git push gitea coolify
# 4. Status pruefen:
# https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-core/actions
```
### Coolify-deployed Services
| Service | Container | Beschreibung |
|---------|-----------|--------------|
| valkey | bp-core-valkey | Session-Cache |
| consent-service | bp-core-consent-service | Consent-Management (Go) |
| rag-service | bp-core-rag-service | Semantische Suche |
| embedding-service | bp-core-embedding-service | Text-Embeddings |
| paddleocr-service | bp-core-paddleocr | OCR Engine (x86_64) |
| health-aggregator | bp-core-health | Health-Check Aggregator |
## breakpilot-compliance → Coolify
### Pipeline
```yaml
# .gitea/workflows/ci.yaml
on:
push:
branches: [main, develop]
jobs:
# Lint (nur PRs)
# Tests (Go, Python, Node.js)
# Validate Canonical Controls
# Deploy (nur main, nach allen Tests)
```
### Workflow
```bash
# Committen und pushen → Coolify deployt automatisch:
git push origin main && git push gitea main
# CI-Status pruefen:
# https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-compliance/actions
# Health Checks:
curl -sf https://api-dev.breakpilot.ai/health
curl -sf https://sdk-dev.breakpilot.ai/health
```
## breakpilot-lehrer → Mac Mini (lokal)
### Workflow
```bash
# 1. Code auf MacBook bearbeiten
# 2. Committen und pushen:
git push origin main && git push gitea main
# 3. Auf Mac Mini pullen und Container neu bauen:
ssh macmini "git -C /Users/benjaminadmin/Projekte/breakpilot-lehrer pull --no-rebase origin main"
ssh macmini "/usr/local/bin/docker compose -f /Users/benjaminadmin/Projekte/breakpilot-lehrer/docker-compose.yml build --no-cache <service>"
ssh macmini "/usr/local/bin/docker compose -f /Users/benjaminadmin/Projekte/breakpilot-lehrer/docker-compose.yml up -d <service>"
```
## Gitea Actions ## Gitea Actions
### Überblick ### Ueberblick
BreakPilot Core nutzt **Gitea Actions** (GitHub Actions-kompatibel) als CI/CD-System. Der `act_runner` läuft als Container auf dem Mac Mini und führt Pipelines direkt bei Code-Push aus. BreakPilot nutzt **Gitea Actions** (GitHub Actions-kompatibel) als CI/CD-System. Der `act_runner` laeuft als Container auf dem Mac Mini und fuehrt Pipelines aus.
| Komponente | Container | Beschreibung | | Komponente | Container | Beschreibung |
|------------|-----------|--------------| |------------|-----------|--------------|
| Gitea | `bp-core-gitea` (Port 3003) | Git-Server + Actions-Trigger | | Gitea | `bp-core-gitea` (Port 3003) | Git-Server + Actions-Trigger |
| Gitea Runner | `bp-core-gitea-runner` | Führt Actions-Workflows aus | | Gitea Runner | `bp-core-gitea-runner` | Fuehrt Actions-Workflows aus |
### Pipeline-Konfiguration ### Pipeline-Konfiguration
Workflows liegen im Repo unter `.gitea/workflows/`: Workflows liegen in jedem Repo unter `.gitea/workflows/`:
```yaml | Repo | Workflow | Branch | Aktion |
# .gitea/workflows/main.yml |------|----------|--------|--------|
on: | breakpilot-core | `deploy-coolify.yml` | `coolify` | Coolify API Deploy |
push: | breakpilot-compliance | `ci.yaml` | `main` | Tests + Coolify Deploy |
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build & Test
run: docker compose build
```
### Runner-Token erneuern ### Runner-Token erneuern
@@ -314,12 +172,79 @@ ssh macmini "/usr/local/bin/docker compose \
up -d --force-recreate gitea-runner" up -d --force-recreate gitea-runner"
``` ```
### Pipeline-Status prüfen ### Pipeline-Status pruefen
```bash ```bash
# Runner-Logs # Runner-Logs
ssh macmini "/usr/local/bin/docker logs -f bp-core-gitea-runner" ssh macmini "/usr/local/bin/docker logs -f bp-core-gitea-runner"
```
# Laufende Jobs
ssh macmini "/usr/local/bin/docker exec bp-core-gitea-runner act_runner list" ## Health Checks
### Production (Coolify)
```bash
# Core PaddleOCR
curl -sf https://ocr.breakpilot.com/health
# Compliance
curl -sf https://api-dev.breakpilot.ai/health
curl -sf https://sdk-dev.breakpilot.ai/health
```
### Lokal (Mac Mini)
```bash
# Core Health Aggregator
curl -sf http://macmini:8099/health
# Lehrer Backend
curl -sf https://macmini:8001/health
# Klausur-Service
curl -sf https://macmini:8086/health
```
## Troubleshooting
### Container startet nicht
```bash
# Logs pruefen (lokal)
ssh macmini "/usr/local/bin/docker logs bp-core-<service>"
# In Container einloggen
ssh macmini "/usr/local/bin/docker exec -it bp-core-<service> /bin/sh"
```
### Build-Fehler
```bash
# Cache komplett leeren
ssh macmini "docker builder prune -a"
# Ohne Cache bauen
ssh macmini "docker compose build --no-cache <service>"
```
## Rollback
### Coolify
Ein Redeploy mit einem aelteren Commit kann durch Zuruecksetzen des Branches ausgeloest werden:
```bash
# Branch auf vorherigen Commit zuruecksetzen und pushen
git reset --hard <previous-commit>
git push gitea coolify --force
```
### Lokal (Mac Mini)
```bash
# Image taggen als Backup
ssh macmini "docker tag breakpilot-lehrer-klausur-service:latest breakpilot-lehrer-klausur-service:backup"
# Bei Problemen: Backup wiederherstellen
ssh macmini "docker tag breakpilot-lehrer-klausur-service:backup breakpilot-lehrer-klausur-service:latest"
``` ```

View File

@@ -12,6 +12,14 @@ BreakPilot besteht aus drei unabhaengigen Projekten:
| **breakpilot-lehrer** | Bildungs-Stack (Team A) | `bp-lehrer-*` | Blau | | **breakpilot-lehrer** | Bildungs-Stack (Team A) | `bp-lehrer-*` | Blau |
| **breakpilot-compliance** | DSGVO/Compliance-Stack (Team B) | `bp-compliance-*` | Lila | | **breakpilot-compliance** | DSGVO/Compliance-Stack (Team B) | `bp-compliance-*` | Lila |
### Deployment-Modell
| Repo | Deployment | Trigger |
|------|-----------|---------|
| **breakpilot-core** | Coolify (automatisch) | Push auf gitea main |
| **breakpilot-compliance** | Coolify (automatisch) | Push auf gitea main |
| **breakpilot-lehrer** | Mac Mini (lokal) | Manuell docker compose |
## Core Services ## Core Services
| Service | Container | Port | Beschreibung | | Service | Container | Port | Beschreibung |
@@ -30,32 +38,11 @@ BreakPilot besteht aus drei unabhaengigen Projekten:
| Admin Core | bp-core-admin | 3008 | Admin-Dashboard (Next.js) | | Admin Core | bp-core-admin | 3008 | Admin-Dashboard (Next.js) |
| Health Aggregator | bp-core-health | 8099 | Service-Health Monitoring | | Health Aggregator | bp-core-health | 8099 | Service-Health Monitoring |
| Night Scheduler | bp-core-night-scheduler | 8096 | Nachtabschaltung | | Night Scheduler | bp-core-night-scheduler | 8096 | Nachtabschaltung |
| Pitch Deck | bp-core-pitch-deck | 3012 | Investor-Praesentation |
| Mailpit | bp-core-mailpit | 8025 | E-Mail (Entwicklung) | | Mailpit | bp-core-mailpit | 8025 | E-Mail (Entwicklung) |
| Gitea | bp-core-gitea | 3003 | Git-Server | | Gitea | bp-core-gitea | 3003 | Git-Server |
| Gitea Runner | bp-core-gitea-runner | - | CI/CD (Gitea Actions) | | Gitea Runner | bp-core-gitea-runner | - | CI/CD (Gitea Actions) |
| Jitsi (5 Container) | bp-core-jitsi-* | 8443 | Videokonferenzen | | Jitsi (5 Container) | bp-core-jitsi-* | 8443 | Videokonferenzen |
## Nginx Routing-Tabelle
| Port | Upstream | Projekt |
|------|----------|---------|
| 443 | bp-lehrer-studio-v2:3001 | Lehrer |
| 3000 | bp-lehrer-website:3000 | Lehrer |
| 3002 | bp-lehrer-admin:3000 | Lehrer |
| 3006 | bp-compliance-developer-portal:3000 | Compliance |
| 3007 | bp-compliance-admin:3000 | Compliance |
| 3008 | bp-core-admin:3000 | Core |
| 8000 | bp-core-backend:8000 | Core |
| 8001 | bp-lehrer-backend:8001 | Lehrer |
| 8002 | bp-compliance-backend:8002 | Compliance |
| 8086 | bp-lehrer-klausur-service:8086 | Lehrer |
| 8087 | bp-core-embedding-service:8087 | Core |
| 8091 | bp-lehrer-voice-service:8091 | Lehrer |
| 8093 | bp-compliance-ai-sdk:8090 | Compliance |
| 8097 | bp-core-rag-service:8097 | Core |
| 8443 | bp-core-jitsi-web:80 | Core |
## Architektur ## Architektur
- [System-Architektur](architecture/system-architecture.md) - [System-Architektur](architecture/system-architecture.md)

View File

@@ -0,0 +1,16 @@
FROM python:3.11-slim
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
libgl1 libglib2.0-0 libgomp1 curl \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8095
HEALTHCHECK --interval=30s --timeout=10s --start-period=120s --retries=3 \
CMD curl -f http://127.0.0.1:8095/health || exit 1
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8095"]

110
paddleocr-service/main.py Normal file
View File

@@ -0,0 +1,110 @@
"""PaddleOCR Remote Service — PP-OCRv4 on x86_64 (CPU)."""
import io
import logging
import os
import threading
import numpy as np
from fastapi import FastAPI, File, Header, HTTPException, UploadFile
from PIL import Image
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI(title="PaddleOCR Service")
_engine = None
_ready = False
_loading = False
API_KEY = os.environ.get("PADDLEOCR_API_KEY", "")
def _load_model():
"""Load PaddleOCR model in background thread."""
global _engine, _ready
try:
logger.info("Importing paddleocr...")
from paddleocr import PaddleOCR
logger.info("Loading PaddleOCR model (PP-OCRv4, lang=en)...")
_engine = PaddleOCR(
lang="en",
use_angle_cls=True,
show_log=False,
enable_mkldnn=False,
use_gpu=False,
)
logger.info("PaddleOCR model loaded — running warmup...")
# Warmup with tiny image to trigger any lazy init
dummy = np.ones((30, 100, 3), dtype=np.uint8) * 255
_engine.ocr(dummy)
_ready = True
logger.info("PaddleOCR ready to serve")
except Exception as e:
logger.error(f"Failed to load PaddleOCR: {e}", exc_info=True)
@app.on_event("startup")
def startup_load_model():
"""Start model loading in background so health check passes immediately."""
global _loading
_loading = True
threading.Thread(target=_load_model, daemon=True).start()
logger.info("Model loading started in background thread")
@app.get("/health")
def health():
if _ready:
return {"status": "ok", "model": "PP-OCRv4"}
if _loading:
return {"status": "loading"}
return {"status": "error"}
@app.post("/ocr")
async def ocr(
file: UploadFile = File(...),
x_api_key: str = Header(default=""),
):
if API_KEY and x_api_key != API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
if not _ready:
raise HTTPException(status_code=503, detail="Model still loading")
img_bytes = await file.read()
img = Image.open(io.BytesIO(img_bytes)).convert("RGB")
img_np = np.array(img)
try:
result = _engine.ocr(img_np)
except Exception as e:
logger.error(f"OCR failed: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"OCR failed: {e}")
if not result or not result[0]:
return {"words": [], "image_width": img_np.shape[1], "image_height": img_np.shape[0]}
words = []
for line in result[0]:
box, (text, conf) = line[0], line[1]
x_min = min(p[0] for p in box)
y_min = min(p[1] for p in box)
x_max = max(p[0] for p in box)
y_max = max(p[1] for p in box)
words.append({
"text": str(text).strip(),
"left": int(x_min),
"top": int(y_min),
"width": int(x_max - x_min),
"height": int(y_max - y_min),
"conf": round(float(conf) * 100, 1),
})
return {
"words": words,
"image_width": img_np.shape[1],
"image_height": img_np.shape[0],
}

View File

@@ -0,0 +1,7 @@
paddlepaddle>=2.6.0,<3.0.0
paddleocr>=2.7.0,<3.0.0
fastapi>=0.110.0
uvicorn>=0.25.0
python-multipart>=0.0.6
Pillow>=10.0.0
numpy>=1.24.0