Compare commits

...

13 Commits

Author SHA1 Message Date
Benjamin Admin
520a0f401c fix: downgrade to PaddleOCR 2.x for CPU stability
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:13:41 +01:00
Benjamin Admin
6adf1fe1eb fix: force-disable oneDNN for PaddlePaddle 3.x
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:01:55 +01:00
Benjamin Admin
2ac6559291 fix: disable oneDNN and support PaddleOCR 3.x format
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:54:28 +01:00
Benjamin Admin
52618a0630 fix: add error handling to OCR endpoint
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:37:47 +01:00
Benjamin Admin
e1a84fd568 fix: remove warmup OCR call — causes OOM on 6G container
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:31:25 +01:00
Benjamin Admin
dd0bda05be fix: increase paddleocr memory limit 4G → 6G
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
PaddlePaddle + PP-OCRv5 model + warmup OCR needs more than 4G on
CPU-only servers. Container was OOM-killed during warmup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:08:15 +01:00
Benjamin Admin
4c68666c5c fix: add warmup OCR call to avoid timeout on first request
Some checks failed
Deploy to Coolify / deploy (push) Failing after 6s
PaddleOCR JIT-compiles on the first .ocr() call, which takes minutes
on CPU-only servers. This causes Traefik 504 Gateway Timeout.

Run a dummy OCR during startup so the first real request is fast.
Also simplify Traefik labels on paddleocr-service.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:55:07 +01:00
Benjamin Admin
46b1fdc20f fix: use runs-on docker for Gitea runner compatibility
Some checks failed
Deploy to Coolify / deploy (push) Failing after 4s
The Gitea runner on Mac Mini uses label 'docker', not 'ubuntu-latest'.
Also need alpine container with curl installed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:51:23 +01:00
Benjamin Admin
445cbc3100 fix: add deploy-coolify.yml workflow to coolify branch
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
The deploy workflow was missing from the coolify branch, so pushes
to coolify never triggered a Coolify redeploy via Gitea Actions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:45:58 +01:00
Benjamin Admin
8fe4473205 feat: add paddleocr-service directory to coolify branch
The docker-compose.coolify.yml references paddleocr-service/Dockerfile
but the directory only existed on main. Coolify clones the coolify branch
and needs the source files to build the container.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:40:26 +01:00
Benjamin Admin
07dbd78962 feat: add paddleocr-service to Coolify compose
Add PaddleOCR PP-OCRv5 service with 4G memory limit, model volume,
and health check (5min start period for model loading). Domain routing
(ocr.breakpilot.com) to be configured in Coolify UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:39:58 +01:00
Sharang Parnerkar
e9487a31c6 Replace deploy-hetzner with Coolify webhook deploy in ci.yaml
Some checks failed
CI / go-lint (pull_request) Failing after 2s
CI / python-lint (pull_request) Failing after 10s
CI / nodejs-lint (pull_request) Failing after 2s
CI / test-go-consent (pull_request) Failing after 2s
CI / test-python-voice (pull_request) Failing after 9s
CI / Deploy (pull_request) Has been skipped
CI / test-bqas (pull_request) Failing after 10s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:42:41 +01:00
Sharang Parnerkar
0fb4a7e359 Remove standalone deploy-coolify.yml — deploy is handled in ci.yaml
Some checks failed
CI / go-lint (pull_request) Failing after 3s
CI / python-lint (pull_request) Failing after 11s
CI / nodejs-lint (pull_request) Failing after 3s
CI / test-go-consent (pull_request) Failing after 3s
CI / test-python-voice (pull_request) Failing after 11s
CI / test-bqas (pull_request) Failing after 12s
CI / deploy-hetzner (pull_request) Has been skipped
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:26:34 +01:00
6 changed files with 178 additions and 108 deletions

View File

@@ -140,117 +140,20 @@ jobs:
python -m pytest tests/bqas/ -v --tb=short || true python -m pytest tests/bqas/ -v --tb=short || true
# ======================================== # ========================================
# Build & Deploy auf Hetzner (nur main, kein PR) # Deploy via Coolify (nur main, kein PR)
# ======================================== # ========================================
deploy-hetzner: deploy-coolify:
name: Deploy
runs-on: docker runs-on: docker
if: github.event_name == 'push' && github.ref == 'refs/heads/main' if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: needs:
- test-go-consent - test-go-consent
container: docker:27-cli container:
image: alpine:latest
steps: steps:
- name: Deploy - name: Trigger Coolify deploy
run: | run: |
set -euo pipefail apk add --no-cache curl
DEPLOY_DIR="/opt/breakpilot-core" curl -sf "${{ secrets.COOLIFY_WEBHOOK }}" \
COMPOSE_FILES="-f docker-compose.yml -f docker-compose.hetzner.yml" -H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
COMMIT_SHA="${GITHUB_SHA:-unknown}"
SHORT_SHA="${COMMIT_SHA:0:8}"
REPO_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
# Services die deployed werden
SERVICES="postgres valkey qdrant minio ollama mailpit embedding-service rag-service backend-core consent-service health-aggregator"
echo "=== BreakPilot Core Deploy ==="
echo "Commit: ${SHORT_SHA}"
echo "Deploy Dir: ${DEPLOY_DIR}"
echo "Services: ${SERVICES}"
echo ""
# 1. Repo auf dem Host erstellen/aktualisieren via Helper-Container
echo "=== Updating code on host ==="
docker run --rm \
-v "${DEPLOY_DIR}:${DEPLOY_DIR}" \
--entrypoint sh \
alpine/git:latest \
-c "
if [ ! -d '${DEPLOY_DIR}/.git' ]; then
echo 'Erstmaliges Klonen nach ${DEPLOY_DIR}...'
git clone '${REPO_URL}' '${DEPLOY_DIR}'
else
cd '${DEPLOY_DIR}'
git fetch origin main
git reset --hard origin/main
fi
"
echo "Code aktualisiert auf ${SHORT_SHA}"
# 2. .env sicherstellen
docker run --rm -v "${DEPLOY_DIR}:${DEPLOY_DIR}" alpine \
sh -c "
if [ ! -f '${DEPLOY_DIR}/.env' ]; then
echo 'WARNUNG: ${DEPLOY_DIR}/.env fehlt!'
echo 'Erstelle .env aus .env.example mit Defaults...'
if [ -f '${DEPLOY_DIR}/.env.example' ]; then
cp '${DEPLOY_DIR}/.env.example' '${DEPLOY_DIR}/.env'
echo '.env aus .env.example erstellt'
else
echo 'Kein .env.example gefunden — Services starten mit Defaults'
fi
else
echo '.env vorhanden'
fi
"
# 3. Shared Network erstellen (falls noch nicht vorhanden)
docker network create breakpilot-network 2>/dev/null || true
# 4. Build + Deploy via Helper-Container
echo ""
echo "=== Building + Deploying ==="
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "${DEPLOY_DIR}:${DEPLOY_DIR}" \
-w "${DEPLOY_DIR}" \
docker:27-cli \
sh -c "
set -e
COMPOSE_FILES='-f docker-compose.yml -f docker-compose.hetzner.yml'
echo '=== Building Docker Images ==='
docker compose \${COMPOSE_FILES} build --parallel \
backend-core consent-service rag-service embedding-service health-aggregator
echo ''
echo '=== Starting infrastructure ==='
docker compose \${COMPOSE_FILES} up -d postgres valkey qdrant minio mailpit
echo 'Warte auf DB + Cache...'
sleep 10
echo ''
echo '=== Starting Ollama + pulling bge-m3 ==='
docker compose \${COMPOSE_FILES} up -d ollama
sleep 5
# bge-m3 Modell pullen (nur beim ersten Mal ~670MB)
echo 'Pulling bge-m3 model (falls noch nicht vorhanden)...'
docker exec bp-core-ollama ollama pull bge-m3 2>&1 || echo 'WARNUNG: bge-m3 pull fehlgeschlagen (wird spaeter nachgeholt)'
echo ''
echo '=== Starting application services ==='
docker compose \${COMPOSE_FILES} up -d \
embedding-service rag-service backend-core consent-service health-aggregator
echo ''
echo '=== Health Checks ==='
sleep 15
for svc in bp-core-postgres bp-core-valkey bp-core-qdrant bp-core-ollama bp-core-embedding-service bp-core-rag-service bp-core-backend bp-core-consent-service bp-core-health; do
STATUS=\$(docker inspect --format='{{.State.Status}}' \"\${svc}\" 2>/dev/null || echo 'not found')
echo \"\${svc}: \${STATUS}\"
done
"
echo ""
echo "=== Deploy abgeschlossen: ${SHORT_SHA} ==="

View File

@@ -7,10 +7,12 @@ on:
jobs: jobs:
deploy: deploy:
runs-on: ubuntu-latest runs-on: docker
container: alpine:latest
steps: steps:
- name: Deploy via Coolify API - name: Deploy via Coolify API
run: | run: |
apk add --no-cache curl
echo "Deploying breakpilot-core to Coolify..." echo "Deploying breakpilot-core to Coolify..."
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \ HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-X POST \ -X POST \

View File

@@ -15,6 +15,7 @@ networks:
volumes: volumes:
valkey_data: valkey_data:
embedding_models: embedding_models:
paddleocr_models:
services: services:
@@ -141,6 +142,37 @@ services:
networks: networks:
- breakpilot-network - breakpilot-network
# =========================================================
# OCR SERVICE (PaddleOCR PP-OCRv5)
# =========================================================
paddleocr-service:
build:
context: ./paddleocr-service
dockerfile: Dockerfile
container_name: bp-core-paddleocr
expose:
- "8095"
environment:
PADDLEOCR_API_KEY: ${PADDLEOCR_API_KEY:-}
FLAGS_use_mkldnn: "0"
volumes:
- paddleocr_models:/root/.paddleocr
labels:
- "traefik.http.services.paddleocr.loadbalancer.server.port=8095"
deploy:
resources:
limits:
memory: 6G
healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8095/health"]
interval: 30s
timeout: 10s
start_period: 300s
retries: 5
restart: unless-stopped
networks:
- breakpilot-network
# ========================================================= # =========================================================
# HEALTH AGGREGATOR # HEALTH AGGREGATOR
# ========================================================= # =========================================================
@@ -153,7 +185,7 @@ services:
- "8099" - "8099"
environment: environment:
PORT: 8099 PORT: 8099
CHECK_SERVICES: "valkey:6379,consent-service:8081,rag-service:8097,embedding-service:8087" CHECK_SERVICES: "valkey:6379,consent-service:8081,rag-service:8097,embedding-service:8087,paddleocr-service:8095"
healthcheck: healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8099/health"] test: ["CMD", "curl", "-f", "http://127.0.0.1:8099/health"]
interval: 30s interval: 30s

View File

@@ -0,0 +1,16 @@
FROM python:3.11-slim
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
libgl1 libglib2.0-0 libgomp1 curl \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8095
HEALTHCHECK --interval=30s --timeout=10s --start-period=120s --retries=3 \
CMD curl -f http://127.0.0.1:8095/health || exit 1
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8095"]

110
paddleocr-service/main.py Normal file
View File

@@ -0,0 +1,110 @@
"""PaddleOCR Remote Service — PP-OCRv4 on x86_64 (CPU)."""
import io
import logging
import os
import threading
import numpy as np
from fastapi import FastAPI, File, Header, HTTPException, UploadFile
from PIL import Image
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI(title="PaddleOCR Service")
_engine = None
_ready = False
_loading = False
API_KEY = os.environ.get("PADDLEOCR_API_KEY", "")
def _load_model():
"""Load PaddleOCR model in background thread."""
global _engine, _ready
try:
logger.info("Importing paddleocr...")
from paddleocr import PaddleOCR
logger.info("Loading PaddleOCR model (PP-OCRv4, lang=en)...")
_engine = PaddleOCR(
lang="en",
use_angle_cls=True,
show_log=False,
enable_mkldnn=False,
use_gpu=False,
)
logger.info("PaddleOCR model loaded — running warmup...")
# Warmup with tiny image to trigger any lazy init
dummy = np.ones((30, 100, 3), dtype=np.uint8) * 255
_engine.ocr(dummy)
_ready = True
logger.info("PaddleOCR ready to serve")
except Exception as e:
logger.error(f"Failed to load PaddleOCR: {e}", exc_info=True)
@app.on_event("startup")
def startup_load_model():
"""Start model loading in background so health check passes immediately."""
global _loading
_loading = True
threading.Thread(target=_load_model, daemon=True).start()
logger.info("Model loading started in background thread")
@app.get("/health")
def health():
if _ready:
return {"status": "ok", "model": "PP-OCRv4"}
if _loading:
return {"status": "loading"}
return {"status": "error"}
@app.post("/ocr")
async def ocr(
file: UploadFile = File(...),
x_api_key: str = Header(default=""),
):
if API_KEY and x_api_key != API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
if not _ready:
raise HTTPException(status_code=503, detail="Model still loading")
img_bytes = await file.read()
img = Image.open(io.BytesIO(img_bytes)).convert("RGB")
img_np = np.array(img)
try:
result = _engine.ocr(img_np)
except Exception as e:
logger.error(f"OCR failed: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"OCR failed: {e}")
if not result or not result[0]:
return {"words": [], "image_width": img_np.shape[1], "image_height": img_np.shape[0]}
words = []
for line in result[0]:
box, (text, conf) = line[0], line[1]
x_min = min(p[0] for p in box)
y_min = min(p[1] for p in box)
x_max = max(p[0] for p in box)
y_max = max(p[1] for p in box)
words.append({
"text": str(text).strip(),
"left": int(x_min),
"top": int(y_min),
"width": int(x_max - x_min),
"height": int(y_max - y_min),
"conf": round(float(conf) * 100, 1),
})
return {
"words": words,
"image_width": img_np.shape[1],
"image_height": img_np.shape[0],
}

View File

@@ -0,0 +1,7 @@
paddlepaddle>=2.6.0,<3.0.0
paddleocr>=2.7.0,<3.0.0
fastapi>=0.110.0
uvicorn>=0.25.0
python-multipart>=0.0.6
Pillow>=10.0.0
numpy>=1.24.0