Compare commits

...

31 Commits

Author SHA1 Message Date
Benjamin Admin
520a0f401c fix: downgrade to PaddleOCR 2.x for CPU stability
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:13:41 +01:00
Benjamin Admin
6adf1fe1eb fix: force-disable oneDNN for PaddlePaddle 3.x
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 19:01:55 +01:00
Benjamin Admin
2ac6559291 fix: disable oneDNN and support PaddleOCR 3.x format
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:54:28 +01:00
Benjamin Admin
52618a0630 fix: add error handling to OCR endpoint
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:37:47 +01:00
Benjamin Admin
e1a84fd568 fix: remove warmup OCR call — causes OOM on 6G container
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:31:25 +01:00
Benjamin Admin
dd0bda05be fix: increase paddleocr memory limit 4G → 6G
Some checks failed
Deploy to Coolify / deploy (push) Failing after 2s
PaddlePaddle + PP-OCRv5 model + warmup OCR needs more than 4G on
CPU-only servers. Container was OOM-killed during warmup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 18:08:15 +01:00
Benjamin Admin
4c68666c5c fix: add warmup OCR call to avoid timeout on first request
Some checks failed
Deploy to Coolify / deploy (push) Failing after 6s
PaddleOCR JIT-compiles on the first .ocr() call, which takes minutes
on CPU-only servers. This causes Traefik 504 Gateway Timeout.

Run a dummy OCR during startup so the first real request is fast.
Also simplify Traefik labels on paddleocr-service.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:55:07 +01:00
Benjamin Admin
46b1fdc20f fix: use runs-on docker for Gitea runner compatibility
Some checks failed
Deploy to Coolify / deploy (push) Failing after 4s
The Gitea runner on Mac Mini uses label 'docker', not 'ubuntu-latest'.
Also need alpine container with curl installed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:51:23 +01:00
Benjamin Admin
445cbc3100 fix: add deploy-coolify.yml workflow to coolify branch
Some checks failed
Deploy to Coolify / deploy (push) Has been cancelled
The deploy workflow was missing from the coolify branch, so pushes
to coolify never triggered a Coolify redeploy via Gitea Actions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:45:58 +01:00
Benjamin Admin
8fe4473205 feat: add paddleocr-service directory to coolify branch
The docker-compose.coolify.yml references paddleocr-service/Dockerfile
but the directory only existed on main. Coolify clones the coolify branch
and needs the source files to build the container.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:40:26 +01:00
Benjamin Admin
07dbd78962 feat: add paddleocr-service to Coolify compose
Add PaddleOCR PP-OCRv5 service with 4G memory limit, model volume,
and health check (5min start period for model loading). Domain routing
(ocr.breakpilot.com) to be configured in Coolify UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:39:58 +01:00
Sharang Parnerkar
e9487a31c6 Replace deploy-hetzner with Coolify webhook deploy in ci.yaml
Some checks failed
CI / go-lint (pull_request) Failing after 2s
CI / python-lint (pull_request) Failing after 10s
CI / nodejs-lint (pull_request) Failing after 2s
CI / test-go-consent (pull_request) Failing after 2s
CI / test-python-voice (pull_request) Failing after 9s
CI / Deploy (pull_request) Has been skipped
CI / test-bqas (pull_request) Failing after 10s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:42:41 +01:00
Sharang Parnerkar
0fb4a7e359 Remove standalone deploy-coolify.yml — deploy is handled in ci.yaml
Some checks failed
CI / go-lint (pull_request) Failing after 3s
CI / python-lint (pull_request) Failing after 11s
CI / nodejs-lint (pull_request) Failing after 3s
CI / test-go-consent (pull_request) Failing after 3s
CI / test-python-voice (pull_request) Failing after 11s
CI / test-bqas (pull_request) Failing after 12s
CI / deploy-hetzner (pull_request) Has been skipped
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:26:34 +01:00
Sharang Parnerkar
cf2cabd098 Remove services not needed by SDK from Coolify deployment
Some checks failed
CI / go-lint (pull_request) Failing after 15s
CI / nodejs-lint (pull_request) Failing after 2s
CI / test-python-voice (pull_request) Failing after 11s
CI / deploy-hetzner (pull_request) Has been skipped
CI / python-lint (pull_request) Failing after 10s
CI / test-go-consent (pull_request) Failing after 2s
CI / test-bqas (pull_request) Failing after 10s
Deploy to Coolify / deploy (push) Has been cancelled
Remove backend-core, billing-service, night-scheduler, and admin-core
as they are not used by any compliance/SDK service. Update
health-aggregator CHECK_SERVICES to reference consent-service instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
8ee02bd2e4 Add healthchecks to backend-core, consent-service, billing-service, admin-core
Coolify/Traefik requires healthchecks to route traffic to containers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
d9687725e5 Remove Traefik labels from coolify compose — Coolify handles routing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
6c3911ca47 Fix admin-core build: ensure public directory exists before build
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
30807d1ce1 Fix backend-core TARGETARCH: auto-detect instead of hardcoded arm64
The Dockerfile hardcoded TARGETARCH=arm64 for Mac Mini. Coolify server
is x86_64, causing exit code 126 (wrong binary arch). Now uses Docker
BuildKit's auto-detected TARGETARCH with dpkg fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
82c28a2b6e Add QDRANT_API_KEY support to rag-service
- Add QDRANT_API_KEY to config.py (empty string = no auth)
- Pass api_key to QdrantClient constructor (None when empty)
- Add QDRANT_API_KEY to coolify compose and env example

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
86624d72dd Sync coolify compose with main: remove voice-service, update rag/embedding
- Remove voice-service (removed in main branch)
- Remove voice_session_data volume
- Add OLLAMA_URL and OLLAMA_EMBED_MODEL to rag-service
- Update embedding-service default model to BAAI/bge-m3, memory 4G→8G
- Update health-aggregator CHECK_SERVICES (remove voice-service)
- Update .env.coolify.example accordingly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
9218664400 fix: use Alpine-compatible addgroup/adduser flags in Dockerfiles
Replace --system/--gid/--uid (Debian syntax) with -S/-g/-u (BusyBox/Alpine).
Coolify ARG injection causes exit code 255 with Debian-style flags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
8fa5d9061a refactor(coolify): externalize postgres, qdrant, S3; remove jitsi/synapse
- Remove PostgreSQL, Qdrant, MinIO services (managed separately in Coolify)
- Remove Jitsi stack (web, xmpp, jicofo, jvb) and Synapse/synapse-db
- Add POSTGRES_HOST, QDRANT_URL, S3_ENDPOINT/S3_ACCESS_KEY/S3_SECRET_KEY env vars
- Remove Traefik labels from internal-only services
- Health aggregator no longer checks external services
- Core now has 10 services: valkey + 9 application services

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Sharang Parnerkar
84002f5719 feat: add Coolify deployment configuration
Add docker-compose.coolify.yml (17 services), .env.coolify.example,
and Gitea Action workflow for Coolify API deployment. Removes nginx,
vault, gitea, woodpecker, mailpit, and dev-only services. Adds Traefik
labels for *.breakpilot.ai domain routing with Let's Encrypt SSL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:16:59 +01:00
Benjamin Admin
8b87b90cbb fix(qdrant): Increase ulimits for RocksDB (Too many open files)
All checks were successful
CI / nodejs-lint (push) Has been skipped
CI / test-python-voice (push) Successful in 33s
CI / test-bqas (push) Successful in 31s
CI / deploy-hetzner (push) Successful in 40s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 32s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:31:16 +01:00
Benjamin Admin
be45adb975 fix(rag): Auto-create Qdrant collection on first index
All checks were successful
CI / go-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 33s
CI / test-python-voice (push) Successful in 36s
CI / deploy-hetzner (push) Successful in 38s
CI / python-lint (push) Has been skipped
CI / test-bqas (push) Successful in 31s
Collections may not exist if init_collections() failed at startup
(e.g. Qdrant not ready). Now index_documents() ensures the
collection exists before upserting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:02:05 +01:00
Benjamin Admin
7c932c441f feat(rag): Add bp_compliance_gesetze + bp_compliance_ce collections
All checks were successful
CI / go-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 35s
CI / test-python-voice (push) Successful in 50s
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-bqas (push) Successful in 33s
CI / deploy-hetzner (push) Successful in 39s
Required for Verbraucherschutz + EU law ingestion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:41:26 +01:00
Benjamin Admin
1eb402b3da fix(ci): Remove Ollama host port binding — port 11434 already in use
All checks were successful
CI / nodejs-lint (push) Has been skipped
CI / test-python-voice (push) Successful in 33s
CI / test-bqas (push) Successful in 31s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 31s
CI / deploy-hetzner (push) Successful in 1m18s
Host already has Ollama running (LibreChat). Our container only needs
internal docker network access via container name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:04:32 +01:00
Benjamin Admin
963e824328 fix(ci): Use external network + pre-create breakpilot-network
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-bqas (push) Successful in 30s
CI / deploy-hetzner (push) Failing after 15s
CI / test-go-consent (push) Successful in 33s
CI / test-python-voice (push) Successful in 35s
Network already exists from compliance project — use external: true
and pre-create with docker network create before docker compose up.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:01:17 +01:00
Benjamin Admin
c0782e0039 fix(ci): Fix backend-core TARGETARCH for amd64 + set -e in deploy
Some checks failed
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 33s
CI / deploy-hetzner (push) Failing after 1m17s
CI / test-python-voice (push) Successful in 33s
CI / test-bqas (push) Successful in 33s
- backend-core Dockerfile defaults TARGETARCH=arm64, override with build arg
- Add set -e in helper container to fail fast on build errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:51:19 +01:00
Benjamin Admin
44d66e2d6c feat(ci): Add Hetzner deployment for Core services
All checks were successful
CI / test-go-consent (push) Successful in 32s
CI / test-python-voice (push) Successful in 35s
CI / test-bqas (push) Successful in 34s
CI / deploy-hetzner (push) Successful in 3m29s
CI / go-lint (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
- docker-compose.hetzner.yml: Override for x86_64 (platform, ports,
  Ollama container for CPU embeddings, mailpit dummy, disabled services)
- CI: deploy-hetzner job using helper-container pattern
- Services: postgres, valkey, qdrant, ollama, backend-core, consent-service,
  rag-service, embedding-service, health-aggregator

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:42:41 +01:00
Benjamin Admin
f9b475db8f fix: Ensure public/ dir exists in Docker build for levis-holzbau
All checks were successful
CI / python-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / test-go-consent (push) Successful in 38s
CI / test-python-voice (push) Successful in 35s
CI / test-bqas (push) Successful in 38s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 10:06:54 +01:00
13 changed files with 646 additions and 5 deletions

65
.env.coolify.example Normal file
View File

@@ -0,0 +1,65 @@
# =========================================================
# BreakPilot Core — Coolify Environment Variables
# =========================================================
# Copy these into Coolify's environment variable UI
# for the breakpilot-core Docker Compose resource.
# =========================================================
# --- External PostgreSQL (Coolify-managed) ---
POSTGRES_HOST=<coolify-postgres-hostname>
POSTGRES_PORT=5432
POSTGRES_USER=breakpilot
POSTGRES_PASSWORD=CHANGE_ME_STRONG_PASSWORD
POSTGRES_DB=breakpilot_db
# --- Security ---
JWT_SECRET=CHANGE_ME_RANDOM_64_CHARS
JWT_REFRESH_SECRET=CHANGE_ME_ANOTHER_RANDOM_64_CHARS
INTERNAL_API_KEY=CHANGE_ME_INTERNAL_KEY
# --- External S3 Storage ---
S3_ENDPOINT=<s3-endpoint-host:port>
S3_ACCESS_KEY=CHANGE_ME_S3_ACCESS_KEY
S3_SECRET_KEY=CHANGE_ME_S3_SECRET_KEY
S3_BUCKET=breakpilot-rag
S3_SECURE=true
# --- External Qdrant (Coolify-managed) ---
QDRANT_URL=http://<coolify-qdrant-hostname>:6333
QDRANT_API_KEY=
# --- SMTP (Real mail server) ---
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USERNAME=noreply@breakpilot.ai
SMTP_PASSWORD=CHANGE_ME_SMTP_PASSWORD
SMTP_FROM_NAME=BreakPilot
SMTP_FROM_ADDR=noreply@breakpilot.ai
# --- Session ---
SESSION_TTL_HOURS=24
# --- Frontend URLs (build args) ---
NEXT_PUBLIC_CORE_API_URL=https://api-core.breakpilot.ai
FRONTEND_URL=https://www.breakpilot.ai
# --- Stripe (Billing) ---
STRIPE_SECRET_KEY=
STRIPE_WEBHOOK_SECRET=
STRIPE_PUBLISHABLE_KEY=
BILLING_SUCCESS_URL=https://www.breakpilot.ai/billing/success
BILLING_CANCEL_URL=https://www.breakpilot.ai/billing/cancel
TRIAL_PERIOD_DAYS=14
# --- Embedding Service ---
EMBEDDING_BACKEND=local
LOCAL_EMBEDDING_MODEL=BAAI/bge-m3
LOCAL_RERANKER_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
PDF_EXTRACTION_BACKEND=pymupdf
OPENAI_API_KEY=
COHERE_API_KEY=
LOG_LEVEL=INFO
# --- Ollama (optional, for RAG embeddings) ---
OLLAMA_URL=
OLLAMA_EMBED_MODEL=bge-m3

View File

@@ -138,3 +138,22 @@ jobs:
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
pip install --quiet --no-cache-dir fastapi uvicorn pydantic pytest pytest-asyncio
python -m pytest tests/bqas/ -v --tb=short || true
# ========================================
# Deploy via Coolify (nur main, kein PR)
# ========================================
deploy-coolify:
name: Deploy
runs-on: docker
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs:
- test-go-consent
container:
image: alpine:latest
steps:
- name: Trigger Coolify deploy
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"

View File

@@ -0,0 +1,29 @@
name: Deploy to Coolify
on:
push:
branches:
- coolify
jobs:
deploy:
runs-on: docker
container: alpine:latest
steps:
- name: Deploy via Coolify API
run: |
apk add --no-cache curl
echo "Deploying breakpilot-core to Coolify..."
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-X POST \
-H "Authorization: Bearer ${{ secrets.COOLIFY_API_TOKEN }}" \
-H "Content-Type: application/json" \
-d '{"uuid": "${{ secrets.COOLIFY_RESOURCE_UUID }}", "force_rebuild": true}' \
"${{ secrets.COOLIFY_BASE_URL }}/api/v1/deploy")
echo "HTTP Status: $HTTP_STATUS"
if [ "$HTTP_STATUS" -ne 200 ] && [ "$HTTP_STATUS" -ne 201 ]; then
echo "Deployment failed with status $HTTP_STATUS"
exit 1
fi
echo "Deployment triggered successfully!"

View File

@@ -18,6 +18,9 @@ ARG NEXT_PUBLIC_API_URL
# Set environment variables for build
ENV NEXT_PUBLIC_API_URL=$NEXT_PUBLIC_API_URL
# Ensure public directory exists
RUN mkdir -p public
# Build the application
RUN npm run build
@@ -30,8 +33,8 @@ WORKDIR /app
ENV NODE_ENV=production
# Create non-root user
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
RUN addgroup -S -g 1001 nodejs
RUN adduser -S -u 1001 -G nodejs nextjs
# Copy built assets
COPY --from=builder /app/public ./public

View File

@@ -43,11 +43,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
# Install DevSecOps tools (gitleaks, trivy, grype, syft)
ARG TARGETARCH=arm64
ARG TARGETARCH
RUN set -eux; \
ARCH="${TARGETARCH:-$(dpkg --print-architecture)}"; \
# Gitleaks
GITLEAKS_VERSION=8.21.2; \
if [ "$TARGETARCH" = "arm64" ]; then GITLEAKS_ARCH=arm64; else GITLEAKS_ARCH=x64; fi; \
if [ "$ARCH" = "arm64" ]; then GITLEAKS_ARCH=arm64; else GITLEAKS_ARCH=x64; fi; \
curl -sSfL "https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_${GITLEAKS_ARCH}.tar.gz" \
| tar xz -C /usr/local/bin gitleaks; \
# Trivy

197
docker-compose.coolify.yml Normal file
View File

@@ -0,0 +1,197 @@
# =========================================================
# BreakPilot Core — Shared Infrastructure (Coolify)
# =========================================================
# Deployed via Coolify. SSL termination handled by Traefik.
# External services (managed separately in Coolify):
# - PostgreSQL (PostGIS), Qdrant, S3-compatible storage
# Network: breakpilot-network (shared across all 3 repos)
# =========================================================
networks:
breakpilot-network:
name: breakpilot-network
driver: bridge
volumes:
valkey_data:
embedding_models:
paddleocr_models:
services:
# =========================================================
# CACHE
# =========================================================
valkey:
image: valkey/valkey:8-alpine
container_name: bp-core-valkey
volumes:
- valkey_data:/data
command: valkey-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
restart: unless-stopped
networks:
- breakpilot-network
# =========================================================
# SHARED SERVICES
# =========================================================
consent-service:
build:
context: ./consent-service
dockerfile: Dockerfile
container_name: bp-core-consent-service
expose:
- "8081"
environment:
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT:-5432}/${POSTGRES_DB}
JWT_SECRET: ${JWT_SECRET}
JWT_REFRESH_SECRET: ${JWT_REFRESH_SECRET}
PORT: 8081
ENVIRONMENT: production
ALLOWED_ORIGINS: "*"
VALKEY_URL: redis://valkey:6379/0
SESSION_TTL_HOURS: ${SESSION_TTL_HOURS:-24}
SMTP_HOST: ${SMTP_HOST}
SMTP_PORT: ${SMTP_PORT:-587}
SMTP_USERNAME: ${SMTP_USERNAME}
SMTP_PASSWORD: ${SMTP_PASSWORD}
SMTP_FROM_NAME: ${SMTP_FROM_NAME:-BreakPilot}
SMTP_FROM_ADDR: ${SMTP_FROM_ADDR:-noreply@breakpilot.ai}
FRONTEND_URL: ${FRONTEND_URL:-https://www.breakpilot.ai}
depends_on:
valkey:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:8081/health"]
interval: 30s
timeout: 10s
start_period: 15s
retries: 3
restart: unless-stopped
networks:
- breakpilot-network
# =========================================================
# RAG & EMBEDDING SERVICES
# =========================================================
rag-service:
build:
context: ./rag-service
dockerfile: Dockerfile
container_name: bp-core-rag-service
expose:
- "8097"
environment:
PORT: 8097
QDRANT_URL: ${QDRANT_URL}
QDRANT_API_KEY: ${QDRANT_API_KEY:-}
MINIO_ENDPOINT: ${S3_ENDPOINT}
MINIO_ACCESS_KEY: ${S3_ACCESS_KEY}
MINIO_SECRET_KEY: ${S3_SECRET_KEY}
MINIO_BUCKET: ${S3_BUCKET:-breakpilot-rag}
MINIO_SECURE: ${S3_SECURE:-true}
EMBEDDING_SERVICE_URL: http://embedding-service:8087
OLLAMA_URL: ${OLLAMA_URL:-}
OLLAMA_EMBED_MODEL: ${OLLAMA_EMBED_MODEL:-bge-m3}
JWT_SECRET: ${JWT_SECRET}
ENVIRONMENT: production
depends_on:
embedding-service:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8097/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
restart: unless-stopped
networks:
- breakpilot-network
embedding-service:
build:
context: ./embedding-service
dockerfile: Dockerfile
container_name: bp-core-embedding-service
volumes:
- embedding_models:/root/.cache/huggingface
environment:
EMBEDDING_BACKEND: ${EMBEDDING_BACKEND:-local}
LOCAL_EMBEDDING_MODEL: ${LOCAL_EMBEDDING_MODEL:-BAAI/bge-m3}
LOCAL_RERANKER_MODEL: ${LOCAL_RERANKER_MODEL:-cross-encoder/ms-marco-MiniLM-L-6-v2}
PDF_EXTRACTION_BACKEND: ${PDF_EXTRACTION_BACKEND:-pymupdf}
OPENAI_API_KEY: ${OPENAI_API_KEY:-}
COHERE_API_KEY: ${COHERE_API_KEY:-}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
deploy:
resources:
limits:
memory: 8G
healthcheck:
test: ["CMD", "python", "-c", "import httpx; r=httpx.get('http://127.0.0.1:8087/health'); r.raise_for_status()"]
interval: 30s
timeout: 10s
start_period: 120s
retries: 3
restart: unless-stopped
networks:
- breakpilot-network
# =========================================================
# OCR SERVICE (PaddleOCR PP-OCRv5)
# =========================================================
paddleocr-service:
build:
context: ./paddleocr-service
dockerfile: Dockerfile
container_name: bp-core-paddleocr
expose:
- "8095"
environment:
PADDLEOCR_API_KEY: ${PADDLEOCR_API_KEY:-}
FLAGS_use_mkldnn: "0"
volumes:
- paddleocr_models:/root/.paddleocr
labels:
- "traefik.http.services.paddleocr.loadbalancer.server.port=8095"
deploy:
resources:
limits:
memory: 6G
healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8095/health"]
interval: 30s
timeout: 10s
start_period: 300s
retries: 5
restart: unless-stopped
networks:
- breakpilot-network
# =========================================================
# HEALTH AGGREGATOR
# =========================================================
health-aggregator:
build:
context: ./scripts
dockerfile: Dockerfile.health
container_name: bp-core-health
expose:
- "8099"
environment:
PORT: 8099
CHECK_SERVICES: "valkey:6379,consent-service:8081,rag-service:8097,embedding-service:8087,paddleocr-service:8095"
healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1:8099/health"]
interval: 30s
timeout: 10s
retries: 3
restart: unless-stopped
networks:
- breakpilot-network

175
docker-compose.hetzner.yml Normal file
View File

@@ -0,0 +1,175 @@
# =========================================================
# BreakPilot Core — Hetzner Override (x86_64)
# =========================================================
# Verwendung:
# docker compose -f docker-compose.yml -f docker-compose.hetzner.yml up -d \
# postgres valkey qdrant ollama embedding-service rag-service \
# backend-core consent-service health-aggregator
#
# Aenderungen gegenueber Basis (docker-compose.yml):
# - platform: linux/amd64 (statt arm64)
# - Ollama Container fuer CPU-Embeddings (bge-m3)
# - Mailpit ersetzt durch Dummy (kein Mail-Dev-Server noetig)
# - Vault, Nginx, Gitea etc. deaktiviert via Profile
# - Netzwerk: auto-create (nicht external)
# =========================================================
networks:
breakpilot-network:
external: true
name: breakpilot-network
services:
# =========================================================
# NEUE SERVICES
# =========================================================
# Ollama fuer Embeddings (CPU-only, bge-m3)
ollama:
image: ollama/ollama:latest
container_name: bp-core-ollama
platform: linux/amd64
volumes:
- ollama_models:/root/.ollama
healthcheck:
test: ["CMD-SHELL", "curl -sf http://127.0.0.1:11434/api/tags || exit 1"]
interval: 15s
timeout: 10s
retries: 5
start_period: 30s
restart: unless-stopped
networks:
- breakpilot-network
# =========================================================
# PLATFORM OVERRIDES (arm64 → amd64)
# =========================================================
backend-core:
platform: linux/amd64
build:
context: ./backend-core
dockerfile: Dockerfile
args:
TARGETARCH: amd64
ports:
- "8000:8000"
environment:
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-breakpilot}:${POSTGRES_PASSWORD:-breakpilot123}@postgres:5432/${POSTGRES_DB:-breakpilot_db}?options=-csearch_path%3Dcore,public
JWT_SECRET: ${JWT_SECRET:-your-super-secret-jwt-key-change-in-production}
ENVIRONMENT: ${ENVIRONMENT:-production}
VALKEY_URL: redis://valkey:6379/0
SESSION_TTL_HOURS: ${SESSION_TTL_HOURS:-24}
CONSENT_SERVICE_URL: http://consent-service:8081
USE_VAULT_SECRETS: "false"
SMTP_HOST: ${SMTP_HOST:-smtp.example.com}
SMTP_PORT: ${SMTP_PORT:-587}
SMTP_USERNAME: ${SMTP_USERNAME:-}
SMTP_PASSWORD: ${SMTP_PASSWORD:-}
SMTP_FROM_NAME: ${SMTP_FROM_NAME:-BreakPilot}
SMTP_FROM_ADDR: ${SMTP_FROM_ADDR:-noreply@breakpilot.app}
consent-service:
platform: linux/amd64
environment:
DATABASE_URL: postgres://${POSTGRES_USER:-breakpilot}:${POSTGRES_PASSWORD:-breakpilot123}@postgres:5432/${POSTGRES_DB:-breakpilot_db}
JWT_SECRET: ${JWT_SECRET:-your-super-secret-jwt-key-change-in-production}
JWT_REFRESH_SECRET: ${JWT_REFRESH_SECRET:-your-refresh-secret}
PORT: 8081
ENVIRONMENT: ${ENVIRONMENT:-production}
ALLOWED_ORIGINS: "*"
VALKEY_URL: redis://valkey:6379/0
SESSION_TTL_HOURS: ${SESSION_TTL_HOURS:-24}
SMTP_HOST: ${SMTP_HOST:-smtp.example.com}
SMTP_PORT: ${SMTP_PORT:-587}
SMTP_USERNAME: ${SMTP_USERNAME:-}
SMTP_PASSWORD: ${SMTP_PASSWORD:-}
SMTP_FROM_NAME: ${SMTP_FROM_NAME:-BreakPilot}
SMTP_FROM_ADDR: ${SMTP_FROM_ADDR:-noreply@breakpilot.app}
FRONTEND_URL: ${FRONTEND_URL:-https://admin-dev.breakpilot.ai}
billing-service:
platform: linux/amd64
rag-service:
platform: linux/amd64
ports:
- "8097:8097"
environment:
PORT: 8097
QDRANT_URL: http://qdrant:6333
MINIO_ENDPOINT: nbg1.your-objectstorage.com
MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY:-T18RGFVXXG2ZHQ5404TP}
MINIO_SECRET_KEY: ${MINIO_SECRET_KEY:-KOUU4WO6wh07cQjNgh0IZHkeKQrVfBz6hnIGpNss}
MINIO_BUCKET: ${MINIO_BUCKET:-breakpilot-rag}
MINIO_SECURE: "true"
EMBEDDING_SERVICE_URL: http://embedding-service:8087
OLLAMA_URL: http://ollama:11434
OLLAMA_EMBED_MODEL: ${OLLAMA_EMBED_MODEL:-bge-m3}
JWT_SECRET: ${JWT_SECRET:-your-super-secret-jwt-key-change-in-production}
ENVIRONMENT: ${ENVIRONMENT:-production}
embedding-service:
platform: linux/amd64
ports:
- "8087:8087"
health-aggregator:
platform: linux/amd64
environment:
PORT: 8099
CHECK_SERVICES: "postgres:5432,valkey:6379,qdrant:6333,backend-core:8000,rag-service:8097,embedding-service:8087"
# =========================================================
# DUMMY-ERSATZ FUER ABHAENGIGKEITEN
# =========================================================
# backend-core + consent-service haengen von mailpit ab
# (depends_on merged bei compose override, kann nicht entfernt werden)
# → Mailpit durch leichtgewichtigen Dummy ersetzen
mailpit:
image: alpine:3.19
entrypoint: ["sh", "-c", "echo 'Mailpit dummy on Hetzner' && tail -f /dev/null"]
volumes: []
ports: []
environment: {}
# Qdrant: RocksDB braucht mehr open files
qdrant:
ulimits:
nofile:
soft: 65536
hard: 65536
# minio: rag-service haengt davon ab (depends_on)
# Lokal laufen lassen, aber rag-service nutzt externe Hetzner Object Storage
# minio bleibt unveraendert (klein, ~50MB RAM)
# =========================================================
# DEAKTIVIERTE SERVICES (via profiles)
# =========================================================
nginx:
profiles: ["disabled"]
vault:
profiles: ["disabled"]
vault-init:
profiles: ["disabled"]
vault-agent:
profiles: ["disabled"]
gitea:
profiles: ["disabled"]
gitea-runner:
profiles: ["disabled"]
night-scheduler:
profiles: ["disabled"]
admin-core:
profiles: ["disabled"]
pitch-deck:
profiles: ["disabled"]
levis-holzbau:
profiles: ["disabled"]
volumes:
ollama_models:

View File

@@ -9,6 +9,7 @@ FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN mkdir -p public
RUN npm run build
FROM base AS runner

View File

@@ -0,0 +1,16 @@
FROM python:3.11-slim
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
libgl1 libglib2.0-0 libgomp1 curl \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8095
HEALTHCHECK --interval=30s --timeout=10s --start-period=120s --retries=3 \
CMD curl -f http://127.0.0.1:8095/health || exit 1
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8095"]

110
paddleocr-service/main.py Normal file
View File

@@ -0,0 +1,110 @@
"""PaddleOCR Remote Service — PP-OCRv4 on x86_64 (CPU)."""
import io
import logging
import os
import threading
import numpy as np
from fastapi import FastAPI, File, Header, HTTPException, UploadFile
from PIL import Image
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI(title="PaddleOCR Service")
_engine = None
_ready = False
_loading = False
API_KEY = os.environ.get("PADDLEOCR_API_KEY", "")
def _load_model():
"""Load PaddleOCR model in background thread."""
global _engine, _ready
try:
logger.info("Importing paddleocr...")
from paddleocr import PaddleOCR
logger.info("Loading PaddleOCR model (PP-OCRv4, lang=en)...")
_engine = PaddleOCR(
lang="en",
use_angle_cls=True,
show_log=False,
enable_mkldnn=False,
use_gpu=False,
)
logger.info("PaddleOCR model loaded — running warmup...")
# Warmup with tiny image to trigger any lazy init
dummy = np.ones((30, 100, 3), dtype=np.uint8) * 255
_engine.ocr(dummy)
_ready = True
logger.info("PaddleOCR ready to serve")
except Exception as e:
logger.error(f"Failed to load PaddleOCR: {e}", exc_info=True)
@app.on_event("startup")
def startup_load_model():
"""Start model loading in background so health check passes immediately."""
global _loading
_loading = True
threading.Thread(target=_load_model, daemon=True).start()
logger.info("Model loading started in background thread")
@app.get("/health")
def health():
if _ready:
return {"status": "ok", "model": "PP-OCRv4"}
if _loading:
return {"status": "loading"}
return {"status": "error"}
@app.post("/ocr")
async def ocr(
file: UploadFile = File(...),
x_api_key: str = Header(default=""),
):
if API_KEY and x_api_key != API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
if not _ready:
raise HTTPException(status_code=503, detail="Model still loading")
img_bytes = await file.read()
img = Image.open(io.BytesIO(img_bytes)).convert("RGB")
img_np = np.array(img)
try:
result = _engine.ocr(img_np)
except Exception as e:
logger.error(f"OCR failed: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=f"OCR failed: {e}")
if not result or not result[0]:
return {"words": [], "image_width": img_np.shape[1], "image_height": img_np.shape[0]}
words = []
for line in result[0]:
box, (text, conf) = line[0], line[1]
x_min = min(p[0] for p in box)
y_min = min(p[1] for p in box)
x_max = max(p[0] for p in box)
y_max = max(p[1] for p in box)
words.append({
"text": str(text).strip(),
"left": int(x_min),
"top": int(y_min),
"width": int(x_max - x_min),
"height": int(y_max - y_min),
"conf": round(float(conf) * 100, 1),
})
return {
"words": words,
"image_width": img_np.shape[1],
"image_height": img_np.shape[0],
}

View File

@@ -0,0 +1,7 @@
paddlepaddle>=2.6.0,<3.0.0
paddleocr>=2.7.0,<3.0.0
fastapi>=0.110.0
uvicorn>=0.25.0
python-multipart>=0.0.6
Pillow>=10.0.0
numpy>=1.24.0

View File

@@ -6,6 +6,7 @@ class Settings:
# Qdrant
QDRANT_URL: str = os.getenv("QDRANT_URL", "http://localhost:6333")
QDRANT_API_KEY: str = os.getenv("QDRANT_API_KEY", "")
# MinIO
MINIO_ENDPOINT: str = os.getenv("MINIO_ENDPOINT", "localhost:9000")

View File

@@ -27,6 +27,8 @@ _COMPLIANCE_COLLECTIONS = {
"bp_compliance_gdpr": 1024,
"bp_compliance_schulrecht": 1024,
"bp_compliance_datenschutz": 1024,
"bp_compliance_gesetze": 1024,
"bp_compliance_ce": 1024,
"bp_dsfa_templates": 1024,
"bp_dsfa_risks": 1024,
}
@@ -46,7 +48,11 @@ class QdrantClientWrapper:
@property
def client(self) -> QdrantClient:
if self._client is None:
self._client = QdrantClient(url=settings.QDRANT_URL, timeout=30)
self._client = QdrantClient(
url=settings.QDRANT_URL,
api_key=settings.QDRANT_API_KEY or None,
timeout=30,
)
logger.info("Connected to Qdrant at %s", settings.QDRANT_URL)
return self._client
@@ -103,6 +109,13 @@ class QdrantClientWrapper:
# Indexing
# ------------------------------------------------------------------
async def ensure_collection(self, name: str, vector_size: int = 1024) -> None:
"""Create collection if it doesn't exist."""
try:
self.client.get_collection(name)
except Exception:
await self.create_collection(name, vector_size)
async def index_documents(
self,
collection: str,
@@ -116,6 +129,10 @@ class QdrantClientWrapper:
f"vectors ({len(vectors)}) and payloads ({len(payloads)}) must have equal length"
)
# Auto-create collection if missing
vector_size = len(vectors[0]) if vectors else 1024
await self.ensure_collection(collection, vector_size)
if ids is None:
ids = [str(uuid.uuid4()) for _ in vectors]