Compare commits
4 Commits
coolify
...
b4d2be83eb
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b4d2be83eb | ||
|
|
38c7cf0a00 | ||
|
|
399fa62267 | ||
| f1710fdb9e |
@@ -2,53 +2,49 @@
|
||||
|
||||
## Entwicklungsumgebung (WICHTIG - IMMER ZUERST LESEN)
|
||||
|
||||
### Zwei-Rechner-Setup + Hetzner
|
||||
### Zwei-Rechner-Setup + Coolify
|
||||
|
||||
| Geraet | Rolle | Aufgaben |
|
||||
|--------|-------|----------|
|
||||
| **MacBook** | Entwicklung | Claude Terminal, Code-Entwicklung, Browser (Frontend-Tests) |
|
||||
| **Mac Mini** | Lokaler Server | Docker fuer lokale Dev/Tests (NICHT mehr fuer Production!) |
|
||||
| **Hetzner** | Production | CI/CD Build + Deploy via Gitea Actions |
|
||||
| **Mac Mini** | Lokaler Server | Docker fuer lokale Dev/Tests (NICHT fuer Production!) |
|
||||
| **Coolify** | Production | Automatisches Build + Deploy bei Push auf gitea |
|
||||
|
||||
**WICHTIG:** Code wird auf dem MacBook bearbeitet. Production-Deployment laeuft automatisch auf Hetzner via CI/CD.
|
||||
**WICHTIG:** Code wird auf dem MacBook bearbeitet. Production-Deployment laeuft automatisch ueber Coolify.
|
||||
|
||||
### Entwicklungsworkflow (CI/CD — seit 2026-03-11)
|
||||
### Entwicklungsworkflow (CI/CD — Coolify)
|
||||
|
||||
```bash
|
||||
# 1. Code auf MacBook bearbeiten (dieses Verzeichnis)
|
||||
# 2. Committen und zu BEIDEN Remotes pushen:
|
||||
git push origin main && git push gitea main
|
||||
|
||||
# 3. FERTIG! Gitea Actions auf Hetzner uebernimmt automatisch:
|
||||
# Push auf main → Lint → Tests → Build → Deploy
|
||||
# Pipeline: .gitea/workflows/ci.yaml
|
||||
# 3. FERTIG! Push auf gitea triggert automatisch:
|
||||
# - Gitea Actions: Lint → Tests → Validierung
|
||||
# - Coolify: Build → Deploy
|
||||
# Dauer: ca. 3 Minuten
|
||||
# Status pruefen: https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-compliance/actions
|
||||
```
|
||||
|
||||
**NICHT MEHR NOETIG:** Manuelles `ssh macmini "docker compose build"` — das macht jetzt die CI/CD Pipeline!
|
||||
**NICHT MEHR NOETIG:** Manuelles `ssh macmini "docker compose build"` fuer Production.
|
||||
**NIEMALS** manuell in Coolify auf "Redeploy" klicken — Gitea Actions triggert Coolify automatisch.
|
||||
|
||||
### CI/CD Pipeline (Gitea Actions → Hetzner)
|
||||
### CI/CD Pipeline (Gitea Actions → Coolify)
|
||||
|
||||
```
|
||||
Push auf main → go-lint/python-lint/nodejs-lint (nur PRs)
|
||||
→ test-go-ai-compliance
|
||||
→ test-python-backend-compliance
|
||||
→ test-python-document-crawler
|
||||
→ test-python-dsms-gateway
|
||||
→ deploy-hetzner (nur wenn ALLE Tests gruen)
|
||||
Push auf gitea main → go-lint/python-lint/nodejs-lint (nur PRs)
|
||||
→ test-go-ai-compliance
|
||||
→ test-python-backend-compliance
|
||||
→ test-python-document-crawler
|
||||
→ test-python-dsms-gateway
|
||||
→ validate-canonical-controls
|
||||
→ Coolify: Build + Deploy (automatisch bei Push)
|
||||
```
|
||||
|
||||
**Dateien:**
|
||||
- `.gitea/workflows/ci.yaml` — Pipeline-Definition
|
||||
- `docker-compose.hetzner.yml` — Override: arm64→amd64 fuer Hetzner (x86_64)
|
||||
- Deploy-Pfad auf Hetzner: `/opt/breakpilot-compliance/`
|
||||
|
||||
**Ablauf deploy-hetzner:**
|
||||
1. `git pull` im Deploy-Dir
|
||||
2. `docker compose -f docker-compose.yml -f docker-compose.hetzner.yml build --parallel`
|
||||
3. `docker compose up -d --remove-orphans`
|
||||
4. Health Checks
|
||||
- `.gitea/workflows/ci.yaml` — Pipeline-Definition (Tests + Validierung)
|
||||
- `docker-compose.yml` — Haupt-Compose
|
||||
- `docker-compose.hetzner.yml` — Override: arm64→amd64 fuer Coolify Production (x86_64)
|
||||
|
||||
### Lokale Entwicklung (Mac Mini — optional)
|
||||
|
||||
@@ -76,20 +72,18 @@ rsync -avz --exclude node_modules --exclude .next --exclude .git \
|
||||
- RAG-Service (Vektorsuche fuer Compliance-Dokumente)
|
||||
- Nginx (Reverse Proxy)
|
||||
|
||||
**Externe Services (Hetzner/meghshakka) — seit 2026-03-06:**
|
||||
- PostgreSQL 17 @ `46.225.100.82:54321` (sslmode=require) — Schemas: `compliance` (51), `public` (compliance_* + training_* + ucca_* + academy_*)
|
||||
**Externe Services (Production):**
|
||||
- PostgreSQL 17 (sslmode=require) — Schemas: `compliance`, `public`
|
||||
- Qdrant @ `qdrant-dev.breakpilot.ai` (HTTPS, API-Key)
|
||||
- Object Storage @ `nbg1.your-objectstorage.com` (S3-kompatibel, TLS)
|
||||
- Object Storage (S3-kompatibel, TLS)
|
||||
|
||||
Config via `.env` auf Mac Mini (nicht im Repo): `COMPLIANCE_DATABASE_URL`, `QDRANT_URL`, `QDRANT_API_KEY`
|
||||
|
||||
Pruefen: `curl -sf http://macmini:8099/health`
|
||||
Config via `.env` (nicht im Repo): `COMPLIANCE_DATABASE_URL`, `QDRANT_URL`, `QDRANT_API_KEY`
|
||||
|
||||
---
|
||||
|
||||
## Haupt-URLs
|
||||
|
||||
### Production (Hetzner — primaer)
|
||||
### Production (Coolify-deployed)
|
||||
|
||||
| URL | Service | Beschreibung |
|
||||
|-----|---------|--------------|
|
||||
@@ -145,18 +139,6 @@ Pruefen: `curl -sf http://macmini:8099/health`
|
||||
| docs | MkDocs/nginx | 8011 | bp-compliance-docs |
|
||||
| core-wait | curl health-check | - | bp-compliance-core-wait |
|
||||
|
||||
### compliance-tts-service
|
||||
- Piper TTS + FFmpeg fuer Schulungsvideos
|
||||
- Speichert Audio/Video in Hetzner Object Storage (nbg1.your-objectstorage.com)
|
||||
- TTS-Modell: `de_DE-thorsten-high.onnx`
|
||||
- Dateien: `main.py`, `tts_engine.py`, `video_generator.py`, `storage.py`
|
||||
|
||||
### document-crawler
|
||||
- Dokument-Analyse: PDF, DOCX, XLSX, PPTX
|
||||
- Gap-Analyse zwischen bestehenden Dokumenten und Compliance-Anforderungen
|
||||
- IPFS-Archivierung via dsms-gateway
|
||||
- Kommuniziert mit ai-compliance-sdk (LLM Gateway)
|
||||
|
||||
### Docker-Netzwerk
|
||||
Nutzt das externe Core-Netzwerk:
|
||||
```yaml
|
||||
@@ -202,8 +184,8 @@ breakpilot-compliance/
|
||||
├── dsms-gateway/ # IPFS Gateway
|
||||
├── scripts/ # Helper Scripts
|
||||
├── docker-compose.yml # Compliance Compose (~10 Services, platform: arm64)
|
||||
├── docker-compose.hetzner.yml # Override: arm64→amd64 fuer Hetzner
|
||||
└── .gitea/workflows/ci.yaml # CI/CD Pipeline (Lint → Tests → Deploy)
|
||||
├── docker-compose.hetzner.yml # Override: arm64→amd64 fuer Coolify Production
|
||||
└── .gitea/workflows/ci.yaml # CI/CD Pipeline (Lint → Tests → Validierung)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -213,7 +195,7 @@ breakpilot-compliance/
|
||||
### Deployment (CI/CD — Standardweg)
|
||||
|
||||
```bash
|
||||
# Committen und pushen → CI/CD deployt automatisch auf Hetzner:
|
||||
# Committen und pushen → Coolify deployt automatisch:
|
||||
git push origin main && git push gitea main
|
||||
|
||||
# CI-Status pruefen (im Browser):
|
||||
@@ -326,10 +308,6 @@ DELETE /api/v1/projects/{project_id} → Projekt archivieren (Soft Delete)
|
||||
- `app/sdk/layout.tsx` — liest `?project=` aus searchParams
|
||||
- `app/api/sdk/v1/projects/` — Next.js Proxy zum Backend
|
||||
|
||||
**Multi-Tab:** Tab A (Projekt X) und Tab B (Projekt Y) interferieren nicht — separate BroadcastChannel + localStorage Keys.
|
||||
|
||||
**Stammdaten-Kopie:** Neues Projekt mit `copy_from_project_id` → Backend kopiert `companyProfile` aus dem Quell-State. Danach unabhaengig editierbar.
|
||||
|
||||
### Backend-Compliance APIs
|
||||
```
|
||||
POST/GET /api/v1/compliance/risks
|
||||
@@ -340,7 +318,7 @@ POST/GET /api/v1/dsr/requests
|
||||
POST/GET /api/v1/gdpr/exports
|
||||
POST/GET /api/v1/consent/admin
|
||||
|
||||
# Stammdaten, Versionierung & Change-Requests (Phase 1-6, 2026-03-07)
|
||||
# Stammdaten, Versionierung & Change-Requests
|
||||
GET/POST/DELETE /api/compliance/company-profile
|
||||
GET /api/compliance/company-profile/template-context
|
||||
GET /api/compliance/change-requests
|
||||
@@ -358,24 +336,6 @@ GET /api/compliance/{doc}/{id}/versions
|
||||
- UUID-Format, kein `"default"` mehr
|
||||
- Header `X-Tenant-ID` > Query `tenant_id` > ENV-Fallback
|
||||
|
||||
### Migrations (035-038)
|
||||
| Nr | Datei | Beschreibung |
|
||||
|----|-------|--------------|
|
||||
| 035 | `migrations/035_vvt_tenant_isolation.sql` | VVT tenant_id + DSFA/Vendor default→UUID |
|
||||
| 036 | `migrations/036_company_profile_extend.sql` | Stammdaten JSONB + Regulierungs-Flags |
|
||||
| 037 | `migrations/037_document_versions.sql` | 5 Versions-Tabellen + current_version |
|
||||
| 038 | `migrations/038_change_requests.sql` | Change-Requests + Audit-Log |
|
||||
|
||||
### Neue Backend-Module
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `compliance/api/tenant_utils.py` | Shared Tenant-ID Dependency |
|
||||
| `compliance/api/versioning_utils.py` | Shared Versioning Helper |
|
||||
| `compliance/api/change_request_routes.py` | CR CRUD + Accept/Reject/Edit |
|
||||
| `compliance/api/change_request_engine.py` | Regelbasierte CR-Generierung |
|
||||
| `compliance/api/generation_routes.py` | Dokumentengenerierung aus Stammdaten |
|
||||
| `compliance/api/document_templates/` | 5 Template-Generatoren (DSFA, VVT, TOM, etc.) |
|
||||
|
||||
---
|
||||
|
||||
## Wichtige Dateien (Referenz)
|
||||
@@ -383,9 +343,7 @@ GET /api/compliance/{doc}/{id}/versions
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `admin-compliance/app/(sdk)/` | Alle 37+ SDK-Routes |
|
||||
| `admin-compliance/app/(sdk)/sdk/change-requests/page.tsx` | Change-Request Inbox |
|
||||
| `admin-compliance/components/sdk/Sidebar/SDKSidebar.tsx` | SDK Navigation (mit CR-Badge) |
|
||||
| `admin-compliance/components/sdk/VersionHistory.tsx` | Versions-Timeline-Komponente |
|
||||
| `admin-compliance/components/sdk/Sidebar/SDKSidebar.tsx` | SDK Navigation |
|
||||
| `admin-compliance/components/sdk/CommandBar.tsx` | Command Palette |
|
||||
| `admin-compliance/lib/sdk/context.tsx` | SDK State (Provider) |
|
||||
| `backend-compliance/compliance/` | Haupt-Package (50+ Dateien) |
|
||||
|
||||
61
.env.coolify.example
Normal file
61
.env.coolify.example
Normal file
@@ -0,0 +1,61 @@
|
||||
# =========================================================
|
||||
# BreakPilot Compliance — Coolify Environment Variables
|
||||
# =========================================================
|
||||
# Copy these into Coolify's environment variable UI
|
||||
# for the breakpilot-compliance Docker Compose resource.
|
||||
# =========================================================
|
||||
|
||||
# --- External PostgreSQL (Coolify-managed, same as Core) ---
|
||||
COMPLIANCE_DATABASE_URL=postgresql://breakpilot:CHANGE_ME@<coolify-postgres-hostname>:5432/breakpilot_db
|
||||
|
||||
# --- Security ---
|
||||
JWT_SECRET=CHANGE_ME_SAME_AS_CORE
|
||||
|
||||
# --- External S3 Storage (same as Core) ---
|
||||
S3_ENDPOINT=<s3-endpoint-host:port>
|
||||
S3_ACCESS_KEY=CHANGE_ME_SAME_AS_CORE
|
||||
S3_SECRET_KEY=CHANGE_ME_SAME_AS_CORE
|
||||
S3_SECURE=true
|
||||
|
||||
# --- External Qdrant ---
|
||||
QDRANT_URL=https://<qdrant-hostname>
|
||||
QDRANT_API_KEY=CHANGE_ME_QDRANT_API_KEY
|
||||
|
||||
# --- Session ---
|
||||
SESSION_TTL_HOURS=24
|
||||
|
||||
# --- SMTP (Real mail server) ---
|
||||
SMTP_HOST=smtp.example.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USERNAME=compliance@breakpilot.ai
|
||||
SMTP_PASSWORD=CHANGE_ME_SMTP_PASSWORD
|
||||
SMTP_FROM_NAME=BreakPilot Compliance
|
||||
SMTP_FROM_ADDR=compliance@breakpilot.ai
|
||||
|
||||
# --- LLM Configuration ---
|
||||
COMPLIANCE_LLM_PROVIDER=anthropic
|
||||
SELF_HOSTED_LLM_URL=
|
||||
SELF_HOSTED_LLM_MODEL=
|
||||
COMPLIANCE_LLM_MAX_TOKENS=4096
|
||||
COMPLIANCE_LLM_TEMPERATURE=0.3
|
||||
COMPLIANCE_LLM_TIMEOUT=120
|
||||
ANTHROPIC_API_KEY=CHANGE_ME_ANTHROPIC_KEY
|
||||
ANTHROPIC_DEFAULT_MODEL=claude-sonnet-4-5-20250929
|
||||
|
||||
# --- Ollama (optional) ---
|
||||
OLLAMA_URL=
|
||||
OLLAMA_DEFAULT_MODEL=
|
||||
COMPLIANCE_LLM_MODEL=
|
||||
|
||||
# --- LLM Fallback ---
|
||||
LLM_FALLBACK_PROVIDER=
|
||||
|
||||
# --- PII & Audit ---
|
||||
PII_REDACTION_ENABLED=true
|
||||
PII_REDACTION_LEVEL=standard
|
||||
AUDIT_RETENTION_DAYS=365
|
||||
AUDIT_LOG_PROMPTS=true
|
||||
|
||||
# --- Frontend URLs (build args) ---
|
||||
NEXT_PUBLIC_API_URL=https://api-compliance.breakpilot.ai
|
||||
NEXT_PUBLIC_SDK_URL=https://sdk.breakpilot.ai
|
||||
@@ -7,7 +7,7 @@
|
||||
# Node.js: admin-compliance, developer-portal
|
||||
#
|
||||
# Workflow:
|
||||
# Push auf main → Tests → Build → Deploy (Hetzner)
|
||||
# Push auf main → Tests → Deploy (Coolify)
|
||||
# Pull Request → Lint + Tests (kein Deploy)
|
||||
|
||||
name: CI/CD
|
||||
@@ -186,10 +186,11 @@ jobs:
|
||||
python scripts/validate-controls.py
|
||||
|
||||
# ========================================
|
||||
# Build & Deploy auf Hetzner (nur main, kein PR)
|
||||
# Deploy via Coolify (nur main, kein PR)
|
||||
# ========================================
|
||||
|
||||
deploy-hetzner:
|
||||
deploy-coolify:
|
||||
name: Deploy
|
||||
runs-on: docker
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
needs:
|
||||
@@ -198,92 +199,11 @@ jobs:
|
||||
- test-python-document-crawler
|
||||
- test-python-dsms-gateway
|
||||
- validate-canonical-controls
|
||||
container: docker:27-cli
|
||||
container:
|
||||
image: alpine:latest
|
||||
steps:
|
||||
- name: Deploy
|
||||
- name: Trigger Coolify deploy
|
||||
run: |
|
||||
set -euo pipefail
|
||||
DEPLOY_DIR="/opt/breakpilot-compliance"
|
||||
COMPOSE_FILES="-f docker-compose.yml -f docker-compose.hetzner.yml"
|
||||
COMMIT_SHA="${GITHUB_SHA:-unknown}"
|
||||
SHORT_SHA="${COMMIT_SHA:0:8}"
|
||||
REPO_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
|
||||
|
||||
echo "=== BreakPilot Compliance Deploy ==="
|
||||
echo "Commit: ${SHORT_SHA}"
|
||||
echo "Deploy Dir: ${DEPLOY_DIR}"
|
||||
echo ""
|
||||
|
||||
# Der Runner laeuft in einem Container mit Docker-Socket-Zugriff,
|
||||
# hat aber KEINEN direkten Zugriff auf das Host-Dateisystem.
|
||||
# Loesung: Alpine-Helper-Container mit Host-Bind-Mount fuer Git-Ops.
|
||||
|
||||
# 1. Repo auf dem Host erstellen/aktualisieren via Helper-Container
|
||||
echo "=== Updating code on host ==="
|
||||
docker run --rm \
|
||||
-v "${DEPLOY_DIR}:${DEPLOY_DIR}" \
|
||||
--entrypoint sh \
|
||||
alpine/git:latest \
|
||||
-c "
|
||||
if [ ! -d '${DEPLOY_DIR}/.git' ]; then
|
||||
echo 'Erstmaliges Klonen nach ${DEPLOY_DIR}...'
|
||||
git clone '${REPO_URL}' '${DEPLOY_DIR}'
|
||||
else
|
||||
cd '${DEPLOY_DIR}'
|
||||
git fetch origin main
|
||||
git reset --hard origin/main
|
||||
fi
|
||||
"
|
||||
echo "Code aktualisiert auf ${SHORT_SHA}"
|
||||
|
||||
# 2. .env sicherstellen (muss einmalig manuell angelegt werden)
|
||||
docker run --rm -v "${DEPLOY_DIR}:${DEPLOY_DIR}" alpine \
|
||||
sh -c "
|
||||
if [ ! -f '${DEPLOY_DIR}/.env' ]; then
|
||||
echo 'WARNUNG: ${DEPLOY_DIR}/.env fehlt!'
|
||||
echo 'Bitte einmalig auf dem Host anlegen.'
|
||||
echo 'Deploy wird fortgesetzt (Services starten ggf. mit Defaults).'
|
||||
else
|
||||
echo '.env vorhanden'
|
||||
fi
|
||||
"
|
||||
|
||||
# 3. Build + Deploy via Helper-Container mit Docker-Socket + Deploy-Dir
|
||||
# docker compose muss die YAML-Dateien lesen koennen, daher
|
||||
# alles in einem Container mit beiden Mounts ausfuehren.
|
||||
echo ""
|
||||
echo "=== Building + Deploying ==="
|
||||
docker run --rm \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-v "${DEPLOY_DIR}:${DEPLOY_DIR}" \
|
||||
-w "${DEPLOY_DIR}" \
|
||||
docker:27-cli \
|
||||
sh -c "
|
||||
COMPOSE_FILES='-f docker-compose.yml -f docker-compose.hetzner.yml'
|
||||
|
||||
echo '=== Building Docker Images ==='
|
||||
docker compose \${COMPOSE_FILES} build --parallel \
|
||||
admin-compliance \
|
||||
backend-compliance \
|
||||
ai-compliance-sdk \
|
||||
developer-portal
|
||||
|
||||
echo ''
|
||||
echo '=== Starting containers ==='
|
||||
docker compose \${COMPOSE_FILES} up -d --remove-orphans \
|
||||
admin-compliance \
|
||||
backend-compliance \
|
||||
ai-compliance-sdk \
|
||||
developer-portal
|
||||
|
||||
echo ''
|
||||
echo '=== Health Checks ==='
|
||||
sleep 10
|
||||
for svc in bp-compliance-admin bp-compliance-backend bp-compliance-ai-sdk bp-compliance-developer-portal; do
|
||||
STATUS=\$(docker inspect --format='{{.State.Status}}' \"\${svc}\" 2>/dev/null || echo 'not found')
|
||||
echo \"\${svc}: \${STATUS}\"
|
||||
done
|
||||
"
|
||||
|
||||
echo ""
|
||||
echo "=== Deploy abgeschlossen: ${SHORT_SHA} ==="
|
||||
apk add --no-cache curl
|
||||
curl -sf "${{ secrets.COOLIFY_WEBHOOK }}" \
|
||||
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
#
|
||||
# Phasen: gesetze, eu, templates, datenschutz, verbraucherschutz, verify, version, all
|
||||
#
|
||||
# Voraussetzung: RAG-Service und Qdrant muessen auf Hetzner laufen.
|
||||
# Die BreakPilot-Services muessen deployed sein (ci.yaml deploy-hetzner).
|
||||
# Voraussetzung: RAG-Service und Qdrant muessen auf Coolify laufen.
|
||||
# Die BreakPilot-Services muessen deployed sein (ci.yaml deploy-coolify).
|
||||
|
||||
name: RAG Ingestion
|
||||
|
||||
|
||||
@@ -37,8 +37,8 @@ WORKDIR /app
|
||||
ENV NODE_ENV=production
|
||||
|
||||
# Create non-root user
|
||||
RUN addgroup --system --gid 1001 nodejs
|
||||
RUN adduser --system --uid 1001 nextjs
|
||||
RUN addgroup -S -g 1001 nodejs
|
||||
RUN adduser -S -u 1001 -G nodejs nextjs
|
||||
|
||||
# Copy built assets
|
||||
COPY --from=builder /app/public ./public
|
||||
|
||||
@@ -17,7 +17,7 @@ COPY . .
|
||||
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o /ai-compliance-sdk ./cmd/server
|
||||
|
||||
# Runtime stage
|
||||
FROM alpine:3.19
|
||||
FROM alpine:3.21
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ go build -o server ./cmd/server
|
||||
|
||||
# Production: CI/CD (automatisch bei Push auf main)
|
||||
git push origin main && git push gitea main
|
||||
# → Gitea Actions: Tests → Build → Deploy auf Hetzner
|
||||
# → Gitea Actions: Tests → Build → Deploy auf Coolify
|
||||
# → Status: https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-compliance/actions
|
||||
|
||||
# Alternativ: mit Docker (lokal)
|
||||
@@ -466,7 +466,7 @@ Tests laufen automatisch bei jedem Push via Gitea Actions (`.gitea/workflows/ci.
|
||||
| `test-python-document-crawler` | `python:3.12-slim` | `pytest tests/` |
|
||||
| `test-python-dsms-gateway` | `python:3.12-slim` | `pytest test_main.py` |
|
||||
|
||||
Nach erfolgreichen Tests: automatisches Deploy auf Hetzner (`deploy-hetzner` Job).
|
||||
Nach erfolgreichen Tests: automatisches Deploy auf Coolify (`deploy-coolify` Job).
|
||||
|
||||
### Spezifische Tests
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ from typing import Any, Optional
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Header
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import text
|
||||
|
||||
from database import SessionLocal
|
||||
|
||||
@@ -75,13 +76,13 @@ async def get_compliance_scope(
|
||||
db = SessionLocal()
|
||||
try:
|
||||
row = db.execute(
|
||||
"""SELECT tenant_id,
|
||||
text("""SELECT tenant_id,
|
||||
state->'compliance_scope' AS scope,
|
||||
created_at,
|
||||
updated_at
|
||||
FROM sdk_states
|
||||
WHERE tenant_id = :tid
|
||||
AND state ? 'compliance_scope'""",
|
||||
AND state ? 'compliance_scope'"""),
|
||||
{"tid": tid},
|
||||
).fetchone()
|
||||
|
||||
@@ -106,22 +107,22 @@ async def upsert_compliance_scope(
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.execute(
|
||||
"""INSERT INTO sdk_states (tenant_id, state)
|
||||
text("""INSERT INTO sdk_states (tenant_id, state)
|
||||
VALUES (:tid, jsonb_build_object('compliance_scope', :scope::jsonb))
|
||||
ON CONFLICT (tenant_id) DO UPDATE
|
||||
SET state = sdk_states.state || jsonb_build_object('compliance_scope', :scope::jsonb),
|
||||
updated_at = NOW()""",
|
||||
updated_at = NOW()"""),
|
||||
{"tid": tid, "scope": scope_json},
|
||||
)
|
||||
db.commit()
|
||||
|
||||
row = db.execute(
|
||||
"""SELECT tenant_id,
|
||||
text("""SELECT tenant_id,
|
||||
state->'compliance_scope' AS scope,
|
||||
created_at,
|
||||
updated_at
|
||||
FROM sdk_states
|
||||
WHERE tenant_id = :tid""",
|
||||
WHERE tenant_id = :tid"""),
|
||||
{"tid": tid},
|
||||
).fetchone()
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ from typing import Optional
|
||||
import httpx
|
||||
from fastapi import APIRouter, File, Form, Header, UploadFile, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import text
|
||||
|
||||
from database import SessionLocal
|
||||
|
||||
@@ -291,11 +292,11 @@ async def analyze_document(
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.execute(
|
||||
"""INSERT INTO compliance_imported_documents
|
||||
text("""INSERT INTO compliance_imported_documents
|
||||
(id, tenant_id, filename, file_type, file_size, detected_type, detection_confidence,
|
||||
extracted_text, extracted_entities, recommendations, status, analyzed_at)
|
||||
VALUES (:id, :tenant_id, :filename, :file_type, :file_size, :detected_type, :confidence,
|
||||
:text, :entities::jsonb, :recommendations::jsonb, 'analyzed', NOW())""",
|
||||
:text, :entities::jsonb, :recommendations::jsonb, 'analyzed', NOW())"""),
|
||||
{
|
||||
"id": doc_id,
|
||||
"tenant_id": tenant_id,
|
||||
@@ -313,9 +314,9 @@ async def analyze_document(
|
||||
if total_gaps > 0:
|
||||
import json
|
||||
db.execute(
|
||||
"""INSERT INTO compliance_gap_analyses
|
||||
text("""INSERT INTO compliance_gap_analyses
|
||||
(tenant_id, document_id, total_gaps, critical_gaps, high_gaps, medium_gaps, low_gaps, gaps, recommended_packages)
|
||||
VALUES (:tenant_id, :document_id, :total, :critical, :high, :medium, :low, :gaps::jsonb, :packages::jsonb)""",
|
||||
VALUES (:tenant_id, :document_id, :total, :critical, :high, :medium, :low, :gaps::jsonb, :packages::jsonb)"""),
|
||||
{
|
||||
"tenant_id": tenant_id,
|
||||
"document_id": doc_id,
|
||||
@@ -358,7 +359,7 @@ async def get_gap_analysis(
|
||||
db = SessionLocal()
|
||||
try:
|
||||
result = db.execute(
|
||||
"SELECT * FROM compliance_gap_analyses WHERE document_id = :doc_id AND tenant_id = :tid",
|
||||
text("SELECT * FROM compliance_gap_analyses WHERE document_id = :doc_id AND tenant_id = :tid"),
|
||||
{"doc_id": document_id, "tid": tid},
|
||||
).fetchone()
|
||||
if not result:
|
||||
@@ -374,11 +375,11 @@ async def list_documents(tenant_id: str = "default"):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
result = db.execute(
|
||||
"""SELECT id, filename, file_type, file_size, detected_type, detection_confidence,
|
||||
text("""SELECT id, filename, file_type, file_size, detected_type, detection_confidence,
|
||||
extracted_entities, recommendations, status, analyzed_at, created_at
|
||||
FROM compliance_imported_documents
|
||||
WHERE tenant_id = :tenant_id
|
||||
ORDER BY created_at DESC""",
|
||||
ORDER BY created_at DESC"""),
|
||||
{"tenant_id": tenant_id},
|
||||
)
|
||||
rows = result.fetchall()
|
||||
@@ -424,11 +425,11 @@ async def delete_document(
|
||||
try:
|
||||
# Delete gap analysis first (FK dependency)
|
||||
db.execute(
|
||||
"DELETE FROM compliance_gap_analyses WHERE document_id = :doc_id AND tenant_id = :tid",
|
||||
text("DELETE FROM compliance_gap_analyses WHERE document_id = :doc_id AND tenant_id = :tid"),
|
||||
{"doc_id": document_id, "tid": tid},
|
||||
)
|
||||
result = db.execute(
|
||||
"DELETE FROM compliance_imported_documents WHERE id = :doc_id AND tenant_id = :tid",
|
||||
text("DELETE FROM compliance_imported_documents WHERE id = :doc_id AND tenant_id = :tid"),
|
||||
{"doc_id": document_id, "tid": tid},
|
||||
)
|
||||
db.commit()
|
||||
|
||||
@@ -17,6 +17,7 @@ from typing import Optional
|
||||
import httpx
|
||||
from fastapi import APIRouter, File, Form, UploadFile, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy import text
|
||||
|
||||
from database import SessionLocal
|
||||
|
||||
@@ -366,13 +367,13 @@ async def scan_dependencies(
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db.execute(
|
||||
"""INSERT INTO compliance_screenings
|
||||
text("""INSERT INTO compliance_screenings
|
||||
(id, tenant_id, status, sbom_format, sbom_version,
|
||||
total_components, total_issues, critical_issues, high_issues, medium_issues, low_issues,
|
||||
sbom_data, started_at, completed_at)
|
||||
VALUES (:id, :tenant_id, 'completed', 'CycloneDX', '1.5',
|
||||
:total_components, :total_issues, :critical, :high, :medium, :low,
|
||||
:sbom_data::jsonb, :started_at, :completed_at)""",
|
||||
:sbom_data::jsonb, :started_at, :completed_at)"""),
|
||||
{
|
||||
"id": screening_id,
|
||||
"tenant_id": tenant_id,
|
||||
@@ -391,11 +392,11 @@ async def scan_dependencies(
|
||||
# Persist security issues
|
||||
for issue in issues:
|
||||
db.execute(
|
||||
"""INSERT INTO compliance_security_issues
|
||||
text("""INSERT INTO compliance_security_issues
|
||||
(id, screening_id, severity, title, description, cve, cvss,
|
||||
affected_component, affected_version, fixed_in, remediation, status)
|
||||
VALUES (:id, :screening_id, :severity, :title, :description, :cve, :cvss,
|
||||
:component, :version, :fixed_in, :remediation, :status)""",
|
||||
:component, :version, :fixed_in, :remediation, :status)"""),
|
||||
{
|
||||
"id": issue["id"],
|
||||
"screening_id": screening_id,
|
||||
@@ -486,10 +487,10 @@ async def get_screening(screening_id: str):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
result = db.execute(
|
||||
"""SELECT id, status, sbom_format, sbom_version,
|
||||
text("""SELECT id, status, sbom_format, sbom_version,
|
||||
total_components, total_issues, critical_issues, high_issues,
|
||||
medium_issues, low_issues, sbom_data, started_at, completed_at
|
||||
FROM compliance_screenings WHERE id = :id""",
|
||||
FROM compliance_screenings WHERE id = :id"""),
|
||||
{"id": screening_id},
|
||||
)
|
||||
row = result.fetchone()
|
||||
@@ -498,9 +499,9 @@ async def get_screening(screening_id: str):
|
||||
|
||||
# Fetch issues
|
||||
issues_result = db.execute(
|
||||
"""SELECT id, severity, title, description, cve, cvss,
|
||||
text("""SELECT id, severity, title, description, cve, cvss,
|
||||
affected_component, affected_version, fixed_in, remediation, status
|
||||
FROM compliance_security_issues WHERE screening_id = :id""",
|
||||
FROM compliance_security_issues WHERE screening_id = :id"""),
|
||||
{"id": screening_id},
|
||||
)
|
||||
issues_rows = issues_result.fetchall()
|
||||
@@ -566,12 +567,12 @@ async def list_screenings(tenant_id: str = "default"):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
result = db.execute(
|
||||
"""SELECT id, status, total_components, total_issues,
|
||||
text("""SELECT id, status, total_components, total_issues,
|
||||
critical_issues, high_issues, medium_issues, low_issues,
|
||||
started_at, completed_at, created_at
|
||||
FROM compliance_screenings
|
||||
WHERE tenant_id = :tenant_id
|
||||
ORDER BY created_at DESC""",
|
||||
ORDER BY created_at DESC"""),
|
||||
{"tenant_id": tenant_id},
|
||||
)
|
||||
rows = result.fetchall()
|
||||
|
||||
@@ -12,7 +12,7 @@ RUN npm install
|
||||
# Copy source code
|
||||
COPY . .
|
||||
|
||||
# Ensure public directory exists
|
||||
# Ensure public directory exists (may not have static assets)
|
||||
RUN mkdir -p public
|
||||
|
||||
# Build the application
|
||||
@@ -27,8 +27,8 @@ WORKDIR /app
|
||||
ENV NODE_ENV=production
|
||||
|
||||
# Create non-root user
|
||||
RUN addgroup --system --gid 1001 nodejs
|
||||
RUN adduser --system --uid 1001 nextjs
|
||||
RUN addgroup -S -g 1001 nodejs
|
||||
RUN adduser -S -u 1001 -G nodejs nextjs
|
||||
|
||||
# Copy built assets
|
||||
COPY --from=builder /app/public ./public
|
||||
|
||||
272
docker-compose.coolify.yml
Normal file
272
docker-compose.coolify.yml
Normal file
@@ -0,0 +1,272 @@
|
||||
# =========================================================
|
||||
# BreakPilot Compliance — Compliance SDK Platform (Coolify)
|
||||
# =========================================================
|
||||
# Requires: breakpilot-core must be running
|
||||
# Deployed via Coolify. SSL termination handled by Traefik.
|
||||
# External services (managed separately in Coolify):
|
||||
# - PostgreSQL, Qdrant, S3-compatible storage
|
||||
# =========================================================
|
||||
|
||||
networks:
|
||||
breakpilot-network:
|
||||
external: true
|
||||
name: breakpilot-network
|
||||
coolify:
|
||||
external: true
|
||||
name: coolify
|
||||
|
||||
volumes:
|
||||
dsms_data:
|
||||
|
||||
services:
|
||||
|
||||
# =========================================================
|
||||
# FRONTEND
|
||||
# =========================================================
|
||||
admin-compliance:
|
||||
build:
|
||||
context: ./admin-compliance
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
NEXT_PUBLIC_API_URL: ${NEXT_PUBLIC_API_URL:-https://api-compliance.breakpilot.ai}
|
||||
NEXT_PUBLIC_SDK_URL: ${NEXT_PUBLIC_SDK_URL:-https://sdk.breakpilot.ai}
|
||||
container_name: bp-compliance-admin
|
||||
labels:
|
||||
- "traefik.docker.network=coolify"
|
||||
expose:
|
||||
- "3000"
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
DATABASE_URL: ${COMPLIANCE_DATABASE_URL}
|
||||
BACKEND_URL: http://backend-compliance:8002
|
||||
CONSENT_SERVICE_URL: http://bp-core-consent-service:8081
|
||||
SDK_URL: http://ai-compliance-sdk:8090
|
||||
OLLAMA_URL: ${OLLAMA_URL:-}
|
||||
COMPLIANCE_LLM_MODEL: ${COMPLIANCE_LLM_MODEL:-}
|
||||
depends_on:
|
||||
backend-compliance:
|
||||
condition: service_started
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:3000/"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 30s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
- coolify
|
||||
|
||||
developer-portal:
|
||||
build:
|
||||
context: ./developer-portal
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-developer-portal
|
||||
labels:
|
||||
- "traefik.docker.network=coolify"
|
||||
expose:
|
||||
- "3000"
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:3000/"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 30s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
- coolify
|
||||
|
||||
# =========================================================
|
||||
# BACKEND
|
||||
# =========================================================
|
||||
backend-compliance:
|
||||
build:
|
||||
context: ./backend-compliance
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-backend
|
||||
labels:
|
||||
- "traefik.docker.network=coolify"
|
||||
expose:
|
||||
- "8002"
|
||||
environment:
|
||||
PORT: 8002
|
||||
DATABASE_URL: ${COMPLIANCE_DATABASE_URL}
|
||||
JWT_SECRET: ${JWT_SECRET}
|
||||
ENVIRONMENT: production
|
||||
CONSENT_SERVICE_URL: http://bp-core-consent-service:8081
|
||||
VALKEY_URL: redis://bp-core-valkey:6379/0
|
||||
SESSION_TTL_HOURS: ${SESSION_TTL_HOURS:-24}
|
||||
COMPLIANCE_LLM_PROVIDER: ${COMPLIANCE_LLM_PROVIDER:-anthropic}
|
||||
SELF_HOSTED_LLM_URL: ${SELF_HOSTED_LLM_URL:-}
|
||||
SELF_HOSTED_LLM_MODEL: ${SELF_HOSTED_LLM_MODEL:-}
|
||||
COMPLIANCE_LLM_MAX_TOKENS: ${COMPLIANCE_LLM_MAX_TOKENS:-4096}
|
||||
COMPLIANCE_LLM_TEMPERATURE: ${COMPLIANCE_LLM_TEMPERATURE:-0.3}
|
||||
COMPLIANCE_LLM_TIMEOUT: ${COMPLIANCE_LLM_TIMEOUT:-120}
|
||||
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
|
||||
SMTP_HOST: ${SMTP_HOST}
|
||||
SMTP_PORT: ${SMTP_PORT:-587}
|
||||
SMTP_USERNAME: ${SMTP_USERNAME}
|
||||
SMTP_PASSWORD: ${SMTP_PASSWORD}
|
||||
SMTP_FROM_NAME: ${SMTP_FROM_NAME:-BreakPilot Compliance}
|
||||
SMTP_FROM_ADDR: ${SMTP_FROM_ADDR:-compliance@breakpilot.ai}
|
||||
RAG_SERVICE_URL: http://bp-core-rag-service:8097
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://127.0.0.1:8002/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 15s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
- coolify
|
||||
|
||||
# =========================================================
|
||||
# SDK SERVICES
|
||||
# =========================================================
|
||||
ai-compliance-sdk:
|
||||
build:
|
||||
context: ./ai-compliance-sdk
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-ai-sdk
|
||||
labels:
|
||||
- "traefik.docker.network=coolify"
|
||||
expose:
|
||||
- "8090"
|
||||
environment:
|
||||
PORT: 8090
|
||||
ENVIRONMENT: production
|
||||
DATABASE_URL: ${COMPLIANCE_DATABASE_URL}
|
||||
JWT_SECRET: ${JWT_SECRET}
|
||||
LLM_PROVIDER: ${COMPLIANCE_LLM_PROVIDER:-anthropic}
|
||||
LLM_FALLBACK_PROVIDER: ${LLM_FALLBACK_PROVIDER:-}
|
||||
OLLAMA_URL: ${OLLAMA_URL:-}
|
||||
OLLAMA_DEFAULT_MODEL: ${OLLAMA_DEFAULT_MODEL:-}
|
||||
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
|
||||
ANTHROPIC_DEFAULT_MODEL: ${ANTHROPIC_DEFAULT_MODEL:-claude-sonnet-4-5-20250929}
|
||||
PII_REDACTION_ENABLED: ${PII_REDACTION_ENABLED:-true}
|
||||
PII_REDACTION_LEVEL: ${PII_REDACTION_LEVEL:-standard}
|
||||
AUDIT_RETENTION_DAYS: ${AUDIT_RETENTION_DAYS:-365}
|
||||
AUDIT_LOG_PROMPTS: ${AUDIT_LOG_PROMPTS:-true}
|
||||
ALLOWED_ORIGINS: "*"
|
||||
TTS_SERVICE_URL: http://compliance-tts-service:8095
|
||||
QDRANT_URL: ${QDRANT_URL}
|
||||
QDRANT_API_KEY: ${QDRANT_API_KEY:-}
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:8090/health"]
|
||||
interval: 30s
|
||||
timeout: 3s
|
||||
start_period: 10s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
- coolify
|
||||
|
||||
# =========================================================
|
||||
# TTS SERVICE (Piper TTS + FFmpeg)
|
||||
# =========================================================
|
||||
compliance-tts-service:
|
||||
build:
|
||||
context: ./compliance-tts-service
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-tts
|
||||
expose:
|
||||
- "8095"
|
||||
environment:
|
||||
MINIO_ENDPOINT: ${S3_ENDPOINT}
|
||||
MINIO_ACCESS_KEY: ${S3_ACCESS_KEY}
|
||||
MINIO_SECRET_KEY: ${S3_SECRET_KEY}
|
||||
MINIO_SECURE: ${S3_SECURE:-true}
|
||||
PIPER_MODEL_PATH: /app/models/de_DE-thorsten-high.onnx
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8095/health')"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 60s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
|
||||
# =========================================================
|
||||
# DATA SOVEREIGNTY
|
||||
# =========================================================
|
||||
dsms-node:
|
||||
build:
|
||||
context: ./dsms-node
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-dsms-node
|
||||
expose:
|
||||
- "4001"
|
||||
- "5001"
|
||||
- "8080"
|
||||
volumes:
|
||||
- dsms_data:/data/ipfs
|
||||
environment:
|
||||
IPFS_PROFILE: server
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "ipfs id"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 30s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
|
||||
dsms-gateway:
|
||||
build:
|
||||
context: ./dsms-gateway
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-dsms-gateway
|
||||
expose:
|
||||
- "8082"
|
||||
environment:
|
||||
IPFS_API_URL: http://dsms-node:5001
|
||||
IPFS_GATEWAY_URL: http://dsms-node:8080
|
||||
JWT_SECRET: ${JWT_SECRET}
|
||||
depends_on:
|
||||
dsms-node:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://127.0.0.1:8082/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 15s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
|
||||
# =========================================================
|
||||
# DOCUMENT CRAWLER & AUTO-ONBOARDING
|
||||
# =========================================================
|
||||
document-crawler:
|
||||
build:
|
||||
context: ./document-crawler
|
||||
dockerfile: Dockerfile
|
||||
container_name: bp-compliance-document-crawler
|
||||
expose:
|
||||
- "8098"
|
||||
environment:
|
||||
PORT: 8098
|
||||
DATABASE_URL: ${COMPLIANCE_DATABASE_URL}
|
||||
LLM_GATEWAY_URL: http://ai-compliance-sdk:8090
|
||||
DSMS_GATEWAY_URL: http://dsms-gateway:8082
|
||||
CRAWL_BASE_PATH: /data/crawl
|
||||
MAX_FILE_SIZE_MB: 50
|
||||
volumes:
|
||||
- /tmp/breakpilot-crawl-data:/data/crawl:ro
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://127.0.0.1:8098/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 15s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- breakpilot-network
|
||||
@@ -1,16 +1,16 @@
|
||||
# =========================================================
|
||||
# BreakPilot Compliance — Hetzner Override
|
||||
# BreakPilot Compliance — Coolify Production Override
|
||||
# =========================================================
|
||||
# Verwendung: docker compose -f docker-compose.yml -f docker-compose.hetzner.yml up -d
|
||||
#
|
||||
# Aenderungen gegenueber docker-compose.yml:
|
||||
# - Platform: arm64 → amd64 (Hetzner = x86_64)
|
||||
# - Network: external → auto-create (kein breakpilot-core auf Hetzner)
|
||||
# - depends_on: core-health-check entfernt (kein Core auf Hetzner)
|
||||
# - API URLs: auf Hetzner-interne Adressen angepasst
|
||||
# - Platform: arm64 → amd64 (Coolify = x86_64)
|
||||
# - Network: external → auto-create (kein breakpilot-core auf Coolify)
|
||||
# - depends_on: core-health-check entfernt (kein Core auf Coolify)
|
||||
# - API URLs: auf Coolify-interne Adressen angepasst
|
||||
# =========================================================
|
||||
|
||||
# Auf Hetzner laeuft kein breakpilot-core, daher Network selbst erstellen
|
||||
# Auf Coolify laeuft kein breakpilot-core, daher Network selbst erstellen
|
||||
networks:
|
||||
breakpilot-network:
|
||||
external: false
|
||||
@@ -18,9 +18,9 @@ networks:
|
||||
|
||||
services:
|
||||
|
||||
# Core-Health-Check deaktivieren (Core laeuft nicht auf Hetzner)
|
||||
# Core-Health-Check deaktivieren (Core laeuft nicht auf Coolify)
|
||||
core-health-check:
|
||||
entrypoint: ["sh", "-c", "echo 'Core health check skipped on Hetzner' && exit 0"]
|
||||
entrypoint: ["sh", "-c", "echo 'Core health check skipped on Coolify' && exit 0"]
|
||||
restart: "no"
|
||||
|
||||
admin-compliance:
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
# CI/CD Pipeline
|
||||
|
||||
Übersicht über den Deployment-Prozess für Breakpilot.
|
||||
Uebersicht ueber den Deployment-Prozess fuer BreakPilot Compliance.
|
||||
|
||||
## Übersicht
|
||||
## Uebersicht
|
||||
|
||||
| Komponente | Build-Tool | Deployment |
|
||||
|------------|------------|------------|
|
||||
| Frontend (Next.js) | Docker | Mac Mini |
|
||||
| Backend (FastAPI) | Docker | Mac Mini |
|
||||
| Go Services | Docker (Multi-stage) | Mac Mini |
|
||||
| Documentation | MkDocs | Docker (Nginx) |
|
||||
| Frontend (Next.js) | Docker | Coolify (automatisch) |
|
||||
| Backend (FastAPI) | Docker | Coolify (automatisch) |
|
||||
| Go Services | Docker (Multi-stage) | Coolify (automatisch) |
|
||||
| Documentation | MkDocs | Docker (Nginx, lokal) |
|
||||
|
||||
## Deployment-Architektur
|
||||
|
||||
@@ -17,386 +17,129 @@
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Entwickler-MacBook │
|
||||
│ │
|
||||
│ breakpilot-pwa/ │
|
||||
│ ├── studio-v2/ (Next.js Frontend) │
|
||||
│ ├── admin-v2/ (Next.js Admin) │
|
||||
│ ├── backend/ (Python FastAPI) │
|
||||
│ ├── consent-service/ (Go Service) │
|
||||
│ ├── klausur-service/ (Python FastAPI) │
|
||||
│ ├── voice-service/ (Python FastAPI) │
|
||||
│ ├── ai-compliance-sdk/ (Go Service) │
|
||||
│ └── docs-src/ (MkDocs) │
|
||||
│ breakpilot-compliance/ │
|
||||
│ ├── admin-compliance/ (Next.js Dashboard) │
|
||||
│ ├── backend-compliance/ (Python FastAPI) │
|
||||
│ ├── ai-compliance-sdk/ (Go/Gin) │
|
||||
│ ├── developer-portal/ (Next.js) │
|
||||
│ └── docs-src/ (MkDocs) │
|
||||
│ │
|
||||
│ $ ./sync-and-deploy.sh │
|
||||
│ git push origin main && git push gitea main │
|
||||
└───────────────────────────────┬─────────────────────────────────┘
|
||||
│
|
||||
│ rsync + SSH
|
||||
│ git push
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Mac Mini Server │
|
||||
│ Gitea (gitea.meghsakha.com) │
|
||||
│ │
|
||||
│ Docker Compose │
|
||||
│ ├── website (Port 3000) │
|
||||
│ ├── studio-v2 (Port 3001) │
|
||||
│ ├── admin-v2 (Port 3002) │
|
||||
│ ├── backend (Port 8000) │
|
||||
│ ├── consent-service (Port 8081) │
|
||||
│ ├── klausur-service (Port 8086) │
|
||||
│ ├── voice-service (Port 8082) │
|
||||
│ ├── ai-compliance-sdk (Port 8090) │
|
||||
│ ├── docs (Port 8009) │
|
||||
│ ├── postgres │
|
||||
│ ├── valkey (Redis) │
|
||||
│ ├── qdrant (extern: qdrant-dev.breakpilot.ai) │
|
||||
│ └── object-storage (extern: nbg1.your-objectstorage.com) │
|
||||
│ Gitea Actions CI: │
|
||||
│ ├── test-go-ai-compliance │
|
||||
│ ├── test-python-backend-compliance │
|
||||
│ ├── test-python-document-crawler │
|
||||
│ ├── test-python-dsms-gateway │
|
||||
│ └── validate-canonical-controls │
|
||||
│ │
|
||||
│ Coolify Webhook → Build + Deploy (automatisch) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ auto-deploy
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Production (Coolify) │
|
||||
│ │
|
||||
│ ├── admin-dev.breakpilot.ai (Admin Compliance) │
|
||||
│ ├── api-dev.breakpilot.ai (Backend API) │
|
||||
│ ├── sdk-dev.breakpilot.ai (AI SDK) │
|
||||
│ └── developers-dev.breakpilot.ai (Developer Portal) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Sync & Deploy Workflow
|
||||
## Workflow
|
||||
|
||||
### 1. Dateien synchronisieren
|
||||
### 1. Code entwickeln und committen
|
||||
|
||||
```bash
|
||||
# Sync aller relevanten Verzeichnisse zum Mac Mini
|
||||
rsync -avz --delete \
|
||||
--exclude 'node_modules' \
|
||||
--exclude '.next' \
|
||||
--exclude '.git' \
|
||||
--exclude '__pycache__' \
|
||||
--exclude 'venv' \
|
||||
--exclude '.pytest_cache' \
|
||||
/Users/benjaminadmin/Projekte/breakpilot-pwa/ \
|
||||
macmini:/Users/benjaminadmin/Projekte/breakpilot-pwa/
|
||||
# Code auf MacBook bearbeiten
|
||||
# Committen und zu beiden Remotes pushen:
|
||||
git push origin main && git push gitea main
|
||||
```
|
||||
|
||||
### 2. Container bauen
|
||||
### 2. Automatische Tests (Gitea Actions)
|
||||
|
||||
Push auf gitea triggert automatisch die CI-Pipeline:
|
||||
|
||||
- **Go Tests:** `ai-compliance-sdk` Unit Tests
|
||||
- **Python Tests:** `backend-compliance`, `document-crawler`, `dsms-gateway`
|
||||
- **Validierung:** Canonical Controls JSON-Validierung
|
||||
- **Lint:** Go, Python, Node.js (nur bei PRs)
|
||||
|
||||
### 3. Automatisches Deployment (Coolify)
|
||||
|
||||
Nach erfolgreichem Push baut Coolify automatisch alle Services und deployt sie.
|
||||
|
||||
**WICHTIG:** Niemals manuell in Coolify auf "Redeploy" klicken!
|
||||
|
||||
### 4. Health Checks
|
||||
|
||||
```bash
|
||||
# Einzelnen Service bauen
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache <service-name>"
|
||||
|
||||
# Beispiele:
|
||||
# studio-v2, admin-v2, website, backend, klausur-service, docs
|
||||
# Production Health pruefen
|
||||
curl -sf https://api-dev.breakpilot.ai/health
|
||||
curl -sf https://sdk-dev.breakpilot.ai/health
|
||||
```
|
||||
|
||||
### 3. Container deployen
|
||||
## CI Pipeline-Konfiguration
|
||||
|
||||
```bash
|
||||
# Container neu starten
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d <service-name>"
|
||||
**Datei:** `.gitea/workflows/ci.yaml`
|
||||
|
||||
```yaml
|
||||
on:
|
||||
push:
|
||||
branches: [main, develop]
|
||||
pull_request:
|
||||
branches: [main, develop]
|
||||
|
||||
jobs:
|
||||
test-go-ai-compliance: # Go Unit Tests
|
||||
test-python-backend: # Python Unit Tests
|
||||
test-python-document-crawler:
|
||||
test-python-dsms-gateway:
|
||||
validate-canonical-controls: # JSON Validierung
|
||||
go-lint: # Nur bei PRs
|
||||
python-lint: # Nur bei PRs
|
||||
nodejs-lint: # Nur bei PRs
|
||||
```
|
||||
|
||||
### 4. Logs prüfen
|
||||
## Lokale Entwicklung (Mac Mini)
|
||||
|
||||
Fuer lokale Tests ohne Coolify:
|
||||
|
||||
```bash
|
||||
# Container-Logs anzeigen
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
logs -f <service-name>"
|
||||
```
|
||||
|
||||
## Service-spezifische Deployments
|
||||
|
||||
### Next.js Frontend (studio-v2, admin-v2, website)
|
||||
|
||||
```bash
|
||||
# 1. Sync
|
||||
rsync -avz --delete \
|
||||
--exclude 'node_modules' --exclude '.next' --exclude '.git' \
|
||||
/Users/benjaminadmin/Projekte/breakpilot-pwa/studio-v2/ \
|
||||
macmini:/Users/benjaminadmin/Projekte/breakpilot-pwa/studio-v2/
|
||||
|
||||
# 2. Build & Deploy
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache studio-v2 && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d studio-v2"
|
||||
```
|
||||
|
||||
### Python Services (backend, klausur-service, voice-service)
|
||||
|
||||
```bash
|
||||
# Build mit requirements.txt
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build klausur-service && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d klausur-service"
|
||||
```
|
||||
|
||||
### Go Services (consent-service, ai-compliance-sdk)
|
||||
|
||||
```bash
|
||||
# Multi-stage Build (Go → Alpine)
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache consent-service && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d consent-service"
|
||||
```
|
||||
|
||||
### MkDocs Dokumentation
|
||||
|
||||
```bash
|
||||
# Build & Deploy
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache docs && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d docs"
|
||||
|
||||
# Verfügbar unter: http://macmini:8009
|
||||
```
|
||||
|
||||
## Health Checks
|
||||
|
||||
### Service-Status prüfen
|
||||
|
||||
```bash
|
||||
# Alle Container-Status
|
||||
ssh macmini "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
|
||||
|
||||
# Health-Endpoints prüfen
|
||||
curl -s http://macmini:8000/health
|
||||
curl -s http://macmini:8081/health
|
||||
curl -s http://macmini:8086/health
|
||||
curl -s http://macmini:8090/health
|
||||
```
|
||||
|
||||
### Logs analysieren
|
||||
|
||||
```bash
|
||||
# Letzte 100 Zeilen
|
||||
ssh macmini "docker logs --tail 100 breakpilot-pwa-backend-1"
|
||||
|
||||
# Live-Logs folgen
|
||||
ssh macmini "docker logs -f breakpilot-pwa-backend-1"
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
### Container auf vorherige Version zurücksetzen
|
||||
|
||||
```bash
|
||||
# 1. Aktuelles Image taggen
|
||||
ssh macmini "docker tag breakpilot-pwa-backend:latest breakpilot-pwa-backend:backup"
|
||||
|
||||
# 2. Altes Image deployen
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d backend"
|
||||
|
||||
# 3. Bei Problemen: Backup wiederherstellen
|
||||
ssh macmini "docker tag breakpilot-pwa-backend:backup breakpilot-pwa-backend:latest"
|
||||
# Auf Mac Mini pullen und bauen
|
||||
ssh macmini "git -C ~/Projekte/breakpilot-compliance pull --no-rebase origin main"
|
||||
ssh macmini "/usr/local/bin/docker compose -f ~/Projekte/breakpilot-compliance/docker-compose.yml build --no-cache <service>"
|
||||
ssh macmini "/usr/local/bin/docker compose -f ~/Projekte/breakpilot-compliance/docker-compose.yml up -d <service>"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container startet nicht
|
||||
### CI-Status pruefen
|
||||
|
||||
```bash
|
||||
# 1. Logs prüfen
|
||||
ssh macmini "docker logs breakpilot-pwa-<service>-1"
|
||||
|
||||
# 2. Container manuell starten für Debug-Output
|
||||
ssh macmini "docker compose -f .../docker-compose.yml run --rm <service>"
|
||||
|
||||
# 3. In Container einloggen
|
||||
ssh macmini "docker exec -it breakpilot-pwa-<service>-1 /bin/sh"
|
||||
# Im Browser:
|
||||
# https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-compliance/actions
|
||||
```
|
||||
|
||||
### Port bereits belegt
|
||||
### Container-Logs (lokal)
|
||||
|
||||
```bash
|
||||
# Port-Belegung prüfen
|
||||
ssh macmini "lsof -i :8000"
|
||||
|
||||
# Container mit dem Port finden
|
||||
ssh macmini "docker ps --filter publish=8000"
|
||||
ssh macmini "/usr/local/bin/docker logs -f bp-compliance-<service>"
|
||||
```
|
||||
|
||||
### Build-Fehler
|
||||
|
||||
```bash
|
||||
# Cache komplett leeren
|
||||
ssh macmini "docker builder prune -a"
|
||||
|
||||
# Ohne Cache bauen
|
||||
ssh macmini "docker compose build --no-cache <service>"
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Resource-Nutzung
|
||||
|
||||
```bash
|
||||
# CPU/Memory aller Container
|
||||
ssh macmini "docker stats --no-stream"
|
||||
|
||||
# Disk-Nutzung
|
||||
ssh macmini "docker system df"
|
||||
```
|
||||
|
||||
### Cleanup
|
||||
|
||||
```bash
|
||||
# Ungenutzte Images/Container entfernen
|
||||
ssh macmini "docker system prune -a --volumes"
|
||||
|
||||
# Nur dangling Images
|
||||
ssh macmini "docker image prune"
|
||||
```
|
||||
|
||||
## Umgebungsvariablen
|
||||
|
||||
Umgebungsvariablen werden über `.env` Dateien und docker-compose.yml verwaltet:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
backend:
|
||||
environment:
|
||||
- DATABASE_URL=postgresql://...
|
||||
- REDIS_URL=redis://valkey:6379
|
||||
- SECRET_KEY=${SECRET_KEY}
|
||||
```
|
||||
|
||||
**Wichtig**: Sensible Werte niemals in Git committen. Stattdessen:
|
||||
- `.env` Datei auf dem Server pflegen
|
||||
- Secrets über HashiCorp Vault (siehe unten)
|
||||
|
||||
## Woodpecker CI - Automatisierte OAuth Integration
|
||||
|
||||
### Überblick
|
||||
|
||||
Die OAuth-Integration zwischen Woodpecker CI und Gitea ist **vollständig automatisiert**. Credentials werden in HashiCorp Vault gespeichert und bei Bedarf automatisch regeneriert.
|
||||
|
||||
!!! info "Warum automatisiert?"
|
||||
Diese Automatisierung ist eine DevSecOps Best Practice:
|
||||
|
||||
- **Infrastructure-as-Code**: Alles ist reproduzierbar
|
||||
- **Disaster Recovery**: Verlorene Credentials können automatisch regeneriert werden
|
||||
- **Security**: Secrets werden zentral in Vault verwaltet
|
||||
- **Onboarding**: Neue Entwickler müssen nichts manuell konfigurieren
|
||||
|
||||
### Architektur
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Mac Mini Server │
|
||||
│ │
|
||||
│ ┌───────────────┐ OAuth 2.0 ┌───────────────┐ │
|
||||
│ │ Gitea │ ←─────────────────────────→│ Woodpecker │ │
|
||||
│ │ (Port 3003) │ Client ID + Secret │ (Port 8090) │ │
|
||||
│ └───────────────┘ └───────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ OAuth App │ Env Vars│
|
||||
│ │ (DB: oauth2_application) │ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌───────────────────────────────────────────────────────────┐ │
|
||||
│ │ HashiCorp Vault (Port 8200) │ │
|
||||
│ │ │ │
|
||||
│ │ secret/cicd/woodpecker: │ │
|
||||
│ │ - gitea_client_id │ │
|
||||
│ │ - gitea_client_secret │ │
|
||||
│ │ │ │
|
||||
│ │ secret/cicd/api-tokens: │ │
|
||||
│ │ - gitea_token (für API-Zugriff) │ │
|
||||
│ │ - woodpecker_token (für Pipeline-Trigger) │ │
|
||||
│ └───────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Credentials-Speicherorte
|
||||
|
||||
| Ort | Pfad | Inhalt |
|
||||
|-----|------|--------|
|
||||
| **HashiCorp Vault** | `secret/cicd/woodpecker` | Client ID + Secret (Quelle der Wahrheit) |
|
||||
| **.env Datei** | `WOODPECKER_GITEA_CLIENT/SECRET` | Für Docker Compose (aus Vault geladen) |
|
||||
| **Gitea PostgreSQL** | `oauth2_application` Tabelle | OAuth App Registration (gehashtes Secret) |
|
||||
|
||||
### Troubleshooting: OAuth Fehler
|
||||
|
||||
Falls der Fehler "Client ID not registered" oder "user does not exist [uid: 0]" auftritt:
|
||||
|
||||
```bash
|
||||
# Option 1: Automatisches Regenerieren (empfohlen)
|
||||
./scripts/sync-woodpecker-credentials.sh --regenerate
|
||||
|
||||
# Option 2: Manuelles Vorgehen
|
||||
# 1. Credentials aus Vault laden
|
||||
vault kv get secret/cicd/woodpecker
|
||||
|
||||
# 2. .env aktualisieren
|
||||
WOODPECKER_GITEA_CLIENT=<client_id>
|
||||
WOODPECKER_GITEA_SECRET=<client_secret>
|
||||
|
||||
# 3. Zu Mac Mini synchronisieren
|
||||
rsync .env macmini:~/Projekte/breakpilot-pwa/
|
||||
|
||||
# 4. Woodpecker neu starten
|
||||
ssh macmini "cd ~/Projekte/breakpilot-pwa && \
|
||||
docker compose up -d --force-recreate woodpecker-server"
|
||||
```
|
||||
|
||||
### Das Sync-Script
|
||||
|
||||
Das Script `scripts/sync-woodpecker-credentials.sh` automatisiert den gesamten Prozess:
|
||||
|
||||
```bash
|
||||
# Credentials aus Vault laden und .env aktualisieren
|
||||
./scripts/sync-woodpecker-credentials.sh
|
||||
|
||||
# Neue Credentials generieren (OAuth App in Gitea + Vault + .env)
|
||||
./scripts/sync-woodpecker-credentials.sh --regenerate
|
||||
```
|
||||
|
||||
Was das Script macht:
|
||||
|
||||
1. **Liest** die aktuellen Credentials aus Vault
|
||||
2. **Aktualisiert** die .env Datei automatisch
|
||||
3. **Bei `--regenerate`**:
|
||||
- Löscht alte OAuth Apps in Gitea
|
||||
- Erstellt neue OAuth App mit neuem Client ID/Secret
|
||||
- Speichert Credentials in Vault
|
||||
- Aktualisiert .env
|
||||
|
||||
### Vault-Zugriff
|
||||
|
||||
```bash
|
||||
# Vault Token (Development)
|
||||
export VAULT_TOKEN=breakpilot-dev-token
|
||||
|
||||
# Credentials lesen
|
||||
docker exec -e VAULT_TOKEN=$VAULT_TOKEN breakpilot-pwa-vault \
|
||||
vault kv get secret/cicd/woodpecker
|
||||
|
||||
# Credentials setzen
|
||||
docker exec -e VAULT_TOKEN=$VAULT_TOKEN breakpilot-pwa-vault \
|
||||
vault kv put secret/cicd/woodpecker \
|
||||
gitea_client_id="..." \
|
||||
gitea_client_secret="..."
|
||||
```
|
||||
|
||||
### Services neustarten nach Credentials-Änderung
|
||||
|
||||
```bash
|
||||
# Wichtig: --force-recreate um neue Env Vars zu laden
|
||||
cd /Users/benjaminadmin/Projekte/breakpilot-pwa
|
||||
docker compose up -d --force-recreate woodpecker-server
|
||||
|
||||
# Logs prüfen
|
||||
docker logs breakpilot-pwa-woodpecker-server --tail 50
|
||||
# Lokalen Build-Cache leeren
|
||||
ssh macmini "/usr/local/bin/docker builder prune -a"
|
||||
```
|
||||
|
||||
@@ -64,131 +64,39 @@ Module die Compliance-Kunden im SDK sehen und nutzen:
|
||||
| **Document Crawler** | Automatisches Crawling von Rechtstexten | /sdk/document-crawler |
|
||||
| **Advisory Board** | KI-Compliance-Beirat | /sdk/advisory-board |
|
||||
|
||||
## Admin-Module (Plattform-Verwaltung)
|
||||
|
||||
Interne Tools fuer die BreakPilot-Plattformverwaltung:
|
||||
|
||||
| Modul | Beschreibung | Frontend |
|
||||
|-------|--------------|----------|
|
||||
| **Katalogverwaltung** | SDK-Kataloge & Auswahltabellen | /dashboard/catalog-manager |
|
||||
| **Mandantenverwaltung** | B2B-Kundenverwaltung & Mandanten | /dashboard/multi-tenant |
|
||||
| **SSO-Konfiguration** | Single Sign-On & Authentifizierung | /dashboard/sso |
|
||||
| **DSB Portal** | Datenschutzbeauftragter-Arbeitsbereich | /dashboard/dsb-portal |
|
||||
|
||||
---
|
||||
|
||||
## URLs
|
||||
|
||||
### Production (Coolify-deployed)
|
||||
|
||||
| URL | Service | Beschreibung |
|
||||
|-----|---------|--------------|
|
||||
| https://macmini:3007/ | Admin Compliance | Compliance-Dashboard |
|
||||
| https://macmini:3006/ | Developer Portal | API-Dokumentation |
|
||||
| https://macmini:8002/ | Backend API | Compliance REST API |
|
||||
| https://macmini:8093/ | AI SDK API | SDK Backend-API |
|
||||
| https://admin-dev.breakpilot.ai/ | Admin Compliance | Compliance-Dashboard |
|
||||
| https://developers-dev.breakpilot.ai/ | Developer Portal | API-Dokumentation |
|
||||
| https://api-dev.breakpilot.ai/ | Backend API | Compliance REST API |
|
||||
| https://sdk-dev.breakpilot.ai/ | AI SDK API | SDK Backend-API |
|
||||
|
||||
### SDK-Module (Admin Compliance)
|
||||
### Lokal (Mac Mini — nur Dev/Tests)
|
||||
|
||||
| URL | Modul |
|
||||
|-----|-------|
|
||||
| https://macmini:3007/sdk | SDK Uebersicht |
|
||||
| https://macmini:3007/sdk/requirements | Requirements |
|
||||
| https://macmini:3007/sdk/controls | Controls |
|
||||
| https://macmini:3007/sdk/evidence | Evidence |
|
||||
| https://macmini:3007/sdk/risks | Risk Matrix |
|
||||
| https://macmini:3007/sdk/ai-act | AI Act |
|
||||
| https://macmini:3007/sdk/audit-checklist | Audit Checklist |
|
||||
| https://macmini:3007/sdk/audit-report | Audit Report |
|
||||
| https://macmini:3007/sdk/obligations | Obligations v2 |
|
||||
| https://macmini:3007/sdk/iace | IACE (CE-Risikobeurteilung) |
|
||||
| https://macmini:3007/sdk/import | Document Import |
|
||||
| https://macmini:3007/sdk/screening | System Screening |
|
||||
| https://macmini:3007/sdk/rag | RAG/Quellen |
|
||||
| https://macmini:3007/sdk/tom | TOM |
|
||||
| https://macmini:3007/sdk/dsfa | DSFA |
|
||||
| https://macmini:3007/sdk/vvt | VVT |
|
||||
| https://macmini:3007/sdk/loeschfristen | Loeschfristen |
|
||||
| https://macmini:3007/sdk/email-templates | E-Mail-Templates |
|
||||
| https://macmini:3007/sdk/academy | Academy |
|
||||
| https://macmini:3007/sdk/training | Training Engine |
|
||||
| https://macmini:3007/sdk/whistleblower | Whistleblower |
|
||||
| https://macmini:3007/sdk/incidents | Incidents |
|
||||
| https://macmini:3007/sdk/reporting | Reporting |
|
||||
| https://macmini:3007/sdk/vendor-compliance | Vendor Compliance |
|
||||
| https://macmini:3007/sdk/industry-templates | Branchenvorlagen |
|
||||
| https://macmini:3007/sdk/document-crawler | Document Crawler |
|
||||
| https://macmini:3007/sdk/advisory-board | Advisory Board |
|
||||
|
||||
### Admin-Module (Dashboard)
|
||||
|
||||
| URL | Modul |
|
||||
|-----|-------|
|
||||
| https://macmini:3007/dashboard | Dashboard |
|
||||
| https://macmini:3007/dashboard/catalog-manager | Katalogverwaltung |
|
||||
| https://macmini:3007/dashboard/multi-tenant | Mandantenverwaltung |
|
||||
| https://macmini:3007/dashboard/sso | SSO-Konfiguration |
|
||||
| https://macmini:3007/dashboard/dsb-portal | DSB Portal |
|
||||
|
||||
---
|
||||
|
||||
## Abhaengigkeiten zu Core
|
||||
|
||||
Compliance-Services nutzen folgende Core-Infrastruktur:
|
||||
|
||||
| Core Service | Genutzt von | Zweck |
|
||||
|-------------|-------------|-------|
|
||||
| PostgreSQL (46.225.100.82:54321, extern) | Alle | Compliance-Datenbank (Hetzner/meghshakka, TLS) |
|
||||
| Valkey (6379) | Backend, Admin | Session Cache |
|
||||
| Vault (8200) | Alle | Secrets Management |
|
||||
| Qdrant (qdrant-dev.breakpilot.ai) | AI SDK, Document Crawler | Vector-Suche (gehostet, API-Key) |
|
||||
| Hetzner Object Storage | TTS Service, Document Crawler | Datei-Storage (S3-kompatibel) |
|
||||
| Embedding (8087) | AI SDK | Text-Embeddings |
|
||||
| RAG Service (8097) | AI SDK | Retrieval Augmented Generation |
|
||||
| Nginx | Alle | HTTPS Reverse Proxy |
|
||||
|
||||
---
|
||||
|
||||
## Services-Dokumentation
|
||||
|
||||
- [AI Compliance SDK](services/ai-compliance-sdk/index.md)
|
||||
- [Architektur](services/ai-compliance-sdk/ARCHITECTURE.md)
|
||||
- [Developer Guide](services/ai-compliance-sdk/DEVELOPER.md)
|
||||
- [Auditor-Dokumentation](services/ai-compliance-sdk/AUDITOR_DOCUMENTATION.md)
|
||||
- [SBOM](services/ai-compliance-sdk/SBOM.md)
|
||||
- [Document Crawler](services/document-crawler/index.md)
|
||||
- SDK-Module:
|
||||
- [Analyse-Module (Paket 2)](services/sdk-modules/analyse-module.md) — Requirements, Controls, Evidence, Risk Matrix, AI Act, Audit Checklist, Audit Report
|
||||
- [Dokumentations-Module (Paket 3+)](services/sdk-modules/dokumentations-module.md) — VVT, Source Policy, Document Generator, Audit Checklist, Training Engine
|
||||
- [DSFA (Art. 35 DSGVO)](services/sdk-modules/dsfa.md) — vollständig backend-persistent, Migration 024
|
||||
- [Rechtliche Texte (Paket 4)](services/sdk-modules/rechtliche-texte.md) — Einwilligungen, Consent, Cookie Banner, Workflow
|
||||
- [Academy](services/sdk-modules/academy.md)
|
||||
- [Whistleblower](services/sdk-modules/whistleblower.md)
|
||||
- [Incidents](services/sdk-modules/incidents.md)
|
||||
- [Reporting](services/sdk-modules/reporting.md)
|
||||
- [Vendors](services/sdk-modules/vendors.md)
|
||||
- [Industry Templates](services/sdk-modules/industry-templates.md)
|
||||
- [Document Crawler](services/sdk-modules/document-crawler.md)
|
||||
- [Advisory Board](services/sdk-modules/advisory-board.md)
|
||||
- [DSB Portal](services/sdk-modules/dsb-portal.md)
|
||||
|
||||
## Entwicklung
|
||||
|
||||
- [Testing](development/testing.md)
|
||||
- [Dokumentation](development/documentation.md)
|
||||
- [CI/CD Pipeline](development/ci-cd-pipeline.md)
|
||||
| URL | Service |
|
||||
|-----|---------|
|
||||
| https://macmini:3007/ | Admin Compliance |
|
||||
| https://macmini:3006/ | Developer Portal |
|
||||
| https://macmini:8002/ | Backend API |
|
||||
| https://macmini:8093/ | AI SDK API |
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
# Voraussetzung: breakpilot-core muss laufen
|
||||
# Production (Coolify — Standardweg):
|
||||
git push origin main && git push gitea main
|
||||
# Coolify baut und deployt automatisch.
|
||||
|
||||
# Alle Compliance-Services starten
|
||||
# Lokal (Mac Mini — nur Dev/Tests):
|
||||
docker compose -f breakpilot-compliance/docker-compose.yml up -d
|
||||
|
||||
# Einzelnen Service neu bauen
|
||||
docker compose -f breakpilot-compliance/docker-compose.yml build --no-cache <service>
|
||||
docker compose -f breakpilot-compliance/docker-compose.yml up -d <service>
|
||||
```
|
||||
|
||||
---
|
||||
@@ -203,3 +111,17 @@ git push origin main && git push gitea main
|
||||
# origin: http://macmini:3003/pilotadmin/breakpilot-compliance.git
|
||||
# gitea: git@gitea.meghsakha.com:Benjamin_Boenisch/breakpilot-compliance.git
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Services-Dokumentation
|
||||
|
||||
- [AI Compliance SDK](services/ai-compliance-sdk/index.md)
|
||||
- [Document Crawler](services/document-crawler/index.md)
|
||||
- SDK-Module: siehe Unterverzeichnisse
|
||||
|
||||
## Entwicklung
|
||||
|
||||
- [Testing](development/testing.md)
|
||||
- [Dokumentation](development/documentation.md)
|
||||
- [CI/CD Pipeline](development/ci-cd-pipeline.md)
|
||||
|
||||
@@ -128,7 +128,7 @@ KI-generierte Inhalte werden via `compliance-tts-service` (Port 8095) in Audio u
|
||||
|
||||
- **Audio:** Piper TTS → MP3 (Modell: `de_DE-thorsten-high.onnx`)
|
||||
- **Video:** FFmpeg → MP4 (Skript + Stimme + Untertitel)
|
||||
- **Storage:** Hetzner Object Storage (`nbg1.your-objectstorage.com`, S3-kompatibel)
|
||||
- **Storage:** S3-kompatibles Object Storage (TLS)
|
||||
|
||||
```
|
||||
AudioPlayer → /sdk/v1/training/modules/:id/media (audio)
|
||||
|
||||
Reference in New Issue
Block a user