Add CLAUDE.md, MkDocs docs, docs page in admin, .claude/rules
- CLAUDE.md: Comprehensive documentation for Compliance SDK platform - docs-src: AI-Compliance-SDK docs (architecture, developer, auditor, SBOM) - mkdocs.yml: Compliance-specific nav with purple theme - docker-compose: Added docs service (port 8011, profile: docs) - admin-compliance: New /development/docs page with iframe + quick links - navigation.ts: Added development category with docs module - .claude/rules: testing, docs, open-source, compliance-checklist Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Benjamin Boenisch
402
docs-src/development/ci-cd-pipeline.md
Normal file
402
docs-src/development/ci-cd-pipeline.md
Normal file
@@ -0,0 +1,402 @@
|
||||
# CI/CD Pipeline
|
||||
|
||||
Übersicht über den Deployment-Prozess für Breakpilot.
|
||||
|
||||
## Übersicht
|
||||
|
||||
| Komponente | Build-Tool | Deployment |
|
||||
|------------|------------|------------|
|
||||
| Frontend (Next.js) | Docker | Mac Mini |
|
||||
| Backend (FastAPI) | Docker | Mac Mini |
|
||||
| Go Services | Docker (Multi-stage) | Mac Mini |
|
||||
| Documentation | MkDocs | Docker (Nginx) |
|
||||
|
||||
## Deployment-Architektur
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Entwickler-MacBook │
|
||||
│ │
|
||||
│ breakpilot-pwa/ │
|
||||
│ ├── studio-v2/ (Next.js Frontend) │
|
||||
│ ├── admin-v2/ (Next.js Admin) │
|
||||
│ ├── backend/ (Python FastAPI) │
|
||||
│ ├── consent-service/ (Go Service) │
|
||||
│ ├── klausur-service/ (Python FastAPI) │
|
||||
│ ├── voice-service/ (Python FastAPI) │
|
||||
│ ├── ai-compliance-sdk/ (Go Service) │
|
||||
│ └── docs-src/ (MkDocs) │
|
||||
│ │
|
||||
│ $ ./sync-and-deploy.sh │
|
||||
└───────────────────────────────┬─────────────────────────────────┘
|
||||
│
|
||||
│ rsync + SSH
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Mac Mini Server │
|
||||
│ │
|
||||
│ Docker Compose │
|
||||
│ ├── website (Port 3000) │
|
||||
│ ├── studio-v2 (Port 3001) │
|
||||
│ ├── admin-v2 (Port 3002) │
|
||||
│ ├── backend (Port 8000) │
|
||||
│ ├── consent-service (Port 8081) │
|
||||
│ ├── klausur-service (Port 8086) │
|
||||
│ ├── voice-service (Port 8082) │
|
||||
│ ├── ai-compliance-sdk (Port 8090) │
|
||||
│ ├── docs (Port 8009) │
|
||||
│ ├── postgres │
|
||||
│ ├── valkey (Redis) │
|
||||
│ ├── qdrant │
|
||||
│ └── minio │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Sync & Deploy Workflow
|
||||
|
||||
### 1. Dateien synchronisieren
|
||||
|
||||
```bash
|
||||
# Sync aller relevanten Verzeichnisse zum Mac Mini
|
||||
rsync -avz --delete \
|
||||
--exclude 'node_modules' \
|
||||
--exclude '.next' \
|
||||
--exclude '.git' \
|
||||
--exclude '__pycache__' \
|
||||
--exclude 'venv' \
|
||||
--exclude '.pytest_cache' \
|
||||
/Users/benjaminadmin/Projekte/breakpilot-pwa/ \
|
||||
macmini:/Users/benjaminadmin/Projekte/breakpilot-pwa/
|
||||
```
|
||||
|
||||
### 2. Container bauen
|
||||
|
||||
```bash
|
||||
# Einzelnen Service bauen
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache <service-name>"
|
||||
|
||||
# Beispiele:
|
||||
# studio-v2, admin-v2, website, backend, klausur-service, docs
|
||||
```
|
||||
|
||||
### 3. Container deployen
|
||||
|
||||
```bash
|
||||
# Container neu starten
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d <service-name>"
|
||||
```
|
||||
|
||||
### 4. Logs prüfen
|
||||
|
||||
```bash
|
||||
# Container-Logs anzeigen
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
logs -f <service-name>"
|
||||
```
|
||||
|
||||
## Service-spezifische Deployments
|
||||
|
||||
### Next.js Frontend (studio-v2, admin-v2, website)
|
||||
|
||||
```bash
|
||||
# 1. Sync
|
||||
rsync -avz --delete \
|
||||
--exclude 'node_modules' --exclude '.next' --exclude '.git' \
|
||||
/Users/benjaminadmin/Projekte/breakpilot-pwa/studio-v2/ \
|
||||
macmini:/Users/benjaminadmin/Projekte/breakpilot-pwa/studio-v2/
|
||||
|
||||
# 2. Build & Deploy
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache studio-v2 && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d studio-v2"
|
||||
```
|
||||
|
||||
### Python Services (backend, klausur-service, voice-service)
|
||||
|
||||
```bash
|
||||
# Build mit requirements.txt
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build klausur-service && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d klausur-service"
|
||||
```
|
||||
|
||||
### Go Services (consent-service, ai-compliance-sdk)
|
||||
|
||||
```bash
|
||||
# Multi-stage Build (Go → Alpine)
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache consent-service && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d consent-service"
|
||||
```
|
||||
|
||||
### MkDocs Dokumentation
|
||||
|
||||
```bash
|
||||
# Build & Deploy
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
build --no-cache docs && \
|
||||
/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d docs"
|
||||
|
||||
# Verfügbar unter: http://macmini:8009
|
||||
```
|
||||
|
||||
## Health Checks
|
||||
|
||||
### Service-Status prüfen
|
||||
|
||||
```bash
|
||||
# Alle Container-Status
|
||||
ssh macmini "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
|
||||
|
||||
# Health-Endpoints prüfen
|
||||
curl -s http://macmini:8000/health
|
||||
curl -s http://macmini:8081/health
|
||||
curl -s http://macmini:8086/health
|
||||
curl -s http://macmini:8090/health
|
||||
```
|
||||
|
||||
### Logs analysieren
|
||||
|
||||
```bash
|
||||
# Letzte 100 Zeilen
|
||||
ssh macmini "docker logs --tail 100 breakpilot-pwa-backend-1"
|
||||
|
||||
# Live-Logs folgen
|
||||
ssh macmini "docker logs -f breakpilot-pwa-backend-1"
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
### Container auf vorherige Version zurücksetzen
|
||||
|
||||
```bash
|
||||
# 1. Aktuelles Image taggen
|
||||
ssh macmini "docker tag breakpilot-pwa-backend:latest breakpilot-pwa-backend:backup"
|
||||
|
||||
# 2. Altes Image deployen
|
||||
ssh macmini "/usr/local/bin/docker compose \
|
||||
-f /Users/benjaminadmin/Projekte/breakpilot-pwa/docker-compose.yml \
|
||||
up -d backend"
|
||||
|
||||
# 3. Bei Problemen: Backup wiederherstellen
|
||||
ssh macmini "docker tag breakpilot-pwa-backend:backup breakpilot-pwa-backend:latest"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container startet nicht
|
||||
|
||||
```bash
|
||||
# 1. Logs prüfen
|
||||
ssh macmini "docker logs breakpilot-pwa-<service>-1"
|
||||
|
||||
# 2. Container manuell starten für Debug-Output
|
||||
ssh macmini "docker compose -f .../docker-compose.yml run --rm <service>"
|
||||
|
||||
# 3. In Container einloggen
|
||||
ssh macmini "docker exec -it breakpilot-pwa-<service>-1 /bin/sh"
|
||||
```
|
||||
|
||||
### Port bereits belegt
|
||||
|
||||
```bash
|
||||
# Port-Belegung prüfen
|
||||
ssh macmini "lsof -i :8000"
|
||||
|
||||
# Container mit dem Port finden
|
||||
ssh macmini "docker ps --filter publish=8000"
|
||||
```
|
||||
|
||||
### Build-Fehler
|
||||
|
||||
```bash
|
||||
# Cache komplett leeren
|
||||
ssh macmini "docker builder prune -a"
|
||||
|
||||
# Ohne Cache bauen
|
||||
ssh macmini "docker compose build --no-cache <service>"
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Resource-Nutzung
|
||||
|
||||
```bash
|
||||
# CPU/Memory aller Container
|
||||
ssh macmini "docker stats --no-stream"
|
||||
|
||||
# Disk-Nutzung
|
||||
ssh macmini "docker system df"
|
||||
```
|
||||
|
||||
### Cleanup
|
||||
|
||||
```bash
|
||||
# Ungenutzte Images/Container entfernen
|
||||
ssh macmini "docker system prune -a --volumes"
|
||||
|
||||
# Nur dangling Images
|
||||
ssh macmini "docker image prune"
|
||||
```
|
||||
|
||||
## Umgebungsvariablen
|
||||
|
||||
Umgebungsvariablen werden über `.env` Dateien und docker-compose.yml verwaltet:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
backend:
|
||||
environment:
|
||||
- DATABASE_URL=postgresql://...
|
||||
- REDIS_URL=redis://valkey:6379
|
||||
- SECRET_KEY=${SECRET_KEY}
|
||||
```
|
||||
|
||||
**Wichtig**: Sensible Werte niemals in Git committen. Stattdessen:
|
||||
- `.env` Datei auf dem Server pflegen
|
||||
- Secrets über HashiCorp Vault (siehe unten)
|
||||
|
||||
## Woodpecker CI - Automatisierte OAuth Integration
|
||||
|
||||
### Überblick
|
||||
|
||||
Die OAuth-Integration zwischen Woodpecker CI und Gitea ist **vollständig automatisiert**. Credentials werden in HashiCorp Vault gespeichert und bei Bedarf automatisch regeneriert.
|
||||
|
||||
!!! info "Warum automatisiert?"
|
||||
Diese Automatisierung ist eine DevSecOps Best Practice:
|
||||
|
||||
- **Infrastructure-as-Code**: Alles ist reproduzierbar
|
||||
- **Disaster Recovery**: Verlorene Credentials können automatisch regeneriert werden
|
||||
- **Security**: Secrets werden zentral in Vault verwaltet
|
||||
- **Onboarding**: Neue Entwickler müssen nichts manuell konfigurieren
|
||||
|
||||
### Architektur
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Mac Mini Server │
|
||||
│ │
|
||||
│ ┌───────────────┐ OAuth 2.0 ┌───────────────┐ │
|
||||
│ │ Gitea │ ←─────────────────────────→│ Woodpecker │ │
|
||||
│ │ (Port 3003) │ Client ID + Secret │ (Port 8090) │ │
|
||||
│ └───────────────┘ └───────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ OAuth App │ Env Vars│
|
||||
│ │ (DB: oauth2_application) │ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌───────────────────────────────────────────────────────────┐ │
|
||||
│ │ HashiCorp Vault (Port 8200) │ │
|
||||
│ │ │ │
|
||||
│ │ secret/cicd/woodpecker: │ │
|
||||
│ │ - gitea_client_id │ │
|
||||
│ │ - gitea_client_secret │ │
|
||||
│ │ │ │
|
||||
│ │ secret/cicd/api-tokens: │ │
|
||||
│ │ - gitea_token (für API-Zugriff) │ │
|
||||
│ │ - woodpecker_token (für Pipeline-Trigger) │ │
|
||||
│ └───────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Credentials-Speicherorte
|
||||
|
||||
| Ort | Pfad | Inhalt |
|
||||
|-----|------|--------|
|
||||
| **HashiCorp Vault** | `secret/cicd/woodpecker` | Client ID + Secret (Quelle der Wahrheit) |
|
||||
| **.env Datei** | `WOODPECKER_GITEA_CLIENT/SECRET` | Für Docker Compose (aus Vault geladen) |
|
||||
| **Gitea PostgreSQL** | `oauth2_application` Tabelle | OAuth App Registration (gehashtes Secret) |
|
||||
|
||||
### Troubleshooting: OAuth Fehler
|
||||
|
||||
Falls der Fehler "Client ID not registered" oder "user does not exist [uid: 0]" auftritt:
|
||||
|
||||
```bash
|
||||
# Option 1: Automatisches Regenerieren (empfohlen)
|
||||
./scripts/sync-woodpecker-credentials.sh --regenerate
|
||||
|
||||
# Option 2: Manuelles Vorgehen
|
||||
# 1. Credentials aus Vault laden
|
||||
vault kv get secret/cicd/woodpecker
|
||||
|
||||
# 2. .env aktualisieren
|
||||
WOODPECKER_GITEA_CLIENT=<client_id>
|
||||
WOODPECKER_GITEA_SECRET=<client_secret>
|
||||
|
||||
# 3. Zu Mac Mini synchronisieren
|
||||
rsync .env macmini:~/Projekte/breakpilot-pwa/
|
||||
|
||||
# 4. Woodpecker neu starten
|
||||
ssh macmini "cd ~/Projekte/breakpilot-pwa && \
|
||||
docker compose up -d --force-recreate woodpecker-server"
|
||||
```
|
||||
|
||||
### Das Sync-Script
|
||||
|
||||
Das Script `scripts/sync-woodpecker-credentials.sh` automatisiert den gesamten Prozess:
|
||||
|
||||
```bash
|
||||
# Credentials aus Vault laden und .env aktualisieren
|
||||
./scripts/sync-woodpecker-credentials.sh
|
||||
|
||||
# Neue Credentials generieren (OAuth App in Gitea + Vault + .env)
|
||||
./scripts/sync-woodpecker-credentials.sh --regenerate
|
||||
```
|
||||
|
||||
Was das Script macht:
|
||||
|
||||
1. **Liest** die aktuellen Credentials aus Vault
|
||||
2. **Aktualisiert** die .env Datei automatisch
|
||||
3. **Bei `--regenerate`**:
|
||||
- Löscht alte OAuth Apps in Gitea
|
||||
- Erstellt neue OAuth App mit neuem Client ID/Secret
|
||||
- Speichert Credentials in Vault
|
||||
- Aktualisiert .env
|
||||
|
||||
### Vault-Zugriff
|
||||
|
||||
```bash
|
||||
# Vault Token (Development)
|
||||
export VAULT_TOKEN=breakpilot-dev-token
|
||||
|
||||
# Credentials lesen
|
||||
docker exec -e VAULT_TOKEN=$VAULT_TOKEN breakpilot-pwa-vault \
|
||||
vault kv get secret/cicd/woodpecker
|
||||
|
||||
# Credentials setzen
|
||||
docker exec -e VAULT_TOKEN=$VAULT_TOKEN breakpilot-pwa-vault \
|
||||
vault kv put secret/cicd/woodpecker \
|
||||
gitea_client_id="..." \
|
||||
gitea_client_secret="..."
|
||||
```
|
||||
|
||||
### Services neustarten nach Credentials-Änderung
|
||||
|
||||
```bash
|
||||
# Wichtig: --force-recreate um neue Env Vars zu laden
|
||||
cd /Users/benjaminadmin/Projekte/breakpilot-pwa
|
||||
docker compose up -d --force-recreate woodpecker-server
|
||||
|
||||
# Logs prüfen
|
||||
docker logs breakpilot-pwa-woodpecker-server --tail 50
|
||||
```
|
||||
Reference in New Issue
Block a user