All checks were successful
CI/CD / go-lint (push) Has been skipped
CI/CD / python-lint (push) Has been skipped
CI/CD / nodejs-lint (push) Has been skipped
CI/CD / test-go-ai-compliance (push) Successful in 36s
CI/CD / test-python-backend-compliance (push) Successful in 44s
CI/CD / test-python-document-crawler (push) Successful in 24s
CI/CD / test-python-dsms-gateway (push) Successful in 21s
CI/CD / validate-canonical-controls (push) Successful in 12s
CI/CD / Deploy (push) Successful in 2s
After every push to gitea, Claude now automatically polls health endpoints and notifies the user when the deployment is ready for testing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
375 lines
14 KiB
Markdown
375 lines
14 KiB
Markdown
# BreakPilot Compliance - DSGVO/AI-Act SDK Platform
|
|
|
|
## Entwicklungsumgebung (WICHTIG - IMMER ZUERST LESEN)
|
|
|
|
### Zwei-Rechner-Setup + Coolify
|
|
|
|
| Geraet | Rolle | Aufgaben |
|
|
|--------|-------|----------|
|
|
| **MacBook** | Entwicklung | Claude Terminal, Code-Entwicklung, Browser (Frontend-Tests) |
|
|
| **Mac Mini** | Lokaler Server | Docker fuer lokale Dev/Tests (NICHT fuer Production!) |
|
|
| **Coolify** | Production | Automatisches Build + Deploy bei Push auf gitea |
|
|
|
|
**WICHTIG:** Code wird auf dem MacBook bearbeitet. Production-Deployment laeuft automatisch ueber Coolify.
|
|
|
|
### Entwicklungsworkflow (CI/CD — Coolify)
|
|
|
|
```bash
|
|
# 1. Code auf MacBook bearbeiten (dieses Verzeichnis)
|
|
# 2. Committen und zu BEIDEN Remotes pushen:
|
|
git push origin main && git push gitea main
|
|
|
|
# 3. FERTIG! Push auf gitea triggert automatisch:
|
|
# - Gitea Actions: Lint → Tests → Validierung
|
|
# - Coolify: Build → Deploy
|
|
# Dauer: ca. 3 Minuten
|
|
# Status pruefen: https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-compliance/actions
|
|
```
|
|
|
|
**NICHT MEHR NOETIG:** Manuelles `ssh macmini "docker compose build"` fuer Production.
|
|
**NIEMALS** manuell in Coolify auf "Redeploy" klicken — Gitea Actions triggert Coolify automatisch.
|
|
|
|
### Post-Push Deploy-Monitoring (PFLICHT nach jedem Push auf gitea)
|
|
|
|
**IMMER wenn Claude auf gitea pusht, MUSS danach automatisch das Deploy-Monitoring laufen:**
|
|
|
|
1. Dem User sofort mitteilen: "Deploy gestartet, ich ueberwache den Status..."
|
|
2. Im Hintergrund Health-Checks pollen (alle 20 Sekunden, max 5 Minuten):
|
|
```bash
|
|
# Compliance Health-Endpoints:
|
|
curl -sf https://api-dev.breakpilot.ai/health # Backend Compliance
|
|
curl -sf https://sdk-dev.breakpilot.ai/health # AI Compliance SDK
|
|
```
|
|
3. Sobald ALLE Endpoints healthy sind, dem User im Chat melden:
|
|
**"Deploy abgeschlossen! Du kannst jetzt testen: https://admin-dev.breakpilot.ai"**
|
|
4. Falls nach 5 Minuten noch nicht healthy → Fehlermeldung mit Hinweis auf Coolify-Logs.
|
|
|
|
**Ablauf im Terminal:**
|
|
```
|
|
> git push gitea main ✓
|
|
> "Deploy gestartet, ich ueberwache den Status..."
|
|
> [Hintergrund-Polling laeuft]
|
|
> "Deploy abgeschlossen! Alle Services healthy. Du kannst jetzt testen."
|
|
```
|
|
|
|
### CI/CD Pipeline (Gitea Actions → Coolify)
|
|
|
|
```
|
|
Push auf gitea main → go-lint/python-lint/nodejs-lint (nur PRs)
|
|
→ test-go-ai-compliance
|
|
→ test-python-backend-compliance
|
|
→ test-python-document-crawler
|
|
→ test-python-dsms-gateway
|
|
→ validate-canonical-controls
|
|
→ Coolify: Build + Deploy (automatisch bei Push)
|
|
```
|
|
|
|
**Dateien:**
|
|
- `.gitea/workflows/ci.yaml` — Pipeline-Definition (Tests + Validierung)
|
|
- `docker-compose.yml` — Haupt-Compose
|
|
- `docker-compose.hetzner.yml` — Override: arm64→amd64 fuer Coolify Production (x86_64)
|
|
|
|
### Lokale Entwicklung (Mac Mini — optional)
|
|
|
|
```bash
|
|
# Nur fuer lokale Tests, NICHT fuer Production:
|
|
ssh macmini "git -C /Users/benjaminadmin/Projekte/breakpilot-compliance pull --no-rebase origin main"
|
|
ssh macmini "/usr/local/bin/docker compose -f /Users/benjaminadmin/Projekte/breakpilot-compliance/docker-compose.yml build --no-cache <service>"
|
|
ssh macmini "/usr/local/bin/docker compose -f /Users/benjaminadmin/Projekte/breakpilot-compliance/docker-compose.yml up -d <service>"
|
|
|
|
# Fuer schnelle Iteration ohne Commit (rsync):
|
|
rsync -avz --exclude node_modules --exclude .next --exclude .git \
|
|
admin-compliance/ macmini:~/Projekte/breakpilot-compliance/admin-compliance/
|
|
```
|
|
|
|
**WICHTIG:** Docker-Pfad auf Mac Mini ist `/usr/local/bin/docker` (nicht im Standard-SSH-PATH).
|
|
**WICHTIG:** `cd` funktioniert NICHT in SSH-Einzelbefehlen — immer `-f <pfad>/docker-compose.yml` verwenden!
|
|
|
|
---
|
|
|
|
## Voraussetzung
|
|
|
|
**breakpilot-core MUSS laufen!** Dieses Projekt nutzt Core-Services:
|
|
- Valkey (Session-Cache)
|
|
- Vault (Secrets)
|
|
- RAG-Service (Vektorsuche fuer Compliance-Dokumente)
|
|
- Nginx (Reverse Proxy)
|
|
|
|
**Externe Services (Production):**
|
|
- PostgreSQL 17 (sslmode=require) — Schemas: `compliance`, `public`
|
|
- Qdrant @ `qdrant-dev.breakpilot.ai` (HTTPS, API-Key)
|
|
- Object Storage (S3-kompatibel, TLS)
|
|
|
|
Config via `.env` (nicht im Repo): `COMPLIANCE_DATABASE_URL`, `QDRANT_URL`, `QDRANT_API_KEY`
|
|
|
|
---
|
|
|
|
## Haupt-URLs
|
|
|
|
### Production (Coolify-deployed)
|
|
|
|
| URL | Service | Beschreibung |
|
|
|-----|---------|--------------|
|
|
| **https://admin-dev.breakpilot.ai/** | Admin Compliance | SDK-Dashboard, alle Compliance-Module |
|
|
| **https://developers-dev.breakpilot.ai/** | Developer Portal | API-Dokumentation fuer Kunden |
|
|
| https://api-dev.breakpilot.ai/ | Backend Compliance | Compliance APIs (DSGVO, DSR, GDPR) |
|
|
| https://sdk-dev.breakpilot.ai/ | AI Compliance SDK | KI-konforme Compliance-Analyse |
|
|
|
|
### Lokal (Mac Mini — nur Dev/Tests)
|
|
|
|
| URL | Service | Beschreibung |
|
|
|-----|---------|--------------|
|
|
| https://macmini:3007/ | Admin Compliance | Lokale Entwicklung |
|
|
| https://macmini:3006/ | Developer Portal | Lokale Entwicklung |
|
|
| https://macmini:8002/ | Backend Compliance | Lokale Entwicklung |
|
|
| https://macmini:8093/ | AI Compliance SDK | Lokale Entwicklung |
|
|
|
|
### Admin Compliance Module (https://macmini:3007/)
|
|
|
|
| Pfad | Modul | Beschreibung |
|
|
|------|-------|--------------|
|
|
| `/dashboard` | Dashboard | Uebersicht + Catalog-Manager |
|
|
| `/sdk/tom` | TOM | Technisch-Organisatorische Massnahmen |
|
|
| `/sdk/dsfa` | DSFA | Datenschutz-Folgenabschaetzung |
|
|
| `/sdk/vvt` | VVT | Verzeichnis von Verarbeitungstaetigkeiten |
|
|
| `/sdk/loeschfristen` | Loeschfristen | Loeschfristen-Verwaltung |
|
|
| `/sdk/ai-act` | AI Act | KI-Verordnung Compliance |
|
|
| `/sdk/consent` | Consent | Einwilligungsmanagement |
|
|
| `/sdk/dsr` | DSR | Betroffenenrechte |
|
|
| `/sdk/vendor-compliance` | Vendor | Auftragsverarbeitung |
|
|
| `/sdk/risk-assessment` | Risiko | Risikobewertung |
|
|
| `/sdk/incident-response` | Vorfaelle | Datenschutz-Vorfaelle |
|
|
| `/sdk/training` | Schulung | Mitarbeiter-Schulungen |
|
|
| `/sdk/audit` | Audit | Audit-Management |
|
|
| `/sdk/policy-generator` | Policies | Richtlinien-Generator |
|
|
| `/sdk/data-mapping` | Data Map | Datenfluss-Mapping |
|
|
| `/developers` | Developer Portal | SDK API-Docs |
|
|
|
|
---
|
|
|
|
## Services (10 Container)
|
|
|
|
| Service | Tech | Port | Container |
|
|
|---------|------|------|-----------|
|
|
| admin-compliance | Next.js 15 | 3007 (via nginx) | bp-compliance-admin |
|
|
| backend-compliance | Python/FastAPI | 8002 | bp-compliance-backend |
|
|
| ai-compliance-sdk | Go/Gin | 8090→8093 | bp-compliance-ai-sdk |
|
|
| developer-portal | Next.js 15 | 3006 (via nginx) | bp-compliance-developer-portal |
|
|
| compliance-tts-service | Python/Piper TTS | 8095 | bp-compliance-tts |
|
|
| document-crawler | Python/FastAPI | 8098 | bp-compliance-document-crawler |
|
|
| dsms-node | IPFS Kubo | 4001/5001/8085 | bp-compliance-dsms-node |
|
|
| dsms-gateway | Node.js | 8082 | bp-compliance-dsms-gateway |
|
|
| docs | MkDocs/nginx | 8011 | bp-compliance-docs |
|
|
| core-wait | curl health-check | - | bp-compliance-core-wait |
|
|
|
|
### Docker-Netzwerk
|
|
Nutzt das externe Core-Netzwerk:
|
|
```yaml
|
|
networks:
|
|
breakpilot-network:
|
|
external: true
|
|
name: breakpilot-network
|
|
```
|
|
|
|
### Container-Naming: `bp-compliance-*`
|
|
### DB search_path: `compliance,core,public`
|
|
|
|
---
|
|
|
|
## Verzeichnisstruktur
|
|
|
|
```
|
|
breakpilot-compliance/
|
|
├── .claude/
|
|
│ ├── CLAUDE.md # Diese Datei
|
|
│ └── rules/ # Automatische Regeln
|
|
├── admin-compliance/ # Next.js Compliance Dashboard
|
|
│ ├── app/(sdk)/ # 37 SDK-Route-Dirs
|
|
│ ├── app/(admin)/ # Dashboard + Catalog-Manager
|
|
│ ├── components/sdk/ # SDKSidebar, CommandBar, ComplianceAdvisor
|
|
│ ├── components/catalog-manager/ # Shared Catalog UI
|
|
│ └── lib/sdk/ # SDK Context, Types, API-Client
|
|
├── backend-compliance/ # Python/FastAPI Backend
|
|
│ ├── compliance/ # Haupt-Package (40 Dateien)
|
|
│ │ ├── api/ # API Router
|
|
│ │ ├── db/ # DB Models
|
|
│ │ ├── services/ # Business Logic
|
|
│ │ └── data/ # Stammdaten
|
|
│ ├── consent_admin_api.py
|
|
│ ├── dsr_api.py
|
|
│ └── gdpr_api.py
|
|
├── ai-compliance-sdk/ # KI-Compliance Analyse Service
|
|
├── developer-portal/ # API-Dokumentation (Next.js)
|
|
├── breakpilot-compliance-sdk/ # SDK Package
|
|
├── consent-sdk/ # Consent SDK Package
|
|
├── pca-platform/ # Privacy Compliance Automation
|
|
├── dsms-node/ # IPFS Node
|
|
├── dsms-gateway/ # IPFS Gateway
|
|
├── scripts/ # Helper Scripts
|
|
├── docker-compose.yml # Compliance Compose (~10 Services, platform: arm64)
|
|
├── docker-compose.hetzner.yml # Override: arm64→amd64 fuer Coolify Production
|
|
└── .gitea/workflows/ci.yaml # CI/CD Pipeline (Lint → Tests → Validierung)
|
|
```
|
|
|
|
---
|
|
|
|
## Haeufige Befehle
|
|
|
|
### Deployment (CI/CD — Standardweg)
|
|
|
|
```bash
|
|
# Committen und pushen → Coolify deployt automatisch:
|
|
git push origin main && git push gitea main
|
|
|
|
# CI-Status pruefen (im Browser):
|
|
# https://gitea.meghsakha.com/Benjamin_Boenisch/breakpilot-compliance/actions
|
|
|
|
# Health Checks:
|
|
curl -sf https://api-dev.breakpilot.ai/health
|
|
curl -sf https://sdk-dev.breakpilot.ai/health
|
|
```
|
|
|
|
### Git
|
|
|
|
```bash
|
|
# Zu BEIDEN Remotes pushen (PFLICHT! — vom MacBook):
|
|
git push origin main && git push gitea main
|
|
|
|
# Remotes:
|
|
# origin: lokale Gitea (macmini:3003)
|
|
# gitea: gitea.meghsakha.com:22222
|
|
```
|
|
|
|
### Lokale Docker-Befehle (Mac Mini — nur fuer Dev/Tests)
|
|
|
|
```bash
|
|
# Logs
|
|
ssh macmini "/usr/local/bin/docker logs -f bp-compliance-<service>"
|
|
|
|
# Status
|
|
ssh macmini "/usr/local/bin/docker ps --filter name=bp-compliance"
|
|
|
|
# Lokaler Rebuild (nur wenn noetig):
|
|
ssh macmini "git -C /Users/benjaminadmin/Projekte/breakpilot-compliance pull --no-rebase origin main"
|
|
ssh macmini "/usr/local/bin/docker compose -f /Users/benjaminadmin/Projekte/breakpilot-compliance/docker-compose.yml build --no-cache <service> && /usr/local/bin/docker compose -f /Users/benjaminadmin/Projekte/breakpilot-compliance/docker-compose.yml up -d <service>"
|
|
```
|
|
|
|
---
|
|
|
|
## Kernprinzipien
|
|
|
|
### 1. Open Source Policy
|
|
- **NUR Open Source mit kommerziell nutzbarer Lizenz**
|
|
- Erlaubt: MIT, Apache-2.0, BSD, ISC, MPL-2.0, LGPL
|
|
- **VERBOTEN:** GPL (ausser LGPL), AGPL, proprietaer
|
|
|
|
### 2. DSGVO-Compliance
|
|
- Dieses Projekt implementiert DSGVO-Tools — es muss selbst DSGVO-konform sein
|
|
- Audit-Logging fuer alle Compliance-Aktionen
|
|
- Consent-Management via Core consent-service
|
|
|
|
### 3. AI Act Compliance
|
|
- KI-Risikobewertung fuer alle KI-Features
|
|
- Human Oversight sicherstellen
|
|
- Transparenzpflicht bei KI-Nutzung
|
|
|
|
### 4. Testing & Dokumentation
|
|
- Tests sind Pflicht bei jeder Aenderung
|
|
- Compliance-Checkliste bei neuen Features durchgehen
|
|
|
|
### 5. Sensitive Dateien
|
|
**NIEMALS aendern oder committen:**
|
|
- `.env`, `.env.local`, Vault-Tokens, SSL-Zertifikate
|
|
- `*.pdf`, `*.docx`, kompilierte Binaries, grosse Medien
|
|
|
|
---
|
|
|
|
## Tech-Stack
|
|
|
|
| Sprache | Services |
|
|
|---------|----------|
|
|
| Python/FastAPI | backend-compliance, ai-compliance-sdk, pca-platform |
|
|
| TypeScript/Next.js | admin-compliance, developer-portal |
|
|
| Node.js | dsms-node, dsms-gateway, consent-sdk |
|
|
|
|
---
|
|
|
|
## SDK-Module im Detail
|
|
|
|
### Katalog-System (Shared mit Lehrer)
|
|
- `components/catalog-manager/` — CatalogManagerContent, CatalogTable, CatalogModuleTabs, CatalogEntryForm
|
|
- `lib/sdk/catalog-manager/` — catalog-registry.ts, types.ts
|
|
- 17 DSGVO/AI-Act Kataloge (dsfa, vvt-baseline, vendor-compliance, etc.)
|
|
|
|
### Multi-Projekt-Architektur (seit 2026-03-09)
|
|
|
|
Jeder Tenant kann mehrere Compliance-Projekte anlegen. CompanyProfile ist **pro Projekt** (nicht tenant-weit).
|
|
|
|
**URL-Schema:** `/sdk?project={uuid}` — alle SDK-Seiten enthalten `?project=` Query-Param.
|
|
`/sdk` ohne `?project=` zeigt die Projektliste (ProjectSelector).
|
|
|
|
**Datenbank:**
|
|
- `compliance_projects` — Projekt-Metadaten (Name, Typ, Status, Version)
|
|
- `sdk_states` — UNIQUE auf `(tenant_id, project_id)` statt nur `tenant_id`
|
|
- Migration: `039_compliance_projects.sql`
|
|
|
|
**Backend API (FastAPI):**
|
|
```
|
|
GET /api/v1/projects → Alle Projekte des Tenants
|
|
POST /api/v1/projects → Neues Projekt erstellen (mit copy_from_project_id)
|
|
GET /api/v1/projects/{project_id} → Einzelnes Projekt laden
|
|
PATCH /api/v1/projects/{project_id} → Projekt aktualisieren
|
|
DELETE /api/v1/projects/{project_id} → Projekt archivieren (Soft Delete)
|
|
```
|
|
|
|
**Frontend:**
|
|
- `components/sdk/ProjectSelector/ProjectSelector.tsx` — Projektliste + Erstellen-Dialog
|
|
- `lib/sdk/types.ts` — `ProjectInfo` Interface, `SDKState.projectId`
|
|
- `lib/sdk/context.tsx` — `projectId` Prop, `createProject()`, `listProjects()`, `switchProject()`
|
|
- `lib/sdk/sync.ts` — BroadcastChannel + localStorage pro Projekt
|
|
- `lib/sdk/api-client.ts` — `projectId` in State-API + Projekt-CRUD-Methoden
|
|
- `app/sdk/layout.tsx` — liest `?project=` aus searchParams
|
|
- `app/api/sdk/v1/projects/` — Next.js Proxy zum Backend
|
|
|
|
### Backend-Compliance APIs
|
|
```
|
|
POST/GET /api/v1/compliance/risks
|
|
POST/GET /api/v1/compliance/controls
|
|
POST/GET /api/v1/compliance/requirements
|
|
POST/GET /api/v1/compliance/evidence
|
|
POST/GET /api/v1/dsr/requests
|
|
POST/GET /api/v1/gdpr/exports
|
|
POST/GET /api/v1/consent/admin
|
|
|
|
# Stammdaten, Versionierung & Change-Requests
|
|
GET/POST/DELETE /api/compliance/company-profile
|
|
GET /api/compliance/company-profile/template-context
|
|
GET /api/compliance/change-requests
|
|
GET /api/compliance/change-requests/stats
|
|
POST /api/compliance/change-requests/{id}/accept
|
|
POST /api/compliance/change-requests/{id}/reject
|
|
POST /api/compliance/change-requests/{id}/edit
|
|
GET /api/compliance/generation/preview/{doc_type}
|
|
POST /api/compliance/generation/apply/{doc_type}
|
|
GET /api/compliance/{doc}/{id}/versions
|
|
```
|
|
|
|
### Multi-Tenancy
|
|
- Shared Dependency: `compliance/api/tenant_utils.py` (`get_tenant_id()`)
|
|
- UUID-Format, kein `"default"` mehr
|
|
- Header `X-Tenant-ID` > Query `tenant_id` > ENV-Fallback
|
|
|
|
---
|
|
|
|
## Wichtige Dateien (Referenz)
|
|
|
|
| Datei | Beschreibung |
|
|
|-------|--------------|
|
|
| `admin-compliance/app/(sdk)/` | Alle 37+ SDK-Routes |
|
|
| `admin-compliance/components/sdk/Sidebar/SDKSidebar.tsx` | SDK Navigation |
|
|
| `admin-compliance/components/sdk/CommandBar.tsx` | Command Palette |
|
|
| `admin-compliance/lib/sdk/context.tsx` | SDK State (Provider) |
|
|
| `backend-compliance/compliance/` | Haupt-Package (50+ Dateien) |
|
|
| `ai-compliance-sdk/` | KI-Compliance Analyse |
|
|
| `developer-portal/` | API-Dokumentation |
|