Initial commit: breakpilot-core - Shared Infrastructure
Docker Compose with 24+ services: - PostgreSQL (PostGIS), Valkey, MinIO, Qdrant - Vault (PKI/TLS), Nginx (Reverse Proxy) - Backend Core API, Consent Service, Billing Service - RAG Service, Embedding Service - Gitea, Woodpecker CI/CD - Night Scheduler, Health Aggregator - Jitsi (Web/XMPP/JVB/Jicofo), Mailpit Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Benjamin Boenisch
251
docs-src/architecture/secrets-management.md
Normal file
251
docs-src/architecture/secrets-management.md
Normal file
@@ -0,0 +1,251 @@
|
||||
# BreakPilot Secrets Management
|
||||
|
||||
## Uebersicht
|
||||
|
||||
BreakPilot verwendet **HashiCorp Vault** als zentrales Secrets-Management-System.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ SECRETS MANAGEMENT │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ HashiCorp Vault │ │
|
||||
│ │ Port 8200 │ │
|
||||
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │ │
|
||||
│ │ │ KV v2 Engine │ │ AppRole Auth │ │ Audit Logging │ │ │
|
||||
│ │ │ secret/ │ │ Token Auth │ │ Verschluesselung │ │ │
|
||||
│ │ └──────────────┘ └──────────────┘ └──────────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────────┼─────────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Python Backend │ │ Go Services │ │ Frontend │ │
|
||||
│ │ (hvac client) │ │ (vault-client) │ │ (via Backend) │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Warum Vault?
|
||||
|
||||
| Alternative | Nachteil |
|
||||
|-------------|----------|
|
||||
| Environment Variables | Keine Audit-Logs, keine Verschluesselung, keine Rotation |
|
||||
| Docker Secrets | Nur fuer Docker Swarm, keine zentrale Verwaltung |
|
||||
| AWS Secrets Manager | Cloud Lock-in, Kosten |
|
||||
| Kubernetes Secrets | Keine Verschluesselung by default, nur K8s |
|
||||
| **HashiCorp Vault** | Open Source (BSL 1.1), Self-Hosted, Enterprise Features |
|
||||
|
||||
## Architektur
|
||||
|
||||
### Secret-Hierarchie
|
||||
|
||||
```
|
||||
secret/breakpilot/
|
||||
├── api_keys/
|
||||
│ ├── anthropic # Anthropic Claude API Key
|
||||
│ ├── vast # vast.ai GPU API Key
|
||||
│ ├── stripe # Stripe Payment Key
|
||||
│ ├── stripe_webhook
|
||||
│ └── tavily # Tavily Search API Key
|
||||
├── database/
|
||||
│ ├── postgres # username, password, url
|
||||
│ └── synapse # Matrix Synapse DB
|
||||
├── auth/
|
||||
│ ├── jwt # secret, refresh_secret
|
||||
│ └── keycloak # client_secret
|
||||
├── communication/
|
||||
│ ├── matrix # access_token, db_password
|
||||
│ └── jitsi # app_secret, jicofo, jvb passwords
|
||||
├── storage/
|
||||
│ └── minio # access_key, secret_key
|
||||
└── infra/
|
||||
└── vast # api_key, instance_id, control_key
|
||||
```
|
||||
|
||||
### Python Integration
|
||||
|
||||
```python
|
||||
from secrets import get_secret
|
||||
|
||||
# Einzelnes Secret abrufen
|
||||
api_key = get_secret("ANTHROPIC_API_KEY")
|
||||
|
||||
# Mit Default-Wert
|
||||
debug = get_secret("DEBUG", default="false")
|
||||
|
||||
# Als Pflicht-Secret
|
||||
db_url = get_secret("DATABASE_URL", required=True)
|
||||
```
|
||||
|
||||
### Fallback-Reihenfolge
|
||||
|
||||
```
|
||||
1. HashiCorp Vault (wenn VAULT_ADDR gesetzt)
|
||||
↓ falls nicht verfuegbar
|
||||
2. Environment Variables
|
||||
↓ falls nicht gesetzt
|
||||
3. Docker Secrets (/run/secrets/)
|
||||
↓ falls nicht vorhanden
|
||||
4. Default-Wert (wenn angegeben)
|
||||
↓ sonst
|
||||
5. SecretNotFoundError (wenn required=True)
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
### Entwicklung (Dev Mode)
|
||||
|
||||
```bash
|
||||
# Vault starten (Dev Mode - NICHT fuer Produktion!)
|
||||
docker-compose -f docker-compose.vault.yml up -d vault
|
||||
|
||||
# Warten bis healthy
|
||||
docker-compose -f docker-compose.vault.yml up vault-init
|
||||
|
||||
# Environment setzen
|
||||
export VAULT_ADDR=http://localhost:8200
|
||||
export VAULT_TOKEN=breakpilot-dev-token
|
||||
```
|
||||
|
||||
### Secrets setzen
|
||||
|
||||
```bash
|
||||
# Anthropic API Key
|
||||
vault kv put secret/breakpilot/api_keys/anthropic value='sk-ant-api03-...'
|
||||
|
||||
# vast.ai Credentials
|
||||
vault kv put secret/breakpilot/infra/vast \
|
||||
api_key='xxx' \
|
||||
instance_id='123' \
|
||||
control_key='yyy'
|
||||
|
||||
# Database
|
||||
vault kv put secret/breakpilot/database/postgres \
|
||||
username='breakpilot' \
|
||||
password='supersecret' \
|
||||
url='postgres://breakpilot:supersecret@localhost:5432/breakpilot_db'
|
||||
```
|
||||
|
||||
### Secrets lesen
|
||||
|
||||
```bash
|
||||
# Liste aller Secrets
|
||||
vault kv list secret/breakpilot/
|
||||
|
||||
# Secret anzeigen
|
||||
vault kv get secret/breakpilot/api_keys/anthropic
|
||||
|
||||
# Nur den Wert
|
||||
vault kv get -field=value secret/breakpilot/api_keys/anthropic
|
||||
```
|
||||
|
||||
## Produktion
|
||||
|
||||
### AppRole Authentication
|
||||
|
||||
In Produktion verwenden Services AppRole statt Token-Auth:
|
||||
|
||||
```bash
|
||||
# 1. AppRole aktivieren (einmalig)
|
||||
vault auth enable approle
|
||||
|
||||
# 2. Policy erstellen
|
||||
vault policy write breakpilot-backend - <<EOF
|
||||
path "secret/data/breakpilot/*" {
|
||||
capabilities = ["read", "list"]
|
||||
}
|
||||
EOF
|
||||
|
||||
# 3. Role erstellen
|
||||
vault write auth/approle/role/breakpilot-backend \
|
||||
token_policies="breakpilot-backend" \
|
||||
token_ttl=1h \
|
||||
token_max_ttl=4h
|
||||
|
||||
# 4. Role-ID holen (fix)
|
||||
vault read -field=role_id auth/approle/role/breakpilot-backend/role-id
|
||||
|
||||
# 5. Secret-ID generieren (bei jedem Deploy neu)
|
||||
vault write -f auth/approle/role/breakpilot-backend/secret-id
|
||||
```
|
||||
|
||||
### Environment fuer Services
|
||||
|
||||
```bash
|
||||
# Docker-Compose / Kubernetes
|
||||
VAULT_ADDR=https://vault.breakpilot.app:8200
|
||||
VAULT_AUTH_METHOD=approle
|
||||
VAULT_ROLE_ID=<role-id>
|
||||
VAULT_SECRET_ID=<secret-id>
|
||||
VAULT_SECRETS_PATH=breakpilot
|
||||
```
|
||||
|
||||
## Sicherheits-Checkliste
|
||||
|
||||
### Muss erfuellt sein
|
||||
|
||||
- [ ] Keine echten Secrets in `.env` Dateien
|
||||
- [ ] `.env` in `.gitignore`
|
||||
- [ ] Vault im Sealed-State wenn nicht in Verwendung
|
||||
- [ ] TLS fuer Vault in Produktion
|
||||
- [ ] AppRole statt Token-Auth in Produktion
|
||||
- [ ] Audit-Logging aktiviert
|
||||
- [ ] Minimale Policies (Least Privilege)
|
||||
|
||||
### Sollte erfuellt sein
|
||||
|
||||
- [ ] Automatische Secret-Rotation
|
||||
- [ ] Separate Vault-Instanz fuer Produktion
|
||||
- [ ] HSM-basiertes Auto-Unseal
|
||||
- [ ] Disaster Recovery Plan
|
||||
|
||||
## Dateien
|
||||
|
||||
| Datei | Beschreibung |
|
||||
|-------|--------------|
|
||||
| `backend/secrets/__init__.py` | Secrets-Modul Exports |
|
||||
| `backend/secrets/vault_client.py` | Vault Client Implementation |
|
||||
| `docker-compose.vault.yml` | Vault Docker Configuration |
|
||||
| `vault/init-secrets.sh` | Entwicklungs-Secrets Initialisierung |
|
||||
| `vault/policies/` | Vault Policy Files |
|
||||
|
||||
## Fehlerbehebung
|
||||
|
||||
### Vault nicht erreichbar
|
||||
|
||||
```bash
|
||||
# Status pruefen
|
||||
vault status
|
||||
|
||||
# Falls sealed
|
||||
vault operator unseal <unseal-key>
|
||||
```
|
||||
|
||||
### Secret nicht gefunden
|
||||
|
||||
```bash
|
||||
# Pfad pruefen
|
||||
vault kv list secret/breakpilot/
|
||||
|
||||
# Cache leeren (Python)
|
||||
from secrets import get_secrets_manager
|
||||
get_secrets_manager().clear_cache()
|
||||
```
|
||||
|
||||
### Token abgelaufen
|
||||
|
||||
```bash
|
||||
# Neuen Token holen (AppRole)
|
||||
vault write auth/approle/login \
|
||||
role_id=$VAULT_ROLE_ID \
|
||||
secret_id=$VAULT_SECRET_ID
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Referenzen
|
||||
|
||||
- [HashiCorp Vault Documentation](https://developer.hashicorp.com/vault/docs)
|
||||
- [hvac Python Client](https://hvac.readthedocs.io/)
|
||||
- [Vault Best Practices](https://developer.hashicorp.com/vault/tutorials/recommended-patterns)
|
||||
Reference in New Issue
Block a user