feat: Add Academy, Whistleblower, Incidents SDK modules, pitch-deck, blog and CI/CD config
Some checks failed
ci/woodpecker/push/integration Pipeline failed
ci/woodpecker/push/main Pipeline failed
CI/CD Pipeline / Go Tests (push) Has been cancelled
CI/CD Pipeline / Python Tests (push) Has been cancelled
CI/CD Pipeline / Website Tests (push) Has been cancelled
CI/CD Pipeline / Linting (push) Has been cancelled
CI/CD Pipeline / Security Scan (push) Has been cancelled
CI/CD Pipeline / Docker Build & Push (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / CI Summary (push) Has been cancelled
Security Scanning / Secret Scanning (push) Has been cancelled
Security Scanning / Dependency Vulnerability Scan (push) Has been cancelled
Security Scanning / Go Security Scan (push) Has been cancelled
Security Scanning / Python Security Scan (push) Has been cancelled
Security Scanning / Node.js Security Scan (push) Has been cancelled
Security Scanning / Docker Image Security (push) Has been cancelled
Security Scanning / Security Summary (push) Has been cancelled
Tests / Go Tests (push) Has been cancelled
Tests / Python Tests (push) Has been cancelled
Tests / Integration Tests (push) Has been cancelled
Tests / Go Lint (push) Has been cancelled
Tests / Python Lint (push) Has been cancelled
Tests / Security Scan (push) Has been cancelled
Tests / All Checks Passed (push) Has been cancelled
Some checks failed
ci/woodpecker/push/integration Pipeline failed
ci/woodpecker/push/main Pipeline failed
CI/CD Pipeline / Go Tests (push) Has been cancelled
CI/CD Pipeline / Python Tests (push) Has been cancelled
CI/CD Pipeline / Website Tests (push) Has been cancelled
CI/CD Pipeline / Linting (push) Has been cancelled
CI/CD Pipeline / Security Scan (push) Has been cancelled
CI/CD Pipeline / Docker Build & Push (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / CI Summary (push) Has been cancelled
Security Scanning / Secret Scanning (push) Has been cancelled
Security Scanning / Dependency Vulnerability Scan (push) Has been cancelled
Security Scanning / Go Security Scan (push) Has been cancelled
Security Scanning / Python Security Scan (push) Has been cancelled
Security Scanning / Node.js Security Scan (push) Has been cancelled
Security Scanning / Docker Image Security (push) Has been cancelled
Security Scanning / Security Summary (push) Has been cancelled
Tests / Go Tests (push) Has been cancelled
Tests / Python Tests (push) Has been cancelled
Tests / Integration Tests (push) Has been cancelled
Tests / Go Lint (push) Has been cancelled
Tests / Python Lint (push) Has been cancelled
Tests / Security Scan (push) Has been cancelled
Tests / All Checks Passed (push) Has been cancelled
- Academy, Whistleblower, Incidents frontend pages with API proxies and types - Vendor compliance API proxy route - Go backend handlers and models for all new SDK modules - Investor pitch-deck app with interactive slides - Blog section with DSGVO, AI Act, NIS2, glossary articles - MkDocs documentation site - CI/CD pipelines (Woodpecker, GitHub Actions), security scanning config - Planning and implementation documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
132
admin-v2/.woodpecker/auto-fix.yml
Normal file
132
admin-v2/.woodpecker/auto-fix.yml
Normal file
@@ -0,0 +1,132 @@
|
||||
# Woodpecker CI Auto-Fix Pipeline
|
||||
# Automatische Reparatur fehlgeschlagener Tests
|
||||
#
|
||||
# Laeuft taeglich um 2:00 Uhr nachts
|
||||
# Analysiert offene Backlog-Items und versucht automatische Fixes
|
||||
|
||||
when:
|
||||
- event: cron
|
||||
cron: "0 2 * * *" # Taeglich um 2:00 Uhr
|
||||
|
||||
clone:
|
||||
git:
|
||||
image: woodpeckerci/plugin-git
|
||||
settings:
|
||||
depth: 1
|
||||
extra_hosts:
|
||||
- macmini:192.168.178.100
|
||||
|
||||
steps:
|
||||
# ========================================
|
||||
# 1. Fetch Failed Tests from Backlog
|
||||
# ========================================
|
||||
|
||||
fetch-backlog:
|
||||
image: curlimages/curl:latest
|
||||
commands:
|
||||
- |
|
||||
curl -s "http://backend:8000/api/tests/backlog?status=open&priority=critical" \
|
||||
-o backlog-critical.json
|
||||
curl -s "http://backend:8000/api/tests/backlog?status=open&priority=high" \
|
||||
-o backlog-high.json
|
||||
- echo "=== Kritische Tests ==="
|
||||
- cat backlog-critical.json | head -50
|
||||
- echo "=== Hohe Prioritaet ==="
|
||||
- cat backlog-high.json | head -50
|
||||
|
||||
# ========================================
|
||||
# 2. Analyze and Classify Errors
|
||||
# ========================================
|
||||
|
||||
analyze-errors:
|
||||
image: python:3.12-slim
|
||||
commands:
|
||||
- pip install --quiet jq-py
|
||||
- |
|
||||
python3 << 'EOF'
|
||||
import json
|
||||
import os
|
||||
|
||||
def classify_error(error_type, error_msg):
|
||||
"""Klassifiziert Fehler nach Auto-Fix-Potential"""
|
||||
auto_fixable = {
|
||||
'nil_pointer': 'high',
|
||||
'import_error': 'high',
|
||||
'undefined_variable': 'medium',
|
||||
'type_error': 'medium',
|
||||
'assertion': 'low',
|
||||
'timeout': 'low',
|
||||
'logic_error': 'manual'
|
||||
}
|
||||
return auto_fixable.get(error_type, 'manual')
|
||||
|
||||
# Lade Backlog
|
||||
try:
|
||||
with open('backlog-critical.json') as f:
|
||||
critical = json.load(f)
|
||||
with open('backlog-high.json') as f:
|
||||
high = json.load(f)
|
||||
except:
|
||||
print("Keine Backlog-Daten gefunden")
|
||||
exit(0)
|
||||
|
||||
all_items = critical.get('items', []) + high.get('items', [])
|
||||
|
||||
auto_fix_candidates = []
|
||||
for item in all_items:
|
||||
fix_potential = classify_error(
|
||||
item.get('error_type', 'unknown'),
|
||||
item.get('error_message', '')
|
||||
)
|
||||
if fix_potential in ['high', 'medium']:
|
||||
auto_fix_candidates.append({
|
||||
'id': item.get('id'),
|
||||
'test_name': item.get('test_name'),
|
||||
'error_type': item.get('error_type'),
|
||||
'fix_potential': fix_potential
|
||||
})
|
||||
|
||||
print(f"Auto-Fix Kandidaten: {len(auto_fix_candidates)}")
|
||||
with open('auto-fix-candidates.json', 'w') as f:
|
||||
json.dump(auto_fix_candidates, f, indent=2)
|
||||
EOF
|
||||
depends_on:
|
||||
- fetch-backlog
|
||||
|
||||
# ========================================
|
||||
# 3. Generate Fix Suggestions (Placeholder)
|
||||
# ========================================
|
||||
|
||||
generate-fixes:
|
||||
image: python:3.12-slim
|
||||
commands:
|
||||
- |
|
||||
echo "Auto-Fix Generation ist in Phase 4 geplant"
|
||||
echo "Aktuell werden nur Vorschlaege generiert"
|
||||
|
||||
# Hier wuerde Claude API oder anderer LLM aufgerufen werden
|
||||
# python3 scripts/auto-fix-agent.py auto-fix-candidates.json
|
||||
|
||||
echo "Fix-Vorschlaege wuerden hier generiert werden"
|
||||
depends_on:
|
||||
- analyze-errors
|
||||
|
||||
# ========================================
|
||||
# 4. Report Results
|
||||
# ========================================
|
||||
|
||||
report-results:
|
||||
image: curlimages/curl:latest
|
||||
commands:
|
||||
- |
|
||||
curl -X POST "http://backend:8000/api/tests/auto-fix/report" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"run_date\": \"$(date -Iseconds)\",
|
||||
\"candidates_found\": $(cat auto-fix-candidates.json | wc -l),
|
||||
\"fixes_attempted\": 0,
|
||||
\"fixes_successful\": 0,
|
||||
\"status\": \"analysis_only\"
|
||||
}" || true
|
||||
when:
|
||||
status: [success, failure]
|
||||
37
admin-v2/.woodpecker/build-ci-image.yml
Normal file
37
admin-v2/.woodpecker/build-ci-image.yml
Normal file
@@ -0,0 +1,37 @@
|
||||
# One-time pipeline to build the custom Python CI image
|
||||
# Trigger manually, then delete this file
|
||||
#
|
||||
# This builds the breakpilot/python-ci:3.12 image on the CI runner
|
||||
|
||||
when:
|
||||
- event: manual
|
||||
|
||||
clone:
|
||||
git:
|
||||
image: woodpeckerci/plugin-git
|
||||
settings:
|
||||
depth: 1
|
||||
extra_hosts:
|
||||
- macmini:192.168.178.100
|
||||
|
||||
steps:
|
||||
build-python-ci-image:
|
||||
image: docker:27-cli
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
commands:
|
||||
- |
|
||||
echo "=== Building breakpilot/python-ci:3.12 ==="
|
||||
|
||||
docker build \
|
||||
-t breakpilot/python-ci:3.12 \
|
||||
-t breakpilot/python-ci:latest \
|
||||
-f .docker/python-ci.Dockerfile \
|
||||
.
|
||||
|
||||
echo ""
|
||||
echo "=== Build complete ==="
|
||||
docker images | grep breakpilot/python-ci
|
||||
|
||||
echo ""
|
||||
echo "Image is now available for CI pipelines!"
|
||||
161
admin-v2/.woodpecker/integration.yml
Normal file
161
admin-v2/.woodpecker/integration.yml
Normal file
@@ -0,0 +1,161 @@
|
||||
# Integration Tests Pipeline
|
||||
# Separate Datei weil Services auf Pipeline-Ebene definiert werden muessen
|
||||
#
|
||||
# Diese Pipeline laeuft parallel zur main.yml und testet:
|
||||
# - Database Connectivity (PostgreSQL)
|
||||
# - Cache Connectivity (Valkey/Redis)
|
||||
# - Service-to-Service Kommunikation
|
||||
#
|
||||
# Dokumentation: docs/testing/integration-test-environment.md
|
||||
|
||||
when:
|
||||
- event: [push, pull_request]
|
||||
branch: [main, develop]
|
||||
|
||||
clone:
|
||||
git:
|
||||
image: woodpeckerci/plugin-git
|
||||
settings:
|
||||
depth: 1
|
||||
extra_hosts:
|
||||
- macmini:192.168.178.100
|
||||
|
||||
# Services auf Pipeline-Ebene (NICHT Step-Ebene!)
|
||||
# Diese Services sind fuer ALLE Steps verfuegbar
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_USER: breakpilot
|
||||
POSTGRES_PASSWORD: breakpilot_test
|
||||
POSTGRES_DB: breakpilot_test
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:8-alpine
|
||||
|
||||
steps:
|
||||
wait-for-services:
|
||||
image: postgres:16-alpine
|
||||
commands:
|
||||
- |
|
||||
echo "=== Waiting for PostgreSQL ==="
|
||||
for i in $(seq 1 30); do
|
||||
if pg_isready -h postgres -U breakpilot; then
|
||||
echo "PostgreSQL ready after $i attempts!"
|
||||
break
|
||||
fi
|
||||
echo "Attempt $i/30: PostgreSQL not ready, waiting..."
|
||||
sleep 2
|
||||
done
|
||||
# Final check
|
||||
if ! pg_isready -h postgres -U breakpilot; then
|
||||
echo "ERROR: PostgreSQL not ready after 30 attempts"
|
||||
exit 1
|
||||
fi
|
||||
- |
|
||||
echo "=== Waiting for Valkey ==="
|
||||
# Install redis-cli in postgres alpine image
|
||||
apk add --no-cache redis > /dev/null 2>&1 || true
|
||||
for i in $(seq 1 30); do
|
||||
if redis-cli -h valkey ping 2>/dev/null | grep -q PONG; then
|
||||
echo "Valkey ready after $i attempts!"
|
||||
break
|
||||
fi
|
||||
echo "Attempt $i/30: Valkey not ready, waiting..."
|
||||
sleep 2
|
||||
done
|
||||
# Final check
|
||||
if ! redis-cli -h valkey ping 2>/dev/null | grep -q PONG; then
|
||||
echo "ERROR: Valkey not ready after 30 attempts"
|
||||
exit 1
|
||||
fi
|
||||
- echo "=== All services ready ==="
|
||||
|
||||
integration-tests:
|
||||
image: breakpilot/python-ci:3.12
|
||||
environment:
|
||||
CI: "true"
|
||||
DATABASE_URL: postgresql://breakpilot:breakpilot_test@postgres:5432/breakpilot_test
|
||||
VALKEY_URL: redis://valkey:6379
|
||||
REDIS_URL: redis://valkey:6379
|
||||
SKIP_INTEGRATION_TESTS: "false"
|
||||
SKIP_DB_TESTS: "false"
|
||||
SKIP_WEASYPRINT_TESTS: "false"
|
||||
# Test-spezifische Umgebungsvariablen
|
||||
ENVIRONMENT: "testing"
|
||||
JWT_SECRET: "test-secret-key-for-integration-tests"
|
||||
TEACHER_REQUIRE_AUTH: "false"
|
||||
GAME_USE_DATABASE: "false"
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
cd backend
|
||||
|
||||
# PYTHONPATH setzen damit lokale Module gefunden werden
|
||||
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
||||
|
||||
echo "=== Installing dependencies ==="
|
||||
pip install --quiet --no-cache-dir -r requirements.txt
|
||||
|
||||
echo "=== Running Integration Tests ==="
|
||||
set +e
|
||||
python -m pytest tests/test_integration/ -v \
|
||||
--tb=short \
|
||||
--json-report \
|
||||
--json-report-file=../.ci-results/test-integration.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
# Ergebnisse auswerten
|
||||
if [ -f ../.ci-results/test-integration.json ]; then
|
||||
TOTAL=$(python3 -c "import json; d=json.load(open('../.ci-results/test-integration.json')); print(d.get('summary',{}).get('total',0))" 2>/dev/null || echo "0")
|
||||
PASSED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-integration.json')); print(d.get('summary',{}).get('passed',0))" 2>/dev/null || echo "0")
|
||||
FAILED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-integration.json')); print(d.get('summary',{}).get('failed',0))" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-integration.json')); print(d.get('summary',{}).get('skipped',0))" 2>/dev/null || echo "0")
|
||||
else
|
||||
echo "WARNUNG: Keine JSON-Ergebnisse gefunden"
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
echo "{\"service\":\"integration-tests\",\"framework\":\"pytest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../.ci-results/results-integration.json
|
||||
cat ../.ci-results/results-integration.json
|
||||
|
||||
echo ""
|
||||
echo "=== Integration Test Summary ==="
|
||||
echo "Total: $TOTAL | Passed: $PASSED | Failed: $FAILED | Skipped: $SKIPPED"
|
||||
|
||||
if [ "$TEST_EXIT" -ne "0" ]; then
|
||||
echo "Integration tests failed with exit code $TEST_EXIT"
|
||||
exit 1
|
||||
fi
|
||||
depends_on:
|
||||
- wait-for-services
|
||||
|
||||
report-integration-results:
|
||||
image: curlimages/curl:8.10.1
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
echo "=== Sende Integration Test-Ergebnisse an Dashboard ==="
|
||||
|
||||
if [ -f .ci-results/results-integration.json ]; then
|
||||
echo "Sending integration test results..."
|
||||
curl -f -sS -X POST "http://backend:8000/api/tests/ci-result" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"pipeline_id\": \"${CI_PIPELINE_NUMBER}\",
|
||||
\"commit\": \"${CI_COMMIT_SHA}\",
|
||||
\"branch\": \"${CI_COMMIT_BRANCH}\",
|
||||
\"status\": \"${CI_PIPELINE_STATUS:-unknown}\",
|
||||
\"test_results\": $(cat .ci-results/results-integration.json)
|
||||
}" || echo "WARNUNG: Konnte Ergebnisse nicht an Dashboard senden"
|
||||
else
|
||||
echo "Keine Integration-Ergebnisse zum Senden gefunden"
|
||||
fi
|
||||
|
||||
echo "=== Integration Test-Ergebnisse gesendet ==="
|
||||
when:
|
||||
status: [success, failure]
|
||||
depends_on:
|
||||
- integration-tests
|
||||
669
admin-v2/.woodpecker/main.yml
Normal file
669
admin-v2/.woodpecker/main.yml
Normal file
@@ -0,0 +1,669 @@
|
||||
# Woodpecker CI Main Pipeline
|
||||
# BreakPilot PWA - CI/CD Pipeline
|
||||
#
|
||||
# Plattform: ARM64 (Apple Silicon Mac Mini)
|
||||
#
|
||||
# Strategie:
|
||||
# - Tests laufen bei JEDEM Push/PR
|
||||
# - Test-Ergebnisse werden an Dashboard gesendet
|
||||
# - Builds/Scans laufen nur bei Tags oder manuell
|
||||
# - Deployment nur manuell (Sicherheit)
|
||||
|
||||
when:
|
||||
- event: [push, pull_request, manual, tag]
|
||||
branch: [main, develop]
|
||||
|
||||
clone:
|
||||
git:
|
||||
image: woodpeckerci/plugin-git
|
||||
settings:
|
||||
depth: 1
|
||||
extra_hosts:
|
||||
- macmini:192.168.178.100
|
||||
|
||||
variables:
|
||||
- &golang_image golang:1.23-alpine
|
||||
- &python_image python:3.12-slim
|
||||
- &python_ci_image breakpilot/python-ci:3.12 # Custom image with WeasyPrint
|
||||
- &nodejs_image node:20-alpine
|
||||
- &docker_image docker:27-cli
|
||||
|
||||
steps:
|
||||
# ========================================
|
||||
# STAGE 1: Lint (nur bei PRs)
|
||||
# ========================================
|
||||
|
||||
go-lint:
|
||||
image: golangci/golangci-lint:v1.55-alpine
|
||||
commands:
|
||||
- cd consent-service && golangci-lint run --timeout 5m ./...
|
||||
- cd ../billing-service && golangci-lint run --timeout 5m ./...
|
||||
- cd ../school-service && golangci-lint run --timeout 5m ./...
|
||||
when:
|
||||
event: pull_request
|
||||
|
||||
python-lint:
|
||||
image: *python_image
|
||||
commands:
|
||||
- pip install --quiet ruff black
|
||||
- ruff check backend/ --output-format=github || true
|
||||
- black --check backend/ || true
|
||||
when:
|
||||
event: pull_request
|
||||
|
||||
# ========================================
|
||||
# STAGE 2: Unit Tests mit JSON-Ausgabe
|
||||
# Ergebnisse werden im Workspace gespeichert (.ci-results/)
|
||||
# ========================================
|
||||
|
||||
test-go-consent:
|
||||
image: *golang_image
|
||||
environment:
|
||||
CGO_ENABLED: "0"
|
||||
commands:
|
||||
- |
|
||||
set -euo pipefail
|
||||
apk add --no-cache jq bash
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "consent-service" ]; then
|
||||
echo '{"service":"consent-service","framework":"go","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-consent.json
|
||||
echo "WARNUNG: consent-service Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd consent-service
|
||||
set +e
|
||||
go test -v -json -coverprofile=coverage.out ./... 2>&1 | tee ../.ci-results/test-consent.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
# JSON-Zeilen extrahieren und mit jq zählen
|
||||
JSON_FILE="../.ci-results/test-consent.json"
|
||||
if grep -q '^{' "$JSON_FILE" 2>/dev/null; then
|
||||
TOTAL=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="run" and .Test != null)] | length')
|
||||
PASSED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="pass" and .Test != null)] | length')
|
||||
FAILED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="fail" and .Test != null)] | length')
|
||||
SKIPPED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="skip" and .Test != null)] | length')
|
||||
else
|
||||
echo "WARNUNG: Keine JSON-Zeilen in $JSON_FILE gefunden (Build-Fehler?)"
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' | tr -d '%' || echo "0")
|
||||
[ -z "$COVERAGE" ] && COVERAGE=0
|
||||
|
||||
echo "{\"service\":\"consent-service\",\"framework\":\"go\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":$COVERAGE}" > ../.ci-results/results-consent.json
|
||||
cat ../.ci-results/results-consent.json
|
||||
|
||||
# Backlog-Strategie: Fehler werden gemeldet aber Pipeline laeuft weiter
|
||||
if [ "$FAILED" -gt "0" ]; then
|
||||
echo "WARNUNG: $FAILED Tests fehlgeschlagen - werden ins Backlog geschrieben"
|
||||
fi
|
||||
|
||||
test-go-billing:
|
||||
image: *golang_image
|
||||
environment:
|
||||
CGO_ENABLED: "0"
|
||||
commands:
|
||||
- |
|
||||
set -euo pipefail
|
||||
apk add --no-cache jq bash
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "billing-service" ]; then
|
||||
echo '{"service":"billing-service","framework":"go","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-billing.json
|
||||
echo "WARNUNG: billing-service Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd billing-service
|
||||
set +e
|
||||
go test -v -json -coverprofile=coverage.out ./... 2>&1 | tee ../.ci-results/test-billing.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
# JSON-Zeilen extrahieren und mit jq zählen
|
||||
JSON_FILE="../.ci-results/test-billing.json"
|
||||
if grep -q '^{' "$JSON_FILE" 2>/dev/null; then
|
||||
TOTAL=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="run" and .Test != null)] | length')
|
||||
PASSED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="pass" and .Test != null)] | length')
|
||||
FAILED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="fail" and .Test != null)] | length')
|
||||
SKIPPED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="skip" and .Test != null)] | length')
|
||||
else
|
||||
echo "WARNUNG: Keine JSON-Zeilen in $JSON_FILE gefunden (Build-Fehler?)"
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' | tr -d '%' || echo "0")
|
||||
[ -z "$COVERAGE" ] && COVERAGE=0
|
||||
|
||||
echo "{\"service\":\"billing-service\",\"framework\":\"go\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":$COVERAGE}" > ../.ci-results/results-billing.json
|
||||
cat ../.ci-results/results-billing.json
|
||||
|
||||
# Backlog-Strategie: Fehler werden gemeldet aber Pipeline laeuft weiter
|
||||
if [ "$FAILED" -gt "0" ]; then
|
||||
echo "WARNUNG: $FAILED Tests fehlgeschlagen - werden ins Backlog geschrieben"
|
||||
fi
|
||||
|
||||
test-go-school:
|
||||
image: *golang_image
|
||||
environment:
|
||||
CGO_ENABLED: "0"
|
||||
commands:
|
||||
- |
|
||||
set -euo pipefail
|
||||
apk add --no-cache jq bash
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "school-service" ]; then
|
||||
echo '{"service":"school-service","framework":"go","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-school.json
|
||||
echo "WARNUNG: school-service Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd school-service
|
||||
set +e
|
||||
go test -v -json -coverprofile=coverage.out ./... 2>&1 | tee ../.ci-results/test-school.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
# JSON-Zeilen extrahieren und mit jq zählen
|
||||
JSON_FILE="../.ci-results/test-school.json"
|
||||
if grep -q '^{' "$JSON_FILE" 2>/dev/null; then
|
||||
TOTAL=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="run" and .Test != null)] | length')
|
||||
PASSED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="pass" and .Test != null)] | length')
|
||||
FAILED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="fail" and .Test != null)] | length')
|
||||
SKIPPED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="skip" and .Test != null)] | length')
|
||||
else
|
||||
echo "WARNUNG: Keine JSON-Zeilen in $JSON_FILE gefunden (Build-Fehler?)"
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' | tr -d '%' || echo "0")
|
||||
[ -z "$COVERAGE" ] && COVERAGE=0
|
||||
|
||||
echo "{\"service\":\"school-service\",\"framework\":\"go\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":$COVERAGE}" > ../.ci-results/results-school.json
|
||||
cat ../.ci-results/results-school.json
|
||||
|
||||
# Backlog-Strategie: Fehler werden gemeldet aber Pipeline laeuft weiter
|
||||
if [ "$FAILED" -gt "0" ]; then
|
||||
echo "WARNUNG: $FAILED Tests fehlgeschlagen - werden ins Backlog geschrieben"
|
||||
fi
|
||||
|
||||
test-go-edu-search:
|
||||
image: *golang_image
|
||||
environment:
|
||||
CGO_ENABLED: "0"
|
||||
commands:
|
||||
- |
|
||||
set -euo pipefail
|
||||
apk add --no-cache jq bash
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "edu-search-service" ]; then
|
||||
echo '{"service":"edu-search-service","framework":"go","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-edu-search.json
|
||||
echo "WARNUNG: edu-search-service Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd edu-search-service
|
||||
set +e
|
||||
go test -v -json -coverprofile=coverage.out ./internal/... 2>&1 | tee ../.ci-results/test-edu-search.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
# JSON-Zeilen extrahieren und mit jq zählen
|
||||
JSON_FILE="../.ci-results/test-edu-search.json"
|
||||
if grep -q '^{' "$JSON_FILE" 2>/dev/null; then
|
||||
TOTAL=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="run" and .Test != null)] | length')
|
||||
PASSED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="pass" and .Test != null)] | length')
|
||||
FAILED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="fail" and .Test != null)] | length')
|
||||
SKIPPED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="skip" and .Test != null)] | length')
|
||||
else
|
||||
echo "WARNUNG: Keine JSON-Zeilen in $JSON_FILE gefunden (Build-Fehler?)"
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' | tr -d '%' || echo "0")
|
||||
[ -z "$COVERAGE" ] && COVERAGE=0
|
||||
|
||||
echo "{\"service\":\"edu-search-service\",\"framework\":\"go\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":$COVERAGE}" > ../.ci-results/results-edu-search.json
|
||||
cat ../.ci-results/results-edu-search.json
|
||||
|
||||
# Backlog-Strategie: Fehler werden gemeldet aber Pipeline laeuft weiter
|
||||
if [ "$FAILED" -gt "0" ]; then
|
||||
echo "WARNUNG: $FAILED Tests fehlgeschlagen - werden ins Backlog geschrieben"
|
||||
fi
|
||||
|
||||
test-go-ai-compliance:
|
||||
image: *golang_image
|
||||
environment:
|
||||
CGO_ENABLED: "0"
|
||||
commands:
|
||||
- |
|
||||
set -euo pipefail
|
||||
apk add --no-cache jq bash
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "ai-compliance-sdk" ]; then
|
||||
echo '{"service":"ai-compliance-sdk","framework":"go","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-ai-compliance.json
|
||||
echo "WARNUNG: ai-compliance-sdk Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd ai-compliance-sdk
|
||||
set +e
|
||||
go test -v -json -coverprofile=coverage.out ./... 2>&1 | tee ../.ci-results/test-ai-compliance.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
# JSON-Zeilen extrahieren und mit jq zählen
|
||||
JSON_FILE="../.ci-results/test-ai-compliance.json"
|
||||
if grep -q '^{' "$JSON_FILE" 2>/dev/null; then
|
||||
TOTAL=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="run" and .Test != null)] | length')
|
||||
PASSED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="pass" and .Test != null)] | length')
|
||||
FAILED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="fail" and .Test != null)] | length')
|
||||
SKIPPED=$(grep '^{' "$JSON_FILE" | jq -s '[.[] | select(.Action=="skip" and .Test != null)] | length')
|
||||
else
|
||||
echo "WARNUNG: Keine JSON-Zeilen in $JSON_FILE gefunden (Build-Fehler?)"
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' | tr -d '%' || echo "0")
|
||||
[ -z "$COVERAGE" ] && COVERAGE=0
|
||||
|
||||
echo "{\"service\":\"ai-compliance-sdk\",\"framework\":\"go\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":$COVERAGE}" > ../.ci-results/results-ai-compliance.json
|
||||
cat ../.ci-results/results-ai-compliance.json
|
||||
|
||||
# Backlog-Strategie: Fehler werden gemeldet aber Pipeline laeuft weiter
|
||||
if [ "$FAILED" -gt "0" ]; then
|
||||
echo "WARNUNG: $FAILED Tests fehlgeschlagen - werden ins Backlog geschrieben"
|
||||
fi
|
||||
|
||||
test-python-backend:
|
||||
image: *python_ci_image
|
||||
environment:
|
||||
CI: "true"
|
||||
DATABASE_URL: "postgresql://test:test@localhost:5432/test_db"
|
||||
SKIP_DB_TESTS: "true"
|
||||
SKIP_WEASYPRINT_TESTS: "false"
|
||||
SKIP_INTEGRATION_TESTS: "true"
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "backend" ]; then
|
||||
echo '{"service":"backend","framework":"pytest","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-backend.json
|
||||
echo "WARNUNG: backend Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd backend
|
||||
# Set PYTHONPATH to current directory (backend) so local packages like classroom_engine, alerts_agent are found
|
||||
# IMPORTANT: Use absolute path and export before pip install to ensure modules are available
|
||||
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
||||
|
||||
# Test tools are pre-installed in breakpilot/python-ci image
|
||||
# Only install project-specific dependencies
|
||||
pip install --quiet --no-cache-dir -r requirements.txt
|
||||
|
||||
# NOTE: PostgreSQL service removed - tests that require DB are skipped via SKIP_DB_TESTS=true
|
||||
# For full integration tests, use: docker compose -f docker-compose.test.yml up -d
|
||||
|
||||
set +e
|
||||
# Use python -m pytest to ensure PYTHONPATH is properly applied before pytest starts
|
||||
python -m pytest tests/ -v --tb=short --cov=. --cov-report=term-missing --json-report --json-report-file=../.ci-results/test-backend.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ -f ../.ci-results/test-backend.json ]; then
|
||||
TOTAL=$(python3 -c "import json; d=json.load(open('../.ci-results/test-backend.json')); print(d.get('summary',{}).get('total',0))" 2>/dev/null || echo "0")
|
||||
PASSED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-backend.json')); print(d.get('summary',{}).get('passed',0))" 2>/dev/null || echo "0")
|
||||
FAILED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-backend.json')); print(d.get('summary',{}).get('failed',0))" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-backend.json')); print(d.get('summary',{}).get('skipped',0))" 2>/dev/null || echo "0")
|
||||
else
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
echo "{\"service\":\"backend\",\"framework\":\"pytest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../.ci-results/results-backend.json
|
||||
cat ../.ci-results/results-backend.json
|
||||
|
||||
if [ "$TEST_EXIT" -ne "0" ]; then exit 1; fi
|
||||
|
||||
test-python-voice:
|
||||
image: *python_image
|
||||
environment:
|
||||
CI: "true"
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "voice-service" ]; then
|
||||
echo '{"service":"voice-service","framework":"pytest","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-voice.json
|
||||
echo "WARNUNG: voice-service Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd voice-service
|
||||
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
||||
pip install --quiet --no-cache-dir -r requirements.txt
|
||||
pip install --quiet --no-cache-dir pytest-json-report
|
||||
|
||||
set +e
|
||||
python -m pytest tests/ -v --tb=short --json-report --json-report-file=../.ci-results/test-voice.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ -f ../.ci-results/test-voice.json ]; then
|
||||
TOTAL=$(python3 -c "import json; d=json.load(open('../.ci-results/test-voice.json')); print(d.get('summary',{}).get('total',0))" 2>/dev/null || echo "0")
|
||||
PASSED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-voice.json')); print(d.get('summary',{}).get('passed',0))" 2>/dev/null || echo "0")
|
||||
FAILED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-voice.json')); print(d.get('summary',{}).get('failed',0))" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-voice.json')); print(d.get('summary',{}).get('skipped',0))" 2>/dev/null || echo "0")
|
||||
else
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
echo "{\"service\":\"voice-service\",\"framework\":\"pytest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../.ci-results/results-voice.json
|
||||
cat ../.ci-results/results-voice.json
|
||||
|
||||
if [ "$TEST_EXIT" -ne "0" ]; then exit 1; fi
|
||||
|
||||
test-bqas-golden:
|
||||
image: *python_image
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "voice-service/tests/bqas" ]; then
|
||||
echo '{"service":"bqas-golden","framework":"pytest","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-bqas-golden.json
|
||||
echo "WARNUNG: voice-service/tests/bqas Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd voice-service
|
||||
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
||||
pip install --quiet --no-cache-dir -r requirements.txt
|
||||
pip install --quiet --no-cache-dir pytest-json-report pytest-asyncio
|
||||
|
||||
set +e
|
||||
python -m pytest tests/bqas/test_golden.py tests/bqas/test_regression.py tests/bqas/test_synthetic.py -v --tb=short --json-report --json-report-file=../.ci-results/test-bqas-golden.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ -f ../.ci-results/test-bqas-golden.json ]; then
|
||||
TOTAL=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-golden.json')); print(d.get('summary',{}).get('total',0))" 2>/dev/null || echo "0")
|
||||
PASSED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-golden.json')); print(d.get('summary',{}).get('passed',0))" 2>/dev/null || echo "0")
|
||||
FAILED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-golden.json')); print(d.get('summary',{}).get('failed',0))" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-golden.json')); print(d.get('summary',{}).get('skipped',0))" 2>/dev/null || echo "0")
|
||||
else
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
echo "{\"service\":\"bqas-golden\",\"framework\":\"pytest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../.ci-results/results-bqas-golden.json
|
||||
cat ../.ci-results/results-bqas-golden.json
|
||||
|
||||
# BQAS tests may skip if Ollama not available - don't fail pipeline
|
||||
if [ "$FAILED" -gt "0" ]; then exit 1; fi
|
||||
|
||||
test-bqas-rag:
|
||||
image: *python_image
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "voice-service/tests/bqas" ]; then
|
||||
echo '{"service":"bqas-rag","framework":"pytest","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-bqas-rag.json
|
||||
echo "WARNUNG: voice-service/tests/bqas Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd voice-service
|
||||
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
||||
pip install --quiet --no-cache-dir -r requirements.txt
|
||||
pip install --quiet --no-cache-dir pytest-json-report pytest-asyncio
|
||||
|
||||
set +e
|
||||
python -m pytest tests/bqas/test_rag.py tests/bqas/test_notifier.py -v --tb=short --json-report --json-report-file=../.ci-results/test-bqas-rag.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ -f ../.ci-results/test-bqas-rag.json ]; then
|
||||
TOTAL=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-rag.json')); print(d.get('summary',{}).get('total',0))" 2>/dev/null || echo "0")
|
||||
PASSED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-rag.json')); print(d.get('summary',{}).get('passed',0))" 2>/dev/null || echo "0")
|
||||
FAILED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-rag.json')); print(d.get('summary',{}).get('failed',0))" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(python3 -c "import json; d=json.load(open('../.ci-results/test-bqas-rag.json')); print(d.get('summary',{}).get('skipped',0))" 2>/dev/null || echo "0")
|
||||
else
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
echo "{\"service\":\"bqas-rag\",\"framework\":\"pytest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../.ci-results/results-bqas-rag.json
|
||||
cat ../.ci-results/results-bqas-rag.json
|
||||
|
||||
# BQAS tests may skip if Ollama not available - don't fail pipeline
|
||||
if [ "$FAILED" -gt "0" ]; then exit 1; fi
|
||||
|
||||
test-python-klausur:
|
||||
image: *python_image
|
||||
environment:
|
||||
CI: "true"
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "klausur-service/backend" ]; then
|
||||
echo '{"service":"klausur-service","framework":"pytest","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-klausur.json
|
||||
echo "WARNUNG: klausur-service/backend Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd klausur-service/backend
|
||||
# Set PYTHONPATH to current directory so local modules like hyde, hybrid_search, etc. are found
|
||||
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
||||
|
||||
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || pip install --quiet --no-cache-dir fastapi uvicorn pytest pytest-asyncio pytest-json-report
|
||||
pip install --quiet --no-cache-dir pytest-json-report
|
||||
|
||||
set +e
|
||||
python -m pytest tests/ -v --tb=short --json-report --json-report-file=../../.ci-results/test-klausur.json
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ -f ../../.ci-results/test-klausur.json ]; then
|
||||
TOTAL=$(python3 -c "import json; d=json.load(open('../../.ci-results/test-klausur.json')); print(d.get('summary',{}).get('total',0))" 2>/dev/null || echo "0")
|
||||
PASSED=$(python3 -c "import json; d=json.load(open('../../.ci-results/test-klausur.json')); print(d.get('summary',{}).get('passed',0))" 2>/dev/null || echo "0")
|
||||
FAILED=$(python3 -c "import json; d=json.load(open('../../.ci-results/test-klausur.json')); print(d.get('summary',{}).get('failed',0))" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(python3 -c "import json; d=json.load(open('../../.ci-results/test-klausur.json')); print(d.get('summary',{}).get('skipped',0))" 2>/dev/null || echo "0")
|
||||
else
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
echo "{\"service\":\"klausur-service\",\"framework\":\"pytest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../../.ci-results/results-klausur.json
|
||||
cat ../../.ci-results/results-klausur.json
|
||||
|
||||
if [ "$TEST_EXIT" -ne "0" ]; then exit 1; fi
|
||||
|
||||
test-nodejs-h5p:
|
||||
image: *nodejs_image
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
mkdir -p .ci-results
|
||||
|
||||
if [ ! -d "h5p-service" ]; then
|
||||
echo '{"service":"h5p-service","framework":"jest","total":0,"passed":0,"failed":0,"skipped":0,"coverage":0}' > .ci-results/results-h5p.json
|
||||
echo "WARNUNG: h5p-service Verzeichnis nicht gefunden"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd h5p-service
|
||||
npm ci --silent 2>/dev/null || npm install --silent
|
||||
|
||||
set +e
|
||||
npm run test:ci -- --json --outputFile=../.ci-results/test-h5p.json 2>&1
|
||||
TEST_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ -f ../.ci-results/test-h5p.json ]; then
|
||||
TOTAL=$(node -e "const d=require('../.ci-results/test-h5p.json'); console.log(d.numTotalTests || 0)" 2>/dev/null || echo "0")
|
||||
PASSED=$(node -e "const d=require('../.ci-results/test-h5p.json'); console.log(d.numPassedTests || 0)" 2>/dev/null || echo "0")
|
||||
FAILED=$(node -e "const d=require('../.ci-results/test-h5p.json'); console.log(d.numFailedTests || 0)" 2>/dev/null || echo "0")
|
||||
SKIPPED=$(node -e "const d=require('../.ci-results/test-h5p.json'); console.log(d.numPendingTests || 0)" 2>/dev/null || echo "0")
|
||||
else
|
||||
TOTAL=0; PASSED=0; FAILED=0; SKIPPED=0
|
||||
fi
|
||||
|
||||
[ -z "$TOTAL" ] && TOTAL=0
|
||||
[ -z "$PASSED" ] && PASSED=0
|
||||
[ -z "$FAILED" ] && FAILED=0
|
||||
[ -z "$SKIPPED" ] && SKIPPED=0
|
||||
|
||||
echo "{\"service\":\"h5p-service\",\"framework\":\"jest\",\"total\":$TOTAL,\"passed\":$PASSED,\"failed\":$FAILED,\"skipped\":$SKIPPED,\"coverage\":0}" > ../.ci-results/results-h5p.json
|
||||
cat ../.ci-results/results-h5p.json
|
||||
|
||||
if [ "$TEST_EXIT" -ne "0" ]; then exit 1; fi
|
||||
|
||||
# ========================================
|
||||
# STAGE 2.5: Integration Tests
|
||||
# ========================================
|
||||
# Integration Tests laufen in separater Pipeline:
|
||||
# .woodpecker/integration.yml
|
||||
# (benötigt Pipeline-Level Services für PostgreSQL und Valkey)
|
||||
|
||||
# ========================================
|
||||
# STAGE 3: Test-Ergebnisse an Dashboard senden
|
||||
# ========================================
|
||||
|
||||
report-test-results:
|
||||
image: curlimages/curl:8.10.1
|
||||
commands:
|
||||
- |
|
||||
set -uo pipefail
|
||||
echo "=== Sende Test-Ergebnisse an Dashboard ==="
|
||||
echo "Pipeline Status: ${CI_PIPELINE_STATUS:-unknown}"
|
||||
ls -la .ci-results/ || echo "Verzeichnis nicht gefunden"
|
||||
|
||||
PIPELINE_STATUS="${CI_PIPELINE_STATUS:-unknown}"
|
||||
|
||||
for f in .ci-results/results-*.json; do
|
||||
[ -f "$f" ] || continue
|
||||
echo "Sending: $f"
|
||||
curl -f -sS -X POST "http://backend:8000/api/tests/ci-result" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"pipeline_id\": \"${CI_PIPELINE_NUMBER}\",
|
||||
\"commit\": \"${CI_COMMIT_SHA}\",
|
||||
\"branch\": \"${CI_COMMIT_BRANCH}\",
|
||||
\"status\": \"${PIPELINE_STATUS}\",
|
||||
\"test_results\": $(cat "$f")
|
||||
}" || echo "WARNUNG: Konnte $f nicht senden"
|
||||
done
|
||||
|
||||
echo "=== Test-Ergebnisse gesendet ==="
|
||||
when:
|
||||
status: [success, failure]
|
||||
depends_on:
|
||||
- test-go-consent
|
||||
- test-go-billing
|
||||
- test-go-school
|
||||
- test-go-edu-search
|
||||
- test-go-ai-compliance
|
||||
- test-python-backend
|
||||
- test-python-voice
|
||||
- test-bqas-golden
|
||||
- test-bqas-rag
|
||||
- test-python-klausur
|
||||
- test-nodejs-h5p
|
||||
|
||||
# ========================================
|
||||
# STAGE 4: Build & Security (nur Tags/manuell)
|
||||
# ========================================
|
||||
|
||||
build-consent-service:
|
||||
image: *docker_image
|
||||
commands:
|
||||
- docker build -t breakpilot/consent-service:${CI_COMMIT_SHA:0:8} ./consent-service
|
||||
- docker tag breakpilot/consent-service:${CI_COMMIT_SHA:0:8} breakpilot/consent-service:latest
|
||||
- echo "Built breakpilot/consent-service:${CI_COMMIT_SHA:0:8}"
|
||||
when:
|
||||
- event: tag
|
||||
- event: manual
|
||||
|
||||
build-backend:
|
||||
image: *docker_image
|
||||
commands:
|
||||
- docker build -t breakpilot/backend:${CI_COMMIT_SHA:0:8} ./backend
|
||||
- docker tag breakpilot/backend:${CI_COMMIT_SHA:0:8} breakpilot/backend:latest
|
||||
- echo "Built breakpilot/backend:${CI_COMMIT_SHA:0:8}"
|
||||
when:
|
||||
- event: tag
|
||||
- event: manual
|
||||
|
||||
build-voice-service:
|
||||
image: *docker_image
|
||||
commands:
|
||||
- |
|
||||
if [ -d ./voice-service ]; then
|
||||
docker build -t breakpilot/voice-service:${CI_COMMIT_SHA:0:8} ./voice-service
|
||||
docker tag breakpilot/voice-service:${CI_COMMIT_SHA:0:8} breakpilot/voice-service:latest
|
||||
echo "Built breakpilot/voice-service:${CI_COMMIT_SHA:0:8}"
|
||||
else
|
||||
echo "voice-service Verzeichnis nicht gefunden - ueberspringe"
|
||||
fi
|
||||
when:
|
||||
- event: tag
|
||||
- event: manual
|
||||
|
||||
generate-sbom:
|
||||
image: *golang_image
|
||||
commands:
|
||||
- |
|
||||
echo "Installing syft for ARM64..."
|
||||
wget -qO- https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
|
||||
syft dir:./consent-service -o cyclonedx-json > sbom-consent.json
|
||||
syft dir:./backend -o cyclonedx-json > sbom-backend.json
|
||||
if [ -d ./voice-service ]; then
|
||||
syft dir:./voice-service -o cyclonedx-json > sbom-voice.json
|
||||
fi
|
||||
echo "SBOMs generated successfully"
|
||||
when:
|
||||
- event: tag
|
||||
- event: manual
|
||||
|
||||
vulnerability-scan:
|
||||
image: *golang_image
|
||||
commands:
|
||||
- |
|
||||
echo "Installing grype for ARM64..."
|
||||
wget -qO- https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
|
||||
grype sbom:sbom-consent.json -o table --fail-on critical || true
|
||||
grype sbom:sbom-backend.json -o table --fail-on critical || true
|
||||
if [ -f sbom-voice.json ]; then
|
||||
grype sbom:sbom-voice.json -o table --fail-on critical || true
|
||||
fi
|
||||
when:
|
||||
- event: tag
|
||||
- event: manual
|
||||
depends_on:
|
||||
- generate-sbom
|
||||
|
||||
# ========================================
|
||||
# STAGE 5: Deploy (nur manuell)
|
||||
# ========================================
|
||||
|
||||
deploy-production:
|
||||
image: *docker_image
|
||||
commands:
|
||||
- echo "Deploying to production..."
|
||||
- docker compose -f docker-compose.yml pull || true
|
||||
- docker compose -f docker-compose.yml up -d --remove-orphans || true
|
||||
when:
|
||||
event: manual
|
||||
depends_on:
|
||||
- build-consent-service
|
||||
- build-backend
|
||||
314
admin-v2/.woodpecker/security.yml
Normal file
314
admin-v2/.woodpecker/security.yml
Normal file
@@ -0,0 +1,314 @@
|
||||
# Woodpecker CI Security Pipeline
|
||||
# Dedizierte Security-Scans fuer DevSecOps
|
||||
#
|
||||
# Laeuft taeglich via Cron und bei jedem PR
|
||||
|
||||
when:
|
||||
- event: cron
|
||||
cron: "0 3 * * *" # Taeglich um 3:00 Uhr
|
||||
- event: pull_request
|
||||
|
||||
clone:
|
||||
git:
|
||||
image: woodpeckerci/plugin-git
|
||||
settings:
|
||||
depth: 1
|
||||
extra_hosts:
|
||||
- macmini:192.168.178.100
|
||||
|
||||
steps:
|
||||
# ========================================
|
||||
# Static Analysis
|
||||
# ========================================
|
||||
|
||||
semgrep-scan:
|
||||
image: returntocorp/semgrep:latest
|
||||
commands:
|
||||
- semgrep scan --config auto --json -o semgrep-results.json . || true
|
||||
- |
|
||||
if [ -f semgrep-results.json ]; then
|
||||
echo "=== Semgrep Findings ==="
|
||||
cat semgrep-results.json | head -100
|
||||
fi
|
||||
when:
|
||||
event: [pull_request, cron]
|
||||
|
||||
bandit-python:
|
||||
image: python:3.12-slim
|
||||
commands:
|
||||
- pip install --quiet bandit
|
||||
- bandit -r backend/ -f json -o bandit-results.json || true
|
||||
- |
|
||||
if [ -f bandit-results.json ]; then
|
||||
echo "=== Bandit Findings ==="
|
||||
cat bandit-results.json | head -50
|
||||
fi
|
||||
when:
|
||||
event: [pull_request, cron]
|
||||
|
||||
gosec-go:
|
||||
image: securego/gosec:latest
|
||||
commands:
|
||||
- gosec -fmt json -out gosec-consent.json ./consent-service/... || true
|
||||
- gosec -fmt json -out gosec-billing.json ./billing-service/... || true
|
||||
- echo "Go Security Scan abgeschlossen"
|
||||
when:
|
||||
event: [pull_request, cron]
|
||||
|
||||
# ========================================
|
||||
# Secrets Detection
|
||||
# ========================================
|
||||
|
||||
gitleaks-scan:
|
||||
image: zricethezav/gitleaks:latest
|
||||
commands:
|
||||
- gitleaks detect --source . --report-format json --report-path gitleaks-report.json || true
|
||||
- |
|
||||
if [ -s gitleaks-report.json ]; then
|
||||
echo "=== WARNUNG: Potentielle Secrets gefunden ==="
|
||||
cat gitleaks-report.json
|
||||
else
|
||||
echo "Keine Secrets gefunden"
|
||||
fi
|
||||
|
||||
trufflehog-scan:
|
||||
image: trufflesecurity/trufflehog:latest
|
||||
commands:
|
||||
- trufflehog filesystem . --json > trufflehog-results.json 2>&1 || true
|
||||
- echo "TruffleHog Scan abgeschlossen"
|
||||
|
||||
# ========================================
|
||||
# Dependency Vulnerabilities
|
||||
# ========================================
|
||||
|
||||
npm-audit:
|
||||
image: node:20-alpine
|
||||
commands:
|
||||
- cd website && npm audit --json > ../npm-audit-website.json || true
|
||||
- cd ../studio-v2 && npm audit --json > ../npm-audit-studio.json || true
|
||||
- cd ../admin-v2 && npm audit --json > ../npm-audit-admin.json || true
|
||||
- echo "NPM Audit abgeschlossen"
|
||||
when:
|
||||
event: [pull_request, cron]
|
||||
|
||||
pip-audit:
|
||||
image: python:3.12-slim
|
||||
commands:
|
||||
- pip install --quiet pip-audit
|
||||
- pip-audit -r backend/requirements.txt --format json -o pip-audit-backend.json || true
|
||||
- pip-audit -r voice-service/requirements.txt --format json -o pip-audit-voice.json || true
|
||||
- echo "Pip Audit abgeschlossen"
|
||||
when:
|
||||
event: [pull_request, cron]
|
||||
|
||||
go-vulncheck:
|
||||
image: golang:1.21-alpine
|
||||
commands:
|
||||
- go install golang.org/x/vuln/cmd/govulncheck@latest
|
||||
- cd consent-service && govulncheck ./... || true
|
||||
- cd ../billing-service && govulncheck ./... || true
|
||||
- echo "Go Vulncheck abgeschlossen"
|
||||
when:
|
||||
event: [pull_request, cron]
|
||||
|
||||
# ========================================
|
||||
# Container Security
|
||||
# ========================================
|
||||
|
||||
trivy-filesystem:
|
||||
image: aquasec/trivy:latest
|
||||
commands:
|
||||
- trivy fs --severity HIGH,CRITICAL --format json -o trivy-fs.json . || true
|
||||
- echo "Trivy Filesystem Scan abgeschlossen"
|
||||
when:
|
||||
event: cron
|
||||
|
||||
# ========================================
|
||||
# SBOM Generation (taeglich)
|
||||
# ========================================
|
||||
|
||||
daily-sbom:
|
||||
image: anchore/syft:latest
|
||||
commands:
|
||||
- mkdir -p sbom-reports
|
||||
- syft dir:. -o cyclonedx-json > sbom-reports/sbom-full-$(date +%Y%m%d).json
|
||||
- echo "SBOM generiert"
|
||||
when:
|
||||
event: cron
|
||||
|
||||
# ========================================
|
||||
# AUTO-FIX: Dependency Vulnerabilities
|
||||
# Laeuft nur bei Cron (nightly), nicht bei PRs
|
||||
# ========================================
|
||||
|
||||
auto-fix-npm:
|
||||
image: node:20-alpine
|
||||
commands:
|
||||
- apk add --no-cache git
|
||||
- |
|
||||
echo "=== Auto-Fix: NPM Dependencies ==="
|
||||
FIXES_APPLIED=0
|
||||
|
||||
for dir in website studio-v2 admin-v2 h5p-service; do
|
||||
if [ -d "$dir" ] && [ -f "$dir/package.json" ]; then
|
||||
echo "Pruefe $dir..."
|
||||
cd $dir
|
||||
|
||||
# Speichere Hash vor Fix
|
||||
BEFORE=$(md5sum package-lock.json 2>/dev/null || echo "none")
|
||||
|
||||
# npm audit fix (ohne --force fuer sichere Updates)
|
||||
npm audit fix --package-lock-only 2>/dev/null || true
|
||||
|
||||
# Pruefe ob Aenderungen
|
||||
AFTER=$(md5sum package-lock.json 2>/dev/null || echo "none")
|
||||
if [ "$BEFORE" != "$AFTER" ]; then
|
||||
echo " -> Fixes angewendet in $dir"
|
||||
FIXES_APPLIED=$((FIXES_APPLIED + 1))
|
||||
fi
|
||||
|
||||
cd ..
|
||||
fi
|
||||
done
|
||||
|
||||
echo "NPM Auto-Fix abgeschlossen: $FIXES_APPLIED Projekte aktualisiert"
|
||||
echo "NPM_FIXES=$FIXES_APPLIED" >> /tmp/autofix-results.env
|
||||
when:
|
||||
event: cron
|
||||
|
||||
auto-fix-python:
|
||||
image: python:3.12-slim
|
||||
commands:
|
||||
- apt-get update && apt-get install -y git
|
||||
- pip install --quiet pip-audit
|
||||
- |
|
||||
echo "=== Auto-Fix: Python Dependencies ==="
|
||||
FIXES_APPLIED=0
|
||||
|
||||
for reqfile in backend/requirements.txt voice-service/requirements.txt klausur-service/backend/requirements.txt; do
|
||||
if [ -f "$reqfile" ]; then
|
||||
echo "Pruefe $reqfile..."
|
||||
DIR=$(dirname $reqfile)
|
||||
|
||||
# pip-audit mit --fix (aktualisiert requirements.txt)
|
||||
pip-audit -r $reqfile --fix 2>/dev/null || true
|
||||
|
||||
# Pruefe ob requirements.txt geaendert wurde
|
||||
if git diff --quiet $reqfile 2>/dev/null; then
|
||||
echo " -> Keine Aenderungen in $reqfile"
|
||||
else
|
||||
echo " -> Fixes angewendet in $reqfile"
|
||||
FIXES_APPLIED=$((FIXES_APPLIED + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Python Auto-Fix abgeschlossen: $FIXES_APPLIED Dateien aktualisiert"
|
||||
echo "PYTHON_FIXES=$FIXES_APPLIED" >> /tmp/autofix-results.env
|
||||
when:
|
||||
event: cron
|
||||
|
||||
auto-fix-go:
|
||||
image: golang:1.21-alpine
|
||||
commands:
|
||||
- apk add --no-cache git
|
||||
- |
|
||||
echo "=== Auto-Fix: Go Dependencies ==="
|
||||
FIXES_APPLIED=0
|
||||
|
||||
for dir in consent-service billing-service school-service edu-search ai-compliance-sdk; do
|
||||
if [ -d "$dir" ] && [ -f "$dir/go.mod" ]; then
|
||||
echo "Pruefe $dir..."
|
||||
cd $dir
|
||||
|
||||
# Go mod tidy und update
|
||||
go get -u ./... 2>/dev/null || true
|
||||
go mod tidy 2>/dev/null || true
|
||||
|
||||
# Pruefe ob go.mod/go.sum geaendert wurden
|
||||
if git diff --quiet go.mod go.sum 2>/dev/null; then
|
||||
echo " -> Keine Aenderungen in $dir"
|
||||
else
|
||||
echo " -> Updates angewendet in $dir"
|
||||
FIXES_APPLIED=$((FIXES_APPLIED + 1))
|
||||
fi
|
||||
|
||||
cd ..
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Go Auto-Fix abgeschlossen: $FIXES_APPLIED Module aktualisiert"
|
||||
echo "GO_FIXES=$FIXES_APPLIED" >> /tmp/autofix-results.env
|
||||
when:
|
||||
event: cron
|
||||
|
||||
# ========================================
|
||||
# Commit & Push Auto-Fixes
|
||||
# ========================================
|
||||
|
||||
commit-security-fixes:
|
||||
image: alpine/git:latest
|
||||
commands:
|
||||
- |
|
||||
echo "=== Commit Security Fixes ==="
|
||||
|
||||
# Git konfigurieren
|
||||
git config --global user.email "security-bot@breakpilot.de"
|
||||
git config --global user.name "Security Bot"
|
||||
git config --global --add safe.directory /woodpecker/src
|
||||
|
||||
# Pruefe ob es Aenderungen gibt
|
||||
if git diff --quiet && git diff --cached --quiet; then
|
||||
echo "Keine Security-Fixes zum Committen"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Zeige was geaendert wurde
|
||||
echo "Geaenderte Dateien:"
|
||||
git status --short
|
||||
|
||||
# Stage alle relevanten Dateien
|
||||
git add -A \
|
||||
*/package-lock.json \
|
||||
*/requirements.txt \
|
||||
*/go.mod \
|
||||
*/go.sum \
|
||||
2>/dev/null || true
|
||||
|
||||
# Commit erstellen
|
||||
TIMESTAMP=$(date +%Y-%m-%d)
|
||||
git commit -m "fix(security): auto-fix vulnerable dependencies [$TIMESTAMP]
|
||||
|
||||
Automatische Sicherheitsupdates durch CI/CD Pipeline:
|
||||
- npm audit fix fuer Node.js Projekte
|
||||
- pip-audit --fix fuer Python Projekte
|
||||
- go get -u fuer Go Module
|
||||
|
||||
Co-Authored-By: Security Bot <security-bot@breakpilot.de>" || echo "Nichts zu committen"
|
||||
|
||||
# Push zum Repository
|
||||
git push origin HEAD:main || echo "Push fehlgeschlagen - manueller Review erforderlich"
|
||||
|
||||
echo "Security-Fixes committed und gepusht"
|
||||
when:
|
||||
event: cron
|
||||
status: success
|
||||
|
||||
# ========================================
|
||||
# Report to Dashboard
|
||||
# ========================================
|
||||
|
||||
update-security-dashboard:
|
||||
image: curlimages/curl:latest
|
||||
commands:
|
||||
- |
|
||||
curl -X POST "http://backend:8000/api/security/scan-results" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"scan_type\": \"daily\",
|
||||
\"timestamp\": \"$(date -Iseconds)\",
|
||||
\"tools\": [\"semgrep\", \"bandit\", \"gosec\", \"gitleaks\", \"trivy\"]
|
||||
}" || true
|
||||
when:
|
||||
status: [success, failure]
|
||||
event: cron
|
||||
Reference in New Issue
Block a user