Phase 1 Step 4 follow-up addressing the debt flagged in the worked-example
commit (4a91814).
## mypy --strict policy
Adds backend-compliance/mypy.ini declaring the strict-mode scope:
Fully strict (enforced today):
- compliance/domain/
- compliance/schemas/
- compliance/api/_http_errors.py
- compliance/api/audit_routes.py (refactored in Step 4)
- compliance/services/audit_session_service.py
- compliance/services/audit_signoff_service.py
Loose (ignore_errors=True) with a migration path:
- compliance/db/* — SQLAlchemy 1.x Column[] vs
runtime T; unblocks Phase 1
until a Mapped[T] migration.
- compliance/api/<route>.py — each route file flips to
strict as its own Step 4
refactor lands.
- compliance/services/<legacy util> — 14 utility services
(llm_provider, pdf_extractor,
seeder, ...) that predate the
clean-arch refactor.
- compliance/tests/ — excluded (legacy placeholder
style). The new TestClient-
based integration suite is
type-annotated.
The two new service files carry a scoped `# mypy: disable-error-code="arg-type,assignment"`
header for the ORM Column[T] issue — same underlying SQLAlchemy limitation,
narrowly scoped rather than wholesale ignore_errors.
Flow: `cd backend-compliance && mypy compliance/` -> clean on 119 files.
CI yaml updated to use the config instead of ad-hoc package lists.
## Bugs fixed while enabling strict
mypy --strict surfaced two latent bugs in the pre-refactor code. Both
were invisible because the old `compliance/tests/test_audit_routes.py`
is a placeholder suite that asserts on request-data shape and never
calls the handlers:
- AuditSessionResponse.updated_at is a required field in the schema,
but the original handler didn't pass it. Fixed in
AuditSessionService._to_response.
- PaginationMeta requires has_next + has_prev. The original audit
checklist handler didn't compute them. Fixed in
AuditSignOffService.get_checklist.
Both are behavior-preserving at the HTTP level because the old code
would have raised Pydantic ValidationError at response serialization
had the endpoint actually been exercised.
## Integration test suite
Adds backend-compliance/tests/test_audit_routes_integration.py — 26
real TestClient tests against an in-memory sqlite backend (StaticPool).
Replaces the coverage gap left by the placeholder suite.
Covers:
- Session CRUD + lifecycle transitions (draft -> in_progress -> completed
-> archived), including the 409 paths for illegal transitions
- Checklist pagination, filtering, search
- Sign-off create / update / auto-start-session / count-flipping
- Sign-off 400 (invalid result), 404 (missing requirement), 409 (completed session)
- Get-signoff 404 / 200 round-trip
Uses a module-scoped schema fixture + per-test DELETE-sweep so the
suite runs in ~2.3s despite the ~50-table ORM surface.
Verified:
- 199/199 pytest (173 original + 26 new audit integration) pass
- tests/contracts/test_openapi_baseline.py green, OpenAPI 360/484 unchanged
- mypy compliance/ -> Success: no issues found in 119 source files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
302 lines
11 KiB
YAML
302 lines
11 KiB
YAML
# Gitea Actions CI/CD Pipeline
|
|
# BreakPilot Compliance
|
|
#
|
|
# Services:
|
|
# Go: ai-compliance-sdk
|
|
# Python: backend-compliance, document-crawler, dsms-gateway
|
|
# Node.js: admin-compliance, developer-portal
|
|
#
|
|
# Workflow:
|
|
# Push auf main → Tests → Deploy (Coolify)
|
|
# Pull Request → Lint + Tests (kein Deploy)
|
|
|
|
name: CI/CD
|
|
|
|
on:
|
|
push:
|
|
branches: [main, develop]
|
|
pull_request:
|
|
branches: [main, develop]
|
|
|
|
jobs:
|
|
# ========================================
|
|
# Guardrails — LOC budget + architecture gates
|
|
# Runs on every push/PR. Fails fast and cheap.
|
|
# ========================================
|
|
|
|
loc-budget:
|
|
runs-on: docker
|
|
container: alpine:3.20
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apk add --no-cache git bash
|
|
git clone --depth 50 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Enforce 500-line hard cap on changed files
|
|
run: |
|
|
chmod +x scripts/check-loc.sh
|
|
if [ "${GITHUB_EVENT_NAME}" = "pull_request" ]; then
|
|
git fetch origin ${GITHUB_BASE_REF}:base
|
|
mapfile -t changed < <(git diff --name-only --diff-filter=ACM base...HEAD)
|
|
[ ${#changed[@]} -eq 0 ] && { echo "No changed files."; exit 0; }
|
|
scripts/check-loc.sh "${changed[@]}"
|
|
else
|
|
# Push to main: only warn on whole-repo state; blocking gate is on PRs.
|
|
scripts/check-loc.sh || true
|
|
fi
|
|
# Phase 0 intentionally gates only changed files so the 205-file legacy
|
|
# baseline doesn't block every PR. Phases 1-4 drain the baseline; Phase 5
|
|
# flips this to a whole-repo blocking gate.
|
|
|
|
guardrail-integrity:
|
|
runs-on: docker
|
|
container: alpine:3.20
|
|
if: github.event_name == 'pull_request'
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apk add --no-cache git bash
|
|
git clone --depth 20 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
git fetch origin ${GITHUB_BASE_REF}:base
|
|
- name: Require [guardrail-change] label in PR commits touching guardrails
|
|
run: |
|
|
changed=$(git diff --name-only base...HEAD)
|
|
echo "$changed" | grep -E '^(\.claude/settings\.json|\.claude/rules/loc-exceptions\.txt|scripts/check-loc\.sh|scripts/githooks/pre-commit|AGENTS\.(python|go|typescript)\.md)$' || exit 0
|
|
if ! git log base..HEAD --format=%B | grep -q '\[guardrail-change\]'; then
|
|
echo "::error:: Guardrail files were modified but no commit in this PR carries [guardrail-change]."
|
|
echo "If intentional, amend one commit message with [guardrail-change] and explain why in the body."
|
|
exit 1
|
|
fi
|
|
|
|
# ========================================
|
|
# Lint (nur bei PRs)
|
|
# ========================================
|
|
|
|
go-lint:
|
|
runs-on: docker
|
|
if: github.event_name == 'pull_request'
|
|
container: golangci/golangci-lint:v1.62-alpine
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apk add --no-cache git
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Lint ai-compliance-sdk
|
|
run: |
|
|
if [ -d "ai-compliance-sdk" ]; then
|
|
cd ai-compliance-sdk && golangci-lint run --timeout 5m ./...
|
|
fi
|
|
|
|
python-lint:
|
|
runs-on: docker
|
|
if: github.event_name == 'pull_request'
|
|
container: python:3.12-slim
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Lint Python services (ruff)
|
|
run: |
|
|
pip install --quiet ruff
|
|
fail=0
|
|
for svc in backend-compliance document-crawler dsms-gateway compliance-tts-service; do
|
|
if [ -d "$svc" ]; then
|
|
echo "=== ruff: $svc ==="
|
|
ruff check "$svc/" --output-format=github || fail=1
|
|
fi
|
|
done
|
|
exit $fail
|
|
- name: Type-check (mypy via backend-compliance/mypy.ini)
|
|
# Policy is declared in backend-compliance/mypy.ini: strict mode globally,
|
|
# with per-module overrides for legacy utility services, the SQLAlchemy
|
|
# ORM layer, and yet-unrefactored route files. Each Phase 1 Step 4
|
|
# refactor flips a route file from loose->strict via its own mypy.ini
|
|
# override block.
|
|
run: |
|
|
pip install --quiet mypy
|
|
if [ -f "backend-compliance/mypy.ini" ]; then
|
|
cd backend-compliance && mypy compliance/
|
|
fi
|
|
|
|
nodejs-lint:
|
|
runs-on: docker
|
|
if: github.event_name == 'pull_request'
|
|
container: node:20-alpine
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apk add --no-cache git
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Lint + type-check Node.js services
|
|
run: |
|
|
fail=0
|
|
for svc in admin-compliance developer-portal; do
|
|
if [ -d "$svc" ]; then
|
|
echo "=== $svc: install ==="
|
|
(cd "$svc" && (npm ci --silent 2>/dev/null || npm install --silent))
|
|
echo "=== $svc: next lint ==="
|
|
(cd "$svc" && npx next lint) || fail=1
|
|
echo "=== $svc: tsc --noEmit ==="
|
|
(cd "$svc" && npx tsc --noEmit) || fail=1
|
|
fi
|
|
done
|
|
exit $fail
|
|
|
|
# ========================================
|
|
# Unit Tests
|
|
# ========================================
|
|
|
|
test-go-ai-compliance:
|
|
runs-on: docker
|
|
container: golang:1.24-alpine
|
|
env:
|
|
CGO_ENABLED: "0"
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apk add --no-cache git
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Test ai-compliance-sdk
|
|
run: |
|
|
if [ ! -d "ai-compliance-sdk" ]; then
|
|
echo "WARNUNG: ai-compliance-sdk nicht gefunden"
|
|
exit 0
|
|
fi
|
|
cd ai-compliance-sdk
|
|
go test -v -coverprofile=coverage.out ./... 2>&1
|
|
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' || echo "0%")
|
|
echo "Coverage: $COVERAGE"
|
|
|
|
test-python-backend-compliance:
|
|
runs-on: docker
|
|
container: python:3.12-slim
|
|
env:
|
|
CI: "true"
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Test backend-compliance
|
|
run: |
|
|
if [ ! -d "backend-compliance" ]; then
|
|
echo "WARNUNG: backend-compliance nicht gefunden"
|
|
exit 0
|
|
fi
|
|
cd backend-compliance
|
|
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
|
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
|
|
pip install --quiet --no-cache-dir fastapi uvicorn pytest pytest-asyncio
|
|
python -m pytest compliance/tests/ -v --tb=short
|
|
|
|
test-python-document-crawler:
|
|
runs-on: docker
|
|
container: python:3.12-slim
|
|
env:
|
|
CI: "true"
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Test document-crawler
|
|
run: |
|
|
if [ ! -d "document-crawler" ]; then
|
|
echo "WARNUNG: document-crawler nicht gefunden"
|
|
exit 0
|
|
fi
|
|
cd document-crawler
|
|
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
|
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
|
|
pip install --quiet --no-cache-dir pytest pytest-asyncio
|
|
python -m pytest tests/ -v --tb=short
|
|
|
|
test-python-dsms-gateway:
|
|
runs-on: docker
|
|
container: python:3.12-slim
|
|
env:
|
|
CI: "true"
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Test dsms-gateway
|
|
run: |
|
|
if [ ! -d "dsms-gateway" ]; then
|
|
echo "WARNUNG: dsms-gateway nicht gefunden"
|
|
exit 0
|
|
fi
|
|
cd dsms-gateway
|
|
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
|
|
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
|
|
pip install --quiet --no-cache-dir pytest pytest-asyncio
|
|
python -m pytest test_main.py -v --tb=short
|
|
|
|
# ========================================
|
|
# SBOM + license scan (compliance product → we eat our own dog food)
|
|
# ========================================
|
|
|
|
sbom-scan:
|
|
runs-on: docker
|
|
if: github.event_name == 'pull_request'
|
|
container: alpine:3.20
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apk add --no-cache git curl bash
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Install syft + grype
|
|
run: |
|
|
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
|
|
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
|
|
- name: Generate SBOM
|
|
run: |
|
|
mkdir -p sbom-out
|
|
syft dir:. -o cyclonedx-json=sbom-out/sbom.cdx.json -q
|
|
- name: Vulnerability scan (fail on high+)
|
|
run: |
|
|
grype sbom:sbom-out/sbom.cdx.json --fail-on high -q || true
|
|
# Initially non-blocking ('|| true'). Flip to blocking after baseline is clean.
|
|
|
|
# ========================================
|
|
# Validate Canonical Controls
|
|
# ========================================
|
|
|
|
validate-canonical-controls:
|
|
runs-on: docker
|
|
container: python:3.12-slim
|
|
steps:
|
|
- name: Checkout
|
|
run: |
|
|
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
|
|
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
|
- name: Validate controls
|
|
run: |
|
|
python scripts/validate-controls.py
|
|
|
|
# ========================================
|
|
# Deploy via Coolify (nur main, kein PR)
|
|
# ========================================
|
|
|
|
deploy-coolify:
|
|
name: Deploy
|
|
runs-on: docker
|
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
needs:
|
|
- loc-budget
|
|
- test-go-ai-compliance
|
|
- test-python-backend-compliance
|
|
- test-python-document-crawler
|
|
- test-python-dsms-gateway
|
|
- validate-canonical-controls
|
|
container:
|
|
image: alpine:latest
|
|
steps:
|
|
- name: Trigger Coolify deploy
|
|
run: |
|
|
apk add --no-cache curl
|
|
curl -sf "${{ secrets.COOLIFY_WEBHOOK }}" \
|
|
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
|