ci: optimize pipeline for feature branch workflow
Some checks failed
CI / branch-name (push) Has been skipped
CI / guardrail-integrity (push) Has been skipped
CI / loc-budget (push) Successful in 25s
CI / secret-scan (push) Has been skipped
CI / python-lint (push) Has been skipped
CI / go-lint (push) Has been skipped
CI / nodejs-lint (push) Has been skipped
CI / nodejs-build (push) Successful in 2m51s
CI / dep-audit (push) Has been skipped
CI / sbom-scan (push) Has been skipped
CI / test-go (push) Successful in 47s
CI / test-python-backend (push) Failing after 43s
CI / test-python-document-crawler (push) Successful in 30s
CI / test-python-dsms-gateway (push) Successful in 22s
CI / validate-canonical-controls (push) Successful in 22s

Trigger changes:
- Remove dead 'develop' branch trigger
- PR gate runs full suite; push-to-main re-runs tests + build only

New jobs:
- branch-name: enforce feat/*/feature/*/fix/*/hotfix/* naming on PRs
- secret-scan: gitleaks v8 — blocks secrets from merging
- nodejs-build: 'next build' for admin-compliance + developer-portal
  (catches webpack/TS errors like the duplicate-export that broke CI)
- dep-audit: pip-audit (Python), npm audit --moderate (Node),
  govulncheck (Go, non-blocking until modules are pinned)

Existing job improvements:
- go-lint: add 'go build ./...' compile check
- python-lint: add import sanity check (catches NameError at collection)
- Rename test jobs for consistency

[guardrail-change]

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Sharang Parnerkar
2026-04-19 16:46:02 +02:00
parent c05a71163b
commit f96536ebbe

View File

@@ -1,45 +1,41 @@
# Gitea Actions CI/CD Pipeline # BreakPilot Compliance — CI Pipeline
# BreakPilot Compliance
# #
# Services: # Feature branch workflow:
# Go: ai-compliance-sdk # feat/* | feature/* | fix/* | hotfix/* | chore/* | refactor/* | docs/* | test/* | ci/*
# Python: backend-compliance, document-crawler, dsms-gateway # → open PR targeting main
# Node.js: admin-compliance, developer-portal # → all jobs run as PR gates
# → squash merge to main
# → subset of jobs re-run on main to catch merge surprises
# #
# Workflow: # Deploy is handled by build-push-deploy.yml on push to main.
# Push auf main → Tests → Deploy (Orca)
# Pull Request → Lint + Tests (kein Deploy)
name: CI/CD name: CI
on: on:
push: push:
branches: [main, develop] branches: [main]
pull_request: pull_request:
branches: [main, develop] branches: [main]
jobs: jobs:
# ========================================
# Guardrails — LOC budget + architecture gates
# Runs on every push/PR. Fails fast and cheap.
# ========================================
loc-budget: # ── Branch naming convention (PR only) ──────────────────────────────────
branch-name:
runs-on: docker runs-on: docker
container: alpine:3.20 container: alpine:3.20
if: github.event_name == 'pull_request'
steps: steps:
- name: Checkout - name: Validate branch name
run: | run: |
apk add --no-cache git bash BRANCH="${GITHUB_HEAD_REF}"
git clone --depth 50 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . if ! echo "$BRANCH" | grep -qE '^(feat|feature|fix|hotfix|chore|refactor|docs|test|ci)/.+'; then
- name: Enforce 500-line hard cap (whole repo) echo "::error::Branch '$BRANCH' does not follow naming convention."
run: | echo "Required prefix: feat/ feature/ fix/ hotfix/ chore/ refactor/ docs/ test/ ci/"
chmod +x scripts/check-loc.sh exit 1
scripts/check-loc.sh fi
# Phase 5: whole-repo blocking gate. Phases 1-4 have drained the legacy echo "Branch name OK: $BRANCH"
# baseline; any remaining oversized files must be listed in
# .claude/rules/loc-exceptions.txt with a written rationale.
# ── Guardrail integrity (PR only) ────────────────────────────────────────
guardrail-integrity: guardrail-integrity:
runs-on: docker runs-on: docker
container: alpine:3.20 container: alpine:3.20
@@ -50,20 +46,47 @@ jobs:
apk add --no-cache git bash apk add --no-cache git bash
git clone --depth 20 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 20 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
git fetch origin ${GITHUB_BASE_REF}:base git fetch origin ${GITHUB_BASE_REF}:base
- name: Require [guardrail-change] label in PR commits touching guardrails - name: Require [guardrail-change] in commits touching guardrails
run: | run: |
changed=$(git diff --name-only base...HEAD) changed=$(git diff --name-only base...HEAD)
echo "$changed" | grep -E '^(\.claude/settings\.json|\.claude/rules/loc-exceptions\.txt|scripts/check-loc\.sh|scripts/githooks/pre-commit|AGENTS\.(python|go|typescript)\.md)$' || exit 0 echo "$changed" | grep -qE '^(\.claude/settings\.json|\.claude/rules/loc-exceptions\.txt|scripts/check-loc\.sh|scripts/githooks/pre-commit|AGENTS\.(python|go|typescript)\.md)$' || exit 0
if ! git log base..HEAD --format=%B | grep -q '\[guardrail-change\]'; then if ! git log base..HEAD --format=%B | grep -q '\[guardrail-change\]'; then
echo "::error:: Guardrail files were modified but no commit in this PR carries [guardrail-change]." echo "::error::Guardrail files modified without [guardrail-change] in any commit message."
echo "If intentional, amend one commit message with [guardrail-change] and explain why in the body."
exit 1 exit 1
fi fi
# ======================================== # ── LOC budget (always) ──────────────────────────────────────────────────
# Lint (nur bei PRs) loc-budget:
# ======================================== runs-on: docker
container: alpine:3.20
steps:
- name: Checkout
run: |
apk add --no-cache git bash
git clone --depth 50 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Enforce 500-line hard cap
run: |
chmod +x scripts/check-loc.sh
scripts/check-loc.sh
# ── Secret scanning (PR only) ────────────────────────────────────────────
secret-scan:
runs-on: docker
container: zricethezav/gitleaks:v8.21.2
if: github.event_name == 'pull_request'
steps:
- name: Checkout
run: |
apk add --no-cache git
git clone --depth 50 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Scan for secrets
run: |
gitleaks detect --source . --no-git \
--exit-code 1 \
--redact \
|| { echo "::error::Secrets detected — remove them before merging."; exit 1; }
# ── Go lint + build (PR only) ────────────────────────────────────────────
go-lint: go-lint:
runs-on: docker runs-on: docker
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
@@ -75,10 +98,16 @@ jobs:
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Lint ai-compliance-sdk - name: Lint ai-compliance-sdk
run: | run: |
if [ -d "ai-compliance-sdk" ]; then [ -d "ai-compliance-sdk" ] || exit 0
cd ai-compliance-sdk && golangci-lint run --timeout 5m ./... cd ai-compliance-sdk
fi golangci-lint run --timeout 5m ./...
- name: Build ai-compliance-sdk
run: |
[ -d "ai-compliance-sdk" ] || exit 0
cd ai-compliance-sdk
go build ./...
# ── Python lint + import check (PR only) ────────────────────────────────
python-lint: python-lint:
runs-on: docker runs-on: docker
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
@@ -88,29 +117,27 @@ jobs:
run: | run: |
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1 apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Lint Python services (ruff) - name: Lint (ruff) + type-check (mypy)
run: | run: |
pip install --quiet ruff pip install --quiet ruff mypy
fail=0 fail=0
for svc in backend-compliance document-crawler dsms-gateway compliance-tts-service; do for svc in backend-compliance document-crawler dsms-gateway compliance-tts-service; do
if [ -d "$svc" ]; then [ -d "$svc" ] || continue
echo "=== ruff: $svc ===" echo "=== ruff: $svc ===" && ruff check "$svc/" --output-format=github || fail=1
ruff check "$svc/" --output-format=github || fail=1
fi
done done
exit $fail
- name: Type-check (mypy via backend-compliance/mypy.ini)
# Policy is declared in backend-compliance/mypy.ini: strict mode globally,
# with per-module overrides for legacy utility services, the SQLAlchemy
# ORM layer, and yet-unrefactored route files. Each Phase 1 Step 4
# refactor flips a route file from loose->strict via its own mypy.ini
# override block.
run: |
pip install --quiet mypy
if [ -f "backend-compliance/mypy.ini" ]; then if [ -f "backend-compliance/mypy.ini" ]; then
cd backend-compliance && mypy compliance/ cd backend-compliance && mypy compliance/ || fail=1
fi fi
exit $fail
- name: Import sanity check (catches NameError at collection time)
run: |
cd backend-compliance
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
python -c "import compliance; print('Import OK')" \
|| { echo "::error::compliance package fails to import — missing import or syntax error."; exit 1; }
# ── Node.js lint + type-check (PR only) ─────────────────────────────────
nodejs-lint: nodejs-lint:
runs-on: docker runs-on: docker
if: github.event_name == 'pull_request' if: github.event_name == 'pull_request'
@@ -120,26 +147,105 @@ jobs:
run: | run: |
apk add --no-cache git apk add --no-cache git
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Lint + type-check Node.js services - name: Lint + type-check
run: | run: |
fail=0 fail=0
for svc in admin-compliance developer-portal; do for svc in admin-compliance developer-portal; do
if [ -d "$svc" ]; then [ -d "$svc" ] || continue
echo "=== $svc: install ===" echo "=== $svc: install ===" && (cd "$svc" && npm ci --silent 2>/dev/null || npm install --silent)
(cd "$svc" && (npm ci --silent 2>/dev/null || npm install --silent)) echo "=== $svc: next lint ===" && (cd "$svc" && npx next lint) || fail=1
echo "=== $svc: next lint ===" echo "=== $svc: tsc ===" && (cd "$svc" && npx tsc --noEmit) || fail=1
(cd "$svc" && npx next lint) || fail=1
echo "=== $svc: tsc --noEmit ==="
(cd "$svc" && npx tsc --noEmit) || fail=1
fi
done done
exit $fail exit $fail
# ======================================== # ── Node.js build — next build (PR + push to main) ───────────────────────
# Unit Tests nodejs-build:
# ======================================== runs-on: docker
container: node:20-alpine
steps:
- name: Checkout
run: |
apk add --no-cache git
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Build Next.js services
run: |
fail=0
for svc in admin-compliance developer-portal; do
[ -d "$svc" ] || continue
echo "=== $svc: install ==="
(cd "$svc" && npm ci --silent 2>/dev/null || npm install --silent)
echo "=== $svc: next build ==="
(cd "$svc" && \
NEXT_PUBLIC_API_URL=https://api-dev.breakpilot.ai \
NEXT_PUBLIC_SDK_URL=https://sdk-dev.breakpilot.ai \
npm run build) || fail=1
done
exit $fail
test-go-ai-compliance: # ── Dependency audit (PR only) ───────────────────────────────────────────
dep-audit:
runs-on: docker
if: github.event_name == 'pull_request'
container: python:3.12-slim
steps:
- name: Checkout
run: |
apt-get update -qq && apt-get install -y -qq git curl > /dev/null 2>&1
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Install Node.js + Go
run: |
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - > /dev/null 2>&1
apt-get install -y nodejs golang-go > /dev/null 2>&1
- name: Python — pip-audit
run: |
pip install --quiet pip-audit
fail=0
for svc in backend-compliance document-crawler dsms-gateway compliance-tts-service; do
[ -f "$svc/requirements.txt" ] || continue
echo "=== pip-audit: $svc ==="
pip-audit -r "$svc/requirements.txt" --skip-editable -f columns || fail=1
done
exit $fail
- name: Node.js — npm audit
run: |
fail=0
for svc in admin-compliance developer-portal; do
[ -d "$svc" ] || continue
echo "=== npm audit: $svc ==="
(cd "$svc" && npm audit --audit-level=moderate --json 2>/dev/null | \
node -e "const d=JSON.parse(require('fs').readFileSync('/dev/stdin','utf8')); \
const hi=Object.values(d.vulnerabilities||{}).filter(v=>['high','critical'].includes(v.severity)).length; \
if(hi>0){console.error('HIGH/CRITICAL: '+hi);process.exit(1)}") || fail=1
done
exit $fail
- name: Go — govulncheck
run: |
[ -d "ai-compliance-sdk" ] || exit 0
go install golang.org/x/vuln/cmd/govulncheck@latest 2>/dev/null
cd ai-compliance-sdk && govulncheck ./... || true
# Non-blocking until Go module versions are pinned
# ── SBOM + vulnerability scan (PR only) ─────────────────────────────────
sbom-scan:
runs-on: docker
if: github.event_name == 'pull_request'
container: alpine:3.20
steps:
- name: Checkout
run: |
apk add --no-cache git curl bash
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Install syft + grype
run: |
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
- name: Generate SBOM
run: mkdir -p sbom-out && syft dir:. -o cyclonedx-json=sbom-out/sbom.cdx.json -q
- name: Vulnerability scan (fail on high+)
run: grype sbom:sbom-out/sbom.cdx.json --fail-on high -q
# ── Tests (PR + push to main) ─────────────────────────────────────────────
test-go:
runs-on: docker runs-on: docker
container: golang:1.24-alpine container: golang:1.24-alpine
env: env:
@@ -151,16 +257,12 @@ jobs:
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Test ai-compliance-sdk - name: Test ai-compliance-sdk
run: | run: |
if [ ! -d "ai-compliance-sdk" ]; then [ -d "ai-compliance-sdk" ] || exit 0
echo "WARNUNG: ai-compliance-sdk nicht gefunden"
exit 0
fi
cd ai-compliance-sdk cd ai-compliance-sdk
go test -v -coverprofile=coverage.out ./... 2>&1 go test -v -coverprofile=coverage.out ./...
COVERAGE=$(go tool cover -func=coverage.out 2>/dev/null | tail -1 | awk '{print $3}' || echo "0%") go tool cover -func=coverage.out | tail -1
echo "Coverage: $COVERAGE"
test-python-backend-compliance: test-python-backend:
runs-on: docker runs-on: docker
container: python:3.12-slim container: python:3.12-slim
env: env:
@@ -172,14 +274,11 @@ jobs:
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Test backend-compliance - name: Test backend-compliance
run: | run: |
if [ ! -d "backend-compliance" ]; then [ -d "backend-compliance" ] || exit 0
echo "WARNUNG: backend-compliance nicht gefunden"
exit 0
fi
cd backend-compliance cd backend-compliance
export PYTHONPATH="$(pwd):${PYTHONPATH:-}" export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
pip install --quiet --no-cache-dir fastapi uvicorn pytest pytest-asyncio pip install --quiet --no-cache-dir pytest pytest-asyncio
python -m pytest compliance/tests/ -v --tb=short python -m pytest compliance/tests/ -v --tb=short
test-python-document-crawler: test-python-document-crawler:
@@ -194,10 +293,7 @@ jobs:
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Test document-crawler - name: Test document-crawler
run: | run: |
if [ ! -d "document-crawler" ]; then [ -d "document-crawler" ] || exit 0
echo "WARNUNG: document-crawler nicht gefunden"
exit 0
fi
cd document-crawler cd document-crawler
export PYTHONPATH="$(pwd):${PYTHONPATH:-}" export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
@@ -216,46 +312,14 @@ jobs:
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Test dsms-gateway - name: Test dsms-gateway
run: | run: |
if [ ! -d "dsms-gateway" ]; then [ -d "dsms-gateway" ] || exit 0
echo "WARNUNG: dsms-gateway nicht gefunden"
exit 0
fi
cd dsms-gateway cd dsms-gateway
export PYTHONPATH="$(pwd):${PYTHONPATH:-}" export PYTHONPATH="$(pwd):${PYTHONPATH:-}"
pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true pip install --quiet --no-cache-dir -r requirements.txt 2>/dev/null || true
pip install --quiet --no-cache-dir pytest pytest-asyncio pip install --quiet --no-cache-dir pytest pytest-asyncio
python -m pytest test_main.py -v --tb=short python -m pytest test_main.py -v --tb=short
# ======================================== # ── OpenAPI contract validation (always) ─────────────────────────────────
# SBOM + license scan (compliance product → we eat our own dog food)
# ========================================
sbom-scan:
runs-on: docker
if: github.event_name == 'pull_request'
container: alpine:3.20
steps:
- name: Checkout
run: |
apk add --no-cache git curl bash
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Install syft + grype
run: |
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin
- name: Generate SBOM
run: |
mkdir -p sbom-out
syft dir:. -o cyclonedx-json=sbom-out/sbom.cdx.json -q
- name: Vulnerability scan (fail on high+)
run: |
grype sbom:sbom-out/sbom.cdx.json --fail-on high -q
# Phase 5: blocking. Any high+ CVE in the dependency graph fails the PR.
# ========================================
# Validate Canonical Controls
# ========================================
validate-canonical-controls: validate-canonical-controls:
runs-on: docker runs-on: docker
container: python:3.12-slim container: python:3.12-slim
@@ -265,8 +329,4 @@ jobs:
apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1 apt-get update -qq && apt-get install -y -qq git > /dev/null 2>&1
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git . git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
- name: Validate controls - name: Validate controls
run: | run: python scripts/validate-controls.py
python scripts/validate-controls.py
# Deploy is handled by .gitea/workflows/build-push-deploy.yml
# which builds images, pushes to registry.meghsakha.com, and triggers orca.