sharang/compliance-scanner-agent
1
0
Fork 0
Code Issues 79 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity

CI: use crates.meghsakha.com (Kellnr) mirror + persistent sccache (Hetzner S3), matching werkpilot #139

New Issue
Closed
opened 2026-07-10 13:32:21 +00:00 by sharang · 0 comments
sharang commented 2026-07-10 13:32:21 +00:00
Owner
Copy Link
Copy Source

Problem

compliance-scanner CI (.gitea/workflows/ci.yml) is neither crates.io-independent nor cross-run cached, unlike werkpilot which already has both:

  • No self-hosted crate mirror. CI resolves dependencies straight from crates.io (671 crates). No .cargo/config.toml in the repo, and the CI image (rust:1.94-bookworm) uses the default crates.io source. This makes CI slower and vulnerable to crates.io rate-limits/outages.
  • sccache is only intra-job. We set RUSTC_WRAPPER=/usr/local/bin/sccache with SCCACHE_DIR=/tmp/sccache — an ephemeral per-container dir, so the cache is thrown away after every run. It only shares work within one job (the two dashboard feature compiles); there is no persistent backend, so every PR recompiles from scratch.

Werkpilot's setup (reference — werkpilot/.gitea/workflows/ci.yml)

  1. Kellnr crates.io mirror (crates.meghsakha.com), written into .cargo/config.toml in a CI step:

    [source.crates-io]
replace-with = "kellnr"
[registries.kellnr]
index = "sparse+https://crates.meghsakha.com/api/v1/cratesio/"

    Self-hosted, cached, crates.io-independent.

  2. sccache → Hetzner S3 (runner-independent, persists across runs):

    RUSTC_WRAPPER: sccache
SCCACHE_BUCKET: breakpilot-sccache
SCCACHE_ENDPOINT: https://nbg1.your-objectstorage.com
SCCACHE_REGION: auto
SCCACHE_S3_USE_SSL: "true"
SCCACHE_S3_KEY_PREFIX: werkpilot
AWS_ACCESS_KEY_ID: ${{ secrets.HETZNER_S3_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.HETZNER_S3_SECRET_KEY }}

Proposed change

Adopt the same two mechanisms in .gitea/workflows/ci.yml:

  • Add a "Use Kellnr crates.io mirror" step that writes .cargo/config.toml (as above) before any cargo command.
  • Point sccache at the shared Hetzner S3 bucket with SCCACHE_S3_KEY_PREFIX: compliance-scanner (own key prefix, same bucket/creds as werkpilot). Drop SCCACHE_DIR.
  • Reuse the existing repo secrets (HETZNER_S3_ACCESS_KEY, HETZNER_S3_SECRET_KEY) — confirm they're set on this repo/org in Gitea, or copy them from werkpilot's repo settings.

Outcome

Faster CI (warm compile cache across runs) and crates.io-independent dependency resolution. No source changes — CI config only.

(Filed as a follow-up per discussion; not blocking the onboarding PRs.)

## Problem compliance-scanner CI (`.gitea/workflows/ci.yml`) is neither crates.io-independent nor cross-run cached, unlike **werkpilot** which already has both: - **No self-hosted crate mirror.** CI resolves dependencies straight from crates.io (671 crates). No `.cargo/config.toml` in the repo, and the CI image (`rust:1.94-bookworm`) uses the default crates.io source. This makes CI slower and vulnerable to crates.io rate-limits/outages. - **sccache is only intra-job.** We set `RUSTC_WRAPPER=/usr/local/bin/sccache` with `SCCACHE_DIR=/tmp/sccache` — an ephemeral per-container dir, so the cache is thrown away after every run. It only shares work *within* one job (the two dashboard feature compiles); there is **no persistent backend**, so every PR recompiles from scratch. ## Werkpilot's setup (reference — `werkpilot/.gitea/workflows/ci.yml`) 1. **Kellnr crates.io mirror** (`crates.meghsakha.com`), written into `.cargo/config.toml` in a CI step: ```toml [source.crates-io] replace-with = "kellnr" [registries.kellnr] index = "sparse+https://crates.meghsakha.com/api/v1/cratesio/" ``` Self-hosted, cached, crates.io-independent. 2. **sccache → Hetzner S3** (runner-independent, persists across runs): ```yaml RUSTC_WRAPPER: sccache SCCACHE_BUCKET: breakpilot-sccache SCCACHE_ENDPOINT: https://nbg1.your-objectstorage.com SCCACHE_REGION: auto SCCACHE_S3_USE_SSL: "true" SCCACHE_S3_KEY_PREFIX: werkpilot AWS_ACCESS_KEY_ID: ${{ secrets.HETZNER_S3_ACCESS_KEY }} AWS_SECRET_ACCESS_KEY: ${{ secrets.HETZNER_S3_SECRET_KEY }} ``` ## Proposed change Adopt the same two mechanisms in `.gitea/workflows/ci.yml`: - Add a "Use Kellnr crates.io mirror" step that writes `.cargo/config.toml` (as above) before any cargo command. - Point sccache at the shared Hetzner S3 bucket with `SCCACHE_S3_KEY_PREFIX: compliance-scanner` (own key prefix, same bucket/creds as werkpilot). Drop `SCCACHE_DIR`. - Reuse the existing repo secrets (`HETZNER_S3_ACCESS_KEY`, `HETZNER_S3_SECRET_KEY`) — confirm they're set on this repo/org in Gitea, or copy them from werkpilot's repo settings. ## Outcome Faster CI (warm compile cache across runs) and crates.io-independent dependency resolution. No source changes — CI config only. *(Filed as a follow-up per discussion; not blocking the onboarding PRs.)*
sharang added the infrastructure label 2026-07-10 13:32:21 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P0-demo
CERTifAI P0 demo build tickets (CS-*)
 bug
critical
enhancement
epic
Tracking issue spanning multiple sub-issues
 high
infrastructure
onboarding
Multi-target onboarding wizard redesign
 performance
security
v0.3.0
No labels infrastructure
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
Benjamin_Boenisch sharang (Sharang Parnerkar)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sharang/compliance-scanner-agent#139