ci: Kellnr crates.io mirror + persistent S3-backed sccache #140

2026-07-10T16:04:04Z

sharang commented

2026-07-10 16:04:04 +00:00

Closes #139. Adopts werkpilot's CI compile-acceleration setup so builds are crates.io-independent and cached across runs (today every PR recompiles all ~670 crates cold).

Changes (`.gitea/workflows/ci.yml`, `check` job only)

Kellnr crates.io mirror — a new "Use Kellnr crates.io mirror" step writes $CARGO_HOME/config.toml with [source.crates-io] replace-with = "kellnr" → sparse+https://crates.meghsakha.com/api/v1/cratesio/. Runs before any cargo command. Git deps (tramiton-core) are unaffected — source replacement only touches crates.io-sourced crates.
Persistent sccache → Hetzner S3 — replaced the ephemeral SCCACHE_DIR: /tmp/sccache with the S3 backend (breakpilot-sccache bucket, SCCACHE_S3_KEY_PREFIX: compliance-scanner — own prefix, same bucket as werkpilot). Bumped sccache to v0.10.0.
Throttle CARGO_NET_RETRY=10 + CARGO_HTTP_MULTIPLEXING=false so the concurrent download burst doesn't 429 the mirror.

⚠️ Requires two repo secrets

Add these to compliance-scanner-agent → Settings → Actions → Secrets (copy the values from werkpilot's repo/org settings — same bucket & creds):

HETZNER_S3_ACCESS_KEY
HETZNER_S3_SECRET_KEY

Without them, sccache can't reach S3 and the build fails (RUSTC_WRAPPER=sccache). This is the only manual step.

Expected effect

First run after merge: cold (populates the S3 cache).
Subsequent runs: warm — sccache pulls object files from S3, so the ~9-min cold compile drops substantially. Show sccache stats reports the hit rate.

Closes #139. Adopts werkpilot's CI compile-acceleration setup so builds are **crates.io-independent** and **cached across runs** (today every PR recompiles all ~670 crates cold). ## Changes (`.gitea/workflows/ci.yml`, `check` job only) - **Kellnr crates.io mirror** — a new "Use Kellnr crates.io mirror" step writes `$CARGO_HOME/config.toml` with `[source.crates-io] replace-with = "kellnr"` → `sparse+https://crates.meghsakha.com/api/v1/cratesio/`. Runs before any cargo command. Git deps (tramiton-core) are unaffected — source replacement only touches crates.io-sourced crates. - **Persistent sccache → Hetzner S3** — replaced the ephemeral `SCCACHE_DIR: /tmp/sccache` with the S3 backend (`breakpilot-sccache` bucket, `SCCACHE_S3_KEY_PREFIX: compliance-scanner` — own prefix, same bucket as werkpilot). Bumped sccache to v0.10.0. - **Throttle** `CARGO_NET_RETRY=10` + `CARGO_HTTP_MULTIPLEXING=false` so the concurrent download burst doesn't 429 the mirror. ## ⚠️ Requires two repo secrets Add these to `compliance-scanner-agent` → Settings → Actions → Secrets (copy the values from werkpilot's repo/org settings — same bucket & creds): - **`HETZNER_S3_ACCESS_KEY`** - **`HETZNER_S3_SECRET_KEY`** Without them, sccache can't reach S3 and the build fails (RUSTC_WRAPPER=sccache). This is the only manual step. ## Expected effect - First run after merge: cold (populates the S3 cache). - Subsequent runs: warm — sccache pulls object files from S3, so the ~9-min cold compile drops substantially. `Show sccache stats` reports the hit rate.

sharang added the infrastructure label 2026-07-10 16:04:04 +00:00

sharang added 1 commit 2026-07-10 16:20:59 +00:00

ci: Kellnr crates.io mirror + persistent S3-backed sccache

CI / Check (pull_request) Successful in 9m10s

Details

CI / Deploy Docs (pull_request) Has been skipped

Details

CI / Detect Changes (pull_request) Has been skipped

Details

CI / Deploy Agent (pull_request) Has been skipped

Details

CI / Deploy Dashboard (pull_request) Has been skipped

Details

CI / Deploy MCP (pull_request) Has been skipped

Details

63c60f4c45

Adopt werkpilot's CI compile-acceleration setup so builds are crates.io-
independent and cached across runs:

- Resolve crates.io deps through the self-hosted Kellnr mirror
  (crates.meghsakha.com) via a .cargo/config.toml source replacement. Git deps
  (tramiton-core) are unaffected.
- Back sccache with Hetzner S3 (breakpilot-sccache bucket, key prefix
  compliance-scanner) instead of an ephemeral /tmp dir — persistent across CI
  runs. Bump sccache to v0.10.0.
- Throttle cargo (CARGO_NET_RETRY, HTTP/1.1) so the ~670-crate download burst
  doesn't 429 the mirror.

Requires repo secrets HETZNER_S3_ACCESS_KEY and HETZNER_S3_SECRET_KEY (same
bucket/creds as werkpilot). Closes #139.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

sharang force-pushed feat/ci-kellnr-sccache from c483fc846a to 63c60f4c45

2026-07-10 16:20:59 +00:00

Compare

sharang merged commit a074efd0b4 into main

2026-07-10 16:30:51 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sharang/compliance-scanner-agent#140

ci: Kellnr crates.io mirror + persistent S3-backed sccache #140

Changes (.gitea/workflows/ci.yml, check job only)

⚠️ Requires two repo secrets

Expected effect

Changes (`.gitea/workflows/ci.yml`, `check` job only)