Compare commits

..
Author SHA1 Message Date
Sharang ParnerkarandClaude Opus 4.8 9717a1efda fix(onboarding): targets visibility + unified pipeline by default
CI / Check (pull_request) Successful in 5m36s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
The onboarding wizard created OnboardedTargets that were invisible in the
dashboard, and triggering a scan failed with "Repository <id> not found":
`/targets/{id}/scan` went through `run_scan`, which consulted the global
`unified_pipeline` flag and fell to the legacy repository pipeline (reads
`repositories`, not `onboarded_targets`).

Agent
- Add `ComplianceAgent::run_target_scan`, always dispatching to the unified
  `run_target` pipeline. The target-scan endpoint operates on
  `onboarded_targets` by construction, so it must not depend on the
  transition flag. `trigger_target_scan` now calls it.
- Default `UNIFIED_PIPELINE` to on (no legacy `repositories` data in prod);
  set `UNIFIED_PIPELINE=0` to opt back to the legacy pipeline.
- Scheduler now scans `onboarded_targets` (via `run_target_scan`) instead of
  the legacy `repositories` collection.

Dashboard
- New Targets page (`/targets`): lists onboarded targets with detected type,
  artifacts, findings count, applicable-scans matrix (on expand), plus Run
  scan and Delete. Sidebar "Repositories" nav becomes "Targets".
- Remove the "Add Repository" form from the Repositories page — onboarding
  is the single entry point (private-repo auth + issue tracker move into the
  onboarding flow, revisable on the target).
- Add `delete_target` server fn.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 09:22:03 +02:00
sharang 669e1f1b03 feat(onboarding): scan-trigger endpoint + wizard Run-Scan button (#153)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Successful in 3m34s
CI / Deploy Dashboard (push) Successful in 2m56s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 22:19:32 +00:00
sharang 9e70bd1c8e ci: don't cancel-in-progress for main-branch runs (only pull_request) (#152)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 22:08:58 +00:00
sharang 9ec07ff7a1 fix(ci): authenticate tramiton fetch in dashboard + mcp image builds (#147)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m47s
CI / Deploy Dashboard (push) Successful in 2m37s
CI / Deploy Docs (push) Has been skipped
2026-07-12 21:52:38 +00:00
sharang 0e57c2d7a7 feat(pipeline): run tramiton classification + provision DAST in run_target (#146)
CI / Deploy Agent (push) Successful in 3m40s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Has been skipped
2026-07-12 21:49:22 +00:00
sharang c6a02f0ac3 feat(pipeline): unified run_target execution behind UNIFIED_PIPELINE (#145)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m47s
CI / Deploy Dashboard (push) Failing after 4m28s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Failing after 5s
2026-07-12 21:31:21 +00:00
sharang 17f788858d feat(dashboard): onboarding wizard UI (#144)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Agent (push) Has been skipped
2026-07-12 21:28:52 +00:00
sharangandSharang Parnerkar c6b5622113 fix(ci): authenticate tramiton-core fetch in the agent image build (main deploys) (#143)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m46s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
The PR `check` job authenticates the private tramiton-core git fetch, but the main-branch **`deploy-agent`** job builds `Dockerfile.agent` where cargo also fetches tramiton-core — inside the image build, with no credentials — so **agent image builds on main fail**.

Fix: inject the PAT as a **BuildKit secret** (never baked into an image layer).
- `Dockerfile.agent`: `RUN --mount=type=secret,id=tramiton_token …` applies the same `https`-insteadOf rewrite + `CARGO_NET_GIT_FETCH_WITH_CLI=true` before `cargo build`.
- `deploy-agent`: `DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN …`, reusing the existing `TRAMITON_FETCH_TOKEN` secret.

Only the agent image depends on tramiton-core (dashboard/mcp/docs unaffected). **Self-tests on merge** — changing `Dockerfile.agent` makes `detect-changes` run `deploy-agent`.

Note: couldn't fully run the image build locally (no PAT value on hand + no `.dockerignore` so the context is large), but this mirrors the working PR-stage auth and uses standard BuildKit secret injection. Assumes the deploy runner's Docker daemon supports BuildKit (docker:27-cli → yes).
---------

Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #143
2026-07-12 20:25:39 +00:00
sharang a204f0c59c feat(api): onboarding endpoints for unified targets (#142)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 20:15:22 +00:00
sharang cf4afdda1b feat(migrate): onboarding backfill (repositories + dast_targets -> onboarded_targets) (#141)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 20:11:16 +00:00
sharang a074efd0b4 ci: Kellnr crates.io mirror + persistent S3-backed sccache (#140)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-10 16:30:51 +00:00
sharang ef6ee3dcd1 feat(onboarding): artifact ingest + classifier + native tramiton + suite seams (#138)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Failing after 5s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Failing after 4s
2026-07-10 16:00:51 +00:00
sharang 675c4ef699 feat(onboarding): unified multi-target model + scan matrix foundation (#134)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m36s
CI / Deploy Dashboard (push) Successful in 7m9s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m44s
2026-07-10 13:41:25 +00:00
sharang aed551231c feat(dashboard): UI for managing MCP tokens (#94)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 3m3s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
Adds /mcp-tokens page so a logged-in user can mint, list, and revoke bearer tokens for the MCP server without curl. Pairs with #92's tenant-scoped MCP middleware — copy a token from the dashboard straight into an LLM client config.
2026-06-30 16:32:54 +00:00
sharang b851f4267a feat(m7.3): scheduler pulls tenants from registry, env as fallback (#96)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
Replaces M7.2-C static SCHEDULER_TENANT_IDS with a live query to the tenant-registry at every tick. New tenants picked up without an agent restart; env stays as fallback so a registry outage never silences the scheduler. Resolution order: registry -> SCHEDULER_TENANT_IDS env -> DEFAULT_SCHEDULER_TENANT_ID. Logs the active source on startup.
2026-06-30 16:32:35 +00:00
sharang e9536b6d98 fix(audit): bump quinn-proto + ignore rmcp DNS-rebinding advisory (#97)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m55s
CI / Deploy Dashboard (push) Successful in 2m44s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m50s
RUSTSEC-2026-0185 (quinn-proto 0.11.14): patch-bump to 0.11.15. RUSTSEC-2026-0189 (rmcp 0.16 DNS rebinding): added to ignore with public-hostname + bearer-auth threat-model justification; rmcp 0.16->2.x migration tracked as a separate multi-hour PR.
2026-06-30 16:07:01 +00:00
sharang a3a96fe2cc feat(m7.3): MCP tenant-scoped bearer tokens (#92)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m13s
CI / Deploy Dashboard (push) Successful in 7m3s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m50s
MCP server validates per-tenant bearer tokens on incoming calls and routes each tool to the caller's tenant DB. Closes the cross-tenant data leak in the MCP path identified in M7.3.
2026-06-30 15:27:21 +00:00
sharang ac24ca766a feat(m7.3): cross-tenant admin HTTP endpoints (#95)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
GET /api/admin/tenants lists tenant DBs; DELETE /api/admin/tenants/{tenant_id} drops them (GDPR). Behind a separate auth path that rejects customer realm tokens.
2026-06-30 15:23:39 +00:00
sharang 485c3ff45e chore(agent): remove stale unscoped webhook routes from API router (#93)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
Webhook routes live on the separate webhook server (port 3002). M7.2-C URL form is /webhook/{tenant_id}/{platform}/{repo_id}; mounting unscoped variants on the API router would mismatch handler signatures.
2026-06-30 15:18:31 +00:00
sharang 69c4f7bb78 feat(dashboard): proactively refresh expired Keycloak tokens (#91)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 8s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 2m55s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-06-17 20:01:37 +00:00
sharang 56482911b8 fix(dashboard): attach Keycloak token on agent API calls (#90)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 6s
CI / Deploy Agent (push) Successful in 4m8s
CI / Deploy Dashboard (push) Successful in 4m58s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-06-17 18:35:59 +00:00
sharang 183234f9af feat(m7.1): wire compliance-agent to compliance-core auth + status gate (#85)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m38s
CI / Deploy Dashboard (push) Successful in 7m30s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m55s
2026-06-17 09:36:52 +00:00
sharang dbadff0aac fix(m7.1): JWKS refresh-on-failure in auth middleware (#84)
CI / Deploy Agent (push) Successful in 11m44s
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Dashboard (push) Successful in 13m1s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m53s
2026-06-04 14:46:14 +00:00
sharang 116293519d M7.1 smoke harness: lift auth to compliance-core + compliance-smoke service (#83)
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
2026-06-04 14:38:35 +00:00
sharang a8cef58e02 feat(dashboard): add light/dark theme with sidebar toggle (#81)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Dashboard (push) Successful in 9m46s
CI / Deploy Docs (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 11:44:22 +00:00
sharang 927fbc8ecb fix: live progress + concurrency for embedding builds (#80)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m59s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 10:01:05 +00:00
sharang e67a13535a fix: add HTTP timeout to reqwest client and CVE stage timeout (#79)
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m26s
CI / Deploy Dashboard (push) Has been skipped
CI / Check (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 07:30:26 +00:00
sharangandSharang Parnerkar df0063abc0 fix: scanner timeouts, semgrep memory cap, syft remote lookups, Script error (#78)
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 9m41s
CI / Check (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 15m19s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 3m7s
## Summary

- **Scan produces no results in Orca** — semgrep (`--config=auto`, unbounded memory) and syft (remote license network calls) were getting OOM-killed or hanging in resource-constrained Orca containers. Scan would "complete" with 0 findings/SBOMs silently because each scanner failure is caught and logged as a warning.
- **Dashboard Script error spam** — `document::Script` in Dioxus 0.7 needs a single text node child for inline scripts; `dangerous_inner_html` was invalid and spammed the error log on every unauthenticated page load.

## Changes

| File | Change |
|------|--------|
| `semgrep.rs` | Add `--max-memory 500 --jobs 1`; 10-minute timeout |
| `syft.rs` | Remove remote license lookup env vars; 5-minute timeout |
| `gitleaks.rs` | 5-minute timeout |
| `app_shell.rs` | Fix `dangerous_inner_html` → text child in `document::Script` |

## Test plan

- [ ] Trigger a scan on a repo in Orca — findings and SBOM entries should now appear
- [ ] Agent logs should show timeout/error warnings rather than silent empty results when tools are killed
- [ ] Navigate to dashboard unauthenticated — Script error gone from logs
- [ ] Verify scans work end-to-end with `docker compose up`

---------

Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #78
2026-05-12 11:27:24 +00:00
Sharang Parnerkar 5cafd13f44 ci: log orca webhook response so deploy steps arent silent
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
Nightly E2E Tests / E2E Tests (push) Failing after 2m59s
2026-04-08 15:09:27 +02:00
Sharang ParnerkarandClaude Opus 4.6 69209649a5 ci: trigger first orca build for all services
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 7m5s
CI / Deploy Dashboard (push) Failing after 21m28s
CI / Detect Changes (push) Successful in 4s
CI / Deploy Docs (push) Successful in 30s
CI / Deploy MCP (push) Successful in 1m31s
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:10:07 +02:00
Sharang ParnerkarandClaude Opus 4.6 d5439adc0d ci: trigger build of dashboard, docs, mcp images for orca
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:09:49 +02:00
Sharang ParnerkarandClaude Opus 4.6 bc7cdd35e4 ci: replace coolify webhook with orca deploy
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
Each deploy job now builds the per-service image, pushes to the
private registry as :latest and :sha, then triggers an HMAC-signed
orca redeploy webhook. Coolify webhooks are no longer used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:06:11 +02:00
Sharang ParnerkarandClaude Opus 4.6 c062d834a1 fix: downgrade dotenv missing file from FAILED to info message
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
Nightly E2E Tests / E2E Tests (push) Failing after 2m16s
Non-fatal in Docker where env vars come from container config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 15:33:24 +02:00
sharang 23cf37b6c3 fix: CVE notifications during scan + help chat doc loading + Dockerfile (#55)
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Has been skipped
2026-03-30 13:10:56 +00:00
sharang 49d5cd4e0a feat: hourly CVE alerting with notification bell and API (#53)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
2026-03-30 10:39:39 +00:00
sharang 4388e98b5b feat: add E2E test suite with nightly CI, fix dashboard Dockerfile (#52)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 2s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-30 10:04:07 +00:00
sharang a8bb05d7b1 feat: add floating help chat widget, remove settings page (#51)
CI / Deploy Docs (push) Successful in 2s
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3s
CI / Deploy Dashboard (push) Successful in 2s
2026-03-30 08:05:29 +00:00
sharang bae24f9cf8 fix: cascade-delete DAST targets, pentests, and downstream data on repo delete (#50)
CI / Deploy Agent (push) Successful in 9s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
2026-03-30 07:11:23 +00:00
sharang dd53132746 feat: refine all LLM system prompts for precision and reduced false positives (#49)
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
2026-03-30 07:11:17 +00:00
sharang ff088f9eb4 feat: deduplicate code review findings across LLM passes (#48)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 7s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-29 20:38:52 +00:00
745ad8a441 fix: check Gitea API response status and fallback for PR reviews (#47)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
## Summary
- Add HTTP response status checking to all Gitea tracker methods that were silently swallowing errors
- Add fallback in create_pr_review: if inline comments fail, retry as plain PR comment

## Test plan
- [ ] Deploy and trigger a PR review, check logs for actual error details
- [ ] Verify fallback posts summary comment when inline comments fail

Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com>
Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #47
2026-03-25 16:26:09 +00:00
sharang a9d039dad3 fix: stop storing code review findings in dashboard (#22)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-18 15:18:07 +00:00
Sharang ParnerkarandClaude Opus 4.6 a509bdcb2e fix: require TLS for IMAP auth, close port 143 (CERT-Bund compliance)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 7s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 1s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
- Remove port 143 from mailserver (only expose 993/IMAPS)
- Enable SSL_TYPE=manual with Let's Encrypt certs
- Set DOVECOT_DISABLE_PLAINTEXT_AUTH=yes
- Add pentest_imap_tls config field (defaults to true)

Fixes CERT-Bund report: IMAP PLAIN/LOGIN without TLS on 46.225.100.82:143

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 09:29:34 +01:00
sharangandSharang Parnerkar c461faa2fb feat: pentest onboarding — streaming, browser automation, reports, user cleanup (#16)
CI / Deploy MCP (push) Successful in 2s
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 7s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Successful in 2s
Complete pentest feature overhaul: SSE streaming, session-persistent browser tool (CDP), AES-256 credential encryption, auto-screenshots in reports, code-level remediation correlation, SAST triage chunking, context window optimization, test user cleanup (Keycloak/Auth0/Okta), wizard dropdowns, attack chain improvements, architecture docs with Mermaid diagrams.

Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com>
Reviewed-on: #16
2026-03-17 20:32:20 +00:00
152 changed files with 13897 additions and 1592 deletions
+23
View File
@@ -0,0 +1,23 @@
[advisories]
ignore = [
# hickory-proto 0.25.x pulled in transitively via mongodb → hickory-resolver.
# MongoDB 3.x has not yet released with hickory-resolver 0.26.x, so we cannot
# upgrade past this without a mongodb release. Both are DNS-layer DoS vectors
# requiring a MITM/controlled DNS server against MongoDB's hostname resolution —
# not a realistic attack surface here. Revisit when mongodb bumps hickory.
"RUSTSEC-2026-0118", # NSEC3 loop, no fix available upstream
"RUSTSEC-2026-0119", # O(n²) name compression, fixed in hickory-proto >=0.26.1
# rmcp 0.16.0 — DNS rebinding in Streamable HTTP server transport (missing
# Host header validation). Patched in rmcp >= 1.4.0, which is a major API
# version jump from our pin; rmcp shipped 0.x → 1.x → 2.x in three months
# and the migration touches every tool handler + the auth middleware we
# just landed in #92. Threat model in our deployment: the MCP server is
# exposed at a public hostname (comp-mcp-dev.meghsakha.com) behind orca's
# TLS-terminating ingress with per-tenant bearer auth — the attack model
# (browser DNS-rebinding into localhost MCP server) doesn't directly apply.
# Defense-in-depth Host-header check is still a worthwhile follow-up.
# FOLLOW-UP: bump rmcp to 2.x in a dedicated PR (M7.3 follow-up, sized
# multi-hour due to API surface change).
"RUSTSEC-2026-0189",
]
+113 -28
View File
@@ -9,15 +9,33 @@ on:
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-D warnings"
# sccache caches compilation artifacts within a job so that compiling
# both --features server and --features web shares common crate work.
# Compile cache: sccache -> Hetzner S3 (breakpilot-sccache), runner-independent
# and persistent across CI runs (own key prefix). Reuses the shared cluster S3
# creds (same bucket as werkpilot). Requires repo secrets HETZNER_S3_ACCESS_KEY
# and HETZNER_S3_SECRET_KEY.
RUSTC_WRAPPER: /usr/local/bin/sccache
SCCACHE_DIR: /tmp/sccache
SCCACHE_BUCKET: breakpilot-sccache
SCCACHE_ENDPOINT: https://nbg1.your-objectstorage.com
SCCACHE_REGION: auto
SCCACHE_S3_USE_SSL: "true"
SCCACHE_S3_KEY_PREFIX: compliance-scanner
AWS_ACCESS_KEY_ID: ${{ secrets.HETZNER_S3_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.HETZNER_S3_SECRET_KEY }}
# compliance-agent depends on tramiton-core via git; use the system git so the
# credential rewrite below (see "Configure git auth ...") is honored on fetch.
CARGO_NET_GIT_FETCH_WITH_CLI: "true"
# Throttle cargo so a ~670-crate concurrent download burst doesn't 429 the
# Kellnr mirror: fewer concurrent connections (HTTP/1.1) + more retries.
CARGO_NET_RETRY: "10"
CARGO_HTTP_MULTIPLEXING: "false"
# Cancel in-progress runs for the same branch/PR
# Cancel superseded PR runs, but NEVER cancel main-branch runs — those build and
# deploy per-service images, and cancelling one merge's deploy when the next
# merge lands leaves a service un-deployed (as happened between two back-to-back
# merges). So cancel-in-progress only for pull_request events.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
# ---------------------------------------------------------------------------
@@ -36,16 +54,44 @@ jobs:
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}"
git checkout FETCH_HEAD
# Resolve crates.io deps through the self-hosted Kellnr mirror (cached,
# crates.io-independent). Git deps (tramiton-core) are unaffected — source
# replacement only applies to crates.io-sourced crates.
- name: Use Kellnr crates.io mirror
run: |
: "${CARGO_HOME:=/usr/local/cargo}"
mkdir -p "$CARGO_HOME"
{
echo '[source.crates-io]'
echo 'replace-with = "kellnr"'
echo '[registries.kellnr]'
echo 'index = "sparse+https://crates.meghsakha.com/api/v1/cratesio/"'
} >> "$CARGO_HOME/config.toml"
env:
RUSTC_WRAPPER: ""
- name: Install tools
run: |
rustup component add rustfmt clippy
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.9.1/sccache-v0.9.1-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.9.1-x86_64-unknown-linux-musl/sccache
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.10.0-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
cargo install cargo-audit --locked
env:
RUSTC_WRAPPER: ""
# compliance-agent has a git dependency on tramiton-core (a private repo on
# this Gitea instance). Rewrite its SSH URL to HTTPS + a PAT so the runner
# can fetch it. Requires the repo secret TRAMITON_FETCH_TOKEN (a Gitea PAT
# with read:repository, owned by a user with access to sharang/tramiton).
# (Honored on fetch because CARGO_NET_GIT_FETCH_WITH_CLI=true uses system git.)
- name: Configure git auth for private tramiton dependency
run: |
git config --global \
url."https://sharang:${{ secrets.TRAMITON_FETCH_TOKEN }}@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"
env:
RUSTC_WRAPPER: ""
# Format (no compilation needed)
- name: Format
run: cargo fmt --all --check
@@ -70,7 +116,7 @@ jobs:
# Tests (reuses compilation artifacts from clippy)
- name: Tests (core + agent)
run: cargo test -p compliance-core -p compliance-agent
run: cargo test -p compliance-core -p compliance-agent --lib
- name: Tests (dashboard server)
run: cargo test -p compliance-dashboard --features server --no-default-features
- name: Tests (dashboard web)
@@ -145,13 +191,25 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.agent == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
env:
# PAT for fetching the private tramiton-core git dependency during the
# image build (injected as a BuildKit secret, never baked into a layer).
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_AGENT }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-agent
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy agent"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-dashboard:
name: Deploy Dashboard
@@ -159,13 +217,23 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.dashboard == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
env:
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_DASHBOARD }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-dashboard
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy dashboard"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-docs:
name: Deploy Docs
@@ -173,13 +241,20 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.docs == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_DOCS }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-docs
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.docs -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy docs"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-mcp:
name: Deploy MCP
@@ -187,10 +262,20 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.mcp == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
env:
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_MCP }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-mcp
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy mcp"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
+52
View File
@@ -0,0 +1,52 @@
name: Nightly E2E Tests
on:
schedule:
- cron: '0 3 * * *' # 3 AM UTC daily
workflow_dispatch: # Allow manual trigger
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-D warnings"
RUSTC_WRAPPER: /usr/local/bin/sccache
SCCACHE_DIR: /tmp/sccache
TEST_MONGODB_URI: "mongodb://root:example@mongo:27017/?authSource=admin"
concurrency:
group: nightly-e2e
cancel-in-progress: true
jobs:
e2e:
name: E2E Tests
runs-on: docker
container:
image: rust:1.94-bookworm
services:
mongo:
image: mongo:7
env:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
steps:
- name: Checkout
run: |
git init
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA:-refs/heads/main}"
git checkout FETCH_HEAD
- name: Install sccache
run: |
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.9.1/sccache-v0.9.1-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.9.1-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
env:
RUSTC_WRAPPER: ""
- name: Run E2E tests
run: cargo test -p compliance-agent --test e2e -- --test-threads=4
- name: Show sccache stats
run: sccache --show-stats
if: always()
Generated
+99 -13
View File
@@ -676,6 +676,7 @@ dependencies = [
"jsonwebtoken",
"mongodb",
"octocrab",
"rand 0.9.2",
"regex",
"reqwest",
"secrecy",
@@ -687,9 +688,11 @@ dependencies = [
"tokio-cron-scheduler",
"tokio-stream",
"tokio-tungstenite 0.26.2",
"tower",
"tower-http",
"tracing",
"tracing-subscriber",
"tramiton-core",
"urlencoding",
"uuid",
"walkdir",
@@ -700,19 +703,23 @@ dependencies = [
name = "compliance-core"
version = "0.1.0"
dependencies = [
"axum",
"bson",
"chrono",
"hex",
"jsonwebtoken",
"mongodb",
"opentelemetry",
"opentelemetry-appender-tracing",
"opentelemetry-otlp",
"opentelemetry_sdk",
"reqwest",
"secrecy",
"serde",
"serde_json",
"sha2",
"thiserror 2.0.18",
"tokio",
"tracing",
"tracing-opentelemetry",
"tracing-subscriber",
@@ -813,12 +820,15 @@ dependencies = [
"bson",
"chrono",
"compliance-core",
"dashmap",
"dotenvy",
"hex",
"mongodb",
"rmcp",
"schemars 1.2.1",
"serde",
"serde_json",
"sha2",
"thiserror 2.0.18",
"tokio",
"tower-http",
@@ -826,6 +836,20 @@ dependencies = [
"tracing-subscriber",
]
[[package]]
name = "compliance-smoke"
version = "0.1.0"
dependencies = [
"axum",
"compliance-core",
"reqwest",
"serde",
"serde_json",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "console_error_panic_hook"
version = "0.1.7"
@@ -1095,9 +1119,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
version = "0.9.18"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
@@ -3524,9 +3548,9 @@ checksum = "224484c5d09285a7b8cb0a0c117e847ebd14cb6e4470ecf68cdb89c503b0edb9"
[[package]]
name = "mongodb"
version = "3.5.1"
version = "3.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "803dd859e8afa084c255a8effd8000ff86f7c8076a50cd6d8c99e8f3496f75c2"
checksum = "1ef2c933617431ad0246fb5b43c425ebdae18c7f7259c87de0726d93b0e7e91b"
dependencies = [
"base64",
"bitflags",
@@ -3570,9 +3594,9 @@ dependencies = [
[[package]]
name = "mongodb-internal-macros"
version = "3.5.1"
version = "3.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a973ef3dd3dbc6f6e65bbdecfd9ec5e781b9e7493b0f369a7c62e35d8e5ae2c8"
checksum = "9e5758dc828eb2d02ec30563cba365609d56ddd833190b192beaee2b475a7bb3"
dependencies = [
"macro_magic",
"proc-macro2",
@@ -4174,7 +4198,7 @@ version = "3.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "219cb19e96be00ab2e37d6e299658a0cfa83e52429179969b0f0121b4ac46983"
dependencies = [
"toml_edit",
"toml_edit 0.23.10+spec-1.0.0",
]
[[package]]
@@ -4259,9 +4283,9 @@ dependencies = [
[[package]]
name = "quinn-proto"
version = "0.11.14"
version = "0.11.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "434b42fec591c96ef50e21e886936e66d3cc3f737104fdb9b737c40ffb94c098"
checksum = "4fcb935c5bec503c2f0e306bdd3e58bb9029dcb14fa8d9ac76e3a5256ac0763e"
dependencies = [
"bytes",
"getrandom 0.3.4",
@@ -4699,9 +4723,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.9"
version = "0.103.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7df23109aa6c1567d1c575b9952556388da57401e4ace1d15f79eedad0d8f53"
checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e"
dependencies = [
"ring",
"rustls-pki-types",
@@ -4973,6 +4997,15 @@ dependencies = [
"syn",
]
[[package]]
name = "serde_spanned"
version = "0.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
dependencies = [
"serde",
]
[[package]]
name = "serde_urlencoded"
version = "0.7.1"
@@ -5171,7 +5204,7 @@ version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1c97747dbf44bb1ca44a561ece23508e99cb592e862f22222dcf42f51d1e451"
dependencies = [
"heck 0.5.0",
"heck 0.4.1",
"proc-macro2",
"quote",
"syn",
@@ -5787,6 +5820,27 @@ dependencies = [
"tokio",
]
[[package]]
name = "toml"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
dependencies = [
"serde",
"serde_spanned",
"toml_datetime 0.6.11",
"toml_edit 0.22.27",
]
[[package]]
name = "toml_datetime"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
dependencies = [
"serde",
]
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
@@ -5796,6 +5850,20 @@ dependencies = [
"serde_core",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap 2.13.0",
"serde",
"serde_spanned",
"toml_datetime 0.6.11",
"toml_write",
"winnow",
]
[[package]]
name = "toml_edit"
version = "0.23.10+spec-1.0.0"
@@ -5803,7 +5871,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84c8b9f757e028cee9fa244aea147aab2a9ec09d5325a9b01e0a49730c2b5269"
dependencies = [
"indexmap 2.13.0",
"toml_datetime",
"toml_datetime 0.7.5+spec-1.1.0",
"toml_parser",
"winnow",
]
@@ -5817,6 +5885,12 @@ dependencies = [
"winnow",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "tonic"
version = "0.12.3"
@@ -6063,6 +6137,18 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "tramiton-core"
version = "0.4.0"
source = "git+ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git?tag=v0.4.0#e3dc1bf7027a2f6d7b1fe43043d6dfa887ce4af3"
dependencies = [
"serde",
"tempfile",
"thiserror 1.0.69",
"toml",
"walkdir",
]
[[package]]
name = "tree-sitter"
version = "0.24.7"
+3
View File
@@ -6,6 +6,7 @@ members = [
"compliance-graph",
"compliance-dast",
"compliance-mcp",
"compliance-smoke",
]
resolver = "2"
@@ -33,3 +34,5 @@ zip = { version = "2", features = ["aes-crypto", "deflate"] }
dashmap = "6"
tokio-stream = { version = "0.1", features = ["sync"] }
aes-gcm = "0.10"
rand = "0.9"
base64 = "0.22"
+16 -1
View File
@@ -2,7 +2,16 @@ FROM rust:1.94-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release -p compliance-agent
# compliance-agent depends on the private tramiton-core git repo. Authenticate
# the fetch with a PAT passed as a BuildKit secret (never baked into a layer).
# Build with: DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN ...
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release -p compliance-agent
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 git curl python3 python3-pip npm golang-go php-cli && rm -rf /var/lib/apt/lists/*
@@ -33,9 +42,15 @@ RUN pip3 install --break-system-packages ruff
COPY --from=builder /app/target/release/compliance-agent /usr/local/bin/compliance-agent
# Copy documentation for the help chat assistant
COPY --from=builder /app/README.md /app/README.md
COPY --from=builder /app/docs /app/docs
ENV HELP_DOCS_PATH=/app
# Ensure SSH key directory exists
RUN mkdir -p /data/compliance-scanner/ssh
EXPOSE 3001 3002
ENTRYPOINT ["compliance-agent"]
+12 -2
View File
@@ -1,13 +1,22 @@
FROM rust:1.94-bookworm AS builder
RUN cargo install dioxus-cli --version 0.7.3
RUN cargo install dioxus-cli --version 0.7.3 --locked
ARG DOCS_URL=/docs
WORKDIR /app
COPY . .
ENV DOCS_URL=${DOCS_URL}
RUN dx build --release --package compliance-dashboard
# compliance-agent (a workspace member) depends on the private tramiton-core git
# repo, so the workspace resolve needs it even to build the dashboard.
# Authenticate the fetch with a PAT passed as a BuildKit secret.
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true dx build --release --package compliance-dashboard
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 && rm -rf /var/lib/apt/lists/*
@@ -20,3 +29,4 @@ ENV IP=0.0.0.0
EXPOSE 8080
ENTRYPOINT ["./compliance-dashboard"]
+1
View File
@@ -12,3 +12,4 @@ RUN rm /etc/nginx/conf.d/default.conf
COPY docs/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /app/.vitepress/dist /usr/share/nginx/html
EXPOSE 80
+11 -1
View File
@@ -2,7 +2,16 @@ FROM rust:1.94-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release -p compliance-mcp
# compliance-agent (a workspace member) depends on the private tramiton-core git
# repo, so the workspace resolve needs it even to build the mcp binary.
# Authenticate the fetch with a PAT passed as a BuildKit secret.
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release -p compliance-mcp
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 && rm -rf /var/lib/apt/lists/*
@@ -14,3 +23,4 @@ EXPOSE 8090
ENV MCP_PORT=8090
ENTRYPOINT ["compliance-mcp"]
+77 -43
View File
@@ -28,9 +28,9 @@
## About
Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira) with evidence and remediation suggestions, reviews pull requests, and exposes a Dioxus-based dashboard for visualization.
Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira/Gitea) with evidence and remediation suggestions, reviews pull requests with multi-pass LLM analysis, runs autonomous penetration tests, and exposes a Dioxus-based dashboard for visualization.
> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation.
> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation with multi-language awareness.
## Features
@@ -41,31 +41,38 @@ Compliance Scanner is an autonomous agent that continuously monitors git reposit
| **CVE Monitoring** | OSV.dev batch queries, NVD CVSS enrichment, SearXNG context |
| **GDPR Patterns** | Detect PII logging, missing consent, hardcoded retention, missing deletion |
| **OAuth Patterns** | Detect implicit grant, missing PKCE, token in localStorage, token in URLs |
| **LLM Triage** | Confidence scoring via LiteLLM to filter false positives |
| **Issue Creation** | Auto-create issues in GitHub, GitLab, or Jira with code evidence |
| **PR Reviews** | Post security review comments on pull requests |
| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, and statistics |
| **Webhooks** | GitHub (HMAC-SHA256) and GitLab webhook receivers for push/PR events |
| **LLM Triage** | Multi-language-aware confidence scoring (Rust, Python, Go, Java, Ruby, PHP, C++) |
| **Issue Creation** | Auto-create issues in GitHub, GitLab, Jira, or Gitea with dedup via fingerprints |
| **PR Reviews** | Multi-pass security review (logic, security, convention, complexity) with dedup |
| **DAST Scanning** | Black-box security testing with endpoint discovery and parameter fuzzing |
| **AI Pentesting** | Autonomous LLM-orchestrated penetration testing with encrypted reports |
| **Code Graph** | Interactive code knowledge graph with impact analysis |
| **AI Chat (RAG)** | Natural language Q&A grounded in repository source code |
| **Help Assistant** | Documentation-grounded help chat accessible from every dashboard page |
| **MCP Server** | Expose live security data to Claude, Cursor, and other AI tools |
| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, DAST, pentest, and graph |
| **Webhooks** | GitHub, GitLab, and Gitea webhook receivers for push/PR events |
| **Finding Dedup** | SHA-256 fingerprint dedup for SAST, CWE-based dedup for DAST findings |
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Cargo Workspace │
├──────────────┬──────────────────┬───────────────────────────┤
│ compliance- │ compliance- │ compliance-
│ core │ agent │ dashboard
(lib)(bin) │ (bin, Dioxus 0.7.3)
│ │
Models │ Scan Pipeline │ Fullstack Web UI
TraitsLLM Client │ Server Functions
ConfigIssue Trackers Charts + Tables
Errors │ Scheduler │ Settings Page
│ │ REST API │ │
│ │ Webhooks │ │
└──────────────┴──────────────────┴───────────────────────────┘
MongoDB (shared)
┌──────────────────────────────────────────────────────────────────────────
Cargo Workspace
├──────────────┬──────────────────┬──────────────┬──────────┬─────────────┤
│ compliance- │ compliance- │ compliance- │ complian-│ compliance-
│ core (lib) │ agent (bin) │ dashboard │ ce-graph │ mcp (bin)
│ (bin) │ (lib) │
Models │ Scan Pipeline │ Dioxus 0.7 │ Tree- │ MCP Server
Traits │ LLM Client │ Fullstack UI │ sitter │ Live data
ConfigIssue Trackers │ Help Chat │ Graph │ for AI
ErrorsPentest Engine Server Fns │ Embedds │ tools
│ DAST Tools │ │ RAG │
│ │ REST API │ │ │
│ │ Webhooks │ │ │
└──────────────┴──────────────────┴──────────────┴──────────┴─────────────┘
MongoDB (shared)
```
## Scan Pipeline (7 Stages)
@@ -84,11 +91,16 @@ Compliance Scanner is an autonomous agent that continuously monitors git reposit
|-------|-----------|
| Shared Library | `compliance-core` -- models, traits, config |
| Agent | Axum REST API, git2, tokio-cron-scheduler, Semgrep, Syft |
| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS |
| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS 4 |
| Code Graph | `compliance-graph` -- tree-sitter parsing, embeddings, RAG |
| MCP Server | `compliance-mcp` -- Model Context Protocol for AI tools |
| DAST | `compliance-dast` -- dynamic application security testing |
| Database | MongoDB with typed collections |
| LLM | LiteLLM (OpenAI-compatible API) |
| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3) |
| LLM | LiteLLM (OpenAI-compatible API for chat, triage, embeddings) |
| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3), Gitea |
| CVE Sources | OSV.dev, NVD, SearXNG |
| Auth | Keycloak (OAuth2/PKCE, SSO) |
| Browser Automation | Chromium (headless, for pentesting and PDF generation) |
## Getting Started
@@ -151,20 +163,35 @@ The agent exposes a REST API on port 3001:
| `GET` | `/api/v1/sbom` | List dependencies |
| `GET` | `/api/v1/issues` | List cross-tracker issues |
| `GET` | `/api/v1/scan-runs` | Scan execution history |
| `GET` | `/api/v1/graph/:repo_id` | Code knowledge graph |
| `POST` | `/api/v1/graph/:repo_id/build` | Trigger graph build |
| `GET` | `/api/v1/dast/targets` | List DAST targets |
| `POST` | `/api/v1/dast/targets` | Add DAST target |
| `GET` | `/api/v1/dast/findings` | List DAST findings |
| `POST` | `/api/v1/chat/:repo_id` | RAG-powered code chat |
| `POST` | `/api/v1/help/chat` | Documentation-grounded help chat |
| `POST` | `/api/v1/pentest/sessions` | Create pentest session |
| `POST` | `/api/v1/pentest/sessions/:id/export` | Export encrypted pentest report |
| `POST` | `/webhook/github` | GitHub webhook (HMAC-SHA256) |
| `POST` | `/webhook/gitlab` | GitLab webhook (token verify) |
| `POST` | `/webhook/gitea` | Gitea webhook |
## Dashboard Pages
| Page | Description |
|------|-------------|
| **Overview** | Stat cards, severity distribution chart |
| **Repositories** | Add/manage tracked repos, trigger scans |
| **Findings** | Filterable table by severity, type, status |
| **Overview** | Stat cards, severity distribution, AI chat cards, MCP status |
| **Repositories** | Add/manage tracked repos, trigger scans, webhook config |
| **Findings** | Filterable table by severity, type, status, scanner |
| **Finding Detail** | Code evidence, remediation, suggested fix, linked issue |
| **SBOM** | Dependency inventory with vulnerability badges |
| **Issues** | Cross-tracker view (GitHub + GitLab + Jira) |
| **Settings** | Configure LiteLLM, tracker tokens, SearXNG URL |
| **SBOM** | Dependency inventory with vulnerability badges, license summary |
| **Issues** | Cross-tracker view (GitHub + GitLab + Jira + Gitea) |
| **Code Graph** | Interactive architecture visualization, impact analysis |
| **AI Chat** | RAG-powered Q&A about repository code |
| **DAST** | Dynamic scanning targets, findings, and scan history |
| **Pentest** | AI-driven pentest sessions, attack chain visualization |
| **MCP Servers** | Model Context Protocol server management |
| **Help Chat** | Floating assistant (available on every page) for product Q&A |
## Project Structure
@@ -173,19 +200,24 @@ compliance-scanner/
├── compliance-core/ Shared library (models, traits, config, errors)
├── compliance-agent/ Agent daemon (pipeline, LLM, trackers, API, webhooks)
│ └── src/
│ ├── pipeline/ 7-stage scan pipeline
│ ├── llm/ LiteLLM client, triage, descriptions, fixes, PR review
│ ├── trackers/ GitHub, GitLab, Jira integrations
│ ├── api/ REST API (Axum)
── webhooks/ GitHub + GitLab webhook receivers
│ ├── pipeline/ 7-stage scan pipeline, dedup, PR reviews, code review
│ ├── llm/ LiteLLM client, triage, descriptions, fixes, review prompts
│ ├── trackers/ GitHub, GitLab, Jira, Gitea integrations
│ ├── pentest/ AI-driven pentest orchestrator, tools, reports
── rag/ RAG pipeline, chunking, embedding
│ ├── api/ REST API (Axum), help chat
│ └── webhooks/ GitHub, GitLab, Gitea webhook receivers
├── compliance-dashboard/ Dioxus fullstack dashboard
│ └── src/
│ ├── components/ Reusable UI components
│ ├── infrastructure/ Server functions, DB, config
│ └── pages/ Full page views
│ ├── components/ Reusable UI (sidebar, help chat, attack chain, etc.)
│ ├── infrastructure/ Server functions, DB, config, auth
│ └── pages/ Full page views (overview, DAST, pentest, graph, etc.)
├── compliance-graph/ Code knowledge graph (tree-sitter, embeddings, RAG)
├── compliance-dast/ Dynamic application security testing
├── compliance-mcp/ Model Context Protocol server
├── docs/ VitePress documentation site
├── assets/ Static assets (CSS, icons)
── styles/ Tailwind input stylesheet
└── bin/ Dashboard binary entrypoint
── styles/ Tailwind input stylesheet
```
## External Services
@@ -193,10 +225,12 @@ compliance-scanner/
| Service | Purpose | Default URL |
|---------|---------|-------------|
| MongoDB | Persistence | `mongodb://localhost:27017` |
| LiteLLM | LLM proxy for triage and generation | `http://localhost:4000` |
| LiteLLM | LLM proxy (chat, triage, embeddings) | `http://localhost:4000` |
| SearXNG | CVE context search | `http://localhost:8888` |
| Keycloak | Authentication (OAuth2/PKCE, SSO) | `http://localhost:8080` |
| Semgrep | SAST scanning | CLI tool |
| Syft | SBOM generation | CLI tool |
| Chromium | Headless browser (pentesting, PDF) | Managed via Docker |
---
+20 -2
View File
@@ -7,9 +7,14 @@ edition = "2021"
workspace = true
[dependencies]
compliance-core = { workspace = true, features = ["mongodb", "telemetry"] }
compliance-core = { workspace = true, features = ["mongodb", "telemetry", "axum"] }
compliance-graph = { path = "../compliance-graph" }
compliance-dast = { path = "../compliance-dast" }
# Native firmware build/target detection for bare-metal & RTOS artifacts.
# Same-company IP, used directly (not via CLI) so the whole tramiton suite is
# available to the onboarding classifier. NOTE: CI must be able to fetch this
# private repo (see the git-auth step in .gitea/workflows/ci.yml).
tramiton-core = { git = "ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git", tag = "v0.4.0" }
serde = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true }
@@ -25,7 +30,7 @@ uuid = { workspace = true }
secrecy = { workspace = true }
regex = { workspace = true }
axum = "0.8"
tower-http = { version = "0.6", features = ["cors", "trace"] }
tower-http = { version = "0.6", features = ["cors", "trace", "set-header"] }
git2 = "0.20"
octocrab = "0.44"
tokio-cron-scheduler = "0.13"
@@ -42,3 +47,16 @@ tokio-tungstenite = { version = "0.26", features = ["rustls-tls-webpki-roots"] }
futures-core = "0.3"
dashmap = { workspace = true }
tokio-stream = { workspace = true }
rand = { workspace = true }
[dev-dependencies]
compliance-core = { workspace = true, features = ["mongodb", "axum"] }
tower = { version = "0.5", features = ["util"] }
reqwest = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true }
mongodb = { workspace = true }
uuid = { workspace = true }
secrecy = { workspace = true }
axum = "0.8"
tower-http = { version = "0.6", features = ["cors"] }
+47 -20
View File
@@ -6,7 +6,7 @@ use tokio::sync::{broadcast, watch, Semaphore};
use compliance_core::models::pentest::PentestEvent;
use compliance_core::AgentConfig;
use crate::database::Database;
use crate::database::DatabasePool;
use crate::llm::LlmClient;
use crate::pipeline::orchestrator::PipelineOrchestrator;
@@ -16,7 +16,10 @@ const DEFAULT_MAX_CONCURRENT_SESSIONS: usize = 5;
#[derive(Clone)]
pub struct ComplianceAgent {
pub config: AgentConfig,
pub db: Database,
/// Per-tenant Mongo broker. Every code path must obtain a
/// tenant-scoped [`crate::database::Database`] from this pool —
/// there is no single shared database any more.
pub db_pool: DatabasePool,
pub llm: Arc<LlmClient>,
pub http: reqwest::Client,
/// Per-session broadcast senders for SSE streaming.
@@ -28,18 +31,23 @@ pub struct ComplianceAgent {
}
impl ComplianceAgent {
pub fn new(config: AgentConfig, db: Database) -> Self {
pub fn new(config: AgentConfig, db_pool: DatabasePool) -> Self {
let llm = Arc::new(LlmClient::new(
config.litellm_url.clone(),
config.litellm_api_key.clone(),
config.litellm_model.clone(),
config.litellm_embed_model.clone(),
));
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self {
config,
db,
db_pool,
llm,
http: reqwest::Client::new(),
http,
session_streams: Arc::new(DashMap::new()),
session_pause: Arc::new(DashMap::new()),
session_semaphore: Arc::new(Semaphore::new(DEFAULT_MAX_CONCURRENT_SESSIONS)),
@@ -48,28 +56,51 @@ impl ComplianceAgent {
pub async fn run_scan(
&self,
tenant_id: &str,
repo_id: &str,
trigger: compliance_core::models::ScanTrigger,
) -> Result<(), crate::error::AgentError> {
let orchestrator = PipelineOrchestrator::new(
self.config.clone(),
self.db.clone(),
self.llm.clone(),
self.http.clone(),
);
orchestrator.run(repo_id, trigger).await
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
if self.config.unified_pipeline {
orchestrator.run_target(repo_id, trigger).await
} else {
orchestrator.run(repo_id, trigger).await
}
}
/// Run a scan for an onboarded target through the unified pipeline,
/// unconditionally.
///
/// Unlike [`Self::run_scan`], this does *not* consult the
/// `unified_pipeline` transition flag: the caller (the `/targets/{id}/scan`
/// endpoint) operates on `onboarded_targets` by construction, so it must
/// always dispatch to `run_target` regardless of how the legacy paths
/// (scheduler, webhooks, `/repositories/{id}/scan`) are configured.
pub async fn run_target_scan(
&self,
tenant_id: &str,
target_id: &str,
trigger: compliance_core::models::ScanTrigger,
) -> Result<(), crate::error::AgentError> {
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator.run_target(target_id, trigger).await
}
/// Run a PR review: scan the diff and post review comments.
pub async fn run_pr_review(
&self,
tenant_id: &str,
repo_id: &str,
pr_number: u64,
base_sha: &str,
head_sha: &str,
) -> Result<(), crate::error::AgentError> {
let repo = self
.db
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let repo = db
.repositories()
.find_one(mongodb::bson::doc! {
"_id": mongodb::bson::oid::ObjectId::parse_str(repo_id)
@@ -80,12 +111,8 @@ impl ComplianceAgent {
crate::error::AgentError::Other(format!("Repository {repo_id} not found"))
})?;
let orchestrator = PipelineOrchestrator::new(
self.config.clone(),
self.db.clone(),
self.llm.clone(),
self.http.clone(),
);
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator
.run_pr_review(&repo, repo_id, pr_number, base_sha, head_sha)
.await
-113
View File
@@ -1,113 +0,0 @@
use std::sync::Arc;
use axum::{
extract::Request,
middleware::Next,
response::{IntoResponse, Response},
};
use jsonwebtoken::{decode, decode_header, jwk::JwkSet, DecodingKey, Validation};
use reqwest::StatusCode;
use serde::Deserialize;
use tokio::sync::RwLock;
/// Cached JWKS from Keycloak for token validation.
#[derive(Clone)]
pub struct JwksState {
pub jwks: Arc<RwLock<Option<JwkSet>>>,
pub jwks_url: String,
}
#[derive(Debug, Deserialize)]
struct Claims {
#[allow(dead_code)]
sub: String,
}
const PUBLIC_ENDPOINTS: &[&str] = &["/api/v1/health"];
/// Middleware that validates Bearer JWT tokens against Keycloak's JWKS.
///
/// Skips validation for health check endpoints.
/// If `JwksState` is not present as an extension (keycloak not configured),
/// all requests pass through.
pub async fn require_jwt_auth(request: Request, next: Next) -> Response {
let path = request.uri().path();
if PUBLIC_ENDPOINTS.contains(&path) {
return next.run(request).await;
}
let jwks_state = match request.extensions().get::<JwksState>() {
Some(s) => s.clone(),
None => return next.run(request).await,
};
let auth_header = match request.headers().get("authorization") {
Some(h) => h,
None => return (StatusCode::UNAUTHORIZED, "Missing authorization header").into_response(),
};
let token = match auth_header.to_str() {
Ok(s) if s.starts_with("Bearer ") => &s[7..],
_ => return (StatusCode::UNAUTHORIZED, "Invalid authorization header").into_response(),
};
match validate_token(token, &jwks_state).await {
Ok(()) => next.run(request).await,
Err(e) => {
tracing::warn!("JWT validation failed: {e}");
(StatusCode::UNAUTHORIZED, "Invalid token").into_response()
}
}
}
async fn validate_token(token: &str, state: &JwksState) -> Result<(), String> {
let header = decode_header(token).map_err(|e| format!("failed to decode JWT header: {e}"))?;
let kid = header
.kid
.ok_or_else(|| "JWT missing kid header".to_string())?;
let jwks = fetch_or_get_jwks(state).await?;
let jwk = jwks
.keys
.iter()
.find(|k| k.common.key_id.as_deref() == Some(&kid))
.ok_or_else(|| "no matching key found in JWKS".to_string())?;
let decoding_key =
DecodingKey::from_jwk(jwk).map_err(|e| format!("failed to create decoding key: {e}"))?;
let mut validation = Validation::new(header.alg);
validation.validate_exp = true;
validation.validate_aud = false;
decode::<Claims>(token, &decoding_key, &validation)
.map_err(|e| format!("token validation failed: {e}"))?;
Ok(())
}
async fn fetch_or_get_jwks(state: &JwksState) -> Result<JwkSet, String> {
{
let cached = state.jwks.read().await;
if let Some(ref jwks) = *cached {
return Ok(jwks.clone());
}
}
let resp = reqwest::get(&state.jwks_url)
.await
.map_err(|e| format!("failed to fetch JWKS: {e}"))?;
let jwks: JwkSet = resp
.json()
.await
.map_err(|e| format!("failed to parse JWKS: {e}"))?;
let mut cached = state.jwks.write().await;
*cached = Some(jwks.clone());
Ok(jwks)
}
+115
View File
@@ -0,0 +1,115 @@
//! Cross-tenant admin endpoints (`/api/v1/admin/*`).
//!
//! Operator-only. Auth is a **static bearer token** (`ADMIN_API_TOKEN`
//! env on the agent) — explicitly NOT a Keycloak JWT, because the
//! whole point of these endpoints is to operate ACROSS tenants. A
//! customer JWT (which always carries a single tenant_id) has no
//! business mounting them.
//!
//! Routes are only registered when `ADMIN_API_TOKEN` is set. With no
//! token, the endpoints don't exist at all (404), which is a stronger
//! guarantee than "401 if you guess the path".
//!
//! Operations:
//! - `GET /api/v1/admin/tenants` — list tenant DBs
//! - `DELETE /api/v1/admin/tenants/{tenant_id}` — GDPR delete
//!
//! Tenant ids in URLs are passed as-is to `DatabasePool::drop_tenant`,
//! which sanitises them the same way it does for creation. Listing
//! returns the raw DB names from `list_tenant_db_names` — operators
//! can reverse-derive the tenant_id from the prefix.
use axum::extract::{Extension, Path, Request};
use axum::http::{header, StatusCode};
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
use axum::Json;
use secrecy::ExposeSecret;
use serde::Serialize;
use super::dto::AgentExt;
#[derive(Serialize)]
pub struct ListTenantDbsResponse {
pub tenant_db_names: Vec<String>,
}
#[tracing::instrument(skip_all)]
pub async fn list_tenant_dbs(
Extension(agent): AgentExt,
) -> Result<Json<ListTenantDbsResponse>, StatusCode> {
let names = agent.db_pool.list_tenant_db_names().await.map_err(|e| {
tracing::error!("admin: list_tenant_db_names failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
Ok(Json(ListTenantDbsResponse {
tenant_db_names: names,
}))
}
#[tracing::instrument(skip_all, fields(tenant_id = %tenant_id))]
pub async fn drop_tenant_db(
Extension(agent): AgentExt,
Path(tenant_id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
agent.db_pool.drop_tenant(&tenant_id).await.map_err(|e| {
tracing::error!("admin: drop_tenant failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
Ok(Json(serde_json::json!({ "status": "dropped" })))
}
/// Constant-time-ish comparison of the configured admin token against
/// the incoming bearer. Uses `subtle`-style byte equality so timing
/// attacks can't probe the token character by character.
fn tokens_eq(a: &str, b: &str) -> bool {
if a.len() != b.len() {
return false;
}
let mut diff = 0u8;
for (x, y) in a.bytes().zip(b.bytes()) {
diff |= x ^ y;
}
diff == 0
}
/// Middleware enforcing the static `ADMIN_API_TOKEN`. Mounted only on
/// the admin sub-router, so this never runs on customer routes.
pub async fn require_admin_token(
Extension(agent): AgentExt,
request: Request,
next: Next,
) -> Response {
let Some(expected) = agent.config.admin_api_token.as_ref() else {
// Belt-and-braces — if the routes were somehow mounted without
// a token configured, refuse rather than no-op-pass.
return (StatusCode::NOT_FOUND, "admin disabled").into_response();
};
let presented = request
.headers()
.get(header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|s| s.strip_prefix("Bearer "))
.map(|s| s.trim());
let Some(presented) = presented.filter(|s| !s.is_empty()) else {
return (StatusCode::UNAUTHORIZED, "Missing bearer token").into_response();
};
if !tokens_eq(presented, expected.expose_secret()) {
return (StatusCode::UNAUTHORIZED, "Invalid admin token").into_response();
}
next.run(request).await
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tokens_eq_basic() {
assert!(tokens_eq("abc", "abc"));
assert!(!tokens_eq("abc", "abd"));
assert!(!tokens_eq("abc", "abcd"));
assert!(!tokens_eq("", "x"));
assert!(tokens_eq("", ""));
}
}
+37 -30
View File
@@ -7,11 +7,13 @@ use mongodb::bson::doc;
use compliance_core::models::chat::{ChatRequest, ChatResponse, SourceReference};
use compliance_core::models::embedding::EmbeddingBuildRun;
use compliance_core::tenant_ctx::TenantCtx;
use compliance_graph::graph::embedding_store::EmbeddingStore;
use crate::agent::ComplianceAgent;
use crate::rag::pipeline::RagPipeline;
use super::dto::tenant_db;
use super::ApiResponse;
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -20,10 +22,12 @@ type AgentExt = Extension<Arc<ComplianceAgent>>;
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn chat(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
Json(req): Json<ChatRequest>,
) -> Result<Json<ApiResponse<ChatResponse>>, StatusCode> {
let pipeline = RagPipeline::new(agent.llm.clone(), agent.db.inner());
let db = tenant_db(&agent, &tenant).await?;
let pipeline = RagPipeline::new(agent.llm.clone(), db.inner());
// Step 1: Embed the user's message
let query_vectors = agent
@@ -90,10 +94,13 @@ pub async fn chat(
};
let system_prompt = format!(
"You are an expert code assistant for a software repository. \
Answer the user's question based on the code context below. \
Reference specific files and functions when relevant. \
If the context doesn't contain enough information, say so.\n\n\
"You are a code assistant for this repository. Answer questions using the code context below.\n\n\
Rules:\n\
- Reference specific files, functions, and line numbers\n\
- Show code snippets when they help explain the answer\n\
- If the context is insufficient, say what's missing rather than guessing\n\
- Be concise — lead with the answer, then explain if needed\n\
- For security questions, note relevant CWEs and link to the finding if one exists\n\n\
## Code Context\n\n{code_context}"
);
@@ -130,12 +137,15 @@ pub async fn chat(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn build_embeddings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
// Resolve the tenant DB up front so we can move it into the spawn;
// the JWT/dev context isn't available inside detached tasks.
let db = tenant_db(&agent, &tenant).await?;
let agent_clone = (*agent).clone();
tokio::spawn(async move {
let repo = match agent_clone
.db
let repo = match db
.repositories()
.find_one(doc! { "_id": mongodb::bson::oid::ObjectId::parse_str(&repo_id).ok() })
.await
@@ -148,8 +158,7 @@ pub async fn build_embeddings(
};
// Get latest graph build
let build = match agent_clone
.db
let build = match db
.graph_builds()
.find_one(doc! { "repo_id": &repo_id })
.sort(doc! { "started_at": -1 })
@@ -168,26 +177,22 @@ pub async fn build_embeddings(
.unwrap_or_else(|| "unknown".to_string());
// Get nodes
let nodes: Vec<compliance_core::models::graph::CodeNode> = match agent_clone
.db
.graph_nodes()
.find(doc! { "repo_id": &repo_id })
.await
{
Ok(cursor) => {
use futures_util::StreamExt;
let mut items = Vec::new();
let mut cursor = cursor;
while let Some(Ok(item)) = cursor.next().await {
items.push(item);
let nodes: Vec<compliance_core::models::graph::CodeNode> =
match db.graph_nodes().find(doc! { "repo_id": &repo_id }).await {
Ok(cursor) => {
use futures_util::StreamExt;
let mut items = Vec::new();
let mut cursor = cursor;
while let Some(Ok(item)) = cursor.next().await {
items.push(item);
}
items
}
items
}
Err(e) => {
tracing::error!("[{repo_id}] Failed to fetch nodes: {e}");
return;
}
};
Err(e) => {
tracing::error!("[{repo_id}] Failed to fetch nodes: {e}");
return;
}
};
let creds = crate::pipeline::git::RepoCredentials {
ssh_key_path: Some(agent_clone.config.ssh_key_path.clone()),
@@ -204,7 +209,7 @@ pub async fn build_embeddings(
}
};
let pipeline = RagPipeline::new(agent_clone.llm.clone(), agent_clone.db.inner());
let pipeline = RagPipeline::new(agent_clone.llm.clone(), db.inner());
match pipeline
.build_embeddings(&repo_id, &repo_path, &graph_build_id, &nodes)
.await
@@ -231,9 +236,11 @@ pub async fn build_embeddings(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn embedding_status(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<Option<EmbeddingBuildRun>>>, StatusCode> {
let store = EmbeddingStore::new(agent.db.inner());
let db = tenant_db(&agent, &tenant).await?;
let store = EmbeddingStore::new(db.inner());
let build = store.get_latest_build(&repo_id).await.map_err(|e| {
tracing::error!("Failed to get embedding status: {e}");
StatusCode::INTERNAL_SERVER_ERROR
+20 -11
View File
@@ -7,9 +7,11 @@ use mongodb::bson::doc;
use serde::Deserialize;
use compliance_core::models::dast::{DastFinding, DastScanRun, DastTarget, DastTargetType};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -45,9 +47,11 @@ fn default_rate_limit() -> u32 {
#[tracing::instrument(skip_all)]
pub async fn list_targets(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<DastTarget>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_targets()
@@ -80,6 +84,7 @@ pub async fn list_targets(
#[tracing::instrument(skip_all)]
pub async fn add_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<AddTargetRequest>,
) -> Result<Json<ApiResponse<DastTarget>>, StatusCode> {
let mut target = DastTarget::new(req.name, req.base_url, req.target_type);
@@ -89,9 +94,8 @@ pub async fn add_target(
target.rate_limit = req.rate_limit;
target.allow_destructive = req.allow_destructive;
agent
.db
.dast_targets()
let db = tenant_db(&agent, &tenant).await?;
db.dast_targets()
.insert_one(&target)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
@@ -107,19 +111,19 @@ pub async fn add_target(
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn trigger_scan(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let target = agent
.db
let target = db
.dast_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
let db = agent.db.clone();
tokio::spawn(async move {
let orchestrator = compliance_dast::DastOrchestrator::new(100);
match orchestrator.run_scan(&target, Vec::new()).await {
@@ -147,9 +151,11 @@ pub async fn trigger_scan(
#[tracing::instrument(skip_all)]
pub async fn list_scan_runs(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<DastScanRun>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_scan_runs()
@@ -183,9 +189,11 @@ pub async fn list_scan_runs(
#[tracing::instrument(skip_all)]
pub async fn list_findings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<DastFinding>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_findings()
@@ -219,12 +227,13 @@ pub async fn list_findings(
#[tracing::instrument(skip_all, fields(finding_id = %id))]
pub async fn get_finding(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<DastFinding>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let finding = agent
.db
let finding = db
.dast_findings()
.find_one(doc! { "_id": oid })
.await
+21
View File
@@ -180,6 +180,27 @@ pub struct SbomVersionDiff {
pub(crate) type AgentExt = axum::extract::Extension<std::sync::Arc<crate::agent::ComplianceAgent>>;
pub(crate) type ApiResult<T> = Result<axum::Json<ApiResponse<T>>, axum::http::StatusCode>;
/// Resolve a tenant-scoped [`Database`] from the request's
/// [`TenantContext`] (inserted by the M7.1 JWT middleware, or by the
/// dev fallback in unsecured environments). The pool ensures the
/// tenant's indexes idempotently.
///
/// Returns 500 on the rare path where Mongo refuses the database
/// handle — the M7.1 auth/status middleware already rejects every
/// other failure mode with 4xx before we get here.
pub(crate) async fn tenant_db(
agent: &crate::agent::ComplianceAgent,
tenant: &compliance_core::tenant_ctx::TenantCtx,
) -> Result<crate::database::Database, axum::http::StatusCode> {
agent.db_pool.for_tenant(&tenant.0).await.map_err(|e| {
tracing::error!(
tenant_id = %tenant.0.tenant_id,
"Failed to acquire tenant database: {e}"
);
axum::http::StatusCode::INTERNAL_SERVER_ERROR
})
}
pub(crate) async fn collect_cursor_async<T: serde::de::DeserializeOwned + Unpin + Send>(
mut cursor: mongodb::Cursor<T>,
) -> Vec<T> {
+16 -11
View File
@@ -5,13 +5,16 @@ use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::Finding;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all, fields(repo_id = ?filter.repo_id, severity = ?filter.severity, scan_type = ?filter.scan_type))]
pub async fn list_findings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(filter): Query<FindingsFilter>,
) -> ApiResult<Vec<Finding>> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let mut query = doc! {};
if let Some(repo_id) = &filter.repo_id {
query.insert("repo_id", repo_id);
@@ -81,11 +84,12 @@ pub async fn list_findings(
#[tracing::instrument(skip_all, fields(finding_id = %id))]
pub async fn get_finding(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<Finding>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let finding = agent
.db
let db = tenant_db(&agent, &tenant).await?;
let finding = db
.findings()
.find_one(doc! { "_id": oid })
.await
@@ -102,14 +106,14 @@ pub async fn get_finding(
#[tracing::instrument(skip_all, fields(finding_id = %id))]
pub async fn update_finding_status(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateStatusRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
agent
.db
.findings()
db.findings()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "status": &req.status, "updated_at": mongodb::bson::DateTime::now() } },
@@ -123,6 +127,7 @@ pub async fn update_finding_status(
#[tracing::instrument(skip_all)]
pub async fn bulk_update_finding_status(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<BulkUpdateStatusRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oids: Vec<mongodb::bson::oid::ObjectId> = req
@@ -135,8 +140,8 @@ pub async fn bulk_update_finding_status(
return Err(StatusCode::BAD_REQUEST);
}
let result = agent
.db
let db = tenant_db(&agent, &tenant).await?;
let result = db
.findings()
.update_many(
doc! { "_id": { "$in": oids } },
@@ -153,14 +158,14 @@ pub async fn bulk_update_finding_status(
#[tracing::instrument(skip_all)]
pub async fn update_finding_feedback(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateFeedbackRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
agent
.db
.findings()
db.findings()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "developer_feedback": &req.feedback, "updated_at": mongodb::bson::DateTime::now() } },
+24 -10
View File
@@ -7,9 +7,11 @@ use mongodb::bson::doc;
use serde::{Deserialize, Serialize};
use compliance_core::models::graph::{CodeEdge, CodeNode, GraphBuildRun, ImpactAnalysis};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -36,9 +38,11 @@ fn default_search_limit() -> usize {
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_graph(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<GraphData>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Get latest build
let build: Option<GraphBuildRun> = db
@@ -98,9 +102,11 @@ pub async fn get_graph(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_nodes(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<Vec<CodeNode>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let filter = doc! { "repo_id": &repo_id };
let nodes: Vec<CodeNode> = match db.graph_nodes().find(filter).await {
@@ -123,9 +129,11 @@ pub async fn get_nodes(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_communities(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<Vec<CommunityInfo>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let filter = doc! { "repo_id": &repo_id };
let nodes: Vec<CodeNode> = match db.graph_nodes().find(filter).await {
@@ -176,9 +184,11 @@ pub struct CommunityInfo {
#[tracing::instrument(skip_all, fields(repo_id = %repo_id, finding_id = %finding_id))]
pub async fn get_impact(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path((repo_id, finding_id)): Path<(String, String)>,
) -> Result<Json<ApiResponse<Option<ImpactAnalysis>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let filter = doc! { "repo_id": &repo_id, "finding_id": &finding_id };
let impact = db
@@ -198,10 +208,12 @@ pub async fn get_impact(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id, query = %params.q))]
pub async fn search_symbols(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
Query(params): Query<SearchParams>,
) -> Result<Json<ApiResponse<Vec<CodeNode>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Simple text search on qualified_name and name fields
let filter = doc! {
@@ -234,10 +246,12 @@ pub async fn search_symbols(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_file_content(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
Query(params): Query<FileContentParams>,
) -> Result<Json<ApiResponse<FileContent>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Look up the repository to get repo name
let repo = db
@@ -296,12 +310,13 @@ pub struct FileContent {
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn trigger_build(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let agent_clone = (*agent).clone();
tokio::spawn(async move {
let repo = match agent_clone
.db
let repo = match db
.repositories()
.find_one(doc! { "_id": mongodb::bson::oid::ObjectId::parse_str(&repo_id).ok() })
.await
@@ -333,8 +348,7 @@ pub async fn trigger_build(
match engine.build_graph(&repo_path, &repo_id, &graph_build_id) {
Ok((code_graph, build_run)) => {
let store =
compliance_graph::graph::persistence::GraphStore::new(agent_clone.db.inner());
let store = compliance_graph::graph::persistence::GraphStore::new(db.inner());
let _ = store.delete_repo_graph(&repo_id).await;
let _ = store
.store_graph(&build_run, &code_graph.nodes, &code_graph.edges)
+7 -2
View File
@@ -3,6 +3,7 @@ use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::ScanRun;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn health() -> Json<serde_json::Value> {
@@ -10,8 +11,12 @@ pub async fn health() -> Json<serde_json::Value> {
}
#[tracing::instrument(skip_all)]
pub async fn stats_overview(axum::extract::Extension(agent): AgentExt) -> ApiResult<OverviewStats> {
let db = &agent.db;
pub async fn stats_overview(
axum::extract::Extension(agent): AgentExt,
tenant: TenantCtx,
) -> ApiResult<OverviewStats> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let total_repositories = db
.repositories()
@@ -0,0 +1,217 @@
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
use axum::extract::Extension;
use axum::http::StatusCode;
use axum::Json;
use serde::{Deserialize, Serialize};
use walkdir::WalkDir;
use super::dto::{AgentExt, ApiResponse};
// ── DTOs ─────────────────────────────────────────────────────────────────────
#[derive(Debug, Deserialize)]
pub struct HelpChatMessage {
pub role: String,
pub content: String,
}
#[derive(Debug, Deserialize)]
pub struct HelpChatRequest {
pub message: String,
#[serde(default)]
pub history: Vec<HelpChatMessage>,
}
#[derive(Debug, Serialize)]
pub struct HelpChatResponse {
pub message: String,
}
// ── Doc cache ────────────────────────────────────────────────────────────────
static DOC_CONTEXT: OnceLock<String> = OnceLock::new();
/// Walk upward from `start` until we find a directory containing both
/// `README.md` and a `docs/` subdirectory.
fn find_project_root(start: &Path) -> Option<PathBuf> {
let mut current = start.to_path_buf();
loop {
if current.join("README.md").is_file() && current.join("docs").is_dir() {
return Some(current);
}
if !current.pop() {
return None;
}
}
}
/// Read README.md + all docs/**/*.md (excluding node_modules).
fn load_docs(root: &Path) -> String {
let mut parts: Vec<String> = Vec::new();
// Root README first
if let Ok(content) = std::fs::read_to_string(root.join("README.md")) {
parts.push(format!("<!-- file: README.md -->\n{content}"));
}
// docs/**/*.md, skipping node_modules
for entry in WalkDir::new(root.join("docs"))
.follow_links(false)
.into_iter()
.filter_entry(|e| {
!e.path()
.components()
.any(|c| c.as_os_str() == "node_modules")
})
.filter_map(|e| e.ok())
{
let path = entry.path();
if !path.is_file() {
continue;
}
if path
.extension()
.and_then(|s| s.to_str())
.map(|s| !s.eq_ignore_ascii_case("md"))
.unwrap_or(true)
{
continue;
}
let rel = path.strip_prefix(root).unwrap_or(path);
if let Ok(content) = std::fs::read_to_string(path) {
parts.push(format!("<!-- file: {} -->\n{content}", rel.display()));
}
}
if parts.is_empty() {
tracing::warn!(
"help_chat: no documentation files found under {}",
root.display()
);
} else {
tracing::info!(
"help_chat: loaded {} documentation file(s) from {}",
parts.len(),
root.display()
);
}
parts.join("\n\n---\n\n")
}
/// Returns a reference to the cached doc context string, initialised on
/// first call via `OnceLock`.
///
/// Discovery order:
/// 1. `HELP_DOCS_PATH` env var (explicit override)
/// 2. Walk up from the binary location
/// 3. Current working directory
/// 4. Common Docker paths (/app, /opt/compliance-scanner)
fn doc_context() -> &'static str {
DOC_CONTEXT.get_or_init(|| {
// 1. Explicit env var
if let Ok(path) = std::env::var("HELP_DOCS_PATH") {
let p = PathBuf::from(&path);
if p.join("README.md").is_file() || p.join("docs").is_dir() {
tracing::info!("help_chat: loading docs from HELP_DOCS_PATH={path}");
return load_docs(&p);
}
tracing::warn!("help_chat: HELP_DOCS_PATH={path} has no README.md or docs/");
}
// 2. Walk up from binary location
let start = std::env::current_exe()
.ok()
.and_then(|p| p.parent().map(Path::to_path_buf))
.unwrap_or_else(|| PathBuf::from("."));
if let Some(root) = find_project_root(&start) {
return load_docs(&root);
}
// 3. Current working directory
if let Ok(cwd) = std::env::current_dir() {
if let Some(root) = find_project_root(&cwd) {
return load_docs(&root);
}
if cwd.join("README.md").is_file() {
return load_docs(&cwd);
}
}
// 4. Common Docker/deployment paths
for candidate in ["/app", "/opt/compliance-scanner", "/srv/compliance-scanner"] {
let p = PathBuf::from(candidate);
if p.join("README.md").is_file() || p.join("docs").is_dir() {
tracing::info!("help_chat: found docs at {candidate}");
return load_docs(&p);
}
}
tracing::error!(
"help_chat: could not locate project root; doc context will be empty. \
Set HELP_DOCS_PATH to the directory containing README.md and docs/"
);
String::new()
})
}
// ── Handler ──────────────────────────────────────────────────────────────────
/// POST /api/v1/help/chat — Answer questions about the compliance-scanner
/// using the project documentation as grounding context.
#[tracing::instrument(skip_all)]
pub async fn help_chat(
Extension(agent): AgentExt,
Json(req): Json<HelpChatRequest>,
) -> Result<Json<ApiResponse<HelpChatResponse>>, StatusCode> {
let context = doc_context();
let system_prompt = if context.is_empty() {
"You are a helpful assistant for the Compliance Scanner project. \
Answer questions about how to use and configure it. \
No documentation was loaded at startup, so rely on your general knowledge."
.to_string()
} else {
format!(
"You are a helpful assistant for the Compliance Scanner project. \
Answer questions about how to use, configure, and understand it \
using the documentation below as your primary source of truth.\n\n\
Rules:\n\
- Prefer information from the provided docs over general knowledge\n\
- Quote or reference the relevant doc section when it helps\n\
- If the docs do not cover the topic, say so clearly\n\
- Be concise — lead with the answer, then explain if needed\n\
- Use markdown formatting for readability\n\n\
## Project Documentation\n\n{context}"
)
};
let mut messages: Vec<(String, String)> = Vec::with_capacity(req.history.len() + 2);
messages.push(("system".to_string(), system_prompt));
for msg in &req.history {
messages.push((msg.role.clone(), msg.content.clone()));
}
messages.push(("user".to_string(), req.message));
let response_text = agent
.llm
.chat_with_messages(messages, Some(0.3))
.await
.map_err(|e| {
tracing::error!("LLM help chat failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
Ok(Json(ApiResponse {
data: HelpChatResponse {
message: response_text,
},
total: None,
page: None,
}))
}
+4 -1
View File
@@ -4,13 +4,16 @@ use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::TrackerIssue;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn list_issues(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> ApiResult<Vec<TrackerIssue>> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.tracker_issues()
@@ -0,0 +1,186 @@
//! `/api/v1/mcp-tokens` — per-tenant API tokens for the MCP server.
//!
//! These are opaque static bearers issued via the dashboard (or a
//! direct curl with a KC JWT) and copied into LLM clients (Claude
//! Desktop / Cursor / ChatGPT). The MCP server hashes incoming bearers
//! and looks them up in the cross-tenant `<prefix>__admin.mcp_tokens`
//! collection to derive the tenant_id for routing.
//!
//! The raw token is shown to the caller exactly once at creation; the
//! database only ever stores the SHA-256 hash. Revocation is a soft
//! delete (sets `revoked: true`) so the audit log keeps the record.
use axum::extract::{Extension, Path};
use axum::http::StatusCode;
use axum::Json;
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine as _};
use compliance_core::models::{McpToken, McpTokenView};
use compliance_core::tenant_ctx::TenantCtx;
use mongodb::bson::doc;
use rand::RngCore;
use sha2::{Digest, Sha256};
use super::dto::{AgentExt, ApiResponse};
/// Mongo collection name inside the admin DB.
const COLLECTION: &str = "mcp_tokens";
/// Token prefix the MCP server expects on every bearer.
const TOKEN_PREFIX: &str = "mcpt_";
/// Bytes of randomness behind each token. 32 → ~256 bits.
/// Encoded as URL-safe base64 without padding → 43 chars.
/// Combined with `mcpt_` → 48-char tokens.
const TOKEN_RAND_BYTES: usize = 32;
#[derive(serde::Deserialize)]
pub struct CreateMcpTokenRequest {
pub name: String,
}
/// Returned exactly once at creation. The `token` field is gone from
/// the listing endpoint — the user must save it now.
#[derive(serde::Serialize)]
pub struct CreateMcpTokenResponse {
pub token: String,
pub view: McpTokenView,
}
/// `POST /api/v1/mcp-tokens` — mint a new token for the caller's tenant.
#[tracing::instrument(skip_all)]
pub async fn create_mcp_token(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateMcpTokenRequest>,
) -> Result<Json<CreateMcpTokenResponse>, StatusCode> {
if req.name.trim().is_empty() {
return Err(StatusCode::BAD_REQUEST);
}
let raw = generate_token();
let token_hash = sha256_hex(&raw);
let token_prefix: String = raw.chars().take(12).collect();
let mut token = McpToken {
id: None,
token_hash,
token_prefix,
tenant_id: tenant.0.tenant_id.clone(),
name: req.name.trim().to_string(),
created_by: tenant.0.user_id.clone(),
created_at: chrono::Utc::now(),
last_used_at: None,
revoked: false,
};
let col = agent.db_pool.admin_db().collection::<McpToken>(COLLECTION);
let res = col.insert_one(&token).await.map_err(|e| {
tracing::error!("Failed to insert MCP token: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
token.id = res.inserted_id.as_object_id();
Ok(Json(CreateMcpTokenResponse {
view: McpTokenView::from(&token),
token: raw,
}))
}
/// `GET /api/v1/mcp-tokens` — list tokens for the caller's tenant.
/// Hash is never returned; only metadata + the 12-char prefix so the
/// user can identify which row is which.
#[tracing::instrument(skip_all)]
pub async fn list_mcp_tokens(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<ApiResponse<Vec<McpTokenView>>>, StatusCode> {
let col = agent.db_pool.admin_db().collection::<McpToken>(COLLECTION);
let mut cursor = col
.find(doc! { "tenant_id": &tenant.0.tenant_id })
.sort(doc! { "created_at": -1 })
.await
.map_err(|e| {
tracing::error!("Failed to list MCP tokens: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
let mut out = Vec::new();
while cursor.advance().await.map_err(|e| {
tracing::warn!("MCP tokens cursor advance failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})? {
match cursor.deserialize_current() {
Ok(t) => out.push(McpTokenView::from(&t)),
Err(e) => tracing::warn!("Failed to deserialize MCP token: {e}"),
}
}
Ok(Json(ApiResponse {
data: out,
total: None,
page: None,
}))
}
/// `DELETE /api/v1/mcp-tokens/{id}` — revoke (soft delete).
/// Scoped to the caller's tenant: a user can't revoke another tenant's
/// token even if they guess its id.
#[tracing::instrument(skip_all, fields(id = %id))]
pub async fn revoke_mcp_token(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let col = agent.db_pool.admin_db().collection::<McpToken>(COLLECTION);
let result = col
.update_one(
doc! { "_id": oid, "tenant_id": &tenant.0.tenant_id },
doc! { "$set": { "revoked": true } },
)
.await
.map_err(|e| {
tracing::error!("Failed to revoke MCP token: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
if result.matched_count == 0 {
return Err(StatusCode::NOT_FOUND);
}
Ok(Json(serde_json::json!({ "status": "revoked" })))
}
/// 32 bytes random → URL-safe base64 → 43 chars, no padding.
/// Prefixed with `mcpt_` so the MCP server can sniff the format
/// before bothering with the DB lookup.
fn generate_token() -> String {
let mut bytes = [0u8; TOKEN_RAND_BYTES];
rand::rng().fill_bytes(&mut bytes);
format!("{TOKEN_PREFIX}{}", URL_SAFE_NO_PAD.encode(bytes))
}
fn sha256_hex(s: &str) -> String {
let mut h = Sha256::new();
h.update(s.as_bytes());
hex::encode(h.finalize())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn generated_tokens_are_unique_and_prefixed() {
let a = generate_token();
let b = generate_token();
assert_ne!(a, b);
assert!(a.starts_with(TOKEN_PREFIX));
assert!(b.starts_with(TOKEN_PREFIX));
// 5 + 43 = 48 chars
assert_eq!(a.len(), 5 + 43);
}
#[test]
fn sha256_is_stable_and_64_hex() {
let h = sha256_hex("mcpt_abc");
assert_eq!(h.len(), 64);
assert!(h.chars().all(|c| c.is_ascii_hexdigit()));
assert_eq!(sha256_hex("mcpt_abc"), h);
}
}
+5
View File
@@ -1,10 +1,15 @@
pub mod admin;
pub mod chat;
pub mod dast;
pub mod dto;
pub mod findings;
pub mod graph;
pub mod health;
pub mod help_chat;
pub mod issues;
pub mod mcp_tokens;
pub mod notifications;
pub mod onboarding;
pub mod pentest_handlers;
pub use pentest_handlers as pentest;
pub mod repos;
@@ -0,0 +1,183 @@
use axum::extract::Extension;
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::doc;
use serde::Deserialize;
use compliance_core::models::notification::CveNotification;
use compliance_core::tenant_ctx::TenantCtx;
use super::dto::{tenant_db, AgentExt, ApiResponse};
/// GET /api/v1/notifications — List CVE notifications (newest first)
#[tracing::instrument(skip_all)]
pub async fn list_notifications(
Extension(agent): AgentExt,
tenant: TenantCtx,
axum::extract::Query(params): axum::extract::Query<NotificationFilter>,
) -> Result<Json<ApiResponse<Vec<CveNotification>>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let mut filter = doc! {};
// Filter by status (default: show new + read, exclude dismissed)
match params.status.as_deref() {
Some("all") => {}
Some(s) => {
filter.insert("status", s);
}
None => {
filter.insert("status", doc! { "$in": ["new", "read"] });
}
}
// Filter by severity
if let Some(ref sev) = params.severity {
filter.insert("severity", sev.as_str());
}
// Filter by repo
if let Some(ref repo_id) = params.repo_id {
filter.insert("repo_id", repo_id.as_str());
}
let page = params.page.unwrap_or(1).max(1);
let limit = params.limit.unwrap_or(50).min(200);
let skip = (page - 1) * limit as u64;
let total = db
.cve_notifications()
.count_documents(filter.clone())
.await
.unwrap_or(0);
let notifications: Vec<CveNotification> = match db
.cve_notifications()
.find(filter)
.sort(doc! { "created_at": -1 })
.skip(skip)
.limit(limit)
.await
{
Ok(cursor) => {
use futures_util::StreamExt;
let mut items = Vec::new();
let mut cursor = cursor;
while let Some(Ok(n)) = cursor.next().await {
items.push(n);
}
items
}
Err(e) => {
tracing::error!("Failed to list notifications: {e}");
return Err(StatusCode::INTERNAL_SERVER_ERROR);
}
};
Ok(Json(ApiResponse {
data: notifications,
total: Some(total),
page: Some(page),
}))
}
/// GET /api/v1/notifications/count — Count of unread notifications
#[tracing::instrument(skip_all)]
pub async fn notification_count(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let count = db
.cve_notifications()
.count_documents(doc! { "status": "new" })
.await
.unwrap_or(0);
Ok(Json(serde_json::json!({ "count": count })))
}
/// PATCH /api/v1/notifications/:id/read — Mark a notification as read
#[tracing::instrument(skip_all, fields(id = %id))]
pub async fn mark_read(
Extension(agent): AgentExt,
tenant: TenantCtx,
axum::extract::Path(id): axum::extract::Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let result = db
.cve_notifications()
.update_one(
doc! { "_id": oid },
doc! { "$set": {
"status": "read",
"read_at": mongodb::bson::DateTime::now(),
}},
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
if result.matched_count == 0 {
return Err(StatusCode::NOT_FOUND);
}
Ok(Json(serde_json::json!({ "status": "read" })))
}
/// PATCH /api/v1/notifications/:id/dismiss — Dismiss a notification
#[tracing::instrument(skip_all, fields(id = %id))]
pub async fn dismiss_notification(
Extension(agent): AgentExt,
tenant: TenantCtx,
axum::extract::Path(id): axum::extract::Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let result = db
.cve_notifications()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "status": "dismissed" } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
if result.matched_count == 0 {
return Err(StatusCode::NOT_FOUND);
}
Ok(Json(serde_json::json!({ "status": "dismissed" })))
}
/// POST /api/v1/notifications/read-all — Mark all new notifications as read
#[tracing::instrument(skip_all)]
pub async fn mark_all_read(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let result = db
.cve_notifications()
.update_many(
doc! { "status": "new" },
doc! { "$set": {
"status": "read",
"read_at": mongodb::bson::DateTime::now(),
}},
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
Ok(Json(
serde_json::json!({ "updated": result.modified_count }),
))
}
#[derive(Debug, Deserialize)]
pub struct NotificationFilter {
pub status: Option<String>,
pub severity: Option<String>,
pub repo_id: Option<String>,
pub page: Option<u64>,
pub limit: Option<i64>,
}
@@ -0,0 +1,393 @@
//! Onboarding API — CRUD for unified targets, artifact add, classification, and
//! the scan-applicability matrix. The wizard (and future integrations) drive
//! onboarding through these endpoints.
use std::collections::HashMap;
use std::sync::Arc;
use axum::extract::{Extension, Path, Query};
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::{doc, oid::ObjectId, to_bson};
use serde::{Deserialize, Serialize};
use compliance_core::models::{
Artifact, ArtifactKind, ComplianceProfile, OnboardedTarget, PlcFormat, TargetScanConfig,
TargetType,
};
use compliance_core::scan_matrix::{applicable_scans, supports_pentest};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use crate::classify::{classify_target, MockFirmwareDetector};
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
/// A client-supplied artifact spec. The server builds the [`Artifact`] (and its
/// id) from it, so clients never set internal fields.
#[derive(Deserialize)]
pub struct ArtifactInput {
pub kind: ArtifactKind,
pub source_ref: String,
#[serde(default)]
pub branch: Option<String>,
#[serde(default)]
pub plc_format: Option<PlcFormat>,
}
impl ArtifactInput {
fn build(&self) -> Artifact {
let s = self.source_ref.clone();
match self.kind {
ArtifactKind::GitRepo => {
Artifact::git_repo(s, self.branch.clone().unwrap_or_else(|| "main".to_string()))
}
ArtifactKind::LiveUrl => Artifact::live_url(s),
ArtifactKind::FirmwareImage => Artifact::firmware_image(s),
ArtifactKind::SourceArchive => Artifact::source_archive(s),
ArtifactKind::MobilePackage => Artifact::mobile_package(s),
ArtifactKind::ContainerImage => Artifact::container_image(s),
ArtifactKind::PlcProject => {
Artifact::plc_project(s, self.plc_format.unwrap_or(PlcFormat::PlcopenXml))
}
ArtifactKind::PlaintextDescription => Artifact::plaintext(s),
}
}
}
#[derive(Deserialize)]
pub struct CreateTargetRequest {
pub name: String,
pub target_type: TargetType,
#[serde(default)]
pub description: Option<String>,
#[serde(default)]
pub artifacts: Vec<ArtifactInput>,
}
#[derive(Deserialize)]
pub struct UpdateTargetRequest {
pub name: Option<String>,
pub target_type: Option<TargetType>,
pub scan_config: Option<TargetScanConfig>,
pub compliance_profile: Option<ComplianceProfile>,
pub scan_schedule: Option<String>,
}
/// One applicable-scan option, serialized for the wizard.
#[derive(Serialize)]
pub struct ScanOptionDto {
pub scan: String,
pub default_on: bool,
pub rationale: String,
pub required_artifact: Option<String>,
pub blocked_reason: Option<String>,
}
#[derive(Serialize)]
pub struct ApplicableScansResponse {
pub scans: Vec<ScanOptionDto>,
pub pentest_supported: bool,
}
fn parse_oid(id: &str) -> Result<ObjectId, StatusCode> {
ObjectId::parse_str(id).map_err(|_| StatusCode::BAD_REQUEST)
}
/// GET /api/v1/targets — list onboarded targets (paginated).
#[tracing::instrument(skip_all)]
pub async fn list_targets(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<OnboardedTarget>>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.onboarded_targets()
.count_documents(doc! {})
.await
.unwrap_or(0);
let targets = match db
.onboarded_targets()
.find(doc! {})
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch onboarded targets: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: targets,
total: Some(total),
page: Some(params.page),
}))
}
/// POST /api/v1/targets — create an onboarded target.
#[tracing::instrument(skip_all)]
pub async fn create_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateTargetRequest>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let mut target = OnboardedTarget::new(req.name, req.target_type);
target.description = req.description;
target.artifacts = req.artifacts.iter().map(ArtifactInput::build).collect();
let db = tenant_db(&agent, &tenant).await?;
let res = db
.onboarded_targets()
.insert_one(&target)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
target.id = res.inserted_id.as_object_id();
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// GET /api/v1/targets/{id} — fetch one target.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn get_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// PATCH /api/v1/targets/{id} — update mutable fields.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn update_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateTargetRequest>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let mut set = doc! { "updated_at": mongodb::bson::DateTime::now() };
if let Some(name) = req.name {
set.insert("name", name);
}
if let Some(tt) = req.target_type {
set.insert(
"target_type",
to_bson(&tt).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(sc) = req.scan_config {
set.insert(
"scan_config",
to_bson(&sc).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(cp) = req.compliance_profile {
set.insert(
"compliance_profile",
to_bson(&cp).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(ss) = req.scan_schedule {
set.insert("scan_schedule", ss);
}
db.onboarded_targets()
.update_one(doc! { "_id": oid }, doc! { "$set": set })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// DELETE /api/v1/targets/{id} — remove the target and its findings/scans.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn delete_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
db.onboarded_targets()
.delete_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
// Cascade the collections keyed by repo_id == target id (best-effort).
let by_repo = doc! { "repo_id": &id };
let _ = db.findings().delete_many(by_repo.clone()).await;
let _ = db.scan_runs().delete_many(by_repo.clone()).await;
let _ = db.sbom_entries().delete_many(by_repo.clone()).await;
let _ = db.cve_alerts().delete_many(by_repo).await;
Ok(Json(serde_json::json!({ "status": "deleted" })))
}
/// POST /api/v1/targets/{id}/artifacts — attach an artifact (by reference).
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn add_artifact(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(input): Json<ArtifactInput>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let artifact = to_bson(&input.build()).map_err(|_| StatusCode::BAD_REQUEST)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$push": { "artifacts": artifact }, "$set": { "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// GET /api/v1/targets/{id}/applicable-scans — the scan-applicability matrix.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn applicable_scans_for_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<ApplicableScansResponse>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
let scans = applicable_scans(&target)
.into_iter()
.map(|o| ScanOptionDto {
scan: o.scan.to_string(),
default_on: o.default_on,
rationale: o.rationale,
required_artifact: o.required_artifact.map(|k| k.to_string()),
blocked_reason: o.blocked_reason,
})
.collect();
Ok(Json(ApiResponse {
data: ApplicableScansResponse {
scans,
pentest_supported: supports_pentest(target.target_type),
},
total: None,
page: None,
}))
}
/// POST /api/v1/targets/{id}/detect — classify the target from its artifacts.
///
/// This is the lightweight pass: it classifies from artifact kinds without
/// ingesting (cloning) sources, so it returns immediately. Deep detection (after
/// ingest, with tramiton firmware analysis) is a follow-up background step.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn detect_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let mut target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
// No ingested working paths here → kind-based classification only; the mock
// firmware detector is never invoked (no firmware working path present).
let empty = HashMap::new();
let detector = MockFirmwareDetector { detection: None };
let classification = classify_target(&target, &empty, &detector)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let classification_bson =
to_bson(&classification).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "classification": classification_bson, "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
target.classification = Some(classification);
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// POST /api/v1/targets/{id}/scan — trigger a scan for the target.
///
/// Dispatches to the unified pipeline when `UNIFIED_PIPELINE` is set (else the
/// legacy path). Runs in the background and returns immediately.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn trigger_target_scan(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
// 404 if the target doesn't exist for this tenant.
if db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.is_none()
{
return Err(StatusCode::NOT_FOUND);
}
let agent_clone = (*agent).clone();
let tenant_id = tenant.0.tenant_id.clone();
tokio::spawn(async move {
// Always the unified target pipeline — this endpoint is about an
// onboarded target by construction, independent of the global
// `unified_pipeline` transition flag used by the legacy paths.
if let Err(e) = agent_clone
.run_target_scan(
&tenant_id,
&id,
compliance_core::models::ScanTrigger::Manual,
)
.await
{
tracing::error!("Manual target scan failed for {id}: {e}");
}
});
Ok(Json(serde_json::json!({ "status": "scan_triggered" })))
}
@@ -13,10 +13,11 @@ use compliance_core::models::dast::DastFinding;
use compliance_core::models::finding::Finding;
use compliance_core::models::pentest::*;
use compliance_core::models::sbom::SbomEntry;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::super::dto::collect_cursor_async;
use super::super::dto::{collect_cursor_async, tenant_db};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -35,11 +36,15 @@ pub struct ExportBody {
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn export_session_report(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(body): Json<ExportBody>,
) -> Result<axum::response::Response, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
if body.password.len() < 8 {
return Err((
@@ -49,8 +54,7 @@ pub async fn export_session_report(
}
// Fetch session
let session = agent
.db
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -64,9 +68,7 @@ pub async fn export_session_report(
// Resolve target name
let target = if let Ok(tid) = mongodb::bson::oid::ObjectId::parse_str(&session.target_id) {
agent
.db
.dast_targets()
db.dast_targets()
.find_one(doc! { "_id": tid })
.await
.ok()
@@ -84,8 +86,7 @@ pub async fn export_session_report(
.unwrap_or_default();
// Fetch attack chain nodes
let nodes: Vec<AttackChainNode> = match agent
.db
let nodes: Vec<AttackChainNode> = match db
.attack_chain_nodes()
.find(doc! { "session_id": &id })
.sort(doc! { "started_at": 1 })
@@ -95,9 +96,8 @@ pub async fn export_session_report(
Err(_) => Vec::new(),
};
// Fetch DAST findings for this session
let findings: Vec<DastFinding> = match agent
.db
// Fetch DAST findings for this session, then deduplicate
let raw_findings: Vec<DastFinding> = match db
.dast_findings()
.find(doc! { "session_id": &id })
.sort(doc! { "severity": -1, "created_at": -1 })
@@ -106,6 +106,14 @@ pub async fn export_session_report(
Ok(cursor) => collect_cursor_async(cursor).await,
Err(_) => Vec::new(),
};
let raw_count = raw_findings.len();
let findings = crate::pipeline::dedup::dedup_dast_findings(raw_findings);
if findings.len() < raw_count {
tracing::info!(
"Deduped DAST findings for session {id}: {raw_count} → {}",
findings.len()
);
}
// Fetch SAST findings, SBOM, and code context for the linked repository
let repo_id = session
@@ -114,8 +122,7 @@ pub async fn export_session_report(
.or_else(|| target.as_ref().and_then(|t| t.repo_id.clone()));
let (sast_findings, sbom_entries, code_context) = if let Some(ref rid) = repo_id {
let sast: Vec<Finding> = match agent
.db
let sast: Vec<Finding> = match db
.findings()
.find(doc! {
"repo_id": rid,
@@ -135,8 +142,7 @@ pub async fn export_session_report(
Err(_) => Vec::new(),
};
let sbom: Vec<SbomEntry> = match agent
.db
let sbom: Vec<SbomEntry> = match db
.sbom_entries()
.find(doc! {
"repo_id": rid,
@@ -156,8 +162,7 @@ pub async fn export_session_report(
};
// Build code context from graph nodes
let code_ctx: Vec<CodeContextHint> = match agent
.db
let code_ctx: Vec<CodeContextHint> = match db
.graph_nodes()
.find(doc! { "repo_id": rid, "is_entry_point": true })
.limit(50)
@@ -7,11 +7,12 @@ use mongodb::bson::doc;
use serde::Deserialize;
use compliance_core::models::pentest::*;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use crate::pentest::PentestOrchestrator;
use super::super::dto::{collect_cursor_async, ApiResponse, PaginationParams};
use super::super::dto::{collect_cursor_async, tenant_db, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -43,6 +44,7 @@ pub struct LookupRepoQuery {
#[tracing::instrument(skip_all)]
pub async fn create_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateSessionRequest>,
) -> Result<Json<ApiResponse<PentestSession>>, (StatusCode, String)> {
// Try to acquire a concurrency permit
@@ -57,6 +59,10 @@ pub async fn create_session(
)
})?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
if let Some(ref config) = req.config {
// ── Wizard path ──────────────────────────────────────────────
if !config.disclaimer_accepted {
@@ -67,8 +73,7 @@ pub async fn create_session(
}
// Look up or auto-create DastTarget by app_url
let target = match agent
.db
let target = match db
.dast_targets()
.find_one(doc! { "base_url": &config.app_url })
.await
@@ -87,7 +92,7 @@ pub async fn create_session(
}
t.allow_destructive = config.allow_destructive;
t.excluded_paths = config.scope_exclusions.clone();
let res = agent.db.dast_targets().insert_one(&t).await.map_err(|e| {
let res = db.dast_targets().insert_one(&t).await.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create target: {e}"),
@@ -110,8 +115,7 @@ pub async fn create_session(
// Resolve repo_id from git_repo_url if provided
if let Some(ref git_url) = config.git_repo_url {
if let Ok(Some(repo)) = agent
.db
if let Ok(Some(repo)) = db
.repositories()
.find_one(doc! { "git_url": git_url })
.await
@@ -120,8 +124,7 @@ pub async fn create_session(
}
}
let insert_result = agent
.db
let insert_result = db
.pentest_sessions()
.insert_one(&session)
.await
@@ -212,8 +215,7 @@ pub async fn create_session(
// Persist encrypted credentials to DB
if session_for_task.config.is_some() {
if let Some(sid) = session.id {
let _ = agent
.db
let _ = db
.pentest_sessions()
.update_one(
doc! { "_id": sid },
@@ -245,12 +247,13 @@ pub async fn create_session(
});
let llm = agent.llm.clone();
let db = agent.db.clone();
let db_for_orchestrator = db.clone();
let session_clone = session.clone();
let target_clone = target.clone();
let agent_ref = agent.clone();
tokio::spawn(async move {
let orchestrator = PentestOrchestrator::new(llm, db, event_tx, Some(pause_rx));
let orchestrator =
PentestOrchestrator::new(llm, db_for_orchestrator, event_tx, Some(pause_rx));
orchestrator
.run_session_guarded(&session_clone, &target_clone, &initial_message)
.await;
@@ -292,8 +295,7 @@ pub async fn create_session(
)
})?;
let target = agent
.db
let target = db
.dast_targets()
.find_one(doc! { "_id": oid })
.await
@@ -310,8 +312,7 @@ pub async fn create_session(
let mut session = PentestSession::new(target_id, strategy);
session.repo_id = target.repo_id.clone();
let insert_result = agent
.db
let insert_result = db
.pentest_sessions()
.insert_one(&session)
.await
@@ -338,12 +339,13 @@ pub async fn create_session(
});
let llm = agent.llm.clone();
let db = agent.db.clone();
let db_for_orchestrator = db.clone();
let session_clone = session.clone();
let target_clone = target.clone();
let agent_ref = agent.clone();
tokio::spawn(async move {
let orchestrator = PentestOrchestrator::new(llm, db, event_tx, Some(pause_rx));
let orchestrator =
PentestOrchestrator::new(llm, db_for_orchestrator, event_tx, Some(pause_rx));
orchestrator
.run_session_guarded(&session_clone, &target_clone, &initial_message)
.await;
@@ -373,10 +375,11 @@ fn parse_strategy(s: &str) -> PentestStrategy {
#[tracing::instrument(skip_all)]
pub async fn lookup_repo(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<LookupRepoQuery>,
) -> Result<Json<ApiResponse<serde_json::Value>>, StatusCode> {
let repo = agent
.db
let db = tenant_db(&agent, &tenant).await?;
let repo = db
.repositories()
.find_one(doc! { "git_url": &params.url })
.await
@@ -402,9 +405,11 @@ pub async fn lookup_repo(
#[tracing::instrument(skip_all)]
pub async fn list_sessions(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<PentestSession>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.pentest_sessions()
@@ -438,12 +443,13 @@ pub async fn list_sessions(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<PentestSession>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let mut session = agent
.db
let mut session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -471,15 +477,18 @@ pub async fn get_session(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn send_message(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<SendMessageRequest>,
) -> Result<Json<ApiResponse<PentestMessage>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
// Verify session exists and is running
let session = agent
.db
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -506,8 +515,7 @@ pub async fn send_message(
)
})?;
let target = agent
.db
let target = db
.dast_targets()
.find_one(doc! { "_id": target_oid })
.await
@@ -527,13 +535,13 @@ pub async fn send_message(
// Store user message
let session_id = id.clone();
let user_msg = PentestMessage::user(session_id.clone(), req.message.clone());
let _ = agent.db.pentest_messages().insert_one(&user_msg).await;
let _ = db.pentest_messages().insert_one(&user_msg).await;
let response_msg = user_msg.clone();
// Spawn orchestrator to continue the session
let llm = agent.llm.clone();
let db = agent.db.clone();
let db_for_orchestrator = db.clone();
let message = req.message.clone();
// Use existing broadcast sender if available, otherwise create a new one
@@ -548,7 +556,7 @@ pub async fn send_message(
.unwrap_or_else(|| agent.register_session_stream(&session_id));
tokio::spawn(async move {
let orchestrator = PentestOrchestrator::new(llm, db, event_tx, None);
let orchestrator = PentestOrchestrator::new(llm, db_for_orchestrator, event_tx, None);
orchestrator
.run_session_guarded(&session, &target, &message)
.await;
@@ -565,13 +573,16 @@ pub async fn send_message(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn stop_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<PentestSession>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
let session = agent
.db
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -590,9 +601,7 @@ pub async fn stop_session(
));
}
agent
.db
.pentest_sessions()
db.pentest_sessions()
.update_one(
doc! { "_id": oid },
doc! { "$set": {
@@ -612,8 +621,7 @@ pub async fn stop_session(
// Clean up session resources
agent.cleanup_session(&id);
let updated = agent
.db
let updated = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -641,13 +649,16 @@ pub async fn stop_session(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn pause_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<serde_json::Value>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
let session = agent
.db
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -684,13 +695,16 @@ pub async fn pause_session(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn resume_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<serde_json::Value>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
let session = agent
.db
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -727,12 +741,13 @@ pub async fn resume_session(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_attack_chain(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<Vec<AttackChainNode>>>, StatusCode> {
let _oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let nodes = match agent
.db
let nodes = match db
.attack_chain_nodes()
.find(doc! { "session_id": &id })
.sort(doc! { "started_at": 1 })
@@ -757,21 +772,21 @@ pub async fn get_attack_chain(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_messages(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<PentestMessage>>>, StatusCode> {
let _oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = agent
.db
let total = db
.pentest_messages()
.count_documents(doc! { "session_id": &id })
.await
.unwrap_or(0);
let messages = match agent
.db
let messages = match db
.pentest_messages()
.find(doc! { "session_id": &id })
.sort(doc! { "created_at": 1 })
@@ -797,21 +812,21 @@ pub async fn get_messages(
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_session_findings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<compliance_core::models::dast::DastFinding>>>, StatusCode> {
let _oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = agent
.db
let total = db
.dast_findings()
.count_documents(doc! { "session_id": &id })
.await
.unwrap_or(0);
let findings = match agent
.db
let findings = match db
.dast_findings()
.find(doc! { "session_id": &id })
.sort(doc! { "created_at": -1 })
@@ -6,10 +6,11 @@ use axum::Json;
use mongodb::bson::doc;
use compliance_core::models::pentest::*;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::super::dto::{collect_cursor_async, ApiResponse};
use super::super::dto::{collect_cursor_async, tenant_db, ApiResponse};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -17,8 +18,10 @@ type AgentExt = Extension<Arc<ComplianceAgent>>;
#[tracing::instrument(skip_all)]
pub async fn pentest_stats(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<ApiResponse<PentestStats>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let running_sessions = db
.pentest_sessions()
@@ -11,10 +11,11 @@ use tokio_stream::wrappers::BroadcastStream;
use tokio_stream::StreamExt;
use compliance_core::models::pentest::*;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::super::dto::collect_cursor_async;
use super::super::dto::{collect_cursor_async, tenant_db};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -25,13 +26,14 @@ type AgentExt = Extension<Arc<ComplianceAgent>>;
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn session_stream(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Sse<impl futures_util::Stream<Item = Result<Event, Infallible>>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
// Verify session exists
let _session = agent
.db
let _session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
@@ -43,8 +45,7 @@ pub async fn session_stream(
let mut initial_events: Vec<Result<Event, Infallible>> = Vec::new();
// Fetch recent messages for this session
let messages: Vec<PentestMessage> = match agent
.db
let messages: Vec<PentestMessage> = match db
.pentest_messages()
.find(doc! { "session_id": &id })
.sort(doc! { "created_at": 1 })
@@ -56,8 +57,7 @@ pub async fn session_stream(
};
// Fetch recent attack chain nodes
let nodes: Vec<AttackChainNode> = match agent
.db
let nodes: Vec<AttackChainNode> = match db
.attack_chain_nodes()
.find(doc! { "session_id": &id })
.sort(doc! { "started_at": 1 })
@@ -94,8 +94,7 @@ pub async fn session_stream(
}
// Add current session status event
let session = agent
.db
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
+115 -17
View File
@@ -5,13 +5,16 @@ use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::*;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn list_repositories(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> ApiResult<Vec<TrackedRepository>> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.repositories()
@@ -43,6 +46,7 @@ pub async fn list_repositories(
#[tracing::instrument(skip_all)]
pub async fn add_repository(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<AddRepositoryRequest>,
) -> Result<Json<ApiResponse<TrackedRepository>>, (StatusCode, String)> {
// Validate repository access before saving
@@ -69,17 +73,15 @@ pub async fn add_repository(
repo.tracker_token = req.tracker_token;
repo.scan_schedule = req.scan_schedule;
agent
.db
.repositories()
.insert_one(&repo)
let db = tenant_db(&agent, &tenant)
.await
.map_err(|_| {
(
StatusCode::CONFLICT,
"Repository already exists".to_string(),
)
})?;
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
db.repositories().insert_one(&repo).await.map_err(|_| {
(
StatusCode::CONFLICT,
"Repository already exists".to_string(),
)
})?;
Ok(Json(ApiResponse {
data: repo,
@@ -91,10 +93,12 @@ pub async fn add_repository(
#[tracing::instrument(skip_all, fields(repo_id = %id))]
pub async fn update_repository(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateRepositoryRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let mut set_doc = doc! { "updated_at": mongodb::bson::DateTime::now() };
@@ -126,8 +130,7 @@ pub async fn update_repository(
set_doc.insert("scan_schedule", schedule);
}
let result = agent
.db
let result = db
.repositories()
.update_one(doc! { "_id": oid }, doc! { "$set": set_doc })
.await
@@ -155,11 +158,16 @@ pub async fn get_ssh_public_key(
#[tracing::instrument(skip_all, fields(repo_id = %id))]
pub async fn trigger_scan(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let agent_clone = (*agent).clone();
let tenant_id = tenant.0.tenant_id.clone();
tokio::spawn(async move {
if let Err(e) = agent_clone.run_scan(&id, ScanTrigger::Manual).await {
if let Err(e) = agent_clone
.run_scan(&tenant_id, &id, ScanTrigger::Manual)
.await
{
tracing::error!("Manual scan failed for {id}: {e}");
}
});
@@ -170,11 +178,12 @@ pub async fn trigger_scan(
/// Return the webhook secret for a repository (used by dashboard to display it)
pub async fn get_webhook_config(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let repo = agent
.db
let db = tenant_db(&agent, &tenant).await?;
let repo = db
.repositories()
.find_one(doc! { "_id": oid })
.await
@@ -196,10 +205,12 @@ pub async fn get_webhook_config(
#[tracing::instrument(skip_all, fields(repo_id = %id))]
pub async fn delete_repository(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Delete the repository
let result = db
@@ -237,5 +248,92 @@ pub async fn delete_repository(
.delete_many(doc! { "repo_id": &id })
.await;
// Cascade delete DAST targets linked to this repo, and all their downstream data
// (scan runs, findings, pentest sessions, attack chains, messages)
if let Ok(mut cursor) = db.dast_targets().find(doc! { "repo_id": &id }).await {
use futures_util::StreamExt;
while let Some(Ok(target)) = cursor.next().await {
let target_id = target.id.map(|oid| oid.to_hex()).unwrap_or_default();
if !target_id.is_empty() {
cascade_delete_dast_target(db, &target_id).await;
}
}
}
// Also delete pentest sessions linked directly to this repo (not via target)
if let Ok(mut cursor) = db.pentest_sessions().find(doc! { "repo_id": &id }).await {
use futures_util::StreamExt;
while let Some(Ok(session)) = cursor.next().await {
let session_id = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
if !session_id.is_empty() {
let _ = db
.attack_chain_nodes()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.pentest_messages()
.delete_many(doc! { "session_id": &session_id })
.await;
// Delete DAST findings produced by this session
let _ = db
.dast_findings()
.delete_many(doc! { "session_id": &session_id })
.await;
}
}
}
let _ = db
.pentest_sessions()
.delete_many(doc! { "repo_id": &id })
.await;
Ok(Json(serde_json::json!({ "status": "deleted" })))
}
/// Cascade-delete a DAST target and all its downstream data.
async fn cascade_delete_dast_target(db: &crate::database::Database, target_id: &str) {
// Delete pentest sessions for this target (and their attack chains + messages)
if let Ok(mut cursor) = db
.pentest_sessions()
.find(doc! { "target_id": target_id })
.await
{
use futures_util::StreamExt;
while let Some(Ok(session)) = cursor.next().await {
let session_id = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
if !session_id.is_empty() {
let _ = db
.attack_chain_nodes()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.pentest_messages()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.dast_findings()
.delete_many(doc! { "session_id": &session_id })
.await;
}
}
}
let _ = db
.pentest_sessions()
.delete_many(doc! { "target_id": target_id })
.await;
// Delete DAST scan runs and their findings
let _ = db
.dast_findings()
.delete_many(doc! { "target_id": target_id })
.await;
let _ = db
.dast_scan_runs()
.delete_many(doc! { "target_id": target_id })
.await;
// Delete the target itself
if let Ok(oid) = mongodb::bson::oid::ObjectId::parse_str(target_id) {
let _ = db.dast_targets().delete_one(doc! { "_id": oid }).await;
}
}
+16 -5
View File
@@ -6,6 +6,7 @@ use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::SbomEntry;
use compliance_core::tenant_ctx::TenantCtx;
const COPYLEFT_LICENSES: &[&str] = &[
"GPL-2.0",
@@ -29,8 +30,10 @@ const COPYLEFT_LICENSES: &[&str] = &[
#[tracing::instrument(skip_all)]
pub async fn sbom_filters(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let managers: Vec<String> = db
.sbom_entries()
@@ -61,9 +64,11 @@ pub async fn sbom_filters(
#[tracing::instrument(skip_all, fields(repo_id = ?filter.repo_id, package_manager = ?filter.package_manager))]
pub async fn list_sbom(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(filter): Query<SbomFilter>,
) -> ApiResult<Vec<SbomEntry>> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let mut query = doc! {};
if let Some(repo_id) = &filter.repo_id {
@@ -120,9 +125,11 @@ pub async fn list_sbom(
#[tracing::instrument(skip_all)]
pub async fn export_sbom(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<SbomExportParams>,
) -> Result<impl IntoResponse, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let entries: Vec<SbomEntry> = match db
.sbom_entries()
.find(doc! { "repo_id": &params.repo_id })
@@ -236,9 +243,11 @@ pub async fn export_sbom(
#[tracing::instrument(skip_all)]
pub async fn license_summary(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<SbomFilter>,
) -> ApiResult<Vec<LicenseSummary>> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let mut query = doc! {};
if let Some(repo_id) = &params.repo_id {
query.insert("repo_id", repo_id);
@@ -285,9 +294,11 @@ pub async fn license_summary(
#[tracing::instrument(skip_all)]
pub async fn sbom_diff(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<SbomDiffParams>,
) -> ApiResult<SbomDiffResult> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let entries_a: Vec<SbomEntry> = match db
.sbom_entries()
+4 -1
View File
@@ -4,13 +4,16 @@ use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::ScanRun;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn list_scan_runs(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> ApiResult<Vec<ScanRun>> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db.scan_runs().count_documents(doc! {}).await.unwrap_or(0);
-1
View File
@@ -1,4 +1,3 @@
pub mod auth_middleware;
pub mod handlers;
pub mod routes;
pub mod server;
+65 -14
View File
@@ -2,7 +2,6 @@ use axum::routing::{delete, get, patch, post};
use axum::Router;
use crate::api::handlers;
use crate::webhooks;
pub fn build_router() -> Router {
Router::new()
@@ -26,6 +25,33 @@ pub fn build_router() -> Router {
"/api/v1/repositories/{id}/webhook-config",
get(handlers::get_webhook_config),
)
// Unified onboarding targets (#131).
.route(
"/api/v1/targets",
get(handlers::onboarding::list_targets).post(handlers::onboarding::create_target),
)
.route(
"/api/v1/targets/{id}",
get(handlers::onboarding::get_target)
.patch(handlers::onboarding::update_target)
.delete(handlers::onboarding::delete_target),
)
.route(
"/api/v1/targets/{id}/artifacts",
post(handlers::onboarding::add_artifact),
)
.route(
"/api/v1/targets/{id}/applicable-scans",
get(handlers::onboarding::applicable_scans_for_target),
)
.route(
"/api/v1/targets/{id}/detect",
post(handlers::onboarding::detect_target),
)
.route(
"/api/v1/targets/{id}/scan",
post(handlers::onboarding::trigger_target_scan),
)
.route("/api/v1/findings", get(handlers::list_findings))
.route("/api/v1/findings/{id}", get(handlers::get_finding))
.route(
@@ -47,6 +73,15 @@ pub fn build_router() -> Router {
.route("/api/v1/sbom/diff", get(handlers::sbom_diff))
.route("/api/v1/issues", get(handlers::list_issues))
.route("/api/v1/scan-runs", get(handlers::list_scan_runs))
// MCP token management (per-tenant API tokens for the MCP server)
.route(
"/api/v1/mcp-tokens",
get(handlers::mcp_tokens::list_mcp_tokens).post(handlers::mcp_tokens::create_mcp_token),
)
.route(
"/api/v1/mcp-tokens/{id}",
delete(handlers::mcp_tokens::revoke_mcp_token),
)
// Graph API endpoints
.route("/api/v1/graph/{repo_id}", get(handlers::graph::get_graph))
.route(
@@ -99,6 +134,29 @@ pub fn build_router() -> Router {
"/api/v1/chat/{repo_id}/status",
get(handlers::chat::embedding_status),
)
// Help chat (documentation-grounded Q&A)
.route("/api/v1/help/chat", post(handlers::help_chat::help_chat))
// CVE notification endpoints
.route(
"/api/v1/notifications",
get(handlers::notifications::list_notifications),
)
.route(
"/api/v1/notifications/count",
get(handlers::notifications::notification_count),
)
.route(
"/api/v1/notifications/read-all",
post(handlers::notifications::mark_all_read),
)
.route(
"/api/v1/notifications/{id}/read",
patch(handlers::notifications::mark_read),
)
.route(
"/api/v1/notifications/{id}/dismiss",
patch(handlers::notifications::dismiss_notification),
)
// Pentest API endpoints
.route(
"/api/v1/pentest/lookup-repo",
@@ -152,17 +210,10 @@ pub fn build_router() -> Router {
"/api/v1/pentest/stats",
get(handlers::pentest::pentest_stats),
)
// Webhook endpoints (proxied through dashboard)
.route(
"/webhook/github/{repo_id}",
post(webhooks::github::handle_github_webhook),
)
.route(
"/webhook/gitlab/{repo_id}",
post(webhooks::gitlab::handle_gitlab_webhook),
)
.route(
"/webhook/gitea/{repo_id}",
post(webhooks::gitea::handle_gitea_webhook),
)
// Webhook routes live on the separate webhook server (port 3002,
// see crate::webhooks::server). The M7.2-C tenant-in-URL form is
// `/webhook/{tenant_id}/{platform}/{repo_id}` and the handlers
// expect a (tenant_id, repo_id) path tuple. Anything mounting
// them here on the API server would mismatch the handler
// signature, so the routes are not exported.
}
+96 -6
View File
@@ -1,20 +1,99 @@
use std::sync::Arc;
use axum::{middleware, Extension};
use axum::extract::Request;
use axum::http::HeaderValue;
use axum::middleware::Next;
use axum::response::Response;
use axum::routing::{delete, get};
use axum::{middleware, Extension, Router};
use tokio::sync::RwLock;
use tower_http::cors::CorsLayer;
use tower_http::set_header::SetResponseHeaderLayer;
use tower_http::trace::TraceLayer;
use compliance_core::auth::{require_jwt_auth, require_tenant_status, JwksState};
use compliance_core::{TenantContext, TenantStatus};
use crate::agent::ComplianceAgent;
use crate::api::auth_middleware::{require_jwt_auth, JwksState};
use crate::api::handlers;
use crate::api::routes;
use crate::error::AgentError;
/// Synthetic tenant id used when Keycloak isn't configured (local dev,
/// `cargo run` against a bare Mongo). Lets the handler stack stay
/// uniformly tenant-scoped without the operator having to spin up KC
/// just to poke at the API. Override via `DEV_TENANT_ID`.
const DEFAULT_DEV_TENANT_ID: &str = "dev";
/// Inject a synthetic [`TenantContext`] for any request that lacks one.
/// Only mounted when Keycloak is NOT configured; with KC, the real
/// `require_jwt_auth` middleware owns this and we never reach here
/// without a context.
///
/// Public so the integration-test harness can mount it without
/// duplicating the synthetic-context shape.
pub async fn inject_dev_tenant(mut request: Request, next: Next) -> Response {
if request.extensions().get::<TenantContext>().is_none() {
let tenant_id =
std::env::var("DEV_TENANT_ID").unwrap_or_else(|_| DEFAULT_DEV_TENANT_ID.to_string());
let ctx = TenantContext {
tenant_slug: tenant_id.clone(),
tenant_id,
org_roles: vec![],
products: vec![],
plan: "dev".to_string(),
status: TenantStatus::Active,
user_id: "dev-user".to_string(),
user_name: None,
};
request.extensions_mut().insert(ctx);
}
next.run(request).await
}
pub async fn start_api_server(agent: ComplianceAgent, port: u16) -> Result<(), AgentError> {
// Admin sub-router. Routes are only mounted when ADMIN_API_TOKEN is
// configured — without it, the paths don't exist at all (404 rather
// than 401), so an operator who hasn't opted in can't fingerprint
// the surface area.
let admin_router: Router = if agent.config.admin_api_token.is_some() {
tracing::info!("Admin API enabled — /api/v1/admin/* mounted behind ADMIN_API_TOKEN bearer");
Router::new()
.route(
"/api/v1/admin/tenants",
get(handlers::admin::list_tenant_dbs),
)
.route(
"/api/v1/admin/tenants/{tenant_id}",
delete(handlers::admin::drop_tenant_db),
)
.layer(middleware::from_fn(handlers::admin::require_admin_token))
} else {
Router::new()
};
let mut app = routes::build_router()
.merge(admin_router)
.layer(Extension(Arc::new(agent.clone())))
.layer(CorsLayer::permissive())
.layer(TraceLayer::new_for_http());
.layer(TraceLayer::new_for_http())
// Security headers (defense-in-depth, primary enforcement via Traefik)
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::STRICT_TRANSPORT_SECURITY,
HeaderValue::from_static("max-age=31536000; includeSubDomains"),
))
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::X_FRAME_OPTIONS,
HeaderValue::from_static("DENY"),
))
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::X_CONTENT_TYPE_OPTIONS,
HeaderValue::from_static("nosniff"),
))
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::REFERRER_POLICY,
HeaderValue::from_static("strict-origin-when-cross-origin"),
));
if let (Some(kc_url), Some(kc_realm)) =
(&agent.config.keycloak_url, &agent.config.keycloak_realm)
@@ -25,11 +104,22 @@ pub async fn start_api_server(agent: ComplianceAgent, port: u16) -> Result<(), A
jwks_url,
};
tracing::info!("Keycloak JWT auth enabled for realm '{kc_realm}'");
// Layers execute outermost-first. Extension(jwks_state) must run
// before require_jwt_auth so the middleware can read it; the
// status gate runs after JWT so TenantContext is in extensions.
app = app
.layer(Extension(jwks_state))
.layer(middleware::from_fn(require_jwt_auth));
.layer(middleware::from_fn(require_tenant_status))
.layer(middleware::from_fn(require_jwt_auth))
.layer(Extension(jwks_state));
} else {
tracing::warn!("Keycloak not configured - API endpoints are unprotected");
let tenant_id =
std::env::var("DEV_TENANT_ID").unwrap_or_else(|_| DEFAULT_DEV_TENANT_ID.to_string());
tracing::warn!(
tenant_id = %tenant_id,
"Keycloak not configured — running unauthenticated against the dev tenant. \
DO NOT use in any environment with real customer data."
);
app = app.layer(middleware::from_fn(inject_dev_tenant));
}
let addr = format!("0.0.0.0:{port}");
+217
View File
@@ -0,0 +1,217 @@
//! Firmware classification via tramiton.
//!
//! tramiton is the company's firmware build/repro engine; we do not re-implement
//! its detection. We depend on `tramiton-core` directly (same-company IP) and run
//! its provider analysis in-process behind a [`FirmwareDetector`] port, mapping
//! tramiton's `BuildPlan` onto a [`TargetType`]. A deterministic
//! [`MockFirmwareDetector`] backs the tests so CI unit tests need neither the
//! tramiton sources nor a real firmware tree.
use std::path::Path;
use compliance_core::error::CoreError;
use compliance_core::models::{DetectedFact, TargetType};
use compliance_core::traits::ClassifierVerdict;
/// A minimal firmware-detection summary, mapped from tramiton's `BuildPlan`.
/// Kept small and tramiton-independent so the classifier and the test mock don't
/// need to construct a full tramiton plan.
#[derive(Debug, Clone, Default)]
pub struct FirmwareDetection {
/// The detecting provider (e.g. `zephyr`, `cmake`, `source-archaeology`).
pub provider: String,
/// Detection confidence: `low` | `medium` | `high`.
pub confidence: String,
/// Build-system label (e.g. `Zephyr`, `ESP-IDF`, `CMake`).
pub build_system: String,
/// Framework, when known (`zephyr`, `esp-idf`, `bare-metal`, ...).
pub framework: Option<String>,
/// Target board / MCU / arch.
pub target: FirmwareTarget,
/// Unresolved gaps in the plan.
pub gaps: Vec<String>,
}
/// The detected firmware target (board / MCU / arch).
#[derive(Debug, Clone, Default)]
pub struct FirmwareTarget {
/// Board name.
pub board: Option<String>,
/// MCU part.
pub mcu: Option<String>,
/// Architecture.
pub arch: Option<String>,
}
/// A source of tramiton firmware detection.
#[allow(async_fn_in_trait)]
pub trait FirmwareDetector: Send + Sync {
/// Run detection over a path, returning a firmware detection if tramiton
/// could form a build plan.
async fn detect(&self, path: &Path) -> Result<Option<FirmwareDetection>, CoreError>;
}
/// Uses `tramiton-core` in-process. The analysis is blocking (filesystem walk),
/// so it runs on a blocking thread to avoid stalling the async runtime. A path
/// with no recognizable build system yields `Ok(None)`.
pub struct TramitonNative;
impl FirmwareDetector for TramitonNative {
async fn detect(&self, path: &Path) -> Result<Option<FirmwareDetection>, CoreError> {
let path = path.to_path_buf();
let plan = tokio::task::spawn_blocking(move || {
let repo = tramiton_core::Repo::new(&path);
tramiton_core::provider::analyze(&repo)
})
.await
.map_err(|e| CoreError::Other(format!("tramiton detect task join error: {e}")))?
.map_err(|e| CoreError::Other(format!("tramiton analyze error: {e}")))?;
Ok(plan.map(|bp| detection_from_build_plan(&bp)))
}
}
/// Map tramiton's `BuildPlan` onto our minimal detection summary.
fn detection_from_build_plan(bp: &tramiton_core::BuildPlan) -> FirmwareDetection {
FirmwareDetection {
provider: bp.provider.clone(),
confidence: bp.confidence.to_string(),
build_system: bp.build_system.label().to_string(),
framework: bp.framework.clone(),
target: FirmwareTarget {
board: bp.target.board.clone(),
mcu: bp.target.mcu.clone(),
arch: bp.target.arch.clone(),
},
gaps: bp.gaps.clone(),
}
}
/// Map a firmware detection to a target type. Framework/build-system signals
/// distinguish RTOS from bare-metal from Yocto.
pub fn detection_to_target_type(det: &FirmwareDetection) -> TargetType {
let framework = det.framework.as_deref().unwrap_or("").to_lowercase();
let build_system = det.build_system.to_lowercase();
let signal = format!("{framework} {build_system} {}", det.provider.to_lowercase());
const RTOS: [&str; 6] = ["zephyr", "esp-idf", "freertos", "nuttx", "riot", "chibios"];
if signal.contains("bitbake") || signal.contains("yocto") || signal.contains("openembedded") {
TargetType::EmbeddedLinuxYocto
} else if RTOS.iter().any(|k| signal.contains(k)) {
TargetType::FirmwareRtos
} else {
TargetType::FirmwareBareMetal
}
}
/// Map tramiton's confidence label to a `[0,1]` score.
fn confidence_score(label: &str) -> f32 {
match label.to_lowercase().as_str() {
"high" => 0.9,
"medium" => 0.6,
"low" => 0.3,
_ => 0.4,
}
}
/// Turn a firmware detection into a classifier verdict, carrying the MCU / board
/// / build-system as facts.
pub fn detection_to_verdict(det: &FirmwareDetection) -> ClassifierVerdict {
let target_type = detection_to_target_type(det);
let mut facts = vec![DetectedFact::new(
"build_system",
det.build_system.clone(),
"tramiton",
)];
if let Some(fw) = &det.framework {
facts.push(DetectedFact::new("framework", fw.clone(), "tramiton"));
}
if let Some(mcu) = &det.target.mcu {
facts.push(DetectedFact::new("mcu", mcu.clone(), "tramiton"));
}
if let Some(board) = &det.target.board {
facts.push(DetectedFact::new("board", board.clone(), "tramiton"));
}
if let Some(arch) = &det.target.arch {
facts.push(DetectedFact::new("arch", arch.clone(), "tramiton"));
}
ClassifierVerdict {
target_type,
confidence: confidence_score(&det.confidence),
facts,
rationale: format!(
"tramiton detected build system '{}'{}",
det.build_system,
det.framework
.as_ref()
.map(|f| format!(" (framework {f})"))
.unwrap_or_default()
),
}
}
/// A deterministic [`FirmwareDetector`] for tests — returns a preset detection.
pub struct MockFirmwareDetector {
/// The detection to return (or `None` for "no detection").
pub detection: Option<FirmwareDetection>,
}
impl FirmwareDetector for MockFirmwareDetector {
async fn detect(&self, _path: &Path) -> Result<Option<FirmwareDetection>, CoreError> {
Ok(self.detection.clone())
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
fn detection(build_system: &str, framework: Option<&str>) -> FirmwareDetection {
FirmwareDetection {
provider: build_system.to_string(),
confidence: "high".to_string(),
build_system: build_system.to_string(),
framework: framework.map(|s| s.to_string()),
target: FirmwareTarget {
mcu: Some("stm32f429".to_string()),
..Default::default()
},
gaps: Vec::new(),
}
}
#[test]
fn zephyr_maps_to_rtos() {
assert_eq!(
detection_to_target_type(&detection("zephyr", Some("zephyr"))),
TargetType::FirmwareRtos
);
}
#[test]
fn bare_cmake_maps_to_bare_metal() {
assert_eq!(
detection_to_target_type(&detection("cmake", Some("bare-metal"))),
TargetType::FirmwareBareMetal
);
}
#[test]
fn bitbake_maps_to_yocto() {
assert_eq!(
detection_to_target_type(&detection("bitbake", None)),
TargetType::EmbeddedLinuxYocto
);
}
#[test]
fn verdict_carries_mcu_fact_and_confidence() {
let v = detection_to_verdict(&detection("esp-idf", Some("esp-idf")));
assert_eq!(v.target_type, TargetType::FirmwareRtos);
assert!((v.confidence - 0.9).abs() < f32::EPSILON);
assert!(v
.facts
.iter()
.any(|f| f.key == "mcu" && f.value == "stm32f429"));
}
}
+357
View File
@@ -0,0 +1,357 @@
//! Heuristic target-type classification from artifact kinds and source markers.
//!
//! Complements the tramiton firmware detector: this handles web / backend /
//! mobile / desktop / PLC by sniffing manifest files and file extensions in the
//! ingested code trees, plus strong priors from the artifact kinds themselves
//! (a PLC-project artifact is a PLC target; an `.ipa` is an iOS app).
use std::collections::HashSet;
use std::fs;
use std::path::Path;
use compliance_core::error::CoreError;
use compliance_core::models::{ArtifactKind, DetectedFact, TargetType};
use compliance_core::traits::{ClassificationInput, ClassifierVerdict, TargetClassifier};
/// Max directory depth scanned for marker files.
const SCAN_DEPTH: usize = 2;
/// Markers collected from a code tree.
#[derive(Default)]
struct Markers {
files: HashSet<String>,
dirs: HashSet<String>,
exts: HashSet<String>,
}
impl Markers {
fn has_file(&self, name: &str) -> bool {
self.files.contains(name)
}
fn has_ext(&self, ext: &str) -> bool {
self.exts.contains(ext)
}
fn any_dir_ends_with(&self, suffix: &str) -> bool {
self.dirs.iter().any(|d| d.ends_with(suffix))
}
}
/// Recursively collect marker file/dir/extension names up to [`SCAN_DEPTH`].
fn collect_markers(root: &Path) -> Markers {
let mut m = Markers::default();
scan_dir(root, 0, &mut m);
m
}
fn scan_dir(dir: &Path, depth: usize, m: &mut Markers) {
let Ok(entries) = fs::read_dir(dir) else {
return;
};
for entry in entries.flatten() {
let path = entry.path();
let name = entry.file_name().to_string_lossy().to_lowercase();
if path.is_dir() {
m.dirs.insert(name);
if depth < SCAN_DEPTH {
scan_dir(&path, depth + 1, m);
}
} else {
if let Some(ext) = path.extension() {
m.exts.insert(ext.to_string_lossy().to_lowercase());
}
m.files.insert(name);
}
}
}
/// Whether a `package.json` at `root` looks like a front-end app.
fn package_json_is_frontend(root: &Path) -> bool {
let Ok(content) = fs::read_to_string(root.join("package.json")) else {
return false;
};
let c = content.to_lowercase();
["react", "next", "vue", "@angular", "svelte", "vite"]
.iter()
.any(|f| c.contains(f))
}
/// The heuristic classifier: artifact-kind priors + source-tree markers.
pub struct HeuristicClassifier;
impl HeuristicClassifier {
/// Verdicts from the artifact kinds alone (no filesystem needed).
fn kind_priors(&self, input: &ClassificationInput<'_>) -> Vec<ClassifierVerdict> {
let mut out = Vec::new();
for a in input.artifacts {
let lower = a.source_ref.to_lowercase();
match a.kind {
ArtifactKind::PlcProject => out.push(verdict(
TargetType::PlcSps,
0.85,
"PLC project artifact",
vec![],
)),
ArtifactKind::MobilePackage => {
let (tt, why) = if lower.ends_with(".ipa") {
(TargetType::IosApp, "iOS package (.ipa)")
} else {
(TargetType::AndroidApp, "Android package (.apk/.aab)")
};
out.push(verdict(tt, 0.85, why, vec![]));
}
ArtifactKind::ContainerImage => out.push(verdict(
TargetType::BackendService,
0.4,
"container image",
vec![],
)),
ArtifactKind::FirmwareImage => out.push(verdict(
TargetType::FirmwareBareMetal,
0.35,
"firmware image (pending tramiton detection)",
vec![],
)),
ArtifactKind::LiveUrl if input.artifacts.len() == 1 => {
out.push(verdict(TargetType::WebApp, 0.3, "live URL only", vec![]))
}
_ => {}
}
}
out
}
/// Verdicts from scanning the ingested code trees for manifest markers.
fn source_verdicts(&self, input: &ClassificationInput<'_>) -> Vec<ClassifierVerdict> {
let mut out = Vec::new();
for a in input.artifacts {
if !matches!(a.kind, ArtifactKind::GitRepo | ArtifactKind::SourceArchive) {
continue;
}
let Some(path) = input.working_paths.get(&a.id) else {
continue;
};
let m = collect_markers(path);
// Mobile (checked first — strongest signal).
if m.has_file("androidmanifest.xml") || m.has_ext("apk") || m.has_ext("aab") {
out.push(verdict(
TargetType::AndroidApp,
0.8,
"Android manifest / gradle",
facts_lang("kotlin/java"),
));
}
if m.any_dir_ends_with(".xcodeproj")
|| m.has_file("info.plist")
|| m.has_file("podfile")
|| m.has_ext("ipa")
{
out.push(verdict(
TargetType::IosApp,
0.8,
"Xcode project / Info.plist",
facts_lang("swift/objc"),
));
}
// Desktop.
if m.has_ext("sln")
|| m.has_ext("csproj")
|| m.has_ext("vcxproj")
|| m.has_ext("desktop")
{
out.push(verdict(
TargetType::DesktopApp,
0.7,
"desktop project files",
facts_lang("dotnet/native"),
));
}
// PLC.
if m.has_ext("st") {
out.push(verdict(
TargetType::PlcSps,
0.8,
"Structured Text sources",
facts_lang("iec-61131-3"),
));
}
// Web vs backend from package.json.
if m.has_file("package.json") {
if package_json_is_frontend(path) {
out.push(verdict(
TargetType::WebApp,
0.65,
"package.json with a front-end framework",
facts_lang("javascript"),
));
} else {
out.push(verdict(
TargetType::BackendService,
0.55,
"package.json (no front-end framework)",
facts_lang("javascript"),
));
}
}
// Backend languages.
for (file, lang) in [
("cargo.toml", "rust"),
("go.mod", "go"),
("pom.xml", "java"),
("requirements.txt", "python"),
("pyproject.toml", "python"),
] {
if m.has_file(file) {
out.push(verdict(
TargetType::BackendService,
0.6,
"backend build manifest",
facts_lang(lang),
));
}
}
// Container-only.
if m.has_file("dockerfile") && out.is_empty() {
out.push(verdict(
TargetType::BackendService,
0.4,
"Dockerfile",
facts_lang("container"),
));
}
}
out
}
}
impl TargetClassifier for HeuristicClassifier {
fn name(&self) -> &str {
"heuristic"
}
async fn classify(
&self,
input: &ClassificationInput<'_>,
) -> Result<Vec<ClassifierVerdict>, CoreError> {
let mut out = self.kind_priors(input);
out.extend(self.source_verdicts(input));
Ok(out)
}
}
fn verdict(
target_type: TargetType,
confidence: f32,
rationale: &str,
facts: Vec<DetectedFact>,
) -> ClassifierVerdict {
ClassifierVerdict {
target_type,
confidence,
facts,
rationale: rationale.to_string(),
}
}
fn facts_lang(lang: &str) -> Vec<DetectedFact> {
vec![DetectedFact::new("language", lang, "heuristic")]
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::Artifact;
use std::collections::HashMap;
use std::path::PathBuf;
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-classify-{}", uuid::Uuid::new_v4()));
fs::create_dir_all(&p).expect("mkdir");
Self(p)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
async fn classify_tree(setup: impl FnOnce(&Path)) -> Vec<ClassifierVerdict> {
let scratch = Scratch::new();
setup(&scratch.0);
let artifact = Artifact::git_repo("https://git/x", "main");
let mut wp = HashMap::new();
wp.insert(artifact.id.clone(), scratch.0.clone());
let artifacts = vec![artifact];
let input = ClassificationInput {
artifacts: &artifacts,
working_paths: &wp,
description: None,
};
HeuristicClassifier
.classify(&input)
.await
.expect("classify")
}
#[tokio::test]
async fn frontend_package_json_is_webapp() {
let v = classify_tree(|root| {
fs::write(
root.join("package.json"),
r#"{"dependencies":{"react":"18"}}"#,
)
.unwrap();
})
.await;
assert!(v.iter().any(|x| x.target_type == TargetType::WebApp));
}
#[tokio::test]
async fn cargo_toml_is_backend() {
let v = classify_tree(|root| {
fs::write(root.join("Cargo.toml"), "[package]\nname='x'").unwrap();
})
.await;
assert!(v
.iter()
.any(|x| x.target_type == TargetType::BackendService));
}
#[tokio::test]
async fn android_manifest_is_android() {
let v = classify_tree(|root| {
fs::write(root.join("AndroidManifest.xml"), "<manifest/>").unwrap();
})
.await;
assert!(v.iter().any(|x| x.target_type == TargetType::AndroidApp));
}
#[tokio::test]
async fn structured_text_is_plc() {
let v = classify_tree(|root| {
fs::write(root.join("main.st"), "PROGRAM main END_PROGRAM").unwrap();
})
.await;
assert!(v.iter().any(|x| x.target_type == TargetType::PlcSps));
}
#[tokio::test]
async fn ipa_artifact_prior_is_ios() {
let artifacts = vec![Artifact::mobile_package("app.ipa")];
let wp = HashMap::new();
let input = ClassificationInput {
artifacts: &artifacts,
working_paths: &wp,
description: None,
};
let v = HeuristicClassifier
.classify(&input)
.await
.expect("classify");
assert!(v.iter().any(|x| x.target_type == TargetType::IosApp));
}
}
+226
View File
@@ -0,0 +1,226 @@
//! Target classification.
//!
//! Runs the classifier registry over a target's artifacts and their ingested
//! working paths, then merges and ranks the verdicts into a [`Classification`].
//! The registry is the heuristic classifier (artifact kinds + source markers)
//! plus the tramiton firmware detector (behind a [`FirmwareDetector`] port).
mod firmware;
mod language;
pub use firmware::{
FirmwareDetection, FirmwareDetector, FirmwareTarget, MockFirmwareDetector, TramitonNative,
};
pub use language::HeuristicClassifier;
use std::collections::HashMap;
use std::path::PathBuf;
use compliance_core::error::CoreError;
use compliance_core::models::{
ArtifactKind, Classification, DetectedFact, OnboardedTarget, TargetType, TargetTypeCandidate,
};
use compliance_core::traits::{ClassificationInput, ClassifierVerdict, TargetClassifier};
use firmware::detection_to_verdict;
/// Classify a target from its artifacts and their ingested working paths, using
/// the heuristic classifier plus the tramiton firmware detector. Verdicts are
/// merged (max confidence per target type) and ranked into a [`Classification`].
pub async fn classify_target<D: FirmwareDetector>(
target: &OnboardedTarget,
working_paths: &HashMap<String, PathBuf>,
firmware_detector: &D,
) -> Result<Classification, CoreError> {
let input = ClassificationInput {
artifacts: &target.artifacts,
working_paths,
description: target.description.as_deref(),
};
let mut verdicts = Vec::new();
let mut detected_by = Vec::new();
let heuristic = HeuristicClassifier.classify(&input).await?;
if !heuristic.is_empty() {
detected_by.push("heuristic".to_string());
}
verdicts.extend(heuristic);
// Tramiton firmware detection over firmware / code working paths.
let mut tramiton_used = false;
for artifact in &target.artifacts {
if !matches!(
artifact.kind,
ArtifactKind::FirmwareImage | ArtifactKind::GitRepo | ArtifactKind::SourceArchive
) {
continue;
}
let Some(path) = working_paths.get(&artifact.id) else {
continue;
};
if let Some(detection) = firmware_detector.detect(path).await? {
verdicts.push(detection_to_verdict(&detection));
tramiton_used = true;
}
}
if tramiton_used {
detected_by.push("tramiton".to_string());
}
Ok(rank(verdicts, detected_by, target.target_type))
}
/// Merge verdicts by target type (keeping the max confidence and its rationale),
/// dedupe facts, rank by descending confidence, and assemble a [`Classification`].
/// Falls back to the declared type when no verdict is produced.
fn rank(
verdicts: Vec<ClassifierVerdict>,
detected_by: Vec<String>,
fallback: TargetType,
) -> Classification {
let mut best: HashMap<TargetType, (f32, String)> = HashMap::new();
let mut facts: Vec<DetectedFact> = Vec::new();
for verdict in verdicts {
for fact in verdict.facts {
if !facts
.iter()
.any(|e| e.key == fact.key && e.value == fact.value)
{
facts.push(fact);
}
}
let entry = best
.entry(verdict.target_type)
.or_insert((0.0, String::new()));
if verdict.confidence > entry.0 {
*entry = (verdict.confidence, verdict.rationale);
}
}
let mut candidates: Vec<TargetTypeCandidate> = best
.into_iter()
.map(
|(target_type, (confidence, rationale))| TargetTypeCandidate {
target_type,
confidence,
rationale,
},
)
.collect();
// Descending confidence; ties broken by type name for deterministic ordering.
candidates.sort_by(|a, b| {
b.confidence
.partial_cmp(&a.confidence)
.unwrap_or(std::cmp::Ordering::Equal)
.then_with(|| a.target_type.to_string().cmp(&b.target_type.to_string()))
});
let suggested = candidates
.first()
.map(|c| c.target_type)
.unwrap_or(fallback);
Classification {
suggested,
candidates,
facts,
detected_by,
detected_at: chrono::Utc::now(),
confirmed: false,
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::Artifact;
use std::fs;
use std::path::Path;
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-classify-mod-{}", uuid::Uuid::new_v4()));
fs::create_dir_all(&p).expect("mkdir");
Self(p)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
fn no_firmware() -> MockFirmwareDetector {
MockFirmwareDetector { detection: None }
}
#[tokio::test]
async fn backend_repo_classifies_as_backend() {
let scratch = Scratch::new();
fs::write(scratch.0.join("go.mod"), "module x").unwrap();
let artifact = Artifact::git_repo("https://git/x", "main");
let mut wp = HashMap::new();
wp.insert(artifact.id.clone(), scratch.0.clone());
let mut target = OnboardedTarget::new("x".to_string(), TargetType::WebApp);
target.artifacts.push(artifact);
let c = classify_target(&target, &wp, &no_firmware())
.await
.expect("classify");
assert_eq!(c.suggested, TargetType::BackendService);
assert!(c.detected_by.contains(&"heuristic".to_string()));
assert!(!c.confirmed);
}
#[tokio::test]
async fn firmware_detector_verdict_ranks_top() {
let scratch = Scratch::new();
fs::write(scratch.0.join("fw.bin"), b"x").unwrap();
let artifact =
Artifact::firmware_image(scratch.0.join("fw.bin").to_string_lossy().to_string());
let mut wp = HashMap::new();
wp.insert(artifact.id.clone(), scratch.0.clone());
let mut target = OnboardedTarget::new("fw".to_string(), TargetType::FirmwareBareMetal);
target.artifacts.push(artifact);
let detector = MockFirmwareDetector {
detection: Some(FirmwareDetection {
provider: "zephyr".to_string(),
confidence: "high".to_string(),
build_system: "zephyr".to_string(),
framework: Some("zephyr".to_string()),
target: FirmwareTarget {
mcu: Some("nrf52840".to_string()),
..Default::default()
},
gaps: vec![],
}),
};
let c = classify_target(&target, &wp, &detector)
.await
.expect("classify");
// tramiton's high-confidence RTOS verdict beats the weak firmware prior.
assert_eq!(c.suggested, TargetType::FirmwareRtos);
assert!(c.detected_by.contains(&"tramiton".to_string()));
assert!(c.facts.iter().any(|f| f.key == "mcu"));
}
#[tokio::test]
async fn no_signal_falls_back_to_declared_type() {
let scratch = Scratch::new();
let _ = Path::new(&scratch.0);
let target = OnboardedTarget::new("empty".to_string(), TargetType::DesktopApp);
let wp = HashMap::new();
let c = classify_target(&target, &wp, &no_firmware())
.await
.expect("classify");
assert_eq!(c.suggested, TargetType::DesktopApp);
assert!(c.candidates.is_empty());
}
}
+14 -1
View File
@@ -42,9 +42,17 @@ pub fn load_config() -> Result<AgentConfig, AgentError> {
.unwrap_or(3001),
scan_schedule: env_var_opt("SCAN_SCHEDULE").unwrap_or_else(|| "0 0 */6 * * *".to_string()),
cve_monitor_schedule: env_var_opt("CVE_MONITOR_SCHEDULE")
.unwrap_or_else(|| "0 0 0 * * *".to_string()),
.unwrap_or_else(|| "0 0 * * * *".to_string()),
git_clone_base_path: env_var_opt("GIT_CLONE_BASE_PATH")
.unwrap_or_else(|| "/tmp/compliance-scanner/repos".to_string()),
artifact_store_base_path: env_var_opt("ARTIFACT_STORE_BASE_PATH")
.unwrap_or_else(|| "/data/compliance-scanner/artifacts".to_string()),
// Defaults ON: the unified onboarded-target pipeline is now the primary
// path (no legacy `repositories` data in production). Set
// `UNIFIED_PIPELINE=0` to fall back to the legacy repository pipeline.
unified_pipeline: env_var_opt("UNIFIED_PIPELINE")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(true),
ssh_key_path: env_var_opt("SSH_KEY_PATH")
.unwrap_or_else(|| "/data/compliance-scanner/ssh/id_ed25519".to_string()),
keycloak_url: env_var_opt("KEYCLOAK_URL"),
@@ -54,7 +62,12 @@ pub fn load_config() -> Result<AgentConfig, AgentError> {
pentest_verification_email: env_var_opt("PENTEST_VERIFICATION_EMAIL"),
pentest_imap_host: env_var_opt("PENTEST_IMAP_HOST"),
pentest_imap_port: env_var_opt("PENTEST_IMAP_PORT").and_then(|p| p.parse().ok()),
pentest_imap_tls: env_var_opt("PENTEST_IMAP_TLS")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(true),
pentest_imap_username: env_var_opt("PENTEST_IMAP_USERNAME"),
pentest_imap_password: env_secret_opt("PENTEST_IMAP_PASSWORD"),
admin_api_token: env_secret_opt("ADMIN_API_TOKEN"),
tenant_registry_url: env_var_opt("TENANT_REGISTRY_URL"),
})
}
+297
View File
@@ -1,11 +1,233 @@
use std::sync::Arc;
use dashmap::DashMap;
use mongodb::bson::doc;
use mongodb::options::IndexOptions;
use mongodb::{Client, Collection, IndexModel};
use sha2::{Digest, Sha256};
use compliance_core::models::*;
use compliance_core::TenantContext;
use crate::error::AgentError;
/// Mongo enforces a 63-byte cap on database names (older clusters: 64
/// on Linux, 63 on Windows; we target the conservative limit).
const MAX_DB_NAME_LEN: usize = 63;
/// Hex length of the SHA-256 truncation used for the hash fallback
/// tenant DB name (16 bytes → 32 hex chars). 16 bytes gives ~2^64
/// birthday-collision resistance — at our 10s-100s tenant scale this
/// is effectively impossible to hit.
const HASH_HEX_LEN: usize = 32;
/// Largest `db_prefix` that still guarantees the hash-fallback name
/// fits in the 63-byte cap: `prefix + "_" + 32 hex chars`.
const MAX_PREFIX_LEN: usize = MAX_DB_NAME_LEN - 1 - HASH_HEX_LEN;
/// Per-tenant Mongo connection broker (M7.2 isolation model).
///
/// Holds one [`Client`] and hands out [`Database`] handles physically
/// scoped to `<db_prefix>_<tenant_id>`. The driver is the isolation
/// boundary — a handle for tenant A cannot see tenant B's documents
/// because it is connected to a different database, not because of an
/// application-level filter.
///
/// Index creation runs idempotently the first time each tenant is seen
/// in the process's lifetime. Mongo's `createIndex` is itself idempotent
/// by index name; the in-memory `ensured` set just skips the round-trip.
#[derive(Clone, Debug)]
pub struct DatabasePool {
client: Client,
db_prefix: String,
ensured: Arc<DashMap<String, ()>>,
}
impl DatabasePool {
/// Connect to the cluster and prepare to hand out tenant databases
/// named `<db_prefix>_<tenant_id>`.
///
/// Validates `db_prefix.len() <= MAX_PREFIX_LEN` so the
/// hash-fallback path is provably within Mongo's 63-byte db-name
/// cap. Refuses to construct a pool that could ever produce an
/// over-long name.
pub async fn connect(uri: &str, db_prefix: &str) -> Result<Self, AgentError> {
if db_prefix.len() > MAX_PREFIX_LEN {
return Err(AgentError::Other(format!(
"db_prefix '{db_prefix}' is {} chars; max is {MAX_PREFIX_LEN} so the \
hash-fallback tenant DB name fits Mongo's {MAX_DB_NAME_LEN}-byte cap",
db_prefix.len()
)));
}
let client = Client::with_uri_str(uri).await?;
client
.database("admin")
.run_command(doc! { "ping": 1 })
.await?;
tracing::info!(
"MongoDB cluster reachable; per-tenant pool ready (db prefix '{db_prefix}')"
);
Ok(Self {
client,
db_prefix: db_prefix.to_string(),
ensured: Arc::new(DashMap::new()),
})
}
/// Return a [`Database`] scoped to this tenant. Ensures indexes on
/// first call per tenant (per process). Cheap on the hot path —
/// subsequent calls skip the round-trip.
pub async fn for_tenant(&self, ctx: &TenantContext) -> Result<Database, AgentError> {
self.for_tenant_id(&ctx.tenant_id).await
}
/// Like [`Self::for_tenant`] but accepts a bare tenant_id.
/// For background paths (scheduler, webhooks, pipeline orchestrators)
/// that don't have a full [`TenantContext`] but know which tenant
/// they're operating on (typically resolved from a URL path, a job
/// argument, or the registry).
pub async fn for_tenant_id(&self, tenant_id: &str) -> Result<Database, AgentError> {
let db_name = self.tenant_db_name(tenant_id);
let db = Database::from_database(self.client.database(&db_name));
// `DashMap::insert` returns the previous value; `None` means we
// were the first writer for this tenant_id and own the
// index-ensure work.
if self.ensured.insert(tenant_id.to_string(), ()).is_none() {
if let Err(e) = db.ensure_indexes().await {
// Roll the marker back so the next request retries.
self.ensured.remove(tenant_id);
return Err(e);
}
tracing::debug!(
tenant_id = %tenant_id,
db_name = %db_name,
"Indexes ensured for tenant database"
);
}
Ok(db)
}
/// Compute the Mongo database name for a tenant. Public for tests
/// and tenant offboarding (`pool.client().database(name).drop()`).
///
/// Format: `<prefix>_<sanitized_tenant_id>` if it fits the 63-byte
/// cap, else `<prefix>_<sha256-16-byte-hex-of-tenant_id>`. The
/// `db_prefix` length invariant established at [`Self::connect`]
/// guarantees the hash-fallback name always fits — no runtime
/// assertion needed.
///
/// Collision resistance: the hash fallback is a 16-byte SHA-256
/// truncation, which gives ~2^64 birthday-collision resistance. At
/// our 10s100s tenant scale the probability of two tenant_ids
/// colliding is effectively zero. (8-byte truncation would have
/// been ~2^32 — too close for comfort on a regulated product.)
pub fn tenant_db_name(&self, tenant_id: &str) -> String {
let sanitized = sanitize_tenant_id(tenant_id);
let natural = format!("{}_{}", self.db_prefix, sanitized);
if natural.len() <= MAX_DB_NAME_LEN {
natural
} else {
let mut hasher = Sha256::new();
hasher.update(tenant_id.as_bytes());
let digest = hasher.finalize();
let suffix = hex::encode(&digest[..HASH_HEX_LEN / 2]);
format!("{}_{}", self.db_prefix, suffix)
}
}
/// Raw client handle. Reserved for cross-tenant admin flows that
/// must opt in explicitly (tenant listing, drop-on-offboard).
pub fn client(&self) -> &Client {
&self.client
}
/// Cross-tenant admin database used by features that intentionally
/// span tenants (today: MCP bearer tokens — each token row carries
/// a `tenant_id` and the MCP server reads them to route requests).
///
/// The name `<db_prefix>__admin` (double underscore) is reserved —
/// the sanitizer never produces it for a normal tenant DB because
/// the natural format is `<db_prefix>_<sanitized_tenant_id>` (one
/// underscore) and tenant_ids would have to start with `_admin` to
/// collide. New tenant provisioning should reject such ids.
pub fn admin_db(&self) -> mongodb::Database {
self.client.database(&self.admin_db_name())
}
/// Name of the admin database — public so tests / operators can
/// drop it via the raw client.
pub fn admin_db_name(&self) -> String {
format!("{}__admin", self.db_prefix)
}
/// List every Mongo database currently belonging to this pool,
/// identified by the `<db_prefix>_` prefix. The result is the raw
/// database names — opening one for offboarding/cleanup goes
/// through [`Self::client`].
///
/// Note: hashed-fallback names (very long tenant_ids) lose the
/// original tenant_id at the cluster level — we know a database
/// exists for *some* tenant but not which one. In practice
/// tenant_ids are UUIDs (36 chars) and never hit the fallback,
/// so this is a theoretical concern, not an operational one.
pub async fn list_tenant_db_names(&self) -> Result<Vec<String>, AgentError> {
let prefix = format!("{}_", self.db_prefix);
let names = self.client.list_database_names().await?;
Ok(names
.into_iter()
.filter(|n| n.starts_with(&prefix))
.collect())
}
/// Tenant ids for every provisioned tenant database, derived by stripping
/// the `<prefix>_` from the database names. Skips the admin database
/// (`<prefix>__admin`). Hash-fallback names (very long tenant_ids) are lost
/// at the cluster level and cannot be recovered here — in practice tenant
/// ids are UUIDs and never hit that path. Used by the migration CLI's
/// `--all` mode.
pub async fn list_tenant_ids(&self) -> Result<Vec<String>, AgentError> {
let prefix = format!("{}_", self.db_prefix);
Ok(self
.list_tenant_db_names()
.await?
.into_iter()
.filter_map(|n| n.strip_prefix(&prefix).map(str::to_string))
.filter(|id| !id.starts_with('_'))
.collect())
}
/// Drop the database for a specific tenant. Used by GDPR delete
/// and tenant offboarding. Idempotent — dropping a non-existent
/// database is a no-op at the driver level.
///
/// Also evicts the tenant from the in-memory `ensured` set so a
/// later re-provision triggers fresh `ensure_indexes`.
pub async fn drop_tenant(&self, tenant_id: &str) -> Result<(), AgentError> {
let db_name = self.tenant_db_name(tenant_id);
self.client.database(&db_name).drop().await?;
self.ensured.remove(tenant_id);
tracing::info!(
tenant_id = %tenant_id,
db_name = %db_name,
"Dropped tenant database"
);
Ok(())
}
}
/// Mongo database names disallow `/`, `\`, `.`, `"`, `$`, ` `, and NUL.
/// breakpilot-dev tenant_ids are UUIDs so this is belt-and-braces, but
/// it lets the pool tolerate any future tenant_id shape without surprise.
fn sanitize_tenant_id(tenant_id: &str) -> String {
tenant_id
.chars()
.map(|c| match c {
'/' | '\\' | '.' | '"' | '$' | ' ' | '\0' => '_',
c => c,
})
.collect()
}
#[derive(Clone, Debug)]
pub struct Database {
inner: mongodb::Database,
@@ -20,6 +242,12 @@ impl Database {
Ok(Self { inner: db })
}
/// Wrap an already-resolved Mongo database. Used by [`DatabasePool`]
/// to hand out tenant-scoped handles without a fresh client per tenant.
pub(crate) fn from_database(inner: mongodb::Database) -> Self {
Self { inner }
}
pub async fn ensure_indexes(&self) -> Result<(), AgentError> {
// repositories: unique git_url
self.repositories()
@@ -78,6 +306,25 @@ impl Database {
)
.await?;
// cve_notifications: unique cve_id + repo_id + package, status filter
self.cve_notifications()
.create_index(
IndexModel::builder()
.keys(
doc! { "cve_id": 1, "repo_id": 1, "package_name": 1, "package_version": 1 },
)
.options(IndexOptions::builder().unique(true).build())
.build(),
)
.await?;
self.cve_notifications()
.create_index(
IndexModel::builder()
.keys(doc! { "status": 1, "created_at": -1 })
.build(),
)
.await?;
// tracker_issues: unique finding_id
self.tracker_issues()
.create_index(
@@ -198,6 +445,36 @@ impl Database {
)
.await?;
// onboarded_targets: multikey on artifact source ref (webhook + dedupe
// lookup). Non-unique — "one git URL per tenant" is enforced in the
// create handler, since a unique multikey index on an array field has
// null-collision caveats.
self.onboarded_targets()
.create_index(
IndexModel::builder()
.keys(doc! { "artifacts.source_ref": 1 })
.build(),
)
.await?;
// onboarded_targets: multikey on artifact kind
self.onboarded_targets()
.create_index(
IndexModel::builder()
.keys(doc! { "artifacts.kind": 1 })
.build(),
)
.await?;
// onboarded_targets: target_type filter
self.onboarded_targets()
.create_index(
IndexModel::builder()
.keys(doc! { "target_type": 1 })
.build(),
)
.await?;
tracing::info!("Database indexes ensured");
Ok(())
}
@@ -222,6 +499,12 @@ impl Database {
self.inner.collection("cve_alerts")
}
pub fn cve_notifications(
&self,
) -> Collection<compliance_core::models::notification::CveNotification> {
self.inner.collection("cve_notifications")
}
pub fn tracker_issues(&self) -> Collection<TrackerIssue> {
self.inner.collection("tracker_issues")
}
@@ -248,6 +531,20 @@ impl Database {
self.inner.collection("dast_targets")
}
/// The unified onboarding targets that replace `repositories` and
/// `dast_targets`. Ids are preserved from the legacy collections during
/// migration so downstream `repo_id` / `target_id` references keep resolving.
pub fn onboarded_targets(&self) -> Collection<OnboardedTarget> {
self.inner.collection("onboarded_targets")
}
/// A typed handle to an arbitrary collection by name. For bookkeeping
/// collections without a dedicated model (e.g. `schema_migrations`,
/// `onboarding_migration_log`).
pub fn collection_named<T: Send + Sync>(&self, name: &str) -> Collection<T> {
self.inner.collection(name)
}
pub fn dast_scan_runs(&self) -> Collection<DastScanRun> {
self.inner.collection("dast_scan_runs")
}
+154
View File
@@ -0,0 +1,154 @@
//! Content-addressed blob storage and archive extraction for ingest.
//!
//! Blobs are stored at `<base>/blobs/<sha[0:2]>/<sha>` and deduplicated by
//! digest; per-run working directories live under `<base>/work/`.
use std::fs::{self, File};
use std::io::{self, Read};
use std::path::{Path, PathBuf};
use sha2::{Digest, Sha256};
use crate::error::AgentError;
/// Read buffer size for streaming hashes/copies (64 KiB).
const BUF_LEN: usize = 64 * 1024;
/// Stream-hash a file with SHA-256, returning the lowercase-hex digest and the
/// byte length. Streams so large firmware images never load fully into memory.
pub fn hash_file(path: &Path) -> Result<(String, u64), AgentError> {
let mut file = File::open(path)?;
let mut hasher = Sha256::new();
let mut buf = [0u8; BUF_LEN];
let mut total: u64 = 0;
loop {
let n = file.read(&mut buf)?;
if n == 0 {
break;
}
hasher.update(&buf[..n]);
total += n as u64;
}
Ok((hex::encode(hasher.finalize()), total))
}
/// Copy `src` into the content-addressed blob store under `base`, returning the
/// stored path. Idempotent: an already-present blob is not rewritten.
pub fn store_file(base: &Path, src: &Path, sha: &str) -> Result<PathBuf, AgentError> {
if sha.len() < 2 {
return Err(AgentError::Other(format!("invalid content hash '{sha}'")));
}
let dir = base.join("blobs").join(&sha[0..2]);
fs::create_dir_all(&dir)?;
let dest = dir.join(sha);
if !dest.exists() {
fs::copy(src, &dest)?;
}
Ok(dest)
}
/// Extract a zip archive into `dest` (created if needed). `enclosed_name`
/// sanitizes each entry path, so this is safe against zip-slip traversal.
pub fn extract_zip(archive: &Path, dest: &Path) -> Result<(), AgentError> {
let file = File::open(archive)?;
let mut zip =
zip::ZipArchive::new(file).map_err(|e| AgentError::Other(format!("open zip: {e}")))?;
fs::create_dir_all(dest)?;
for i in 0..zip.len() {
let mut entry = zip
.by_index(i)
.map_err(|e| AgentError::Other(format!("read zip entry: {e}")))?;
// `enclosed_name` returns `None` for traversal-unsafe paths — skip them.
let Some(rel) = entry.enclosed_name() else {
continue;
};
let out = dest.join(rel);
if entry.is_dir() {
fs::create_dir_all(&out)?;
} else {
if let Some(parent) = out.parent() {
fs::create_dir_all(parent)?;
}
let mut outfile = File::create(&out)?;
io::copy(&mut entry, &mut outfile)?;
}
}
Ok(())
}
/// The working directory for one artifact of a target: `<base>/work/<target>/<artifact>`.
pub fn work_dir(base: &Path, target_id: &str, artifact_id: &str) -> PathBuf {
base.join("work").join(target_id).join(artifact_id)
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
/// A unique scratch directory, removed on drop.
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-ingest-{}", uuid::Uuid::new_v4()));
fs::create_dir_all(&p).expect("mkdir scratch");
Self(p)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
#[test]
fn hash_is_stable_and_reports_size() {
let dir = Scratch::new();
let f = dir.path().join("a.bin");
fs::write(&f, b"hello world").expect("write");
let (sha, size) = hash_file(&f).expect("hash");
assert_eq!(size, 11);
// Known SHA-256 of "hello world".
assert_eq!(
sha,
"b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"
);
}
#[test]
fn store_is_content_addressed_and_idempotent() {
let base = Scratch::new();
let src = base.path().join("src.bin");
fs::write(&src, b"payload").expect("write");
let (sha, _) = hash_file(&src).expect("hash");
let p1 = store_file(base.path(), &src, &sha).expect("store");
let p2 = store_file(base.path(), &src, &sha).expect("store again");
assert_eq!(p1, p2);
assert!(p1.ends_with(&sha));
assert!(p1.starts_with(base.path().join("blobs").join(&sha[0..2])));
assert_eq!(fs::read(&p1).expect("read"), b"payload");
}
#[test]
fn extract_zip_writes_entries() {
let base = Scratch::new();
let archive = base.path().join("a.zip");
{
let file = File::create(&archive).expect("create");
let mut w = zip::ZipWriter::new(file);
let opts: zip::write::SimpleFileOptions = Default::default();
w.start_file("dir/hello.txt", opts).expect("start");
io::Write::write_all(&mut w, b"hi").expect("write");
w.finish().expect("finish");
}
let dest = base.path().join("out");
extract_zip(&archive, &dest).expect("extract");
assert_eq!(
fs::read_to_string(dest.join("dir/hello.txt")).expect("read"),
"hi"
);
}
}
+334
View File
@@ -0,0 +1,334 @@
//! Artifact ingest.
//!
//! Normalizes each [`Artifact`] on an [`OnboardedTarget`] into a local working
//! path plus recorded metadata (content hash, size, discovered facts) that the
//! classifier and scanners consume. Every blob is SHA-256 hashed — that digest
//! is also the reconciliation key against sibling products (a firmware sha256
//! matches tramiton's `Artifact.sha256`).
mod blob;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use compliance_core::models::{Artifact, ArtifactKind, DetectedFact, OnboardedTarget};
use compliance_core::AgentConfig;
use crate::error::AgentError;
use crate::pipeline::git::{GitOps, RepoCredentials};
/// The paths and identifiers an ingest needs. Decoupled from the full
/// [`AgentConfig`] so ingest is testable without a complete config.
pub struct IngestContext<'a> {
/// Base directory for content-addressed blobs and working dirs.
pub artifact_store_base: &'a Path,
/// Base directory for git clones.
pub git_clone_base: &'a str,
/// Default SSH key path (used when an artifact provides none).
pub ssh_key_path: &'a str,
/// The id of the target these artifacts belong to (namespaces working dirs).
pub target_id: &'a str,
}
impl<'a> IngestContext<'a> {
/// Build an ingest context from the agent config for a given target.
pub fn from_config(config: &'a AgentConfig, target_id: &'a str) -> Self {
Self {
artifact_store_base: Path::new(&config.artifact_store_base_path),
git_clone_base: &config.git_clone_base_path,
ssh_key_path: &config.ssh_key_path,
target_id,
}
}
}
/// The result of ingesting one artifact.
pub struct IngestedArtifact {
/// The artifact this corresponds to ([`Artifact::id`]).
pub artifact_id: String,
/// The artifact kind.
pub kind: ArtifactKind,
/// Local working path (clone dir, extracted dir, or blob file). `None` for
/// artifacts with no on-disk form (live URL, plaintext, container ref).
pub working_path: Option<PathBuf>,
/// SHA-256 of the content (blobs) or git head SHA (git repos).
pub content_hash: Option<String>,
/// Stored blob size in bytes, when applicable.
pub size_bytes: Option<u64>,
/// Facts discovered during ingest.
pub facts: Vec<DetectedFact>,
}
/// All ingested artifacts for a target, keyed by artifact id.
pub struct IngestSet {
/// The ingested artifacts, keyed by [`Artifact::id`].
pub by_artifact: HashMap<String, IngestedArtifact>,
}
impl IngestSet {
/// The working paths of every ingested artifact that has one — the input the
/// classifier expects.
pub fn working_paths(&self) -> HashMap<String, PathBuf> {
self.by_artifact
.iter()
.filter_map(|(id, a)| a.working_path.clone().map(|p| (id.clone(), p)))
.collect()
}
/// The ingest result for a specific artifact.
pub fn get(&self, artifact_id: &str) -> Option<&IngestedArtifact> {
self.by_artifact.get(artifact_id)
}
}
/// Ingest every artifact on a target.
pub fn ingest_all(
target: &OnboardedTarget,
ctx: &IngestContext<'_>,
) -> Result<IngestSet, AgentError> {
let mut by_artifact = HashMap::new();
for artifact in &target.artifacts {
let ingested = ingest_artifact(artifact, ctx)?;
by_artifact.insert(artifact.id.clone(), ingested);
}
Ok(IngestSet { by_artifact })
}
/// Ingest a single artifact, dispatching on its kind.
pub fn ingest_artifact(
artifact: &Artifact,
ctx: &IngestContext<'_>,
) -> Result<IngestedArtifact, AgentError> {
match artifact.kind {
ArtifactKind::GitRepo => ingest_git(artifact, ctx),
ArtifactKind::SourceArchive | ArtifactKind::MobilePackage | ArtifactKind::PlcProject => {
ingest_blob(artifact, ctx, true)
}
ArtifactKind::FirmwareImage => ingest_blob(artifact, ctx, false),
ArtifactKind::ContainerImage => Ok(metadata_only(
artifact,
DetectedFact::new("container_ref", artifact.source_ref.as_str(), "ingest"),
)),
ArtifactKind::LiveUrl => Ok(metadata_only(
artifact,
DetectedFact::new("live_url", artifact.source_ref.as_str(), "ingest"),
)),
ArtifactKind::PlaintextDescription => Ok(metadata_only(
artifact,
DetectedFact::new(
"description_len",
artifact.source_ref.len().to_string(),
"ingest",
),
)),
}
}
/// Clone (or fetch) a git artifact, recording the head SHA as the content hash.
fn ingest_git(
artifact: &Artifact,
ctx: &IngestContext<'_>,
) -> Result<IngestedArtifact, AgentError> {
let creds = credentials_for(artifact, ctx.ssh_key_path);
let git_ops = GitOps::new(ctx.git_clone_base, creds);
let repo_path = git_ops.clone_or_fetch(&artifact.source_ref, &artifact.id)?;
let head = GitOps::get_head_sha(&repo_path).ok();
Ok(IngestedArtifact {
artifact_id: artifact.id.clone(),
kind: artifact.kind,
working_path: Some(repo_path),
content_hash: head,
size_bytes: None,
facts: Vec::new(),
})
}
/// Store a blob artifact content-addressed. When `extract` is set and the blob
/// is a zip container (source archive, APK/AAB/IPA), also unpack it into a
/// working directory; otherwise the working path is the stored blob.
fn ingest_blob(
artifact: &Artifact,
ctx: &IngestContext<'_>,
extract: bool,
) -> Result<IngestedArtifact, AgentError> {
let base = ctx.artifact_store_base;
let src = local_source(artifact)?;
let (sha, size) = blob::hash_file(&src)?;
let stored = blob::store_file(base, &src, &sha)?;
let mut facts = Vec::new();
let working_path = if extract {
let dest = blob::work_dir(base, ctx.target_id, &artifact.id);
match blob::extract_zip(&stored, &dest) {
Ok(()) => dest,
Err(e) => {
// Not a zip (e.g. a tar.gz source archive) — keep the blob and
// note it so later stages can decide what to do.
facts.push(DetectedFact::new(
"archive_unextracted",
e.to_string(),
"ingest",
));
stored.clone()
}
}
} else {
stored.clone()
};
Ok(IngestedArtifact {
artifact_id: artifact.id.clone(),
kind: artifact.kind,
working_path: Some(working_path),
content_hash: Some(sha),
size_bytes: Some(size),
facts,
})
}
/// An artifact with no on-disk form: record a single fact, no hash/path.
fn metadata_only(artifact: &Artifact, fact: DetectedFact) -> IngestedArtifact {
IngestedArtifact {
artifact_id: artifact.id.clone(),
kind: artifact.kind,
working_path: None,
content_hash: None,
size_bytes: None,
facts: vec![fact],
}
}
/// The local file backing a blob artifact: its `stored_path` if already
/// uploaded, else its `source_ref` interpreted as a filesystem path.
fn local_source(artifact: &Artifact) -> Result<PathBuf, AgentError> {
let path = artifact
.stored_path
.as_deref()
.unwrap_or(artifact.source_ref.as_str());
let path = PathBuf::from(path);
if !path.exists() {
return Err(AgentError::Other(format!(
"artifact {} source not found at {}",
artifact.id,
path.display()
)));
}
Ok(path)
}
/// Build git credentials from an artifact's auth plus a default SSH key path.
fn credentials_for(artifact: &Artifact, default_ssh_key_path: &str) -> RepoCredentials {
let auth = artifact.auth.as_ref();
RepoCredentials {
ssh_key_path: auth
.and_then(|a| a.ssh_key_path.clone())
.or_else(|| Some(default_ssh_key_path.to_string())),
auth_token: auth.and_then(|a| a.secret.clone()),
auth_username: auth.and_then(|a| a.username.clone()),
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{ArtifactAuth, TargetType};
/// A unique scratch directory, removed on drop.
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-ingest-mod-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&p).expect("mkdir scratch");
Self(p)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn ctx_for<'a>(store: &'a Path, target_id: &'a str) -> IngestContext<'a> {
IngestContext {
artifact_store_base: store,
git_clone_base: "/tmp/cs-ingest-test-repos",
ssh_key_path: "/tmp/cs-ingest-test-ssh",
target_id,
}
}
#[test]
fn firmware_blob_is_hashed_and_stored() {
let scratch = Scratch::new();
let store = scratch.0.join("store");
let fw = scratch.0.join("fw.bin");
std::fs::write(&fw, b"firmware-bytes").expect("write");
let ctx = ctx_for(&store, "t1");
let artifact = Artifact::firmware_image(fw.to_string_lossy().to_string());
let out = ingest_artifact(&artifact, &ctx).expect("ingest");
assert_eq!(out.kind, ArtifactKind::FirmwareImage);
assert_eq!(out.size_bytes, Some(14));
let sha = out.content_hash.expect("hash");
assert_eq!(sha.len(), 64);
// working path is the content-addressed blob
let wp = out.working_path.expect("working path");
assert!(wp.starts_with(store.join("blobs")));
}
#[test]
fn live_url_has_no_blob() {
let scratch = Scratch::new();
let store = scratch.0.join("store");
let ctx = ctx_for(&store, "t1");
let artifact = Artifact::live_url("https://example.com");
let out = ingest_artifact(&artifact, &ctx).expect("ingest");
assert!(out.working_path.is_none());
assert!(out.content_hash.is_none());
assert!(out.facts.iter().any(|f| f.key == "live_url"));
}
#[test]
fn ingest_all_collects_working_paths() {
let scratch = Scratch::new();
let store = scratch.0.join("store");
let fw = scratch.0.join("fw.bin");
std::fs::write(&fw, b"abc").expect("write");
let ctx = ctx_for(&store, "t1");
let mut target = OnboardedTarget::new("t".to_string(), TargetType::FirmwareBareMetal);
target
.artifacts
.push(Artifact::firmware_image(fw.to_string_lossy().to_string()));
target.artifacts.push(Artifact::live_url("https://x"));
let set = ingest_all(&target, &ctx).expect("ingest all");
assert_eq!(set.by_artifact.len(), 2);
// Only the firmware artifact yields a working path.
assert_eq!(set.working_paths().len(), 1);
}
#[test]
fn credentials_prefer_artifact_auth() {
let mut artifact = Artifact::git_repo("https://git/x", "main");
artifact.auth = Some(ArtifactAuth {
method: "token".to_string(),
username: Some("bob".to_string()),
secret: Some("pat".to_string()),
..Default::default()
});
let creds = credentials_for(&artifact, "/default/ssh/key");
assert_eq!(creds.auth_token.as_deref(), Some("pat"));
assert_eq!(creds.auth_username.as_deref(), Some("bob"));
}
#[test]
fn credentials_fall_back_to_default_ssh_key() {
let artifact = Artifact::git_repo("git@host:x.git", "main");
let creds = credentials_for(&artifact, "/default/ssh/key");
assert_eq!(creds.ssh_key_path.as_deref(), Some("/default/ssh/key"));
assert!(creds.auth_token.is_none());
}
}
+19
View File
@@ -0,0 +1,19 @@
// Library entrypoint — re-exports for integration tests and the binary.
pub mod agent;
pub mod api;
pub mod classify;
pub mod config;
pub mod database;
pub mod error;
pub mod ingest;
pub mod llm;
pub mod migrate;
pub mod pentest;
pub mod pipeline;
pub mod rag;
pub mod scheduler;
pub mod ssh;
#[allow(dead_code)]
pub mod trackers;
pub mod webhooks;
+6 -1
View File
@@ -19,12 +19,17 @@ impl LlmClient {
model: String,
embed_model: String,
) -> Self {
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(300))
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self {
base_url,
api_key,
model,
embed_model,
http: reqwest::Client::new(),
http,
}
}
+12 -7
View File
@@ -5,15 +5,20 @@ use compliance_core::models::Finding;
use crate::error::AgentError;
use crate::llm::LlmClient;
const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing issue descriptions for a bug tracker. Generate a clear, actionable issue body in Markdown format that includes:
const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing a bug tracker issue for a developer to fix. Be direct and actionable — developers skim issue descriptions, so lead with what matters.
1. **Summary**: 1-2 sentence overview
2. **Evidence**: Code location, snippet, and what was detected
3. **Impact**: What could happen if not fixed
4. **Remediation**: Step-by-step fix instructions
5. **References**: Relevant CWE/CVE links if applicable
Format in Markdown:
Keep it concise and professional. Use code blocks for code snippets."#;
1. **What**: 1 sentence what's wrong and where (file:line)
2. **Why it matters**: 1-2 sentences concrete impact if not fixed. Avoid generic "could lead to" phrasing; describe the specific attack or failure scenario.
3. **Fix**: The specific code change needed. Use a code block with the corrected code if possible. If the fix is configuration-based, show the exact config change.
4. **References**: CWE/CVE link if applicable (one line, not a section)
Rules:
- No filler paragraphs or background explanations
- No restating the finding title in the body
- Code blocks should show the FIX, not the vulnerable code (the developer can see that in the diff)
- If the remediation is a one-liner, just say it don't wrap it in a section header"#;
pub async fn generate_issue_description(
llm: &Arc<LlmClient>,
+18 -1
View File
@@ -5,7 +5,24 @@ use compliance_core::models::Finding;
use crate::error::AgentError;
use crate::llm::LlmClient;
const FIX_SYSTEM_PROMPT: &str = r#"You are a security engineer. Given a security finding with code context, suggest a concrete code fix. Return ONLY the fixed code snippet that can directly replace the vulnerable code. Include brief inline comments explaining the fix."#;
const FIX_SYSTEM_PROMPT: &str = r#"You are a security engineer suggesting a code fix. Return ONLY the corrected code that replaces the vulnerable snippet — no explanations, no markdown fences, no before/after comparison.
Rules:
- The fix must be a drop-in replacement for the vulnerable code
- Preserve the original code's style, indentation, and naming conventions
- Add at most one brief inline comment on the changed line explaining the security fix
- If the fix requires importing a new module, include the import on a separate line prefixed with the language's comment syntax + "Add import: "
- Do not refactor, rename variables, or "improve" unrelated code
- If the vulnerability is a false positive and the code is actually safe, return the original code unchanged with a comment explaining why no fix is needed
Language-specific fix guidance:
- Rust: use `?` for error propagation, prefer `SecretString` for secrets, use parameterized queries with `sqlx`/`diesel`
- Python: use parameterized queries (never f-strings in SQL), use `secrets` module not `random`, use `subprocess.run([...])` list form, use `markupsafe.escape()` for HTML
- Go: use `sql.Query` with `$1`/`?` placeholders, use `crypto/rand` not `math/rand`, use `html/template` not `text/template`, return errors don't panic
- Java/Kotlin: use `PreparedStatement` with `?` params, use `SecureRandom`, use `Jsoup.clean()` for HTML sanitization, use `@Valid` for input validation
- Ruby: use ActiveRecord parameterized finders, use `SecureRandom`, use `ERB::Util.html_escape`, use `strong_parameters`
- PHP: use PDO prepared statements with `:param` or `?`, use `random_bytes()`/`random_int()`, use `htmlspecialchars()` with `ENT_QUOTES`, use `password_hash(PASSWORD_BCRYPT)`
- C/C++: use `snprintf` not `sprintf`, use bounds-checked APIs, free resources in reverse allocation order, use `memset_s` for secret cleanup"#;
pub async fn suggest_fix(llm: &Arc<LlmClient>, finding: &Finding) -> Result<String, AgentError> {
let user_prompt = format!(
+108 -39
View File
@@ -1,69 +1,138 @@
// System prompts for multi-pass LLM code review.
// Each pass focuses on a different aspect to avoid overloading a single prompt.
pub const LOGIC_REVIEW_PROMPT: &str = r#"You are a senior software engineer reviewing code changes. Focus ONLY on logic and correctness issues.
pub const LOGIC_REVIEW_PROMPT: &str = r#"You are a senior software engineer reviewing a code diff. Report ONLY genuine logic bugs that would cause incorrect behavior at runtime.
Look for:
- Off-by-one errors, wrong comparisons, missing edge cases
- Incorrect control flow (unreachable code, missing returns, wrong loop conditions)
- Race conditions or concurrency bugs
- Resource leaks (unclosed handles, missing cleanup)
- Wrong variable used (copy-paste errors)
- Incorrect error handling (swallowed errors, wrong error type)
Report:
- Off-by-one errors, wrong comparisons, missing edge cases that cause wrong results
- Incorrect control flow that produces wrong output (not style preferences)
- Actual race conditions with concrete shared-state mutation (not theoretical ones)
- Resource leaks where cleanup is truly missing (not just "could be improved")
- Wrong variable used (copy-paste errors) must be provably wrong, not just suspicious
- Swallowed errors that silently hide failures in a way that matters
Ignore: style, naming, formatting, documentation, minor improvements.
Do NOT report:
- Style, naming, formatting, documentation, or code organization preferences
- Theoretical issues without a concrete triggering scenario
- "Potential" problems that require assumptions not supported by the visible code
- Complexity or function length that's a separate review pass
For each issue found, respond with a JSON array:
Language-idiomatic patterns that are NOT bugs (do not flag these):
- Rust: `||`/`&&` short-circuit evaluation, variable shadowing, `let` rebinding, `clone()`, `impl` blocks, `match` arms with guards, `?` operator chaining, `unsafe` blocks with safety comments
- Python: duck typing, EAFP pattern (try/except vs check-first), `*args`/`**kwargs`, walrus operator `:=`, truthiness checks on containers, bare `except:` in top-level handlers
- Go: multiple return values for errors, `if err != nil` patterns, goroutine + channel patterns, blank identifier `_`, named returns, `defer` for cleanup, `init()` functions
- Java/Kotlin: checked exception patterns, method overloading, `Optional` vs null checks, Kotlin `?.` safe calls, `!!` non-null assertions in tests, `when` exhaustive matching, companion objects, `lateinit`
- Ruby: monkey patching in libraries, method_missing, blocks/procs/lambdas, `rescue => e` patterns, `send`/`respond_to?` metaprogramming, `nil` checks via `&.` safe navigation
- PHP: loose comparisons with `==` (only flag if `===` was clearly intended), `@` error suppression in legacy code, `isset()`/`empty()` patterns, magic methods (`__get`, `__call`), array functions as callbacks
- C/C++: RAII patterns, move semantics, `const_cast`/`static_cast` in appropriate contexts, macro usage for platform compat, pointer arithmetic in low-level code, `goto` for cleanup in C
Severity guide:
- high: Will cause incorrect behavior in normal usage
- medium: Will cause incorrect behavior in edge cases
- low: Minor correctness concern with limited blast radius
Prefer returning [] over reporting low-confidence guesses. A false positive wastes more developer time than a missed low-severity issue.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "high|medium|low", "file": "...", "line": N, "suggestion": "..."}]
If no issues found, respond with: []"#;
pub const SECURITY_REVIEW_PROMPT: &str = r#"You are a security engineer reviewing code changes. Focus ONLY on security vulnerabilities.
pub const SECURITY_REVIEW_PROMPT: &str = r#"You are a security engineer reviewing a code diff. Report ONLY exploitable security vulnerabilities with a realistic attack scenario.
Look for:
- Injection vulnerabilities (SQL, command, XSS, template injection)
- Authentication/authorization bypasses
- Sensitive data exposure (logging secrets, hardcoded credentials)
- Insecure cryptography (weak algorithms, predictable randomness)
- Path traversal, SSRF, open redirects
- Unsafe deserialization
- Missing input validation at trust boundaries
Report:
- Injection vulnerabilities (SQL, command, XSS, template) where untrusted input reaches a sink
- Authentication/authorization bypasses with a concrete exploit path
- Sensitive data exposure: secrets in code, credentials in logs, PII leaks
- Insecure cryptography: weak algorithms, predictable randomness, hardcoded keys
- Path traversal, SSRF, open redirects only where user input reaches the vulnerable API
- Unsafe deserialization of untrusted data
- Missing input validation at EXTERNAL trust boundaries (user input, API responses)
Ignore: code style, performance, general quality.
Do NOT report:
- Internal code that only handles trusted/validated data
- Hash functions used for non-security purposes (dedup fingerprints, cache keys, content addressing)
- Logging of non-sensitive operational data (finding titles, counts, performance metrics)
- "Information disclosure" for data that is already public or user-facing
- Code style, performance, or general quality issues
- Missing validation on internal function parameters (trust the caller within the same module/crate/package)
- Theoretical attacks that require preconditions not present in the code
For each issue found, respond with a JSON array:
Language-specific patterns that are NOT vulnerabilities (do not flag these):
- Python: `pickle` used on trusted internal data, `eval()`/`exec()` on hardcoded strings, `subprocess` with hardcoded commands, Django `mark_safe()` on static content, `assert` in non-security contexts
- Go: `crypto/rand` is secure (don't confuse with `math/rand`), `sql.DB` with parameterized queries is safe, `http.ListenAndServe` without TLS in dev/internal, error strings in responses (Go convention)
- Java/Kotlin: Spring Security annotations are sufficient auth checks, `@Transactional` provides atomicity, JPA parameterized queries are safe, Kotlin `require()`/`check()` are assertion patterns not vulnerabilities
- Ruby: Rails `params.permit()` is input validation, `render html:` with `html_safe` on generated content, ActiveRecord parameterized finders are safe, Devise/Warden patterns for auth
- PHP: PDO prepared statements are safe, Laravel Eloquent is parameterized, `htmlspecialchars()` is XSS mitigation, Symfony security voters are auth checks, `password_hash()`/`password_verify()` are correct bcrypt usage
- C/C++: `strncpy`/`snprintf` are bounds-checked (vs `strcpy`/`sprintf`), smart pointers manage memory, RAII handles cleanup, `static_assert` is compile-time only, OpenSSL with proper context setup
- Rust: `sha2`/`blake3` for fingerprinting is not "weak crypto", `unsafe` with documented invariants, `secrecy::SecretString` properly handles secrets
Severity guide:
- critical: Remote code execution, auth bypass, or data breach with no preconditions
- high: Exploitable vulnerability requiring minimal preconditions
- medium: Vulnerability requiring specific conditions or limited impact
Prefer returning [] over reporting speculative vulnerabilities. Every false positive erodes trust in the scanner.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "critical|high|medium", "file": "...", "line": N, "cwe": "CWE-XXX", "suggestion": "..."}]
If no issues found, respond with: []"#;
pub const CONVENTION_REVIEW_PROMPT: &str = r#"You are a code reviewer checking adherence to project conventions. Focus ONLY on patterns that indicate likely bugs or maintenance problems.
pub const CONVENTION_REVIEW_PROMPT: &str = r#"You are a code reviewer checking for convention violations that indicate likely bugs. Report ONLY deviations from the project's visible patterns that could cause real problems.
Look for:
- Inconsistent error handling patterns within the same module
- Public API that doesn't follow the project's established patterns
- Missing or incorrect type annotations that could cause runtime issues
- Anti-patterns specific to the language (e.g. unwrap in Rust library code, any in TypeScript)
Report:
- Inconsistent error handling within the same module where the inconsistency could hide failures
- Public API that breaks the module's established contract (not just different style)
- Anti-patterns that are bugs in this language: e.g. `unwrap()` in Rust library code where the CI enforces `clippy::unwrap_used`, `any` defeating TypeScript's type system
Do NOT report: minor style preferences, documentation gaps, formatting.
Only report issues with HIGH confidence that they deviate from the visible codebase conventions.
Do NOT report:
- Style preferences, formatting, naming conventions, or documentation
- Code organization suggestions ("this function should be split")
- Patterns that are valid in the language even if you'd write them differently
- "Missing type annotations" unless the code literally won't compile or causes a type inference bug
For each issue found, respond with a JSON array:
Language-specific patterns that are conventional (do not flag these):
- Rust: variable shadowing, `||`/`&&` short-circuit, `let` rebinding, builder patterns, `clone()`, `From`/`Into` impl chains, `#[allow(...)]` attributes
- Python: `**kwargs` forwarding, `@property` setters, `__dunder__` methods, list comprehensions with conditions, `if TYPE_CHECKING` imports, `noqa` comments
- Go: stuttering names (`http.HTTPClient`) discouraged but not a bug, `context.Context` as first param, init() functions, `//nolint` directives, returning concrete types vs interfaces in internal code
- Java/Kotlin: builder pattern boilerplate, Lombok annotations (`@Data`, `@Builder`), Kotlin data classes, `companion object` factories, `@Suppress` annotations, checked exception wrapping
- Ruby: `attr_accessor` usage, `Enumerable` mixin patterns, `module_function`, `class << self` syntax, DSL blocks (Rake, RSpec, Sinatra routes)
- PHP: `__construct` with property promotion, Laravel facades, static factory methods, nullable types with `?`, attribute syntax `#[...]`
- C/C++: header guards vs `#pragma once`, forward declarations, `const` correctness patterns, template specialization, `auto` type deduction
Severity guide:
- medium: Convention violation that will likely cause a bug or maintenance problem
- low: Convention violation that is a minor concern
Return at most 3 findings. Prefer [] over marginal findings.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "medium|low", "file": "...", "line": N, "suggestion": "..."}]
If no issues found, respond with: []"#;
pub const COMPLEXITY_REVIEW_PROMPT: &str = r#"You are reviewing code changes for excessive complexity that could lead to bugs.
pub const COMPLEXITY_REVIEW_PROMPT: &str = r#"You are reviewing code changes for complexity that is likely to cause bugs. Report ONLY complexity that makes the code demonstrably harder to reason about.
Look for:
- Functions over 50 lines that should be decomposed
- Deeply nested control flow (4+ levels)
- Complex boolean expressions that are hard to reason about
- Functions with 5+ parameters
- Code duplication within the changed files
Report:
- Functions over 80 lines with multiple interleaved responsibilities (not just long)
- Deeply nested control flow (5+ levels) where flattening would prevent bugs
- Complex boolean expressions that a reader would likely misinterpret
Only report complexity issues that are HIGH risk for future bugs. Ignore acceptable complexity in configuration, CLI argument parsing, or generated code.
Do NOT report:
- Functions that are long but linear and easy to follow
- Acceptable complexity: configuration setup, CLI parsing, test helpers, builder patterns
- Code that is complex because the problem is complex only report if restructuring would reduce bug risk
- "This function does multiple things" unless you can identify a specific bug risk from the coupling
- Suggestions that would just move complexity elsewhere without reducing it
For each issue found, respond with a JSON array:
Severity guide:
- medium: Complexity that has a concrete risk of causing bugs during future changes
- low: Complexity that makes review harder but is unlikely to cause bugs
Return at most 2 findings. Prefer [] over reporting complexity that is justified.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "medium|low", "file": "...", "line": N, "suggestion": "..."}]
If no issues found, respond with: []"#;
+36 -12
View File
@@ -8,22 +8,46 @@ use crate::pipeline::orchestrator::GraphContext;
/// Maximum number of findings to include in a single LLM triage call.
const TRIAGE_CHUNK_SIZE: usize = 30;
const TRIAGE_SYSTEM_PROMPT: &str = r#"You are a security finding triage expert. Analyze each of the following security findings with its code context and determine the appropriate action.
const TRIAGE_SYSTEM_PROMPT: &str = r#"You are a pragmatic security triage expert. Your job is to filter out noise and keep only findings that a developer should actually fix. Be aggressive about dismissing false positives — a clean, high-signal list is more valuable than a comprehensive one.
Actions:
- "confirm": The finding is a true positive at the reported severity. Keep as-is.
- "downgrade": The finding is real but over-reported. Lower severity recommended.
- "upgrade": The finding is under-reported. Higher severity recommended.
- "dismiss": The finding is a false positive. Should be removed.
- "confirm": True positive with real impact. Keep severity as-is.
- "downgrade": Real issue but over-reported severity. Lower it.
- "upgrade": Under-reported higher severity warranted.
- "dismiss": False positive, not exploitable, or not actionable. Remove it.
Consider:
- Is the code in a test, example, or generated file? (lower confidence for test code)
- Does the surrounding code context confirm or refute the finding?
- Is the finding actionable by a developer?
- Would a real attacker be able to exploit this?
Dismiss when:
- The scanner flagged a language idiom as a bug (see examples below)
- The finding is in test/example/generated/vendored code
- The "vulnerability" requires preconditions that don't exist in the code
- The finding is about code style, complexity, or theoretical concerns rather than actual bugs
- A hash function is used for non-security purposes (dedup, caching, content addressing)
- Internal logging of non-sensitive operational data is flagged as "information disclosure"
- The finding duplicates another finding already in the list
- Framework-provided security is already in place (e.g. ORM parameterized queries, CSRF middleware, auth decorators)
Respond with a JSON array, one entry per finding in the same order they were presented:
[{"id": "<fingerprint>", "action": "confirm|downgrade|upgrade|dismiss", "confidence": 0-10, "rationale": "brief explanation", "remediation": "optional fix suggestion"}, ...]"#;
Common false positive patterns by language (dismiss these):
- Rust: short-circuit `||`/`&&`, variable shadowing, `clone()`, `unsafe` with safety docs, `sha2` for fingerprinting
- Python: EAFP try/except, `subprocess` with hardcoded args, `pickle` on trusted data, Django `mark_safe` on static content
- Go: `if err != nil` is not "swallowed error", `crypto/rand` is secure, returning errors is not "information disclosure"
- Java/Kotlin: Spring Security annotations are valid auth, JPA parameterized queries are safe, Kotlin `!!` in tests is fine
- Ruby: Rails `params.permit` is validation, ActiveRecord finders are parameterized, `html_safe` on generated content
- PHP: PDO prepared statements are safe, Laravel Eloquent is parameterized, `htmlspecialchars` is XSS mitigation
- C/C++: `strncpy`/`snprintf` are bounds-checked, smart pointers manage memory, RAII handles cleanup
Confirm only when:
- You can describe a concrete scenario where the bug manifests or the vulnerability is exploitable
- The fix is actionable (developer can change specific code to resolve it)
- The finding is in production code that handles external input or sensitive data
Confidence scoring (0-10):
- 8-10: Certain true positive with clear exploit/bug scenario
- 5-7: Likely true positive, some assumptions required
- 3-4: Uncertain, needs manual review
- 0-2: Almost certainly a false positive
Respond with a JSON array, one entry per finding in the same order presented (no markdown fences):
[{"id": "<fingerprint>", "action": "confirm|downgrade|upgrade|dismiss", "confidence": 0-10, "rationale": "1-2 sentences", "remediation": "optional fix"}, ...]"#;
pub async fn triage_findings(
llm: &Arc<LlmClient>,
+61 -18
View File
@@ -1,23 +1,56 @@
mod agent;
mod api;
pub(crate) mod config;
mod database;
mod error;
mod llm;
mod pentest;
mod pipeline;
mod rag;
mod scheduler;
mod ssh;
#[allow(dead_code)]
mod trackers;
mod webhooks;
use compliance_agent::{agent, api, config, database, migrate, scheduler, ssh, webhooks};
/// Run the `migrate onboarding` subcommand and exit. Backfills (or reverts) the
/// unified `onboarded_targets` collection per tenant.
///
/// Usage: `compliance-agent migrate onboarding [--all | --tenant <id>] [--dry-run] [--revert]`
async fn run_migration(
args: &[String],
pool: &database::DatabasePool,
) -> Result<(), compliance_agent::error::AgentError> {
if args.get(2).map(String::as_str) != Some("onboarding") {
eprintln!(
"usage: compliance-agent migrate onboarding [--all | --tenant <id>] [--dry-run] [--revert]"
);
std::process::exit(2);
}
let has = |flag: &str| args.iter().any(|a| a == flag);
let dry_run = has("--dry-run");
let revert = has("--revert");
let tenant = args
.iter()
.position(|a| a == "--tenant")
.and_then(|i| args.get(i + 1))
.cloned();
let tenants: Vec<String> = if has("--all") {
pool.list_tenant_ids().await?
} else if let Some(t) = tenant {
vec![t]
} else {
eprintln!("specify --all or --tenant <id>");
std::process::exit(2);
};
for tenant_id in tenants {
let db = pool.for_tenant_id(&tenant_id).await?;
if revert {
migrate::onboarding::revert(&db).await?;
println!("[{tenant_id}] reverted onboarding backfill");
} else {
let report = migrate::onboarding::backfill_onboarded_targets(&db, dry_run).await?;
let prefix = if dry_run { "(dry-run) " } else { "" };
println!("[{tenant_id}] {prefix}{report:?}");
}
}
Ok(())
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
match dotenvy::dotenv() {
Ok(path) => eprintln!("[dotenv] Loaded from: {}", path.display()),
Err(e) => eprintln!("[dotenv] FAILED: {e}"),
Err(_) => eprintln!("[dotenv] No .env file found, using environment variables"),
}
let _telemetry_guard = compliance_core::telemetry::init_telemetry("compliance-agent");
@@ -38,10 +71,20 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
}
tracing::info!("Connecting to MongoDB...");
let db = database::Database::connect(&config.mongodb_uri, &config.mongodb_database).await?;
db.ensure_indexes().await?;
// Per-tenant pool only — the agent has no shared "default" database
// after M7.2-D. `mongodb_database` is now the db-name prefix used
// for tenant databases (`<prefix>_<tenant_id>`).
let db_pool =
database::DatabasePool::connect(&config.mongodb_uri, &config.mongodb_database).await?;
let agent = agent::ComplianceAgent::new(config.clone(), db.clone());
// One-shot subcommands run and exit without starting the servers.
let args: Vec<String> = std::env::args().collect();
if args.get(1).map(String::as_str) == Some("migrate") {
run_migration(&args, &db_pool).await?;
return Ok(());
}
let agent = agent::ComplianceAgent::new(config.clone(), db_pool);
tracing::info!("Starting scheduler...");
let scheduler_agent = agent.clone();
+8
View File
@@ -0,0 +1,8 @@
//! One-time data migrations.
//!
//! Currently just the onboarding backfill ([`onboarding`]), which folds the
//! legacy `repositories` and `dast_targets` collections into the unified
//! `onboarded_targets` collection, preserving `_id` so every downstream record
//! keyed by `repo_id` / `target_id` keeps resolving.
pub mod onboarding;
+406
View File
@@ -0,0 +1,406 @@
//! Backfill: legacy `repositories` + `dast_targets` → `onboarded_targets`.
//!
//! The transforms here are **id-preserving**: an [`OnboardedTarget`] keeps the
//! same `_id` as the `TrackedRepository` / `DastTarget` it came from, so every
//! downstream collection keyed by that hex id (findings, sbom, scan_runs,
//! graph, dast_*, pentest_*) keeps resolving with zero row rewrites, and
//! existing webhook URLs keep working. The mapping functions are pure and unit
//! tested; the DB orchestration (idempotent per-tenant backfill + revert) is a
//! thin driver over them.
use compliance_core::models::{
Artifact, ArtifactKind, DastTarget, DastTargetType, GitArtifactConfig, IssueTrackerConfig,
OnboardedTarget, TargetType, TrackedRepository, WebArtifactConfig,
};
use futures_util::TryStreamExt;
use mongodb::bson::{doc, Document};
use crate::database::Database;
use crate::error::AgentError;
/// Marker id in `schema_migrations` recording that the backfill has run.
const MIGRATION_MARKER: &str = "onboarding_backfill_v1";
/// Summary of a backfill run.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct MigrationReport {
/// Repositories turned into onboarded targets.
pub repos_migrated: u64,
/// DAST targets folded into an existing (repo-linked) target as a LiveUrl.
pub dast_targets_folded: u64,
/// DAST targets with no repo link, migrated as standalone targets.
pub dast_targets_standalone: u64,
/// Records skipped because a target with that `_id` already existed.
pub skipped_existing: u64,
}
/// Map a legacy `DastTargetType` to a unified [`TargetType`]. REST/GraphQL APIs
/// are backend services; a browser app is a web app.
fn target_type_for_dast(kind: &DastTargetType) -> TargetType {
match kind {
DastTargetType::WebApp => TargetType::WebApp,
DastTargetType::RestApi | DastTargetType::GraphQl => TargetType::BackendService,
}
}
/// Build the LiveUrl artifact for a DAST target (its base URL + crawl config +
/// auth). Shared by fold-in and standalone migration.
pub fn dast_to_artifact(dast: &DastTarget) -> Artifact {
let mut artifact = Artifact::live_url(dast.base_url.clone());
artifact.web = Some(WebArtifactConfig {
target_kind: dast.target_type.clone(),
excluded_paths: dast.excluded_paths.clone(),
max_crawl_depth: dast.max_crawl_depth,
rate_limit: dast.rate_limit,
allow_destructive: dast.allow_destructive,
});
artifact.auth = dast.auth_config.clone().map(Into::into);
artifact
}
/// Map a `TrackedRepository` to an onboarded target, preserving `_id`. The git
/// remote becomes a `GitRepo` artifact carrying the repo's branch, watermark,
/// and auth; tracker config folds into `scan_config`.
///
/// `target_type` is a safe default (`BackendService`) — the classifier can
/// refine it later; `classification` is left `None` (unconfirmed).
pub fn repo_to_target(repo: &TrackedRepository) -> OnboardedTarget {
let mut target = OnboardedTarget::new(repo.name.clone(), TargetType::BackendService);
target.id = repo.id;
let mut artifact = Artifact::git_repo(repo.git_url.clone(), repo.default_branch.clone());
artifact.git = Some(GitArtifactConfig {
default_branch: repo.default_branch.clone(),
last_scanned_commit: repo.last_scanned_commit.clone(),
local_path: repo.local_path.clone(),
});
if repo.auth_token.is_some() || repo.auth_username.is_some() {
artifact.auth = Some(compliance_core::models::ArtifactAuth {
method: "token".to_string(),
username: repo.auth_username.clone(),
secret: repo.auth_token.clone(),
..Default::default()
});
}
target.artifacts.push(artifact);
if repo.tracker_type.is_some() {
target.scan_config.issue_tracker = Some(IssueTrackerConfig {
tracker_type: repo.tracker_type.clone(),
owner: repo.tracker_owner.clone(),
repo: repo.tracker_repo.clone(),
token: repo.tracker_token.clone(),
});
}
target.scan_schedule = repo.scan_schedule.clone();
target.webhook_enabled = repo.webhook_enabled;
target.webhook_secret = repo.webhook_secret.clone();
target.findings_count = repo.findings_count;
target.created_at = repo.created_at;
target.updated_at = repo.updated_at;
target
}
/// Append a DAST target's LiveUrl artifact onto an existing (repo-derived)
/// target. If the repo default was `BackendService` but the DAST target is a
/// browser web app, promote the type to `WebApp`.
pub fn fold_dast_into_target(target: &mut OnboardedTarget, dast: &DastTarget) {
if matches!(dast.target_type, DastTargetType::WebApp)
&& target.target_type == TargetType::BackendService
{
target.target_type = TargetType::WebApp;
}
if !target.has(ArtifactKind::LiveUrl) {
target.artifacts.push(dast_to_artifact(dast));
}
}
/// Map a repo-less DAST target to a standalone onboarded target, preserving `_id`.
pub fn dast_to_standalone_target(dast: &DastTarget) -> OnboardedTarget {
let mut target =
OnboardedTarget::new(dast.name.clone(), target_type_for_dast(&dast.target_type));
target.id = dast.id;
target.artifacts.push(dast_to_artifact(dast));
target.created_at = dast.created_at;
target.updated_at = dast.updated_at;
target
}
/// Whether the onboarding backfill has already been applied to this database.
pub async fn already_applied(db: &Database) -> Result<bool, AgentError> {
let found = db
.collection_named::<Document>("schema_migrations")
.find_one(doc! { "_id": MIGRATION_MARKER })
.await?;
Ok(found.is_some())
}
/// Backfill `onboarded_targets` from `repositories` + `dast_targets` for one
/// tenant database.
///
/// Id-preserving and **idempotent**: targets that already exist (by `_id`) are
/// skipped, so re-running is safe. With `dry_run`, computes the report without
/// writing. The legacy collections are never deleted; the only mutation outside
/// `onboarded_targets` is the history relink of folded DAST targets, which is
/// logged so [`revert`] can undo it.
pub async fn backfill_onboarded_targets(
db: &Database,
dry_run: bool,
) -> Result<MigrationReport, AgentError> {
let mut report = MigrationReport::default();
// 1. repositories -> onboarded_targets (preserve _id, skip existing).
let mut repos = db.repositories().find(doc! {}).await?;
while let Some(repo) = repos.try_next().await? {
let Some(id) = repo.id else { continue };
if db
.onboarded_targets()
.find_one(doc! { "_id": id })
.await?
.is_some()
{
report.skipped_existing += 1;
continue;
}
if !dry_run {
db.onboarded_targets()
.insert_one(repo_to_target(&repo))
.await?;
}
report.repos_migrated += 1;
}
// 2. dast_targets -> fold into the linked repo target, or migrate standalone.
let mut dasts = db.dast_targets().find(doc! {}).await?;
while let Some(dast) = dasts.try_next().await? {
let Some(dast_id) = dast.id else { continue };
let repo_oid = dast
.repo_id
.as_deref()
.and_then(|r| mongodb::bson::oid::ObjectId::parse_str(r).ok());
let linked = match repo_oid {
Some(oid) => db.onboarded_targets().find_one(doc! { "_id": oid }).await?,
None => None,
};
match (linked, repo_oid) {
// Fold into an existing repo-derived target.
(Some(mut target), Some(oid)) => {
if target.has(ArtifactKind::LiveUrl) {
report.skipped_existing += 1; // already folded on a prior run
continue;
}
fold_dast_into_target(&mut target, &dast);
if !dry_run {
db.onboarded_targets()
.replace_one(doc! { "_id": oid }, &target)
.await?;
relink_history(db, &dast_id.to_hex(), &oid.to_hex()).await?;
}
report.dast_targets_folded += 1;
}
// No linked repo target: migrate as a standalone target (keeps _id).
_ => {
if db
.onboarded_targets()
.find_one(doc! { "_id": dast_id })
.await?
.is_some()
{
report.skipped_existing += 1;
continue;
}
if !dry_run {
db.onboarded_targets()
.insert_one(dast_to_standalone_target(&dast))
.await?;
}
report.dast_targets_standalone += 1;
}
}
}
if !dry_run {
db.collection_named::<Document>("schema_migrations")
.update_one(
doc! { "_id": MIGRATION_MARKER },
doc! { "$set": { "applied_at": mongodb::bson::DateTime::now() } },
)
.upsert(true)
.await?;
}
Ok(report)
}
/// Relink DAST scan runs and pentest sessions from the old DAST target id to the
/// unified target id, logging each move so [`revert`] can undo it.
///
/// Note: if multiple DAST targets fold into the same repo target, revert
/// restores only the last-logged mapping — a rare edge. The source collections
/// (`repositories`, `dast_targets`) are never deleted, so no data is lost.
async fn relink_history(db: &Database, old_id: &str, new_id: &str) -> Result<(), AgentError> {
db.dast_scan_runs()
.update_many(
doc! { "target_id": old_id },
doc! { "$set": { "target_id": new_id } },
)
.await?;
db.pentest_sessions()
.update_many(
doc! { "target_id": old_id },
doc! { "$set": { "target_id": new_id } },
)
.await?;
db.collection_named::<Document>("onboarding_migration_log")
.insert_one(doc! { "old_target_id": old_id, "new_target_id": new_id })
.await?;
Ok(())
}
/// Undo the backfill: replay the relink log in reverse, drop `onboarded_targets`
/// and the log, and clear the marker. The legacy collections are untouched, so
/// this restores the pre-migration state.
pub async fn revert(db: &Database) -> Result<(), AgentError> {
let log = db.collection_named::<Document>("onboarding_migration_log");
let mut cursor = log.find(doc! {}).await?;
while let Some(entry) = cursor.try_next().await? {
if let (Ok(old), Ok(new)) = (
entry.get_str("old_target_id"),
entry.get_str("new_target_id"),
) {
db.dast_scan_runs()
.update_many(
doc! { "target_id": new },
doc! { "$set": { "target_id": old } },
)
.await?;
db.pentest_sessions()
.update_many(
doc! { "target_id": new },
doc! { "$set": { "target_id": old } },
)
.await?;
}
}
db.onboarded_targets().drop().await?;
log.drop().await?;
db.collection_named::<Document>("schema_migrations")
.delete_one(doc! { "_id": MIGRATION_MARKER })
.await?;
Ok(())
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{DastAuthConfig, TrackerType};
fn repo() -> TrackedRepository {
let mut r = TrackedRepository::new("acme".to_string(), "https://git/acme.git".to_string());
r.id = Some(mongodb::bson::oid::ObjectId::new());
r.default_branch = "develop".to_string();
r.last_scanned_commit = Some("abc123".to_string());
r.auth_token = Some("pat".to_string());
r.auth_username = Some("bob".to_string());
r.tracker_type = Some(TrackerType::Gitea);
r.tracker_owner = Some("acme".to_string());
r.findings_count = 7;
r
}
fn dast(repo_id: Option<String>, kind: DastTargetType) -> DastTarget {
let mut d = DastTarget::new(
"acme-web".to_string(),
"https://acme.example.com".to_string(),
kind,
);
d.id = Some(mongodb::bson::oid::ObjectId::new());
d.repo_id = repo_id;
d.max_crawl_depth = 5;
d.auth_config = Some(DastAuthConfig {
method: "bearer".to_string(),
login_url: None,
username: None,
password: None,
token: Some("tok".to_string()),
headers: None,
});
d
}
#[test]
fn repo_maps_preserving_id_and_git_artifact() {
let r = repo();
let t = repo_to_target(&r);
assert_eq!(t.id, r.id); // id preserved
assert_eq!(t.findings_count, 7);
assert_eq!(t.scan_schedule, r.scan_schedule);
let git = t.code_artifact().expect("git artifact");
assert_eq!(git.kind, ArtifactKind::GitRepo);
assert_eq!(git.source_ref, "https://git/acme.git");
let gc = git.git.as_ref().expect("git config");
assert_eq!(gc.default_branch, "develop");
assert_eq!(gc.last_scanned_commit.as_deref(), Some("abc123"));
let auth = git.auth.as_ref().expect("auth");
assert_eq!(auth.secret.as_deref(), Some("pat"));
assert_eq!(auth.username.as_deref(), Some("bob"));
assert_eq!(
t.scan_config
.issue_tracker
.as_ref()
.and_then(|it| it.tracker_type.clone()),
Some(TrackerType::Gitea)
);
}
#[test]
fn standalone_dast_maps_preserving_id_and_live_url() {
let d = dast(None, DastTargetType::WebApp);
let t = dast_to_standalone_target(&d);
assert_eq!(t.id, d.id);
assert_eq!(t.target_type, TargetType::WebApp);
let url = t.live_url().expect("live url");
assert_eq!(url.source_ref, "https://acme.example.com");
let web = url.web.as_ref().expect("web config");
assert_eq!(web.max_crawl_depth, 5);
assert_eq!(
url.auth.as_ref().and_then(|a| a.secret.clone()),
Some("tok".to_string())
);
}
#[test]
fn rest_api_dast_maps_to_backend_service() {
let d = dast(None, DastTargetType::RestApi);
assert_eq!(
dast_to_standalone_target(&d).target_type,
TargetType::BackendService
);
}
#[test]
fn fold_adds_live_url_and_promotes_webapp() {
let mut t = repo_to_target(&repo());
assert_eq!(t.target_type, TargetType::BackendService);
fold_dast_into_target(&mut t, &dast(Some("x".to_string()), DastTargetType::WebApp));
assert_eq!(t.target_type, TargetType::WebApp); // promoted
assert!(t.has(ArtifactKind::LiveUrl));
assert!(t.has(ArtifactKind::GitRepo));
}
#[test]
fn fold_is_idempotent_on_live_url() {
let mut t = repo_to_target(&repo());
let d = dast(Some("x".to_string()), DastTargetType::WebApp);
fold_dast_into_target(&mut t, &d);
fold_dast_into_target(&mut t, &d);
let live_urls = t
.artifacts
.iter()
.filter(|a| a.kind == ArtifactKind::LiveUrl)
.count();
assert_eq!(live_urls, 1);
}
}
+5
View File
@@ -328,6 +328,7 @@ mod tests {
scan_schedule: String::new(),
cve_monitor_schedule: String::new(),
git_clone_base_path: String::new(),
artifact_store_base_path: String::new(),
ssh_key_path: String::new(),
keycloak_url: None,
keycloak_realm: None,
@@ -336,8 +337,12 @@ mod tests {
pentest_verification_email: None,
pentest_imap_host: None,
pentest_imap_port: None,
pentest_imap_tls: true,
pentest_imap_username: None,
pentest_imap_password: None,
admin_api_token: None,
tenant_registry_url: None,
unified_pipeline: false,
}
}
+30 -1
View File
@@ -321,9 +321,38 @@ impl PentestOrchestrator {
total_findings += findings_count;
let mut finding_ids: Vec<String> = Vec::new();
for mut finding in result.findings {
// Dedup findings within this tool result before inserting
let deduped_findings =
crate::pipeline::dedup::dedup_dast_findings(
result.findings,
);
for mut finding in deduped_findings {
finding.scan_run_id = session_id.clone();
finding.session_id = Some(session_id.clone());
// Check for existing duplicate in this session
let fp = crate::pipeline::dedup::compute_dast_fingerprint(
&finding,
);
let existing = self
.db
.dast_findings()
.find_one(doc! {
"session_id": &session_id,
"title": &finding.title,
"endpoint": &finding.endpoint,
"method": &finding.method,
})
.await;
if matches!(existing, Ok(Some(_))) {
tracing::debug!(
"Skipping duplicate DAST finding: {} (fp={:.12})",
finding.title,
fp,
);
continue;
}
let insert_result =
self.db.dast_findings().insert_one(&finding).await;
if let Ok(res) = &insert_result {
@@ -314,6 +314,21 @@ impl PentestOrchestrator {
- For SPA apps: a 200 HTTP status does NOT mean the page is accessible check the actual
page content with the browser tool to verify if it shows real data or a login redirect.
## Finding Quality Rules
- **Do not report the same issue twice.** If multiple tools detect the same missing header or
vulnerability on the same endpoint, report it ONCE with the most specific tool's output.
For example, if the recon tool and the header scanner both find missing HSTS, report it only
from the header scanner (more specific).
- **Group related findings.** Missing security headers on the same endpoint are ONE finding
("Missing security headers") listing all missing headers, not separate findings per header.
- **Severity must match real impact:**
- critical/high: Exploitable vulnerability (you can demonstrate the exploit)
- medium: Real misconfiguration with security implications but not directly exploitable
- low: Best-practice recommendation, defense-in-depth, or informational
- **Missing headers are medium at most** unless you can demonstrate a concrete exploit enabled
by the missing header (e.g., missing CSP + confirmed XSS = high for CSP finding).
- Console.log in third-party/vendored JS (node_modules, minified libraries) is informational only.
## Important
- This is an authorized penetration test. All testing is permitted within the target scope.
- Respect the rate limit of {rate_limit} requests per second.
+51 -1
View File
@@ -66,8 +66,10 @@ impl CodeReviewScanner {
}
}
let deduped = dedup_cross_pass(all_findings);
ScanOutput {
findings: all_findings,
findings: deduped,
sbom_entries: Vec::new(),
}
}
@@ -184,3 +186,51 @@ struct ReviewIssue {
#[serde(default)]
suggestion: Option<String>,
}
/// Deduplicate findings across review passes.
///
/// Multiple passes often flag the same issue (e.g. SQL injection reported by
/// logic, security, and convention passes). We group by file + nearby line +
/// normalized title keywords and keep the highest-severity finding.
fn dedup_cross_pass(findings: Vec<Finding>) -> Vec<Finding> {
use std::collections::HashMap;
// Build a dedup key: (file, line bucket, normalized title words)
fn dedup_key(f: &Finding) -> String {
let file = f.file_path.as_deref().unwrap_or("");
// Group lines within 3 of each other
let line_bucket = f.line_number.unwrap_or(0) / 4;
// Normalize: lowercase, keep only alphanumeric, sort words for order-independence
let title_lower = f.title.to_lowercase();
let mut words: Vec<&str> = title_lower
.split(|c: char| !c.is_alphanumeric())
.filter(|w| w.len() > 2)
.collect();
words.sort();
format!("{file}:{line_bucket}:{}", words.join(","))
}
let mut groups: HashMap<String, Finding> = HashMap::new();
for finding in findings {
let key = dedup_key(&finding);
groups
.entry(key)
.and_modify(|existing| {
// Keep the higher severity; on tie, keep the one with more detail
if finding.severity > existing.severity
|| (finding.severity == existing.severity
&& finding.description.len() > existing.description.len())
{
*existing = finding.clone();
}
// Merge CWE if the existing one is missing it
if existing.cwe.is_none() {
existing.cwe = finding.cwe.clone();
}
})
.or_insert(finding);
}
groups.into_values().collect()
}
+357
View File
@@ -1,5 +1,7 @@
use sha2::{Digest, Sha256};
use compliance_core::models::dast::DastFinding;
pub fn compute_fingerprint(parts: &[&str]) -> String {
let mut hasher = Sha256::new();
for part in parts {
@@ -9,9 +11,209 @@ pub fn compute_fingerprint(parts: &[&str]) -> String {
hex::encode(hasher.finalize())
}
/// Compute a dedup fingerprint for a DAST finding.
///
/// The key is derived from the *canonicalized* title (lowercased, domain names
/// stripped, known synonyms resolved), endpoint, and HTTP method. This lets us
/// detect both exact duplicates (same tool reporting twice across passes) and
/// semantic duplicates (e.g., `security_header_missing` "Missing HSTS header"
/// vs `tls_misconfiguration` "Missing strict-transport-security header").
pub fn compute_dast_fingerprint(f: &DastFinding) -> String {
let canon = canonicalize_dast_title(&f.title);
let endpoint = f.endpoint.to_lowercase().trim_end_matches('/').to_string();
let method = f.method.to_uppercase();
let param = f.parameter.as_deref().unwrap_or("");
compute_fingerprint(&[&canon, &endpoint, &method, param])
}
/// Canonicalize a DAST finding title for dedup purposes.
///
/// 1. Lowercase
/// 2. Strip domain names / URLs (e.g. "for comp-dev.meghsakha.com")
/// 3. Resolve known header synonyms (hsts ↔ strict-transport-security, etc.)
/// 4. Strip extra whitespace
fn canonicalize_dast_title(title: &str) -> String {
let mut s = title.to_lowercase();
// Strip "for <domain>" or "on <domain>" suffixes
// Pattern: "for <word.word...>" or "on <method> <url>"
if let Some(idx) = s.find(" for ") {
// Check if what follows looks like a domain or URL
let rest = &s[idx + 5..];
if rest.contains('.') || rest.starts_with("http") {
s.truncate(idx);
}
}
if let Some(idx) = s.find(" on ") {
let rest = &s[idx + 4..];
if rest.contains("http") || rest.contains('/') {
s.truncate(idx);
}
}
// Resolve known header synonyms
let synonyms: &[(&str, &str)] = &[
("hsts", "strict-transport-security"),
("csp", "content-security-policy"),
("cors", "cross-origin-resource-sharing"),
("xfo", "x-frame-options"),
];
for &(short, canonical) in synonyms {
// Only replace whole words — check boundaries
if let Some(pos) = s.find(short) {
let before_ok = pos == 0 || !s.as_bytes()[pos - 1].is_ascii_alphanumeric();
let after_ok = pos + short.len() >= s.len()
|| !s.as_bytes()[pos + short.len()].is_ascii_alphanumeric();
if before_ok && after_ok {
s = format!("{}{}{}", &s[..pos], canonical, &s[pos + short.len()..]);
}
}
}
// Collapse whitespace
s.split_whitespace().collect::<Vec<_>>().join(" ")
}
/// Deduplicate a list of DAST findings, merging evidence from duplicates.
///
/// Two-phase approach:
/// 1. **Exact dedup** — group by canonicalized `(title, endpoint, method, parameter)`.
/// Merge evidence arrays, keep the highest severity, preserve exploitable flag.
/// 2. **CWE-based dedup** — within the same `(cwe, endpoint, method)` group, merge
/// findings whose canonicalized titles resolve to the same subject (e.g., HSTS
/// reported as both `security_header_missing` and `tls_misconfiguration`).
pub fn dedup_dast_findings(findings: Vec<DastFinding>) -> Vec<DastFinding> {
use std::collections::HashMap;
if findings.len() <= 1 {
return findings;
}
// Phase 1: exact fingerprint dedup
let mut seen: HashMap<String, usize> = HashMap::new();
let mut deduped: Vec<DastFinding> = Vec::new();
for finding in findings {
let fp = compute_dast_fingerprint(&finding);
if let Some(&idx) = seen.get(&fp) {
// Merge into existing
merge_dast_finding(&mut deduped[idx], &finding);
} else {
seen.insert(fp, deduped.len());
deduped.push(finding);
}
}
let before = deduped.len();
// Phase 2: CWE-based related dedup
// Group by (cwe, endpoint_normalized, method) — only when CWE is present
let mut cwe_groups: HashMap<String, Vec<usize>> = HashMap::new();
for (i, f) in deduped.iter().enumerate() {
if let Some(ref cwe) = f.cwe {
let key = format!(
"{}|{}|{}",
cwe,
f.endpoint.to_lowercase().trim_end_matches('/'),
f.method.to_uppercase(),
);
cwe_groups.entry(key).or_default().push(i);
}
}
// For each CWE group with multiple findings, keep the one with highest severity
// and most evidence, merge the rest into it
let mut merge_map: HashMap<usize, Vec<usize>> = HashMap::new();
let mut remove_indices: Vec<usize> = Vec::new();
for indices in cwe_groups.values() {
if indices.len() <= 1 {
continue;
}
// Find the "primary" finding: highest severity, then most evidence, then longest description
let Some(&primary_idx) = indices.iter().max_by(|&&a, &&b| {
deduped[a]
.severity
.cmp(&deduped[b].severity)
.then_with(|| deduped[a].evidence.len().cmp(&deduped[b].evidence.len()))
.then_with(|| {
deduped[a]
.description
.len()
.cmp(&deduped[b].description.len())
})
}) else {
continue;
};
for &idx in indices {
if idx != primary_idx {
remove_indices.push(idx);
merge_map.entry(primary_idx).or_default().push(idx);
}
}
}
if !remove_indices.is_empty() {
remove_indices.sort_unstable();
remove_indices.dedup();
// Merge evidence
for (&primary, secondaries) in &merge_map {
let extra_evidence: Vec<_> = secondaries
.iter()
.flat_map(|&i| deduped[i].evidence.clone())
.collect();
let any_exploitable = secondaries.iter().any(|&i| deduped[i].exploitable);
deduped[primary].evidence.extend(extra_evidence);
if any_exploitable {
deduped[primary].exploitable = true;
}
}
// Remove merged findings (iterate in reverse to preserve indices)
for &idx in remove_indices.iter().rev() {
deduped.remove(idx);
}
}
let after = deduped.len();
if before != after {
tracing::debug!(
"DAST CWE-based dedup: {before} → {after} findings ({} merged)",
before - after
);
}
deduped
}
/// Merge a duplicate DAST finding into a primary one.
fn merge_dast_finding(primary: &mut DastFinding, duplicate: &DastFinding) {
primary.evidence.extend(duplicate.evidence.clone());
if duplicate.severity > primary.severity {
primary.severity = duplicate.severity.clone();
}
if duplicate.exploitable {
primary.exploitable = true;
}
// Keep the longer/better description
if duplicate.description.len() > primary.description.len() {
primary.description.clone_from(&duplicate.description);
}
// Keep remediation if primary doesn't have one
if primary.remediation.is_none() && duplicate.remediation.is_some() {
primary.remediation.clone_from(&duplicate.remediation);
}
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::dast::DastVulnType;
use compliance_core::models::finding::Severity;
#[test]
fn fingerprint_is_deterministic() {
@@ -55,4 +257,159 @@ mod tests {
let b = compute_fingerprint(&["a", "bc"]);
assert_ne!(a, b);
}
fn make_dast(title: &str, endpoint: &str, vuln_type: DastVulnType) -> DastFinding {
let mut f = DastFinding::new(
"run1".into(),
"target1".into(),
vuln_type,
title.into(),
format!("Description for {title}"),
Severity::Medium,
endpoint.into(),
"GET".into(),
);
f.cwe = Some("CWE-319".into());
f
}
#[test]
fn canonicalize_strips_domain_suffix() {
let canon = canonicalize_dast_title("Missing HSTS header for comp-dev.meghsakha.com");
assert!(!canon.contains("meghsakha"), "domain should be stripped");
assert!(
canon.contains("strict-transport-security"),
"hsts should be resolved: {canon}"
);
}
#[test]
fn canonicalize_resolves_synonyms() {
let a = canonicalize_dast_title("Missing HSTS header");
let b = canonicalize_dast_title("Missing strict-transport-security header");
assert_eq!(a, b);
}
#[test]
fn exact_dedup_merges_identical_findings() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let f2 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 1, "exact duplicates should be merged");
}
#[test]
fn synonym_dedup_merges_hsts_variants() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let f2 = make_dast(
"Missing HSTS header for example.com",
"https://example.com",
DastVulnType::TlsMisconfiguration,
);
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(
result.len(),
1,
"HSTS synonym variants should merge to 1 finding"
);
}
#[test]
fn different_headers_not_merged() {
let mut f1 = make_dast(
"Missing x-content-type-options header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f1.cwe = Some("CWE-16".into());
let mut f2 = make_dast(
"Missing permissions-policy header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f2.cwe = Some("CWE-16".into());
// These share CWE-16 but are different headers — phase 2 will merge them
// since they share the same CWE+endpoint. This is acceptable because they
// have the same root cause (missing security headers configuration).
let result = dedup_dast_findings(vec![f1, f2]);
// CWE-based dedup will merge these into 1
assert!(
result.len() <= 2,
"same CWE+endpoint findings may be merged"
);
}
#[test]
fn different_endpoints_not_merged() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let f2 = make_dast(
"Missing strict-transport-security header",
"https://other.com",
DastVulnType::SecurityHeaderMissing,
);
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 2, "different endpoints should not merge");
}
#[test]
fn dedup_preserves_highest_severity() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let mut f2 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f2.severity = Severity::High;
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 1);
assert_eq!(result[0].severity, Severity::High);
}
#[test]
fn dedup_merges_evidence() {
let mut f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f1.evidence
.push(compliance_core::models::dast::DastEvidence {
request_method: "GET".into(),
request_url: "https://example.com".into(),
request_headers: None,
request_body: None,
response_status: 200,
response_headers: None,
response_snippet: Some("pass 1".into()),
screenshot_path: None,
payload: None,
response_time_ms: None,
});
let mut f2 = f1.clone();
f2.evidence[0].response_snippet = Some("pass 2".into());
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 1);
assert_eq!(result[0].evidence.len(), 2, "evidence should be merged");
}
}
+27 -20
View File
@@ -19,26 +19,33 @@ impl Scanner for GitleaksScanner {
#[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let output = tokio::process::Command::new("gitleaks")
.args([
"detect",
"--source",
".",
"--report-format",
"json",
"--report-path",
"/dev/stdout",
"--no-banner",
"--exit-code",
"0",
])
.current_dir(repo_path)
.output()
.await
.map_err(|e| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: Box::new(e),
})?;
let output = tokio::time::timeout(
std::time::Duration::from_secs(300),
tokio::process::Command::new("gitleaks")
.args([
"detect",
"--source",
".",
"--report-format",
"json",
"--report-path",
"/dev/stdout",
"--no-banner",
"--exit-code",
"0",
])
.current_dir(repo_path)
.output(),
)
.await
.map_err(|_| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: "timed out after 5 minutes".into(),
})?
.map_err(|e| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: Box::new(e),
})?;
if output.stdout.is_empty() {
return Ok(ScanOutput::default());
+1
View File
@@ -8,6 +8,7 @@ mod issue_creation;
pub mod lint;
pub mod orchestrator;
pub mod patterns;
pub mod plan;
mod pr_review;
pub mod sbom;
pub mod semgrep;
+409 -38
View File
@@ -10,12 +10,12 @@ use compliance_core::AgentConfig;
use crate::database::Database;
use crate::error::AgentError;
use crate::llm::LlmClient;
use crate::pipeline::code_review::CodeReviewScanner;
use crate::pipeline::cve::CveScanner;
use crate::pipeline::git::GitOps;
use crate::pipeline::gitleaks::GitleaksScanner;
use crate::pipeline::lint::LintScanner;
use crate::pipeline::patterns::{GdprPatternScanner, OAuthPatternScanner};
use crate::pipeline::plan::build_scan_plan;
use crate::pipeline::sbom::SbomScanner;
use crate::pipeline::semgrep::SemgrepScanner;
@@ -175,19 +175,26 @@ impl PipelineOrchestrator {
k.expose_secret().to_string()
}),
);
let cve_alerts = match async {
cve_scanner
.scan_dependencies(&repo_id, &mut sbom_entries)
.await
}
.instrument(tracing::info_span!("stage_cve_scanning"))
let cve_alerts = match tokio::time::timeout(
std::time::Duration::from_secs(600),
async {
cve_scanner
.scan_dependencies(&repo_id, &mut sbom_entries)
.await
}
.instrument(tracing::info_span!("stage_cve_scanning")),
)
.await
{
Ok(alerts) => alerts,
Err(e) => {
Ok(Ok(alerts)) => alerts,
Ok(Err(e)) => {
tracing::warn!("[{repo_id}] CVE scanning failed: {e}");
Vec::new()
}
Err(_) => {
tracing::warn!("[{repo_id}] CVE scanning timed out after 10 minutes");
Vec::new()
}
};
// Stage 4: Pattern Scanning (GDPR + OAuth)
@@ -241,21 +248,6 @@ impl PipelineOrchestrator {
Err(e) => tracing::warn!("[{repo_id}] Lint scanning failed: {e}"),
}
// Stage 4c: LLM Code Review (only on incremental scans)
if let Some(old_sha) = &repo.last_scanned_commit {
tracing::info!("[{repo_id}] Stage 4c: LLM Code Review");
self.update_phase(scan_run_id, "code_review").await;
let review_output = async {
let reviewer = CodeReviewScanner::new(self.llm.clone());
reviewer
.review_diff(&repo_path, &repo_id, old_sha, &current_sha)
.await
}
.instrument(tracing::info_span!("stage_code_review"))
.await;
all_findings.extend(review_output.findings);
}
// Stage 4.5: Graph Building
tracing::info!("[{repo_id}] Stage 4.5: Graph Building");
self.update_phase(scan_run_id, "graph_building").await;
@@ -331,20 +323,67 @@ impl PipelineOrchestrator {
.await?;
}
// Persist CVE alerts (upsert by cve_id + repo_id)
for alert in &cve_alerts {
let filter = doc! {
"cve_id": &alert.cve_id,
"repo_id": &alert.repo_id,
};
let update = mongodb::bson::to_document(alert)
.map(|d| doc! { "$set": d })
.unwrap_or_else(|_| doc! {});
self.db
.cve_alerts()
.update_one(filter, update)
.upsert(true)
.await?;
// Persist CVE alerts and create notifications
{
use compliance_core::models::notification::{parse_severity, CveNotification};
let repo_name = repo.name.clone();
let mut new_notif_count = 0u32;
for alert in &cve_alerts {
// Upsert the alert
let filter = doc! {
"cve_id": &alert.cve_id,
"repo_id": &alert.repo_id,
};
let update = mongodb::bson::to_document(alert)
.map(|d| doc! { "$set": d })
.unwrap_or_else(|_| doc! {});
self.db
.cve_alerts()
.update_one(filter, update)
.upsert(true)
.await?;
// Create notification (dedup by cve_id + repo + package + version)
let notif_filter = doc! {
"cve_id": &alert.cve_id,
"repo_id": &alert.repo_id,
"package_name": &alert.affected_package,
"package_version": &alert.affected_version,
};
let severity = parse_severity(alert.severity.as_deref(), alert.cvss_score);
let mut notification = CveNotification::new(
alert.cve_id.clone(),
repo_id.clone(),
repo_name.clone(),
alert.affected_package.clone(),
alert.affected_version.clone(),
severity,
);
notification.cvss_score = alert.cvss_score;
notification.summary = alert.summary.clone();
notification.url = Some(format!("https://osv.dev/vulnerability/{}", alert.cve_id));
let notif_update = doc! {
"$setOnInsert": mongodb::bson::to_bson(&notification).unwrap_or_default()
};
if let Ok(result) = self
.db
.cve_notifications()
.update_one(notif_filter, notif_update)
.upsert(true)
.await
{
if result.upserted_id.is_some() {
new_notif_count += 1;
}
}
}
if new_notif_count > 0 {
tracing::info!("[{repo_id}] Created {new_notif_count} CVE notification(s)");
}
}
// Stage 6: Issue Creation
@@ -381,6 +420,260 @@ impl PipelineOrchestrator {
Ok(new_count)
}
/// Unified entry point (behind `UNIFIED_PIPELINE`): run a scan for an
/// `OnboardedTarget`. Mirrors [`Self::run`] but sources the target from
/// `onboarded_targets` and dispatches by the scan plan.
#[tracing::instrument(skip_all, fields(target_id = %target_id, trigger = ?trigger))]
pub async fn run_target(
&self,
target_id: &str,
trigger: ScanTrigger,
) -> Result<(), AgentError> {
let oid = mongodb::bson::oid::ObjectId::parse_str(target_id)
.map_err(|e| AgentError::Other(e.to_string()))?;
let target = self
.db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await?
.ok_or_else(|| AgentError::Other(format!("Onboarded target {target_id} not found")))?;
let scan_run = ScanRun::new(target_id.to_string(), trigger);
let insert = self.db.scan_runs().insert_one(&scan_run).await?;
let scan_run_id = insert
.inserted_id
.as_object_id()
.map(|id| id.to_hex())
.unwrap_or_default();
let result = self.run_target_pipeline(&target, &scan_run_id).await;
match &result {
Ok(count) => {
self.db
.scan_runs()
.update_one(
doc! { "_id": &insert.inserted_id },
doc! { "$set": {
"status": "completed",
"current_phase": "completed",
"new_findings_count": *count as i64,
"completed_at": mongodb::bson::DateTime::now(),
} },
)
.await?;
}
Err(e) => {
tracing::error!(target_id, error = %e, "Unified scan pipeline failed");
self.db
.scan_runs()
.update_one(
doc! { "_id": &insert.inserted_id },
doc! { "$set": {
"status": "failed",
"error_message": e.to_string(),
"completed_at": mongodb::bson::DateTime::now(),
} },
)
.await?;
}
}
result.map(|_| ())
}
/// Run the applicable scans for a target. For a code target this reuses the
/// full legacy pipeline over the code artifact (clone → SAST umbrella →
/// triage → persist → issues → DAST); firmware/PLC/mobile scanners are
/// follow-ups (#128/#129/#130). Returns the number of new findings.
async fn run_target_pipeline(
&self,
target: &OnboardedTarget,
scan_run_id: &str,
) -> Result<u32, AgentError> {
let target_id = target.id.map(|id| id.to_hex()).unwrap_or_default();
let plan = build_scan_plan(target);
tracing::info!(
target_id = %target_id,
target_type = %target.target_type,
planned_steps = plan.steps.len(),
"Unified pipeline: scan plan built"
);
// Ingest + classify (tramiton for firmware) and store the detected type.
self.classify_and_store(target, &target_id, scan_run_id)
.await;
// Provision a DAST target from a LiveUrl artifact so DAST fires for
// wizard-created targets, not just migrated ones.
self.ensure_dast_target(target, &plan).await;
match target.code_artifact() {
Some(code) if code.kind == ArtifactKind::GitRepo => {
let repo = repo_view_from_target(target, code);
let new_count = self.run_pipeline(&repo, scan_run_id).await?;
self.finalize_target(target, &repo, new_count).await?;
Ok(new_count)
}
Some(_) => {
tracing::warn!(
target_id = %target_id,
"Unified pipeline: source-archive scanning not yet wired; skipping"
);
Ok(0)
}
None => {
// No code to scan. Firmware/PLC/mobile static scanners land in
// #128/#129/#130; DAST for a running URL still works when a
// DastTarget row exists (migrated targets).
tracing::info!(
target_id = %target_id,
"Unified pipeline: no code artifact; attempting DAST only"
);
self.update_phase(scan_run_id, "dast_scanning").await;
self.maybe_trigger_dast(&target_id, scan_run_id).await;
Ok(0)
}
}
}
/// Ingest the target's artifacts, classify (tramiton for firmware/RTOS/Yocto,
/// heuristics otherwise), and store the detected classification on the target.
/// Best-effort — never fails the scan.
async fn classify_and_store(
&self,
target: &OnboardedTarget,
target_id: &str,
scan_run_id: &str,
) {
self.update_phase(scan_run_id, "classification").await;
let ctx = crate::ingest::IngestContext::from_config(&self.config, target_id);
let ingest_set = match crate::ingest::ingest_all(target, &ctx) {
Ok(set) => set,
Err(e) => {
tracing::warn!(target_id, error = %e, "Unified pipeline: ingest for classification failed");
return;
}
};
let working_paths = ingest_set.working_paths();
match crate::classify::classify_target(
target,
&working_paths,
&crate::classify::TramitonNative,
)
.await
{
Ok(classification) => {
tracing::info!(
target_id,
suggested = %classification.suggested,
"Unified pipeline: classified target"
);
if let (Some(oid), Ok(bson)) = (target.id, mongodb::bson::to_bson(&classification))
{
let _ = self
.db
.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "classification": bson } },
)
.await;
}
}
Err(e) => {
tracing::warn!(target_id, error = %e, "Unified pipeline: classification failed")
}
}
}
/// If the target has a `LiveUrl` artifact and DAST is planned, provision a
/// `DastTarget` (keyed by `repo_id` = target id) so the existing DAST trigger
/// fires for wizard-created targets. Idempotent.
async fn ensure_dast_target(
&self,
target: &OnboardedTarget,
plan: &crate::pipeline::plan::ScanPlan,
) {
if !plan.has(ScanType::Dast) {
return;
}
let (Some(url), Some(oid)) = (target.live_url(), target.id) else {
return;
};
let target_id = oid.to_hex();
if self
.db
.dast_targets()
.find_one(doc! { "repo_id": &target_id })
.await
.ok()
.flatten()
.is_some()
{
return; // already provisioned
}
let kind = url
.web
.as_ref()
.map(|w| w.target_kind.clone())
.unwrap_or(DastTargetType::WebApp);
let mut dast = DastTarget::new(target.name.clone(), url.source_ref.clone(), kind);
dast.repo_id = Some(target_id);
if let Some(web) = &url.web {
dast.excluded_paths = web.excluded_paths.clone();
dast.max_crawl_depth = web.max_crawl_depth;
dast.rate_limit = web.rate_limit;
dast.allow_destructive = web.allow_destructive;
}
if let Some(auth) = &url.auth {
dast.auth_config = Some(DastAuthConfig {
method: auth.method.clone(),
login_url: auth.login_url.clone(),
username: auth.username.clone(),
password: None,
token: auth.secret.clone(),
headers: auth.headers.clone(),
});
}
if let Err(e) = self.db.dast_targets().insert_one(&dast).await {
tracing::warn!(error = %e, "Unified pipeline: failed to provision DAST target");
}
}
/// Sync the onboarded-target document after a scan: bump `findings_count`
/// and advance the git artifact's `last_scanned_commit` watermark.
async fn finalize_target(
&self,
target: &OnboardedTarget,
repo: &TrackedRepository,
new_count: u32,
) -> Result<(), AgentError> {
let oid = match target.id {
Some(id) => id,
None => return Ok(()),
};
self.db
.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! {
"$inc": { "findings_count": new_count as i64 },
"$set": { "updated_at": mongodb::bson::DateTime::now() },
},
)
.await?;
let repo_path = std::path::Path::new(&self.config.git_clone_base_path).join(&repo.name);
if let (Ok(sha), Some(code)) = (GitOps::get_head_sha(&repo_path), target.code_artifact()) {
self.db
.onboarded_targets()
.update_one(
doc! { "_id": oid, "artifacts.id": &code.id },
doc! { "$set": { "artifacts.$.git.last_scanned_commit": sha } },
)
.await?;
}
Ok(())
}
pub(super) async fn update_phase(&self, scan_run_id: &str, phase: &str) {
if let Ok(oid) = mongodb::bson::oid::ObjectId::parse_str(scan_run_id) {
let _ = self
@@ -398,6 +691,37 @@ impl PipelineOrchestrator {
}
}
/// Build a legacy `TrackedRepository` view from an onboarded target's code
/// artifact, so the unified pipeline can reuse the existing repo pipeline. The
/// inverse of the migration's `repo_to_target`. `_id` is preserved so findings
/// and DAST lookups resolve against the same key.
fn repo_view_from_target(target: &OnboardedTarget, code: &Artifact) -> TrackedRepository {
let mut repo = TrackedRepository::new(target.name.clone(), code.source_ref.clone());
repo.id = target.id;
if let Some(git) = &code.git {
repo.default_branch = git.default_branch.clone();
repo.last_scanned_commit = git.last_scanned_commit.clone();
repo.local_path = git.local_path.clone();
}
if let Some(auth) = &code.auth {
repo.auth_token = auth.secret.clone();
repo.auth_username = auth.username.clone();
}
if let Some(it) = &target.scan_config.issue_tracker {
repo.tracker_type = it.tracker_type.clone();
repo.tracker_owner = it.owner.clone();
repo.tracker_repo = it.repo.clone();
repo.tracker_token = it.token.clone();
}
repo.scan_schedule = target.scan_schedule.clone();
repo.webhook_enabled = target.webhook_enabled;
repo.webhook_secret = target.webhook_secret.clone();
repo.findings_count = target.findings_count;
repo.created_at = target.created_at;
repo.updated_at = target.updated_at;
repo
}
/// Extract the scheme + host from a git URL.
/// e.g. "https://gitea.example.com/owner/repo.git" -> "https://gitea.example.com"
/// e.g. "ssh://git@gitea.example.com:22/owner/repo.git" -> "https://gitea.example.com"
@@ -422,3 +746,50 @@ pub(super) fn extract_base_url(git_url: &str) -> Option<String> {
None
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{
Artifact, ArtifactAuth, IssueTrackerConfig, TargetType, TrackerType,
};
#[test]
fn repo_view_preserves_id_git_auth_and_tracker() {
let mut target = OnboardedTarget::new("acme".to_string(), TargetType::WebApp);
target.id = Some(mongodb::bson::oid::ObjectId::new());
target.findings_count = 3;
target.scan_config.issue_tracker = Some(IssueTrackerConfig {
tracker_type: Some(TrackerType::Gitea),
owner: Some("acme".to_string()),
repo: Some("web".to_string()),
token: Some("tt".to_string()),
});
let mut artifact = Artifact::git_repo("https://git/acme.git", "develop");
if let Some(git) = artifact.git.as_mut() {
git.last_scanned_commit = Some("abc123".to_string());
}
artifact.auth = Some(ArtifactAuth {
method: "token".to_string(),
username: Some("bob".to_string()),
secret: Some("pat".to_string()),
..Default::default()
});
target.artifacts.push(artifact);
let code = target.code_artifact().expect("code artifact");
let repo = repo_view_from_target(&target, code);
assert_eq!(repo.id, target.id); // preserved
assert_eq!(repo.git_url, "https://git/acme.git");
assert_eq!(repo.default_branch, "develop");
assert_eq!(repo.last_scanned_commit.as_deref(), Some("abc123"));
assert_eq!(repo.auth_token.as_deref(), Some("pat"));
assert_eq!(repo.auth_username.as_deref(), Some("bob"));
assert_eq!(repo.tracker_type, Some(TrackerType::Gitea));
assert_eq!(repo.tracker_owner.as_deref(), Some("acme"));
assert_eq!(repo.findings_count, 3);
}
}
+3 -1
View File
@@ -33,6 +33,7 @@ struct PatternRule {
file_extensions: Vec<String>,
}
#[allow(clippy::new_without_default)]
impl GdprPatternScanner {
pub fn new() -> Self {
let patterns = vec![
@@ -98,6 +99,7 @@ impl Scanner for GdprPatternScanner {
}
}
#[allow(clippy::new_without_default)]
impl OAuthPatternScanner {
pub fn new() -> Self {
let patterns = vec![
@@ -213,7 +215,7 @@ fn scan_with_patterns(
repo_id.to_string(),
fingerprint,
scanner_name.to_string(),
scan_type.clone(),
scan_type,
pattern.title.clone(),
pattern.description.clone(),
pattern.severity.clone(),
+195
View File
@@ -0,0 +1,195 @@
//! The scan plan.
//!
//! [`build_scan_plan`] turns an [`OnboardedTarget`] into the concrete ordered
//! list of scans to run, each bound to the artifact it consumes. It intersects
//! the scan-applicability matrix ([`applicable_scans`]) with the target's
//! `scan_config` overrides: a scan runs when its required artifact is present and
//! it is either on by default or explicitly enabled, and is not explicitly
//! disabled. This is the decision engine the unified pipeline (`run_target`)
//! executes.
use compliance_core::models::{Artifact, ArtifactKind, OnboardedTarget, ScanPhase, ScanType};
use compliance_core::scan_matrix::applicable_scans;
/// One scan to run, bound to the artifact it operates on.
#[derive(Debug, Clone, PartialEq)]
pub struct ScanStep {
/// The scan to run.
pub scan_type: ScanType,
/// The pipeline phase to report while it runs.
pub phase: ScanPhase,
/// The id of the artifact this scan consumes ([`Artifact::id`]).
pub artifact_id: String,
}
/// The ordered set of scans to run for a target.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct ScanPlan {
/// The scans, in matrix order.
pub steps: Vec<ScanStep>,
}
impl ScanPlan {
/// Whether the plan contains a step for the given scan type.
pub fn has(&self, scan: ScanType) -> bool {
self.steps.iter().any(|s| s.scan_type == scan)
}
/// Whether the plan is empty (nothing to run).
pub fn is_empty(&self) -> bool {
self.steps.is_empty()
}
}
/// Build the scan plan for a target: matrix defaults ∩ `scan_config`, each scan
/// bound to the artifact it consumes. Scans whose required artifact is absent, or
/// that are disabled, or off-by-default and not explicitly enabled, are dropped.
pub fn build_scan_plan(target: &OnboardedTarget) -> ScanPlan {
let enabled = &target.scan_config.enabled_scans;
let disabled = &target.scan_config.disabled_scans;
let mut steps = Vec::new();
for option in applicable_scans(target) {
// Required artifact missing → not runnable.
if option.blocked_reason.is_some() {
continue;
}
// Explicit opt-out wins.
if disabled.contains(&option.scan) {
continue;
}
// Run if on by default, or explicitly enabled.
if !option.default_on && !enabled.contains(&option.scan) {
continue;
}
let Some(artifact) = resolve_artifact(target, option.required_artifact) else {
continue;
};
steps.push(ScanStep {
scan_type: option.scan,
phase: phase_for(option.scan),
artifact_id: artifact.id.clone(),
});
}
ScanPlan { steps }
}
/// Resolve the artifact a scan consumes. A "code" requirement (represented by
/// `GitRepo`) is satisfied by a git repo *or* a source archive.
fn resolve_artifact(target: &OnboardedTarget, required: Option<ArtifactKind>) -> Option<&Artifact> {
match required {
Some(ArtifactKind::GitRepo) => target.code_artifact(),
Some(kind) => target.first_of(kind),
None => target.code_artifact().or_else(|| target.artifacts.first()),
}
}
/// The pipeline phase reported while a given scan runs.
fn phase_for(scan: ScanType) -> ScanPhase {
match scan {
ScanType::Sast => ScanPhase::Sast,
ScanType::Sbom => ScanPhase::SbomGeneration,
ScanType::Cve => ScanPhase::CveScanning,
ScanType::Gdpr | ScanType::OAuth => ScanPhase::PatternScanning,
ScanType::SecretDetection => ScanPhase::SecretDetection,
ScanType::Lint => ScanPhase::LintScanning,
ScanType::CodeReview => ScanPhase::CodeReview,
ScanType::Graph => ScanPhase::GraphBuilding,
ScanType::Dast => ScanPhase::DastScanning,
ScanType::FirmwareStatic => ScanPhase::FirmwareStatic,
ScanType::PlcControlLogic => ScanPhase::PlcAnalysis,
ScanType::MobileStatic => ScanPhase::MobileStatic,
ScanType::ContainerScan => ScanPhase::ContainerScan,
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{PlcFormat, TargetType};
fn target(target_type: TargetType, artifacts: Vec<Artifact>) -> OnboardedTarget {
let mut t = OnboardedTarget::new("t".to_string(), target_type);
t.artifacts = artifacts;
t
}
fn step_for<'a>(plan: &'a ScanPlan, scan: ScanType) -> Option<&'a ScanStep> {
plan.steps.iter().find(|s| s.scan_type == scan)
}
#[test]
fn webapp_with_code_and_url_runs_sast_and_dast_bound_to_the_right_artifacts() {
let code = Artifact::git_repo("https://git/x", "main");
let url = Artifact::live_url("https://x");
let (code_id, url_id) = (code.id.clone(), url.id.clone());
let t = target(TargetType::WebApp, vec![code, url]);
let plan = build_scan_plan(&t);
let sast = step_for(&plan, ScanType::Sast).expect("sast planned");
assert_eq!(sast.artifact_id, code_id);
assert_eq!(sast.phase, ScanPhase::Sast);
let dast = step_for(&plan, ScanType::Dast).expect("dast planned");
assert_eq!(dast.artifact_id, url_id);
}
#[test]
fn webapp_without_url_omits_dast() {
let t = target(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
let plan = build_scan_plan(&t);
assert!(plan.has(ScanType::Sast));
assert!(!plan.has(ScanType::Dast));
}
#[test]
fn code_scan_binds_to_source_archive_when_no_git_repo() {
let arc = Artifact::source_archive("src.zip");
let arc_id = arc.id.clone();
let t = target(TargetType::BackendService, vec![arc]);
let plan = build_scan_plan(&t);
let sast = step_for(&plan, ScanType::Sast).expect("sast planned");
assert_eq!(sast.artifact_id, arc_id);
}
#[test]
fn firmware_sbom_and_cve_bind_to_the_firmware_image() {
let fw = Artifact::firmware_image("fw.bin");
let fw_id = fw.id.clone();
let t = target(TargetType::FirmwareBareMetal, vec![fw]);
let plan = build_scan_plan(&t);
let sbom = step_for(&plan, ScanType::Sbom).expect("sbom planned");
assert_eq!(sbom.artifact_id, fw_id);
assert!(step_for(&plan, ScanType::FirmwareStatic).is_some());
assert!(!plan.has(ScanType::Dast));
}
#[test]
fn plc_plans_only_control_logic() {
let plc = Artifact::plc_project("p.xml", PlcFormat::PlcopenXml);
let t = target(TargetType::PlcSps, vec![plc]);
let plan = build_scan_plan(&t);
assert_eq!(plan.steps.len(), 1);
assert_eq!(plan.steps[0].scan_type, ScanType::PlcControlLogic);
assert_eq!(plan.steps[0].phase, ScanPhase::PlcAnalysis);
}
#[test]
fn disabled_scan_is_dropped_and_off_by_default_can_be_enabled() {
let mut t = target(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
t.scan_config.disabled_scans = vec![ScanType::Lint];
// CodeReview is off by default for web; enable it explicitly.
t.scan_config.enabled_scans = vec![ScanType::CodeReview];
let plan = build_scan_plan(&t);
assert!(!plan.has(ScanType::Lint));
assert!(plan.has(ScanType::CodeReview));
assert!(plan.has(ScanType::Sast));
}
#[test]
fn no_code_artifact_yields_empty_plan_for_web() {
let t = target(TargetType::WebApp, vec![]);
let plan = build_scan_plan(&t);
assert!(plan.is_empty());
}
}
+38 -1
View File
@@ -1,5 +1,6 @@
use compliance_core::models::*;
use super::dedup::compute_fingerprint;
use super::orchestrator::PipelineOrchestrator;
use crate::error::AgentError;
use crate::pipeline::code_review::CodeReviewScanner;
@@ -89,12 +90,37 @@ impl PipelineOrchestrator {
return Ok(());
}
// Dedup findings by fingerprint to avoid duplicate comments
let mut seen_fps = std::collections::HashSet::new();
let mut unique_findings: Vec<&Finding> = Vec::new();
for finding in &pr_findings {
let fp = compute_fingerprint(&[
repo_id,
&pr_number.to_string(),
finding.file_path.as_deref().unwrap_or(""),
&finding.line_number.unwrap_or(0).to_string(),
&finding.title,
]);
if seen_fps.insert(fp) {
unique_findings.push(finding);
}
}
let pr_findings = unique_findings;
// Build review comments from findings
let mut review_comments = Vec::new();
for finding in &pr_findings {
if let (Some(path), Some(line)) = (&finding.file_path, finding.line_number) {
let fp = compute_fingerprint(&[
repo_id,
&pr_number.to_string(),
path,
&line.to_string(),
&finding.title,
]);
let comment_body = format!(
"**[{}] {}**\n\n{}\n\n*Scanner: {} | {}*",
"**[{}] {}**\n\n{}\n\n*Scanner: {} | {}*\n\n<!-- compliance-fp:{fp} -->",
finding.severity,
finding.title,
finding.description,
@@ -123,6 +149,17 @@ impl PipelineOrchestrator {
.join("\n"),
);
if review_comments.is_empty() {
// All findings were on files/lines we can't comment on inline
if let Err(e) = tracker
.create_pr_review(owner, tracker_repo_name, pr_number, &summary, Vec::new())
.await
{
tracing::warn!("[{repo_id}] Failed to post PR review summary: {e}");
}
return Ok(());
}
if let Err(e) = tracker
.create_pr_review(
owner,
+20 -14
View File
@@ -5,20 +5,26 @@ use compliance_core::CoreError;
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub(super) async fn run_syft(repo_path: &Path, repo_id: &str) -> Result<Vec<SbomEntry>, CoreError> {
let output = tokio::process::Command::new("syft")
.arg(repo_path)
.args(["-o", "cyclonedx-json"])
// Enable remote license lookups for all ecosystems
.env("SYFT_GOLANG_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_JAVASCRIPT_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_PYTHON_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_JAVA_USE_NETWORK", "true")
.output()
.await
.map_err(|e| CoreError::Scanner {
scanner: "syft".to_string(),
source: Box::new(e),
})?;
let output = tokio::time::timeout(
std::time::Duration::from_secs(300),
tokio::process::Command::new("syft")
.arg(repo_path)
.args(["-o", "cyclonedx-json"])
.env("SYFT_GOLANG_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_JAVASCRIPT_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_PYTHON_SEARCH_REMOTE_LICENSES", "true")
.env("SYFT_JAVA_USE_NETWORK", "true")
.output(),
)
.await
.map_err(|_| CoreError::Scanner {
scanner: "syft".to_string(),
source: "timed out after 5 minutes".into(),
})?
.map_err(|e| CoreError::Scanner {
scanner: "syft".to_string(),
source: Box::new(e),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
+24 -9
View File
@@ -19,15 +19,30 @@ impl Scanner for SemgrepScanner {
#[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let output = tokio::process::Command::new("semgrep")
.args(["--config=auto", "--json", "--quiet"])
.arg(repo_path)
.output()
.await
.map_err(|e| CoreError::Scanner {
scanner: "semgrep".to_string(),
source: Box::new(e),
})?;
let output = tokio::time::timeout(
std::time::Duration::from_secs(600),
tokio::process::Command::new("semgrep")
.args([
"--config=auto",
"--json",
"--quiet",
"--max-memory",
"500",
"--jobs",
"1",
])
.arg(repo_path)
.output(),
)
.await
.map_err(|_| CoreError::Scanner {
scanner: "semgrep".to_string(),
source: "timed out after 10 minutes".into(),
})?
.map_err(|e| CoreError::Scanner {
scanner: "semgrep".to_string(),
source: Box::new(e),
})?;
if !output.status.success() && output.stdout.is_empty() {
let stderr = String::from_utf8_lossy(&output.stderr);
+87 -19
View File
@@ -6,11 +6,16 @@ use compliance_core::models::embedding::{CodeEmbedding, EmbeddingBuildRun, Embed
use compliance_core::models::graph::CodeNode;
use compliance_graph::graph::chunking::extract_chunks;
use compliance_graph::graph::embedding_store::EmbeddingStore;
use futures_util::stream::{FuturesUnordered, StreamExt};
use tracing::{error, info};
use crate::error::AgentError;
use crate::llm::LlmClient;
const EMBED_BATCH_SIZE: usize = 20;
const EMBED_CONCURRENCY: usize = 4;
const EMBED_FLUSH_EVERY: usize = 200;
/// RAG pipeline for building embeddings and performing retrieval
pub struct RagPipeline {
llm: Arc<LlmClient>,
@@ -77,25 +82,33 @@ impl RagPipeline {
.await
.map_err(|e| AgentError::Other(format!("Failed to delete old embeddings: {e}")))?;
// Step 3: Batch embed (small batches to stay within model limits)
let batch_size = 20;
let mut all_embeddings = Vec::new();
// Step 3: Batch embed with bounded concurrency. Flush to Mongo and
// update progress periodically so the dashboard can show live status.
let mut pending = Vec::with_capacity(EMBED_FLUSH_EVERY);
let mut embedded_count = 0u32;
for batch_start in (0..chunks.len()).step_by(batch_size) {
let batch_end = (batch_start + batch_size).min(chunks.len());
let batch_chunks = &chunks[batch_start..batch_end];
// Build the list of batch indices to process.
let batches: Vec<(usize, usize)> = (0..chunks.len())
.step_by(EMBED_BATCH_SIZE)
.map(|start| (start, (start + EMBED_BATCH_SIZE).min(chunks.len())))
.collect();
// Prepare texts: context_header + content
let texts: Vec<String> = batch_chunks
.iter()
.map(|c| format!("{}\n{}", c.context_header, c.content))
.collect();
let mut batch_iter = batches.into_iter();
let mut in_flight = FuturesUnordered::new();
match self.llm.embed(texts).await {
Ok(vectors) => {
// Prime up to EMBED_CONCURRENCY batches.
for _ in 0..EMBED_CONCURRENCY {
if let Some((start, end)) = batch_iter.next() {
in_flight.push(self.embed_batch(&chunks[start..end], start, end));
}
}
while let Some(result) = in_flight.next().await {
match result {
Ok((start, end, vectors)) => {
let batch_chunks = &chunks[start..end];
for (chunk, embedding) in batch_chunks.iter().zip(vectors) {
all_embeddings.push(CodeEmbedding {
pending.push(CodeEmbedding {
id: None,
repo_id: repo_id.to_string(),
graph_build_id: graph_build_id.to_string(),
@@ -113,9 +126,45 @@ impl RagPipeline {
});
}
embedded_count += batch_chunks.len() as u32;
// Flush pending embeddings to Mongo periodically and update progress.
if pending.len() >= EMBED_FLUSH_EVERY {
self.embedding_store
.store_embeddings(&pending)
.await
.map_err(|e| {
AgentError::Other(format!("Failed to store embeddings: {e}"))
})?;
pending.clear();
}
// Always update the progress counter on the build doc — even if
// we haven't flushed embeddings yet — so the UI shows movement.
if let Err(e) = self
.embedding_store
.update_build(
repo_id,
graph_build_id,
EmbeddingBuildStatus::Running,
embedded_count,
None,
)
.await
{
error!("[{repo_id}] Failed to update build progress: {e}");
}
// Queue the next batch to keep concurrency saturated.
if let Some((s, e)) = batch_iter.next() {
in_flight.push(self.embed_batch(&chunks[s..e], s, e));
}
}
Err(e) => {
error!("[{repo_id}] Embedding batch failed: {e}");
// Flush whatever we have so partial progress isn't lost.
if !pending.is_empty() {
let _ = self.embedding_store.store_embeddings(&pending).await;
}
build.status = EmbeddingBuildStatus::Failed;
build.error_message = Some(e.to_string());
build.completed_at = Some(Utc::now());
@@ -134,11 +183,13 @@ impl RagPipeline {
}
}
// Step 4: Store all embeddings
self.embedding_store
.store_embeddings(&all_embeddings)
.await
.map_err(|e| AgentError::Other(format!("Failed to store embeddings: {e}")))?;
// Step 4: Flush any remaining embeddings
if !pending.is_empty() {
self.embedding_store
.store_embeddings(&pending)
.await
.map_err(|e| AgentError::Other(format!("Failed to store embeddings: {e}")))?;
}
// Step 5: Update build status
build.status = EmbeddingBuildStatus::Completed;
@@ -161,4 +212,21 @@ impl RagPipeline {
);
Ok(build)
}
/// Embed one batch of chunks. Returns the (start, end, vectors) tuple so
/// out-of-order completion from `FuturesUnordered` can still be reconciled
/// against the original chunk slice.
async fn embed_batch(
&self,
batch_chunks: &[compliance_graph::graph::chunking::CodeChunk],
start: usize,
end: usize,
) -> Result<(usize, usize, Vec<Vec<f64>>), AgentError> {
let texts: Vec<String> = batch_chunks
.iter()
.map(|c| format!("{}\n{}", c.context_header, c.content))
.collect();
let vectors = self.llm.embed(texts).await?;
Ok((start, end, vectors))
}
}
+394 -25
View File
@@ -4,8 +4,21 @@ use tokio_cron_scheduler::{Job, JobScheduler};
use compliance_core::models::ScanTrigger;
use crate::agent::ComplianceAgent;
use crate::database::Database;
use crate::error::AgentError;
/// Default tenant the scheduler runs against when neither the tenant
/// registry nor `SCHEDULER_TENANT_IDS` are configured. Matches the
/// dev-injector default so a bare `cargo run` has the scheduler
/// scanning whatever lives in `<prefix>_dev`.
const DEFAULT_SCHEDULER_TENANT_ID: &str = "dev";
/// Request timeout when fetching the live tenant list from the
/// registry. Kept short — if the registry is slow we'd rather fall
/// back to env-configured ids and finish the tick than block the
/// scheduler loop.
const REGISTRY_FETCH_TIMEOUT_SECS: u64 = 5;
pub async fn start_scheduler(agent: &ComplianceAgent) -> Result<(), AgentError> {
let sched = JobScheduler::new()
.await
@@ -18,7 +31,14 @@ pub async fn start_scheduler(agent: &ComplianceAgent) -> Result<(), AgentError>
let agent = scan_agent.clone();
Box::pin(async move {
tracing::info!("Scheduled scan triggered");
scan_all_repos(&agent).await;
let tenants = scheduler_tenants(&agent).await;
tracing::debug!(
tenant_count = tenants.len(),
"Scheduled scan: tenants resolved"
);
for tenant_id in tenants {
scan_all_repos(&agent, &tenant_id).await;
}
})
})
.map_err(|e| AgentError::Scheduler(format!("Failed to create scan job: {e}")))?;
@@ -34,7 +54,14 @@ pub async fn start_scheduler(agent: &ComplianceAgent) -> Result<(), AgentError>
let agent = cve_agent.clone();
Box::pin(async move {
tracing::info!("CVE monitor triggered");
monitor_cves(&agent).await;
let tenants = scheduler_tenants(&agent).await;
tracing::debug!(
tenant_count = tenants.len(),
"CVE monitor: tenants resolved"
);
for tenant_id in tenants {
monitor_cves(&agent, &tenant_id).await;
}
})
})
.map_err(|e| AgentError::Scheduler(format!("Failed to create CVE monitor job: {e}")))?;
@@ -48,8 +75,14 @@ pub async fn start_scheduler(agent: &ComplianceAgent) -> Result<(), AgentError>
.await
.map_err(|e| AgentError::Scheduler(format!("Failed to start scheduler: {e}")))?;
let tenants = scheduler_tenants(agent).await;
let source = if agent.config.tenant_registry_url.is_some() {
"tenant-registry (env fallback)"
} else {
"env (SCHEDULER_TENANT_IDS)"
};
tracing::info!(
"Scheduler started: scans='{}', CVE monitor='{}'",
"Scheduler started: scans='{}', CVE monitor='{}', tenant source={source}, tenants={tenants:?}",
agent.config.scan_schedule,
agent.config.cve_monitor_schedule,
);
@@ -60,46 +93,382 @@ pub async fn start_scheduler(agent: &ComplianceAgent) -> Result<(), AgentError>
}
}
async fn scan_all_repos(agent: &ComplianceAgent) {
use futures_util::StreamExt;
let cursor = match agent.db.repositories().find(doc! {}).await {
Ok(c) => c,
Err(e) => {
tracing::error!("Failed to list repos for scheduled scan: {e}");
return;
/// Tenants the scheduler iterates each tick.
///
/// Resolution order:
/// 1. **Tenant registry** at `agent.config.tenant_registry_url`
/// (`GET /v1/tenants`). Fresh on every tick — picks up newly
/// provisioned tenants without an agent restart.
/// 2. **`SCHEDULER_TENANT_IDS`** env (comma-separated) — fallback when
/// the registry is unreachable, the response is malformed, or no
/// registry URL is configured.
/// 3. **`DEFAULT_SCHEDULER_TENANT_ID`** (`"dev"`) — last-ditch fallback
/// so the scheduler keeps doing something useful in dev.
///
/// We never panic out of this function — the scheduler must keep
/// firing even if the registry is offline.
async fn scheduler_tenants(agent: &ComplianceAgent) -> Vec<String> {
if let Some(url) = agent.config.tenant_registry_url.as_deref() {
match fetch_tenants_from_registry(&agent.http, url).await {
Ok(v) if !v.is_empty() => return v,
Ok(_) => {
tracing::warn!("tenant-registry returned empty list; falling back to env");
}
Err(e) => {
tracing::warn!(
url = %url,
error = %e,
"tenant-registry fetch failed; falling back to env"
);
}
}
};
}
tenants_from_env()
}
let repos: Vec<_> = cursor.filter_map(|r| async { r.ok() }).collect().await;
fn tenants_from_env() -> Vec<String> {
std::env::var("SCHEDULER_TENANT_IDS")
.ok()
.map(|s| {
s.split(',')
.map(str::trim)
.filter(|s| !s.is_empty())
.map(String::from)
.collect::<Vec<_>>()
})
.filter(|v| !v.is_empty())
.unwrap_or_else(|| vec![DEFAULT_SCHEDULER_TENANT_ID.to_string()])
}
for repo in repos {
let repo_id = repo.id.map(|id| id.to_hex()).unwrap_or_default();
if let Err(e) = agent.run_scan(&repo_id, ScanTrigger::Scheduled).await {
tracing::error!("Scheduled scan failed for {}: {e}", repo.name);
/// Shape we accept from the registry. Liberal in what we accept:
/// the registry can return any field shape as long as either `id` or
/// `tenant_id` is present. Other fields are ignored.
#[derive(serde::Deserialize)]
struct RegistryTenant {
#[serde(alias = "tenant_id")]
id: String,
/// Filter out non-running tenants if status is present. Missing
/// status defaults to "active" so older registry deployments keep
/// working.
#[serde(default = "default_status")]
status: String,
}
fn default_status() -> String {
"active".to_string()
}
#[derive(serde::Deserialize)]
struct RegistryListResponse {
data: Vec<RegistryTenant>,
}
async fn fetch_tenants_from_registry(
http: &reqwest::Client,
base_url: &str,
) -> Result<Vec<String>, String> {
let url = format!("{}/v1/tenants", base_url.trim_end_matches('/'));
let resp = http
.get(&url)
.timeout(std::time::Duration::from_secs(REGISTRY_FETCH_TIMEOUT_SECS))
.send()
.await
.map_err(|e| format!("request failed: {e}"))?;
if !resp.status().is_success() {
return Err(format!("registry returned {}", resp.status()));
}
let body: RegistryListResponse = resp
.json()
.await
.map_err(|e| format!("invalid JSON: {e}"))?;
Ok(filter_active(body.data))
}
/// Frozen/Archived tenants don't need scheduled scans; the M7.1
/// status gate would 402/410 anyway. Skip them so we don't waste
/// cycles. Active / trial / demo / anything-else-unknown all run.
fn filter_active(rows: Vec<RegistryTenant>) -> Vec<String> {
rows.into_iter()
.filter(|t| !matches!(t.status.as_str(), "frozen" | "archived"))
.map(|t| t.id)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
fn tenant(id: &str, status: &str) -> RegistryTenant {
RegistryTenant {
id: id.to_string(),
status: status.to_string(),
}
}
#[test]
fn filter_active_keeps_running_skips_frozen_archived() {
let rows = vec![
tenant("a", "active"),
tenant("b", "trial"),
tenant("c", "demo"),
tenant("d", "frozen"),
tenant("e", "archived"),
tenant("f", "weird-but-not-known-dead"),
];
let out = filter_active(rows);
assert_eq!(out, vec!["a", "b", "c", "f"]);
}
#[test]
fn deserialize_registry_response_accepts_id_or_tenant_id() {
let body = r#"{"data":[
{"id":"a","status":"active"},
{"tenant_id":"b","status":"trial"},
{"id":"c"}
]}"#;
let parsed: RegistryListResponse = serde_json::from_str(body).unwrap();
assert_eq!(parsed.data.len(), 3);
assert_eq!(parsed.data[0].id, "a");
assert_eq!(parsed.data[1].id, "b");
assert_eq!(parsed.data[2].id, "c");
// Default status for the third entry should be "active"
assert_eq!(parsed.data[2].status, "active");
}
/// Combined into a single test: cargo runs tests in parallel and
/// env vars are process-global, so two separate tests touching
/// `SCHEDULER_TENANT_IDS` race each other. Doing both checks in
/// one test keeps them in a deterministic order.
#[test]
fn tenants_from_env_resolution() {
std::env::remove_var("SCHEDULER_TENANT_IDS");
assert_eq!(
tenants_from_env(),
vec![DEFAULT_SCHEDULER_TENANT_ID.to_string()],
"unset → default"
);
std::env::set_var("SCHEDULER_TENANT_IDS", "acme, globex ,,hello");
let out = tenants_from_env();
std::env::remove_var("SCHEDULER_TENANT_IDS");
assert_eq!(
out,
vec!["acme", "globex", "hello"],
"splits + trims + drops empty"
);
std::env::set_var("SCHEDULER_TENANT_IDS", "");
let out = tenants_from_env();
std::env::remove_var("SCHEDULER_TENANT_IDS");
assert_eq!(
out,
vec![DEFAULT_SCHEDULER_TENANT_ID.to_string()],
"empty → default"
);
}
}
/// Resolve the per-tenant database. Logs and returns `None` on failure
/// so the loop in the caller can continue with other tenants.
async fn tenant_db(agent: &ComplianceAgent, tenant_id: &str) -> Option<Database> {
match agent.db_pool.for_tenant_id(tenant_id).await {
Ok(db) => Some(db),
Err(e) => {
tracing::error!("Scheduler: cannot open tenant database '{tenant_id}': {e}");
None
}
}
}
async fn monitor_cves(agent: &ComplianceAgent) {
async fn scan_all_repos(agent: &ComplianceAgent, tenant_id: &str) {
use futures_util::StreamExt;
// Re-scan all SBOM entries for new CVEs
let cursor = match agent.db.sbom_entries().find(doc! {}).await {
let db = match tenant_db(agent, tenant_id).await {
Some(db) => db,
None => return,
};
let cursor = match db.onboarded_targets().find(doc! {}).await {
Ok(c) => c,
Err(e) => {
tracing::error!("Failed to list SBOM entries for CVE monitoring: {e}");
tracing::error!("Failed to list targets for tenant '{tenant_id}': {e}");
return;
}
};
let entries: Vec<_> = cursor.filter_map(|r| async { r.ok() }).collect().await;
let targets: Vec<_> = cursor.filter_map(|r| async { r.ok() }).collect().await;
for target in targets {
let target_id = target.id.map(|id| id.to_hex()).unwrap_or_default();
if let Err(e) = agent
.run_target_scan(tenant_id, &target_id, ScanTrigger::Scheduled)
.await
{
tracing::error!(
"Scheduled scan failed for {} (tenant '{tenant_id}'): {e}",
target.name
);
}
}
}
async fn monitor_cves(agent: &ComplianceAgent, tenant_id: &str) {
use compliance_core::models::notification::{parse_severity, CveNotification};
use compliance_core::models::SbomEntry;
use futures_util::StreamExt;
let db = match tenant_db(agent, tenant_id).await {
Some(db) => db,
None => return,
};
// Fetch all SBOM entries grouped by repo
let cursor = match db.sbom_entries().find(doc! {}).await {
Ok(c) => c,
Err(e) => {
tracing::error!("CVE monitor: failed to list SBOM entries for '{tenant_id}': {e}");
return;
}
};
let entries: Vec<SbomEntry> = cursor.filter_map(|r| async { r.ok() }).collect().await;
if entries.is_empty() {
tracing::debug!("CVE monitor: no SBOM entries for tenant '{tenant_id}', skipping");
return;
}
tracing::info!("CVE monitor: checking {} dependencies", entries.len());
// The actual CVE checking is handled by the CveScanner in the pipeline
// This is a simplified version that just logs the activity
tracing::info!(
"CVE monitor: checking {} dependencies for new CVEs (tenant '{tenant_id}')",
entries.len()
);
// Build a repo_id → repo_name lookup
let repo_ids: std::collections::HashSet<String> =
entries.iter().map(|e| e.repo_id.clone()).collect();
let mut repo_names: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
for rid in &repo_ids {
if let Ok(oid) = mongodb::bson::oid::ObjectId::parse_str(rid) {
if let Ok(Some(repo)) = db.repositories().find_one(doc! { "_id": oid }).await {
repo_names.insert(rid.clone(), repo.name.clone());
}
}
}
// Use the existing CveScanner to query OSV.dev
let nvd_key = agent.config.nvd_api_key.as_ref().map(|k| {
use secrecy::ExposeSecret;
k.expose_secret().to_string()
});
let scanner = crate::pipeline::cve::CveScanner::new(
agent.http.clone(),
agent.config.searxng_url.clone(),
nvd_key,
);
// Group entries by repo for scanning
let mut entries_by_repo: std::collections::HashMap<String, Vec<SbomEntry>> =
std::collections::HashMap::new();
for entry in entries {
entries_by_repo
.entry(entry.repo_id.clone())
.or_default()
.push(entry);
}
let mut new_notifications = 0u32;
for (repo_id, mut repo_entries) in entries_by_repo {
let repo_name = repo_names
.get(&repo_id)
.cloned()
.unwrap_or_else(|| repo_id.clone());
// Scan dependencies for CVEs
let alerts = match scanner.scan_dependencies(&repo_id, &mut repo_entries).await {
Ok(a) => a,
Err(e) => {
tracing::warn!("CVE monitor: scan failed for {repo_name}: {e}");
continue;
}
};
// Upsert CVE alerts (existing logic)
for alert in &alerts {
let filter = doc! { "cve_id": &alert.cve_id, "repo_id": &alert.repo_id };
let update = doc! { "$setOnInsert": mongodb::bson::to_bson(alert).unwrap_or_default() };
let _ = db
.cve_alerts()
.update_one(filter, update)
.upsert(true)
.await;
}
// Update SBOM entries with discovered vulnerabilities
for entry in &repo_entries {
if entry.known_vulnerabilities.is_empty() {
continue;
}
if let Some(entry_id) = &entry.id {
let _ = db
.sbom_entries()
.update_one(
doc! { "_id": entry_id },
doc! { "$set": {
"known_vulnerabilities": mongodb::bson::to_bson(&entry.known_vulnerabilities).unwrap_or_default(),
"updated_at": mongodb::bson::DateTime::now(),
}},
)
.await;
}
}
// Create notifications for NEW CVEs (dedup against existing notifications)
for alert in &alerts {
let filter = doc! {
"cve_id": &alert.cve_id,
"repo_id": &alert.repo_id,
"package_name": &alert.affected_package,
"package_version": &alert.affected_version,
};
// Only insert if not already exists (upsert with $setOnInsert)
let severity = parse_severity(alert.severity.as_deref(), alert.cvss_score);
let mut notification = CveNotification::new(
alert.cve_id.clone(),
repo_id.clone(),
repo_name.clone(),
alert.affected_package.clone(),
alert.affected_version.clone(),
severity,
);
notification.cvss_score = alert.cvss_score;
notification.summary = alert.summary.clone();
notification.url = Some(format!("https://osv.dev/vulnerability/{}", alert.cve_id));
let update = doc! {
"$setOnInsert": mongodb::bson::to_bson(&notification).unwrap_or_default()
};
match db
.cve_notifications()
.update_one(filter, update)
.upsert(true)
.await
{
Ok(result) if result.upserted_id.is_some() => {
new_notifications += 1;
}
Err(e) => {
tracing::warn!("CVE monitor: failed to create notification: {e}");
}
_ => {} // Already exists
}
}
}
if new_notifications > 0 {
tracing::info!(
"CVE monitor: created {new_notifications} new notification(s) for tenant '{tenant_id}'"
);
} else {
tracing::info!("CVE monitor: no new CVEs found for tenant '{tenant_id}'");
}
}
+64 -3
View File
@@ -98,7 +98,8 @@ impl IssueTracker for GiteaTracker {
_ => "open",
};
self.http
let resp = self
.http
.patch(&url)
.header(
"Authorization",
@@ -109,6 +110,14 @@ impl IssueTracker for GiteaTracker {
.await
.map_err(|e| CoreError::IssueTracker(format!("Gitea update issue failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(CoreError::IssueTracker(format!(
"Gitea update issue returned {status}: {text}"
)));
}
Ok(())
}
@@ -123,7 +132,8 @@ impl IssueTracker for GiteaTracker {
"/repos/{owner}/{repo}/issues/{external_id}/comments"
));
self.http
let resp = self
.http
.post(&url)
.header(
"Authorization",
@@ -134,6 +144,14 @@ impl IssueTracker for GiteaTracker {
.await
.map_err(|e| CoreError::IssueTracker(format!("Gitea add comment failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
return Err(CoreError::IssueTracker(format!(
"Gitea add comment returned {status}: {text}"
)));
}
Ok(())
}
@@ -158,7 +176,8 @@ impl IssueTracker for GiteaTracker {
})
.collect();
self.http
let resp = self
.http
.post(&url)
.header(
"Authorization",
@@ -173,6 +192,48 @@ impl IssueTracker for GiteaTracker {
.await
.map_err(|e| CoreError::IssueTracker(format!("Gitea PR review failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
// If inline comments caused the failure, retry with just the summary body
if !comments.is_empty() {
tracing::warn!(
"Gitea PR review with inline comments failed ({status}): {text}, retrying as plain comment"
);
let fallback_url = self.api_url(&format!(
"/repos/{owner}/{repo}/issues/{pr_number}/comments"
));
let fallback_resp = self
.http
.post(&fallback_url)
.header(
"Authorization",
format!("token {}", self.token.expose_secret()),
)
.json(&serde_json::json!({ "body": body }))
.send()
.await
.map_err(|e| {
CoreError::IssueTracker(format!("Gitea PR comment fallback failed: {e}"))
})?;
if !fallback_resp.status().is_success() {
let fb_status = fallback_resp.status();
let fb_text = fallback_resp.text().await.unwrap_or_default();
return Err(CoreError::IssueTracker(format!(
"Gitea PR comment fallback returned {fb_status}: {fb_text}"
)));
}
return Ok(());
}
return Err(CoreError::IssueTracker(format!(
"Gitea PR review returned {status}: {text}"
)));
}
Ok(())
}
+23 -9
View File
@@ -14,24 +14,30 @@ type HmacSha256 = Hmac<Sha256>;
pub async fn handle_gitea_webhook(
Extension(agent): Extension<Arc<ComplianceAgent>>,
Path(repo_id): Path<String>,
Path((tenant_id, repo_id)): Path<(String, String)>,
headers: HeaderMap,
body: Bytes,
) -> StatusCode {
// Look up the repo to get its webhook secret
// Look up the repo in the tenant's database to get its webhook secret
let oid = match mongodb::bson::oid::ObjectId::parse_str(&repo_id) {
Ok(oid) => oid,
Err(_) => return StatusCode::NOT_FOUND,
};
let repo = match agent
.db
let db = match agent.db_pool.for_tenant_id(&tenant_id).await {
Ok(db) => db,
Err(e) => {
tracing::warn!("Gitea webhook: cannot open tenant database '{tenant_id}': {e}");
return StatusCode::NOT_FOUND;
}
};
let repo = match db
.repositories()
.find_one(mongodb::bson::doc! { "_id": oid })
.await
{
Ok(Some(repo)) => repo,
_ => {
tracing::warn!("Gitea webhook: repo {repo_id} not found");
tracing::warn!("Gitea webhook: repo {repo_id} not found in tenant '{tenant_id}'");
return StatusCode::NOT_FOUND;
}
};
@@ -66,15 +72,21 @@ pub async fn handle_gitea_webhook(
"push" => {
let agent_clone = (*agent).clone();
let repo_id = repo_id.clone();
let tenant_id = tenant_id.clone();
tokio::spawn(async move {
tracing::info!("Gitea push webhook: triggering scan for {repo_id}");
if let Err(e) = agent_clone.run_scan(&repo_id, ScanTrigger::Webhook).await {
tracing::info!(
"Gitea push webhook: triggering scan for {repo_id} in tenant {tenant_id}"
);
if let Err(e) = agent_clone
.run_scan(&tenant_id, &repo_id, ScanTrigger::Webhook)
.await
{
tracing::error!("Webhook-triggered scan failed: {e}");
}
});
StatusCode::OK
}
"pull_request" => handle_pull_request(agent, &repo_id, &payload).await,
"pull_request" => handle_pull_request(agent, &tenant_id, &repo_id, &payload).await,
_ => {
tracing::debug!("Gitea webhook: ignoring event '{event}'");
StatusCode::OK
@@ -84,6 +96,7 @@ pub async fn handle_gitea_webhook(
async fn handle_pull_request(
agent: Arc<ComplianceAgent>,
tenant_id: &str,
repo_id: &str,
payload: &serde_json::Value,
) -> StatusCode {
@@ -106,13 +119,14 @@ async fn handle_pull_request(
}
let repo_id = repo_id.to_string();
let tenant_id = tenant_id.to_string();
let head_sha = head_sha.to_string();
let base_sha = base_sha.to_string();
let agent_clone = (*agent).clone();
tokio::spawn(async move {
tracing::info!("Gitea PR webhook: reviewing PR #{pr_number} on {repo_id}");
if let Err(e) = agent_clone
.run_pr_review(&repo_id, pr_number, &base_sha, &head_sha)
.run_pr_review(&tenant_id, &repo_id, pr_number, &base_sha, &head_sha)
.await
{
tracing::error!("PR review failed for #{pr_number}: {e}");
+23 -9
View File
@@ -14,24 +14,30 @@ type HmacSha256 = Hmac<Sha256>;
pub async fn handle_github_webhook(
Extension(agent): Extension<Arc<ComplianceAgent>>,
Path(repo_id): Path<String>,
Path((tenant_id, repo_id)): Path<(String, String)>,
headers: HeaderMap,
body: Bytes,
) -> StatusCode {
// Look up the repo to get its webhook secret
// Look up the repo in the tenant's database to get its webhook secret
let oid = match mongodb::bson::oid::ObjectId::parse_str(&repo_id) {
Ok(oid) => oid,
Err(_) => return StatusCode::NOT_FOUND,
};
let repo = match agent
.db
let db = match agent.db_pool.for_tenant_id(&tenant_id).await {
Ok(db) => db,
Err(e) => {
tracing::warn!("GitHub webhook: cannot open tenant database '{tenant_id}': {e}");
return StatusCode::NOT_FOUND;
}
};
let repo = match db
.repositories()
.find_one(mongodb::bson::doc! { "_id": oid })
.await
{
Ok(Some(repo)) => repo,
_ => {
tracing::warn!("GitHub webhook: repo {repo_id} not found");
tracing::warn!("GitHub webhook: repo {repo_id} not found in tenant '{tenant_id}'");
return StatusCode::NOT_FOUND;
}
};
@@ -66,15 +72,21 @@ pub async fn handle_github_webhook(
"push" => {
let agent_clone = (*agent).clone();
let repo_id = repo_id.clone();
let tenant_id = tenant_id.clone();
tokio::spawn(async move {
tracing::info!("GitHub push webhook: triggering scan for {repo_id}");
if let Err(e) = agent_clone.run_scan(&repo_id, ScanTrigger::Webhook).await {
tracing::info!(
"GitHub push webhook: triggering scan for {repo_id} in tenant {tenant_id}"
);
if let Err(e) = agent_clone
.run_scan(&tenant_id, &repo_id, ScanTrigger::Webhook)
.await
{
tracing::error!("Webhook-triggered scan failed: {e}");
}
});
StatusCode::OK
}
"pull_request" => handle_pull_request(agent, &repo_id, &payload).await,
"pull_request" => handle_pull_request(agent, &tenant_id, &repo_id, &payload).await,
_ => {
tracing::debug!("GitHub webhook: ignoring event '{event}'");
StatusCode::OK
@@ -84,6 +96,7 @@ pub async fn handle_github_webhook(
async fn handle_pull_request(
agent: Arc<ComplianceAgent>,
tenant_id: &str,
repo_id: &str,
payload: &serde_json::Value,
) -> StatusCode {
@@ -105,13 +118,14 @@ async fn handle_pull_request(
}
let repo_id = repo_id.to_string();
let tenant_id = tenant_id.to_string();
let head_sha = head_sha.to_string();
let base_sha = base_sha.to_string();
let agent_clone = (*agent).clone();
tokio::spawn(async move {
tracing::info!("GitHub PR webhook: reviewing PR #{pr_number} on {repo_id}");
if let Err(e) = agent_clone
.run_pr_review(&repo_id, pr_number, &base_sha, &head_sha)
.run_pr_review(&tenant_id, &repo_id, pr_number, &base_sha, &head_sha)
.await
{
tracing::error!("PR review failed for #{pr_number}: {e}");
+23 -9
View File
@@ -10,24 +10,30 @@ use crate::agent::ComplianceAgent;
pub async fn handle_gitlab_webhook(
Extension(agent): Extension<Arc<ComplianceAgent>>,
Path(repo_id): Path<String>,
Path((tenant_id, repo_id)): Path<(String, String)>,
headers: HeaderMap,
body: Bytes,
) -> StatusCode {
// Look up the repo to get its webhook secret
// Look up the repo in the tenant's database to get its webhook secret
let oid = match mongodb::bson::oid::ObjectId::parse_str(&repo_id) {
Ok(oid) => oid,
Err(_) => return StatusCode::NOT_FOUND,
};
let repo = match agent
.db
let db = match agent.db_pool.for_tenant_id(&tenant_id).await {
Ok(db) => db,
Err(e) => {
tracing::warn!("GitLab webhook: cannot open tenant database '{tenant_id}': {e}");
return StatusCode::NOT_FOUND;
}
};
let repo = match db
.repositories()
.find_one(mongodb::bson::doc! { "_id": oid })
.await
{
Ok(Some(repo)) => repo,
_ => {
tracing::warn!("GitLab webhook: repo {repo_id} not found");
tracing::warn!("GitLab webhook: repo {repo_id} not found in tenant '{tenant_id}'");
return StatusCode::NOT_FOUND;
}
};
@@ -59,15 +65,21 @@ pub async fn handle_gitlab_webhook(
"push" => {
let agent_clone = (*agent).clone();
let repo_id = repo_id.clone();
let tenant_id = tenant_id.clone();
tokio::spawn(async move {
tracing::info!("GitLab push webhook: triggering scan for {repo_id}");
if let Err(e) = agent_clone.run_scan(&repo_id, ScanTrigger::Webhook).await {
tracing::info!(
"GitLab push webhook: triggering scan for {repo_id} in tenant {tenant_id}"
);
if let Err(e) = agent_clone
.run_scan(&tenant_id, &repo_id, ScanTrigger::Webhook)
.await
{
tracing::error!("Webhook-triggered scan failed: {e}");
}
});
StatusCode::OK
}
"merge_request" => handle_merge_request(agent, &repo_id, &payload).await,
"merge_request" => handle_merge_request(agent, &tenant_id, &repo_id, &payload).await,
_ => {
tracing::debug!("GitLab webhook: ignoring event '{event_type}'");
StatusCode::OK
@@ -77,6 +89,7 @@ pub async fn handle_gitlab_webhook(
async fn handle_merge_request(
agent: Arc<ComplianceAgent>,
tenant_id: &str,
repo_id: &str,
payload: &serde_json::Value,
) -> StatusCode {
@@ -101,13 +114,14 @@ async fn handle_merge_request(
}
let repo_id = repo_id.to_string();
let tenant_id = tenant_id.to_string();
let head_sha = head_sha.to_string();
let base_sha = base_sha.to_string();
let agent_clone = (*agent).clone();
tokio::spawn(async move {
tracing::info!("GitLab MR webhook: reviewing MR !{mr_iid} on {repo_id}");
if let Err(e) = agent_clone
.run_pr_review(&repo_id, mr_iid, &base_sha, &head_sha)
.run_pr_review(&tenant_id, &repo_id, mr_iid, &base_sha, &head_sha)
.await
{
tracing::error!("MR review failed for !{mr_iid}: {e}");
+8 -4
View File
@@ -9,17 +9,21 @@ use crate::webhooks::{gitea, github, gitlab};
pub async fn start_webhook_server(agent: &ComplianceAgent) -> Result<(), AgentError> {
let app = Router::new()
// Per-repo webhook URLs: /webhook/{platform}/{repo_id}
// Per-tenant per-repo webhook URLs: /webhook/{tenant_id}/{platform}/{repo_id}
// The tenant_id is resolved from the URL path because webhooks
// arrive without a JWT — they're authenticated via per-repo HMAC,
// not via the tenant gate. The dashboard surfaces the full URL
// including the tenant_id when the repo is registered.
.route(
"/webhook/github/{repo_id}",
"/webhook/{tenant_id}/github/{repo_id}",
post(github::handle_github_webhook),
)
.route(
"/webhook/gitlab/{repo_id}",
"/webhook/{tenant_id}/gitlab/{repo_id}",
post(gitlab::handle_gitlab_webhook),
)
.route(
"/webhook/gitea/{repo_id}",
"/webhook/{tenant_id}/gitea/{repo_id}",
post(gitea::handle_gitea_webhook),
)
.layer(Extension(Arc::new(agent.clone())));
+181 -2
View File
@@ -1,3 +1,182 @@
// Shared test helpers for compliance-agent integration tests.
// Shared test harness for E2E / integration tests.
//
// Add database mocks, fixtures, and test utilities here.
// Spins up the agent API server on a random port with an isolated test
// database. Each test gets a fresh database that is dropped on cleanup.
use std::sync::Arc;
use compliance_agent::agent::ComplianceAgent;
use compliance_agent::api;
use compliance_agent::database::DatabasePool;
use compliance_core::AgentConfig;
use secrecy::SecretString;
/// A running test server with a unique database.
pub struct TestServer {
pub base_url: String,
pub client: reqwest::Client,
db_name: String,
mongodb_uri: String,
}
impl TestServer {
/// Start an agent API server on a random port with an isolated database.
pub async fn start() -> Self {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// Unique db-name prefix per run. Must fit the pool's 30-char cap
// (`<prefix>_<32 hex>` <= 63), so use a 16-hex-char suffix.
let db_name = format!("t_{}", &uuid::Uuid::new_v4().simple().to_string()[..16]);
let db_pool = DatabasePool::connect(&mongodb_uri, &db_name)
.await
.expect("Failed to build DatabasePool");
let config = AgentConfig {
mongodb_uri: mongodb_uri.clone(),
mongodb_database: db_name.clone(),
litellm_url: std::env::var("TEST_LITELLM_URL")
.unwrap_or_else(|_| "http://localhost:4000".into()),
litellm_api_key: SecretString::from(String::new()),
litellm_model: "gpt-4o".into(),
litellm_embed_model: "text-embedding-3-small".into(),
agent_port: 0, // not used — we bind ourselves
scan_schedule: String::new(),
cve_monitor_schedule: String::new(),
git_clone_base_path: "/tmp/compliance-scanner-tests/repos".into(),
artifact_store_base_path: "/tmp/compliance-scanner-tests/artifacts".into(),
ssh_key_path: "/tmp/compliance-scanner-tests/ssh/id_ed25519".into(),
github_token: None,
github_webhook_secret: None,
gitlab_url: None,
gitlab_token: None,
gitlab_webhook_secret: None,
jira_url: None,
jira_email: None,
jira_api_token: None,
jira_project_key: None,
searxng_url: None,
nvd_api_key: None,
keycloak_url: None,
keycloak_realm: None,
keycloak_admin_username: None,
keycloak_admin_password: None,
pentest_verification_email: None,
pentest_imap_host: None,
pentest_imap_port: None,
pentest_imap_tls: false,
pentest_imap_username: None,
pentest_imap_password: None,
admin_api_token: None,
tenant_registry_url: None,
unified_pipeline: false,
};
let agent = ComplianceAgent::new(config, db_pool);
// Build the router with the agent extension. After M7.2-B every
// handler takes a TenantCtx extractor; without KC in the test
// harness, the dev-tenant injector mounts a synthetic context so
// tests run end-to-end against `<db_name>_dev`.
let app = api::routes::build_router()
.layer(axum::extract::Extension(Arc::new(agent)))
.layer(axum::middleware::from_fn(api::server::inject_dev_tenant))
.layer(tower_http::cors::CorsLayer::permissive());
// Bind to port 0 to get a random available port
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("Failed to bind test server");
let port = listener.local_addr().expect("no local addr").port();
tokio::spawn(async move {
axum::serve(listener, app).await.ok();
});
let base_url = format!("http://127.0.0.1:{port}");
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.expect("Failed to build HTTP client");
// Wait for server to be ready
for _ in 0..50 {
if client
.get(format!("{base_url}/api/v1/health"))
.send()
.await
.is_ok()
{
break;
}
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
}
Self {
base_url,
client,
db_name,
mongodb_uri,
}
}
/// GET helper
pub async fn get(&self, path: &str) -> reqwest::Response {
self.client
.get(format!("{}{path}", self.base_url))
.send()
.await
.expect("GET request failed")
}
/// POST helper with JSON body
pub async fn post(&self, path: &str, body: &serde_json::Value) -> reqwest::Response {
self.client
.post(format!("{}{path}", self.base_url))
.json(body)
.send()
.await
.expect("POST request failed")
}
/// PATCH helper with JSON body
pub async fn patch(&self, path: &str, body: &serde_json::Value) -> reqwest::Response {
self.client
.patch(format!("{}{path}", self.base_url))
.json(body)
.send()
.await
.expect("PATCH request failed")
}
/// DELETE helper
pub async fn delete(&self, path: &str) -> reqwest::Response {
self.client
.delete(format!("{}{path}", self.base_url))
.send()
.await
.expect("DELETE request failed")
}
/// Get the unique database name for direct MongoDB access in tests.
pub fn db_name(&self) -> &str {
&self.db_name
}
/// Drop every per-tenant database belonging to this test run.
/// Post-M7.2-D the agent never opens a `db_name` directly —
/// data lives only in `<db_name>_<tenant>` per-tenant databases.
pub async fn cleanup(&self) {
if let Ok(client) = mongodb::Client::with_uri_str(&self.mongodb_uri).await {
if let Ok(names) = client.list_database_names().await {
let prefix = format!("{}_", self.db_name);
for name in names {
if name.starts_with(&prefix) {
client.database(&name).drop().await.ok();
}
}
}
}
}
}
+7
View File
@@ -0,0 +1,7 @@
// E2E test entry point.
//
// Run with: cargo test -p compliance-agent --test e2e
// Requires: MongoDB running (set TEST_MONGODB_URI if not default)
mod common;
mod integration;
@@ -0,0 +1,221 @@
use crate::common::TestServer;
use serde_json::json;
/// Insert a DAST target directly into MongoDB linked to a repo.
async fn insert_dast_target(server: &TestServer, repo_id: &str, name: &str) -> String {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
let result = db
.collection::<mongodb::bson::Document>("dast_targets")
.insert_one(mongodb::bson::doc! {
"name": name,
"base_url": format!("https://{name}.example.com"),
"target_type": "webapp",
"repo_id": repo_id,
"rate_limit": 10,
"allow_destructive": false,
"created_at": mongodb::bson::DateTime::now(),
})
.await
.unwrap();
result.inserted_id.as_object_id().unwrap().to_hex()
}
/// Insert a pentest session linked to a target.
async fn insert_pentest_session(server: &TestServer, target_id: &str, repo_id: &str) -> String {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
let result = db
.collection::<mongodb::bson::Document>("pentest_sessions")
.insert_one(mongodb::bson::doc! {
"target_id": target_id,
"repo_id": repo_id,
"strategy": "comprehensive",
"status": "completed",
"findings_count": 1_i32,
"exploitable_count": 0_i32,
"created_at": mongodb::bson::DateTime::now(),
})
.await
.unwrap();
result.inserted_id.as_object_id().unwrap().to_hex()
}
/// Insert an attack chain node linked to a session.
async fn insert_attack_node(server: &TestServer, session_id: &str) {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
db.collection::<mongodb::bson::Document>("attack_chain_nodes")
.insert_one(mongodb::bson::doc! {
"session_id": session_id,
"node_id": "node-1",
"tool_name": "recon",
"status": "completed",
"created_at": mongodb::bson::DateTime::now(),
})
.await
.unwrap();
}
/// Insert a DAST finding linked to a target.
async fn insert_dast_finding(server: &TestServer, target_id: &str, session_id: &str) {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
db.collection::<mongodb::bson::Document>("dast_findings")
.insert_one(mongodb::bson::doc! {
"scan_run_id": "run-1",
"target_id": target_id,
"vuln_type": "xss",
"title": "Reflected XSS",
"description": "XSS in search param",
"severity": "high",
"endpoint": "https://example.com/search",
"method": "GET",
"exploitable": true,
"evidence": [],
"session_id": session_id,
"created_at": mongodb::bson::DateTime::now(),
})
.await
.unwrap();
}
/// Helper to count documents in a collection
async fn count_docs(server: &TestServer, collection: &str) -> u64 {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
db.collection::<mongodb::bson::Document>(collection)
.count_documents(mongodb::bson::doc! {})
.await
.unwrap()
}
#[tokio::test]
async fn delete_repo_cascades_to_dast_and_pentest_data() {
let server = TestServer::start().await;
// Create a repo
let resp = server
.post(
"/api/v1/repositories",
&json!({
"name": "cascade-test",
"git_url": "https://github.com/example/cascade-test.git",
}),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let repo_id = body["data"]["id"].as_str().unwrap().to_string();
// Insert DAST target linked to repo
let target_id = insert_dast_target(&server, &repo_id, "cascade-target").await;
// Insert pentest session linked to target
let session_id = insert_pentest_session(&server, &target_id, &repo_id).await;
// Insert downstream data
insert_attack_node(&server, &session_id).await;
insert_dast_finding(&server, &target_id, &session_id).await;
// Verify data exists
assert_eq!(count_docs(&server, "dast_targets").await, 1);
assert_eq!(count_docs(&server, "pentest_sessions").await, 1);
assert_eq!(count_docs(&server, "attack_chain_nodes").await, 1);
assert_eq!(count_docs(&server, "dast_findings").await, 1);
// Delete the repo
let resp = server
.delete(&format!("/api/v1/repositories/{repo_id}"))
.await;
assert_eq!(resp.status(), 200);
// All downstream data should be gone
assert_eq!(count_docs(&server, "dast_targets").await, 0);
assert_eq!(count_docs(&server, "pentest_sessions").await, 0);
assert_eq!(count_docs(&server, "attack_chain_nodes").await, 0);
assert_eq!(count_docs(&server, "dast_findings").await, 0);
server.cleanup().await;
}
#[tokio::test]
async fn delete_repo_cascades_sast_findings_and_sbom() {
let server = TestServer::start().await;
// Create a repo
let resp = server
.post(
"/api/v1/repositories",
&json!({
"name": "sast-cascade",
"git_url": "https://github.com/example/sast-cascade.git",
}),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let repo_id = body["data"]["id"].as_str().unwrap().to_string();
// Insert SAST finding and SBOM entry
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
let now = mongodb::bson::DateTime::now();
db.collection::<mongodb::bson::Document>("findings")
.insert_one(mongodb::bson::doc! {
"repo_id": &repo_id,
"fingerprint": "fp-test-1",
"scanner": "semgrep",
"scan_type": "sast",
"title": "SQL Injection",
"description": "desc",
"severity": "critical",
"status": "open",
"created_at": now,
"updated_at": now,
})
.await
.unwrap();
db.collection::<mongodb::bson::Document>("sbom_entries")
.insert_one(mongodb::bson::doc! {
"repo_id": &repo_id,
"name": "lodash",
"version": "4.17.20",
"package_manager": "npm",
"known_vulnerabilities": [],
})
.await
.unwrap();
assert_eq!(count_docs(&server, "findings").await, 1);
assert_eq!(count_docs(&server, "sbom_entries").await, 1);
// Delete repo
server
.delete(&format!("/api/v1/repositories/{repo_id}"))
.await;
// Both should be gone
assert_eq!(count_docs(&server, "findings").await, 0);
assert_eq!(count_docs(&server, "sbom_entries").await, 0);
server.cleanup().await;
}
@@ -0,0 +1,48 @@
use crate::common::TestServer;
use serde_json::json;
#[tokio::test]
async fn add_and_list_dast_targets() {
let server = TestServer::start().await;
// Initially empty
let resp = server.get("/api/v1/dast/targets").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
// Add a target
let resp = server
.post(
"/api/v1/dast/targets",
&json!({
"name": "test-app",
"base_url": "https://test-app.example.com",
"target_type": "webapp",
}),
)
.await;
assert_eq!(resp.status(), 200);
// List should return 1
let resp = server.get("/api/v1/dast/targets").await;
let body: serde_json::Value = resp.json().await.unwrap();
let targets = body["data"].as_array().unwrap();
assert_eq!(targets.len(), 1);
assert_eq!(targets[0]["name"], "test-app");
assert_eq!(targets[0]["base_url"], "https://test-app.example.com");
server.cleanup().await;
}
#[tokio::test]
async fn list_dast_findings_empty() {
let server = TestServer::start().await;
let resp = server.get("/api/v1/dast/findings").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
server.cleanup().await;
}
@@ -0,0 +1,144 @@
use crate::common::TestServer;
use serde_json::json;
/// Helper: insert a finding directly via MongoDB for testing query endpoints.
async fn insert_finding(server: &TestServer, repo_id: &str, title: &str, severity: &str) {
// We insert via the agent's DB by posting to the internal test path.
// Since there's no direct "create finding" API, we use MongoDB directly.
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// Extract the database name from the server's unique DB
// We'll use the agent's internal DB through the stats endpoint to verify
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
// Get the DB name from the test server by parsing the health response
// For now, we use a direct insert approach
let db = client.database(&server.db_name());
let now = mongodb::bson::DateTime::now();
db.collection::<mongodb::bson::Document>("findings")
.insert_one(mongodb::bson::doc! {
"repo_id": repo_id,
"fingerprint": format!("fp-{title}-{severity}"),
"scanner": "test-scanner",
"scan_type": "sast",
"title": title,
"description": format!("Test finding: {title}"),
"severity": severity,
"status": "open",
"created_at": now,
"updated_at": now,
})
.await
.unwrap();
}
#[tokio::test]
async fn list_findings_empty() {
let server = TestServer::start().await;
let resp = server.get("/api/v1/findings").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
assert_eq!(body["total"], 0);
server.cleanup().await;
}
#[tokio::test]
async fn list_findings_with_data() {
let server = TestServer::start().await;
insert_finding(&server, "repo1", "SQL Injection", "critical").await;
insert_finding(&server, "repo1", "XSS", "high").await;
insert_finding(&server, "repo2", "Info Leak", "low").await;
let resp = server.get("/api/v1/findings").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["total"], 3);
// Filter by severity
let resp = server.get("/api/v1/findings?severity=critical").await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["total"], 1);
assert_eq!(body["data"][0]["title"], "SQL Injection");
// Filter by repo
let resp = server.get("/api/v1/findings?repo_id=repo1").await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["total"], 2);
server.cleanup().await;
}
#[tokio::test]
async fn update_finding_status() {
let server = TestServer::start().await;
insert_finding(&server, "repo1", "Test Bug", "medium").await;
// Get the finding ID
let resp = server.get("/api/v1/findings").await;
let body: serde_json::Value = resp.json().await.unwrap();
let finding_id = body["data"][0]["_id"]["$oid"].as_str().unwrap();
// Update status to resolved
let resp = server
.patch(
&format!("/api/v1/findings/{finding_id}/status"),
&json!({ "status": "resolved" }),
)
.await;
assert_eq!(resp.status(), 200);
// Verify it's updated
let resp = server.get(&format!("/api/v1/findings/{finding_id}")).await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["status"], "resolved");
server.cleanup().await;
}
#[tokio::test]
async fn bulk_update_finding_status() {
let server = TestServer::start().await;
insert_finding(&server, "repo1", "Bug A", "high").await;
insert_finding(&server, "repo1", "Bug B", "high").await;
// Get both finding IDs
let resp = server.get("/api/v1/findings").await;
let body: serde_json::Value = resp.json().await.unwrap();
let ids: Vec<String> = body["data"]
.as_array()
.unwrap()
.iter()
.map(|f| f["_id"]["$oid"].as_str().unwrap().to_string())
.collect();
// Bulk update
let resp = server
.patch(
"/api/v1/findings/bulk-status",
&json!({
"ids": ids,
"status": "false_positive"
}),
)
.await;
assert_eq!(resp.status(), 200);
// Verify both are updated
for id in &ids {
let resp = server.get(&format!("/api/v1/findings/{id}")).await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["status"], "false_positive");
}
server.cleanup().await;
}
@@ -0,0 +1,29 @@
use crate::common::TestServer;
#[tokio::test]
async fn health_endpoint_returns_ok() {
let server = TestServer::start().await;
let resp = server.get("/api/v1/health").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["status"], "ok");
server.cleanup().await;
}
#[tokio::test]
async fn stats_overview_returns_zeroes_on_empty_db() {
let server = TestServer::start().await;
let resp = server.get("/api/v1/stats/overview").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let data = &body["data"];
assert_eq!(data["repositories"], 0);
assert_eq!(data["total_findings"], 0);
server.cleanup().await;
}
@@ -0,0 +1,7 @@
mod cascade_delete;
mod dast;
mod findings;
mod health;
mod onboarding;
mod repositories;
mod stats;
@@ -0,0 +1,115 @@
use crate::common::TestServer;
use serde_json::json;
#[tokio::test]
async fn create_list_and_applicable_scans() {
let server = TestServer::start().await;
// Initially empty.
let resp = server.get("/api/v1/targets").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
// Create a web-app target with a git repo + a live URL.
let resp = server
.post(
"/api/v1/targets",
&json!({
"name": "acme-web",
"target_type": "web_app",
"artifacts": [
{ "kind": "git_repo", "source_ref": "https://git/acme.git", "branch": "main" },
{ "kind": "live_url", "source_ref": "https://acme.example.com" }
]
}),
)
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let id = body["data"]["_id"]["$oid"].as_str().unwrap().to_string();
assert!(!id.is_empty());
assert_eq!(body["data"]["artifacts"].as_array().unwrap().len(), 2);
// List returns it.
let resp = server.get("/api/v1/targets").await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 1);
// Applicable scans: SAST present + DAST offered (live URL present), pentest supported.
let resp = server
.get(&format!("/api/v1/targets/{id}/applicable-scans"))
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let scans = body["data"]["scans"].as_array().unwrap();
let names: Vec<&str> = scans.iter().filter_map(|s| s["scan"].as_str()).collect();
assert!(names.contains(&"sast"));
assert!(names.contains(&"dast"));
assert_eq!(body["data"]["pentest_supported"], true);
server.cleanup().await;
}
#[tokio::test]
async fn detect_classifies_a_plc_target() {
let server = TestServer::start().await;
// A PLC project artifact is a strong kind-based signal.
let resp = server
.post(
"/api/v1/targets",
&json!({
"name": "line-controller",
"target_type": "backend_service", // deliberately wrong; detect should suggest PLC
"artifacts": [
{ "kind": "plc_project", "source_ref": "line.xml", "plc_format": "plcopen_xml" }
]
}),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let id = body["data"]["_id"]["$oid"].as_str().unwrap().to_string();
let resp = server
.post(&format!("/api/v1/targets/{id}/detect"), &json!({}))
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["classification"]["suggested"], "plc_sps");
server.cleanup().await;
}
#[tokio::test]
async fn add_artifact_and_delete_target() {
let server = TestServer::start().await;
let resp = server
.post(
"/api/v1/targets",
&json!({ "name": "svc", "target_type": "backend_service" }),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let id = body["data"]["_id"]["$oid"].as_str().unwrap().to_string();
// Attach a git repo.
let resp = server
.post(
&format!("/api/v1/targets/{id}/artifacts"),
&json!({ "kind": "git_repo", "source_ref": "https://git/svc.git" }),
)
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["artifacts"].as_array().unwrap().len(), 1);
// Delete it.
let resp = server.delete(&format!("/api/v1/targets/{id}")).await;
assert_eq!(resp.status(), 200);
let resp = server.get(&format!("/api/v1/targets/{id}")).await;
assert_eq!(resp.status(), 404);
server.cleanup().await;
}
@@ -0,0 +1,110 @@
use crate::common::TestServer;
use serde_json::json;
#[tokio::test]
async fn add_and_list_repository() {
let server = TestServer::start().await;
// Initially empty
let resp = server.get("/api/v1/repositories").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
// Add a repository
let resp = server
.post(
"/api/v1/repositories",
&json!({
"name": "test-repo",
"git_url": "https://github.com/example/test-repo.git",
}),
)
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let repo_id = body["data"]["id"].as_str().unwrap().to_string();
assert!(!repo_id.is_empty());
// List should now return 1
let resp = server.get("/api/v1/repositories").await;
let body: serde_json::Value = resp.json().await.unwrap();
let repos = body["data"].as_array().unwrap();
assert_eq!(repos.len(), 1);
assert_eq!(repos[0]["name"], "test-repo");
server.cleanup().await;
}
#[tokio::test]
async fn add_duplicate_repository_fails() {
let server = TestServer::start().await;
let payload = json!({
"name": "dup-repo",
"git_url": "https://github.com/example/dup-repo.git",
});
// First add succeeds
let resp = server.post("/api/v1/repositories", &payload).await;
assert_eq!(resp.status(), 200);
// Second add with same git_url should fail (unique index)
let resp = server.post("/api/v1/repositories", &payload).await;
assert_ne!(resp.status(), 200);
server.cleanup().await;
}
#[tokio::test]
async fn delete_repository() {
let server = TestServer::start().await;
// Add a repo
let resp = server
.post(
"/api/v1/repositories",
&json!({
"name": "to-delete",
"git_url": "https://github.com/example/to-delete.git",
}),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let repo_id = body["data"]["id"].as_str().unwrap();
// Delete it
let resp = server
.delete(&format!("/api/v1/repositories/{repo_id}"))
.await;
assert_eq!(resp.status(), 200);
// List should be empty again
let resp = server.get("/api/v1/repositories").await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
server.cleanup().await;
}
#[tokio::test]
async fn delete_nonexistent_repository_returns_404() {
let server = TestServer::start().await;
let resp = server
.delete("/api/v1/repositories/000000000000000000000000")
.await;
assert_eq!(resp.status(), 404);
server.cleanup().await;
}
#[tokio::test]
async fn delete_invalid_id_returns_400() {
let server = TestServer::start().await;
let resp = server.delete("/api/v1/repositories/not-a-valid-id").await;
assert_eq!(resp.status(), 400);
server.cleanup().await;
}
@@ -0,0 +1,111 @@
use crate::common::TestServer;
use serde_json::json;
#[tokio::test]
async fn stats_overview_reflects_inserted_data() {
let server = TestServer::start().await;
// Add a repo
server
.post(
"/api/v1/repositories",
&json!({
"name": "stats-repo",
"git_url": "https://github.com/example/stats-repo.git",
}),
)
.await;
// Insert findings directly
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
let now = mongodb::bson::DateTime::now();
for (title, severity) in [
("Critical Bug", "critical"),
("High Bug", "high"),
("Medium Bug", "medium"),
("Low Bug", "low"),
] {
db.collection::<mongodb::bson::Document>("findings")
.insert_one(mongodb::bson::doc! {
"repo_id": "test-repo-id",
"fingerprint": format!("fp-{title}"),
"scanner": "test",
"scan_type": "sast",
"title": title,
"description": "desc",
"severity": severity,
"status": "open",
"created_at": now,
"updated_at": now,
})
.await
.unwrap();
}
let resp = server.get("/api/v1/stats/overview").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let data = &body["data"];
assert_eq!(data["repositories"], 1);
assert_eq!(data["total_findings"], 4);
assert_eq!(data["critical"], 1);
assert_eq!(data["high"], 1);
server.cleanup().await;
}
#[tokio::test]
async fn stats_update_after_finding_status_change() {
let server = TestServer::start().await;
// Insert a finding
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let client = mongodb::Client::with_uri_str(&mongodb_uri).await.unwrap();
let db = client.database(&server.db_name());
let now = mongodb::bson::DateTime::now();
let result = db
.collection::<mongodb::bson::Document>("findings")
.insert_one(mongodb::bson::doc! {
"repo_id": "repo-1",
"fingerprint": "fp-stats-test",
"scanner": "test",
"scan_type": "sast",
"title": "Stats Test Finding",
"description": "desc",
"severity": "high",
"status": "open",
"created_at": now,
"updated_at": now,
})
.await
.unwrap();
let finding_id = result.inserted_id.as_object_id().unwrap().to_hex();
// Stats should show 1 finding
let resp = server.get("/api/v1/stats/overview").await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["total_findings"], 1);
// Mark it as resolved
server
.patch(
&format!("/api/v1/findings/{finding_id}/status"),
&json!({ "status": "resolved" }),
)
.await;
// The finding still exists (status changed, not deleted)
let resp = server.get("/api/v1/stats/overview").await;
let body: serde_json::Value = resp.json().await.unwrap();
// total_findings counts all findings regardless of status
assert_eq!(body["data"]["total_findings"], 1);
server.cleanup().await;
}
@@ -0,0 +1,156 @@
// Integration tests for the onboarding backfill migration.
//
// Requires MongoDB (set TEST_MONGODB_URI if not at the default).
// Not run in CI (which is `--lib` only) — run locally:
// cargo test -p compliance-agent --test e2e migration
use compliance_agent::database::{Database, DatabasePool};
use compliance_agent::migrate::onboarding;
use compliance_core::models::{
ArtifactKind, DastTarget, DastTargetType, TargetType, TrackedRepository,
};
use mongodb::bson::{doc, Document};
async fn fresh_db() -> (DatabasePool, String, Database) {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// Prefix must fit the pool's 30-char cap (`<prefix>_<32 hex>` <= 63).
let prefix = format!("t_{}", &uuid::Uuid::new_v4().simple().to_string()[..16]);
let pool = DatabasePool::connect(&uri, &prefix)
.await
.expect("connect mongo");
let db = pool.for_tenant_id("t1").await.expect("tenant db");
(pool, prefix, db)
}
async fn cleanup(pool: &DatabasePool, prefix: &str) {
if let Ok(names) = pool.client().list_database_names().await {
for n in names {
if n.starts_with(prefix) {
pool.client().database(&n).drop().await.ok();
}
}
}
}
#[tokio::test]
async fn backfill_folds_relinks_is_idempotent_and_reversible() {
let (pool, prefix, db) = fresh_db().await;
// Seed a repo.
let repo = TrackedRepository::new("acme".into(), "https://git/acme.git".into());
let repo_id = db
.repositories()
.insert_one(repo)
.await
.expect("insert repo")
.inserted_id
.as_object_id()
.expect("repo oid");
// A DAST target linked to the repo (folds + promotes to WebApp + relinks).
let mut linked = DastTarget::new(
"acme-web".into(),
"https://acme.example.com".into(),
DastTargetType::WebApp,
);
linked.repo_id = Some(repo_id.to_hex());
let linked_id = db
.dast_targets()
.insert_one(linked)
.await
.expect("insert linked dast")
.inserted_id
.as_object_id()
.expect("linked oid");
// A repo-less DAST target (standalone).
let standalone = DastTarget::new(
"acme-api".into(),
"https://api.acme.com".into(),
DastTargetType::RestApi,
);
let standalone_id = db
.dast_targets()
.insert_one(standalone)
.await
.expect("insert standalone dast")
.inserted_id
.as_object_id()
.expect("standalone oid");
// A DAST scan run pointing at the linked target — should be relinked to the repo.
db.collection_named::<Document>("dast_scan_runs")
.insert_one(doc! { "target_id": linked_id.to_hex(), "status": "completed" })
.await
.expect("insert dast run");
// --- Backfill ---
assert!(!onboarding::already_applied(&db).await.unwrap());
let report = onboarding::backfill_onboarded_targets(&db, false)
.await
.expect("backfill");
assert_eq!(report.repos_migrated, 1);
assert_eq!(report.dast_targets_folded, 1);
assert_eq!(report.dast_targets_standalone, 1);
assert!(onboarding::already_applied(&db).await.unwrap());
// Repo target: preserved _id, has git + folded live-url, promoted to WebApp.
let repo_target = db
.onboarded_targets()
.find_one(doc! { "_id": repo_id })
.await
.unwrap()
.expect("repo target");
assert!(repo_target.has(ArtifactKind::GitRepo));
assert!(repo_target.has(ArtifactKind::LiveUrl));
assert_eq!(repo_target.target_type, TargetType::WebApp);
// Standalone target: preserved _id, live-url, backend service.
let standalone_target = db
.onboarded_targets()
.find_one(doc! { "_id": standalone_id })
.await
.unwrap()
.expect("standalone target");
assert!(standalone_target.has(ArtifactKind::LiveUrl));
assert_eq!(standalone_target.target_type, TargetType::BackendService);
// The DAST run was relinked from the old dast id to the repo (unified) id.
let run = db
.collection_named::<Document>("dast_scan_runs")
.find_one(doc! {})
.await
.unwrap()
.expect("run");
assert_eq!(run.get_str("target_id").unwrap(), repo_id.to_hex());
// --- Idempotent: re-run migrates nothing new ---
let again = onboarding::backfill_onboarded_targets(&db, false)
.await
.expect("backfill again");
assert_eq!(again.repos_migrated, 0);
assert_eq!(again.dast_targets_folded, 0);
assert_eq!(again.dast_targets_standalone, 0);
assert!(again.skipped_existing >= 2);
// --- Revert: onboarded targets gone, relink undone, marker cleared ---
onboarding::revert(&db).await.expect("revert");
assert_eq!(
db.onboarded_targets()
.count_documents(doc! {})
.await
.unwrap(),
0
);
let run_after = db
.collection_named::<Document>("dast_scan_runs")
.find_one(doc! {})
.await
.unwrap()
.expect("run");
assert_eq!(run_after.get_str("target_id").unwrap(), linked_id.to_hex());
assert!(!onboarding::already_applied(&db).await.unwrap());
cleanup(&pool, &prefix).await;
}
+9 -3
View File
@@ -1,4 +1,10 @@
// Integration tests for the compliance-agent crate.
// E2E / Integration tests for the compliance-agent API.
//
// Add tests that exercise the full pipeline, API handlers,
// and cross-module interactions here.
// These tests require a running MongoDB instance. Set TEST_MONGODB_URI
// if it's not at the default `mongodb://root:example@localhost:27017`.
//
// Run with: cargo test -p compliance-agent --test e2e
// Or nightly: (via CI with MongoDB service container)
mod api;
mod migration;
+298
View File
@@ -0,0 +1,298 @@
//! M7.2-A — `DatabasePool` isolation proof.
//!
//! Two `TenantContext`s, two databases, one client. Insert on A, query
//! on B → empty. Insert on B, query on A → only A's docs. Proves that
//! the per-tenant database split actually isolates at the driver level
//! and not at "we hope we filter."
//!
//! Requires MongoDB. Set `TEST_MONGODB_URI` to override the default
//! `mongodb://root:example@localhost:27017/?authSource=admin`.
#![allow(clippy::expect_used, clippy::unwrap_used)]
use compliance_agent::database::DatabasePool;
use compliance_core::models::TrackedRepository;
use compliance_core::{OrgRole, TenantContext, TenantStatus};
use mongodb::bson::doc;
fn ctx(tenant_id: &str, slug: &str) -> TenantContext {
TenantContext {
tenant_id: tenant_id.to_string(),
tenant_slug: slug.to_string(),
org_roles: vec![OrgRole::ItAdmin],
products: vec!["compliance-scanner".to_string()],
plan: "starter".to_string(),
status: TenantStatus::Active,
user_id: "u-1".to_string(),
user_name: None,
}
}
fn fixture_repo(name: &str, git_url: &str) -> TrackedRepository {
TrackedRepository {
id: None,
name: name.to_string(),
git_url: git_url.to_string(),
default_branch: "main".to_string(),
local_path: None,
scan_schedule: None,
webhook_enabled: false,
webhook_secret: None,
tracker_type: None,
tracker_owner: None,
tracker_repo: None,
tracker_token: None,
auth_token: None,
auth_username: None,
last_scanned_commit: None,
findings_count: 0,
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
}
}
#[tokio::test]
async fn pool_isolates_tenants_at_driver_level() {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// Unique per run so parallel test invocations don't collide. Kept
// short because Mongo caps db names at 63 bytes (prefix + tenant_id).
let prefix = format!("m72a_{}", short_id());
let pool = DatabasePool::connect(&uri, &prefix)
.await
.expect("Failed to connect to MongoDB — is it running?");
let acme = ctx("00000000-0000-0000-0000-00000000acme", "acme");
let globex = ctx("00000000-0000-0000-0000-0000globex000", "globex");
let acme_db = pool.for_tenant(&acme).await.expect("acme db");
let globex_db = pool.for_tenant(&globex).await.expect("globex db");
// Write distinct repos into each tenant's database.
acme_db
.repositories()
.insert_one(fixture_repo("acme-app", "git@example.com:acme/app.git"))
.await
.expect("insert acme");
globex_db
.repositories()
.insert_one(fixture_repo(
"globex-platform",
"git@example.com:globex/platform.git",
))
.await
.expect("insert globex");
// The point of the whole exercise: acme can ONLY see acme's repo
// and globex can ONLY see globex's, with no filter doc anywhere
// because the isolation is at the database handle, not in the query.
let acme_seen = collect(&acme_db).await;
let globex_seen = collect(&globex_db).await;
assert_eq!(acme_seen.len(), 1, "acme should see exactly its own repo");
assert_eq!(acme_seen[0].name, "acme-app");
assert_eq!(
globex_seen.len(),
1,
"globex should see exactly its own repo"
);
assert_eq!(globex_seen[0].name, "globex-platform");
// Sanity: the two databases really are different by name.
let acme_db_name = pool.tenant_db_name(&acme.tenant_id);
let globex_db_name = pool.tenant_db_name(&globex.tenant_id);
assert_ne!(acme_db_name, globex_db_name);
assert!(acme_db_name.starts_with(&prefix));
// Cleanup — drop both per-tenant databases.
pool.client()
.database(&acme_db_name)
.drop()
.await
.expect("drop acme");
pool.client()
.database(&globex_db_name)
.drop()
.await
.expect("drop globex");
}
#[tokio::test]
async fn for_tenant_is_idempotent_index_creation() {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let prefix = format!("m72a_{}", short_id());
let pool = DatabasePool::connect(&uri, &prefix).await.expect("connect");
let acme = ctx("00000000-0000-0000-0000-00000000acme", "acme");
// Second call must not fail (ensure_indexes already ran, in-memory
// marker is set, Mongo's createIndex is idempotent by name anyway).
let _ = pool.for_tenant(&acme).await.expect("first call");
let _ = pool.for_tenant(&acme).await.expect("second call");
let _ = pool.for_tenant(&acme).await.expect("third call");
// Cleanup
let db_name = pool.tenant_db_name(&acme.tenant_id);
pool.client().database(&db_name).drop().await.expect("drop");
}
#[tokio::test]
async fn tenant_db_name_sanitizes_unsafe_characters() {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let pool = DatabasePool::connect(&uri, "m72a_sanitize")
.await
.expect("connect");
// Mongo db names cannot contain `/ \ . " $ <space> NUL`. The pool
// must rewrite these without exploding on connect.
let funky = "te/n.a\\nt$id\" with spaces";
let name = pool.tenant_db_name(funky);
for c in ['/', '\\', '.', '"', '$', ' '] {
assert!(
!name.contains(c),
"sanitized db name still contains {c:?}: {name}"
);
}
}
#[tokio::test]
async fn admin_helpers_list_and_drop_tenant_dbs() {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let prefix = format!("m72d_{}", short_id());
let pool = DatabasePool::connect(&uri, &prefix).await.expect("connect");
let acme = ctx("00000000-0000-0000-0000-00000000acme", "acme");
let globex = ctx("00000000-0000-0000-0000-0000globex000", "globex");
// Provision two tenants and write a doc into each so the databases
// actually materialize on the cluster (Mongo lazily creates DBs).
let acme_db = pool.for_tenant(&acme).await.expect("acme db");
let globex_db = pool.for_tenant(&globex).await.expect("globex db");
acme_db
.repositories()
.insert_one(fixture_repo("acme-app", "git@example.com:acme/app.git"))
.await
.expect("insert acme");
globex_db
.repositories()
.insert_one(fixture_repo("globex-app", "git@example.com:globex/app.git"))
.await
.expect("insert globex");
// list_tenant_db_names sees both, filtered by prefix
let names = pool.list_tenant_db_names().await.expect("list tenants");
let acme_name = pool.tenant_db_name(&acme.tenant_id);
let globex_name = pool.tenant_db_name(&globex.tenant_id);
assert!(
names.contains(&acme_name),
"expected {acme_name} in {names:?}"
);
assert!(
names.contains(&globex_name),
"expected {globex_name} in {names:?}"
);
for name in &names {
assert!(name.starts_with(&format!("{prefix}_")));
}
// drop_tenant removes acme's DB
pool.drop_tenant(&acme.tenant_id)
.await
.expect("drop acme tenant");
let after = pool
.list_tenant_db_names()
.await
.expect("list tenants after drop");
assert!(
!after.contains(&acme_name),
"acme should be gone after drop, got {after:?}"
);
assert!(
after.contains(&globex_name),
"globex should still be present, got {after:?}"
);
// Cleanup remaining
pool.drop_tenant(&globex.tenant_id)
.await
.expect("drop globex tenant");
}
#[tokio::test]
async fn tenant_db_name_falls_back_to_hash_when_too_long() {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
let pool = DatabasePool::connect(&uri, "m72a_long")
.await
.expect("connect");
// 100-byte tenant_id would overflow the 63-byte db-name cap with
// any reasonable prefix. The pool must hash it down.
let huge = "x".repeat(100);
let name = pool.tenant_db_name(&huge);
assert!(name.len() <= 63, "hashed name should fit: {name}");
assert!(name.starts_with("m72a_long_"));
// The hash suffix is 32 hex chars (16-byte SHA-256 truncation).
let suffix = name.trim_start_matches("m72a_long_");
assert_eq!(
suffix.len(),
32,
"expected 32-hex suffix (16-byte hash), got {suffix:?}"
);
assert!(suffix.chars().all(|c| c.is_ascii_hexdigit()));
// Stable: same input → same output.
assert_eq!(name, pool.tenant_db_name(&huge));
// Different inputs → different outputs (collision check on a tiny
// sample — full birthday-resistance is a proof not a test).
let huge2 = "y".repeat(100);
assert_ne!(pool.tenant_db_name(&huge), pool.tenant_db_name(&huge2));
}
#[tokio::test]
async fn connect_rejects_overlong_db_prefix() {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// MAX_PREFIX_LEN is 30 (= 63 - 1 - 32). A 31-char prefix MUST be
// rejected at construction so the hash-fallback path can never
// produce an over-long db name at runtime.
let too_long = "a".repeat(31);
let err = DatabasePool::connect(&uri, &too_long).await.unwrap_err();
let msg = format!("{err}");
assert!(
msg.contains("max is 30") || msg.contains(&too_long),
"error should explain the cap: {msg}"
);
// Exactly 30 chars is the inclusive bound — must succeed.
let just_right = "a".repeat(30);
let _ = DatabasePool::connect(&uri, &just_right)
.await
.expect("30-char prefix should be accepted");
}
/// Short UUID slug for keeping test prefixes well under Mongo's 63-byte
/// db-name cap.
fn short_id() -> String {
uuid::Uuid::new_v4().simple().to_string()[..8].to_string()
}
/// Drain a `repositories` find cursor on the given tenant database.
async fn collect(db: &compliance_agent::database::Database) -> Vec<TrackedRepository> {
let mut cursor = db
.repositories()
.find(doc! {})
.await
.expect("find repositories");
let mut out = Vec::new();
while cursor.advance().await.expect("advance") {
out.push(cursor.deserialize_current().expect("deserialize"));
}
out
}
@@ -0,0 +1,122 @@
//! M7.1 — integration tests for `compliance_core::auth::require_tenant_status`.
//!
//! Exercises the middleware end-to-end through an Axum router so we
//! catch wiring bugs (extension propagation, method matching) that pure
//! unit tests would miss.
#![allow(clippy::expect_used, clippy::unwrap_used)]
use axum::{
body::Body,
extract::Request,
http::{Method, StatusCode},
middleware::{from_fn, Next},
response::Response,
routing::{get, post},
Router,
};
use compliance_core::{auth::require_tenant_status, TenantContext, TenantStatus};
use tower::ServiceExt;
fn ctx_with(status: TenantStatus) -> TenantContext {
TenantContext {
tenant_id: "t-1".to_string(),
tenant_slug: "acme".to_string(),
org_roles: vec![],
products: vec![],
plan: "starter".to_string(),
status,
user_id: "u-1".to_string(),
user_name: None,
}
}
fn router_with_ctx(ctx: Option<TenantContext>) -> Router {
let injector = move |mut req: Request, next: Next| {
let ctx = ctx.clone();
async move {
if let Some(c) = ctx {
req.extensions_mut().insert(c);
}
next.run(req).await
}
};
Router::new()
.route("/r", get(|| async { "read" }))
.route("/w", post(|| async { "write" }))
.layer(from_fn(require_tenant_status))
.layer(from_fn(injector))
}
async fn call(router: Router, method: Method, path: &str) -> Response {
let req = Request::builder()
.method(method)
.uri(path)
.body(Body::empty())
.expect("request build");
router.oneshot(req).await.expect("oneshot")
}
#[tokio::test]
async fn active_tenant_can_read_and_write() {
let r = router_with_ctx(Some(ctx_with(TenantStatus::Active)));
assert_eq!(
call(r.clone(), Method::GET, "/r").await.status(),
StatusCode::OK
);
assert_eq!(call(r, Method::POST, "/w").await.status(), StatusCode::OK);
}
#[tokio::test]
async fn trial_tenant_can_read_and_write() {
let r = router_with_ctx(Some(ctx_with(TenantStatus::Trial)));
assert_eq!(
call(r.clone(), Method::GET, "/r").await.status(),
StatusCode::OK
);
assert_eq!(call(r, Method::POST, "/w").await.status(), StatusCode::OK);
}
#[tokio::test]
async fn demo_tenant_can_read_and_write() {
let r = router_with_ctx(Some(ctx_with(TenantStatus::Demo)));
assert_eq!(
call(r.clone(), Method::GET, "/r").await.status(),
StatusCode::OK
);
assert_eq!(call(r, Method::POST, "/w").await.status(), StatusCode::OK);
}
#[tokio::test]
async fn frozen_tenant_can_read_but_not_write() {
let r = router_with_ctx(Some(ctx_with(TenantStatus::Frozen)));
assert_eq!(
call(r.clone(), Method::GET, "/r").await.status(),
StatusCode::OK
);
assert_eq!(
call(r, Method::POST, "/w").await.status(),
StatusCode::PAYMENT_REQUIRED
);
}
#[tokio::test]
async fn archived_tenant_is_gone_on_every_method() {
let r = router_with_ctx(Some(ctx_with(TenantStatus::Archived)));
assert_eq!(
call(r.clone(), Method::GET, "/r").await.status(),
StatusCode::GONE
);
assert_eq!(call(r, Method::POST, "/w").await.status(), StatusCode::GONE);
}
#[tokio::test]
async fn no_context_passes_through() {
let r = router_with_ctx(None);
assert_eq!(
call(r.clone(), Method::GET, "/r").await.status(),
StatusCode::OK
);
assert_eq!(call(r, Method::POST, "/w").await.status(), StatusCode::OK);
}
+13
View File
@@ -18,6 +18,15 @@ telemetry = [
"dep:tracing-subscriber",
"dep:tracing",
]
# Pulls in the M7.1 Axum middleware + extractor. Consumers that don't
# embed an HTTP server (e.g. the wasm dashboard frontend) leave it off.
axum = [
"dep:axum",
"dep:jsonwebtoken",
"dep:reqwest",
"dep:tokio",
"dep:tracing",
]
[dependencies]
serde = { workspace = true }
@@ -37,3 +46,7 @@ opentelemetry-appender-tracing = { version = "0.29", optional = true }
tracing-opentelemetry = { version = "0.30", optional = true }
tracing-subscriber = { workspace = true, optional = true }
tracing = { workspace = true, optional = true }
axum = { version = "0.8", optional = true }
jsonwebtoken = { version = "9", optional = true }
reqwest = { workspace = true, optional = true }
tokio = { workspace = true, optional = true }
+398
View File
@@ -0,0 +1,398 @@
//! M7.1 — JWT validation + tenant context propagation.
//!
//! `require_jwt_auth` validates a Bearer JWT against Keycloak's JWKS and
//! attaches a [`TenantContext`] to the request extensions. Downstream
//! middleware ([`require_tenant_status`]) and Axum extractors
//! ([`crate::tenant_ctx::TenantCtx`]) read it from there.
//!
//! Skipped paths:
//! * `/api/v1/health` — Kubernetes liveness; never authenticated.
//!
//! Failure modes:
//! * No `JwksState` extension → pass-through (single-tenant dev mode).
//! * Missing / malformed Bearer header → 401.
//! * Signature / expiry invalid → 401.
//! * Claims present but tenant_id missing → 401 (treated as a malformed
//! token; the realm must always issue tenant_id).
use std::sync::Arc;
use axum::{
extract::Request,
http::Method,
middleware::Next,
response::{IntoResponse, Response},
};
use jsonwebtoken::{decode, decode_header, jwk::JwkSet, DecodingKey, Validation};
use reqwest::StatusCode;
use serde::Deserialize;
use tokio::sync::RwLock;
use crate::{OrgRole, TenantContext, TenantStatus};
/// Cached JWKS from Keycloak for token validation.
#[derive(Clone)]
pub struct JwksState {
pub jwks: Arc<RwLock<Option<JwkSet>>>,
pub jwks_url: String,
}
/// Raw shape of the JWT payload — matches the breakpilot-dev realm's
/// protocol-mapper output. Missing fields default to "" / empty so a
/// realm that hasn't been fully wired yet still validates.
#[derive(Debug, Deserialize)]
struct Claims {
sub: String,
#[serde(default)]
name: Option<String>,
#[serde(default)]
preferred_username: Option<String>,
#[serde(default)]
tenant_id: String,
#[serde(default)]
tenant_slug: String,
#[serde(default)]
org_roles: Vec<String>,
#[serde(default)]
products: Vec<String>,
#[serde(default)]
plan: String,
#[serde(default)]
tenant_status: Option<TenantStatus>,
}
const PUBLIC_ENDPOINTS: &[&str] = &["/api/v1/health"];
/// Path prefixes that bypass JWT validation. The admin sub-router
/// (`/api/v1/admin/*`) has its own static-bearer middleware and must
/// not be routed through the customer-JWT path — a Keycloak token
/// always carries a single tenant_id and would semantically conflict
/// with cross-tenant admin operations.
const PUBLIC_PREFIXES: &[&str] = &["/api/v1/admin/"];
/// Middleware that validates Bearer JWT tokens against Keycloak's JWKS
/// and attaches a `TenantContext` extension on success.
///
/// Skips validation for the health endpoint and any path under one of
/// the [`PUBLIC_PREFIXES`]. If `JwksState` is not present (Keycloak
/// not configured), requests pass through and downstream code must
/// handle the missing context.
pub async fn require_jwt_auth(mut request: Request, next: Next) -> Response {
let path = request.uri().path();
if PUBLIC_ENDPOINTS.contains(&path) || PUBLIC_PREFIXES.iter().any(|p| path.starts_with(p)) {
return next.run(request).await;
}
let jwks_state = match request.extensions().get::<JwksState>() {
Some(s) => s.clone(),
None => return next.run(request).await,
};
let auth_header = match request.headers().get("authorization") {
Some(h) => h,
None => return (StatusCode::UNAUTHORIZED, "Missing authorization header").into_response(),
};
let token = match auth_header.to_str() {
Ok(s) if s.starts_with("Bearer ") => &s[7..],
_ => return (StatusCode::UNAUTHORIZED, "Invalid authorization header").into_response(),
};
match validate_token(token, &jwks_state).await {
Ok(ctx) => {
request.extensions_mut().insert(ctx);
next.run(request).await
}
Err(e) => {
tracing::warn!("JWT validation failed: {e}");
(StatusCode::UNAUTHORIZED, "Invalid token").into_response()
}
}
}
/// Middleware that enforces the M7.1 `tenant_status` contract.
///
/// * `Active` / `Trial` / `Demo` — pass through.
/// * `Frozen` — read-only after cancel / non-payment. Writes return 402.
/// * `Archived` — data-retention window closed. Every request returns 410.
///
/// Pass-through when no `TenantContext` is present (single-tenant dev or
/// the upstream JWT middleware ran without `JwksState`).
pub async fn require_tenant_status(request: Request, next: Next) -> Response {
let ctx = match request.extensions().get::<TenantContext>() {
Some(c) => c.clone(),
None => return next.run(request).await,
};
if ctx.status.is_archived() {
return (
StatusCode::GONE,
"Tenant archived — data retention window closed",
)
.into_response();
}
if ctx.status.is_frozen() && is_write(request.method()) {
return (
StatusCode::PAYMENT_REQUIRED,
"Tenant frozen — read-only. Re-activate to resume writes.",
)
.into_response();
}
next.run(request).await
}
/// Treat anything other than GET/HEAD/OPTIONS as a write. Good enough for
/// REST. The few exceptions (e.g. read-side POSTs) can opt out at the
/// handler level once we have them.
fn is_write(m: &Method) -> bool {
!matches!(m, &Method::GET | &Method::HEAD | &Method::OPTIONS)
}
async fn validate_token(token: &str, state: &JwksState) -> Result<TenantContext, String> {
let header = decode_header(token).map_err(|e| format!("failed to decode JWT header: {e}"))?;
let kid = header
.kid
.clone()
.ok_or_else(|| "JWT missing kid header".to_string())?;
// First try against whatever's currently cached. If the kid isn't
// there or the signature doesn't verify, the cached JWKS is most
// likely stale (KC rotated keys) — refresh once and retry before
// giving up. Without this every key rotation produces a silent 401
// storm that only goes away when the agent restarts.
let jwks = fetch_or_get_jwks(state, false).await?;
match try_validate(token, &header, &kid, &jwks) {
Ok(ctx) => Ok(ctx),
Err(ValidationError::Permanent(e)) => Err(e),
Err(ValidationError::Stale(reason)) => {
tracing::info!(
kid = %kid,
reason = %reason,
"JWKS appears stale — forcing refresh and retrying"
);
let jwks = fetch_or_get_jwks(state, true).await?;
try_validate(token, &header, &kid, &jwks).map_err(|e| match e {
ValidationError::Stale(s) | ValidationError::Permanent(s) => s,
})
}
}
}
#[derive(Debug)]
enum ValidationError {
/// Refresh-eligible: cached JWKS may be stale.
Stale(String),
/// Refusing the token regardless of JWKS freshness.
Permanent(String),
}
fn try_validate(
token: &str,
header: &jsonwebtoken::Header,
kid: &str,
jwks: &JwkSet,
) -> Result<TenantContext, ValidationError> {
let jwk = match jwks
.keys
.iter()
.find(|k| k.common.key_id.as_deref() == Some(kid))
{
Some(j) => j,
None => {
return Err(ValidationError::Stale(
"no matching key found in JWKS".to_string(),
))
}
};
let decoding_key = DecodingKey::from_jwk(jwk)
.map_err(|e| ValidationError::Permanent(format!("failed to create decoding key: {e}")))?;
let mut validation = Validation::new(header.alg);
validation.validate_exp = true;
validation.validate_aud = false;
let data = match decode::<Claims>(token, &decoding_key, &validation) {
Ok(d) => d,
Err(e) => {
// Signature mismatch is the other refresh-eligible failure:
// the matching kid is present but the key bytes don't match.
// Everything else (expired, malformed, etc.) is permanent.
return Err(
if matches!(e.kind(), jsonwebtoken::errors::ErrorKind::InvalidSignature) {
ValidationError::Stale(format!("token validation failed: {e}"))
} else {
ValidationError::Permanent(format!("token validation failed: {e}"))
},
);
}
};
claims_to_context(data.claims).map_err(ValidationError::Permanent)
}
/// Map the decoded JWT payload into the platform-wide `TenantContext`.
/// Pulled out for unit testing — no I/O.
fn claims_to_context(c: Claims) -> Result<TenantContext, String> {
if c.tenant_id.is_empty() {
return Err("JWT is missing tenant_id claim".to_string());
}
let status = c.tenant_status.unwrap_or_else(|| {
tracing::warn!(
"JWT missing tenant_status claim for tenant {} — defaulting to Trial",
c.tenant_id
);
TenantStatus::Trial
});
Ok(TenantContext {
tenant_id: c.tenant_id,
tenant_slug: c.tenant_slug,
org_roles: c.org_roles.iter().map(|r| OrgRole::parse(r)).collect(),
products: c.products,
plan: c.plan,
status,
user_id: c.sub,
user_name: c.name.or(c.preferred_username),
})
}
async fn fetch_or_get_jwks(state: &JwksState, force: bool) -> Result<JwkSet, String> {
if !force {
let cached = state.jwks.read().await;
if let Some(ref jwks) = *cached {
return Ok(jwks.clone());
}
}
// Hold the write lock across the fetch so concurrent refreshers
// don't all hammer Keycloak when keys rotate. If another writer
// already populated a fresh JWKS while we were waiting (and we
// weren't asked to force), use theirs.
let mut cached = state.jwks.write().await;
if !force {
if let Some(ref jwks) = *cached {
return Ok(jwks.clone());
}
}
let resp = reqwest::get(&state.jwks_url)
.await
.map_err(|e| format!("failed to fetch JWKS: {e}"))?;
let jwks: JwkSet = resp
.json()
.await
.map_err(|e| format!("failed to parse JWKS: {e}"))?;
*cached = Some(jwks.clone());
Ok(jwks)
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
fn base_claims() -> Claims {
Claims {
sub: "user-123".to_string(),
name: Some("Alice Acme".to_string()),
preferred_username: None,
tenant_id: "00000000-0000-0000-0000-000000000001".to_string(),
tenant_slug: "acme".to_string(),
org_roles: vec!["IT_ADMIN".to_string()],
products: vec!["compliance".to_string()],
plan: "professional".to_string(),
tenant_status: Some(TenantStatus::Active),
}
}
#[test]
fn claims_to_context_happy_path() {
let ctx = claims_to_context(base_claims()).expect("should map");
assert_eq!(ctx.tenant_id, "00000000-0000-0000-0000-000000000001");
assert_eq!(ctx.tenant_slug, "acme");
assert_eq!(ctx.org_roles, vec![OrgRole::ItAdmin]);
assert_eq!(ctx.products, vec!["compliance"]);
assert_eq!(ctx.plan, "professional");
assert_eq!(ctx.status, TenantStatus::Active);
assert_eq!(ctx.user_id, "user-123");
assert_eq!(ctx.user_name.as_deref(), Some("Alice Acme"));
}
#[test]
fn claims_to_context_rejects_missing_tenant_id() {
let mut c = base_claims();
c.tenant_id = "".to_string();
let err = claims_to_context(c).expect_err("should reject");
assert!(err.contains("tenant_id"));
}
#[test]
fn claims_to_context_defaults_status_when_missing() {
let mut c = base_claims();
c.tenant_status = None;
let ctx = claims_to_context(c).expect("should map");
assert_eq!(ctx.status, TenantStatus::Trial);
}
#[test]
fn claims_to_context_falls_back_to_preferred_username() {
let mut c = base_claims();
c.name = None;
c.preferred_username = Some("alice@acme.dev".to_string());
let ctx = claims_to_context(c).expect("should map");
assert_eq!(ctx.user_name.as_deref(), Some("alice@acme.dev"));
}
#[test]
fn claims_to_context_parses_multiple_roles() {
let mut c = base_claims();
c.org_roles = vec![
"IT_ADMIN".to_string(),
"CXO".to_string(),
"GARBAGE".to_string(),
];
let ctx = claims_to_context(c).expect("should map");
assert_eq!(
ctx.org_roles,
vec![OrgRole::ItAdmin, OrgRole::Cxo, OrgRole::Unknown]
);
}
#[test]
fn try_validate_returns_stale_when_kid_missing_from_jwks() {
// Empty JWKS — the kid we ask for can't possibly match. The error
// must classify as Stale so the caller refreshes JWKS and retries.
let jwks = JwkSet { keys: vec![] };
let header = jsonwebtoken::Header {
alg: jsonwebtoken::Algorithm::RS256,
kid: Some("kid-rotated-out".to_string()),
..Default::default()
};
let err = try_validate("ignored.token.value", &header, "kid-rotated-out", &jwks)
.expect_err("should fail");
match err {
ValidationError::Stale(s) => assert!(s.contains("no matching key")),
ValidationError::Permanent(s) => panic!("must be Stale, got Permanent: {s}"),
}
}
#[test]
fn is_write_detects_methods() {
assert!(!is_write(&Method::GET));
assert!(!is_write(&Method::HEAD));
assert!(!is_write(&Method::OPTIONS));
assert!(is_write(&Method::POST));
assert!(is_write(&Method::PUT));
assert!(is_write(&Method::PATCH));
assert!(is_write(&Method::DELETE));
}
}
+19
View File
@@ -24,6 +24,9 @@ pub struct AgentConfig {
pub scan_schedule: String,
pub cve_monitor_schedule: String,
pub git_clone_base_path: String,
/// Base directory for content-addressed artifact blobs and per-run working
/// dirs (`<base>/blobs/<sha[0:2]>/<sha>`, `<base>/work/<target>/<artifact>/`).
pub artifact_store_base_path: String,
pub ssh_key_path: String,
pub keycloak_url: Option<String>,
pub keycloak_realm: Option<String>,
@@ -33,8 +36,24 @@ pub struct AgentConfig {
pub pentest_verification_email: Option<String>,
pub pentest_imap_host: Option<String>,
pub pentest_imap_port: Option<u16>,
/// Use implicit TLS (IMAPS, port 993) instead of plain IMAP.
pub pentest_imap_tls: bool,
pub pentest_imap_username: Option<String>,
pub pentest_imap_password: Option<SecretString>,
/// Static bearer for the cross-tenant admin endpoints under
/// `/api/v1/admin/*`. When `None`, those endpoints are not
/// mounted at all (defense-in-depth: ops endpoints never reach
/// any auth path if no operator has explicitly opted in).
pub admin_api_token: Option<SecretString>,
/// Live tenant-registry URL the scheduler consults for the list
/// of tenants to iterate. When `None` or unreachable, scheduler
/// falls back to `SCHEDULER_TENANT_IDS` env (M7.2-C).
pub tenant_registry_url: Option<String>,
/// When true, `run_scan` dispatches to the unified `run_target` pipeline
/// (reads `onboarded_targets`) instead of the legacy repository pipeline.
/// Env `UNIFIED_PIPELINE`. Defaults on; set `UNIFIED_PIPELINE=0` to use the
/// legacy repository pipeline.
pub unified_pipeline: bool,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
+75
View File
@@ -0,0 +1,75 @@
//! Database helpers shared across the workspace.
//!
//! `tenant_filter` returns the BSON filter that every query and update
//! against a tenant-scoped collection MUST include. Centralising it here
//! makes the rule grep-able and keeps query call-sites from accidentally
//! omitting it.
//!
//! Future work (M7.2+): each collection model grows a `tenant_id` field
//! and every `find` / `update_*` / `delete_*` call gets this filter
//! merged in. The migration to per-collection scoping is tracked
//! separately — this helper is the building block.
use bson::{doc, Document};
use crate::TenantContext;
/// Returns `{ "tenant_id": <ctx.tenant_id> }`. Merge this into every
/// query filter against a tenant-scoped collection.
///
/// Use [`tenant_filter_merge`] when you need to combine it with other
/// query conditions — it preserves both halves without overwriting.
pub fn tenant_filter(ctx: &TenantContext) -> Document {
doc! { "tenant_id": &ctx.tenant_id }
}
/// Returns the tenant filter merged with caller-supplied conditions.
/// The tenant_id always wins on key conflict — callers cannot
/// accidentally override the scoping.
pub fn tenant_filter_merge(ctx: &TenantContext, mut extra: Document) -> Document {
extra.insert("tenant_id", &ctx.tenant_id);
extra
}
#[cfg(test)]
mod tests {
use super::*;
use crate::TenantStatus;
fn ctx() -> TenantContext {
TenantContext {
tenant_id: "t-abc".to_string(),
tenant_slug: "acme".to_string(),
org_roles: vec![],
products: vec![],
plan: "starter".to_string(),
status: TenantStatus::Active,
user_id: "u-1".to_string(),
user_name: None,
}
}
#[test]
fn produces_tenant_id_filter() {
let f = tenant_filter(&ctx());
assert_eq!(f.get_str("tenant_id"), Ok("t-abc"));
assert_eq!(f.len(), 1);
}
#[test]
fn merge_preserves_extra_conditions() {
let extra = doc! { "status": "open", "severity": "high" };
let f = tenant_filter_merge(&ctx(), extra);
assert_eq!(f.get_str("tenant_id"), Ok("t-abc"));
assert_eq!(f.get_str("status"), Ok("open"));
assert_eq!(f.get_str("severity"), Ok("high"));
}
#[test]
fn merge_overrides_caller_tenant_id() {
let extra = doc! { "tenant_id": "evil-other", "status": "open" };
let f = tenant_filter_merge(&ctx(), extra);
assert_eq!(f.get_str("tenant_id"), Ok("t-abc"));
assert_eq!(f.get_str("status"), Ok("open"));
}
}
+9
View File
@@ -1,9 +1,18 @@
pub mod config;
pub mod db;
pub mod error;
pub mod models;
pub mod scan_matrix;
#[cfg(feature = "telemetry")]
pub mod telemetry;
pub mod tenant;
pub mod traits;
#[cfg(feature = "axum")]
pub mod auth;
#[cfg(feature = "axum")]
pub mod tenant_ctx;
pub use config::{AgentConfig, DashboardConfig};
pub use error::CoreError;
pub use tenant::{OrgRole, TenantContext, TenantStatus};
+69
View File
@@ -0,0 +1,69 @@
//! Per-tenant API tokens used by `compliance-mcp` to authenticate MCP
//! HTTP requests on behalf of LLM clients (Claude Desktop, Cursor,
//! ChatGPT, etc.) that can't run a Keycloak OIDC flow.
//!
//! Tokens are opaque strings of the form `mcpt_<44 url-safe random
//! chars>`. The raw value is shown to the user exactly once at
//! creation; the database only ever sees the SHA-256 hash. Lookups go
//! through the cross-tenant `<prefix>__admin.mcp_tokens` collection
//! and return the `tenant_id` the MCP server should route to.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
/// Persisted token metadata. `token_hash` is the SHA-256 hex of the
/// raw token; the raw token itself is never stored.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct McpToken {
#[serde(rename = "_id", skip_serializing_if = "Option::is_none")]
pub id: Option<bson::oid::ObjectId>,
/// SHA-256 hex of the raw token. Unique index in the collection.
pub token_hash: String,
/// First 8 chars of the raw token — purely for UI display so users
/// can identify which token is which without re-issuing.
pub token_prefix: String,
/// Routes to `<db_prefix>_<tenant_id>` on MCP requests.
pub tenant_id: String,
/// User-given label, e.g. "Claude Desktop" or "Sharang's laptop".
pub name: String,
/// Keycloak `sub` of the user who created this token, for audit.
pub created_by: String,
#[serde(with = "super::serde_helpers::bson_datetime")]
pub created_at: DateTime<Utc>,
#[serde(default, with = "super::serde_helpers::opt_bson_datetime")]
pub last_used_at: Option<DateTime<Utc>>,
/// Soft-delete flag. A revoked token doc stays around for audit
/// but never authenticates.
#[serde(default)]
pub revoked: bool,
}
/// Public projection of a token — never includes the hash.
/// Returned by `GET /api/v1/mcp-tokens`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct McpTokenView {
pub id: String,
pub name: String,
/// `mcpt_xxxx…` so the user can identify which row is which.
pub token_prefix: String,
pub created_by: String,
#[serde(with = "super::serde_helpers::bson_datetime")]
pub created_at: DateTime<Utc>,
#[serde(default, with = "super::serde_helpers::opt_bson_datetime")]
pub last_used_at: Option<DateTime<Utc>>,
pub revoked: bool,
}
impl From<&McpToken> for McpTokenView {
fn from(t: &McpToken) -> Self {
Self {
id: t.id.map(|o| o.to_hex()).unwrap_or_default(),
name: t.name.clone(),
token_prefix: t.token_prefix.clone(),
created_by: t.created_by.clone(),
created_at: t.created_at,
last_used_at: t.last_used_at,
revoked: t.revoked,
}
}
}
+11
View File
@@ -7,6 +7,9 @@ pub mod finding;
pub mod graph;
pub mod issue;
pub mod mcp;
pub mod mcp_token;
pub mod notification;
pub mod onboarding;
pub mod pentest;
pub mod repository;
pub mod sbom;
@@ -27,6 +30,14 @@ pub use graph::{
};
pub use issue::{IssueStatus, TrackerIssue, TrackerType};
pub use mcp::{McpServerConfig, McpServerStatus, McpTransport};
pub use mcp_token::{McpToken, McpTokenView};
pub use notification::{CveNotification, NotificationSeverity, NotificationStatus};
pub use onboarding::{
default_compliance_profile, Artifact, ArtifactAuth, ArtifactKind, Classification,
ComplianceFramework, ComplianceProfile, DetectedFact, ExternalRef, ExternalSystem,
GitArtifactConfig, IssueTrackerConfig, OnboardedTarget, PlcArtifactConfig, PlcFormat,
TargetScanConfig, TargetType, TargetTypeCandidate, WebArtifactConfig,
};
pub use pentest::{
AttackChainNode, AttackNodeStatus, AuthMode, CodeContextHint, Environment, IdentityProvider,
PentestAuthConfig, PentestConfig, PentestEvent, PentestMessage, PentestSession, PentestStats,
+103
View File
@@ -0,0 +1,103 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
/// Status of a CVE notification
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum NotificationStatus {
/// Newly created, not yet seen by the user
New,
/// User has seen it (e.g., opened the notification panel)
Read,
/// User has explicitly acknowledged/dismissed it
Dismissed,
}
/// Severity level for notification filtering
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)]
#[serde(rename_all = "lowercase")]
pub enum NotificationSeverity {
Low,
Medium,
High,
Critical,
}
/// A notification about a newly discovered CVE affecting a tracked dependency.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CveNotification {
#[serde(rename = "_id", skip_serializing_if = "Option::is_none")]
pub id: Option<bson::oid::ObjectId>,
/// The CVE/GHSA identifier
pub cve_id: String,
/// Repository where the vulnerable dependency is used
pub repo_id: String,
/// Repository name (denormalized for display)
pub repo_name: String,
/// Affected package name
pub package_name: String,
/// Affected version
pub package_version: String,
/// Human-readable severity
pub severity: NotificationSeverity,
/// CVSS score if available
pub cvss_score: Option<f64>,
/// Short summary of the vulnerability
pub summary: Option<String>,
/// Link to vulnerability details
pub url: Option<String>,
/// Notification lifecycle status
pub status: NotificationStatus,
/// When the CVE was first detected for this dependency
#[serde(with = "super::serde_helpers::bson_datetime")]
pub created_at: DateTime<Utc>,
/// When the user last interacted with this notification
pub read_at: Option<DateTime<Utc>>,
}
impl CveNotification {
pub fn new(
cve_id: String,
repo_id: String,
repo_name: String,
package_name: String,
package_version: String,
severity: NotificationSeverity,
) -> Self {
Self {
id: None,
cve_id,
repo_id,
repo_name,
package_name,
package_version,
severity,
cvss_score: None,
summary: None,
url: None,
status: NotificationStatus::New,
created_at: Utc::now(),
read_at: None,
}
}
}
/// Map an OSV/NVD severity string to our notification severity
pub fn parse_severity(s: Option<&str>, cvss: Option<f64>) -> NotificationSeverity {
// Prefer CVSS score if available
if let Some(score) = cvss {
return match score {
s if s >= 9.0 => NotificationSeverity::Critical,
s if s >= 7.0 => NotificationSeverity::High,
s if s >= 4.0 => NotificationSeverity::Medium,
_ => NotificationSeverity::Low,
};
}
// Fall back to string severity
match s.map(|s| s.to_uppercase()).as_deref() {
Some("CRITICAL") => NotificationSeverity::Critical,
Some("HIGH") => NotificationSeverity::High,
Some("MODERATE" | "MEDIUM") => NotificationSeverity::Medium,
_ => NotificationSeverity::Low,
}
}
+753
View File
@@ -0,0 +1,753 @@
//! The unified onboarding model.
//!
//! An [`OnboardedTarget`] is the single source of truth for anything the scanner
//! can analyze. It records *what kind of software* the target is ([`TargetType`]),
//! the concrete [`Artifact`]s that were provided for it (a git repo, a firmware
//! image, a live URL, a PLC project, ...), the classifier's verdict, and the scan
//! configuration. It replaces the older git-only `TrackedRepository` and the
//! standalone `DastTarget`, both of which fold into this type as artifacts.
use std::collections::HashMap;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use super::dast::{DastAuthConfig, DastTargetType};
use super::issue::TrackerType;
use super::pentest::{Environment, PentestConfig, PentestStrategy};
use super::scan::ScanType;
/// The family of software a target belongs to.
///
/// Targets look endlessly varied but fall into a small enumerable set classified
/// by where the analyzable signal lives. This drives the scan-applicability
/// matrix and the onboarding wizard's type selection.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum TargetType {
/// Browser-facing web application (front end + server).
WebApp,
/// Headless backend service / API (REST, GraphQL, gRPC).
BackendService,
/// Desktop application (Windows/macOS/Linux GUI or CLI binary).
DesktopApp,
/// Android application (APK / AAB).
AndroidApp,
/// iOS application (IPA).
IosApp,
/// Bare-metal embedded firmware (no operating system).
FirmwareBareMetal,
/// Embedded firmware running on an RTOS (Zephyr, FreeRTOS, ...).
FirmwareRtos,
/// Embedded Linux built with Yocto / OpenEmbedded (BSP + image).
EmbeddedLinuxYocto,
/// Programmable logic controller software (IEC 61131-3, PLCopen / SPS).
PlcSps,
}
impl std::fmt::Display for TargetType {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::WebApp => write!(f, "web_app"),
Self::BackendService => write!(f, "backend_service"),
Self::DesktopApp => write!(f, "desktop_app"),
Self::AndroidApp => write!(f, "android_app"),
Self::IosApp => write!(f, "ios_app"),
Self::FirmwareBareMetal => write!(f, "firmware_bare_metal"),
Self::FirmwareRtos => write!(f, "firmware_rtos"),
Self::EmbeddedLinuxYocto => write!(f, "embedded_linux_yocto"),
Self::PlcSps => write!(f, "plc_sps"),
}
}
}
/// The kind of artifact provided for a target.
///
/// Which scans are possible is a function of the target type *and* which of
/// these are present (SAST needs code, DAST needs a running URL, and so on).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ArtifactKind {
/// A git repository (cloned for static analysis).
GitRepo,
/// A source archive (zip / tarball) with no live git remote.
SourceArchive,
/// A firmware image or binary blob.
FirmwareImage,
/// A mobile package: Android APK/AAB or iOS IPA.
MobilePackage,
/// An OCI/Docker container image reference.
ContainerImage,
/// A reachable running instance (base URL / endpoint) for dynamic testing.
LiveUrl,
/// A PLC project: PLCopen XML or Structured Text source.
PlcProject,
/// Free-form plaintext describing the target (feeds classification only).
PlaintextDescription,
}
impl std::fmt::Display for ArtifactKind {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::GitRepo => write!(f, "git_repo"),
Self::SourceArchive => write!(f, "source_archive"),
Self::FirmwareImage => write!(f, "firmware_image"),
Self::MobilePackage => write!(f, "mobile_package"),
Self::ContainerImage => write!(f, "container_image"),
Self::LiveUrl => write!(f, "live_url"),
Self::PlcProject => write!(f, "plc_project"),
Self::PlaintextDescription => write!(f, "plaintext_description"),
}
}
}
/// Credentials attached to an artifact.
///
/// This folds both `TrackedRepository`'s git auth (`auth_token` / `auth_username`
/// / SSH key) and `DastAuthConfig`'s HTTP auth (form / bearer / cookie) into one
/// shape so a single artifact carries whatever it needs to be fetched or probed.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct ArtifactAuth {
/// Auth method: `none` | `token` | `basic` | `bearer` | `cookie` | `form` | `ssh`.
#[serde(default)]
pub method: String,
/// Username (git user, basic-auth user, or `x-access-token` for PATs).
pub username: Option<String>,
/// The secret credential: PAT, password, or bearer token. Encrypted at rest.
pub secret: Option<String>,
/// Path to an SSH private key for git-over-SSH.
pub ssh_key_path: Option<String>,
/// Login URL for form-based authentication.
pub login_url: Option<String>,
/// Extra headers to send when authenticating / probing.
pub headers: Option<HashMap<String, String>>,
}
impl From<DastAuthConfig> for ArtifactAuth {
fn from(c: DastAuthConfig) -> Self {
Self {
method: c.method,
username: c.username,
// Prefer a bearer token; otherwise fall back to the password.
secret: c.token.or(c.password),
ssh_key_path: None,
login_url: c.login_url,
headers: c.headers,
}
}
}
/// Git-specific configuration for a [`ArtifactKind::GitRepo`] artifact.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GitArtifactConfig {
/// Branch to scan.
pub default_branch: String,
/// Commit SHA of the last completed scan (change-detection watermark).
pub last_scanned_commit: Option<String>,
/// Local clone path once the repo has been fetched.
pub local_path: Option<String>,
}
impl GitArtifactConfig {
/// Config for a fresh git artifact on the given branch.
pub fn on_branch(branch: impl Into<String>) -> Self {
Self {
default_branch: branch.into(),
last_scanned_commit: None,
local_path: None,
}
}
}
impl Default for GitArtifactConfig {
fn default() -> Self {
Self::on_branch("main")
}
}
/// Dynamic-analysis configuration for a [`ArtifactKind::LiveUrl`] artifact.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct WebArtifactConfig {
/// Whether the endpoint is a web app, REST API, or GraphQL API.
pub target_kind: DastTargetType,
/// URL paths to exclude from crawling / scanning.
#[serde(default)]
pub excluded_paths: Vec<String>,
/// Maximum crawl depth.
pub max_crawl_depth: u32,
/// Rate limit in requests per second.
pub rate_limit: u32,
/// Whether destructive methods (DELETE / PUT) are permitted.
#[serde(default)]
pub allow_destructive: bool,
}
impl Default for WebArtifactConfig {
fn default() -> Self {
Self {
target_kind: DastTargetType::WebApp,
excluded_paths: Vec::new(),
max_crawl_depth: 3,
rate_limit: 10,
allow_destructive: false,
}
}
}
/// The source format of a PLC project artifact.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum PlcFormat {
/// PLCopen XML project export.
PlcopenXml,
/// IEC 61131-3 Structured Text source.
StructuredText,
}
/// PLC-specific configuration for a [`ArtifactKind::PlcProject`] artifact.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PlcArtifactConfig {
/// The project source format.
pub format: PlcFormat,
}
/// A single fact discovered about a target by ingest or classification
/// (e.g. `language=rust`, `build_system=cmake`, `mcu=stm32f429`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DetectedFact {
/// The fact name.
pub key: String,
/// The fact value.
pub value: String,
/// What produced the fact (e.g. `tramiton`, `language-fingerprint`).
pub source: String,
}
impl DetectedFact {
/// Build a fact from its parts.
pub fn new(
key: impl Into<String>,
value: impl Into<String>,
source: impl Into<String>,
) -> Self {
Self {
key: key.into(),
value: value.into(),
source: source.into(),
}
}
}
/// One concrete thing provided for a target: code, a binary, a URL, etc.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Artifact {
/// Stable per-artifact id (UUID v4) — scan steps reference this.
pub id: String,
/// What kind of artifact this is.
pub kind: ArtifactKind,
/// The source reference: git URL, blob id, live URL, or image ref.
pub source_ref: String,
/// Optional human-friendly label.
pub display_name: Option<String>,
/// Content-addressed storage path once ingested (blobs only).
pub stored_path: Option<String>,
/// SHA-256 of the ingested content (git artifacts store the head SHA).
pub content_hash: Option<String>,
/// Size of the stored blob in bytes.
pub size_bytes: Option<u64>,
/// Credentials for fetching or probing this artifact.
pub auth: Option<ArtifactAuth>,
/// Git configuration (present for [`ArtifactKind::GitRepo`]).
pub git: Option<GitArtifactConfig>,
/// Dynamic-analysis configuration (present for [`ArtifactKind::LiveUrl`]).
pub web: Option<WebArtifactConfig>,
/// PLC configuration (present for [`ArtifactKind::PlcProject`]).
pub plc: Option<PlcArtifactConfig>,
/// Facts discovered about this artifact by ingest / classification.
#[serde(default)]
pub detected: Vec<DetectedFact>,
/// When this artifact was last ingested.
#[serde(default, with = "super::serde_helpers::opt_bson_datetime")]
pub ingested_at: Option<DateTime<Utc>>,
}
impl Artifact {
/// A bare artifact of the given kind and source reference.
fn bare(kind: ArtifactKind, source_ref: impl Into<String>) -> Self {
Self {
id: uuid::Uuid::new_v4().to_string(),
kind,
source_ref: source_ref.into(),
display_name: None,
stored_path: None,
content_hash: None,
size_bytes: None,
auth: None,
git: None,
web: None,
plc: None,
detected: Vec::new(),
ingested_at: None,
}
}
/// A git-repository artifact tracking the given branch.
pub fn git_repo(url: impl Into<String>, branch: impl Into<String>) -> Self {
let mut a = Self::bare(ArtifactKind::GitRepo, url);
a.git = Some(GitArtifactConfig::on_branch(branch));
a
}
/// A live-URL artifact with default crawl settings.
pub fn live_url(url: impl Into<String>) -> Self {
let mut a = Self::bare(ArtifactKind::LiveUrl, url);
a.web = Some(WebArtifactConfig::default());
a
}
/// A firmware-image artifact referenced by name (blob ingested later).
pub fn firmware_image(source_ref: impl Into<String>) -> Self {
Self::bare(ArtifactKind::FirmwareImage, source_ref)
}
/// A source-archive artifact referenced by name (blob ingested later).
pub fn source_archive(source_ref: impl Into<String>) -> Self {
Self::bare(ArtifactKind::SourceArchive, source_ref)
}
/// A mobile-package artifact (APK/AAB/IPA) referenced by name.
pub fn mobile_package(source_ref: impl Into<String>) -> Self {
Self::bare(ArtifactKind::MobilePackage, source_ref)
}
/// A container-image artifact referenced by OCI ref.
pub fn container_image(source_ref: impl Into<String>) -> Self {
Self::bare(ArtifactKind::ContainerImage, source_ref)
}
/// A PLC-project artifact in the given format.
pub fn plc_project(source_ref: impl Into<String>, format: PlcFormat) -> Self {
let mut a = Self::bare(ArtifactKind::PlcProject, source_ref);
a.plc = Some(PlcArtifactConfig { format });
a
}
/// A plaintext-description artifact (classification input only).
pub fn plaintext(text: impl Into<String>) -> Self {
Self::bare(ArtifactKind::PlaintextDescription, text)
}
}
/// One ranked candidate produced by the classifier.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TargetTypeCandidate {
/// The candidate target type.
pub target_type: TargetType,
/// Confidence in `[0.0, 1.0]`.
pub confidence: f32,
/// Why this candidate was proposed.
pub rationale: String,
}
/// The classifier's verdict for a target: a suggested type plus ranked
/// alternatives and the facts the decision rested on.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Classification {
/// The top-ranked target type.
pub suggested: TargetType,
/// All candidates, sorted by descending confidence.
#[serde(default)]
pub candidates: Vec<TargetTypeCandidate>,
/// Facts gathered during classification.
#[serde(default)]
pub facts: Vec<DetectedFact>,
/// Which classifiers contributed (e.g. `["tramiton", "language-fingerprint"]`).
#[serde(default)]
pub detected_by: Vec<String>,
/// When classification ran.
#[serde(with = "super::serde_helpers::bson_datetime")]
pub detected_at: DateTime<Utc>,
/// Whether a human confirmed the suggestion.
#[serde(default)]
pub confirmed: bool,
}
/// Issue-tracker linkage, migrated from `TrackedRepository`'s `tracker_*` fields.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct IssueTrackerConfig {
/// The tracker platform.
pub tracker_type: Option<TrackerType>,
/// Tracker owner / organization.
pub owner: Option<String>,
/// Tracker repository / project.
pub repo: Option<String>,
/// Per-target tracker access token.
pub token: Option<String>,
}
/// How a target should be scanned.
///
/// `enabled_scans` / `disabled_scans` override the scan-applicability matrix
/// defaults; the pentest and tracker blocks reuse the existing wizard config.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct TargetScanConfig {
/// Scans explicitly turned on (empty means "use matrix defaults").
#[serde(default)]
pub enabled_scans: Vec<ScanType>,
/// Scans explicitly turned off.
#[serde(default)]
pub disabled_scans: Vec<ScanType>,
/// Target environment (gates destructive / active testing).
#[serde(default)]
pub environment: Environment,
/// Whether destructive tests are permitted for this target.
#[serde(default)]
pub allow_destructive: bool,
/// Pentest strategy selector.
pub strategy: Option<PentestStrategy>,
/// Full pentest wizard configuration.
pub pentest: Option<PentestConfig>,
/// Issue-tracker linkage.
pub issue_tracker: Option<IssueTrackerConfig>,
}
/// A sibling product in the company suite that may already hold authoritative
/// data for a target. compliance-scanner reconciles with these rather than
/// recomputing what they already know.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ExternalSystem {
/// Reproducible-build & firmware compliance engine (build plan, SBOM, VEX,
/// attestation).
Tramiton,
/// Code assistant (downstream remediation consumer).
Werkpilot,
/// Compliance-controls RAG (atomic controls derived from laws).
BreakpilotCompliance,
}
impl std::fmt::Display for ExternalSystem {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Tramiton => write!(f, "tramiton"),
Self::Werkpilot => write!(f, "werkpilot"),
Self::BreakpilotCompliance => write!(f, "breakpilot_compliance"),
}
}
}
/// A link from this target to a record in a sibling product, used to reconcile
/// existing evidence instead of recomputing it.
///
/// For tramiton, `project_id` is the shared cross-product key and
/// `subject_sha256` matches a firmware artifact's [`Artifact::content_hash`]
/// (which equals tramiton's `Artifact.sha256`).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ExternalRef {
/// Which sibling product this reference points at.
pub system: ExternalSystem,
/// The sibling product's project identifier, if known.
pub project_id: Option<String>,
/// Content digest of the subject artifact (firmware sha256), if known.
pub subject_sha256: Option<String>,
/// Reconciliation status: `linked` | `reconciled` | `unavailable`.
#[serde(default)]
pub status: String,
/// Opaque, offline-verifiable entitlement grant (e.g. tramiton's signed
/// `LicenseGrant`), if the tenant provided one.
pub license_grant: Option<String>,
/// When evidence was last reconciled from this system.
#[serde(default, with = "super::serde_helpers::opt_bson_datetime")]
pub last_reconciled_at: Option<DateTime<Utc>>,
}
impl ExternalRef {
/// A freshly linked (not yet reconciled) reference to a sibling system.
pub fn linked(system: ExternalSystem) -> Self {
Self {
system,
project_id: None,
subject_sha256: None,
status: "linked".to_string(),
license_grant: None,
last_reconciled_at: None,
}
}
}
/// A regulatory / standards framework a target must comply with. Drives which
/// controls the mapping engine pulls from the [`crate::traits::ControlsProvider`].
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ComplianceFramework {
/// EU Cyber Resilience Act.
Cra,
/// IEC 62443 (industrial automation & control systems security).
Iec62443,
/// EU General Data Protection Regulation.
Gdpr,
/// SOC 2.
Soc2,
/// ISO/IEC 27001.
Iso27001,
/// EU Radio Equipment Directive (RED) cybersecurity articles.
RedDirective,
}
impl std::fmt::Display for ComplianceFramework {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Cra => write!(f, "cra"),
Self::Iec62443 => write!(f, "iec_62443"),
Self::Gdpr => write!(f, "gdpr"),
Self::Soc2 => write!(f, "soc2"),
Self::Iso27001 => write!(f, "iso_27001"),
Self::RedDirective => write!(f, "red_directive"),
}
}
}
/// The compliance scope of a target: which frameworks apply and, optionally, the
/// jurisdiction. Captured at onboarding (with per-target-type defaults from
/// [`default_compliance_profile`]).
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct ComplianceProfile {
/// Applicable frameworks.
#[serde(default)]
pub frameworks: Vec<ComplianceFramework>,
/// Free-form jurisdiction (e.g. `eu`, `us`, `de`).
pub jurisdiction: Option<String>,
}
/// The sensible default compliance scope for a target type. Firmware / PLC /
/// embedded default to CRA + IEC 62443; software defaults to GDPR + SOC 2.
pub fn default_compliance_profile(target_type: TargetType) -> ComplianceProfile {
use ComplianceFramework::{Cra, Gdpr, Iec62443, Soc2};
let frameworks = match target_type {
TargetType::PlcSps
| TargetType::FirmwareBareMetal
| TargetType::FirmwareRtos
| TargetType::EmbeddedLinuxYocto => vec![Cra, Iec62443],
TargetType::WebApp | TargetType::BackendService => vec![Gdpr, Soc2],
TargetType::DesktopApp | TargetType::AndroidApp | TargetType::IosApp => {
vec![Gdpr, Cra]
}
};
ComplianceProfile {
frameworks,
jurisdiction: None,
}
}
/// A target onboarded for scanning: the unified replacement for the legacy
/// `TrackedRepository` (SAST) and `DastTarget` (DAST) records.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct OnboardedTarget {
/// Mongo id. Preserved from the legacy record during migration so every
/// downstream collection keyed by `repo_id` / `target_id` keeps resolving.
#[serde(rename = "_id", skip_serializing_if = "Option::is_none")]
pub id: Option<bson::oid::ObjectId>,
/// Human-friendly name.
#[serde(default)]
pub name: String,
/// The software family this target belongs to.
pub target_type: TargetType,
/// Optional free-form description (also a classification input).
pub description: Option<String>,
/// The artifacts provided for this target.
#[serde(default)]
pub artifacts: Vec<Artifact>,
/// The classifier's verdict, once run.
pub classification: Option<Classification>,
/// How this target should be scanned.
#[serde(default)]
pub scan_config: TargetScanConfig,
/// The compliance scope (applicable frameworks / jurisdiction).
#[serde(default)]
pub compliance_profile: ComplianceProfile,
/// Links to sibling products (tramiton, ...) holding reconcilable evidence.
#[serde(default)]
pub external_refs: Vec<ExternalRef>,
/// Cron schedule for recurring scans, if any.
pub scan_schedule: Option<String>,
/// Whether inbound webhooks are enabled for this target.
#[serde(default)]
pub webhook_enabled: bool,
/// HMAC secret for verifying inbound webhooks.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub webhook_secret: Option<String>,
/// Cached count of findings across this target's scans.
#[serde(default)]
pub findings_count: u32,
/// Creation timestamp.
#[serde(
default = "chrono::Utc::now",
with = "super::serde_helpers::bson_datetime"
)]
pub created_at: DateTime<Utc>,
/// Last-update timestamp.
#[serde(
default = "chrono::Utc::now",
with = "super::serde_helpers::bson_datetime"
)]
pub updated_at: DateTime<Utc>,
}
impl OnboardedTarget {
/// A new target of the given type with a freshly generated webhook secret.
pub fn new(name: String, target_type: TargetType) -> Self {
let now = Utc::now();
let webhook_secret = uuid::Uuid::new_v4().to_string().replace('-', "");
Self {
id: None,
name,
target_type,
description: None,
artifacts: Vec::new(),
classification: None,
scan_config: TargetScanConfig::default(),
compliance_profile: default_compliance_profile(target_type),
external_refs: Vec::new(),
scan_schedule: None,
webhook_enabled: false,
webhook_secret: Some(webhook_secret),
findings_count: 0,
created_at: now,
updated_at: now,
}
}
/// The first artifact of the given kind, if present.
pub fn first_of(&self, kind: ArtifactKind) -> Option<&Artifact> {
self.artifacts.iter().find(|a| a.kind == kind)
}
/// Whether the target has at least one artifact of the given kind.
pub fn has(&self, kind: ArtifactKind) -> bool {
self.artifacts.iter().any(|a| a.kind == kind)
}
/// The primary code artifact (git repo or source archive), if any.
pub fn code_artifact(&self) -> Option<&Artifact> {
self.artifacts
.iter()
.find(|a| matches!(a.kind, ArtifactKind::GitRepo | ArtifactKind::SourceArchive))
}
/// The live-URL artifact, if any.
pub fn live_url(&self) -> Option<&Artifact> {
self.first_of(ArtifactKind::LiveUrl)
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
fn sample_target() -> OnboardedTarget {
let mut t = OnboardedTarget::new("acme-web".to_string(), TargetType::WebApp);
t.artifacts.push(Artifact::git_repo(
"https://git.example.com/acme.git",
"main",
));
t.artifacts
.push(Artifact::live_url("https://acme.example.com"));
t
}
#[test]
fn onboarded_target_bson_round_trip() {
let t = sample_target();
let b = bson::to_bson(&t).expect("serialize");
let back: OnboardedTarget = bson::from_bson(b.clone()).expect("deserialize");
let b2 = bson::to_bson(&back).expect("re-serialize");
assert_eq!(b, b2);
}
#[test]
fn enum_display_is_snake_case() {
assert_eq!(
TargetType::FirmwareBareMetal.to_string(),
"firmware_bare_metal"
);
assert_eq!(TargetType::PlcSps.to_string(), "plc_sps");
assert_eq!(ArtifactKind::PlcProject.to_string(), "plc_project");
assert_eq!(ArtifactKind::MobilePackage.to_string(), "mobile_package");
}
#[test]
fn helpers_locate_artifacts() {
let t = sample_target();
assert!(t.has(ArtifactKind::GitRepo));
assert!(t.live_url().is_some());
assert!(t.code_artifact().is_some());
assert!(!t.has(ArtifactKind::FirmwareImage));
assert_eq!(
t.first_of(ArtifactKind::GitRepo).map(|a| a.kind),
Some(ArtifactKind::GitRepo)
);
}
#[test]
fn new_target_generates_webhook_secret() {
let t = OnboardedTarget::new("t".to_string(), TargetType::BackendService);
let secret = t.webhook_secret.expect("secret present");
assert_eq!(secret.len(), 32);
assert!(!secret.contains('-'));
}
#[test]
fn dast_auth_folds_into_artifact_auth() {
let dast = DastAuthConfig {
method: "bearer".to_string(),
login_url: Some("https://x/login".to_string()),
username: Some("user".to_string()),
password: Some("pw".to_string()),
token: Some("tok".to_string()),
headers: None,
};
let auth = ArtifactAuth::from(dast);
assert_eq!(auth.method, "bearer");
// Bearer token wins over password.
assert_eq!(auth.secret.as_deref(), Some("tok"));
assert_eq!(auth.login_url.as_deref(), Some("https://x/login"));
}
#[test]
fn each_artifact_gets_a_unique_id() {
let a = Artifact::firmware_image("fw.bin");
let b = Artifact::firmware_image("fw.bin");
assert_ne!(a.id, b.id);
}
#[test]
fn firmware_default_profile_is_cra_and_62443() {
let p = default_compliance_profile(TargetType::FirmwareBareMetal);
assert!(p.frameworks.contains(&ComplianceFramework::Cra));
assert!(p.frameworks.contains(&ComplianceFramework::Iec62443));
}
#[test]
fn webapp_default_profile_is_gdpr_and_soc2() {
let p = default_compliance_profile(TargetType::WebApp);
assert!(p.frameworks.contains(&ComplianceFramework::Gdpr));
assert!(p.frameworks.contains(&ComplianceFramework::Soc2));
}
#[test]
fn new_target_gets_default_profile_and_no_external_refs() {
let t = OnboardedTarget::new("fw".to_string(), TargetType::FirmwareRtos);
assert!(!t.compliance_profile.frameworks.is_empty());
assert!(t.external_refs.is_empty());
}
#[test]
fn external_ref_linked_defaults() {
let r = ExternalRef::linked(ExternalSystem::Tramiton);
assert_eq!(r.system, ExternalSystem::Tramiton);
assert_eq!(r.status, "linked");
assert!(r.project_id.is_none());
assert!(r.last_reconciled_at.is_none());
}
}
+19 -1
View File
@@ -3,7 +3,7 @@ use serde::{Deserialize, Serialize};
use super::repository::ScanTrigger;
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum ScanType {
Sast,
@@ -16,6 +16,14 @@ pub enum ScanType {
SecretDetection,
Lint,
CodeReview,
/// Static analysis of a firmware image (unpack + component CVE).
FirmwareStatic,
/// Control-logic security analysis of PLC / SPS programs.
PlcControlLogic,
/// Static analysis of a mobile package (APK / AAB / IPA).
MobileStatic,
/// Static analysis of a container image.
ContainerScan,
}
impl std::fmt::Display for ScanType {
@@ -31,6 +39,10 @@ impl std::fmt::Display for ScanType {
Self::SecretDetection => write!(f, "secret_detection"),
Self::Lint => write!(f, "lint"),
Self::CodeReview => write!(f, "code_review"),
Self::FirmwareStatic => write!(f, "firmware_static"),
Self::PlcControlLogic => write!(f, "plc_control_logic"),
Self::MobileStatic => write!(f, "mobile_static"),
Self::ContainerScan => write!(f, "container_scan"),
}
}
}
@@ -47,6 +59,8 @@ pub enum ScanRunStatus {
#[serde(rename_all = "snake_case")]
pub enum ScanPhase {
ChangeDetection,
ArtifactIngest,
Classification,
Sast,
SbomGeneration,
CveScanning,
@@ -55,6 +69,10 @@ pub enum ScanPhase {
LintScanning,
CodeReview,
GraphBuilding,
FirmwareStatic,
PlcAnalysis,
MobileStatic,
ContainerScan,
LlmTriage,
IssueCreation,
DastScanning,
+379
View File
@@ -0,0 +1,379 @@
//! The scan-applicability matrix.
//!
//! Which scans are possible for a target is a function of its [`TargetType`] and
//! which [`ArtifactKind`]s are actually present: SAST needs code, DAST needs a
//! running URL, firmware-static analysis needs a firmware image, and so on. This
//! module encodes that as a table — one rule set per target type — and resolves
//! it against a concrete [`OnboardedTarget`] into a list of [`ScanOption`]s the
//! onboarding wizard and the scan pipeline both consume.
use crate::models::{ArtifactKind, OnboardedTarget, ScanType, TargetType};
/// What an artifact a scan needs in order to run.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ArtifactRequirement {
/// Source code — a git repo or a source archive.
Code,
/// A reachable running instance (live URL / endpoint).
RunningUrl,
/// A firmware image / binary blob.
Firmware,
/// A PLC project (PLCopen XML or Structured Text).
Plc,
/// A mobile package (APK / AAB / IPA).
Mobile,
/// A container image.
Container,
/// No specific artifact required.
Any,
}
/// A static rule: this scan applies to a target type, needs this artifact, and
/// defaults on/off. The rationale explains the entry to the user.
#[derive(Debug, Clone, Copy)]
pub struct ScanRule {
/// The scan this rule governs.
pub scan: ScanType,
/// Whether the scan is on by default (only when its artifact is present).
pub default_on: bool,
/// Human-readable explanation of what the scan does here.
pub rationale: &'static str,
/// The artifact the scan consumes.
pub requires: ArtifactRequirement,
}
impl ScanRule {
const fn new(
scan: ScanType,
default_on: bool,
rationale: &'static str,
requires: ArtifactRequirement,
) -> Self {
Self {
scan,
default_on,
rationale,
requires,
}
}
}
/// A resolved scan choice for a specific target: a rule intersected with the
/// artifacts actually present. `blocked_reason` is `Some` when the required
/// artifact is missing.
#[derive(Debug, Clone)]
pub struct ScanOption {
/// The scan.
pub scan: ScanType,
/// Whether to pre-select the scan (false when blocked).
pub default_on: bool,
/// Why the scan is offered.
pub rationale: String,
/// The artifact kind the scan needs, if any specific one.
pub required_artifact: Option<ArtifactKind>,
/// Set when the required artifact is absent, explaining the block.
pub blocked_reason: Option<String>,
}
/// The SAST umbrella: every static-analysis sub-scan that runs over source code.
fn sast_umbrella() -> Vec<ScanRule> {
use ArtifactRequirement::Code;
vec![
ScanRule::new(
ScanType::Sast,
true,
"Static analysis (Semgrep) over source",
Code,
),
ScanRule::new(
ScanType::Sbom,
true,
"Software bill of materials from source",
Code,
),
ScanRule::new(
ScanType::Cve,
true,
"Match dependencies against known CVEs",
Code,
),
ScanRule::new(
ScanType::SecretDetection,
true,
"Scan source for committed secrets",
Code,
),
ScanRule::new(ScanType::Lint, true, "Language linters over source", Code),
ScanRule::new(
ScanType::Gdpr,
true,
"GDPR data-handling pattern checks",
Code,
),
ScanRule::new(
ScanType::OAuth,
true,
"OAuth misconfiguration patterns",
Code,
),
ScanRule::new(
ScanType::Graph,
true,
"Build the code graph for impact analysis",
Code,
),
ScanRule::new(
ScanType::CodeReview,
false,
"LLM code review over changed source",
Code,
),
]
}
/// The rule set for a target type. Scans that are never applicable to a type are
/// simply absent (e.g. DAST is not listed for a PLC target).
pub fn rules_for(target_type: TargetType) -> Vec<ScanRule> {
use ArtifactRequirement::{Firmware, Mobile, Plc, RunningUrl};
match target_type {
TargetType::WebApp | TargetType::BackendService => {
let mut r = sast_umbrella();
r.push(ScanRule::new(
ScanType::Dast,
true,
"Dynamic scan of the running endpoint",
RunningUrl,
));
r
}
TargetType::DesktopApp => sast_umbrella(),
TargetType::AndroidApp | TargetType::IosApp => {
let mut r = sast_umbrella();
r.push(ScanRule::new(
ScanType::MobileStatic,
true,
"Static analysis of the mobile package (manifest, permissions, libs)",
Mobile,
));
r
}
TargetType::FirmwareBareMetal | TargetType::FirmwareRtos => {
let mut r = sast_umbrella();
r.push(ScanRule::new(
ScanType::FirmwareStatic,
true,
"Unpack and statically analyze the firmware image",
Firmware,
));
r.push(ScanRule::new(
ScanType::Sbom,
true,
"SBOM from the firmware image (binwalk / tramiton)",
Firmware,
));
r.push(ScanRule::new(
ScanType::Cve,
true,
"Match firmware components against known CVEs",
Firmware,
));
r
}
TargetType::EmbeddedLinuxYocto => {
let mut r = sast_umbrella();
r.push(ScanRule::new(
ScanType::FirmwareStatic,
true,
"EMBA / binwalk static analysis of the image",
Firmware,
));
r.push(ScanRule::new(
ScanType::Sbom,
true,
"SBOM from image layers / recipes",
Firmware,
));
r.push(ScanRule::new(
ScanType::Cve,
true,
"Match image components against known CVEs",
Firmware,
));
r.push(ScanRule::new(
ScanType::Dast,
false,
"Dynamic scan of exposed network services (if any)",
RunningUrl,
));
r
}
TargetType::PlcSps => vec![ScanRule::new(
ScanType::PlcControlLogic,
true,
"Control-logic security rules over the PLC program",
Plc,
)],
}
}
/// Whether an active penetration test is applicable to this target type.
///
/// Pentest runs as its own session (not a [`ScanType`] scan) and needs a
/// reachable running target, so it is offered only for the network-reachable
/// families.
pub fn supports_pentest(target_type: TargetType) -> bool {
matches!(
target_type,
TargetType::WebApp
| TargetType::BackendService
| TargetType::AndroidApp
| TargetType::IosApp
| TargetType::EmbeddedLinuxYocto
)
}
/// The representative artifact kind a requirement is satisfied by.
fn representative_kind(req: ArtifactRequirement) -> Option<ArtifactKind> {
match req {
ArtifactRequirement::Code => Some(ArtifactKind::GitRepo),
ArtifactRequirement::RunningUrl => Some(ArtifactKind::LiveUrl),
ArtifactRequirement::Firmware => Some(ArtifactKind::FirmwareImage),
ArtifactRequirement::Plc => Some(ArtifactKind::PlcProject),
ArtifactRequirement::Mobile => Some(ArtifactKind::MobilePackage),
ArtifactRequirement::Container => Some(ArtifactKind::ContainerImage),
ArtifactRequirement::Any => None,
}
}
/// Whether the target carries an artifact that satisfies the requirement.
fn requirement_satisfied(req: ArtifactRequirement, target: &OnboardedTarget) -> bool {
match req {
ArtifactRequirement::Code => target.code_artifact().is_some(),
ArtifactRequirement::RunningUrl => target.has(ArtifactKind::LiveUrl),
ArtifactRequirement::Firmware => target.has(ArtifactKind::FirmwareImage),
ArtifactRequirement::Plc => target.has(ArtifactKind::PlcProject),
ArtifactRequirement::Mobile => target.has(ArtifactKind::MobilePackage),
ArtifactRequirement::Container => target.has(ArtifactKind::ContainerImage),
ArtifactRequirement::Any => true,
}
}
/// Resolve the matrix for a concrete target into the scans it can run, marking
/// any whose required artifact is missing as blocked.
pub fn applicable_scans(target: &OnboardedTarget) -> Vec<ScanOption> {
rules_for(target.target_type)
.into_iter()
.map(|rule| {
let satisfied = requirement_satisfied(rule.requires, target);
let required_artifact = representative_kind(rule.requires);
let blocked_reason = if satisfied {
None
} else {
Some(match required_artifact {
Some(kind) => format!("no {kind} artifact provided"),
None => "required artifact missing".to_string(),
})
};
ScanOption {
scan: rule.scan,
default_on: rule.default_on && satisfied,
rationale: rule.rationale.to_string(),
required_artifact,
blocked_reason,
}
})
.collect()
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use crate::models::{Artifact, PlcFormat};
fn target_with(target_type: TargetType, artifacts: Vec<Artifact>) -> OnboardedTarget {
let mut t = OnboardedTarget::new("t".to_string(), target_type);
t.artifacts = artifacts;
t
}
fn option<'a>(opts: &'a [ScanOption], scan: ScanType) -> Option<&'a ScanOption> {
opts.iter().find(|o| o.scan == scan)
}
#[test]
fn webapp_with_code_and_url_offers_sast_and_dast() {
let t = target_with(
TargetType::WebApp,
vec![
Artifact::git_repo("u", "main"),
Artifact::live_url("http://x"),
],
);
let opts = applicable_scans(&t);
let sast = option(&opts, ScanType::Sast).expect("sast offered");
assert!(sast.default_on && sast.blocked_reason.is_none());
let dast = option(&opts, ScanType::Dast).expect("dast offered");
assert!(dast.default_on && dast.blocked_reason.is_none());
}
#[test]
fn webapp_without_url_blocks_dast() {
let t = target_with(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
let opts = applicable_scans(&t);
let dast = option(&opts, ScanType::Dast).expect("dast listed");
assert!(!dast.default_on);
assert!(dast.blocked_reason.is_some());
assert_eq!(dast.required_artifact, Some(ArtifactKind::LiveUrl));
}
#[test]
fn firmware_offers_firmware_static_and_not_dast() {
let t = target_with(
TargetType::FirmwareBareMetal,
vec![Artifact::firmware_image("fw.bin")],
);
let opts = applicable_scans(&t);
let fw = option(&opts, ScanType::FirmwareStatic).expect("firmware static offered");
assert!(fw.default_on && fw.blocked_reason.is_none());
assert!(option(&opts, ScanType::Dast).is_none());
}
#[test]
fn plc_offers_only_control_logic() {
let t = target_with(
TargetType::PlcSps,
vec![Artifact::plc_project("p.xml", PlcFormat::PlcopenXml)],
);
let opts = applicable_scans(&t);
assert_eq!(opts.len(), 1);
assert_eq!(opts[0].scan, ScanType::PlcControlLogic);
assert!(opts[0].default_on);
}
#[test]
fn pentest_support_matches_reachable_families() {
assert!(supports_pentest(TargetType::WebApp));
assert!(supports_pentest(TargetType::BackendService));
assert!(!supports_pentest(TargetType::PlcSps));
assert!(!supports_pentest(TargetType::FirmwareBareMetal));
assert!(!supports_pentest(TargetType::DesktopApp));
}
#[test]
fn every_target_type_has_at_least_one_rule() {
for tt in [
TargetType::WebApp,
TargetType::BackendService,
TargetType::DesktopApp,
TargetType::AndroidApp,
TargetType::IosApp,
TargetType::FirmwareBareMetal,
TargetType::FirmwareRtos,
TargetType::EmbeddedLinuxYocto,
TargetType::PlcSps,
] {
assert!(!rules_for(tt).is_empty(), "{tt} has no rules");
}
}
}
+165
View File
@@ -0,0 +1,165 @@
//! Tenant context propagated through every authenticated request.
//!
//! M7.1 single source of truth for "who is this request for". Claims come
//! from a Keycloak-issued JWT and land here via [`crate::auth::require_jwt_auth`]
//! (enabled with the `axum` feature). Handlers reach into the request
//! extensions with the [`crate::tenant_ctx::TenantCtx`] extractor.
//!
//! The shape mirrors the JWT claim names the breakpilot-platform realm
//! emits (see `platform/orca-platform/dev/keycloak/realm-export.json`).
//! Stable contract — adding fields is fine; renaming is a breaking
//! change for every downstream product.
use serde::{Deserialize, Serialize};
/// Tenant lifecycle status from `PLATFORM_ARCHITECTURE.md §5c`.
///
/// Drives the `tenant_status` middleware:
/// * `Demo` / `Trial` / `Active` — full access.
/// * `Frozen` — read-only after cancel / non-payment. Mutating endpoints
/// return 402.
/// * `Archived` — data-retention window closed. Every endpoint returns 410.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum TenantStatus {
Demo,
Trial,
Active,
Frozen,
Archived,
}
impl TenantStatus {
/// True for statuses that block write paths.
pub fn is_frozen(&self) -> bool {
matches!(self, TenantStatus::Frozen)
}
/// True for statuses that block every request.
pub fn is_archived(&self) -> bool {
matches!(self, TenantStatus::Archived)
}
/// True for the shared demo tenant — metering, billing, and audit
/// export are skipped.
pub fn is_demo(&self) -> bool {
matches!(self, TenantStatus::Demo)
}
}
impl std::fmt::Display for TenantStatus {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Demo => write!(f, "demo"),
Self::Trial => write!(f, "trial"),
Self::Active => write!(f, "active"),
Self::Frozen => write!(f, "frozen"),
Self::Archived => write!(f, "archived"),
}
}
}
/// Org-level role baked into the JWT by the realm's protocol mapper.
/// `PLATFORM_ARCHITECTURE.md §6` is the canonical list.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "UPPERCASE")]
pub enum OrgRole {
ItAdmin,
Cxo,
Finance,
Legal,
User,
/// Anything we haven't enumerated yet — forwards-compatible.
#[serde(other)]
Unknown,
}
impl OrgRole {
/// Parses a single role string (Keycloak emits these as `IT_ADMIN`,
/// `CXO`, etc.). Round-trips with the JSON layer.
pub fn parse(s: &str) -> Self {
match s {
"IT_ADMIN" => OrgRole::ItAdmin,
"CXO" => OrgRole::Cxo,
"FINANCE" => OrgRole::Finance,
"LEGAL" => OrgRole::Legal,
"USER" => OrgRole::User,
_ => OrgRole::Unknown,
}
}
}
/// Everything we know about the requesting tenant at the moment a request
/// lands. Cheap to clone (every field is owned + small).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TenantContext {
/// `tenants.id` from the platform's tenant-registry (UUID).
pub tenant_id: String,
/// Lowercase URL-safe slug. Useful for log lines + audit emit.
pub tenant_slug: String,
/// Org-level roles the authenticated user holds inside this tenant.
/// Drives the per-handler RBAC in `M7.1-followup` PRs.
pub org_roles: Vec<OrgRole>,
/// Products this tenant is currently entitled to. Used to short-circuit
/// MCP / API calls for unsubscribed products.
pub products: Vec<String>,
/// Customer plan (`starter` / `professional` / `enterprise`) — gates
/// per-plan feature flags (e.g., MCP server is enterprise-only).
pub plan: String,
/// Lifecycle status — read by `require_tenant_status` middleware.
pub status: TenantStatus,
/// Keycloak user id of the requester (`sub` claim). Required for audit
/// emit so we know WHO did the thing, not just WHICH tenant.
pub user_id: String,
/// Optional user-facing name from the `name` / `preferred_username`
/// claim. Only used in audit + log lines.
pub user_name: Option<String>,
}
impl TenantContext {
/// True if the caller holds at least one of the listed roles.
pub fn has_any_role(&self, roles: &[OrgRole]) -> bool {
self.org_roles.iter().any(|r| roles.contains(r))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn org_role_parses_known_values() {
assert_eq!(OrgRole::parse("IT_ADMIN"), OrgRole::ItAdmin);
assert_eq!(OrgRole::parse("CXO"), OrgRole::Cxo);
assert_eq!(OrgRole::parse("USER"), OrgRole::User);
}
#[test]
fn org_role_unknown_is_forward_compat() {
assert_eq!(OrgRole::parse("FUTURE_ROLE"), OrgRole::Unknown);
}
#[test]
fn tenant_status_predicates() {
assert!(TenantStatus::Frozen.is_frozen());
assert!(!TenantStatus::Active.is_frozen());
assert!(TenantStatus::Archived.is_archived());
assert!(TenantStatus::Demo.is_demo());
assert!(!TenantStatus::Active.is_demo());
}
#[test]
fn has_any_role_matches() {
let ctx = TenantContext {
tenant_id: "t1".into(),
tenant_slug: "acme".into(),
org_roles: vec![OrgRole::ItAdmin],
products: vec![],
plan: "professional".into(),
status: TenantStatus::Active,
user_id: "u".into(),
user_name: None,
};
assert!(ctx.has_any_role(&[OrgRole::ItAdmin]));
assert!(ctx.has_any_role(&[OrgRole::Cxo, OrgRole::ItAdmin]));
assert!(!ctx.has_any_role(&[OrgRole::User, OrgRole::Cxo]));
}
}
+95
View File
@@ -0,0 +1,95 @@
//! Axum extractor for the per-request `TenantContext`.
//!
//! Handlers consume it as a normal extractor argument:
//!
//! ```ignore
//! async fn list_findings(TenantCtx(ctx): TenantCtx) -> Json<...> {
//! let filter = compliance_core::db::tenant_filter(&ctx);
//! ...
//! }
//! ```
//!
//! The middleware ([`crate::auth::require_jwt_auth`]) is responsible for
//! inserting the context into the request extensions. If it's missing on
//! a route that uses this extractor, that's a bug in the wiring — we
//! return 401 so the caller sees an auth failure rather than a 500.
use axum::{
extract::FromRequestParts,
http::{request::Parts, StatusCode},
response::{IntoResponse, Response},
};
use crate::TenantContext;
#[derive(Debug, Clone)]
pub struct TenantCtx(pub TenantContext);
#[derive(Debug)]
pub struct TenantCtxRejection;
impl IntoResponse for TenantCtxRejection {
fn into_response(self) -> Response {
(
StatusCode::UNAUTHORIZED,
"Missing tenant context — request was not authenticated",
)
.into_response()
}
}
impl<S> FromRequestParts<S> for TenantCtx
where
S: Send + Sync,
{
type Rejection = TenantCtxRejection;
async fn from_request_parts(parts: &mut Parts, _state: &S) -> Result<Self, Self::Rejection> {
parts
.extensions
.get::<TenantContext>()
.cloned()
.map(TenantCtx)
.ok_or(TenantCtxRejection)
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use crate::TenantStatus;
use axum::http::Request;
fn ctx() -> TenantContext {
TenantContext {
tenant_id: "t-1".to_string(),
tenant_slug: "acme".to_string(),
org_roles: vec![],
products: vec![],
plan: "starter".to_string(),
status: TenantStatus::Active,
user_id: "u-1".to_string(),
user_name: None,
}
}
#[tokio::test]
async fn extracts_context_when_present() {
let mut req = Request::new(());
req.extensions_mut().insert(ctx());
let (mut parts, _) = req.into_parts();
let TenantCtx(found) = TenantCtx::from_request_parts(&mut parts, &())
.await
.expect("extractor should succeed");
assert_eq!(found.tenant_id, "t-1");
}
#[tokio::test]
async fn rejects_when_missing() {
let req: Request<()> = Request::new(());
let (mut parts, _) = req.into_parts();
let err = TenantCtx::from_request_parts(&mut parts, &()).await;
assert!(err.is_err());
}
}
+51
View File
@@ -0,0 +1,51 @@
//! The target-classification port.
//!
//! A [`TargetClassifier`] inspects a target's artifacts (and optionally their
//! ingested working directories) and proposes one or more [`ClassifierVerdict`]s
//! — a target type, a confidence, and the facts the decision rested on. Concrete
//! classifiers live in the agent (language/build-system fingerprinting, a
//! firmware detector backed by tramiton, etc.); a registry merges and ranks
//! their verdicts. This mirrors the [`crate::traits::Scanner`] port so the two
//! read the same way.
use std::collections::HashMap;
use std::path::PathBuf;
use crate::error::CoreError;
use crate::models::{Artifact, DetectedFact, TargetType};
/// Everything a classifier needs to reason about a target.
pub struct ClassificationInput<'a> {
/// The artifacts declared for the target.
pub artifacts: &'a [Artifact],
/// Ingested working paths, keyed by [`Artifact::id`]. Absent for artifacts
/// with no on-disk form (e.g. a live URL).
pub working_paths: &'a HashMap<String, PathBuf>,
/// Free-form description of the target, if provided.
pub description: Option<&'a str>,
}
/// A single classifier's proposal for a target.
pub struct ClassifierVerdict {
/// The proposed target type.
pub target_type: TargetType,
/// Confidence in `[0.0, 1.0]`.
pub confidence: f32,
/// Facts that informed the proposal.
pub facts: Vec<DetectedFact>,
/// Human-readable explanation.
pub rationale: String,
}
/// A source of target-type classification.
#[allow(async_fn_in_trait)]
pub trait TargetClassifier: Send + Sync {
/// Stable identifier for this classifier (recorded in `detected_by`).
fn name(&self) -> &str;
/// Propose zero or more ranked verdicts for the given input.
async fn classify(
&self,
input: &ClassificationInput<'_>,
) -> Result<Vec<ClassifierVerdict>, CoreError>;
}

Some files were not shown because too many files have changed in this diff Show More