Compare commits

...
Author SHA1 Message Date
Sharang ParnerkarandClaude Opus 4.8 50a54c02bc docs(plc): add PLC Runtime Landscape reference + support watch-list
CI / Check (pull_request) Successful in 5m23s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
Documents the soft-PLC runtime ecosystem as a support watch-list: we support the
CODESYS family today (incl. the many rebranded-CODESYS OEMs — Schneider, WAGO,
ABB, Bosch, Eaton, KEBA), and everything else (ProConOS/ISaGRAF/straton, Siemens
TIA, Rockwell, Beckhoff TwinCAT, B&R, Mitsubishi, Omron, PLCnext) is watch-list —
we add a project-format parser when a customer needs it. The dynamic OT probe is
vendor-agnostic. Also lists the open-source test-bench substrates (OpenPLC,
Beremiz+MatIEC, 4diac). New Reference sidebar entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 23:59:44 +02:00
sharang 0f4c75b6b3 fix(matrix): DAST needs an http(s) endpoint; don't offer/run it on modbus:// (#182)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Successful in 3m34s
CI / Deploy Dashboard (push) Successful in 2m37s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2m0s
2026-07-16 21:28:47 +00:00
sharang cd65fa345c feat(onboarding): enable opt-in scans from the wizard success step (#181)
CI / Check (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 2m38s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Has been skipped
2026-07-16 20:52:50 +00:00
sharang 097f86243e feat(cve): match the CODESYS runtime against NVD by CPE (#180)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 3m45s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-16 20:10:18 +00:00
sharang b7406eca15 feat(onboarding): Project-archive PLC format + auto-detect (#179)
CI / Check (push) Waiting to run
CI / Detect Changes (push) Waiting to run
CI / Deploy Agent (push) Blocked by required conditions
CI / Deploy Dashboard (push) Blocked by required conditions
CI / Deploy Docs (push) Blocked by required conditions
CI / Deploy MCP (push) Blocked by required conditions
2026-07-16 20:05:05 +00:00
sharang 4f9e2c2828 fix(plc): scan every PLC-source artifact, not just the first (#178)
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Has been cancelled
2026-07-16 19:59:58 +00:00
sharang 98357cae10 fix(upload): raise body-size limit to 512 MiB + surface upload errors (#177)
CI / Deploy Agent (push) Successful in 3m33s
CI / Deploy Dashboard (push) Successful in 2m39s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
2026-07-16 18:03:40 +00:00
sharang 55e4117369 fix(dashboard): preselect saved type/kind in edit-target form (#176)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Dashboard (push) Successful in 2m36s
CI / Deploy Docs (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-16 17:52:27 +00:00
sharang 7932f60ab9 feat(ics): EtherNet/IP probe + OT service-discovery port scan (#175)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m29s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-16 16:48:22 +00:00
sharang b96c4d594c feat(ics): OPC UA reachability probe (#174)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m27s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-16 16:35:54 +00:00
sharang d2003eb3c0 feat(cve): notifications for the PLC control-app SBOM (#173)
CI / Check (push) Waiting to run
CI / Detect Changes (push) Waiting to run
CI / Deploy Agent (push) Blocked by required conditions
CI / Deploy Dashboard (push) Blocked by required conditions
CI / Deploy Docs (push) Blocked by required conditions
CI / Deploy MCP (push) Blocked by required conditions
2026-07-16 16:30:19 +00:00
sharang 77eced8314 feat(ics): dynamic Modbus/TCP probe for PLC/SPS devices (#172)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
2026-07-16 16:25:04 +00:00
sharang a3f3f1d4f5 feat(plc): ingest CODESYS projects from a git repo (SAST + SBOM) (#171)
CI / Deploy Agent (push) Successful in 11m7s
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Dashboard (push) Successful in 7m0s
CI / Deploy Docs (push) Successful in 57s
CI / Deploy MCP (push) Successful in 1m48s
2026-07-16 15:47:04 +00:00
sharang 7369e031c4 feat(plc): control-application SBOM from CODESYS .projectarchive (#170)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m28s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-16 15:28:42 +00:00
sharang 386d8457d6 feat(plc): analyse graphical logic (FBD/LD) from PLCopen XML (#169)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 5m23s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-16 11:43:49 +00:00
sharang 3a53a1d7f2 feat(matrix): PlcSps composite device target (SBOM/CVE/DAST/pentest) (#168)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy MCP (push) Successful in 2m9s
CI / Deploy Agent (push) Successful in 3m51s
CI / Deploy Dashboard (push) Successful in 3m1s
CI / Deploy Docs (push) Has been skipped
2026-07-16 10:56:36 +00:00
sharang afa5c985ee feat(onboarding): PLC/blob file upload (multipart) + single-file ingest fix (#163)
CI / Deploy Agent (push) Successful in 5m39s
CI / Deploy Dashboard (push) Successful in 4m7s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2m38s
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
2026-07-16 09:41:12 +00:00
sharang 6d02b138c6 feat(pipeline): PLC/SPS control-logic security scanner (IEC 61131-3) (#162)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m48s
CI / Deploy Dashboard (push) Successful in 2m51s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2m2s
2026-07-16 08:31:43 +00:00
sharang a981311413 refactor: rip out the legacy TrackedRepository / repositories path (#161)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 8m20s
CI / Deploy Dashboard (push) Successful in 3m13s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2m7s
2026-07-16 07:25:45 +00:00
sharang 87f240b26f feat(onboarding): input validation + editable targets (#160)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 4m6s
CI / Deploy Dashboard (push) Successful in 3m11s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-13 17:57:48 +00:00
sharang 613847d4d3 feat(agent): real nix (sandbox=false) for firmware SBOM, replacing nix-portable (#159)
CI / Deploy Agent (push) Successful in 4m7s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
2026-07-13 16:01:09 +00:00
sharang 71aceafa26 fix(deps): bump tramiton to v0.4.1 (proot-safe firmware build) (#158)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Successful in 1m44s
CI / Deploy Agent (push) Successful in 3m44s
CI / Deploy Dashboard (push) Successful in 2m34s
CI / Deploy Docs (push) Has been skipped
2026-07-13 11:21:43 +00:00
sharang b12d18d99d feat(pipeline): firmware SBOM via tramiton reproducible build (phase 2) (#157)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Successful in 4m53s
CI / Deploy Dashboard (push) Successful in 2m37s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m45s
2026-07-13 10:15:06 +00:00
sharang eaaafc0621 feat(pipeline): analysis-based firmware SBOM from tramiton (#156)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 5m3s
CI / Deploy Dashboard (push) Successful in 3m46s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2m14s
2026-07-13 08:31:33 +00:00
sharang 0ec5fd8295 fix(dashboard): Findings/SBOM filter by targets + accurate target findings_count (#155)
CI / Deploy Agent (push) Successful in 5m7s
CI / Deploy Dashboard (push) Successful in 2m57s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
2026-07-13 07:59:45 +00:00
sharang bf32b9939a fix(onboarding): targets visibility + unified pipeline by default (#154)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 3m41s
CI / Deploy Dashboard (push) Successful in 2m46s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m46s
2026-07-13 07:29:04 +00:00
sharang 669e1f1b03 feat(onboarding): scan-trigger endpoint + wizard Run-Scan button (#153)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Successful in 3m34s
CI / Deploy Dashboard (push) Successful in 2m56s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 22:19:32 +00:00
sharang 9e70bd1c8e ci: don't cancel-in-progress for main-branch runs (only pull_request) (#152)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 22:08:58 +00:00
sharang 9ec07ff7a1 fix(ci): authenticate tramiton fetch in dashboard + mcp image builds (#147)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m47s
CI / Deploy Dashboard (push) Successful in 2m37s
CI / Deploy Docs (push) Has been skipped
2026-07-12 21:52:38 +00:00
sharang 0e57c2d7a7 feat(pipeline): run tramiton classification + provision DAST in run_target (#146)
CI / Deploy Agent (push) Successful in 3m40s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Has been skipped
2026-07-12 21:49:22 +00:00
sharang c6a02f0ac3 feat(pipeline): unified run_target execution behind UNIFIED_PIPELINE (#145)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m47s
CI / Deploy Dashboard (push) Failing after 4m28s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Failing after 5s
2026-07-12 21:31:21 +00:00
sharang 17f788858d feat(dashboard): onboarding wizard UI (#144)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Agent (push) Has been skipped
2026-07-12 21:28:52 +00:00
sharangandSharang Parnerkar c6b5622113 fix(ci): authenticate tramiton-core fetch in the agent image build (main deploys) (#143)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m46s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
The PR `check` job authenticates the private tramiton-core git fetch, but the main-branch **`deploy-agent`** job builds `Dockerfile.agent` where cargo also fetches tramiton-core — inside the image build, with no credentials — so **agent image builds on main fail**.

Fix: inject the PAT as a **BuildKit secret** (never baked into an image layer).
- `Dockerfile.agent`: `RUN --mount=type=secret,id=tramiton_token …` applies the same `https`-insteadOf rewrite + `CARGO_NET_GIT_FETCH_WITH_CLI=true` before `cargo build`.
- `deploy-agent`: `DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN …`, reusing the existing `TRAMITON_FETCH_TOKEN` secret.

Only the agent image depends on tramiton-core (dashboard/mcp/docs unaffected). **Self-tests on merge** — changing `Dockerfile.agent` makes `detect-changes` run `deploy-agent`.

Note: couldn't fully run the image build locally (no PAT value on hand + no `.dockerignore` so the context is large), but this mirrors the working PR-stage auth and uses standard BuildKit secret injection. Assumes the deploy runner's Docker daemon supports BuildKit (docker:27-cli → yes).
---------

Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #143
2026-07-12 20:25:39 +00:00
sharang a204f0c59c feat(api): onboarding endpoints for unified targets (#142)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 20:15:22 +00:00
sharang cf4afdda1b feat(migrate): onboarding backfill (repositories + dast_targets -> onboarded_targets) (#141)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 20:11:16 +00:00
sharang a074efd0b4 ci: Kellnr crates.io mirror + persistent S3-backed sccache (#140)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-10 16:30:51 +00:00
sharang ef6ee3dcd1 feat(onboarding): artifact ingest + classifier + native tramiton + suite seams (#138)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Failing after 5s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Failing after 4s
2026-07-10 16:00:51 +00:00
sharang 675c4ef699 feat(onboarding): unified multi-target model + scan matrix foundation (#134)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m36s
CI / Deploy Dashboard (push) Successful in 7m9s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m44s
2026-07-10 13:41:25 +00:00
sharang aed551231c feat(dashboard): UI for managing MCP tokens (#94)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 3m3s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
Adds /mcp-tokens page so a logged-in user can mint, list, and revoke bearer tokens for the MCP server without curl. Pairs with #92's tenant-scoped MCP middleware — copy a token from the dashboard straight into an LLM client config.
2026-06-30 16:32:54 +00:00
sharang b851f4267a feat(m7.3): scheduler pulls tenants from registry, env as fallback (#96)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
Replaces M7.2-C static SCHEDULER_TENANT_IDS with a live query to the tenant-registry at every tick. New tenants picked up without an agent restart; env stays as fallback so a registry outage never silences the scheduler. Resolution order: registry -> SCHEDULER_TENANT_IDS env -> DEFAULT_SCHEDULER_TENANT_ID. Logs the active source on startup.
2026-06-30 16:32:35 +00:00
sharang e9536b6d98 fix(audit): bump quinn-proto + ignore rmcp DNS-rebinding advisory (#97)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m55s
CI / Deploy Dashboard (push) Successful in 2m44s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m50s
RUSTSEC-2026-0185 (quinn-proto 0.11.14): patch-bump to 0.11.15. RUSTSEC-2026-0189 (rmcp 0.16 DNS rebinding): added to ignore with public-hostname + bearer-auth threat-model justification; rmcp 0.16->2.x migration tracked as a separate multi-hour PR.
2026-06-30 16:07:01 +00:00
sharang a3a96fe2cc feat(m7.3): MCP tenant-scoped bearer tokens (#92)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m13s
CI / Deploy Dashboard (push) Successful in 7m3s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m50s
MCP server validates per-tenant bearer tokens on incoming calls and routes each tool to the caller's tenant DB. Closes the cross-tenant data leak in the MCP path identified in M7.3.
2026-06-30 15:27:21 +00:00
sharang ac24ca766a feat(m7.3): cross-tenant admin HTTP endpoints (#95)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
GET /api/admin/tenants lists tenant DBs; DELETE /api/admin/tenants/{tenant_id} drops them (GDPR). Behind a separate auth path that rejects customer realm tokens.
2026-06-30 15:23:39 +00:00
sharang 485c3ff45e chore(agent): remove stale unscoped webhook routes from API router (#93)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
Webhook routes live on the separate webhook server (port 3002). M7.2-C URL form is /webhook/{tenant_id}/{platform}/{repo_id}; mounting unscoped variants on the API router would mismatch handler signatures.
2026-06-30 15:18:31 +00:00
sharang 69c4f7bb78 feat(dashboard): proactively refresh expired Keycloak tokens (#91)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 8s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 2m55s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-06-17 20:01:37 +00:00
sharang 56482911b8 fix(dashboard): attach Keycloak token on agent API calls (#90)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 6s
CI / Deploy Agent (push) Successful in 4m8s
CI / Deploy Dashboard (push) Successful in 4m58s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-06-17 18:35:59 +00:00
sharang 183234f9af feat(m7.1): wire compliance-agent to compliance-core auth + status gate (#85)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m38s
CI / Deploy Dashboard (push) Successful in 7m30s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m55s
2026-06-17 09:36:52 +00:00
sharang dbadff0aac fix(m7.1): JWKS refresh-on-failure in auth middleware (#84)
CI / Deploy Agent (push) Successful in 11m44s
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Dashboard (push) Successful in 13m1s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m53s
2026-06-04 14:46:14 +00:00
sharang 116293519d M7.1 smoke harness: lift auth to compliance-core + compliance-smoke service (#83)
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
2026-06-04 14:38:35 +00:00
sharang a8cef58e02 feat(dashboard): add light/dark theme with sidebar toggle (#81)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Dashboard (push) Successful in 9m46s
CI / Deploy Docs (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 11:44:22 +00:00
sharang 927fbc8ecb fix: live progress + concurrency for embedding builds (#80)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m59s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 10:01:05 +00:00
sharang e67a13535a fix: add HTTP timeout to reqwest client and CVE stage timeout (#79)
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 8m26s
CI / Deploy Dashboard (push) Has been skipped
CI / Check (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-05-13 07:30:26 +00:00
sharangandSharang Parnerkar df0063abc0 fix: scanner timeouts, semgrep memory cap, syft remote lookups, Script error (#78)
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 9m41s
CI / Check (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 15m19s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 3m7s
## Summary

- **Scan produces no results in Orca** — semgrep (`--config=auto`, unbounded memory) and syft (remote license network calls) were getting OOM-killed or hanging in resource-constrained Orca containers. Scan would "complete" with 0 findings/SBOMs silently because each scanner failure is caught and logged as a warning.
- **Dashboard Script error spam** — `document::Script` in Dioxus 0.7 needs a single text node child for inline scripts; `dangerous_inner_html` was invalid and spammed the error log on every unauthenticated page load.

## Changes

| File | Change |
|------|--------|
| `semgrep.rs` | Add `--max-memory 500 --jobs 1`; 10-minute timeout |
| `syft.rs` | Remove remote license lookup env vars; 5-minute timeout |
| `gitleaks.rs` | 5-minute timeout |
| `app_shell.rs` | Fix `dangerous_inner_html` → text child in `document::Script` |

## Test plan

- [ ] Trigger a scan on a repo in Orca — findings and SBOM entries should now appear
- [ ] Agent logs should show timeout/error warnings rather than silent empty results when tools are killed
- [ ] Navigate to dashboard unauthenticated — Script error gone from logs
- [ ] Verify scans work end-to-end with `docker compose up`

---------

Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #78
2026-05-12 11:27:24 +00:00
Sharang Parnerkar 5cafd13f44 ci: log orca webhook response so deploy steps arent silent
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
Nightly E2E Tests / E2E Tests (push) Failing after 2m59s
2026-04-08 15:09:27 +02:00
Sharang ParnerkarandClaude Opus 4.6 69209649a5 ci: trigger first orca build for all services
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 7m5s
CI / Deploy Dashboard (push) Failing after 21m28s
CI / Detect Changes (push) Successful in 4s
CI / Deploy Docs (push) Successful in 30s
CI / Deploy MCP (push) Successful in 1m31s
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:10:07 +02:00
Sharang ParnerkarandClaude Opus 4.6 d5439adc0d ci: trigger build of dashboard, docs, mcp images for orca
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:09:49 +02:00
Sharang ParnerkarandClaude Opus 4.6 bc7cdd35e4 ci: replace coolify webhook with orca deploy
CI / Check (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
Each deploy job now builds the per-service image, pushes to the
private registry as :latest and :sha, then triggers an HMAC-signed
orca redeploy webhook. Coolify webhooks are no longer used.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:06:11 +02:00
Sharang ParnerkarandClaude Opus 4.6 c062d834a1 fix: downgrade dotenv missing file from FAILED to info message
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
Nightly E2E Tests / E2E Tests (push) Failing after 2m16s
Non-fatal in Docker where env vars come from container config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 15:33:24 +02:00
sharang 23cf37b6c3 fix: CVE notifications during scan + help chat doc loading + Dockerfile (#55)
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Has been skipped
2026-03-30 13:10:56 +00:00
sharang 49d5cd4e0a feat: hourly CVE alerting with notification bell and API (#53)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
2026-03-30 10:39:39 +00:00
sharang 4388e98b5b feat: add E2E test suite with nightly CI, fix dashboard Dockerfile (#52)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 2s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-30 10:04:07 +00:00
sharang a8bb05d7b1 feat: add floating help chat widget, remove settings page (#51)
CI / Deploy Docs (push) Successful in 2s
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3s
CI / Deploy Dashboard (push) Successful in 2s
2026-03-30 08:05:29 +00:00
sharang bae24f9cf8 fix: cascade-delete DAST targets, pentests, and downstream data on repo delete (#50)
CI / Deploy Agent (push) Successful in 9s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
2026-03-30 07:11:23 +00:00
sharang dd53132746 feat: refine all LLM system prompts for precision and reduced false positives (#49)
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Detect Changes (push) Has been cancelled
2026-03-30 07:11:17 +00:00
sharang ff088f9eb4 feat: deduplicate code review findings across LLM passes (#48)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 7s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-29 20:38:52 +00:00
745ad8a441 fix: check Gitea API response status and fallback for PR reviews (#47)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
## Summary
- Add HTTP response status checking to all Gitea tracker methods that were silently swallowing errors
- Add fallback in create_pr_review: if inline comments fail, retry as plain PR comment

## Test plan
- [ ] Deploy and trigger a PR review, check logs for actual error details
- [ ] Verify fallback posts summary comment when inline comments fail

Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com>
Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #47
2026-03-25 16:26:09 +00:00
sharang a9d039dad3 fix: stop storing code review findings in dashboard (#22)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-18 15:18:07 +00:00
Sharang ParnerkarandClaude Opus 4.6 a509bdcb2e fix: require TLS for IMAP auth, close port 143 (CERT-Bund compliance)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 7s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 1s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
- Remove port 143 from mailserver (only expose 993/IMAPS)
- Enable SSL_TYPE=manual with Let's Encrypt certs
- Set DOVECOT_DISABLE_PLAINTEXT_AUTH=yes
- Add pentest_imap_tls config field (defaults to true)

Fixes CERT-Bund report: IMAP PLAIN/LOGIN without TLS on 46.225.100.82:143

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 09:29:34 +01:00
sharangandSharang Parnerkar c461faa2fb feat: pentest onboarding — streaming, browser automation, reports, user cleanup (#16)
CI / Deploy MCP (push) Successful in 2s
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 7s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Successful in 2s
Complete pentest feature overhaul: SSE streaming, session-persistent browser tool (CDP), AES-256 credential encryption, auto-screenshots in reports, code-level remediation correlation, SAST triage chunking, context window optimization, test user cleanup (Keycloak/Auth0/Okta), wizard dropdowns, attack chain improvements, architecture docs with Mermaid diagrams.

Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com>
Reviewed-on: #16
2026-03-17 20:32:20 +00:00
Sharang Parnerkar 11e1c5f438 Merge branch 'fix/chrome-in-agent'
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-03-13 11:25:16 +01:00
Sharang ParnerkarandClaude Opus 4.6 77f1c92c7b ci: skip check stage on main push since PRs enforce it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:20:39 +01:00
sharang 4eac1209d8 fix: remote Chrome PDF export & MCP endpoint sync (#15)
CI / Check (push) Successful in 11m16s
CI / Detect Changes (push) Successful in 3s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy Agent (push) Successful in 3s
CI / Deploy MCP (push) Has been skipped
2026-03-13 10:12:20 +00:00
Sharang ParnerkarandClaude Opus 4.6 584ef2c822 fix: remote Chrome PDF via CDP, sync MCP endpoint URL on boot
CI / Check (pull_request) Successful in 11m33s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
- Add CHROME_WS_URL env var support for PDF report generation via
  Chrome DevTools Protocol over WebSocket (falls back to local binary)
- Update seeded MCP server endpoint URLs on boot when MCP_ENDPOINT_URL
  env var differs from stored value (previously only seeded once)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:57:28 +01:00
sharang a529e9af0c ci: consolidate CI into single job; fix sidebar footer (#14)
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
CI / Check (push) Successful in 11m4s
CI / Detect Changes (push) Successful in 2s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Successful in 2s
2026-03-13 09:44:32 +00:00
sharang 3bb690e5bb refactor: modularize codebase and add 404 unit tests (#13)
CI / Format (push) Successful in 4s
CI / Clippy (push) Successful in 4m19s
CI / Detect Changes (push) Successful in 5s
CI / Tests (push) Successful in 5m15s
CI / Deploy Agent (push) Successful in 2s
CI / Deploy Dashboard (push) Successful in 2s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 2s
CI / Security Audit (push) Successful in 1m44s
2026-03-13 08:03:45 +00:00
sharang acc5b86aa4 feat: AI-driven automated penetration testing (#12)
CI / Clippy (push) Failing after 1m51s
CI / Security Audit (push) Successful in 2m1s
CI / Tests (push) Has been skipped
CI / Detect Changes (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Format (push) Failing after 42s
CI / Deploy MCP (push) Has been skipped
2026-03-12 14:42:54 +00:00
268 changed files with 45728 additions and 4879 deletions
+23
View File
@@ -0,0 +1,23 @@
[advisories]
ignore = [
# hickory-proto 0.25.x pulled in transitively via mongodb → hickory-resolver.
# MongoDB 3.x has not yet released with hickory-resolver 0.26.x, so we cannot
# upgrade past this without a mongodb release. Both are DNS-layer DoS vectors
# requiring a MITM/controlled DNS server against MongoDB's hostname resolution —
# not a realistic attack surface here. Revisit when mongodb bumps hickory.
"RUSTSEC-2026-0118", # NSEC3 loop, no fix available upstream
"RUSTSEC-2026-0119", # O(n²) name compression, fixed in hickory-proto >=0.26.1
# rmcp 0.16.0 — DNS rebinding in Streamable HTTP server transport (missing
# Host header validation). Patched in rmcp >= 1.4.0, which is a major API
# version jump from our pin; rmcp shipped 0.x → 1.x → 2.x in three months
# and the migration touches every tool handler + the auth middleware we
# just landed in #92. Threat model in our deployment: the MCP server is
# exposed at a public hostname (comp-mcp-dev.meghsakha.com) behind orca's
# TLS-terminating ingress with per-tenant bearer auth — the attack model
# (browser DNS-rebinding into localhost MCP server) doesn't directly apply.
# Defense-in-depth Host-header check is still a worthwhile follow-up.
# FOLLOW-UP: bump rmcp to 2.x in a dedicated PR (M7.3 follow-up, sized
# multi-hour due to API surface change).
"RUSTSEC-2026-0189",
]
+138 -101
View File
@@ -2,31 +2,48 @@ name: CI
on:
push:
branches:
- "**"
pull_request:
branches:
- main
pull_request:
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-D warnings"
# sccache caches compilation artifacts within a job so that compiling
# both --features server and --features web shares common crate work.
# Compile cache: sccache -> Hetzner S3 (breakpilot-sccache), runner-independent
# and persistent across CI runs (own key prefix). Reuses the shared cluster S3
# creds (same bucket as werkpilot). Requires repo secrets HETZNER_S3_ACCESS_KEY
# and HETZNER_S3_SECRET_KEY.
RUSTC_WRAPPER: /usr/local/bin/sccache
SCCACHE_DIR: /tmp/sccache
SCCACHE_BUCKET: breakpilot-sccache
SCCACHE_ENDPOINT: https://nbg1.your-objectstorage.com
SCCACHE_REGION: auto
SCCACHE_S3_USE_SSL: "true"
SCCACHE_S3_KEY_PREFIX: compliance-scanner
AWS_ACCESS_KEY_ID: ${{ secrets.HETZNER_S3_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.HETZNER_S3_SECRET_KEY }}
# compliance-agent depends on tramiton-core via git; use the system git so the
# credential rewrite below (see "Configure git auth ...") is honored on fetch.
CARGO_NET_GIT_FETCH_WITH_CLI: "true"
# Throttle cargo so a ~670-crate concurrent download burst doesn't 429 the
# Kellnr mirror: fewer concurrent connections (HTTP/1.1) + more retries.
CARGO_NET_RETRY: "10"
CARGO_HTTP_MULTIPLEXING: "false"
# Cancel in-progress runs for the same branch/PR
# Cancel superseded PR runs, but NEVER cancel main-branch runs — those build and
# deploy per-service images, and cancelling one merge's deploy when the next
# merge lands leaves a service un-deployed (as happened between two back-to-back
# merges). So cancel-in-progress only for pull_request events.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
# ---------------------------------------------------------------------------
# Stage 1: Code quality checks (run in parallel)
# Stage 1: Lint, audit, and test (single job to share cargo cache)
# ---------------------------------------------------------------------------
fmt:
name: Format
check:
name: Check
if: github.event_name == 'pull_request'
runs-on: docker
container:
image: rust:1.94-bookworm
@@ -37,105 +54,86 @@ jobs:
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}"
git checkout FETCH_HEAD
- run: rustup component add rustfmt
# Format check does not compile, so sccache is not needed here.
- run: cargo fmt --all --check
# Resolve crates.io deps through the self-hosted Kellnr mirror (cached,
# crates.io-independent). Git deps (tramiton-core) are unaffected — source
# replacement only applies to crates.io-sourced crates.
- name: Use Kellnr crates.io mirror
run: |
: "${CARGO_HOME:=/usr/local/cargo}"
mkdir -p "$CARGO_HOME"
{
echo '[source.crates-io]'
echo 'replace-with = "kellnr"'
echo '[registries.kellnr]'
echo 'index = "sparse+https://crates.meghsakha.com/api/v1/cratesio/"'
} >> "$CARGO_HOME/config.toml"
env:
RUSTC_WRAPPER: ""
- name: Install tools
run: |
rustup component add rustfmt clippy
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.10.0-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
cargo install cargo-audit --locked
env:
RUSTC_WRAPPER: ""
clippy:
name: Clippy
runs-on: docker
container:
image: rust:1.94-bookworm
steps:
- name: Checkout
# compliance-agent has a git dependency on tramiton-core (a private repo on
# this Gitea instance). Rewrite its SSH URL to HTTPS + a PAT so the runner
# can fetch it. Requires the repo secret TRAMITON_FETCH_TOKEN (a Gitea PAT
# with read:repository, owned by a user with access to sharang/tramiton).
# (Honored on fetch because CARGO_NET_GIT_FETCH_WITH_CLI=true uses system git.)
- name: Configure git auth for private tramiton dependency
run: |
git init
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}"
git checkout FETCH_HEAD
- name: Install sccache
run: |
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.9.1/sccache-v0.9.1-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.9.1-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
- run: rustup component add clippy
# Lint the agent (native only).
git config --global \
url."https://sharang:${{ secrets.TRAMITON_FETCH_TOKEN }}@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"
env:
RUSTC_WRAPPER: ""
# Format (no compilation needed)
- name: Format
run: cargo fmt --all --check
env:
RUSTC_WRAPPER: ""
# Clippy (compiles once, sccache reuses across feature sets)
- name: Clippy (agent)
run: cargo clippy -p compliance-agent -- -D warnings
# Lint the dashboard for both feature sets independently.
# sccache deduplicates shared crates between the two compilations.
- name: Clippy (dashboard server)
run: cargo clippy -p compliance-dashboard --features server --no-default-features -- -D warnings
- name: Clippy (dashboard web)
run: cargo clippy -p compliance-dashboard --features web --no-default-features -- -D warnings
- name: Clippy (mcp)
run: cargo clippy -p compliance-mcp -- -D warnings
- name: Show sccache stats
run: sccache --show-stats
if: always()
audit:
name: Security Audit
runs-on: docker
if: github.ref == 'refs/heads/main'
container:
image: rust:1.94-bookworm
steps:
- name: Checkout
run: |
git init
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}"
git checkout FETCH_HEAD
- run: cargo install cargo-audit
env:
RUSTC_WRAPPER: ""
- run: cargo audit
# Security audit
- name: Security Audit
run: cargo audit
env:
RUSTC_WRAPPER: ""
# ---------------------------------------------------------------------------
# Stage 2: Tests (only after all quality checks pass)
# ---------------------------------------------------------------------------
test:
name: Tests
runs-on: docker
needs: [fmt, clippy, audit]
container:
image: rust:1.94-bookworm
steps:
- name: Checkout
run: |
git init
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}"
git checkout FETCH_HEAD
- name: Install sccache
run: |
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.9.1/sccache-v0.9.1-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.9.1-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
- name: Run tests (core + agent)
run: cargo test -p compliance-core -p compliance-agent
- name: Run tests (dashboard server)
# Tests (reuses compilation artifacts from clippy)
- name: Tests (core + agent)
run: cargo test -p compliance-core -p compliance-agent --lib
- name: Tests (dashboard server)
run: cargo test -p compliance-dashboard --features server --no-default-features
- name: Run tests (dashboard web)
- name: Tests (dashboard web)
run: cargo test -p compliance-dashboard --features web --no-default-features
- name: Show sccache stats
run: sccache --show-stats
if: always()
# ---------------------------------------------------------------------------
# Stage 3: Deploy (only on main, after tests pass)
# Stage 2: Deploy (only on main, after checks pass)
# Each service only deploys when its relevant files changed.
# ---------------------------------------------------------------------------
detect-changes:
name: Detect Changes
runs-on: docker
if: github.ref == 'refs/heads/main'
needs: [test]
container:
image: alpine:latest
outputs:
@@ -193,13 +191,25 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.agent == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
env:
# PAT for fetching the private tramiton-core git dependency during the
# image build (injected as a BuildKit secret, never baked into a layer).
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_AGENT }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-agent
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy agent"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-dashboard:
name: Deploy Dashboard
@@ -207,13 +217,23 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.dashboard == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
env:
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_DASHBOARD }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-dashboard
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy dashboard"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-docs:
name: Deploy Docs
@@ -221,13 +241,20 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.docs == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_DOCS }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-docs
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.docs -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy docs"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
deploy-mcp:
name: Deploy MCP
@@ -235,10 +262,20 @@ jobs:
needs: [detect-changes]
if: needs.detect-changes.outputs.mcp == 'true'
container:
image: alpine:latest
image: docker:27-cli
steps:
- name: Trigger Coolify deploy
- name: Build, push and trigger orca redeploy
env:
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache curl
curl -sf "${{ secrets.COOLIFY_WEBHOOK_MCP }}" \
-H "Authorization: Bearer ${{ secrets.COOLIFY_TOKEN }}"
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-mcp
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy mcp"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
RESP=$(curl -fsS -w "\nHTTP %{http_code}" -X POST "http://46.225.100.82:6880/api/v1/webhooks/github" -H "Content-Type: application/json" -H "X-Hub-Signature-256: sha256=$SIG" -d "$PAYLOAD"); echo "$RESP"
+52
View File
@@ -0,0 +1,52 @@
name: Nightly E2E Tests
on:
schedule:
- cron: '0 3 * * *' # 3 AM UTC daily
workflow_dispatch: # Allow manual trigger
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-D warnings"
RUSTC_WRAPPER: /usr/local/bin/sccache
SCCACHE_DIR: /tmp/sccache
TEST_MONGODB_URI: "mongodb://root:example@mongo:27017/?authSource=admin"
concurrency:
group: nightly-e2e
cancel-in-progress: true
jobs:
e2e:
name: E2E Tests
runs-on: docker
container:
image: rust:1.94-bookworm
services:
mongo:
image: mongo:7
env:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
steps:
- name: Checkout
run: |
git init
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA:-refs/heads/main}"
git checkout FETCH_HEAD
- name: Install sccache
run: |
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.9.1/sccache-v0.9.1-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.9.1-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
env:
RUSTC_WRAPPER: ""
- name: Run E2E tests
run: cargo test -p compliance-agent --test e2e -- --test-threads=4
- name: Show sccache stats
run: sccache --show-stats
if: always()
+3
View File
@@ -4,3 +4,6 @@
*.swo
*~
.DS_Store
.playwright-mcp/
report-preview-full.png
compliance-dashboard/attack-chain-final.html
Generated
+564 -31
View File
File diff suppressed because it is too large Load Diff
+8 -1
View File
@@ -6,6 +6,7 @@ members = [
"compliance-graph",
"compliance-dast",
"compliance-mcp",
"compliance-smoke",
]
resolver = "2"
@@ -22,10 +23,16 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
chrono = { version = "0.4", features = ["serde"] }
mongodb = { version = "3", features = ["rustls-tls", "compat-3-0-0"] }
reqwest = { version = "0.12", features = ["json", "rustls-tls"], default-features = false }
reqwest = { version = "0.12", features = ["json", "rustls-tls", "multipart"], default-features = false }
thiserror = "2"
sha2 = "0.10"
hex = "0.4"
uuid = { version = "1", features = ["v4", "serde"] }
secrecy = { version = "0.10", features = ["serde"] }
regex = "1"
zip = { version = "2", features = ["aes-crypto", "deflate"] }
dashmap = "6"
tokio-stream = { version = "0.1", features = ["sync"] }
aes-gcm = "0.10"
rand = "0.9"
base64 = "0.22"
+47 -2
View File
@@ -2,7 +2,22 @@ FROM rust:1.94-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release -p compliance-agent
# compliance-agent depends on the private tramiton-core git repo. Authenticate
# the fetch with a PAT passed as a BuildKit secret (never baked into a layer).
# Build with: DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN ...
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release -p compliance-agent
# A throwaway stage that packs a real nix store (store paths + the validity DB)
# into a compressed bootstrap tarball. Only the tarball is copied into the final
# image, so we don't carry a raw /nix copy layer.
FROM nixos/nix:latest AS nixseed
RUN tar -C / -czf /nix-bootstrap.tar.gz nix
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 git curl python3 python3-pip npm golang-go php-cli && rm -rf /var/lib/apt/lists/*
@@ -31,11 +46,41 @@ RUN pip3 install --break-system-packages semgrep
# Install ruff for Python linting
RUN pip3 install --break-system-packages ruff
# Real nix for the tramiton reproducible-build firmware SBOM.
#
# nix-portable's proot fallback can't run here: user namespaces are blocked by
# the container's default seccomp/apparmor profile, and orca exposes no way to
# relax it. So ship a *real* nix and disable its build sandbox
# (`sandbox = false`) — a plain gcc/make firmware build needs no user namespace,
# so it runs fine under the locked-down profile with no proot involved.
#
# The store is shipped as a bootstrap tarball and seeded onto /nix at first
# start (see docker/agent-entrypoint.sh), so a persistent /nix volume survives
# redeploys. A missing/broken nix just falls back to the analysis-only SBOM.
COPY --from=nixseed /nix-bootstrap.tar.gz /opt/nix-bootstrap.tar.gz
ENV PATH="/nix/var/nix/profiles/default/bin:${PATH}"
RUN mkdir -p /etc/nix && printf '%s\n' \
'experimental-features = nix-command flakes' \
'sandbox = false' \
'build-users-group =' \
'substituters = https://cache.nixos.org' \
'trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=' \
> /etc/nix/nix.conf
COPY --from=builder /app/target/release/compliance-agent /usr/local/bin/compliance-agent
COPY docker/agent-entrypoint.sh /usr/local/bin/agent-entrypoint.sh
RUN chmod +x /usr/local/bin/agent-entrypoint.sh
# Copy documentation for the help chat assistant
COPY --from=builder /app/README.md /app/README.md
COPY --from=builder /app/docs /app/docs
ENV HELP_DOCS_PATH=/app
# Ensure SSH key directory exists
RUN mkdir -p /data/compliance-scanner/ssh
EXPOSE 3001 3002
ENTRYPOINT ["compliance-agent"]
# Seeds /nix (fresh volume) from the bootstrap tarball, then runs the agent.
ENTRYPOINT ["/usr/local/bin/agent-entrypoint.sh"]
+12 -2
View File
@@ -1,13 +1,22 @@
FROM rust:1.94-bookworm AS builder
RUN cargo install dioxus-cli --version 0.7.3
RUN cargo install dioxus-cli --version 0.7.3 --locked
ARG DOCS_URL=/docs
WORKDIR /app
COPY . .
ENV DOCS_URL=${DOCS_URL}
RUN dx build --release --package compliance-dashboard
# compliance-agent (a workspace member) depends on the private tramiton-core git
# repo, so the workspace resolve needs it even to build the dashboard.
# Authenticate the fetch with a PAT passed as a BuildKit secret.
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true dx build --release --package compliance-dashboard
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 && rm -rf /var/lib/apt/lists/*
@@ -20,3 +29,4 @@ ENV IP=0.0.0.0
EXPOSE 8080
ENTRYPOINT ["./compliance-dashboard"]
+1
View File
@@ -12,3 +12,4 @@ RUN rm /etc/nginx/conf.d/default.conf
COPY docs/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /app/.vitepress/dist /usr/share/nginx/html
EXPOSE 80
+11 -1
View File
@@ -2,7 +2,16 @@ FROM rust:1.94-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release -p compliance-mcp
# compliance-agent (a workspace member) depends on the private tramiton-core git
# repo, so the workspace resolve needs it even to build the mcp binary.
# Authenticate the fetch with a PAT passed as a BuildKit secret.
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release -p compliance-mcp
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 && rm -rf /var/lib/apt/lists/*
@@ -14,3 +23,4 @@ EXPOSE 8090
ENV MCP_PORT=8090
ENTRYPOINT ["compliance-mcp"]
+79 -45
View File
@@ -9,7 +9,7 @@
</p>
<p align="center">
<a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/Rust-1.89-orange?logo=rust&logoColor=white" alt="Rust" /></a>
<a href="https://www.rust-lang.org/"><img src="https://img.shields.io/badge/Rust-1.94-orange?logo=rust&logoColor=white" alt="Rust" /></a>
<a href="https://dioxuslabs.com/"><img src="https://img.shields.io/badge/Dioxus-0.7-blue?logo=webassembly&logoColor=white" alt="Dioxus" /></a>
<a href="https://www.mongodb.com/"><img src="https://img.shields.io/badge/MongoDB-8.0-47A248?logo=mongodb&logoColor=white" alt="MongoDB" /></a>
<a href="https://axum.rs/"><img src="https://img.shields.io/badge/Axum-0.8-4A4A55?logo=rust&logoColor=white" alt="Axum" /></a>
@@ -28,9 +28,9 @@
## About
Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira) with evidence and remediation suggestions, reviews pull requests, and exposes a Dioxus-based dashboard for visualization.
Compliance Scanner is an autonomous agent that continuously monitors git repositories for security vulnerabilities, GDPR/OAuth compliance patterns, and dependency risks. It creates issues in external trackers (GitHub/GitLab/Jira/Gitea) with evidence and remediation suggestions, reviews pull requests with multi-pass LLM analysis, runs autonomous penetration tests, and exposes a Dioxus-based dashboard for visualization.
> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation.
> **How it works:** The agent runs as a lazy daemon -- it only scans when new commits are detected, triggered by cron schedules or webhooks. LLM-powered triage filters out false positives and generates actionable remediation with multi-language awareness.
## Features
@@ -41,31 +41,38 @@ Compliance Scanner is an autonomous agent that continuously monitors git reposit
| **CVE Monitoring** | OSV.dev batch queries, NVD CVSS enrichment, SearXNG context |
| **GDPR Patterns** | Detect PII logging, missing consent, hardcoded retention, missing deletion |
| **OAuth Patterns** | Detect implicit grant, missing PKCE, token in localStorage, token in URLs |
| **LLM Triage** | Confidence scoring via LiteLLM to filter false positives |
| **Issue Creation** | Auto-create issues in GitHub, GitLab, or Jira with code evidence |
| **PR Reviews** | Post security review comments on pull requests |
| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, and statistics |
| **Webhooks** | GitHub (HMAC-SHA256) and GitLab webhook receivers for push/PR events |
| **LLM Triage** | Multi-language-aware confidence scoring (Rust, Python, Go, Java, Ruby, PHP, C++) |
| **Issue Creation** | Auto-create issues in GitHub, GitLab, Jira, or Gitea with dedup via fingerprints |
| **PR Reviews** | Multi-pass security review (logic, security, convention, complexity) with dedup |
| **DAST Scanning** | Black-box security testing with endpoint discovery and parameter fuzzing |
| **AI Pentesting** | Autonomous LLM-orchestrated penetration testing with encrypted reports |
| **Code Graph** | Interactive code knowledge graph with impact analysis |
| **AI Chat (RAG)** | Natural language Q&A grounded in repository source code |
| **Help Assistant** | Documentation-grounded help chat accessible from every dashboard page |
| **MCP Server** | Expose live security data to Claude, Cursor, and other AI tools |
| **Dashboard** | Fullstack Dioxus UI with findings, SBOM, issues, DAST, pentest, and graph |
| **Webhooks** | GitHub, GitLab, and Gitea webhook receivers for push/PR events |
| **Finding Dedup** | SHA-256 fingerprint dedup for SAST, CWE-based dedup for DAST findings |
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Cargo Workspace │
├──────────────┬──────────────────┬───────────────────────────┤
│ compliance- │ compliance- │ compliance-
│ core │ agent │ dashboard
(lib)(bin) │ (bin, Dioxus 0.7.3)
│ │
Models │ Scan Pipeline │ Fullstack Web UI
TraitsLLM Client │ Server Functions
ConfigIssue Trackers Charts + Tables
Errors │ Scheduler │ Settings Page
│ │ REST API │ │
│ │ Webhooks │ │
└──────────────┴──────────────────┴───────────────────────────┘
MongoDB (shared)
┌──────────────────────────────────────────────────────────────────────────
Cargo Workspace
├──────────────┬──────────────────┬──────────────┬──────────┬─────────────┤
│ compliance- │ compliance- │ compliance- │ complian-│ compliance-
│ core (lib) │ agent (bin) │ dashboard │ ce-graph │ mcp (bin)
│ (bin) │ (lib) │
Models │ Scan Pipeline │ Dioxus 0.7 │ Tree- │ MCP Server
Traits │ LLM Client │ Fullstack UI │ sitter │ Live data
ConfigIssue Trackers │ Help Chat │ Graph │ for AI
ErrorsPentest Engine Server Fns │ Embedds │ tools
│ DAST Tools │ │ RAG │
│ │ REST API │ │ │
│ │ Webhooks │ │ │
└──────────────┴──────────────────┴──────────────┴──────────┴─────────────┘
MongoDB (shared)
```
## Scan Pipeline (7 Stages)
@@ -84,17 +91,22 @@ Compliance Scanner is an autonomous agent that continuously monitors git reposit
|-------|-----------|
| Shared Library | `compliance-core` -- models, traits, config |
| Agent | Axum REST API, git2, tokio-cron-scheduler, Semgrep, Syft |
| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS |
| Dashboard | Dioxus 0.7.3 fullstack, Tailwind CSS 4 |
| Code Graph | `compliance-graph` -- tree-sitter parsing, embeddings, RAG |
| MCP Server | `compliance-mcp` -- Model Context Protocol for AI tools |
| DAST | `compliance-dast` -- dynamic application security testing |
| Database | MongoDB with typed collections |
| LLM | LiteLLM (OpenAI-compatible API) |
| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3) |
| LLM | LiteLLM (OpenAI-compatible API for chat, triage, embeddings) |
| Issue Trackers | GitHub (octocrab), GitLab (REST v4), Jira (REST v3), Gitea |
| CVE Sources | OSV.dev, NVD, SearXNG |
| Auth | Keycloak (OAuth2/PKCE, SSO) |
| Browser Automation | Chromium (headless, for pentesting and PDF generation) |
## Getting Started
### Prerequisites
- Rust 1.89+
- Rust 1.94+
- [Dioxus CLI](https://dioxuslabs.com/learn/0.7/getting_started) (`dx`)
- MongoDB
- Docker & Docker Compose (optional)
@@ -151,20 +163,35 @@ The agent exposes a REST API on port 3001:
| `GET` | `/api/v1/sbom` | List dependencies |
| `GET` | `/api/v1/issues` | List cross-tracker issues |
| `GET` | `/api/v1/scan-runs` | Scan execution history |
| `GET` | `/api/v1/graph/:repo_id` | Code knowledge graph |
| `POST` | `/api/v1/graph/:repo_id/build` | Trigger graph build |
| `GET` | `/api/v1/dast/targets` | List DAST targets |
| `POST` | `/api/v1/dast/targets` | Add DAST target |
| `GET` | `/api/v1/dast/findings` | List DAST findings |
| `POST` | `/api/v1/chat/:repo_id` | RAG-powered code chat |
| `POST` | `/api/v1/help/chat` | Documentation-grounded help chat |
| `POST` | `/api/v1/pentest/sessions` | Create pentest session |
| `POST` | `/api/v1/pentest/sessions/:id/export` | Export encrypted pentest report |
| `POST` | `/webhook/github` | GitHub webhook (HMAC-SHA256) |
| `POST` | `/webhook/gitlab` | GitLab webhook (token verify) |
| `POST` | `/webhook/gitea` | Gitea webhook |
## Dashboard Pages
| Page | Description |
|------|-------------|
| **Overview** | Stat cards, severity distribution chart |
| **Repositories** | Add/manage tracked repos, trigger scans |
| **Findings** | Filterable table by severity, type, status |
| **Overview** | Stat cards, severity distribution, AI chat cards, MCP status |
| **Repositories** | Add/manage tracked repos, trigger scans, webhook config |
| **Findings** | Filterable table by severity, type, status, scanner |
| **Finding Detail** | Code evidence, remediation, suggested fix, linked issue |
| **SBOM** | Dependency inventory with vulnerability badges |
| **Issues** | Cross-tracker view (GitHub + GitLab + Jira) |
| **Settings** | Configure LiteLLM, tracker tokens, SearXNG URL |
| **SBOM** | Dependency inventory with vulnerability badges, license summary |
| **Issues** | Cross-tracker view (GitHub + GitLab + Jira + Gitea) |
| **Code Graph** | Interactive architecture visualization, impact analysis |
| **AI Chat** | RAG-powered Q&A about repository code |
| **DAST** | Dynamic scanning targets, findings, and scan history |
| **Pentest** | AI-driven pentest sessions, attack chain visualization |
| **MCP Servers** | Model Context Protocol server management |
| **Help Chat** | Floating assistant (available on every page) for product Q&A |
## Project Structure
@@ -173,19 +200,24 @@ compliance-scanner/
├── compliance-core/ Shared library (models, traits, config, errors)
├── compliance-agent/ Agent daemon (pipeline, LLM, trackers, API, webhooks)
│ └── src/
│ ├── pipeline/ 7-stage scan pipeline
│ ├── llm/ LiteLLM client, triage, descriptions, fixes, PR review
│ ├── trackers/ GitHub, GitLab, Jira integrations
│ ├── api/ REST API (Axum)
── webhooks/ GitHub + GitLab webhook receivers
│ ├── pipeline/ 7-stage scan pipeline, dedup, PR reviews, code review
│ ├── llm/ LiteLLM client, triage, descriptions, fixes, review prompts
│ ├── trackers/ GitHub, GitLab, Jira, Gitea integrations
│ ├── pentest/ AI-driven pentest orchestrator, tools, reports
── rag/ RAG pipeline, chunking, embedding
│ ├── api/ REST API (Axum), help chat
│ └── webhooks/ GitHub, GitLab, Gitea webhook receivers
├── compliance-dashboard/ Dioxus fullstack dashboard
│ └── src/
│ ├── components/ Reusable UI components
│ ├── infrastructure/ Server functions, DB, config
│ └── pages/ Full page views
│ ├── components/ Reusable UI (sidebar, help chat, attack chain, etc.)
│ ├── infrastructure/ Server functions, DB, config, auth
│ └── pages/ Full page views (overview, DAST, pentest, graph, etc.)
├── compliance-graph/ Code knowledge graph (tree-sitter, embeddings, RAG)
├── compliance-dast/ Dynamic application security testing
├── compliance-mcp/ Model Context Protocol server
├── docs/ VitePress documentation site
├── assets/ Static assets (CSS, icons)
── styles/ Tailwind input stylesheet
└── bin/ Dashboard binary entrypoint
── styles/ Tailwind input stylesheet
```
## External Services
@@ -193,10 +225,12 @@ compliance-scanner/
| Service | Purpose | Default URL |
|---------|---------|-------------|
| MongoDB | Persistence | `mongodb://localhost:27017` |
| LiteLLM | LLM proxy for triage and generation | `http://localhost:4000` |
| LiteLLM | LLM proxy (chat, triage, embeddings) | `http://localhost:4000` |
| SearXNG | CVE context search | `http://localhost:8888` |
| Keycloak | Authentication (OAuth2/PKCE, SSO) | `http://localhost:8080` |
| Semgrep | SAST scanning | CLI tool |
| Syft | SBOM generation | CLI tool |
| Chromium | Headless browser (pentesting, PDF) | Managed via Docker |
---
+34 -3
View File
@@ -7,9 +7,19 @@ edition = "2021"
workspace = true
[dependencies]
compliance-core = { workspace = true, features = ["mongodb", "telemetry"] }
compliance-core = { workspace = true, features = ["mongodb", "telemetry", "axum"] }
compliance-graph = { path = "../compliance-graph" }
compliance-dast = { path = "../compliance-dast" }
# Native firmware build/target detection for bare-metal & RTOS artifacts.
# Same-company IP, used directly (not via CLI) so the whole tramiton suite is
# available to the onboarding classifier. NOTE: CI must be able to fetch this
# private repo (see the git-auth step in .gitea/workflows/ci.yml).
tramiton-core = { git = "ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git", tag = "v0.4.1" }
# tramiton-repro drives the reproducible build (NixBackend seal_and_build) that
# yields a sealed lock; `libraries_from_inputs` is the analysis-only fallback.
tramiton-repro = { git = "ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git", tag = "v0.4.1" }
# tramiton-sbom renders the bill of materials from a sealed lock (+ binary SCA).
tramiton-sbom = { git = "ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git", tag = "v0.4.1" }
serde = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true }
@@ -24,15 +34,36 @@ hex = { workspace = true }
uuid = { workspace = true }
secrecy = { workspace = true }
regex = { workspace = true }
axum = "0.8"
tower-http = { version = "0.6", features = ["cors", "trace"] }
axum = { version = "0.8", features = ["multipart"] }
tower-http = { version = "0.6", features = ["cors", "trace", "set-header"] }
git2 = "0.20"
octocrab = "0.44"
tokio-cron-scheduler = "0.13"
dotenvy = "0.15"
hmac = "0.12"
walkdir = "2"
# Read-only XML tree parsing for PLCopen project files (POU extraction).
roxmltree = "0.20"
base64 = "0.22"
urlencoding = "2"
futures-util = "0.3"
jsonwebtoken = "9"
zip = { workspace = true }
aes-gcm = { workspace = true }
tokio-tungstenite = { version = "0.26", features = ["rustls-tls-webpki-roots"] }
futures-core = "0.3"
dashmap = { workspace = true }
tokio-stream = { workspace = true }
rand = { workspace = true }
[dev-dependencies]
compliance-core = { workspace = true, features = ["mongodb", "axum"] }
tower = { version = "0.5", features = ["util"] }
reqwest = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true }
mongodb = { workspace = true }
uuid = { workspace = true }
secrecy = { workspace = true }
axum = { version = "0.8", features = ["multipart"] }
tower-http = { version = "0.6", features = ["cors"] }
+113 -26
View File
@@ -1,77 +1,164 @@
use std::sync::Arc;
use dashmap::DashMap;
use tokio::sync::{broadcast, watch, Semaphore};
use compliance_core::models::pentest::PentestEvent;
use compliance_core::AgentConfig;
use crate::database::Database;
use crate::database::DatabasePool;
use crate::llm::LlmClient;
use crate::pipeline::orchestrator::PipelineOrchestrator;
/// Default maximum concurrent pentest sessions.
const DEFAULT_MAX_CONCURRENT_SESSIONS: usize = 5;
#[derive(Clone)]
pub struct ComplianceAgent {
pub config: AgentConfig,
pub db: Database,
/// Per-tenant Mongo broker. Every code path must obtain a
/// tenant-scoped [`crate::database::Database`] from this pool —
/// there is no single shared database any more.
pub db_pool: DatabasePool,
pub llm: Arc<LlmClient>,
pub http: reqwest::Client,
/// Per-session broadcast senders for SSE streaming.
pub session_streams: Arc<DashMap<String, broadcast::Sender<PentestEvent>>>,
/// Per-session pause controls (true = paused).
pub session_pause: Arc<DashMap<String, watch::Sender<bool>>>,
/// Semaphore limiting concurrent pentest sessions.
pub session_semaphore: Arc<Semaphore>,
}
impl ComplianceAgent {
pub fn new(config: AgentConfig, db: Database) -> Self {
pub fn new(config: AgentConfig, db_pool: DatabasePool) -> Self {
let llm = Arc::new(LlmClient::new(
config.litellm_url.clone(),
config.litellm_api_key.clone(),
config.litellm_model.clone(),
config.litellm_embed_model.clone(),
));
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self {
config,
db,
db_pool,
llm,
http: reqwest::Client::new(),
http,
session_streams: Arc::new(DashMap::new()),
session_pause: Arc::new(DashMap::new()),
session_semaphore: Arc::new(Semaphore::new(DEFAULT_MAX_CONCURRENT_SESSIONS)),
}
}
pub async fn run_scan(
&self,
tenant_id: &str,
repo_id: &str,
trigger: compliance_core::models::ScanTrigger,
) -> Result<(), crate::error::AgentError> {
let orchestrator = PipelineOrchestrator::new(
self.config.clone(),
self.db.clone(),
self.llm.clone(),
self.http.clone(),
);
orchestrator.run(repo_id, trigger).await
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator.run_target(repo_id, trigger).await
}
/// Alias for [`Self::run_scan`] — every scan runs the unified onboarded-target
/// pipeline. Kept as a distinct name for the `/targets/{id}/scan` endpoint's
/// intent.
pub async fn run_target_scan(
&self,
tenant_id: &str,
target_id: &str,
trigger: compliance_core::models::ScanTrigger,
) -> Result<(), crate::error::AgentError> {
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator.run_target(target_id, trigger).await
}
/// Run a PR review: scan the diff and post review comments.
pub async fn run_pr_review(
&self,
tenant_id: &str,
repo_id: &str,
pr_number: u64,
base_sha: &str,
head_sha: &str,
) -> Result<(), crate::error::AgentError> {
let repo = self
.db
.repositories()
.find_one(mongodb::bson::doc! {
"_id": mongodb::bson::oid::ObjectId::parse_str(repo_id)
.map_err(|e| crate::error::AgentError::Other(e.to_string()))?
})
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let oid = mongodb::bson::oid::ObjectId::parse_str(repo_id)
.map_err(|e| crate::error::AgentError::Other(e.to_string()))?;
let target = db
.onboarded_targets()
.find_one(mongodb::bson::doc! { "_id": oid })
.await?
.ok_or_else(|| {
crate::error::AgentError::Other(format!("Repository {repo_id} not found"))
crate::error::AgentError::Other(format!("Target {repo_id} not found"))
})?;
let code = target.code_artifact().ok_or_else(|| {
crate::error::AgentError::Other(format!("Target {repo_id} has no code artifact"))
})?;
let repo = crate::pipeline::repo_view::RepoView::from_target(&target, code);
let orchestrator = PipelineOrchestrator::new(
self.config.clone(),
self.db.clone(),
self.llm.clone(),
self.http.clone(),
);
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator
.run_pr_review(&repo, repo_id, pr_number, base_sha, head_sha)
.await
}
// ── Session stream management ──────────────────────────────────
/// Register a broadcast sender for a session. Returns the sender.
pub fn register_session_stream(&self, session_id: &str) -> broadcast::Sender<PentestEvent> {
let (tx, _) = broadcast::channel(256);
self.session_streams
.insert(session_id.to_string(), tx.clone());
tx
}
/// Subscribe to a session's broadcast stream.
pub fn subscribe_session(&self, session_id: &str) -> Option<broadcast::Receiver<PentestEvent>> {
self.session_streams
.get(session_id)
.map(|tx| tx.subscribe())
}
// ── Session pause/resume management ────────────────────────────
/// Register a pause control for a session. Returns the watch receiver.
pub fn register_pause_control(&self, session_id: &str) -> watch::Receiver<bool> {
let (tx, rx) = watch::channel(false);
self.session_pause.insert(session_id.to_string(), tx);
rx
}
/// Pause a session.
pub fn pause_session(&self, session_id: &str) -> bool {
if let Some(tx) = self.session_pause.get(session_id) {
tx.send(true).is_ok()
} else {
false
}
}
/// Resume a session.
pub fn resume_session(&self, session_id: &str) -> bool {
if let Some(tx) = self.session_pause.get(session_id) {
tx.send(false).is_ok()
} else {
false
}
}
/// Clean up all per-session resources.
pub fn cleanup_session(&self, session_id: &str) {
self.session_streams.remove(session_id);
self.session_pause.remove(session_id);
}
}
-113
View File
@@ -1,113 +0,0 @@
use std::sync::Arc;
use axum::{
extract::Request,
middleware::Next,
response::{IntoResponse, Response},
};
use jsonwebtoken::{decode, decode_header, jwk::JwkSet, DecodingKey, Validation};
use reqwest::StatusCode;
use serde::Deserialize;
use tokio::sync::RwLock;
/// Cached JWKS from Keycloak for token validation.
#[derive(Clone)]
pub struct JwksState {
pub jwks: Arc<RwLock<Option<JwkSet>>>,
pub jwks_url: String,
}
#[derive(Debug, Deserialize)]
struct Claims {
#[allow(dead_code)]
sub: String,
}
const PUBLIC_ENDPOINTS: &[&str] = &["/api/v1/health"];
/// Middleware that validates Bearer JWT tokens against Keycloak's JWKS.
///
/// Skips validation for health check endpoints.
/// If `JwksState` is not present as an extension (keycloak not configured),
/// all requests pass through.
pub async fn require_jwt_auth(request: Request, next: Next) -> Response {
let path = request.uri().path();
if PUBLIC_ENDPOINTS.contains(&path) {
return next.run(request).await;
}
let jwks_state = match request.extensions().get::<JwksState>() {
Some(s) => s.clone(),
None => return next.run(request).await,
};
let auth_header = match request.headers().get("authorization") {
Some(h) => h,
None => return (StatusCode::UNAUTHORIZED, "Missing authorization header").into_response(),
};
let token = match auth_header.to_str() {
Ok(s) if s.starts_with("Bearer ") => &s[7..],
_ => return (StatusCode::UNAUTHORIZED, "Invalid authorization header").into_response(),
};
match validate_token(token, &jwks_state).await {
Ok(()) => next.run(request).await,
Err(e) => {
tracing::warn!("JWT validation failed: {e}");
(StatusCode::UNAUTHORIZED, "Invalid token").into_response()
}
}
}
async fn validate_token(token: &str, state: &JwksState) -> Result<(), String> {
let header = decode_header(token).map_err(|e| format!("failed to decode JWT header: {e}"))?;
let kid = header
.kid
.ok_or_else(|| "JWT missing kid header".to_string())?;
let jwks = fetch_or_get_jwks(state).await?;
let jwk = jwks
.keys
.iter()
.find(|k| k.common.key_id.as_deref() == Some(&kid))
.ok_or_else(|| "no matching key found in JWKS".to_string())?;
let decoding_key =
DecodingKey::from_jwk(jwk).map_err(|e| format!("failed to create decoding key: {e}"))?;
let mut validation = Validation::new(header.alg);
validation.validate_exp = true;
validation.validate_aud = false;
decode::<Claims>(token, &decoding_key, &validation)
.map_err(|e| format!("token validation failed: {e}"))?;
Ok(())
}
async fn fetch_or_get_jwks(state: &JwksState) -> Result<JwkSet, String> {
{
let cached = state.jwks.read().await;
if let Some(ref jwks) = *cached {
return Ok(jwks.clone());
}
}
let resp = reqwest::get(&state.jwks_url)
.await
.map_err(|e| format!("failed to fetch JWKS: {e}"))?;
let jwks: JwkSet = resp
.json()
.await
.map_err(|e| format!("failed to parse JWKS: {e}"))?;
let mut cached = state.jwks.write().await;
*cached = Some(jwks.clone());
Ok(jwks)
}
+115
View File
@@ -0,0 +1,115 @@
//! Cross-tenant admin endpoints (`/api/v1/admin/*`).
//!
//! Operator-only. Auth is a **static bearer token** (`ADMIN_API_TOKEN`
//! env on the agent) — explicitly NOT a Keycloak JWT, because the
//! whole point of these endpoints is to operate ACROSS tenants. A
//! customer JWT (which always carries a single tenant_id) has no
//! business mounting them.
//!
//! Routes are only registered when `ADMIN_API_TOKEN` is set. With no
//! token, the endpoints don't exist at all (404), which is a stronger
//! guarantee than "401 if you guess the path".
//!
//! Operations:
//! - `GET /api/v1/admin/tenants` — list tenant DBs
//! - `DELETE /api/v1/admin/tenants/{tenant_id}` — GDPR delete
//!
//! Tenant ids in URLs are passed as-is to `DatabasePool::drop_tenant`,
//! which sanitises them the same way it does for creation. Listing
//! returns the raw DB names from `list_tenant_db_names` — operators
//! can reverse-derive the tenant_id from the prefix.
use axum::extract::{Extension, Path, Request};
use axum::http::{header, StatusCode};
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
use axum::Json;
use secrecy::ExposeSecret;
use serde::Serialize;
use super::dto::AgentExt;
#[derive(Serialize)]
pub struct ListTenantDbsResponse {
pub tenant_db_names: Vec<String>,
}
#[tracing::instrument(skip_all)]
pub async fn list_tenant_dbs(
Extension(agent): AgentExt,
) -> Result<Json<ListTenantDbsResponse>, StatusCode> {
let names = agent.db_pool.list_tenant_db_names().await.map_err(|e| {
tracing::error!("admin: list_tenant_db_names failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
Ok(Json(ListTenantDbsResponse {
tenant_db_names: names,
}))
}
#[tracing::instrument(skip_all, fields(tenant_id = %tenant_id))]
pub async fn drop_tenant_db(
Extension(agent): AgentExt,
Path(tenant_id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
agent.db_pool.drop_tenant(&tenant_id).await.map_err(|e| {
tracing::error!("admin: drop_tenant failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
Ok(Json(serde_json::json!({ "status": "dropped" })))
}
/// Constant-time-ish comparison of the configured admin token against
/// the incoming bearer. Uses `subtle`-style byte equality so timing
/// attacks can't probe the token character by character.
fn tokens_eq(a: &str, b: &str) -> bool {
if a.len() != b.len() {
return false;
}
let mut diff = 0u8;
for (x, y) in a.bytes().zip(b.bytes()) {
diff |= x ^ y;
}
diff == 0
}
/// Middleware enforcing the static `ADMIN_API_TOKEN`. Mounted only on
/// the admin sub-router, so this never runs on customer routes.
pub async fn require_admin_token(
Extension(agent): AgentExt,
request: Request,
next: Next,
) -> Response {
let Some(expected) = agent.config.admin_api_token.as_ref() else {
// Belt-and-braces — if the routes were somehow mounted without
// a token configured, refuse rather than no-op-pass.
return (StatusCode::NOT_FOUND, "admin disabled").into_response();
};
let presented = request
.headers()
.get(header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|s| s.strip_prefix("Bearer "))
.map(|s| s.trim());
let Some(presented) = presented.filter(|s| !s.is_empty()) else {
return (StatusCode::UNAUTHORIZED, "Missing bearer token").into_response();
};
if !tokens_eq(presented, expected.expose_secret()) {
return (StatusCode::UNAUTHORIZED, "Invalid admin token").into_response();
}
next.run(request).await
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tokens_eq_basic() {
assert!(tokens_eq("abc", "abc"));
assert!(!tokens_eq("abc", "abd"));
assert!(!tokens_eq("abc", "abcd"));
assert!(!tokens_eq("", "x"));
assert!(tokens_eq("", ""));
}
}
+47 -32
View File
@@ -7,11 +7,13 @@ use mongodb::bson::doc;
use compliance_core::models::chat::{ChatRequest, ChatResponse, SourceReference};
use compliance_core::models::embedding::EmbeddingBuildRun;
use compliance_core::tenant_ctx::TenantCtx;
use compliance_graph::graph::embedding_store::EmbeddingStore;
use crate::agent::ComplianceAgent;
use crate::rag::pipeline::RagPipeline;
use super::dto::tenant_db;
use super::ApiResponse;
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -20,10 +22,12 @@ type AgentExt = Extension<Arc<ComplianceAgent>>;
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn chat(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
Json(req): Json<ChatRequest>,
) -> Result<Json<ApiResponse<ChatResponse>>, StatusCode> {
let pipeline = RagPipeline::new(agent.llm.clone(), agent.db.inner());
let db = tenant_db(&agent, &tenant).await?;
let pipeline = RagPipeline::new(agent.llm.clone(), db.inner());
// Step 1: Embed the user's message
let query_vectors = agent
@@ -90,10 +94,13 @@ pub async fn chat(
};
let system_prompt = format!(
"You are an expert code assistant for a software repository. \
Answer the user's question based on the code context below. \
Reference specific files and functions when relevant. \
If the context doesn't contain enough information, say so.\n\n\
"You are a code assistant for this repository. Answer questions using the code context below.\n\n\
Rules:\n\
- Reference specific files, functions, and line numbers\n\
- Show code snippets when they help explain the answer\n\
- If the context is insufficient, say what's missing rather than guessing\n\
- Be concise — lead with the answer, then explain if needed\n\
- For security questions, note relevant CWEs and link to the finding if one exists\n\n\
## Code Context\n\n{code_context}"
);
@@ -130,13 +137,16 @@ pub async fn chat(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn build_embeddings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
// Resolve the tenant DB up front so we can move it into the spawn;
// the JWT/dev context isn't available inside detached tasks.
let db = tenant_db(&agent, &tenant).await?;
let agent_clone = (*agent).clone();
tokio::spawn(async move {
let repo = match agent_clone
.db
.repositories()
let repo = match db
.onboarded_targets()
.find_one(doc! { "_id": mongodb::bson::oid::ObjectId::parse_str(&repo_id).ok() })
.await
{
@@ -148,8 +158,7 @@ pub async fn build_embeddings(
};
// Get latest graph build
let build = match agent_clone
.db
let build = match db
.graph_builds()
.find_one(doc! { "repo_id": &repo_id })
.sort(doc! { "started_at": -1 })
@@ -168,35 +177,39 @@ pub async fn build_embeddings(
.unwrap_or_else(|| "unknown".to_string());
// Get nodes
let nodes: Vec<compliance_core::models::graph::CodeNode> = match agent_clone
.db
.graph_nodes()
.find(doc! { "repo_id": &repo_id })
.await
{
Ok(cursor) => {
use futures_util::StreamExt;
let mut items = Vec::new();
let mut cursor = cursor;
while let Some(Ok(item)) = cursor.next().await {
items.push(item);
let nodes: Vec<compliance_core::models::graph::CodeNode> =
match db.graph_nodes().find(doc! { "repo_id": &repo_id }).await {
Ok(cursor) => {
use futures_util::StreamExt;
let mut items = Vec::new();
let mut cursor = cursor;
while let Some(Ok(item)) = cursor.next().await {
items.push(item);
}
items
}
items
}
Err(e) => {
tracing::error!("[{repo_id}] Failed to fetch nodes: {e}");
Err(e) => {
tracing::error!("[{repo_id}] Failed to fetch nodes: {e}");
return;
}
};
let code = match repo.code_artifact() {
Some(c) => c,
None => {
tracing::error!("Target {repo_id} has no code artifact for embedding build");
return;
}
};
let view = crate::pipeline::repo_view::RepoView::from_target(&repo, code);
let creds = crate::pipeline::git::RepoCredentials {
ssh_key_path: Some(agent_clone.config.ssh_key_path.clone()),
auth_token: repo.auth_token.clone(),
auth_username: repo.auth_username.clone(),
auth_token: view.auth_token.clone(),
auth_username: view.auth_username.clone(),
};
let git_ops =
crate::pipeline::git::GitOps::new(&agent_clone.config.git_clone_base_path, creds);
let repo_path = match git_ops.clone_or_fetch(&repo.git_url, &repo.name) {
let repo_path = match git_ops.clone_or_fetch(&view.git_url, &view.name) {
Ok(p) => p,
Err(e) => {
tracing::error!("Failed to clone repo for embedding build: {e}");
@@ -204,7 +217,7 @@ pub async fn build_embeddings(
}
};
let pipeline = RagPipeline::new(agent_clone.llm.clone(), agent_clone.db.inner());
let pipeline = RagPipeline::new(agent_clone.llm.clone(), db.inner());
match pipeline
.build_embeddings(&repo_id, &repo_path, &graph_build_id, &nodes)
.await
@@ -231,9 +244,11 @@ pub async fn build_embeddings(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn embedding_status(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<Option<EmbeddingBuildRun>>>, StatusCode> {
let store = EmbeddingStore::new(agent.db.inner());
let db = tenant_db(&agent, &tenant).await?;
let store = EmbeddingStore::new(db.inner());
let build = store.get_latest_build(&repo_id).await.map_err(|e| {
tracing::error!("Failed to get embedding status: {e}");
StatusCode::INTERNAL_SERVER_ERROR
+20 -11
View File
@@ -7,9 +7,11 @@ use mongodb::bson::doc;
use serde::Deserialize;
use compliance_core::models::dast::{DastFinding, DastScanRun, DastTarget, DastTargetType};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -45,9 +47,11 @@ fn default_rate_limit() -> u32 {
#[tracing::instrument(skip_all)]
pub async fn list_targets(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<DastTarget>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_targets()
@@ -80,6 +84,7 @@ pub async fn list_targets(
#[tracing::instrument(skip_all)]
pub async fn add_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<AddTargetRequest>,
) -> Result<Json<ApiResponse<DastTarget>>, StatusCode> {
let mut target = DastTarget::new(req.name, req.base_url, req.target_type);
@@ -89,9 +94,8 @@ pub async fn add_target(
target.rate_limit = req.rate_limit;
target.allow_destructive = req.allow_destructive;
agent
.db
.dast_targets()
let db = tenant_db(&agent, &tenant).await?;
db.dast_targets()
.insert_one(&target)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
@@ -107,19 +111,19 @@ pub async fn add_target(
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn trigger_scan(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let target = agent
.db
let target = db
.dast_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
let db = agent.db.clone();
tokio::spawn(async move {
let orchestrator = compliance_dast::DastOrchestrator::new(100);
match orchestrator.run_scan(&target, Vec::new()).await {
@@ -147,9 +151,11 @@ pub async fn trigger_scan(
#[tracing::instrument(skip_all)]
pub async fn list_scan_runs(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<DastScanRun>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_scan_runs()
@@ -183,9 +189,11 @@ pub async fn list_scan_runs(
#[tracing::instrument(skip_all)]
pub async fn list_findings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<DastFinding>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_findings()
@@ -219,12 +227,13 @@ pub async fn list_findings(
#[tracing::instrument(skip_all, fields(finding_id = %id))]
pub async fn get_finding(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<DastFinding>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let finding = agent
.db
let finding = db
.dast_findings()
.find_one(doc! { "_id": oid })
.await
+502
View File
@@ -0,0 +1,502 @@
use compliance_core::models::TrackerType;
use serde::{Deserialize, Serialize};
use compliance_core::models::ScanRun;
#[derive(Deserialize)]
pub struct PaginationParams {
#[serde(default = "default_page")]
pub page: u64,
#[serde(default = "default_limit")]
pub limit: i64,
}
pub(crate) fn default_page() -> u64 {
1
}
pub(crate) fn default_limit() -> i64 {
50
}
#[derive(Deserialize)]
pub struct FindingsFilter {
#[serde(default)]
pub repo_id: Option<String>,
#[serde(default)]
pub severity: Option<String>,
#[serde(default)]
pub scan_type: Option<String>,
#[serde(default)]
pub status: Option<String>,
#[serde(default)]
pub q: Option<String>,
#[serde(default)]
pub sort_by: Option<String>,
#[serde(default)]
pub sort_order: Option<String>,
#[serde(default = "default_page")]
pub page: u64,
#[serde(default = "default_limit")]
pub limit: i64,
}
#[derive(Serialize)]
pub struct ApiResponse<T: Serialize> {
pub data: T,
#[serde(skip_serializing_if = "Option::is_none")]
pub total: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub page: Option<u64>,
}
#[derive(Serialize)]
pub struct OverviewStats {
pub total_repositories: u64,
pub total_findings: u64,
pub critical_findings: u64,
pub high_findings: u64,
pub medium_findings: u64,
pub low_findings: u64,
pub total_sbom_entries: u64,
pub total_cve_alerts: u64,
pub total_issues: u64,
pub recent_scans: Vec<ScanRun>,
}
#[derive(Deserialize)]
pub struct AddRepositoryRequest {
pub name: String,
pub git_url: String,
#[serde(default = "default_branch")]
pub default_branch: String,
pub auth_token: Option<String>,
pub auth_username: Option<String>,
pub tracker_type: Option<TrackerType>,
pub tracker_owner: Option<String>,
pub tracker_repo: Option<String>,
pub tracker_token: Option<String>,
pub scan_schedule: Option<String>,
}
#[derive(Deserialize)]
pub struct UpdateRepositoryRequest {
pub name: Option<String>,
pub default_branch: Option<String>,
pub auth_token: Option<String>,
pub auth_username: Option<String>,
pub tracker_type: Option<TrackerType>,
pub tracker_owner: Option<String>,
pub tracker_repo: Option<String>,
pub tracker_token: Option<String>,
pub scan_schedule: Option<String>,
}
fn default_branch() -> String {
"main".to_string()
}
#[derive(Deserialize)]
pub struct UpdateStatusRequest {
pub status: String,
}
#[derive(Deserialize)]
pub struct BulkUpdateStatusRequest {
pub ids: Vec<String>,
pub status: String,
}
#[derive(Deserialize)]
pub struct UpdateFeedbackRequest {
pub feedback: String,
}
#[derive(Deserialize)]
pub struct SbomFilter {
#[serde(default)]
pub repo_id: Option<String>,
#[serde(default)]
pub package_manager: Option<String>,
#[serde(default)]
pub q: Option<String>,
#[serde(default)]
pub has_vulns: Option<bool>,
#[serde(default)]
pub license: Option<String>,
#[serde(default = "default_page")]
pub page: u64,
#[serde(default = "default_limit")]
pub limit: i64,
}
#[derive(Deserialize)]
pub struct SbomExportParams {
pub repo_id: String,
#[serde(default = "default_export_format")]
pub format: String,
}
fn default_export_format() -> String {
"cyclonedx".to_string()
}
#[derive(Deserialize)]
pub struct SbomDiffParams {
pub repo_a: String,
pub repo_b: String,
}
#[derive(Serialize)]
pub struct LicenseSummary {
pub license: String,
pub count: u64,
pub is_copyleft: bool,
pub packages: Vec<String>,
}
#[derive(Serialize)]
pub struct SbomDiffResult {
pub only_in_a: Vec<SbomDiffEntry>,
pub only_in_b: Vec<SbomDiffEntry>,
pub version_changed: Vec<SbomVersionDiff>,
pub common_count: u64,
}
#[derive(Serialize)]
pub struct SbomDiffEntry {
pub name: String,
pub version: String,
pub package_manager: String,
}
#[derive(Serialize)]
pub struct SbomVersionDiff {
pub name: String,
pub package_manager: String,
pub version_a: String,
pub version_b: String,
}
pub(crate) type AgentExt = axum::extract::Extension<std::sync::Arc<crate::agent::ComplianceAgent>>;
pub(crate) type ApiResult<T> = Result<axum::Json<ApiResponse<T>>, axum::http::StatusCode>;
/// Resolve a tenant-scoped [`Database`] from the request's
/// [`TenantContext`] (inserted by the M7.1 JWT middleware, or by the
/// dev fallback in unsecured environments). The pool ensures the
/// tenant's indexes idempotently.
///
/// Returns 500 on the rare path where Mongo refuses the database
/// handle — the M7.1 auth/status middleware already rejects every
/// other failure mode with 4xx before we get here.
pub(crate) async fn tenant_db(
agent: &crate::agent::ComplianceAgent,
tenant: &compliance_core::tenant_ctx::TenantCtx,
) -> Result<crate::database::Database, axum::http::StatusCode> {
agent.db_pool.for_tenant(&tenant.0).await.map_err(|e| {
tracing::error!(
tenant_id = %tenant.0.tenant_id,
"Failed to acquire tenant database: {e}"
);
axum::http::StatusCode::INTERNAL_SERVER_ERROR
})
}
pub(crate) async fn collect_cursor_async<T: serde::de::DeserializeOwned + Unpin + Send>(
mut cursor: mongodb::Cursor<T>,
) -> Vec<T> {
use futures_util::StreamExt;
let mut items = Vec::new();
while let Some(result) = cursor.next().await {
match result {
Ok(item) => items.push(item),
Err(e) => tracing::warn!("Failed to deserialize document: {e}"),
}
}
items
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
// ── PaginationParams ─────────────────────────────────────────
#[test]
fn pagination_params_defaults() {
let p: PaginationParams = serde_json::from_str("{}").unwrap();
assert_eq!(p.page, 1);
assert_eq!(p.limit, 50);
}
#[test]
fn pagination_params_custom_values() {
let p: PaginationParams = serde_json::from_str(r#"{"page":3,"limit":10}"#).unwrap();
assert_eq!(p.page, 3);
assert_eq!(p.limit, 10);
}
#[test]
fn pagination_params_partial_override() {
let p: PaginationParams = serde_json::from_str(r#"{"page":5}"#).unwrap();
assert_eq!(p.page, 5);
assert_eq!(p.limit, 50);
}
#[test]
fn pagination_params_zero_page() {
let p: PaginationParams = serde_json::from_str(r#"{"page":0}"#).unwrap();
assert_eq!(p.page, 0);
}
// ── FindingsFilter ───────────────────────────────────────────
#[test]
fn findings_filter_all_defaults() {
let f: FindingsFilter = serde_json::from_str("{}").unwrap();
assert!(f.repo_id.is_none());
assert!(f.severity.is_none());
assert!(f.scan_type.is_none());
assert!(f.status.is_none());
assert!(f.q.is_none());
assert!(f.sort_by.is_none());
assert!(f.sort_order.is_none());
assert_eq!(f.page, 1);
assert_eq!(f.limit, 50);
}
#[test]
fn findings_filter_with_all_fields() {
let f: FindingsFilter = serde_json::from_str(
r#"{
"repo_id": "abc",
"severity": "high",
"scan_type": "sast",
"status": "open",
"q": "sql injection",
"sort_by": "severity",
"sort_order": "desc",
"page": 2,
"limit": 25
}"#,
)
.unwrap();
assert_eq!(f.repo_id.as_deref(), Some("abc"));
assert_eq!(f.severity.as_deref(), Some("high"));
assert_eq!(f.scan_type.as_deref(), Some("sast"));
assert_eq!(f.status.as_deref(), Some("open"));
assert_eq!(f.q.as_deref(), Some("sql injection"));
assert_eq!(f.sort_by.as_deref(), Some("severity"));
assert_eq!(f.sort_order.as_deref(), Some("desc"));
assert_eq!(f.page, 2);
assert_eq!(f.limit, 25);
}
#[test]
fn findings_filter_empty_string_fields() {
let f: FindingsFilter = serde_json::from_str(r#"{"repo_id":"","severity":""}"#).unwrap();
assert_eq!(f.repo_id.as_deref(), Some(""));
assert_eq!(f.severity.as_deref(), Some(""));
}
// ── ApiResponse ──────────────────────────────────────────────
#[test]
fn api_response_serializes_with_all_fields() {
let resp = ApiResponse {
data: vec!["a", "b"],
total: Some(100),
page: Some(1),
};
let v = serde_json::to_value(&resp).unwrap();
assert_eq!(v["data"], json!(["a", "b"]));
assert_eq!(v["total"], 100);
assert_eq!(v["page"], 1);
}
#[test]
fn api_response_skips_none_fields() {
let resp = ApiResponse {
data: "hello",
total: None,
page: None,
};
let v = serde_json::to_value(&resp).unwrap();
assert_eq!(v["data"], "hello");
assert!(v.get("total").is_none());
assert!(v.get("page").is_none());
}
#[test]
fn api_response_with_nested_struct() {
#[derive(Serialize)]
struct Item {
id: u32,
}
let resp = ApiResponse {
data: Item { id: 42 },
total: Some(1),
page: None,
};
let v = serde_json::to_value(&resp).unwrap();
assert_eq!(v["data"]["id"], 42);
assert_eq!(v["total"], 1);
assert!(v.get("page").is_none());
}
#[test]
fn api_response_empty_vec() {
let resp: ApiResponse<Vec<String>> = ApiResponse {
data: vec![],
total: Some(0),
page: Some(1),
};
let v = serde_json::to_value(&resp).unwrap();
assert!(v["data"].as_array().unwrap().is_empty());
}
// ── SbomFilter ───────────────────────────────────────────────
#[test]
fn sbom_filter_defaults() {
let f: SbomFilter = serde_json::from_str("{}").unwrap();
assert!(f.repo_id.is_none());
assert!(f.package_manager.is_none());
assert!(f.q.is_none());
assert!(f.has_vulns.is_none());
assert!(f.license.is_none());
assert_eq!(f.page, 1);
assert_eq!(f.limit, 50);
}
#[test]
fn sbom_filter_has_vulns_bool() {
let f: SbomFilter = serde_json::from_str(r#"{"has_vulns": true}"#).unwrap();
assert_eq!(f.has_vulns, Some(true));
}
// ── SbomExportParams ─────────────────────────────────────────
#[test]
fn sbom_export_params_default_format() {
let p: SbomExportParams = serde_json::from_str(r#"{"repo_id":"r1"}"#).unwrap();
assert_eq!(p.repo_id, "r1");
assert_eq!(p.format, "cyclonedx");
}
#[test]
fn sbom_export_params_custom_format() {
let p: SbomExportParams =
serde_json::from_str(r#"{"repo_id":"r1","format":"spdx"}"#).unwrap();
assert_eq!(p.format, "spdx");
}
// ── AddRepositoryRequest ─────────────────────────────────────
#[test]
fn add_repository_request_defaults() {
let r: AddRepositoryRequest = serde_json::from_str(
r#"{
"name": "my-repo",
"git_url": "https://github.com/x/y.git"
}"#,
)
.unwrap();
assert_eq!(r.name, "my-repo");
assert_eq!(r.git_url, "https://github.com/x/y.git");
assert_eq!(r.default_branch, "main");
assert!(r.auth_token.is_none());
assert!(r.tracker_type.is_none());
assert!(r.scan_schedule.is_none());
}
#[test]
fn add_repository_request_custom_branch() {
let r: AddRepositoryRequest = serde_json::from_str(
r#"{
"name": "repo",
"git_url": "url",
"default_branch": "develop"
}"#,
)
.unwrap();
assert_eq!(r.default_branch, "develop");
}
// ── UpdateStatusRequest / BulkUpdateStatusRequest ────────────
#[test]
fn update_status_request() {
let r: UpdateStatusRequest = serde_json::from_str(r#"{"status":"resolved"}"#).unwrap();
assert_eq!(r.status, "resolved");
}
#[test]
fn bulk_update_status_request() {
let r: BulkUpdateStatusRequest =
serde_json::from_str(r#"{"ids":["a","b"],"status":"dismissed"}"#).unwrap();
assert_eq!(r.ids, vec!["a", "b"]);
assert_eq!(r.status, "dismissed");
}
#[test]
fn bulk_update_status_empty_ids() {
let r: BulkUpdateStatusRequest =
serde_json::from_str(r#"{"ids":[],"status":"x"}"#).unwrap();
assert!(r.ids.is_empty());
}
// ── SbomDiffResult serialization ─────────────────────────────
#[test]
fn sbom_diff_result_serializes() {
let r = SbomDiffResult {
only_in_a: vec![SbomDiffEntry {
name: "pkg-a".to_string(),
version: "1.0".to_string(),
package_manager: "npm".to_string(),
}],
only_in_b: vec![],
version_changed: vec![SbomVersionDiff {
name: "shared".to_string(),
package_manager: "cargo".to_string(),
version_a: "0.1".to_string(),
version_b: "0.2".to_string(),
}],
common_count: 10,
};
let v = serde_json::to_value(&r).unwrap();
assert_eq!(v["only_in_a"].as_array().unwrap().len(), 1);
assert_eq!(v["only_in_b"].as_array().unwrap().len(), 0);
assert_eq!(v["version_changed"][0]["version_a"], "0.1");
assert_eq!(v["common_count"], 10);
}
// ── LicenseSummary ───────────────────────────────────────────
#[test]
fn license_summary_serializes() {
let ls = LicenseSummary {
license: "MIT".to_string(),
count: 42,
is_copyleft: false,
packages: vec!["serde".to_string()],
};
let v = serde_json::to_value(&ls).unwrap();
assert_eq!(v["license"], "MIT");
assert_eq!(v["is_copyleft"], false);
assert_eq!(v["count"], 42);
}
// ── Default helper functions ─────────────────────────────────
#[test]
fn default_page_returns_1() {
assert_eq!(default_page(), 1);
}
#[test]
fn default_limit_returns_50() {
assert_eq!(default_limit(), 50);
}
}
@@ -0,0 +1,177 @@
use axum::extract::{Extension, Path, Query};
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::Finding;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all, fields(repo_id = ?filter.repo_id, severity = ?filter.severity, scan_type = ?filter.scan_type))]
pub async fn list_findings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(filter): Query<FindingsFilter>,
) -> ApiResult<Vec<Finding>> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let mut query = doc! {};
if let Some(repo_id) = &filter.repo_id {
query.insert("repo_id", repo_id);
}
if let Some(severity) = &filter.severity {
query.insert("severity", severity);
}
if let Some(scan_type) = &filter.scan_type {
query.insert("scan_type", scan_type);
}
if let Some(status) = &filter.status {
query.insert("status", status);
}
// Text search across title, description, file_path, rule_id
if let Some(q) = &filter.q {
if !q.is_empty() {
let regex = doc! { "$regex": q, "$options": "i" };
query.insert(
"$or",
mongodb::bson::bson!([
{ "title": regex.clone() },
{ "description": regex.clone() },
{ "file_path": regex.clone() },
{ "rule_id": regex },
]),
);
}
}
// Dynamic sort
let sort_field = filter.sort_by.as_deref().unwrap_or("created_at");
let sort_dir: i32 = match filter.sort_order.as_deref() {
Some("asc") => 1,
_ => -1,
};
let sort_doc = doc! { sort_field: sort_dir };
let skip = (filter.page.saturating_sub(1)) * filter.limit as u64;
let total = db
.findings()
.count_documents(query.clone())
.await
.unwrap_or(0);
let findings = match db
.findings()
.find(query)
.sort(sort_doc)
.skip(skip)
.limit(filter.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch findings: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: findings,
total: Some(total),
page: Some(filter.page),
}))
}
#[tracing::instrument(skip_all, fields(finding_id = %id))]
pub async fn get_finding(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<Finding>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let finding = db
.findings()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
Ok(Json(ApiResponse {
data: finding,
total: None,
page: None,
}))
}
#[tracing::instrument(skip_all, fields(finding_id = %id))]
pub async fn update_finding_status(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateStatusRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
db.findings()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "status": &req.status, "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
Ok(Json(serde_json::json!({ "status": "updated" })))
}
#[tracing::instrument(skip_all)]
pub async fn bulk_update_finding_status(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<BulkUpdateStatusRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oids: Vec<mongodb::bson::oid::ObjectId> = req
.ids
.iter()
.filter_map(|id| mongodb::bson::oid::ObjectId::parse_str(id).ok())
.collect();
if oids.is_empty() {
return Err(StatusCode::BAD_REQUEST);
}
let db = tenant_db(&agent, &tenant).await?;
let result = db
.findings()
.update_many(
doc! { "_id": { "$in": oids } },
doc! { "$set": { "status": &req.status, "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
Ok(Json(
serde_json::json!({ "status": "updated", "modified_count": result.modified_count }),
))
}
#[tracing::instrument(skip_all)]
pub async fn update_finding_feedback(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateFeedbackRequest>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
db.findings()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "developer_feedback": &req.feedback, "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
Ok(Json(serde_json::json!({ "status": "updated" })))
}
+37 -15
View File
@@ -7,9 +7,11 @@ use mongodb::bson::doc;
use serde::{Deserialize, Serialize};
use compliance_core::models::graph::{CodeEdge, CodeNode, GraphBuildRun, ImpactAnalysis};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse};
type AgentExt = Extension<Arc<ComplianceAgent>>;
@@ -36,9 +38,11 @@ fn default_search_limit() -> usize {
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_graph(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<GraphData>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Get latest build
let build: Option<GraphBuildRun> = db
@@ -98,9 +102,11 @@ pub async fn get_graph(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_nodes(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<Vec<CodeNode>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let filter = doc! { "repo_id": &repo_id };
let nodes: Vec<CodeNode> = match db.graph_nodes().find(filter).await {
@@ -123,9 +129,11 @@ pub async fn get_nodes(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_communities(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<ApiResponse<Vec<CommunityInfo>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let filter = doc! { "repo_id": &repo_id };
let nodes: Vec<CodeNode> = match db.graph_nodes().find(filter).await {
@@ -176,9 +184,11 @@ pub struct CommunityInfo {
#[tracing::instrument(skip_all, fields(repo_id = %repo_id, finding_id = %finding_id))]
pub async fn get_impact(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path((repo_id, finding_id)): Path<(String, String)>,
) -> Result<Json<ApiResponse<Option<ImpactAnalysis>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let filter = doc! { "repo_id": &repo_id, "finding_id": &finding_id };
let impact = db
@@ -198,10 +208,12 @@ pub async fn get_impact(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id, query = %params.q))]
pub async fn search_symbols(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
Query(params): Query<SearchParams>,
) -> Result<Json<ApiResponse<Vec<CodeNode>>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Simple text search on qualified_name and name fields
let filter = doc! {
@@ -234,14 +246,16 @@ pub async fn search_symbols(
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn get_file_content(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
Query(params): Query<FileContentParams>,
) -> Result<Json<ApiResponse<FileContent>>, StatusCode> {
let db = &agent.db;
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
// Look up the repository to get repo name
let repo = db
.repositories()
.onboarded_targets()
.find_one(doc! { "_id": mongodb::bson::oid::ObjectId::parse_str(&repo_id).ok() })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
@@ -296,13 +310,14 @@ pub struct FileContent {
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub async fn trigger_build(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(repo_id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let agent_clone = (*agent).clone();
tokio::spawn(async move {
let repo = match agent_clone
.db
.repositories()
let repo = match db
.onboarded_targets()
.find_one(doc! { "_id": mongodb::bson::oid::ObjectId::parse_str(&repo_id).ok() })
.await
{
@@ -313,14 +328,22 @@ pub async fn trigger_build(
}
};
let code = match repo.code_artifact() {
Some(c) => c,
None => {
tracing::error!("Target {repo_id} has no code artifact for graph build");
return;
}
};
let view = crate::pipeline::repo_view::RepoView::from_target(&repo, code);
let creds = crate::pipeline::git::RepoCredentials {
ssh_key_path: Some(agent_clone.config.ssh_key_path.clone()),
auth_token: repo.auth_token.clone(),
auth_username: repo.auth_username.clone(),
auth_token: view.auth_token.clone(),
auth_username: view.auth_username.clone(),
};
let git_ops =
crate::pipeline::git::GitOps::new(&agent_clone.config.git_clone_base_path, creds);
let repo_path = match git_ops.clone_or_fetch(&repo.git_url, &repo.name) {
let repo_path = match git_ops.clone_or_fetch(&view.git_url, &view.name) {
Ok(p) => p,
Err(e) => {
tracing::error!("Failed to clone repo for graph build: {e}");
@@ -333,8 +356,7 @@ pub async fn trigger_build(
match engine.build_graph(&repo_path, &repo_id, &graph_build_id) {
Ok((code_graph, build_run)) => {
let store =
compliance_graph::graph::persistence::GraphStore::new(agent_clone.db.inner());
let store = compliance_graph::graph::persistence::GraphStore::new(db.inner());
let _ = store.delete_repo_graph(&repo_id).await;
let _ = store
.store_graph(&build_run, &code_graph.nodes, &code_graph.edges)
+101
View File
@@ -0,0 +1,101 @@
use axum::Json;
use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::ScanRun;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn health() -> Json<serde_json::Value> {
Json(serde_json::json!({ "status": "ok" }))
}
/// GET /api/v1/settings/ssh-public-key — the agent's SSH deploy public key,
/// for adding as a read-only deploy key on private git targets.
#[tracing::instrument(skip_all)]
pub async fn get_ssh_public_key(
axum::extract::Extension(agent): AgentExt,
) -> Result<Json<serde_json::Value>, axum::http::StatusCode> {
let public_path = format!("{}.pub", agent.config.ssh_key_path);
let public_key =
std::fs::read_to_string(&public_path).map_err(|_| axum::http::StatusCode::NOT_FOUND)?;
Ok(Json(serde_json::json!({ "public_key": public_key.trim() })))
}
#[tracing::instrument(skip_all)]
pub async fn stats_overview(
axum::extract::Extension(agent): AgentExt,
tenant: TenantCtx,
) -> ApiResult<OverviewStats> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let total_repositories = db
.onboarded_targets()
.count_documents(doc! {})
.await
.unwrap_or(0);
let total_findings = db.findings().count_documents(doc! {}).await.unwrap_or(0);
let critical_findings = db
.findings()
.count_documents(doc! { "severity": "critical" })
.await
.unwrap_or(0);
let high_findings = db
.findings()
.count_documents(doc! { "severity": "high" })
.await
.unwrap_or(0);
let medium_findings = db
.findings()
.count_documents(doc! { "severity": "medium" })
.await
.unwrap_or(0);
let low_findings = db
.findings()
.count_documents(doc! { "severity": "low" })
.await
.unwrap_or(0);
let total_sbom_entries = db
.sbom_entries()
.count_documents(doc! {})
.await
.unwrap_or(0);
let total_cve_alerts = db.cve_alerts().count_documents(doc! {}).await.unwrap_or(0);
let total_issues = db
.tracker_issues()
.count_documents(doc! {})
.await
.unwrap_or(0);
let recent_scans: Vec<ScanRun> = match db
.scan_runs()
.find(doc! {})
.sort(doc! { "started_at": -1 })
.limit(10)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch recent scans: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: OverviewStats {
total_repositories,
total_findings,
critical_findings,
high_findings,
medium_findings,
low_findings,
total_sbom_entries,
total_cve_alerts,
total_issues,
recent_scans,
},
total: None,
page: None,
}))
}
@@ -0,0 +1,217 @@
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
use axum::extract::Extension;
use axum::http::StatusCode;
use axum::Json;
use serde::{Deserialize, Serialize};
use walkdir::WalkDir;
use super::dto::{AgentExt, ApiResponse};
// ── DTOs ─────────────────────────────────────────────────────────────────────
#[derive(Debug, Deserialize)]
pub struct HelpChatMessage {
pub role: String,
pub content: String,
}
#[derive(Debug, Deserialize)]
pub struct HelpChatRequest {
pub message: String,
#[serde(default)]
pub history: Vec<HelpChatMessage>,
}
#[derive(Debug, Serialize)]
pub struct HelpChatResponse {
pub message: String,
}
// ── Doc cache ────────────────────────────────────────────────────────────────
static DOC_CONTEXT: OnceLock<String> = OnceLock::new();
/// Walk upward from `start` until we find a directory containing both
/// `README.md` and a `docs/` subdirectory.
fn find_project_root(start: &Path) -> Option<PathBuf> {
let mut current = start.to_path_buf();
loop {
if current.join("README.md").is_file() && current.join("docs").is_dir() {
return Some(current);
}
if !current.pop() {
return None;
}
}
}
/// Read README.md + all docs/**/*.md (excluding node_modules).
fn load_docs(root: &Path) -> String {
let mut parts: Vec<String> = Vec::new();
// Root README first
if let Ok(content) = std::fs::read_to_string(root.join("README.md")) {
parts.push(format!("<!-- file: README.md -->\n{content}"));
}
// docs/**/*.md, skipping node_modules
for entry in WalkDir::new(root.join("docs"))
.follow_links(false)
.into_iter()
.filter_entry(|e| {
!e.path()
.components()
.any(|c| c.as_os_str() == "node_modules")
})
.filter_map(|e| e.ok())
{
let path = entry.path();
if !path.is_file() {
continue;
}
if path
.extension()
.and_then(|s| s.to_str())
.map(|s| !s.eq_ignore_ascii_case("md"))
.unwrap_or(true)
{
continue;
}
let rel = path.strip_prefix(root).unwrap_or(path);
if let Ok(content) = std::fs::read_to_string(path) {
parts.push(format!("<!-- file: {} -->\n{content}", rel.display()));
}
}
if parts.is_empty() {
tracing::warn!(
"help_chat: no documentation files found under {}",
root.display()
);
} else {
tracing::info!(
"help_chat: loaded {} documentation file(s) from {}",
parts.len(),
root.display()
);
}
parts.join("\n\n---\n\n")
}
/// Returns a reference to the cached doc context string, initialised on
/// first call via `OnceLock`.
///
/// Discovery order:
/// 1. `HELP_DOCS_PATH` env var (explicit override)
/// 2. Walk up from the binary location
/// 3. Current working directory
/// 4. Common Docker paths (/app, /opt/compliance-scanner)
fn doc_context() -> &'static str {
DOC_CONTEXT.get_or_init(|| {
// 1. Explicit env var
if let Ok(path) = std::env::var("HELP_DOCS_PATH") {
let p = PathBuf::from(&path);
if p.join("README.md").is_file() || p.join("docs").is_dir() {
tracing::info!("help_chat: loading docs from HELP_DOCS_PATH={path}");
return load_docs(&p);
}
tracing::warn!("help_chat: HELP_DOCS_PATH={path} has no README.md or docs/");
}
// 2. Walk up from binary location
let start = std::env::current_exe()
.ok()
.and_then(|p| p.parent().map(Path::to_path_buf))
.unwrap_or_else(|| PathBuf::from("."));
if let Some(root) = find_project_root(&start) {
return load_docs(&root);
}
// 3. Current working directory
if let Ok(cwd) = std::env::current_dir() {
if let Some(root) = find_project_root(&cwd) {
return load_docs(&root);
}
if cwd.join("README.md").is_file() {
return load_docs(&cwd);
}
}
// 4. Common Docker/deployment paths
for candidate in ["/app", "/opt/compliance-scanner", "/srv/compliance-scanner"] {
let p = PathBuf::from(candidate);
if p.join("README.md").is_file() || p.join("docs").is_dir() {
tracing::info!("help_chat: found docs at {candidate}");
return load_docs(&p);
}
}
tracing::error!(
"help_chat: could not locate project root; doc context will be empty. \
Set HELP_DOCS_PATH to the directory containing README.md and docs/"
);
String::new()
})
}
// ── Handler ──────────────────────────────────────────────────────────────────
/// POST /api/v1/help/chat — Answer questions about the compliance-scanner
/// using the project documentation as grounding context.
#[tracing::instrument(skip_all)]
pub async fn help_chat(
Extension(agent): AgentExt,
Json(req): Json<HelpChatRequest>,
) -> Result<Json<ApiResponse<HelpChatResponse>>, StatusCode> {
let context = doc_context();
let system_prompt = if context.is_empty() {
"You are a helpful assistant for the Compliance Scanner project. \
Answer questions about how to use and configure it. \
No documentation was loaded at startup, so rely on your general knowledge."
.to_string()
} else {
format!(
"You are a helpful assistant for the Compliance Scanner project. \
Answer questions about how to use, configure, and understand it \
using the documentation below as your primary source of truth.\n\n\
Rules:\n\
- Prefer information from the provided docs over general knowledge\n\
- Quote or reference the relevant doc section when it helps\n\
- If the docs do not cover the topic, say so clearly\n\
- Be concise — lead with the answer, then explain if needed\n\
- Use markdown formatting for readability\n\n\
## Project Documentation\n\n{context}"
)
};
let mut messages: Vec<(String, String)> = Vec::with_capacity(req.history.len() + 2);
messages.push(("system".to_string(), system_prompt));
for msg in &req.history {
messages.push((msg.role.clone(), msg.content.clone()));
}
messages.push(("user".to_string(), req.message));
let response_text = agent
.llm
.chat_with_messages(messages, Some(0.3))
.await
.map_err(|e| {
tracing::error!("LLM help chat failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
Ok(Json(ApiResponse {
data: HelpChatResponse {
message: response_text,
},
total: None,
page: None,
}))
}
@@ -0,0 +1,44 @@
use axum::extract::{Extension, Query};
use axum::Json;
use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::TrackerIssue;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn list_issues(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> ApiResult<Vec<TrackerIssue>> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.tracker_issues()
.count_documents(doc! {})
.await
.unwrap_or(0);
let issues = match db
.tracker_issues()
.find(doc! {})
.sort(doc! { "created_at": -1 })
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch tracker issues: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: issues,
total: Some(total),
page: Some(params.page),
}))
}
@@ -0,0 +1,186 @@
//! `/api/v1/mcp-tokens` — per-tenant API tokens for the MCP server.
//!
//! These are opaque static bearers issued via the dashboard (or a
//! direct curl with a KC JWT) and copied into LLM clients (Claude
//! Desktop / Cursor / ChatGPT). The MCP server hashes incoming bearers
//! and looks them up in the cross-tenant `<prefix>__admin.mcp_tokens`
//! collection to derive the tenant_id for routing.
//!
//! The raw token is shown to the caller exactly once at creation; the
//! database only ever stores the SHA-256 hash. Revocation is a soft
//! delete (sets `revoked: true`) so the audit log keeps the record.
use axum::extract::{Extension, Path};
use axum::http::StatusCode;
use axum::Json;
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine as _};
use compliance_core::models::{McpToken, McpTokenView};
use compliance_core::tenant_ctx::TenantCtx;
use mongodb::bson::doc;
use rand::RngCore;
use sha2::{Digest, Sha256};
use super::dto::{AgentExt, ApiResponse};
/// Mongo collection name inside the admin DB.
const COLLECTION: &str = "mcp_tokens";
/// Token prefix the MCP server expects on every bearer.
const TOKEN_PREFIX: &str = "mcpt_";
/// Bytes of randomness behind each token. 32 → ~256 bits.
/// Encoded as URL-safe base64 without padding → 43 chars.
/// Combined with `mcpt_` → 48-char tokens.
const TOKEN_RAND_BYTES: usize = 32;
#[derive(serde::Deserialize)]
pub struct CreateMcpTokenRequest {
pub name: String,
}
/// Returned exactly once at creation. The `token` field is gone from
/// the listing endpoint — the user must save it now.
#[derive(serde::Serialize)]
pub struct CreateMcpTokenResponse {
pub token: String,
pub view: McpTokenView,
}
/// `POST /api/v1/mcp-tokens` — mint a new token for the caller's tenant.
#[tracing::instrument(skip_all)]
pub async fn create_mcp_token(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateMcpTokenRequest>,
) -> Result<Json<CreateMcpTokenResponse>, StatusCode> {
if req.name.trim().is_empty() {
return Err(StatusCode::BAD_REQUEST);
}
let raw = generate_token();
let token_hash = sha256_hex(&raw);
let token_prefix: String = raw.chars().take(12).collect();
let mut token = McpToken {
id: None,
token_hash,
token_prefix,
tenant_id: tenant.0.tenant_id.clone(),
name: req.name.trim().to_string(),
created_by: tenant.0.user_id.clone(),
created_at: chrono::Utc::now(),
last_used_at: None,
revoked: false,
};
let col = agent.db_pool.admin_db().collection::<McpToken>(COLLECTION);
let res = col.insert_one(&token).await.map_err(|e| {
tracing::error!("Failed to insert MCP token: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
token.id = res.inserted_id.as_object_id();
Ok(Json(CreateMcpTokenResponse {
view: McpTokenView::from(&token),
token: raw,
}))
}
/// `GET /api/v1/mcp-tokens` — list tokens for the caller's tenant.
/// Hash is never returned; only metadata + the 12-char prefix so the
/// user can identify which row is which.
#[tracing::instrument(skip_all)]
pub async fn list_mcp_tokens(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<ApiResponse<Vec<McpTokenView>>>, StatusCode> {
let col = agent.db_pool.admin_db().collection::<McpToken>(COLLECTION);
let mut cursor = col
.find(doc! { "tenant_id": &tenant.0.tenant_id })
.sort(doc! { "created_at": -1 })
.await
.map_err(|e| {
tracing::error!("Failed to list MCP tokens: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
let mut out = Vec::new();
while cursor.advance().await.map_err(|e| {
tracing::warn!("MCP tokens cursor advance failed: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})? {
match cursor.deserialize_current() {
Ok(t) => out.push(McpTokenView::from(&t)),
Err(e) => tracing::warn!("Failed to deserialize MCP token: {e}"),
}
}
Ok(Json(ApiResponse {
data: out,
total: None,
page: None,
}))
}
/// `DELETE /api/v1/mcp-tokens/{id}` — revoke (soft delete).
/// Scoped to the caller's tenant: a user can't revoke another tenant's
/// token even if they guess its id.
#[tracing::instrument(skip_all, fields(id = %id))]
pub async fn revoke_mcp_token(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let col = agent.db_pool.admin_db().collection::<McpToken>(COLLECTION);
let result = col
.update_one(
doc! { "_id": oid, "tenant_id": &tenant.0.tenant_id },
doc! { "$set": { "revoked": true } },
)
.await
.map_err(|e| {
tracing::error!("Failed to revoke MCP token: {e}");
StatusCode::INTERNAL_SERVER_ERROR
})?;
if result.matched_count == 0 {
return Err(StatusCode::NOT_FOUND);
}
Ok(Json(serde_json::json!({ "status": "revoked" })))
}
/// 32 bytes random → URL-safe base64 → 43 chars, no padding.
/// Prefixed with `mcpt_` so the MCP server can sniff the format
/// before bothering with the DB lookup.
fn generate_token() -> String {
let mut bytes = [0u8; TOKEN_RAND_BYTES];
rand::rng().fill_bytes(&mut bytes);
format!("{TOKEN_PREFIX}{}", URL_SAFE_NO_PAD.encode(bytes))
}
fn sha256_hex(s: &str) -> String {
let mut h = Sha256::new();
h.update(s.as_bytes());
hex::encode(h.finalize())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn generated_tokens_are_unique_and_prefixed() {
let a = generate_token();
let b = generate_token();
assert_ne!(a, b);
assert!(a.starts_with(TOKEN_PREFIX));
assert!(b.starts_with(TOKEN_PREFIX));
// 5 + 43 = 48 chars
assert_eq!(a.len(), 5 + 43);
}
#[test]
fn sha256_is_stable_and_64_hex() {
let h = sha256_hex("mcpt_abc");
assert_eq!(h.len(), 64);
assert!(h.chars().all(|c| c.is_ascii_hexdigit()));
assert_eq!(sha256_hex("mcpt_abc"), h);
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,183 @@
use axum::extract::Extension;
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::doc;
use serde::Deserialize;
use compliance_core::models::notification::CveNotification;
use compliance_core::tenant_ctx::TenantCtx;
use super::dto::{tenant_db, AgentExt, ApiResponse};
/// GET /api/v1/notifications — List CVE notifications (newest first)
#[tracing::instrument(skip_all)]
pub async fn list_notifications(
Extension(agent): AgentExt,
tenant: TenantCtx,
axum::extract::Query(params): axum::extract::Query<NotificationFilter>,
) -> Result<Json<ApiResponse<Vec<CveNotification>>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let mut filter = doc! {};
// Filter by status (default: show new + read, exclude dismissed)
match params.status.as_deref() {
Some("all") => {}
Some(s) => {
filter.insert("status", s);
}
None => {
filter.insert("status", doc! { "$in": ["new", "read"] });
}
}
// Filter by severity
if let Some(ref sev) = params.severity {
filter.insert("severity", sev.as_str());
}
// Filter by repo
if let Some(ref repo_id) = params.repo_id {
filter.insert("repo_id", repo_id.as_str());
}
let page = params.page.unwrap_or(1).max(1);
let limit = params.limit.unwrap_or(50).min(200);
let skip = (page - 1) * limit as u64;
let total = db
.cve_notifications()
.count_documents(filter.clone())
.await
.unwrap_or(0);
let notifications: Vec<CveNotification> = match db
.cve_notifications()
.find(filter)
.sort(doc! { "created_at": -1 })
.skip(skip)
.limit(limit)
.await
{
Ok(cursor) => {
use futures_util::StreamExt;
let mut items = Vec::new();
let mut cursor = cursor;
while let Some(Ok(n)) = cursor.next().await {
items.push(n);
}
items
}
Err(e) => {
tracing::error!("Failed to list notifications: {e}");
return Err(StatusCode::INTERNAL_SERVER_ERROR);
}
};
Ok(Json(ApiResponse {
data: notifications,
total: Some(total),
page: Some(page),
}))
}
/// GET /api/v1/notifications/count — Count of unread notifications
#[tracing::instrument(skip_all)]
pub async fn notification_count(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let count = db
.cve_notifications()
.count_documents(doc! { "status": "new" })
.await
.unwrap_or(0);
Ok(Json(serde_json::json!({ "count": count })))
}
/// PATCH /api/v1/notifications/:id/read — Mark a notification as read
#[tracing::instrument(skip_all, fields(id = %id))]
pub async fn mark_read(
Extension(agent): AgentExt,
tenant: TenantCtx,
axum::extract::Path(id): axum::extract::Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let result = db
.cve_notifications()
.update_one(
doc! { "_id": oid },
doc! { "$set": {
"status": "read",
"read_at": mongodb::bson::DateTime::now(),
}},
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
if result.matched_count == 0 {
return Err(StatusCode::NOT_FOUND);
}
Ok(Json(serde_json::json!({ "status": "read" })))
}
/// PATCH /api/v1/notifications/:id/dismiss — Dismiss a notification
#[tracing::instrument(skip_all, fields(id = %id))]
pub async fn dismiss_notification(
Extension(agent): AgentExt,
tenant: TenantCtx,
axum::extract::Path(id): axum::extract::Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let result = db
.cve_notifications()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "status": "dismissed" } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
if result.matched_count == 0 {
return Err(StatusCode::NOT_FOUND);
}
Ok(Json(serde_json::json!({ "status": "dismissed" })))
}
/// POST /api/v1/notifications/read-all — Mark all new notifications as read
#[tracing::instrument(skip_all)]
pub async fn mark_all_read(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let result = db
.cve_notifications()
.update_many(
doc! { "status": "new" },
doc! { "$set": {
"status": "read",
"read_at": mongodb::bson::DateTime::now(),
}},
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
Ok(Json(
serde_json::json!({ "updated": result.modified_count }),
))
}
#[derive(Debug, Deserialize)]
pub struct NotificationFilter {
pub status: Option<String>,
pub severity: Option<String>,
pub repo_id: Option<String>,
pub page: Option<u64>,
pub limit: Option<i64>,
}
@@ -0,0 +1,614 @@
//! Onboarding API — CRUD for unified targets, artifact add, classification, and
//! the scan-applicability matrix. The wizard (and future integrations) drive
//! onboarding through these endpoints.
use std::collections::HashMap;
use std::sync::Arc;
use axum::extract::{Extension, Multipart, Path, Query};
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::{doc, oid::ObjectId, to_bson};
use serde::{Deserialize, Serialize};
use compliance_core::models::{
Artifact, ArtifactKind, ComplianceProfile, OnboardedTarget, PlcFormat, TargetScanConfig,
TargetType,
};
use compliance_core::scan_matrix::{applicable_scans, supports_pentest};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use crate::classify::{classify_target, MockFirmwareDetector};
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
/// A client-supplied artifact spec. The server builds the [`Artifact`] (and its
/// id) from it, so clients never set internal fields.
#[derive(Deserialize)]
pub struct ArtifactInput {
pub kind: ArtifactKind,
pub source_ref: String,
#[serde(default)]
pub branch: Option<String>,
#[serde(default)]
pub plc_format: Option<PlcFormat>,
}
impl ArtifactInput {
fn build(&self) -> Artifact {
let s = self.source_ref.clone();
match self.kind {
ArtifactKind::GitRepo => {
Artifact::git_repo(s, self.branch.clone().unwrap_or_else(|| "main".to_string()))
}
ArtifactKind::LiveUrl => Artifact::live_url(s),
ArtifactKind::FirmwareImage => Artifact::firmware_image(s),
ArtifactKind::SourceArchive => Artifact::source_archive(s),
ArtifactKind::MobilePackage => Artifact::mobile_package(s),
ArtifactKind::ContainerImage => Artifact::container_image(s),
ArtifactKind::PlcProject => {
Artifact::plc_project(s, self.plc_format.unwrap_or(PlcFormat::PlcopenXml))
}
ArtifactKind::PlaintextDescription => Artifact::plaintext(s),
}
}
}
#[derive(Deserialize)]
pub struct CreateTargetRequest {
pub name: String,
pub target_type: TargetType,
#[serde(default)]
pub description: Option<String>,
#[serde(default)]
pub artifacts: Vec<ArtifactInput>,
}
#[derive(Deserialize)]
pub struct UpdateTargetRequest {
pub name: Option<String>,
pub target_type: Option<TargetType>,
pub scan_config: Option<TargetScanConfig>,
pub compliance_profile: Option<ComplianceProfile>,
pub scan_schedule: Option<String>,
/// Replace the target's artifacts wholesale (used by the dashboard editor).
#[serde(default)]
pub artifacts: Option<Vec<ArtifactInput>>,
}
/// One applicable-scan option, serialized for the wizard.
#[derive(Serialize)]
pub struct ScanOptionDto {
pub scan: String,
pub default_on: bool,
pub rationale: String,
pub required_artifact: Option<String>,
pub blocked_reason: Option<String>,
}
#[derive(Serialize)]
pub struct ApplicableScansResponse {
pub scans: Vec<ScanOptionDto>,
pub pentest_supported: bool,
}
fn parse_oid(id: &str) -> Result<ObjectId, StatusCode> {
ObjectId::parse_str(id).map_err(|_| StatusCode::BAD_REQUEST)
}
/// GET /api/v1/targets — list onboarded targets (paginated).
#[tracing::instrument(skip_all)]
pub async fn list_targets(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<OnboardedTarget>>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.onboarded_targets()
.count_documents(doc! {})
.await
.unwrap_or(0);
let targets = match db
.onboarded_targets()
.find(doc! {})
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch onboarded targets: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: targets,
total: Some(total),
page: Some(params.page),
}))
}
/// POST /api/v1/targets — create an onboarded target.
#[tracing::instrument(skip_all)]
pub async fn create_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateTargetRequest>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let mut target = OnboardedTarget::new(req.name, req.target_type);
target.description = req.description;
target.artifacts = req.artifacts.iter().map(ArtifactInput::build).collect();
let db = tenant_db(&agent, &tenant).await?;
let res = db
.onboarded_targets()
.insert_one(&target)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
target.id = res.inserted_id.as_object_id();
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// GET /api/v1/targets/{id} — fetch one target.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn get_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// PATCH /api/v1/targets/{id} — update mutable fields.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn update_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateTargetRequest>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let mut set = doc! { "updated_at": mongodb::bson::DateTime::now() };
if let Some(name) = req.name {
set.insert("name", name);
}
if let Some(tt) = req.target_type {
set.insert(
"target_type",
to_bson(&tt).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(sc) = req.scan_config {
set.insert(
"scan_config",
to_bson(&sc).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(cp) = req.compliance_profile {
set.insert(
"compliance_profile",
to_bson(&cp).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(ss) = req.scan_schedule {
set.insert("scan_schedule", ss);
}
if let Some(arts) = req.artifacts {
let built: Vec<Artifact> = arts.iter().map(ArtifactInput::build).collect();
set.insert(
"artifacts",
to_bson(&built).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
db.onboarded_targets()
.update_one(doc! { "_id": oid }, doc! { "$set": set })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// DELETE /api/v1/targets/{id} — remove the target and its findings/scans.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn delete_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
db.onboarded_targets()
.delete_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
// Cascade all data keyed by repo_id == target id (best-effort).
let db = &db;
let _ = db.findings().delete_many(doc! { "repo_id": &id }).await;
let _ = db.sbom_entries().delete_many(doc! { "repo_id": &id }).await;
let _ = db.scan_runs().delete_many(doc! { "repo_id": &id }).await;
let _ = db.cve_alerts().delete_many(doc! { "repo_id": &id }).await;
let _ = db
.tracker_issues()
.delete_many(doc! { "repo_id": &id })
.await;
let _ = db.graph_nodes().delete_many(doc! { "repo_id": &id }).await;
let _ = db.graph_edges().delete_many(doc! { "repo_id": &id }).await;
let _ = db.graph_builds().delete_many(doc! { "repo_id": &id }).await;
let _ = db
.impact_analyses()
.delete_many(doc! { "repo_id": &id })
.await;
let _ = db
.code_embeddings()
.delete_many(doc! { "repo_id": &id })
.await;
let _ = db
.embedding_builds()
.delete_many(doc! { "repo_id": &id })
.await;
// DAST targets linked to this target, and all their downstream data.
if let Ok(mut cursor) = db.dast_targets().find(doc! { "repo_id": &id }).await {
use futures_util::StreamExt;
while let Some(Ok(dt)) = cursor.next().await {
let dast_target_id = dt.id.map(|oid| oid.to_hex()).unwrap_or_default();
if !dast_target_id.is_empty() {
cascade_delete_dast_target(db, &dast_target_id).await;
}
}
}
// Pentest sessions linked directly to this target (not via a DAST target).
if let Ok(mut cursor) = db.pentest_sessions().find(doc! { "repo_id": &id }).await {
use futures_util::StreamExt;
while let Some(Ok(session)) = cursor.next().await {
let session_id = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
if !session_id.is_empty() {
let _ = db
.attack_chain_nodes()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.pentest_messages()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.dast_findings()
.delete_many(doc! { "session_id": &session_id })
.await;
}
}
}
let _ = db
.pentest_sessions()
.delete_many(doc! { "repo_id": &id })
.await;
Ok(Json(serde_json::json!({ "status": "deleted" })))
}
/// Delete a DAST target and everything downstream of it (pentest sessions +
/// their attack chains / messages / findings, DAST scan runs + findings).
async fn cascade_delete_dast_target(db: &crate::database::Database, target_id: &str) {
use futures_util::StreamExt;
if let Ok(mut cursor) = db
.pentest_sessions()
.find(doc! { "target_id": target_id })
.await
{
while let Some(Ok(session)) = cursor.next().await {
let session_id = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
if !session_id.is_empty() {
let _ = db
.attack_chain_nodes()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.pentest_messages()
.delete_many(doc! { "session_id": &session_id })
.await;
let _ = db
.dast_findings()
.delete_many(doc! { "session_id": &session_id })
.await;
}
}
}
let _ = db
.pentest_sessions()
.delete_many(doc! { "target_id": target_id })
.await;
let _ = db
.dast_findings()
.delete_many(doc! { "target_id": target_id })
.await;
let _ = db
.dast_scan_runs()
.delete_many(doc! { "target_id": target_id })
.await;
if let Ok(oid) = mongodb::bson::oid::ObjectId::parse_str(target_id) {
let _ = db.dast_targets().delete_one(doc! { "_id": oid }).await;
}
}
/// POST /api/v1/targets/{id}/artifacts — attach an artifact (by reference).
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn add_artifact(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(input): Json<ArtifactInput>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let artifact = to_bson(&input.build()).map_err(|_| StatusCode::BAD_REQUEST)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$push": { "artifacts": artifact }, "$set": { "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// POST /api/v1/targets/{id}/artifacts/upload — attach an artifact by uploading
/// its file (PLC project, firmware image, source archive, mobile package). The
/// bytes are written to the artifact blob store and referenced by `stored_path`,
/// so ingest resolves them locally (no URL fetch).
///
/// Multipart fields: `file` (required), `kind` (required, snake_case
/// `ArtifactKind`), `plc_format` (optional, for PLC projects).
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn upload_artifact(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
mut multipart: Multipart,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
if db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.is_none()
{
return Err(StatusCode::NOT_FOUND);
}
let mut kind: Option<ArtifactKind> = None;
let mut plc_format: Option<PlcFormat> = None;
let mut filename = String::from("upload.bin");
let mut bytes: Option<axum::body::Bytes> = None;
while let Some(field) = multipart
.next_field()
.await
.map_err(|_| StatusCode::BAD_REQUEST)?
{
match field.name().unwrap_or("") {
"kind" => {
let v = field.text().await.map_err(|_| StatusCode::BAD_REQUEST)?;
kind = parse_enum(&v);
}
"plc_format" => {
let v = field.text().await.map_err(|_| StatusCode::BAD_REQUEST)?;
plc_format = parse_enum(&v);
}
"file" => {
if let Some(fname) = field.file_name() {
filename = fname.to_string();
}
bytes = Some(field.bytes().await.map_err(|_| StatusCode::BAD_REQUEST)?);
}
_ => {}
}
}
let (Some(kind), Some(bytes)) = (kind, bytes) else {
return Err(StatusCode::BAD_REQUEST);
};
// Store the uploaded bytes under the artifact blob store.
let safe_name: String = filename
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || matches!(c, '.' | '-' | '_') {
c
} else {
'_'
}
})
.collect();
let dir = std::path::Path::new(&agent.config.artifact_store_base_path)
.join("uploads")
.join(&id);
std::fs::create_dir_all(&dir).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let dest = dir.join(format!("{}_{safe_name}", uuid::Uuid::new_v4()));
std::fs::write(&dest, bytes.as_ref()).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
// Build the artifact for this kind, referencing the stored file.
let mut artifact = match kind {
ArtifactKind::PlcProject => Artifact::plc_project(
filename.clone(),
plc_format.unwrap_or(PlcFormat::PlcopenXml),
),
ArtifactKind::FirmwareImage => Artifact::firmware_image(filename.clone()),
ArtifactKind::SourceArchive => Artifact::source_archive(filename.clone()),
ArtifactKind::MobilePackage => Artifact::mobile_package(filename.clone()),
// Non-file kinds (git repo, live URL, container ref, text) use the JSON
// add-artifact endpoint, not upload.
_ => return Err(StatusCode::BAD_REQUEST),
};
artifact.stored_path = Some(dest.to_string_lossy().to_string());
artifact.size_bytes = Some(bytes.len() as u64);
let artifact_bson = to_bson(&artifact).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$push": { "artifacts": artifact_bson }, "$set": { "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// Deserialize a snake_case enum value from a plain string.
fn parse_enum<T: for<'de> Deserialize<'de>>(s: &str) -> Option<T> {
serde_json::from_value(serde_json::Value::String(s.to_string())).ok()
}
/// GET /api/v1/targets/{id}/applicable-scans — the scan-applicability matrix.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn applicable_scans_for_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<ApplicableScansResponse>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
let scans = applicable_scans(&target)
.into_iter()
.map(|o| ScanOptionDto {
scan: o.scan.to_string(),
default_on: o.default_on,
rationale: o.rationale,
required_artifact: o.required_artifact.map(|k| k.to_string()),
blocked_reason: o.blocked_reason,
})
.collect();
Ok(Json(ApiResponse {
data: ApplicableScansResponse {
scans,
pentest_supported: supports_pentest(target.target_type),
},
total: None,
page: None,
}))
}
/// POST /api/v1/targets/{id}/detect — classify the target from its artifacts.
///
/// This is the lightweight pass: it classifies from artifact kinds without
/// ingesting (cloning) sources, so it returns immediately. Deep detection (after
/// ingest, with tramiton firmware analysis) is a follow-up background step.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn detect_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let mut target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
// No ingested working paths here → kind-based classification only; the mock
// firmware detector is never invoked (no firmware working path present).
let empty = HashMap::new();
let detector = MockFirmwareDetector { detection: None };
let classification = classify_target(&target, &empty, &detector)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let classification_bson =
to_bson(&classification).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "classification": classification_bson, "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
target.classification = Some(classification);
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// POST /api/v1/targets/{id}/scan — trigger a scan for the target.
///
/// Dispatches to the unified pipeline when `UNIFIED_PIPELINE` is set (else the
/// legacy path). Runs in the background and returns immediately.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn trigger_target_scan(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
// 404 if the target doesn't exist for this tenant.
if db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.is_none()
{
return Err(StatusCode::NOT_FOUND);
}
let agent_clone = (*agent).clone();
let tenant_id = tenant.0.tenant_id.clone();
tokio::spawn(async move {
// Always the unified target pipeline — this endpoint is about an
// onboarded target by construction, independent of the global
// `unified_pipeline` transition flag used by the legacy paths.
if let Err(e) = agent_clone
.run_target_scan(
&tenant_id,
&id,
compliance_core::models::ScanTrigger::Manual,
)
.await
{
tracing::error!("Manual target scan failed for {id}: {e}");
}
});
Ok(Json(serde_json::json!({ "status": "scan_triggered" })))
}
@@ -0,0 +1,235 @@
use std::sync::Arc;
use axum::extract::{Extension, Path};
use axum::http::StatusCode;
use axum::response::IntoResponse;
use axum::Json;
use mongodb::bson::doc;
use serde::Deserialize;
use futures_util::StreamExt;
use compliance_core::models::dast::DastFinding;
use compliance_core::models::finding::Finding;
use compliance_core::models::pentest::*;
use compliance_core::models::sbom::SbomEntry;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::super::dto::{collect_cursor_async, tenant_db};
type AgentExt = Extension<Arc<ComplianceAgent>>;
#[derive(Deserialize)]
pub struct ExportBody {
pub password: String,
/// Requester display name (from auth)
#[serde(default)]
pub requester_name: String,
/// Requester email (from auth)
#[serde(default)]
pub requester_email: String,
}
/// POST /api/v1/pentest/sessions/:id/export — Export an encrypted pentest report archive
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn export_session_report(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(body): Json<ExportBody>,
) -> Result<axum::response::Response, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
if body.password.len() < 8 {
return Err((
StatusCode::BAD_REQUEST,
"Password must be at least 8 characters".to_string(),
));
}
// Fetch session
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| (StatusCode::NOT_FOUND, "Session not found".to_string()))?;
// Resolve target name
let target = if let Ok(tid) = mongodb::bson::oid::ObjectId::parse_str(&session.target_id) {
db.dast_targets()
.find_one(doc! { "_id": tid })
.await
.ok()
.flatten()
} else {
None
};
let target_name = target
.as_ref()
.map(|t| t.name.clone())
.unwrap_or_else(|| "Unknown Target".to_string());
let target_url = target
.as_ref()
.map(|t| t.base_url.clone())
.unwrap_or_default();
// Fetch attack chain nodes
let nodes: Vec<AttackChainNode> = match db
.attack_chain_nodes()
.find(doc! { "session_id": &id })
.sort(doc! { "started_at": 1 })
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(_) => Vec::new(),
};
// Fetch DAST findings for this session, then deduplicate
let raw_findings: Vec<DastFinding> = match db
.dast_findings()
.find(doc! { "session_id": &id })
.sort(doc! { "severity": -1, "created_at": -1 })
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(_) => Vec::new(),
};
let raw_count = raw_findings.len();
let findings = crate::pipeline::dedup::dedup_dast_findings(raw_findings);
if findings.len() < raw_count {
tracing::info!(
"Deduped DAST findings for session {id}: {raw_count} → {}",
findings.len()
);
}
// Fetch SAST findings, SBOM, and code context for the linked repository
let repo_id = session
.repo_id
.clone()
.or_else(|| target.as_ref().and_then(|t| t.repo_id.clone()));
let (sast_findings, sbom_entries, code_context) = if let Some(ref rid) = repo_id {
let sast: Vec<Finding> = match db
.findings()
.find(doc! {
"repo_id": rid,
"status": { "$in": ["open", "triaged"] },
})
.sort(doc! { "severity": -1 })
.limit(100)
.await
{
Ok(mut cursor) => {
let mut results = Vec::new();
while let Some(Ok(f)) = cursor.next().await {
results.push(f);
}
results
}
Err(_) => Vec::new(),
};
let sbom: Vec<SbomEntry> = match db
.sbom_entries()
.find(doc! {
"repo_id": rid,
"known_vulnerabilities": { "$exists": true, "$ne": [] },
})
.limit(50)
.await
{
Ok(mut cursor) => {
let mut results = Vec::new();
while let Some(Ok(e)) = cursor.next().await {
results.push(e);
}
results
}
Err(_) => Vec::new(),
};
// Build code context from graph nodes
let code_ctx: Vec<CodeContextHint> = match db
.graph_nodes()
.find(doc! { "repo_id": rid, "is_entry_point": true })
.limit(50)
.await
{
Ok(mut cursor) => {
let mut nodes_vec = Vec::new();
while let Some(Ok(n)) = cursor.next().await {
let linked_vulns: Vec<String> = sast
.iter()
.filter(|f| f.file_path.as_deref() == Some(&n.file_path))
.map(|f| {
format!(
"[{}] {}: {} (line {})",
f.severity,
f.scanner,
f.title,
f.line_number.unwrap_or(0)
)
})
.collect();
nodes_vec.push(CodeContextHint {
endpoint_pattern: n.qualified_name.clone(),
handler_function: n.name.clone(),
file_path: n.file_path.clone(),
code_snippet: String::new(),
known_vulnerabilities: linked_vulns,
});
}
nodes_vec
}
Err(_) => Vec::new(),
};
(sast, sbom, code_ctx)
} else {
(Vec::new(), Vec::new(), Vec::new())
};
let config = session.config.clone();
let ctx = crate::pentest::report::ReportContext {
session,
target_name,
target_url,
findings,
attack_chain: nodes,
requester_name: if body.requester_name.is_empty() {
"Unknown".to_string()
} else {
body.requester_name
},
requester_email: body.requester_email,
config,
sast_findings,
sbom_entries,
code_context,
};
let report = crate::pentest::generate_encrypted_report(&ctx, &body.password)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e))?;
let response = serde_json::json!({
"archive_base64": base64::Engine::encode(&base64::engine::general_purpose::STANDARD, &report.archive),
"sha256": report.sha256,
"filename": format!("pentest-report-{id}.zip"),
});
Ok(Json(response).into_response())
}
@@ -0,0 +1,9 @@
mod export;
mod session;
mod stats;
mod stream;
pub use export::*;
pub use session::*;
pub use stats::*;
pub use stream::*;
@@ -0,0 +1,852 @@
use std::sync::Arc;
use axum::extract::{Extension, Path, Query};
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::doc;
use serde::Deserialize;
use compliance_core::models::pentest::*;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use crate::pentest::PentestOrchestrator;
use super::super::dto::{collect_cursor_async, tenant_db, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
#[derive(Deserialize)]
pub struct CreateSessionRequest {
pub target_id: Option<String>,
#[serde(default = "default_strategy")]
pub strategy: String,
pub message: Option<String>,
/// Wizard configuration — if present, takes precedence over legacy fields
pub config: Option<PentestConfig>,
}
fn default_strategy() -> String {
"comprehensive".to_string()
}
#[derive(Deserialize)]
pub struct SendMessageRequest {
pub message: String,
}
#[derive(Deserialize)]
pub struct LookupRepoQuery {
pub url: String,
}
/// POST /api/v1/pentest/sessions — Create a new pentest session and start the orchestrator
#[tracing::instrument(skip_all)]
pub async fn create_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateSessionRequest>,
) -> Result<Json<ApiResponse<PentestSession>>, (StatusCode, String)> {
// Try to acquire a concurrency permit
let permit = agent
.session_semaphore
.clone()
.try_acquire_owned()
.map_err(|_| {
(
StatusCode::TOO_MANY_REQUESTS,
"Maximum concurrent pentest sessions reached. Try again later.".to_string(),
)
})?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
if let Some(ref config) = req.config {
// ── Wizard path ──────────────────────────────────────────────
if !config.disclaimer_accepted {
return Err((
StatusCode::BAD_REQUEST,
"Disclaimer must be accepted".to_string(),
));
}
// Look up or auto-create DastTarget by app_url
let target = match db
.dast_targets()
.find_one(doc! { "base_url": &config.app_url })
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, format!("DB error: {e}")))?
{
Some(t) => t,
None => {
use compliance_core::models::dast::{DastTarget, DastTargetType};
let mut t = DastTarget::new(
config.app_url.clone(),
config.app_url.clone(),
DastTargetType::WebApp,
);
if let Some(rl) = config.rate_limit {
t.rate_limit = rl;
}
t.allow_destructive = config.allow_destructive;
t.excluded_paths = config.scope_exclusions.clone();
let res = db.dast_targets().insert_one(&t).await.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create target: {e}"),
)
})?;
t.id = res.inserted_id.as_object_id();
t
}
};
let target_id = target.id.map(|oid| oid.to_hex()).unwrap_or_default();
// Parse strategy from config or request
let strat_str = config.strategy.as_deref().unwrap_or(req.strategy.as_str());
let strategy = parse_strategy(strat_str);
let mut session = PentestSession::new(target_id, strategy);
session.config = Some(config.clone());
session.repo_id = target.repo_id.clone();
// Resolve repo_id (target id) from git_repo_url if provided
if let Some(ref git_url) = config.git_repo_url {
if let Ok(Some(target)) = db
.onboarded_targets()
.find_one(doc! { "artifacts.source_ref": git_url })
.await
{
session.repo_id = target.id.map(|oid| oid.to_hex());
}
}
let insert_result = db
.pentest_sessions()
.insert_one(&session)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create session: {e}"),
)
})?;
session.id = insert_result.inserted_id.as_object_id();
let session_id_str = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
// Register broadcast stream and pause control
let event_tx = agent.register_session_stream(&session_id_str);
let pause_rx = agent.register_pause_control(&session_id_str);
// Merge server-default IMAP/email settings where wizard left blanks
if let Some(ref mut cfg) = session.config {
if cfg.auth.mode == AuthMode::AutoRegister {
if cfg.auth.verification_email.is_none() {
cfg.auth.verification_email = agent.config.pentest_verification_email.clone();
}
if cfg.auth.imap_host.is_none() {
cfg.auth.imap_host = agent.config.pentest_imap_host.clone();
}
if cfg.auth.imap_port.is_none() {
cfg.auth.imap_port = agent.config.pentest_imap_port;
}
if cfg.auth.imap_username.is_none() {
cfg.auth.imap_username = agent.config.pentest_imap_username.clone();
}
if cfg.auth.imap_password.is_none() {
cfg.auth.imap_password = agent.config.pentest_imap_password.as_ref().map(|s| {
use secrecy::ExposeSecret;
s.expose_secret().to_string()
});
}
}
}
// Pre-populate test user record for auto-register sessions
if let Some(ref cfg) = session.config {
if cfg.auth.mode == AuthMode::AutoRegister {
let verification_email = cfg.auth.verification_email.clone();
// Build plus-addressed email for this session
let test_email = verification_email.as_deref().map(|email| {
let parts: Vec<&str> = email.splitn(2, '@').collect();
if parts.len() == 2 {
format!("{}+{}@{}", parts[0], session_id_str, parts[1])
} else {
email.to_string()
}
});
// Detect identity provider from keycloak config
let provider = if agent.config.keycloak_url.is_some() {
Some(compliance_core::models::pentest::IdentityProvider::Keycloak)
} else {
None
};
session.test_user = Some(compliance_core::models::pentest::TestUserRecord {
username: None, // LLM will choose; updated after registration
email: test_email,
provider_user_id: None,
provider,
cleaned_up: false,
});
}
}
// Encrypt credentials before they linger in memory
let mut session_for_task = session.clone();
if let Some(ref mut cfg) = session_for_task.config {
cfg.auth.username = cfg
.auth
.username
.as_ref()
.map(|u| crate::pentest::crypto::encrypt(u));
cfg.auth.password = cfg
.auth
.password
.as_ref()
.map(|p| crate::pentest::crypto::encrypt(p));
}
// Persist encrypted credentials to DB
if session_for_task.config.is_some() {
if let Some(sid) = session.id {
let _ = db
.pentest_sessions()
.update_one(
doc! { "_id": sid },
doc! { "$set": {
"config.auth.username": session_for_task.config.as_ref()
.and_then(|c| c.auth.username.as_deref())
.map(|s| mongodb::bson::Bson::String(s.to_string()))
.unwrap_or(mongodb::bson::Bson::Null),
"config.auth.password": session_for_task.config.as_ref()
.and_then(|c| c.auth.password.as_deref())
.map(|s| mongodb::bson::Bson::String(s.to_string()))
.unwrap_or(mongodb::bson::Bson::Null),
}},
)
.await;
}
}
let initial_message = config
.initial_instructions
.clone()
.or(req.message.clone())
.unwrap_or_else(|| {
format!(
"Begin a {} penetration test against {} ({}). \
Identify vulnerabilities and provide evidence for each finding.",
session.strategy, target.name, target.base_url,
)
});
let llm = agent.llm.clone();
let db_for_orchestrator = db.clone();
let session_clone = session.clone();
let target_clone = target.clone();
let agent_ref = agent.clone();
tokio::spawn(async move {
let orchestrator =
PentestOrchestrator::new(llm, db_for_orchestrator, event_tx, Some(pause_rx));
orchestrator
.run_session_guarded(&session_clone, &target_clone, &initial_message)
.await;
// Clean up session resources
agent_ref.cleanup_session(&session_id_str);
// Release concurrency permit
drop(permit);
});
// Redact credentials in response
let mut response_session = session;
if let Some(ref mut cfg) = response_session.config {
if cfg.auth.username.is_some() {
cfg.auth.username = Some("********".to_string());
}
if cfg.auth.password.is_some() {
cfg.auth.password = Some("********".to_string());
}
}
Ok(Json(ApiResponse {
data: response_session,
total: None,
page: None,
}))
} else {
// ── Legacy path ──────────────────────────────────────────────
let target_id = req.target_id.clone().ok_or_else(|| {
(
StatusCode::BAD_REQUEST,
"target_id is required for legacy creation".to_string(),
)
})?;
let oid = mongodb::bson::oid::ObjectId::parse_str(&target_id).map_err(|_| {
(
StatusCode::BAD_REQUEST,
"Invalid target_id format".to_string(),
)
})?;
let target = db
.dast_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| (StatusCode::NOT_FOUND, "Target not found".to_string()))?;
let strategy = parse_strategy(&req.strategy);
let mut session = PentestSession::new(target_id, strategy);
session.repo_id = target.repo_id.clone();
let insert_result = db
.pentest_sessions()
.insert_one(&session)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create session: {e}"),
)
})?;
session.id = insert_result.inserted_id.as_object_id();
let session_id_str = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
// Register broadcast stream and pause control
let event_tx = agent.register_session_stream(&session_id_str);
let pause_rx = agent.register_pause_control(&session_id_str);
let initial_message = req.message.unwrap_or_else(|| {
format!(
"Begin a {} penetration test against {} ({}). \
Identify vulnerabilities and provide evidence for each finding.",
session.strategy, target.name, target.base_url,
)
});
let llm = agent.llm.clone();
let db_for_orchestrator = db.clone();
let session_clone = session.clone();
let target_clone = target.clone();
let agent_ref = agent.clone();
tokio::spawn(async move {
let orchestrator =
PentestOrchestrator::new(llm, db_for_orchestrator, event_tx, Some(pause_rx));
orchestrator
.run_session_guarded(&session_clone, &target_clone, &initial_message)
.await;
agent_ref.cleanup_session(&session_id_str);
drop(permit);
});
Ok(Json(ApiResponse {
data: session,
total: None,
page: None,
}))
}
}
fn parse_strategy(s: &str) -> PentestStrategy {
match s {
"quick" => PentestStrategy::Quick,
"targeted" => PentestStrategy::Targeted,
"aggressive" => PentestStrategy::Aggressive,
"stealth" => PentestStrategy::Stealth,
_ => PentestStrategy::Comprehensive,
}
}
/// GET /api/v1/pentest/lookup-repo — Look up a tracked repository by git URL
#[tracing::instrument(skip_all)]
pub async fn lookup_repo(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<LookupRepoQuery>,
) -> Result<Json<ApiResponse<serde_json::Value>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let repo = db
.onboarded_targets()
.find_one(doc! { "artifacts.source_ref": &params.url })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let data = match repo {
Some(r) => {
let git = r.code_artifact().and_then(|c| c.git.as_ref());
serde_json::json!({
"name": r.name,
"default_branch": git.map(|g| g.default_branch.clone()),
"last_scanned_commit": git.and_then(|g| g.last_scanned_commit.clone()),
})
}
None => serde_json::Value::Null,
};
Ok(Json(ApiResponse {
data,
total: None,
page: None,
}))
}
/// GET /api/v1/pentest/sessions — List pentest sessions
#[tracing::instrument(skip_all)]
pub async fn list_sessions(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<PentestSession>>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.pentest_sessions()
.count_documents(doc! {})
.await
.unwrap_or(0);
let sessions = match db
.pentest_sessions()
.find(doc! {})
.sort(doc! { "started_at": -1 })
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch pentest sessions: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: sessions,
total: Some(total),
page: Some(params.page),
}))
}
/// GET /api/v1/pentest/sessions/:id — Get a single pentest session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<PentestSession>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let mut session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
// Redact credentials in response
if let Some(ref mut cfg) = session.config {
if cfg.auth.username.is_some() {
cfg.auth.username = Some("********".to_string());
}
if cfg.auth.password.is_some() {
cfg.auth.password = Some("********".to_string());
}
}
Ok(Json(ApiResponse {
data: session,
total: None,
page: None,
}))
}
/// POST /api/v1/pentest/sessions/:id/chat — Send a user message and trigger next orchestrator iteration
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn send_message(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<SendMessageRequest>,
) -> Result<Json<ApiResponse<PentestMessage>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
// Verify session exists and is running
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| (StatusCode::NOT_FOUND, "Session not found".to_string()))?;
if session.status != PentestStatus::Running && session.status != PentestStatus::Paused {
return Err((
StatusCode::BAD_REQUEST,
format!("Session is {}, cannot send messages", session.status),
));
}
// Look up the target
let target_oid = mongodb::bson::oid::ObjectId::parse_str(&session.target_id).map_err(|_| {
(
StatusCode::INTERNAL_SERVER_ERROR,
"Invalid target_id in session".to_string(),
)
})?;
let target = db
.dast_targets()
.find_one(doc! { "_id": target_oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| {
(
StatusCode::NOT_FOUND,
"Target for session not found".to_string(),
)
})?;
// Store user message
let session_id = id.clone();
let user_msg = PentestMessage::user(session_id.clone(), req.message.clone());
let _ = db.pentest_messages().insert_one(&user_msg).await;
let response_msg = user_msg.clone();
// Spawn orchestrator to continue the session
let llm = agent.llm.clone();
let db_for_orchestrator = db.clone();
let message = req.message.clone();
// Use existing broadcast sender if available, otherwise create a new one
let event_tx = agent
.subscribe_session(&session_id)
.and_then(|_| {
agent
.session_streams
.get(&session_id)
.map(|entry| entry.value().clone())
})
.unwrap_or_else(|| agent.register_session_stream(&session_id));
tokio::spawn(async move {
let orchestrator = PentestOrchestrator::new(llm, db_for_orchestrator, event_tx, None);
orchestrator
.run_session_guarded(&session, &target, &message)
.await;
});
Ok(Json(ApiResponse {
data: response_msg,
total: None,
page: None,
}))
}
/// POST /api/v1/pentest/sessions/:id/stop — Stop a running pentest session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn stop_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<PentestSession>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| (StatusCode::NOT_FOUND, "Session not found".to_string()))?;
if session.status != PentestStatus::Running && session.status != PentestStatus::Paused {
return Err((
StatusCode::BAD_REQUEST,
format!("Session is {}, not running or paused", session.status),
));
}
db.pentest_sessions()
.update_one(
doc! { "_id": oid },
doc! { "$set": {
"status": "failed",
"completed_at": mongodb::bson::DateTime::now(),
"error_message": "Stopped by user",
}},
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?;
// Clean up session resources
agent.cleanup_session(&id);
let updated = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| {
(
StatusCode::NOT_FOUND,
"Session not found after update".to_string(),
)
})?;
Ok(Json(ApiResponse {
data: updated,
total: None,
page: None,
}))
}
/// POST /api/v1/pentest/sessions/:id/pause — Pause a running pentest session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn pause_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<serde_json::Value>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| (StatusCode::NOT_FOUND, "Session not found".to_string()))?;
if session.status != PentestStatus::Running {
return Err((
StatusCode::BAD_REQUEST,
format!("Session is {}, not running", session.status),
));
}
if !agent.pause_session(&id) {
return Err((
StatusCode::INTERNAL_SERVER_ERROR,
"Failed to send pause signal".to_string(),
));
}
Ok(Json(ApiResponse {
data: serde_json::json!({ "status": "paused" }),
total: None,
page: None,
}))
}
/// POST /api/v1/pentest/sessions/:id/resume — Resume a paused pentest session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn resume_session(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<serde_json::Value>>, (StatusCode, String)> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid session ID".to_string()))?;
let db = tenant_db(&agent, &tenant)
.await
.map_err(|s| (s, "failed to acquire tenant database".to_string()))?;
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Database error: {e}"),
)
})?
.ok_or_else(|| (StatusCode::NOT_FOUND, "Session not found".to_string()))?;
if session.status != PentestStatus::Paused {
return Err((
StatusCode::BAD_REQUEST,
format!("Session is {}, not paused", session.status),
));
}
if !agent.resume_session(&id) {
return Err((
StatusCode::INTERNAL_SERVER_ERROR,
"Failed to send resume signal".to_string(),
));
}
Ok(Json(ApiResponse {
data: serde_json::json!({ "status": "running" }),
total: None,
page: None,
}))
}
/// GET /api/v1/pentest/sessions/:id/attack-chain — Get attack chain nodes for a session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_attack_chain(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<Vec<AttackChainNode>>>, StatusCode> {
let _oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let nodes = match db
.attack_chain_nodes()
.find(doc! { "session_id": &id })
.sort(doc! { "started_at": 1 })
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch attack chain nodes: {e}");
Vec::new()
}
};
let total = nodes.len() as u64;
Ok(Json(ApiResponse {
data: nodes,
total: Some(total),
page: None,
}))
}
/// GET /api/v1/pentest/sessions/:id/messages — Get messages for a session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_messages(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<PentestMessage>>>, StatusCode> {
let _oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.pentest_messages()
.count_documents(doc! { "session_id": &id })
.await
.unwrap_or(0);
let messages = match db
.pentest_messages()
.find(doc! { "session_id": &id })
.sort(doc! { "created_at": 1 })
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch pentest messages: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: messages,
total: Some(total),
page: Some(params.page),
}))
}
/// GET /api/v1/pentest/sessions/:id/findings — Get DAST findings for a pentest session
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn get_session_findings(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<compliance_core::models::dast::DastFinding>>>, StatusCode> {
let _oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.dast_findings()
.count_documents(doc! { "session_id": &id })
.await
.unwrap_or(0);
let findings = match db
.dast_findings()
.find(doc! { "session_id": &id })
.sort(doc! { "created_at": -1 })
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch pentest session findings: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: findings,
total: Some(total),
page: Some(params.page),
}))
}
@@ -0,0 +1,105 @@
use std::sync::Arc;
use axum::extract::Extension;
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::doc;
use compliance_core::models::pentest::*;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::super::dto::{collect_cursor_async, tenant_db, ApiResponse};
type AgentExt = Extension<Arc<ComplianceAgent>>;
/// GET /api/v1/pentest/stats — Aggregated pentest statistics
#[tracing::instrument(skip_all)]
pub async fn pentest_stats(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<ApiResponse<PentestStats>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let running_sessions = db
.pentest_sessions()
.count_documents(doc! { "status": "running" })
.await
.unwrap_or(0) as u32;
// Count DAST findings from pentest sessions
let total_vulnerabilities = db
.dast_findings()
.count_documents(doc! { "session_id": { "$exists": true, "$ne": null } })
.await
.unwrap_or(0) as u32;
// Aggregate tool invocations from all sessions
let sessions: Vec<PentestSession> = match db.pentest_sessions().find(doc! {}).await {
Ok(cursor) => collect_cursor_async(cursor).await,
Err(_) => Vec::new(),
};
let total_tool_invocations: u32 = sessions.iter().map(|s| s.tool_invocations).sum();
let total_successes: u32 = sessions.iter().map(|s| s.tool_successes).sum();
let tool_success_rate = if total_tool_invocations == 0 {
100.0
} else {
(total_successes as f64 / total_tool_invocations as f64) * 100.0
};
// Severity distribution from pentest-related DAST findings
let critical = db
.dast_findings()
.count_documents(
doc! { "session_id": { "$exists": true, "$ne": null }, "severity": "critical" },
)
.await
.unwrap_or(0) as u32;
let high = db
.dast_findings()
.count_documents(
doc! { "session_id": { "$exists": true, "$ne": null }, "severity": "high" },
)
.await
.unwrap_or(0) as u32;
let medium = db
.dast_findings()
.count_documents(
doc! { "session_id": { "$exists": true, "$ne": null }, "severity": "medium" },
)
.await
.unwrap_or(0) as u32;
let low = db
.dast_findings()
.count_documents(doc! { "session_id": { "$exists": true, "$ne": null }, "severity": "low" })
.await
.unwrap_or(0) as u32;
let info = db
.dast_findings()
.count_documents(
doc! { "session_id": { "$exists": true, "$ne": null }, "severity": "info" },
)
.await
.unwrap_or(0) as u32;
Ok(Json(ApiResponse {
data: PentestStats {
running_sessions,
total_vulnerabilities,
total_tool_invocations,
tool_success_rate,
severity_distribution: SeverityDistribution {
critical,
high,
medium,
low,
info,
},
},
total: None,
page: None,
}))
}
@@ -0,0 +1,157 @@
use std::convert::Infallible;
use std::sync::Arc;
use std::time::Duration;
use axum::extract::{Extension, Path};
use axum::http::StatusCode;
use axum::response::sse::{Event, KeepAlive, Sse};
use futures_util::stream;
use mongodb::bson::doc;
use tokio_stream::wrappers::BroadcastStream;
use tokio_stream::StreamExt;
use compliance_core::models::pentest::*;
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use super::super::dto::{collect_cursor_async, tenant_db};
type AgentExt = Extension<Arc<ComplianceAgent>>;
/// GET /api/v1/pentest/sessions/:id/stream — SSE endpoint for real-time events
///
/// Replays stored messages/nodes as initial burst, then subscribes to the
/// broadcast channel for live updates. Sends keepalive comments every 15s.
#[tracing::instrument(skip_all, fields(session_id = %id))]
pub async fn session_stream(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Sse<impl futures_util::Stream<Item = Result<Event, Infallible>>>, StatusCode> {
let oid = mongodb::bson::oid::ObjectId::parse_str(&id).map_err(|_| StatusCode::BAD_REQUEST)?;
let db = tenant_db(&agent, &tenant).await?;
// Verify session exists
let _session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
// ── Initial burst: replay stored data ──────────────────────────
let mut initial_events: Vec<Result<Event, Infallible>> = Vec::new();
// Fetch recent messages for this session
let messages: Vec<PentestMessage> = match db
.pentest_messages()
.find(doc! { "session_id": &id })
.sort(doc! { "created_at": 1 })
.limit(100)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(_) => Vec::new(),
};
// Fetch recent attack chain nodes
let nodes: Vec<AttackChainNode> = match db
.attack_chain_nodes()
.find(doc! { "session_id": &id })
.sort(doc! { "started_at": 1 })
.limit(100)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(_) => Vec::new(),
};
for msg in &messages {
let event_data = serde_json::json!({
"type": "message",
"role": msg.role,
"content": msg.content,
"created_at": msg.created_at.to_rfc3339(),
});
if let Ok(data) = serde_json::to_string(&event_data) {
initial_events.push(Ok(Event::default().event("message").data(data)));
}
}
for node in &nodes {
let event_data = serde_json::json!({
"type": "tool_execution",
"node_id": node.node_id,
"tool_name": node.tool_name,
"status": node.status,
"findings_produced": node.findings_produced,
});
if let Ok(data) = serde_json::to_string(&event_data) {
initial_events.push(Ok(Event::default().event("tool").data(data)));
}
}
// Add current session status event
let session = db
.pentest_sessions()
.find_one(doc! { "_id": oid })
.await
.ok()
.flatten();
if let Some(s) = session {
let status_data = serde_json::json!({
"type": "status",
"status": s.status,
"findings_count": s.findings_count,
"tool_invocations": s.tool_invocations,
});
if let Ok(data) = serde_json::to_string(&status_data) {
initial_events.push(Ok(Event::default().event("status").data(data)));
}
}
// ── Live stream: subscribe to broadcast ────────────────────────
let live_stream = if let Some(rx) = agent.subscribe_session(&id) {
let broadcast = BroadcastStream::new(rx).filter_map(|result| match result {
Ok(event) => {
if let Ok(data) = serde_json::to_string(&event) {
let event_type = match &event {
PentestEvent::ToolStart { .. } => "tool_start",
PentestEvent::ToolComplete { .. } => "tool_complete",
PentestEvent::Finding { .. } => "finding",
PentestEvent::Message { .. } => "message",
PentestEvent::Complete { .. } => "complete",
PentestEvent::Error { .. } => "error",
PentestEvent::Thinking { .. } => "thinking",
PentestEvent::Paused => "paused",
PentestEvent::Resumed => "resumed",
};
Some(Ok(Event::default().event(event_type).data(data)))
} else {
None
}
}
Err(_) => None,
});
// Box to unify types
Box::pin(broadcast)
as std::pin::Pin<Box<dyn futures_util::Stream<Item = Result<Event, Infallible>> + Send>>
} else {
// No active broadcast — return empty stream
Box::pin(stream::empty())
as std::pin::Pin<Box<dyn futures_util::Stream<Item = Result<Event, Infallible>> + Send>>
};
// Chain initial burst + live stream
let combined = stream::iter(initial_events).chain(live_stream);
Ok(Sse::new(combined).keep_alive(
KeepAlive::new()
.interval(Duration::from_secs(15))
.text("keepalive"),
))
}
+390
View File
@@ -0,0 +1,390 @@
use axum::extract::{Extension, Query};
use axum::http::{header, StatusCode};
use axum::response::IntoResponse;
use axum::Json;
use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::SbomEntry;
use compliance_core::tenant_ctx::TenantCtx;
const COPYLEFT_LICENSES: &[&str] = &[
"GPL-2.0",
"GPL-2.0-only",
"GPL-2.0-or-later",
"GPL-3.0",
"GPL-3.0-only",
"GPL-3.0-or-later",
"AGPL-3.0",
"AGPL-3.0-only",
"AGPL-3.0-or-later",
"LGPL-2.1",
"LGPL-2.1-only",
"LGPL-2.1-or-later",
"LGPL-3.0",
"LGPL-3.0-only",
"LGPL-3.0-or-later",
"MPL-2.0",
];
#[tracing::instrument(skip_all)]
pub async fn sbom_filters(
Extension(agent): AgentExt,
tenant: TenantCtx,
) -> Result<Json<serde_json::Value>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let managers: Vec<String> = db
.sbom_entries()
.distinct("package_manager", doc! {})
.await
.unwrap_or_default()
.into_iter()
.filter_map(|v| v.as_str().map(|s| s.to_string()))
.filter(|s| !s.is_empty() && s != "unknown" && s != "file")
.collect();
let licenses: Vec<String> = db
.sbom_entries()
.distinct("license", doc! {})
.await
.unwrap_or_default()
.into_iter()
.filter_map(|v| v.as_str().map(|s| s.to_string()))
.filter(|s| !s.is_empty())
.collect();
Ok(Json(serde_json::json!({
"package_managers": managers,
"licenses": licenses,
})))
}
#[tracing::instrument(skip_all, fields(repo_id = ?filter.repo_id, package_manager = ?filter.package_manager))]
pub async fn list_sbom(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(filter): Query<SbomFilter>,
) -> ApiResult<Vec<SbomEntry>> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let mut query = doc! {};
if let Some(repo_id) = &filter.repo_id {
query.insert("repo_id", repo_id);
}
if let Some(pm) = &filter.package_manager {
query.insert("package_manager", pm);
}
if let Some(q) = &filter.q {
if !q.is_empty() {
query.insert("name", doc! { "$regex": q, "$options": "i" });
}
}
if let Some(has_vulns) = filter.has_vulns {
if has_vulns {
query.insert("known_vulnerabilities", doc! { "$exists": true, "$ne": [] });
} else {
query.insert("known_vulnerabilities", doc! { "$size": 0 });
}
}
if let Some(license) = &filter.license {
query.insert("license", license);
}
let skip = (filter.page.saturating_sub(1)) * filter.limit as u64;
let total = db
.sbom_entries()
.count_documents(query.clone())
.await
.unwrap_or(0);
let entries = match db
.sbom_entries()
.find(query)
.sort(doc! { "name": 1 })
.skip(skip)
.limit(filter.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch SBOM entries: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: entries,
total: Some(total),
page: Some(filter.page),
}))
}
#[tracing::instrument(skip_all)]
pub async fn export_sbom(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<SbomExportParams>,
) -> Result<impl IntoResponse, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let entries: Vec<SbomEntry> = match db
.sbom_entries()
.find(doc! { "repo_id": &params.repo_id })
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch SBOM entries for export: {e}");
Vec::new()
}
};
let body = if params.format == "spdx" {
// SPDX 2.3 format
let packages: Vec<serde_json::Value> = entries
.iter()
.enumerate()
.map(|(i, e)| {
serde_json::json!({
"SPDXID": format!("SPDXRef-Package-{i}"),
"name": e.name,
"versionInfo": e.version,
"downloadLocation": "NOASSERTION",
"licenseConcluded": e.license.as_deref().unwrap_or("NOASSERTION"),
"externalRefs": e.purl.as_ref().map(|p| vec![serde_json::json!({
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": p,
})]).unwrap_or_default(),
})
})
.collect();
serde_json::json!({
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": format!("sbom-{}", params.repo_id),
"documentNamespace": format!("https://compliance-scanner/sbom/{}", params.repo_id),
"packages": packages,
})
} else {
// CycloneDX 1.5 format
let components: Vec<serde_json::Value> = entries
.iter()
.map(|e| {
let mut comp = serde_json::json!({
"type": "library",
"name": e.name,
"version": e.version,
"group": e.package_manager,
});
if let Some(purl) = &e.purl {
comp["purl"] = serde_json::Value::String(purl.clone());
}
if let Some(license) = &e.license {
comp["licenses"] = serde_json::json!([{ "license": { "id": license } }]);
}
if !e.known_vulnerabilities.is_empty() {
comp["vulnerabilities"] = serde_json::json!(
e.known_vulnerabilities.iter().map(|v| serde_json::json!({
"id": v.id,
"source": { "name": v.source },
"ratings": v.severity.as_ref().map(|s| vec![serde_json::json!({"severity": s})]).unwrap_or_default(),
})).collect::<Vec<_>>()
);
}
comp
})
.collect();
serde_json::json!({
"bomFormat": "CycloneDX",
"specVersion": "1.5",
"version": 1,
"metadata": {
"component": {
"type": "application",
"name": format!("repo-{}", params.repo_id),
}
},
"components": components,
})
};
let json_str =
serde_json::to_string_pretty(&body).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let filename = if params.format == "spdx" {
format!("sbom-{}-spdx.json", params.repo_id)
} else {
format!("sbom-{}-cyclonedx.json", params.repo_id)
};
let disposition = format!("attachment; filename=\"{filename}\"");
Ok((
[
(
header::CONTENT_TYPE,
header::HeaderValue::from_static("application/json"),
),
(
header::CONTENT_DISPOSITION,
header::HeaderValue::from_str(&disposition)
.unwrap_or_else(|_| header::HeaderValue::from_static("attachment")),
),
],
json_str,
))
}
#[tracing::instrument(skip_all)]
pub async fn license_summary(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<SbomFilter>,
) -> ApiResult<Vec<LicenseSummary>> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let mut query = doc! {};
if let Some(repo_id) = &params.repo_id {
query.insert("repo_id", repo_id);
}
let entries: Vec<SbomEntry> = match db.sbom_entries().find(query).await {
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch SBOM entries for license summary: {e}");
Vec::new()
}
};
let mut license_map: std::collections::HashMap<String, Vec<String>> =
std::collections::HashMap::new();
for entry in &entries {
let lic = entry.license.as_deref().unwrap_or("Unknown").to_string();
license_map.entry(lic).or_default().push(entry.name.clone());
}
let mut summaries: Vec<LicenseSummary> = license_map
.into_iter()
.map(|(license, packages)| {
let is_copyleft = COPYLEFT_LICENSES
.iter()
.any(|c| license.to_uppercase().contains(&c.to_uppercase()));
LicenseSummary {
license,
count: packages.len() as u64,
is_copyleft,
packages,
}
})
.collect();
summaries.sort_by_key(|s| std::cmp::Reverse(s.count));
Ok(Json(ApiResponse {
data: summaries,
total: None,
page: None,
}))
}
#[tracing::instrument(skip_all)]
pub async fn sbom_diff(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<SbomDiffParams>,
) -> ApiResult<SbomDiffResult> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let entries_a: Vec<SbomEntry> = match db
.sbom_entries()
.find(doc! { "repo_id": &params.repo_a })
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch SBOM entries for repo_a: {e}");
Vec::new()
}
};
let entries_b: Vec<SbomEntry> = match db
.sbom_entries()
.find(doc! { "repo_id": &params.repo_b })
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch SBOM entries for repo_b: {e}");
Vec::new()
}
};
// Build maps by (name, package_manager) -> version
let map_a: std::collections::HashMap<(String, String), String> = entries_a
.iter()
.map(|e| {
(
(e.name.clone(), e.package_manager.clone()),
e.version.clone(),
)
})
.collect();
let map_b: std::collections::HashMap<(String, String), String> = entries_b
.iter()
.map(|e| {
(
(e.name.clone(), e.package_manager.clone()),
e.version.clone(),
)
})
.collect();
let mut only_in_a = Vec::new();
let mut version_changed = Vec::new();
let mut common_count: u64 = 0;
for (key, ver_a) in &map_a {
match map_b.get(key) {
None => only_in_a.push(SbomDiffEntry {
name: key.0.clone(),
version: ver_a.clone(),
package_manager: key.1.clone(),
}),
Some(ver_b) if ver_a != ver_b => {
version_changed.push(SbomVersionDiff {
name: key.0.clone(),
package_manager: key.1.clone(),
version_a: ver_a.clone(),
version_b: ver_b.clone(),
});
}
Some(_) => common_count += 1,
}
}
let only_in_b: Vec<SbomDiffEntry> = map_b
.iter()
.filter(|(key, _)| !map_a.contains_key(key))
.map(|(key, ver)| SbomDiffEntry {
name: key.0.clone(),
version: ver.clone(),
package_manager: key.1.clone(),
})
.collect();
Ok(Json(ApiResponse {
data: SbomDiffResult {
only_in_a,
only_in_b,
version_changed,
common_count,
},
total: None,
page: None,
}))
}
@@ -0,0 +1,40 @@
use axum::extract::{Extension, Query};
use axum::Json;
use mongodb::bson::doc;
use super::dto::*;
use compliance_core::models::ScanRun;
use compliance_core::tenant_ctx::TenantCtx;
#[tracing::instrument(skip_all)]
pub async fn list_scan_runs(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> ApiResult<Vec<ScanRun>> {
let db = tenant_db(&agent, &tenant).await?;
let db = &db;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db.scan_runs().count_documents(doc! {}).await.unwrap_or(0);
let scans = match db
.scan_runs()
.find(doc! {})
.sort(doc! { "started_at": -1 })
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch scan runs: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: scans,
total: Some(total),
page: Some(params.page),
}))
}
-1
View File
@@ -1,4 +1,3 @@
pub mod auth_middleware;
pub mod handlers;
pub mod routes;
pub mod server;
+110 -16
View File
@@ -2,7 +2,6 @@ use axum::routing::{delete, get, patch, post};
use axum::Router;
use crate::api::handlers;
use crate::webhooks;
pub fn build_router() -> Router {
Router::new()
@@ -12,19 +11,36 @@ pub fn build_router() -> Router {
"/api/v1/settings/ssh-public-key",
get(handlers::get_ssh_public_key),
)
.route("/api/v1/repositories", get(handlers::list_repositories))
.route("/api/v1/repositories", post(handlers::add_repository))
// Unified onboarding targets (#131).
.route(
"/api/v1/repositories/{id}/scan",
post(handlers::trigger_scan),
"/api/v1/targets",
get(handlers::onboarding::list_targets).post(handlers::onboarding::create_target),
)
.route(
"/api/v1/repositories/{id}",
delete(handlers::delete_repository).patch(handlers::update_repository),
"/api/v1/targets/{id}",
get(handlers::onboarding::get_target)
.patch(handlers::onboarding::update_target)
.delete(handlers::onboarding::delete_target),
)
.route(
"/api/v1/repositories/{id}/webhook-config",
get(handlers::get_webhook_config),
"/api/v1/targets/{id}/artifacts",
post(handlers::onboarding::add_artifact),
)
.route(
"/api/v1/targets/{id}/artifacts/upload",
post(handlers::onboarding::upload_artifact),
)
.route(
"/api/v1/targets/{id}/applicable-scans",
get(handlers::onboarding::applicable_scans_for_target),
)
.route(
"/api/v1/targets/{id}/detect",
post(handlers::onboarding::detect_target),
)
.route(
"/api/v1/targets/{id}/scan",
post(handlers::onboarding::trigger_target_scan),
)
.route("/api/v1/findings", get(handlers::list_findings))
.route("/api/v1/findings/{id}", get(handlers::get_finding))
@@ -47,6 +63,15 @@ pub fn build_router() -> Router {
.route("/api/v1/sbom/diff", get(handlers::sbom_diff))
.route("/api/v1/issues", get(handlers::list_issues))
.route("/api/v1/scan-runs", get(handlers::list_scan_runs))
// MCP token management (per-tenant API tokens for the MCP server)
.route(
"/api/v1/mcp-tokens",
get(handlers::mcp_tokens::list_mcp_tokens).post(handlers::mcp_tokens::create_mcp_token),
)
.route(
"/api/v1/mcp-tokens/{id}",
delete(handlers::mcp_tokens::revoke_mcp_token),
)
// Graph API endpoints
.route("/api/v1/graph/{repo_id}", get(handlers::graph::get_graph))
.route(
@@ -99,17 +124,86 @@ pub fn build_router() -> Router {
"/api/v1/chat/{repo_id}/status",
get(handlers::chat::embedding_status),
)
// Webhook endpoints (proxied through dashboard)
// Help chat (documentation-grounded Q&A)
.route("/api/v1/help/chat", post(handlers::help_chat::help_chat))
// CVE notification endpoints
.route(
"/webhook/github/{repo_id}",
post(webhooks::github::handle_github_webhook),
"/api/v1/notifications",
get(handlers::notifications::list_notifications),
)
.route(
"/webhook/gitlab/{repo_id}",
post(webhooks::gitlab::handle_gitlab_webhook),
"/api/v1/notifications/count",
get(handlers::notifications::notification_count),
)
.route(
"/webhook/gitea/{repo_id}",
post(webhooks::gitea::handle_gitea_webhook),
"/api/v1/notifications/read-all",
post(handlers::notifications::mark_all_read),
)
.route(
"/api/v1/notifications/{id}/read",
patch(handlers::notifications::mark_read),
)
.route(
"/api/v1/notifications/{id}/dismiss",
patch(handlers::notifications::dismiss_notification),
)
// Pentest API endpoints
.route(
"/api/v1/pentest/lookup-repo",
get(handlers::pentest::lookup_repo),
)
.route(
"/api/v1/pentest/sessions",
get(handlers::pentest::list_sessions).post(handlers::pentest::create_session),
)
.route(
"/api/v1/pentest/sessions/{id}",
get(handlers::pentest::get_session),
)
.route(
"/api/v1/pentest/sessions/{id}/chat",
post(handlers::pentest::send_message),
)
.route(
"/api/v1/pentest/sessions/{id}/stop",
post(handlers::pentest::stop_session),
)
.route(
"/api/v1/pentest/sessions/{id}/pause",
post(handlers::pentest::pause_session),
)
.route(
"/api/v1/pentest/sessions/{id}/resume",
post(handlers::pentest::resume_session),
)
.route(
"/api/v1/pentest/sessions/{id}/stream",
get(handlers::pentest::session_stream),
)
.route(
"/api/v1/pentest/sessions/{id}/attack-chain",
get(handlers::pentest::get_attack_chain),
)
.route(
"/api/v1/pentest/sessions/{id}/messages",
get(handlers::pentest::get_messages),
)
.route(
"/api/v1/pentest/sessions/{id}/findings",
get(handlers::pentest::get_session_findings),
)
.route(
"/api/v1/pentest/sessions/{id}/export",
post(handlers::pentest::export_session_report),
)
.route(
"/api/v1/pentest/stats",
get(handlers::pentest::pentest_stats),
)
// Webhook routes live on the separate webhook server (port 3002,
// see crate::webhooks::server). The M7.2-C tenant-in-URL form is
// `/webhook/{tenant_id}/{platform}/{repo_id}` and the handlers
// expect a (tenant_id, repo_id) path tuple. Anything mounting
// them here on the API server would mismatch the handler
// signature, so the routes are not exported.
}
+99 -6
View File
@@ -1,20 +1,102 @@
use std::sync::Arc;
use axum::{middleware, Extension};
use axum::extract::{DefaultBodyLimit, Request};
use axum::http::HeaderValue;
use axum::middleware::Next;
use axum::response::Response;
use axum::routing::{delete, get};
use axum::{middleware, Extension, Router};
use tokio::sync::RwLock;
use tower_http::cors::CorsLayer;
use tower_http::set_header::SetResponseHeaderLayer;
use tower_http::trace::TraceLayer;
use compliance_core::auth::{require_jwt_auth, require_tenant_status, JwksState};
use compliance_core::{TenantContext, TenantStatus};
use crate::agent::ComplianceAgent;
use crate::api::auth_middleware::{require_jwt_auth, JwksState};
use crate::api::handlers;
use crate::api::routes;
use crate::error::AgentError;
/// Synthetic tenant id used when Keycloak isn't configured (local dev,
/// `cargo run` against a bare Mongo). Lets the handler stack stay
/// uniformly tenant-scoped without the operator having to spin up KC
/// just to poke at the API. Override via `DEV_TENANT_ID`.
const DEFAULT_DEV_TENANT_ID: &str = "dev";
/// Inject a synthetic [`TenantContext`] for any request that lacks one.
/// Only mounted when Keycloak is NOT configured; with KC, the real
/// `require_jwt_auth` middleware owns this and we never reach here
/// without a context.
///
/// Public so the integration-test harness can mount it without
/// duplicating the synthetic-context shape.
pub async fn inject_dev_tenant(mut request: Request, next: Next) -> Response {
if request.extensions().get::<TenantContext>().is_none() {
let tenant_id =
std::env::var("DEV_TENANT_ID").unwrap_or_else(|_| DEFAULT_DEV_TENANT_ID.to_string());
let ctx = TenantContext {
tenant_slug: tenant_id.clone(),
tenant_id,
org_roles: vec![],
products: vec![],
plan: "dev".to_string(),
status: TenantStatus::Active,
user_id: "dev-user".to_string(),
user_name: None,
};
request.extensions_mut().insert(ctx);
}
next.run(request).await
}
pub async fn start_api_server(agent: ComplianceAgent, port: u16) -> Result<(), AgentError> {
// Admin sub-router. Routes are only mounted when ADMIN_API_TOKEN is
// configured — without it, the paths don't exist at all (404 rather
// than 401), so an operator who hasn't opted in can't fingerprint
// the surface area.
let admin_router: Router = if agent.config.admin_api_token.is_some() {
tracing::info!("Admin API enabled — /api/v1/admin/* mounted behind ADMIN_API_TOKEN bearer");
Router::new()
.route(
"/api/v1/admin/tenants",
get(handlers::admin::list_tenant_dbs),
)
.route(
"/api/v1/admin/tenants/{tenant_id}",
delete(handlers::admin::drop_tenant_db),
)
.layer(middleware::from_fn(handlers::admin::require_admin_token))
} else {
Router::new()
};
let mut app = routes::build_router()
.merge(admin_router)
// Allow large artifact uploads (PLC .projectarchive, firmware images,
// mobile packages) — axum's default request-body limit is only 2 MiB.
.layer(DefaultBodyLimit::max(512 * 1024 * 1024))
.layer(Extension(Arc::new(agent.clone())))
.layer(CorsLayer::permissive())
.layer(TraceLayer::new_for_http());
.layer(TraceLayer::new_for_http())
// Security headers (defense-in-depth, primary enforcement via Traefik)
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::STRICT_TRANSPORT_SECURITY,
HeaderValue::from_static("max-age=31536000; includeSubDomains"),
))
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::X_FRAME_OPTIONS,
HeaderValue::from_static("DENY"),
))
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::X_CONTENT_TYPE_OPTIONS,
HeaderValue::from_static("nosniff"),
))
.layer(SetResponseHeaderLayer::overriding(
axum::http::header::REFERRER_POLICY,
HeaderValue::from_static("strict-origin-when-cross-origin"),
));
if let (Some(kc_url), Some(kc_realm)) =
(&agent.config.keycloak_url, &agent.config.keycloak_realm)
@@ -25,11 +107,22 @@ pub async fn start_api_server(agent: ComplianceAgent, port: u16) -> Result<(), A
jwks_url,
};
tracing::info!("Keycloak JWT auth enabled for realm '{kc_realm}'");
// Layers execute outermost-first. Extension(jwks_state) must run
// before require_jwt_auth so the middleware can read it; the
// status gate runs after JWT so TenantContext is in extensions.
app = app
.layer(Extension(jwks_state))
.layer(middleware::from_fn(require_jwt_auth));
.layer(middleware::from_fn(require_tenant_status))
.layer(middleware::from_fn(require_jwt_auth))
.layer(Extension(jwks_state));
} else {
tracing::warn!("Keycloak not configured - API endpoints are unprotected");
let tenant_id =
std::env::var("DEV_TENANT_ID").unwrap_or_else(|_| DEFAULT_DEV_TENANT_ID.to_string());
tracing::warn!(
tenant_id = %tenant_id,
"Keycloak not configured — running unauthenticated against the dev tenant. \
DO NOT use in any environment with real customer data."
);
app = app.layer(middleware::from_fn(inject_dev_tenant));
}
let addr = format!("0.0.0.0:{port}");
+217
View File
@@ -0,0 +1,217 @@
//! Firmware classification via tramiton.
//!
//! tramiton is the company's firmware build/repro engine; we do not re-implement
//! its detection. We depend on `tramiton-core` directly (same-company IP) and run
//! its provider analysis in-process behind a [`FirmwareDetector`] port, mapping
//! tramiton's `BuildPlan` onto a [`TargetType`]. A deterministic
//! [`MockFirmwareDetector`] backs the tests so CI unit tests need neither the
//! tramiton sources nor a real firmware tree.
use std::path::Path;
use compliance_core::error::CoreError;
use compliance_core::models::{DetectedFact, TargetType};
use compliance_core::traits::ClassifierVerdict;
/// A minimal firmware-detection summary, mapped from tramiton's `BuildPlan`.
/// Kept small and tramiton-independent so the classifier and the test mock don't
/// need to construct a full tramiton plan.
#[derive(Debug, Clone, Default)]
pub struct FirmwareDetection {
/// The detecting provider (e.g. `zephyr`, `cmake`, `source-archaeology`).
pub provider: String,
/// Detection confidence: `low` | `medium` | `high`.
pub confidence: String,
/// Build-system label (e.g. `Zephyr`, `ESP-IDF`, `CMake`).
pub build_system: String,
/// Framework, when known (`zephyr`, `esp-idf`, `bare-metal`, ...).
pub framework: Option<String>,
/// Target board / MCU / arch.
pub target: FirmwareTarget,
/// Unresolved gaps in the plan.
pub gaps: Vec<String>,
}
/// The detected firmware target (board / MCU / arch).
#[derive(Debug, Clone, Default)]
pub struct FirmwareTarget {
/// Board name.
pub board: Option<String>,
/// MCU part.
pub mcu: Option<String>,
/// Architecture.
pub arch: Option<String>,
}
/// A source of tramiton firmware detection.
#[allow(async_fn_in_trait)]
pub trait FirmwareDetector: Send + Sync {
/// Run detection over a path, returning a firmware detection if tramiton
/// could form a build plan.
async fn detect(&self, path: &Path) -> Result<Option<FirmwareDetection>, CoreError>;
}
/// Uses `tramiton-core` in-process. The analysis is blocking (filesystem walk),
/// so it runs on a blocking thread to avoid stalling the async runtime. A path
/// with no recognizable build system yields `Ok(None)`.
pub struct TramitonNative;
impl FirmwareDetector for TramitonNative {
async fn detect(&self, path: &Path) -> Result<Option<FirmwareDetection>, CoreError> {
let path = path.to_path_buf();
let plan = tokio::task::spawn_blocking(move || {
let repo = tramiton_core::Repo::new(&path);
tramiton_core::provider::analyze(&repo)
})
.await
.map_err(|e| CoreError::Other(format!("tramiton detect task join error: {e}")))?
.map_err(|e| CoreError::Other(format!("tramiton analyze error: {e}")))?;
Ok(plan.map(|bp| detection_from_build_plan(&bp)))
}
}
/// Map tramiton's `BuildPlan` onto our minimal detection summary.
fn detection_from_build_plan(bp: &tramiton_core::BuildPlan) -> FirmwareDetection {
FirmwareDetection {
provider: bp.provider.clone(),
confidence: bp.confidence.to_string(),
build_system: bp.build_system.label().to_string(),
framework: bp.framework.clone(),
target: FirmwareTarget {
board: bp.target.board.clone(),
mcu: bp.target.mcu.clone(),
arch: bp.target.arch.clone(),
},
gaps: bp.gaps.clone(),
}
}
/// Map a firmware detection to a target type. Framework/build-system signals
/// distinguish RTOS from bare-metal from Yocto.
pub fn detection_to_target_type(det: &FirmwareDetection) -> TargetType {
let framework = det.framework.as_deref().unwrap_or("").to_lowercase();
let build_system = det.build_system.to_lowercase();
let signal = format!("{framework} {build_system} {}", det.provider.to_lowercase());
const RTOS: [&str; 6] = ["zephyr", "esp-idf", "freertos", "nuttx", "riot", "chibios"];
if signal.contains("bitbake") || signal.contains("yocto") || signal.contains("openembedded") {
TargetType::EmbeddedLinuxYocto
} else if RTOS.iter().any(|k| signal.contains(k)) {
TargetType::FirmwareRtos
} else {
TargetType::FirmwareBareMetal
}
}
/// Map tramiton's confidence label to a `[0,1]` score.
fn confidence_score(label: &str) -> f32 {
match label.to_lowercase().as_str() {
"high" => 0.9,
"medium" => 0.6,
"low" => 0.3,
_ => 0.4,
}
}
/// Turn a firmware detection into a classifier verdict, carrying the MCU / board
/// / build-system as facts.
pub fn detection_to_verdict(det: &FirmwareDetection) -> ClassifierVerdict {
let target_type = detection_to_target_type(det);
let mut facts = vec![DetectedFact::new(
"build_system",
det.build_system.clone(),
"tramiton",
)];
if let Some(fw) = &det.framework {
facts.push(DetectedFact::new("framework", fw.clone(), "tramiton"));
}
if let Some(mcu) = &det.target.mcu {
facts.push(DetectedFact::new("mcu", mcu.clone(), "tramiton"));
}
if let Some(board) = &det.target.board {
facts.push(DetectedFact::new("board", board.clone(), "tramiton"));
}
if let Some(arch) = &det.target.arch {
facts.push(DetectedFact::new("arch", arch.clone(), "tramiton"));
}
ClassifierVerdict {
target_type,
confidence: confidence_score(&det.confidence),
facts,
rationale: format!(
"tramiton detected build system '{}'{}",
det.build_system,
det.framework
.as_ref()
.map(|f| format!(" (framework {f})"))
.unwrap_or_default()
),
}
}
/// A deterministic [`FirmwareDetector`] for tests — returns a preset detection.
pub struct MockFirmwareDetector {
/// The detection to return (or `None` for "no detection").
pub detection: Option<FirmwareDetection>,
}
impl FirmwareDetector for MockFirmwareDetector {
async fn detect(&self, _path: &Path) -> Result<Option<FirmwareDetection>, CoreError> {
Ok(self.detection.clone())
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
fn detection(build_system: &str, framework: Option<&str>) -> FirmwareDetection {
FirmwareDetection {
provider: build_system.to_string(),
confidence: "high".to_string(),
build_system: build_system.to_string(),
framework: framework.map(|s| s.to_string()),
target: FirmwareTarget {
mcu: Some("stm32f429".to_string()),
..Default::default()
},
gaps: Vec::new(),
}
}
#[test]
fn zephyr_maps_to_rtos() {
assert_eq!(
detection_to_target_type(&detection("zephyr", Some("zephyr"))),
TargetType::FirmwareRtos
);
}
#[test]
fn bare_cmake_maps_to_bare_metal() {
assert_eq!(
detection_to_target_type(&detection("cmake", Some("bare-metal"))),
TargetType::FirmwareBareMetal
);
}
#[test]
fn bitbake_maps_to_yocto() {
assert_eq!(
detection_to_target_type(&detection("bitbake", None)),
TargetType::EmbeddedLinuxYocto
);
}
#[test]
fn verdict_carries_mcu_fact_and_confidence() {
let v = detection_to_verdict(&detection("esp-idf", Some("esp-idf")));
assert_eq!(v.target_type, TargetType::FirmwareRtos);
assert!((v.confidence - 0.9).abs() < f32::EPSILON);
assert!(v
.facts
.iter()
.any(|f| f.key == "mcu" && f.value == "stm32f429"));
}
}
+357
View File
@@ -0,0 +1,357 @@
//! Heuristic target-type classification from artifact kinds and source markers.
//!
//! Complements the tramiton firmware detector: this handles web / backend /
//! mobile / desktop / PLC by sniffing manifest files and file extensions in the
//! ingested code trees, plus strong priors from the artifact kinds themselves
//! (a PLC-project artifact is a PLC target; an `.ipa` is an iOS app).
use std::collections::HashSet;
use std::fs;
use std::path::Path;
use compliance_core::error::CoreError;
use compliance_core::models::{ArtifactKind, DetectedFact, TargetType};
use compliance_core::traits::{ClassificationInput, ClassifierVerdict, TargetClassifier};
/// Max directory depth scanned for marker files.
const SCAN_DEPTH: usize = 2;
/// Markers collected from a code tree.
#[derive(Default)]
struct Markers {
files: HashSet<String>,
dirs: HashSet<String>,
exts: HashSet<String>,
}
impl Markers {
fn has_file(&self, name: &str) -> bool {
self.files.contains(name)
}
fn has_ext(&self, ext: &str) -> bool {
self.exts.contains(ext)
}
fn any_dir_ends_with(&self, suffix: &str) -> bool {
self.dirs.iter().any(|d| d.ends_with(suffix))
}
}
/// Recursively collect marker file/dir/extension names up to [`SCAN_DEPTH`].
fn collect_markers(root: &Path) -> Markers {
let mut m = Markers::default();
scan_dir(root, 0, &mut m);
m
}
fn scan_dir(dir: &Path, depth: usize, m: &mut Markers) {
let Ok(entries) = fs::read_dir(dir) else {
return;
};
for entry in entries.flatten() {
let path = entry.path();
let name = entry.file_name().to_string_lossy().to_lowercase();
if path.is_dir() {
m.dirs.insert(name);
if depth < SCAN_DEPTH {
scan_dir(&path, depth + 1, m);
}
} else {
if let Some(ext) = path.extension() {
m.exts.insert(ext.to_string_lossy().to_lowercase());
}
m.files.insert(name);
}
}
}
/// Whether a `package.json` at `root` looks like a front-end app.
fn package_json_is_frontend(root: &Path) -> bool {
let Ok(content) = fs::read_to_string(root.join("package.json")) else {
return false;
};
let c = content.to_lowercase();
["react", "next", "vue", "@angular", "svelte", "vite"]
.iter()
.any(|f| c.contains(f))
}
/// The heuristic classifier: artifact-kind priors + source-tree markers.
pub struct HeuristicClassifier;
impl HeuristicClassifier {
/// Verdicts from the artifact kinds alone (no filesystem needed).
fn kind_priors(&self, input: &ClassificationInput<'_>) -> Vec<ClassifierVerdict> {
let mut out = Vec::new();
for a in input.artifacts {
let lower = a.source_ref.to_lowercase();
match a.kind {
ArtifactKind::PlcProject => out.push(verdict(
TargetType::PlcSps,
0.85,
"PLC project artifact",
vec![],
)),
ArtifactKind::MobilePackage => {
let (tt, why) = if lower.ends_with(".ipa") {
(TargetType::IosApp, "iOS package (.ipa)")
} else {
(TargetType::AndroidApp, "Android package (.apk/.aab)")
};
out.push(verdict(tt, 0.85, why, vec![]));
}
ArtifactKind::ContainerImage => out.push(verdict(
TargetType::BackendService,
0.4,
"container image",
vec![],
)),
ArtifactKind::FirmwareImage => out.push(verdict(
TargetType::FirmwareBareMetal,
0.35,
"firmware image (pending tramiton detection)",
vec![],
)),
ArtifactKind::LiveUrl if input.artifacts.len() == 1 => {
out.push(verdict(TargetType::WebApp, 0.3, "live URL only", vec![]))
}
_ => {}
}
}
out
}
/// Verdicts from scanning the ingested code trees for manifest markers.
fn source_verdicts(&self, input: &ClassificationInput<'_>) -> Vec<ClassifierVerdict> {
let mut out = Vec::new();
for a in input.artifacts {
if !matches!(a.kind, ArtifactKind::GitRepo | ArtifactKind::SourceArchive) {
continue;
}
let Some(path) = input.working_paths.get(&a.id) else {
continue;
};
let m = collect_markers(path);
// Mobile (checked first — strongest signal).
if m.has_file("androidmanifest.xml") || m.has_ext("apk") || m.has_ext("aab") {
out.push(verdict(
TargetType::AndroidApp,
0.8,
"Android manifest / gradle",
facts_lang("kotlin/java"),
));
}
if m.any_dir_ends_with(".xcodeproj")
|| m.has_file("info.plist")
|| m.has_file("podfile")
|| m.has_ext("ipa")
{
out.push(verdict(
TargetType::IosApp,
0.8,
"Xcode project / Info.plist",
facts_lang("swift/objc"),
));
}
// Desktop.
if m.has_ext("sln")
|| m.has_ext("csproj")
|| m.has_ext("vcxproj")
|| m.has_ext("desktop")
{
out.push(verdict(
TargetType::DesktopApp,
0.7,
"desktop project files",
facts_lang("dotnet/native"),
));
}
// PLC.
if m.has_ext("st") {
out.push(verdict(
TargetType::PlcSps,
0.8,
"Structured Text sources",
facts_lang("iec-61131-3"),
));
}
// Web vs backend from package.json.
if m.has_file("package.json") {
if package_json_is_frontend(path) {
out.push(verdict(
TargetType::WebApp,
0.65,
"package.json with a front-end framework",
facts_lang("javascript"),
));
} else {
out.push(verdict(
TargetType::BackendService,
0.55,
"package.json (no front-end framework)",
facts_lang("javascript"),
));
}
}
// Backend languages.
for (file, lang) in [
("cargo.toml", "rust"),
("go.mod", "go"),
("pom.xml", "java"),
("requirements.txt", "python"),
("pyproject.toml", "python"),
] {
if m.has_file(file) {
out.push(verdict(
TargetType::BackendService,
0.6,
"backend build manifest",
facts_lang(lang),
));
}
}
// Container-only.
if m.has_file("dockerfile") && out.is_empty() {
out.push(verdict(
TargetType::BackendService,
0.4,
"Dockerfile",
facts_lang("container"),
));
}
}
out
}
}
impl TargetClassifier for HeuristicClassifier {
fn name(&self) -> &str {
"heuristic"
}
async fn classify(
&self,
input: &ClassificationInput<'_>,
) -> Result<Vec<ClassifierVerdict>, CoreError> {
let mut out = self.kind_priors(input);
out.extend(self.source_verdicts(input));
Ok(out)
}
}
fn verdict(
target_type: TargetType,
confidence: f32,
rationale: &str,
facts: Vec<DetectedFact>,
) -> ClassifierVerdict {
ClassifierVerdict {
target_type,
confidence,
facts,
rationale: rationale.to_string(),
}
}
fn facts_lang(lang: &str) -> Vec<DetectedFact> {
vec![DetectedFact::new("language", lang, "heuristic")]
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::Artifact;
use std::collections::HashMap;
use std::path::PathBuf;
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-classify-{}", uuid::Uuid::new_v4()));
fs::create_dir_all(&p).expect("mkdir");
Self(p)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
async fn classify_tree(setup: impl FnOnce(&Path)) -> Vec<ClassifierVerdict> {
let scratch = Scratch::new();
setup(&scratch.0);
let artifact = Artifact::git_repo("https://git/x", "main");
let mut wp = HashMap::new();
wp.insert(artifact.id.clone(), scratch.0.clone());
let artifacts = vec![artifact];
let input = ClassificationInput {
artifacts: &artifacts,
working_paths: &wp,
description: None,
};
HeuristicClassifier
.classify(&input)
.await
.expect("classify")
}
#[tokio::test]
async fn frontend_package_json_is_webapp() {
let v = classify_tree(|root| {
fs::write(
root.join("package.json"),
r#"{"dependencies":{"react":"18"}}"#,
)
.unwrap();
})
.await;
assert!(v.iter().any(|x| x.target_type == TargetType::WebApp));
}
#[tokio::test]
async fn cargo_toml_is_backend() {
let v = classify_tree(|root| {
fs::write(root.join("Cargo.toml"), "[package]\nname='x'").unwrap();
})
.await;
assert!(v
.iter()
.any(|x| x.target_type == TargetType::BackendService));
}
#[tokio::test]
async fn android_manifest_is_android() {
let v = classify_tree(|root| {
fs::write(root.join("AndroidManifest.xml"), "<manifest/>").unwrap();
})
.await;
assert!(v.iter().any(|x| x.target_type == TargetType::AndroidApp));
}
#[tokio::test]
async fn structured_text_is_plc() {
let v = classify_tree(|root| {
fs::write(root.join("main.st"), "PROGRAM main END_PROGRAM").unwrap();
})
.await;
assert!(v.iter().any(|x| x.target_type == TargetType::PlcSps));
}
#[tokio::test]
async fn ipa_artifact_prior_is_ios() {
let artifacts = vec![Artifact::mobile_package("app.ipa")];
let wp = HashMap::new();
let input = ClassificationInput {
artifacts: &artifacts,
working_paths: &wp,
description: None,
};
let v = HeuristicClassifier
.classify(&input)
.await
.expect("classify");
assert!(v.iter().any(|x| x.target_type == TargetType::IosApp));
}
}
+226
View File
@@ -0,0 +1,226 @@
//! Target classification.
//!
//! Runs the classifier registry over a target's artifacts and their ingested
//! working paths, then merges and ranks the verdicts into a [`Classification`].
//! The registry is the heuristic classifier (artifact kinds + source markers)
//! plus the tramiton firmware detector (behind a [`FirmwareDetector`] port).
mod firmware;
mod language;
pub use firmware::{
FirmwareDetection, FirmwareDetector, FirmwareTarget, MockFirmwareDetector, TramitonNative,
};
pub use language::HeuristicClassifier;
use std::collections::HashMap;
use std::path::PathBuf;
use compliance_core::error::CoreError;
use compliance_core::models::{
ArtifactKind, Classification, DetectedFact, OnboardedTarget, TargetType, TargetTypeCandidate,
};
use compliance_core::traits::{ClassificationInput, ClassifierVerdict, TargetClassifier};
use firmware::detection_to_verdict;
/// Classify a target from its artifacts and their ingested working paths, using
/// the heuristic classifier plus the tramiton firmware detector. Verdicts are
/// merged (max confidence per target type) and ranked into a [`Classification`].
pub async fn classify_target<D: FirmwareDetector>(
target: &OnboardedTarget,
working_paths: &HashMap<String, PathBuf>,
firmware_detector: &D,
) -> Result<Classification, CoreError> {
let input = ClassificationInput {
artifacts: &target.artifacts,
working_paths,
description: target.description.as_deref(),
};
let mut verdicts = Vec::new();
let mut detected_by = Vec::new();
let heuristic = HeuristicClassifier.classify(&input).await?;
if !heuristic.is_empty() {
detected_by.push("heuristic".to_string());
}
verdicts.extend(heuristic);
// Tramiton firmware detection over firmware / code working paths.
let mut tramiton_used = false;
for artifact in &target.artifacts {
if !matches!(
artifact.kind,
ArtifactKind::FirmwareImage | ArtifactKind::GitRepo | ArtifactKind::SourceArchive
) {
continue;
}
let Some(path) = working_paths.get(&artifact.id) else {
continue;
};
if let Some(detection) = firmware_detector.detect(path).await? {
verdicts.push(detection_to_verdict(&detection));
tramiton_used = true;
}
}
if tramiton_used {
detected_by.push("tramiton".to_string());
}
Ok(rank(verdicts, detected_by, target.target_type))
}
/// Merge verdicts by target type (keeping the max confidence and its rationale),
/// dedupe facts, rank by descending confidence, and assemble a [`Classification`].
/// Falls back to the declared type when no verdict is produced.
fn rank(
verdicts: Vec<ClassifierVerdict>,
detected_by: Vec<String>,
fallback: TargetType,
) -> Classification {
let mut best: HashMap<TargetType, (f32, String)> = HashMap::new();
let mut facts: Vec<DetectedFact> = Vec::new();
for verdict in verdicts {
for fact in verdict.facts {
if !facts
.iter()
.any(|e| e.key == fact.key && e.value == fact.value)
{
facts.push(fact);
}
}
let entry = best
.entry(verdict.target_type)
.or_insert((0.0, String::new()));
if verdict.confidence > entry.0 {
*entry = (verdict.confidence, verdict.rationale);
}
}
let mut candidates: Vec<TargetTypeCandidate> = best
.into_iter()
.map(
|(target_type, (confidence, rationale))| TargetTypeCandidate {
target_type,
confidence,
rationale,
},
)
.collect();
// Descending confidence; ties broken by type name for deterministic ordering.
candidates.sort_by(|a, b| {
b.confidence
.partial_cmp(&a.confidence)
.unwrap_or(std::cmp::Ordering::Equal)
.then_with(|| a.target_type.to_string().cmp(&b.target_type.to_string()))
});
let suggested = candidates
.first()
.map(|c| c.target_type)
.unwrap_or(fallback);
Classification {
suggested,
candidates,
facts,
detected_by,
detected_at: chrono::Utc::now(),
confirmed: false,
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::Artifact;
use std::fs;
use std::path::Path;
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-classify-mod-{}", uuid::Uuid::new_v4()));
fs::create_dir_all(&p).expect("mkdir");
Self(p)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
fn no_firmware() -> MockFirmwareDetector {
MockFirmwareDetector { detection: None }
}
#[tokio::test]
async fn backend_repo_classifies_as_backend() {
let scratch = Scratch::new();
fs::write(scratch.0.join("go.mod"), "module x").unwrap();
let artifact = Artifact::git_repo("https://git/x", "main");
let mut wp = HashMap::new();
wp.insert(artifact.id.clone(), scratch.0.clone());
let mut target = OnboardedTarget::new("x".to_string(), TargetType::WebApp);
target.artifacts.push(artifact);
let c = classify_target(&target, &wp, &no_firmware())
.await
.expect("classify");
assert_eq!(c.suggested, TargetType::BackendService);
assert!(c.detected_by.contains(&"heuristic".to_string()));
assert!(!c.confirmed);
}
#[tokio::test]
async fn firmware_detector_verdict_ranks_top() {
let scratch = Scratch::new();
fs::write(scratch.0.join("fw.bin"), b"x").unwrap();
let artifact =
Artifact::firmware_image(scratch.0.join("fw.bin").to_string_lossy().to_string());
let mut wp = HashMap::new();
wp.insert(artifact.id.clone(), scratch.0.clone());
let mut target = OnboardedTarget::new("fw".to_string(), TargetType::FirmwareBareMetal);
target.artifacts.push(artifact);
let detector = MockFirmwareDetector {
detection: Some(FirmwareDetection {
provider: "zephyr".to_string(),
confidence: "high".to_string(),
build_system: "zephyr".to_string(),
framework: Some("zephyr".to_string()),
target: FirmwareTarget {
mcu: Some("nrf52840".to_string()),
..Default::default()
},
gaps: vec![],
}),
};
let c = classify_target(&target, &wp, &detector)
.await
.expect("classify");
// tramiton's high-confidence RTOS verdict beats the weak firmware prior.
assert_eq!(c.suggested, TargetType::FirmwareRtos);
assert!(c.detected_by.contains(&"tramiton".to_string()));
assert!(c.facts.iter().any(|f| f.key == "mcu"));
}
#[tokio::test]
async fn no_signal_falls_back_to_declared_type() {
let scratch = Scratch::new();
let _ = Path::new(&scratch.0);
let target = OnboardedTarget::new("empty".to_string(), TargetType::DesktopApp);
let wp = HashMap::new();
let c = classify_target(&target, &wp, &no_firmware())
.await
.expect("classify");
assert_eq!(c.suggested, TargetType::DesktopApp);
assert!(c.candidates.is_empty());
}
}
+15 -1
View File
@@ -42,12 +42,26 @@ pub fn load_config() -> Result<AgentConfig, AgentError> {
.unwrap_or(3001),
scan_schedule: env_var_opt("SCAN_SCHEDULE").unwrap_or_else(|| "0 0 */6 * * *".to_string()),
cve_monitor_schedule: env_var_opt("CVE_MONITOR_SCHEDULE")
.unwrap_or_else(|| "0 0 0 * * *".to_string()),
.unwrap_or_else(|| "0 0 * * * *".to_string()),
git_clone_base_path: env_var_opt("GIT_CLONE_BASE_PATH")
.unwrap_or_else(|| "/tmp/compliance-scanner/repos".to_string()),
artifact_store_base_path: env_var_opt("ARTIFACT_STORE_BASE_PATH")
.unwrap_or_else(|| "/data/compliance-scanner/artifacts".to_string()),
ssh_key_path: env_var_opt("SSH_KEY_PATH")
.unwrap_or_else(|| "/data/compliance-scanner/ssh/id_ed25519".to_string()),
keycloak_url: env_var_opt("KEYCLOAK_URL"),
keycloak_realm: env_var_opt("KEYCLOAK_REALM"),
keycloak_admin_username: env_var_opt("KEYCLOAK_ADMIN_USERNAME"),
keycloak_admin_password: env_secret_opt("KEYCLOAK_ADMIN_PASSWORD"),
pentest_verification_email: env_var_opt("PENTEST_VERIFICATION_EMAIL"),
pentest_imap_host: env_var_opt("PENTEST_IMAP_HOST"),
pentest_imap_port: env_var_opt("PENTEST_IMAP_PORT").and_then(|p| p.parse().ok()),
pentest_imap_tls: env_var_opt("PENTEST_IMAP_TLS")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(true),
pentest_imap_username: env_var_opt("PENTEST_IMAP_USERNAME"),
pentest_imap_password: env_secret_opt("PENTEST_IMAP_PASSWORD"),
admin_api_token: env_secret_opt("ADMIN_API_TOKEN"),
tenant_registry_url: env_var_opt("TENANT_REGISTRY_URL"),
})
}
+342 -14
View File
@@ -1,11 +1,233 @@
use std::sync::Arc;
use dashmap::DashMap;
use mongodb::bson::doc;
use mongodb::options::IndexOptions;
use mongodb::{Client, Collection, IndexModel};
use sha2::{Digest, Sha256};
use compliance_core::models::*;
use compliance_core::TenantContext;
use crate::error::AgentError;
/// Mongo enforces a 63-byte cap on database names (older clusters: 64
/// on Linux, 63 on Windows; we target the conservative limit).
const MAX_DB_NAME_LEN: usize = 63;
/// Hex length of the SHA-256 truncation used for the hash fallback
/// tenant DB name (16 bytes → 32 hex chars). 16 bytes gives ~2^64
/// birthday-collision resistance — at our 10s-100s tenant scale this
/// is effectively impossible to hit.
const HASH_HEX_LEN: usize = 32;
/// Largest `db_prefix` that still guarantees the hash-fallback name
/// fits in the 63-byte cap: `prefix + "_" + 32 hex chars`.
const MAX_PREFIX_LEN: usize = MAX_DB_NAME_LEN - 1 - HASH_HEX_LEN;
/// Per-tenant Mongo connection broker (M7.2 isolation model).
///
/// Holds one [`Client`] and hands out [`Database`] handles physically
/// scoped to `<db_prefix>_<tenant_id>`. The driver is the isolation
/// boundary — a handle for tenant A cannot see tenant B's documents
/// because it is connected to a different database, not because of an
/// application-level filter.
///
/// Index creation runs idempotently the first time each tenant is seen
/// in the process's lifetime. Mongo's `createIndex` is itself idempotent
/// by index name; the in-memory `ensured` set just skips the round-trip.
#[derive(Clone, Debug)]
pub struct DatabasePool {
client: Client,
db_prefix: String,
ensured: Arc<DashMap<String, ()>>,
}
impl DatabasePool {
/// Connect to the cluster and prepare to hand out tenant databases
/// named `<db_prefix>_<tenant_id>`.
///
/// Validates `db_prefix.len() <= MAX_PREFIX_LEN` so the
/// hash-fallback path is provably within Mongo's 63-byte db-name
/// cap. Refuses to construct a pool that could ever produce an
/// over-long name.
pub async fn connect(uri: &str, db_prefix: &str) -> Result<Self, AgentError> {
if db_prefix.len() > MAX_PREFIX_LEN {
return Err(AgentError::Other(format!(
"db_prefix '{db_prefix}' is {} chars; max is {MAX_PREFIX_LEN} so the \
hash-fallback tenant DB name fits Mongo's {MAX_DB_NAME_LEN}-byte cap",
db_prefix.len()
)));
}
let client = Client::with_uri_str(uri).await?;
client
.database("admin")
.run_command(doc! { "ping": 1 })
.await?;
tracing::info!(
"MongoDB cluster reachable; per-tenant pool ready (db prefix '{db_prefix}')"
);
Ok(Self {
client,
db_prefix: db_prefix.to_string(),
ensured: Arc::new(DashMap::new()),
})
}
/// Return a [`Database`] scoped to this tenant. Ensures indexes on
/// first call per tenant (per process). Cheap on the hot path —
/// subsequent calls skip the round-trip.
pub async fn for_tenant(&self, ctx: &TenantContext) -> Result<Database, AgentError> {
self.for_tenant_id(&ctx.tenant_id).await
}
/// Like [`Self::for_tenant`] but accepts a bare tenant_id.
/// For background paths (scheduler, webhooks, pipeline orchestrators)
/// that don't have a full [`TenantContext`] but know which tenant
/// they're operating on (typically resolved from a URL path, a job
/// argument, or the registry).
pub async fn for_tenant_id(&self, tenant_id: &str) -> Result<Database, AgentError> {
let db_name = self.tenant_db_name(tenant_id);
let db = Database::from_database(self.client.database(&db_name));
// `DashMap::insert` returns the previous value; `None` means we
// were the first writer for this tenant_id and own the
// index-ensure work.
if self.ensured.insert(tenant_id.to_string(), ()).is_none() {
if let Err(e) = db.ensure_indexes().await {
// Roll the marker back so the next request retries.
self.ensured.remove(tenant_id);
return Err(e);
}
tracing::debug!(
tenant_id = %tenant_id,
db_name = %db_name,
"Indexes ensured for tenant database"
);
}
Ok(db)
}
/// Compute the Mongo database name for a tenant. Public for tests
/// and tenant offboarding (`pool.client().database(name).drop()`).
///
/// Format: `<prefix>_<sanitized_tenant_id>` if it fits the 63-byte
/// cap, else `<prefix>_<sha256-16-byte-hex-of-tenant_id>`. The
/// `db_prefix` length invariant established at [`Self::connect`]
/// guarantees the hash-fallback name always fits — no runtime
/// assertion needed.
///
/// Collision resistance: the hash fallback is a 16-byte SHA-256
/// truncation, which gives ~2^64 birthday-collision resistance. At
/// our 10s100s tenant scale the probability of two tenant_ids
/// colliding is effectively zero. (8-byte truncation would have
/// been ~2^32 — too close for comfort on a regulated product.)
pub fn tenant_db_name(&self, tenant_id: &str) -> String {
let sanitized = sanitize_tenant_id(tenant_id);
let natural = format!("{}_{}", self.db_prefix, sanitized);
if natural.len() <= MAX_DB_NAME_LEN {
natural
} else {
let mut hasher = Sha256::new();
hasher.update(tenant_id.as_bytes());
let digest = hasher.finalize();
let suffix = hex::encode(&digest[..HASH_HEX_LEN / 2]);
format!("{}_{}", self.db_prefix, suffix)
}
}
/// Raw client handle. Reserved for cross-tenant admin flows that
/// must opt in explicitly (tenant listing, drop-on-offboard).
pub fn client(&self) -> &Client {
&self.client
}
/// Cross-tenant admin database used by features that intentionally
/// span tenants (today: MCP bearer tokens — each token row carries
/// a `tenant_id` and the MCP server reads them to route requests).
///
/// The name `<db_prefix>__admin` (double underscore) is reserved —
/// the sanitizer never produces it for a normal tenant DB because
/// the natural format is `<db_prefix>_<sanitized_tenant_id>` (one
/// underscore) and tenant_ids would have to start with `_admin` to
/// collide. New tenant provisioning should reject such ids.
pub fn admin_db(&self) -> mongodb::Database {
self.client.database(&self.admin_db_name())
}
/// Name of the admin database — public so tests / operators can
/// drop it via the raw client.
pub fn admin_db_name(&self) -> String {
format!("{}__admin", self.db_prefix)
}
/// List every Mongo database currently belonging to this pool,
/// identified by the `<db_prefix>_` prefix. The result is the raw
/// database names — opening one for offboarding/cleanup goes
/// through [`Self::client`].
///
/// Note: hashed-fallback names (very long tenant_ids) lose the
/// original tenant_id at the cluster level — we know a database
/// exists for *some* tenant but not which one. In practice
/// tenant_ids are UUIDs (36 chars) and never hit the fallback,
/// so this is a theoretical concern, not an operational one.
pub async fn list_tenant_db_names(&self) -> Result<Vec<String>, AgentError> {
let prefix = format!("{}_", self.db_prefix);
let names = self.client.list_database_names().await?;
Ok(names
.into_iter()
.filter(|n| n.starts_with(&prefix))
.collect())
}
/// Tenant ids for every provisioned tenant database, derived by stripping
/// the `<prefix>_` from the database names. Skips the admin database
/// (`<prefix>__admin`). Hash-fallback names (very long tenant_ids) are lost
/// at the cluster level and cannot be recovered here — in practice tenant
/// ids are UUIDs and never hit that path. Used by the migration CLI's
/// `--all` mode.
pub async fn list_tenant_ids(&self) -> Result<Vec<String>, AgentError> {
let prefix = format!("{}_", self.db_prefix);
Ok(self
.list_tenant_db_names()
.await?
.into_iter()
.filter_map(|n| n.strip_prefix(&prefix).map(str::to_string))
.filter(|id| !id.starts_with('_'))
.collect())
}
/// Drop the database for a specific tenant. Used by GDPR delete
/// and tenant offboarding. Idempotent — dropping a non-existent
/// database is a no-op at the driver level.
///
/// Also evicts the tenant from the in-memory `ensured` set so a
/// later re-provision triggers fresh `ensure_indexes`.
pub async fn drop_tenant(&self, tenant_id: &str) -> Result<(), AgentError> {
let db_name = self.tenant_db_name(tenant_id);
self.client.database(&db_name).drop().await?;
self.ensured.remove(tenant_id);
tracing::info!(
tenant_id = %tenant_id,
db_name = %db_name,
"Dropped tenant database"
);
Ok(())
}
}
/// Mongo database names disallow `/`, `\`, `.`, `"`, `$`, ` `, and NUL.
/// breakpilot-dev tenant_ids are UUIDs so this is belt-and-braces, but
/// it lets the pool tolerate any future tenant_id shape without surprise.
fn sanitize_tenant_id(tenant_id: &str) -> String {
tenant_id
.chars()
.map(|c| match c {
'/' | '\\' | '.' | '"' | '$' | ' ' | '\0' => '_',
c => c,
})
.collect()
}
#[derive(Clone, Debug)]
pub struct Database {
inner: mongodb::Database,
@@ -20,17 +242,13 @@ impl Database {
Ok(Self { inner: db })
}
pub async fn ensure_indexes(&self) -> Result<(), AgentError> {
// repositories: unique git_url
self.repositories()
.create_index(
IndexModel::builder()
.keys(doc! { "git_url": 1 })
.options(IndexOptions::builder().unique(true).build())
.build(),
)
.await?;
/// Wrap an already-resolved Mongo database. Used by [`DatabasePool`]
/// to hand out tenant-scoped handles without a fresh client per tenant.
pub(crate) fn from_database(inner: mongodb::Database) -> Self {
Self { inner }
}
pub async fn ensure_indexes(&self) -> Result<(), AgentError> {
// findings: unique fingerprint
self.findings()
.create_index(
@@ -78,6 +296,25 @@ impl Database {
)
.await?;
// cve_notifications: unique cve_id + repo_id + package, status filter
self.cve_notifications()
.create_index(
IndexModel::builder()
.keys(
doc! { "cve_id": 1, "repo_id": 1, "package_name": 1, "package_version": 1 },
)
.options(IndexOptions::builder().unique(true).build())
.build(),
)
.await?;
self.cve_notifications()
.create_index(
IndexModel::builder()
.keys(doc! { "status": 1, "created_at": -1 })
.build(),
)
.await?;
// tracker_issues: unique finding_id
self.tracker_issues()
.create_index(
@@ -166,14 +403,72 @@ impl Database {
)
.await?;
// pentest_sessions: compound (target_id, started_at DESC)
self.pentest_sessions()
.create_index(
IndexModel::builder()
.keys(doc! { "target_id": 1, "started_at": -1 })
.build(),
)
.await?;
// pentest_sessions: status index
self.pentest_sessions()
.create_index(IndexModel::builder().keys(doc! { "status": 1 }).build())
.await?;
// attack_chain_nodes: compound (session_id, node_id)
self.attack_chain_nodes()
.create_index(
IndexModel::builder()
.keys(doc! { "session_id": 1, "node_id": 1 })
.build(),
)
.await?;
// pentest_messages: compound (session_id, created_at)
self.pentest_messages()
.create_index(
IndexModel::builder()
.keys(doc! { "session_id": 1, "created_at": 1 })
.build(),
)
.await?;
// onboarded_targets: multikey on artifact source ref (webhook + dedupe
// lookup). Non-unique — "one git URL per tenant" is enforced in the
// create handler, since a unique multikey index on an array field has
// null-collision caveats.
self.onboarded_targets()
.create_index(
IndexModel::builder()
.keys(doc! { "artifacts.source_ref": 1 })
.build(),
)
.await?;
// onboarded_targets: multikey on artifact kind
self.onboarded_targets()
.create_index(
IndexModel::builder()
.keys(doc! { "artifacts.kind": 1 })
.build(),
)
.await?;
// onboarded_targets: target_type filter
self.onboarded_targets()
.create_index(
IndexModel::builder()
.keys(doc! { "target_type": 1 })
.build(),
)
.await?;
tracing::info!("Database indexes ensured");
Ok(())
}
pub fn repositories(&self) -> Collection<TrackedRepository> {
self.inner.collection("repositories")
}
pub fn findings(&self) -> Collection<Finding> {
self.inner.collection("findings")
}
@@ -190,6 +485,12 @@ impl Database {
self.inner.collection("cve_alerts")
}
pub fn cve_notifications(
&self,
) -> Collection<compliance_core::models::notification::CveNotification> {
self.inner.collection("cve_notifications")
}
pub fn tracker_issues(&self) -> Collection<TrackerIssue> {
self.inner.collection("tracker_issues")
}
@@ -216,6 +517,20 @@ impl Database {
self.inner.collection("dast_targets")
}
/// The unified onboarding targets that replace `repositories` and
/// `dast_targets`. Ids are preserved from the legacy collections during
/// migration so downstream `repo_id` / `target_id` references keep resolving.
pub fn onboarded_targets(&self) -> Collection<OnboardedTarget> {
self.inner.collection("onboarded_targets")
}
/// A typed handle to an arbitrary collection by name. For bookkeeping
/// collections without a dedicated model (e.g. `schema_migrations`,
/// `onboarding_migration_log`).
pub fn collection_named<T: Send + Sync>(&self, name: &str) -> Collection<T> {
self.inner.collection(name)
}
pub fn dast_scan_runs(&self) -> Collection<DastScanRun> {
self.inner.collection("dast_scan_runs")
}
@@ -235,6 +550,19 @@ impl Database {
self.inner.collection("embedding_builds")
}
// Pentest collections
pub fn pentest_sessions(&self) -> Collection<PentestSession> {
self.inner.collection("pentest_sessions")
}
pub fn attack_chain_nodes(&self) -> Collection<AttackChainNode> {
self.inner.collection("attack_chain_nodes")
}
pub fn pentest_messages(&self) -> Collection<PentestMessage> {
self.inner.collection("pentest_messages")
}
#[allow(dead_code)]
pub fn raw_collection(&self, name: &str) -> Collection<mongodb::bson::Document> {
self.inner.collection(name)
+154
View File
@@ -0,0 +1,154 @@
//! Content-addressed blob storage and archive extraction for ingest.
//!
//! Blobs are stored at `<base>/blobs/<sha[0:2]>/<sha>` and deduplicated by
//! digest; per-run working directories live under `<base>/work/`.
use std::fs::{self, File};
use std::io::{self, Read};
use std::path::{Path, PathBuf};
use sha2::{Digest, Sha256};
use crate::error::AgentError;
/// Read buffer size for streaming hashes/copies (64 KiB).
const BUF_LEN: usize = 64 * 1024;
/// Stream-hash a file with SHA-256, returning the lowercase-hex digest and the
/// byte length. Streams so large firmware images never load fully into memory.
pub fn hash_file(path: &Path) -> Result<(String, u64), AgentError> {
let mut file = File::open(path)?;
let mut hasher = Sha256::new();
let mut buf = [0u8; BUF_LEN];
let mut total: u64 = 0;
loop {
let n = file.read(&mut buf)?;
if n == 0 {
break;
}
hasher.update(&buf[..n]);
total += n as u64;
}
Ok((hex::encode(hasher.finalize()), total))
}
/// Copy `src` into the content-addressed blob store under `base`, returning the
/// stored path. Idempotent: an already-present blob is not rewritten.
pub fn store_file(base: &Path, src: &Path, sha: &str) -> Result<PathBuf, AgentError> {
if sha.len() < 2 {
return Err(AgentError::Other(format!("invalid content hash '{sha}'")));
}
let dir = base.join("blobs").join(&sha[0..2]);
fs::create_dir_all(&dir)?;
let dest = dir.join(sha);
if !dest.exists() {
fs::copy(src, &dest)?;
}
Ok(dest)
}
/// Extract a zip archive into `dest` (created if needed). `enclosed_name`
/// sanitizes each entry path, so this is safe against zip-slip traversal.
pub fn extract_zip(archive: &Path, dest: &Path) -> Result<(), AgentError> {
let file = File::open(archive)?;
let mut zip =
zip::ZipArchive::new(file).map_err(|e| AgentError::Other(format!("open zip: {e}")))?;
fs::create_dir_all(dest)?;
for i in 0..zip.len() {
let mut entry = zip
.by_index(i)
.map_err(|e| AgentError::Other(format!("read zip entry: {e}")))?;
// `enclosed_name` returns `None` for traversal-unsafe paths — skip them.
let Some(rel) = entry.enclosed_name() else {
continue;
};
let out = dest.join(rel);
if entry.is_dir() {
fs::create_dir_all(&out)?;
} else {
if let Some(parent) = out.parent() {
fs::create_dir_all(parent)?;
}
let mut outfile = File::create(&out)?;
io::copy(&mut entry, &mut outfile)?;
}
}
Ok(())
}
/// The working directory for one artifact of a target: `<base>/work/<target>/<artifact>`.
pub fn work_dir(base: &Path, target_id: &str, artifact_id: &str) -> PathBuf {
base.join("work").join(target_id).join(artifact_id)
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
/// A unique scratch directory, removed on drop.
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-ingest-{}", uuid::Uuid::new_v4()));
fs::create_dir_all(&p).expect("mkdir scratch");
Self(p)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
#[test]
fn hash_is_stable_and_reports_size() {
let dir = Scratch::new();
let f = dir.path().join("a.bin");
fs::write(&f, b"hello world").expect("write");
let (sha, size) = hash_file(&f).expect("hash");
assert_eq!(size, 11);
// Known SHA-256 of "hello world".
assert_eq!(
sha,
"b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"
);
}
#[test]
fn store_is_content_addressed_and_idempotent() {
let base = Scratch::new();
let src = base.path().join("src.bin");
fs::write(&src, b"payload").expect("write");
let (sha, _) = hash_file(&src).expect("hash");
let p1 = store_file(base.path(), &src, &sha).expect("store");
let p2 = store_file(base.path(), &src, &sha).expect("store again");
assert_eq!(p1, p2);
assert!(p1.ends_with(&sha));
assert!(p1.starts_with(base.path().join("blobs").join(&sha[0..2])));
assert_eq!(fs::read(&p1).expect("read"), b"payload");
}
#[test]
fn extract_zip_writes_entries() {
let base = Scratch::new();
let archive = base.path().join("a.zip");
{
let file = File::create(&archive).expect("create");
let mut w = zip::ZipWriter::new(file);
let opts: zip::write::SimpleFileOptions = Default::default();
w.start_file("dir/hello.txt", opts).expect("start");
io::Write::write_all(&mut w, b"hi").expect("write");
w.finish().expect("finish");
}
let dest = base.path().join("out");
extract_zip(&archive, &dest).expect("extract");
assert_eq!(
fs::read_to_string(dest.join("dir/hello.txt")).expect("read"),
"hi"
);
}
}
+416
View File
@@ -0,0 +1,416 @@
//! Artifact ingest.
//!
//! Normalizes each [`Artifact`] on an [`OnboardedTarget`] into a local working
//! path plus recorded metadata (content hash, size, discovered facts) that the
//! classifier and scanners consume. Every blob is SHA-256 hashed — that digest
//! is also the reconciliation key against sibling products (a firmware sha256
//! matches tramiton's `Artifact.sha256`).
mod blob;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use compliance_core::models::{Artifact, ArtifactKind, DetectedFact, OnboardedTarget};
use compliance_core::AgentConfig;
use crate::error::AgentError;
use crate::pipeline::git::{GitOps, RepoCredentials};
/// The paths and identifiers an ingest needs. Decoupled from the full
/// [`AgentConfig`] so ingest is testable without a complete config.
pub struct IngestContext<'a> {
/// Base directory for content-addressed blobs and working dirs.
pub artifact_store_base: &'a Path,
/// Base directory for git clones.
pub git_clone_base: &'a str,
/// Default SSH key path (used when an artifact provides none).
pub ssh_key_path: &'a str,
/// The id of the target these artifacts belong to (namespaces working dirs).
pub target_id: &'a str,
}
impl<'a> IngestContext<'a> {
/// Build an ingest context from the agent config for a given target.
pub fn from_config(config: &'a AgentConfig, target_id: &'a str) -> Self {
Self {
artifact_store_base: Path::new(&config.artifact_store_base_path),
git_clone_base: &config.git_clone_base_path,
ssh_key_path: &config.ssh_key_path,
target_id,
}
}
}
/// The result of ingesting one artifact.
pub struct IngestedArtifact {
/// The artifact this corresponds to ([`Artifact::id`]).
pub artifact_id: String,
/// The artifact kind.
pub kind: ArtifactKind,
/// Local working path (clone dir, extracted dir, or blob file). `None` for
/// artifacts with no on-disk form (live URL, plaintext, container ref).
pub working_path: Option<PathBuf>,
/// SHA-256 of the content (blobs) or git head SHA (git repos).
pub content_hash: Option<String>,
/// Stored blob size in bytes, when applicable.
pub size_bytes: Option<u64>,
/// Facts discovered during ingest.
pub facts: Vec<DetectedFact>,
}
/// All ingested artifacts for a target, keyed by artifact id.
pub struct IngestSet {
/// The ingested artifacts, keyed by [`Artifact::id`].
pub by_artifact: HashMap<String, IngestedArtifact>,
}
impl IngestSet {
/// The working paths of every ingested artifact that has one — the input the
/// classifier expects.
pub fn working_paths(&self) -> HashMap<String, PathBuf> {
self.by_artifact
.iter()
.filter_map(|(id, a)| a.working_path.clone().map(|p| (id.clone(), p)))
.collect()
}
/// The ingest result for a specific artifact.
pub fn get(&self, artifact_id: &str) -> Option<&IngestedArtifact> {
self.by_artifact.get(artifact_id)
}
}
/// Ingest every artifact on a target.
pub fn ingest_all(
target: &OnboardedTarget,
ctx: &IngestContext<'_>,
) -> Result<IngestSet, AgentError> {
let mut by_artifact = HashMap::new();
for artifact in &target.artifacts {
let ingested = ingest_artifact(artifact, ctx)?;
by_artifact.insert(artifact.id.clone(), ingested);
}
Ok(IngestSet { by_artifact })
}
/// Ingest a single artifact, dispatching on its kind.
pub fn ingest_artifact(
artifact: &Artifact,
ctx: &IngestContext<'_>,
) -> Result<IngestedArtifact, AgentError> {
match artifact.kind {
ArtifactKind::GitRepo => ingest_git(artifact, ctx),
ArtifactKind::SourceArchive | ArtifactKind::MobilePackage | ArtifactKind::PlcProject => {
ingest_blob(artifact, ctx, true)
}
ArtifactKind::FirmwareImage => ingest_blob(artifact, ctx, false),
ArtifactKind::ContainerImage => Ok(metadata_only(
artifact,
DetectedFact::new("container_ref", artifact.source_ref.as_str(), "ingest"),
)),
ArtifactKind::LiveUrl => Ok(metadata_only(
artifact,
DetectedFact::new("live_url", artifact.source_ref.as_str(), "ingest"),
)),
ArtifactKind::PlaintextDescription => Ok(metadata_only(
artifact,
DetectedFact::new(
"description_len",
artifact.source_ref.len().to_string(),
"ingest",
),
)),
}
}
/// Clone (or fetch) a git artifact, recording the head SHA as the content hash.
fn ingest_git(
artifact: &Artifact,
ctx: &IngestContext<'_>,
) -> Result<IngestedArtifact, AgentError> {
let creds = credentials_for(artifact, ctx.ssh_key_path);
let git_ops = GitOps::new(ctx.git_clone_base, creds);
let repo_path = git_ops.clone_or_fetch(&artifact.source_ref, &artifact.id)?;
let head = GitOps::get_head_sha(&repo_path).ok();
Ok(IngestedArtifact {
artifact_id: artifact.id.clone(),
kind: artifact.kind,
working_path: Some(repo_path),
content_hash: head,
size_bytes: None,
facts: Vec::new(),
})
}
/// Store a blob artifact content-addressed. When `extract` is set and the blob
/// is a zip container (source archive, APK/AAB/IPA), also unpack it into a
/// working directory; otherwise the working path is the stored blob.
fn ingest_blob(
artifact: &Artifact,
ctx: &IngestContext<'_>,
extract: bool,
) -> Result<IngestedArtifact, AgentError> {
let base = ctx.artifact_store_base;
let src = local_source(artifact)?;
let (sha, size) = blob::hash_file(&src)?;
let stored = blob::store_file(base, &src, &sha)?;
let mut facts = Vec::new();
let working_path = if extract {
let dest = blob::work_dir(base, ctx.target_id, &artifact.id);
match blob::extract_zip(&stored, &dest) {
Ok(()) => dest,
Err(e) => {
// Not a zip container — this is a single uploaded file (e.g. a
// `.st`/`.xml` PLC project or a `.tar.gz`). The content-addressed
// blob has no extension, so materialize it into a working dir
// under its original name; extension-based scanners (PLC) can then
// discover it and report a readable path.
facts.push(DetectedFact::new(
"archive_unextracted",
e.to_string(),
"ingest",
));
match materialize_single(&stored, &dest, &blob_file_name(artifact)) {
Ok(dir) => dir,
Err(copy_err) => {
facts.push(DetectedFact::new(
"materialize_failed",
copy_err.to_string(),
"ingest",
));
stored.clone()
}
}
}
}
} else {
stored.clone()
};
Ok(IngestedArtifact {
artifact_id: artifact.id.clone(),
kind: artifact.kind,
working_path: Some(working_path),
content_hash: Some(sha),
size_bytes: Some(size),
facts,
})
}
/// Copy a stored blob into `dest`/`name`, returning `dest`. Used when an
/// "extractable" artifact turns out to be a single file rather than an archive.
fn materialize_single(stored: &Path, dest: &Path, name: &str) -> Result<PathBuf, AgentError> {
std::fs::create_dir_all(dest)?;
std::fs::copy(stored, dest.join(name))?;
Ok(dest.to_path_buf())
}
/// A safe, single-segment file name for an artifact, preserving the original
/// extension so scanners can identify it. Derives from `source_ref` (the
/// uploaded/original file name); `file_name` strips any directory components,
/// so this is traversal-safe. Falls back to the artifact id.
fn blob_file_name(artifact: &Artifact) -> String {
Path::new(&artifact.source_ref)
.file_name()
.and_then(|n| n.to_str())
.map(str::to_string)
.filter(|s| !s.is_empty())
.unwrap_or_else(|| format!("artifact-{}", artifact.id))
}
/// An artifact with no on-disk form: record a single fact, no hash/path.
fn metadata_only(artifact: &Artifact, fact: DetectedFact) -> IngestedArtifact {
IngestedArtifact {
artifact_id: artifact.id.clone(),
kind: artifact.kind,
working_path: None,
content_hash: None,
size_bytes: None,
facts: vec![fact],
}
}
/// The local file backing a blob artifact: its `stored_path` if already
/// uploaded, else its `source_ref` interpreted as a filesystem path.
fn local_source(artifact: &Artifact) -> Result<PathBuf, AgentError> {
let path = artifact
.stored_path
.as_deref()
.unwrap_or(artifact.source_ref.as_str());
let path = PathBuf::from(path);
if !path.exists() {
return Err(AgentError::Other(format!(
"artifact {} source not found at {}",
artifact.id,
path.display()
)));
}
Ok(path)
}
/// Build git credentials from an artifact's auth plus a default SSH key path.
fn credentials_for(artifact: &Artifact, default_ssh_key_path: &str) -> RepoCredentials {
let auth = artifact.auth.as_ref();
RepoCredentials {
ssh_key_path: auth
.and_then(|a| a.ssh_key_path.clone())
.or_else(|| Some(default_ssh_key_path.to_string())),
auth_token: auth.and_then(|a| a.secret.clone()),
auth_username: auth.and_then(|a| a.username.clone()),
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{ArtifactAuth, TargetType};
/// A unique scratch directory, removed on drop.
struct Scratch(PathBuf);
impl Scratch {
fn new() -> Self {
let p = std::env::temp_dir().join(format!("cs-ingest-mod-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&p).expect("mkdir scratch");
Self(p)
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
fn ctx_for<'a>(store: &'a Path, target_id: &'a str) -> IngestContext<'a> {
IngestContext {
artifact_store_base: store,
git_clone_base: "/tmp/cs-ingest-test-repos",
ssh_key_path: "/tmp/cs-ingest-test-ssh",
target_id,
}
}
#[test]
fn firmware_blob_is_hashed_and_stored() {
let scratch = Scratch::new();
let store = scratch.0.join("store");
let fw = scratch.0.join("fw.bin");
std::fs::write(&fw, b"firmware-bytes").expect("write");
let ctx = ctx_for(&store, "t1");
let artifact = Artifact::firmware_image(fw.to_string_lossy().to_string());
let out = ingest_artifact(&artifact, &ctx).expect("ingest");
assert_eq!(out.kind, ArtifactKind::FirmwareImage);
assert_eq!(out.size_bytes, Some(14));
let sha = out.content_hash.expect("hash");
assert_eq!(sha.len(), 64);
// working path is the content-addressed blob
let wp = out.working_path.expect("working path");
assert!(wp.starts_with(store.join("blobs")));
}
#[test]
fn live_url_has_no_blob() {
let scratch = Scratch::new();
let store = scratch.0.join("store");
let ctx = ctx_for(&store, "t1");
let artifact = Artifact::live_url("https://example.com");
let out = ingest_artifact(&artifact, &ctx).expect("ingest");
assert!(out.working_path.is_none());
assert!(out.content_hash.is_none());
assert!(out.facts.iter().any(|f| f.key == "live_url"));
}
#[test]
fn ingest_all_collects_working_paths() {
let scratch = Scratch::new();
let store = scratch.0.join("store");
let fw = scratch.0.join("fw.bin");
std::fs::write(&fw, b"abc").expect("write");
let ctx = ctx_for(&store, "t1");
let mut target = OnboardedTarget::new("t".to_string(), TargetType::FirmwareBareMetal);
target
.artifacts
.push(Artifact::firmware_image(fw.to_string_lossy().to_string()));
target.artifacts.push(Artifact::live_url("https://x"));
let set = ingest_all(&target, &ctx).expect("ingest all");
assert_eq!(set.by_artifact.len(), 2);
// Only the firmware artifact yields a working path.
assert_eq!(set.working_paths().len(), 1);
}
#[test]
fn credentials_prefer_artifact_auth() {
let mut artifact = Artifact::git_repo("https://git/x", "main");
artifact.auth = Some(ArtifactAuth {
method: "token".to_string(),
username: Some("bob".to_string()),
secret: Some("pat".to_string()),
..Default::default()
});
let creds = credentials_for(&artifact, "/default/ssh/key");
assert_eq!(creds.auth_token.as_deref(), Some("pat"));
assert_eq!(creds.auth_username.as_deref(), Some("bob"));
}
#[test]
fn credentials_fall_back_to_default_ssh_key() {
let artifact = Artifact::git_repo("git@host:x.git", "main");
let creds = credentials_for(&artifact, "/default/ssh/key");
assert_eq!(creds.ssh_key_path.as_deref(), Some("/default/ssh/key"));
assert!(creds.auth_token.is_none());
}
/// A single uploaded PLC file (not an archive) must land in a working dir
/// under its original name so the PLC scanner can discover it by extension
/// and report a readable path — the demo's upload → scan path.
#[test]
fn single_uploaded_plc_file_is_materialized_and_scannable() {
use compliance_core::models::PlcFormat;
let scratch = Scratch::new();
let store = scratch.0.join("store");
// Simulate the upload handler: bytes written to an `uploads/` path,
// `source_ref` carrying the original (clean) file name.
let uploads = scratch.0.join("uploads");
std::fs::create_dir_all(&uploads).expect("mkdir uploads");
let uploaded = uploads.join("a1b2c3_pump_station.st");
std::fs::write(
&uploaded,
"PROGRAM P\nVAR\n ApiKey : STRING := 'sk-live-1234';\nEND_VAR\nEND_PROGRAM\n",
)
.expect("write st");
let mut artifact = Artifact::plc_project("pump_station.st", PlcFormat::StructuredText);
artifact.stored_path = Some(uploaded.to_string_lossy().to_string());
let ctx = ctx_for(&store, "t-plc");
let out = ingest_artifact(&artifact, &ctx).expect("ingest");
// Working path is a directory (not the extensionless blob) holding the
// file under its original name.
let wp = out.working_path.expect("working path");
assert!(wp.is_dir(), "expected a working dir, got {wp:?}");
assert!(wp.join("pump_station.st").is_file());
// The PLC scanner finds the hardcoded credential and reports a clean path.
let findings = crate::pipeline::plc::analyze_tree(&wp, "t-plc");
assert!(
!findings.is_empty(),
"scanner should flag the uploaded file"
);
assert!(findings
.iter()
.any(|f| f.rule_id.as_deref() == Some("plc-hardcoded-credential")));
assert_eq!(
findings[0].file_path.as_deref(),
Some("pump_station.st"),
"finding should reference the original file name"
);
}
}
+18
View File
@@ -0,0 +1,18 @@
// Library entrypoint — re-exports for integration tests and the binary.
pub mod agent;
pub mod api;
pub mod classify;
pub mod config;
pub mod database;
pub mod error;
pub mod ingest;
pub mod llm;
pub mod pentest;
pub mod pipeline;
pub mod rag;
pub mod scheduler;
pub mod ssh;
#[allow(dead_code)]
pub mod trackers;
pub mod webhooks;
+139 -165
View File
@@ -1,66 +1,15 @@
use secrecy::{ExposeSecret, SecretString};
use serde::{Deserialize, Serialize};
use super::types::*;
use crate::error::AgentError;
#[derive(Clone)]
pub struct LlmClient {
base_url: String,
api_key: SecretString,
model: String,
embed_model: String,
http: reqwest::Client,
}
#[derive(Serialize)]
struct ChatMessage {
role: String,
content: String,
}
#[derive(Serialize)]
struct ChatCompletionRequest {
model: String,
messages: Vec<ChatMessage>,
#[serde(skip_serializing_if = "Option::is_none")]
temperature: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
max_tokens: Option<u32>,
}
#[derive(Deserialize)]
struct ChatCompletionResponse {
choices: Vec<ChatChoice>,
}
#[derive(Deserialize)]
struct ChatChoice {
message: ChatResponseMessage,
}
#[derive(Deserialize)]
struct ChatResponseMessage {
content: String,
}
/// Request body for the embeddings API
#[derive(Serialize)]
struct EmbeddingRequest {
model: String,
input: Vec<String>,
}
/// Response from the embeddings API
#[derive(Deserialize)]
struct EmbeddingResponse {
data: Vec<EmbeddingData>,
}
/// A single embedding result
#[derive(Deserialize)]
struct EmbeddingData {
embedding: Vec<f64>,
index: usize,
pub(crate) base_url: String,
pub(crate) api_key: SecretString,
pub(crate) model: String,
pub(crate) embed_model: String,
pub(crate) http: reqwest::Client,
}
impl LlmClient {
@@ -70,111 +19,156 @@ impl LlmClient {
model: String,
embed_model: String,
) -> Self {
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(300))
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_default();
Self {
base_url,
api_key,
model,
embed_model,
http: reqwest::Client::new(),
http,
}
}
pub fn embed_model(&self) -> &str {
&self.embed_model
pub(crate) fn chat_url(&self) -> String {
format!(
"{}/v1/chat/completions",
self.base_url.trim_end_matches('/')
)
}
pub(crate) fn auth_header(&self) -> Option<String> {
let key = self.api_key.expose_secret();
if key.is_empty() {
None
} else {
Some(format!("Bearer {key}"))
}
}
/// Simple chat: system + user prompt → text response
pub async fn chat(
&self,
system_prompt: &str,
user_prompt: &str,
temperature: Option<f64>,
) -> Result<String, AgentError> {
let url = format!(
"{}/v1/chat/completions",
self.base_url.trim_end_matches('/')
);
let messages = vec![
ChatMessage {
role: "system".to_string(),
content: Some(system_prompt.to_string()),
tool_calls: None,
tool_call_id: None,
},
ChatMessage {
role: "user".to_string(),
content: Some(user_prompt.to_string()),
tool_calls: None,
tool_call_id: None,
},
];
let request_body = ChatCompletionRequest {
model: self.model.clone(),
messages: vec![
ChatMessage {
role: "system".to_string(),
content: system_prompt.to_string(),
},
ChatMessage {
role: "user".to_string(),
content: user_prompt.to_string(),
},
],
messages,
temperature,
max_tokens: Some(4096),
tools: None,
};
let mut req = self
.http
.post(&url)
.header("content-type", "application/json")
.json(&request_body);
let key = self.api_key.expose_secret();
if !key.is_empty() {
req = req.header("Authorization", format!("Bearer {key}"));
}
let resp = req
.send()
.await
.map_err(|e| AgentError::Other(format!("LiteLLM request failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(AgentError::Other(format!(
"LiteLLM returned {status}: {body}"
)));
}
let body: ChatCompletionResponse = resp
.json()
.await
.map_err(|e| AgentError::Other(format!("Failed to parse LiteLLM response: {e}")))?;
body.choices
.first()
.map(|c| c.message.content.clone())
.ok_or_else(|| AgentError::Other("Empty response from LiteLLM".to_string()))
self.send_chat_request(&request_body).await.map(|resp| {
match resp {
LlmResponse::Content(c) => c,
LlmResponse::ToolCalls { .. } => String::new(), // shouldn't happen without tools
}
})
}
/// Chat with a list of (role, content) messages → text response
#[allow(dead_code)]
pub async fn chat_with_messages(
&self,
messages: Vec<(String, String)>,
temperature: Option<f64>,
) -> Result<String, AgentError> {
let url = format!(
"{}/v1/chat/completions",
self.base_url.trim_end_matches('/')
);
let messages = messages
.into_iter()
.map(|(role, content)| ChatMessage {
role,
content: Some(content),
tool_calls: None,
tool_call_id: None,
})
.collect();
let request_body = ChatCompletionRequest {
model: self.model.clone(),
messages: messages
.into_iter()
.map(|(role, content)| ChatMessage { role, content })
.collect(),
messages,
temperature,
max_tokens: Some(4096),
tools: None,
};
self.send_chat_request(&request_body)
.await
.map(|resp| match resp {
LlmResponse::Content(c) => c,
LlmResponse::ToolCalls { .. } => String::new(),
})
}
/// Chat with tool definitions — returns either content or tool calls.
/// Use this for the AI pentest orchestrator loop.
pub async fn chat_with_tools(
&self,
messages: Vec<ChatMessage>,
tools: &[ToolDefinition],
temperature: Option<f64>,
max_tokens: Option<u32>,
) -> Result<LlmResponse, AgentError> {
let tool_payloads: Vec<ToolDefinitionPayload> = tools
.iter()
.map(|t| ToolDefinitionPayload {
r#type: "function".to_string(),
function: ToolFunctionPayload {
name: t.name.clone(),
description: t.description.clone(),
parameters: t.parameters.clone(),
},
})
.collect();
let request_body = ChatCompletionRequest {
model: self.model.clone(),
messages,
temperature,
max_tokens: Some(max_tokens.unwrap_or(8192)),
tools: if tool_payloads.is_empty() {
None
} else {
Some(tool_payloads)
},
};
self.send_chat_request(&request_body).await
}
/// Internal method to send a chat completion request and parse the response
async fn send_chat_request(
&self,
request_body: &ChatCompletionRequest,
) -> Result<LlmResponse, AgentError> {
let mut req = self
.http
.post(&url)
.post(self.chat_url())
.header("content-type", "application/json")
.json(&request_body);
.json(request_body);
let key = self.api_key.expose_secret();
if !key.is_empty() {
req = req.header("Authorization", format!("Bearer {key}"));
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
let resp = req
@@ -195,54 +189,34 @@ impl LlmClient {
.await
.map_err(|e| AgentError::Other(format!("Failed to parse LiteLLM response: {e}")))?;
body.choices
let choice = body
.choices
.first()
.map(|c| c.message.content.clone())
.ok_or_else(|| AgentError::Other("Empty response from LiteLLM".to_string()))
}
.ok_or_else(|| AgentError::Other("Empty response from LiteLLM".to_string()))?;
/// Generate embeddings for a batch of texts
pub async fn embed(&self, texts: Vec<String>) -> Result<Vec<Vec<f64>>, AgentError> {
let url = format!("{}/v1/embeddings", self.base_url.trim_end_matches('/'));
let request_body = EmbeddingRequest {
model: self.embed_model.clone(),
input: texts,
};
let mut req = self
.http
.post(&url)
.header("content-type", "application/json")
.json(&request_body);
let key = self.api_key.expose_secret();
if !key.is_empty() {
req = req.header("Authorization", format!("Bearer {key}"));
// Check for tool calls first
if let Some(tool_calls) = &choice.message.tool_calls {
if !tool_calls.is_empty() {
let calls: Vec<LlmToolCall> = tool_calls
.iter()
.map(|tc| {
let arguments = serde_json::from_str(&tc.function.arguments)
.unwrap_or(serde_json::Value::Object(serde_json::Map::new()));
LlmToolCall {
id: tc.id.clone(),
name: tc.function.name.clone(),
arguments,
}
})
.collect();
// Capture any reasoning text the LLM included alongside tool calls
let reasoning = choice.message.content.clone().unwrap_or_default();
return Ok(LlmResponse::ToolCalls { calls, reasoning });
}
}
let resp = req
.send()
.await
.map_err(|e| AgentError::Other(format!("Embedding request failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(AgentError::Other(format!(
"Embedding API returned {status}: {body}"
)));
}
let body: EmbeddingResponse = resp
.json()
.await
.map_err(|e| AgentError::Other(format!("Failed to parse embedding response: {e}")))?;
// Sort by index to maintain input order
let mut data = body.data;
data.sort_by_key(|d| d.index);
Ok(data.into_iter().map(|d| d.embedding).collect())
// Otherwise return content
let content = choice.message.content.clone().unwrap_or_default();
Ok(LlmResponse::Content(content))
}
}
+12 -7
View File
@@ -5,15 +5,20 @@ use compliance_core::models::Finding;
use crate::error::AgentError;
use crate::llm::LlmClient;
const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing issue descriptions for a bug tracker. Generate a clear, actionable issue body in Markdown format that includes:
const DESCRIPTION_SYSTEM_PROMPT: &str = r#"You are a security engineer writing a bug tracker issue for a developer to fix. Be direct and actionable — developers skim issue descriptions, so lead with what matters.
1. **Summary**: 1-2 sentence overview
2. **Evidence**: Code location, snippet, and what was detected
3. **Impact**: What could happen if not fixed
4. **Remediation**: Step-by-step fix instructions
5. **References**: Relevant CWE/CVE links if applicable
Format in Markdown:
Keep it concise and professional. Use code blocks for code snippets."#;
1. **What**: 1 sentence — what's wrong and where (file:line)
2. **Why it matters**: 1-2 sentences — concrete impact if not fixed. Avoid generic "could lead to" phrasing; describe the specific attack or failure scenario.
3. **Fix**: The specific code change needed. Use a code block with the corrected code if possible. If the fix is configuration-based, show the exact config change.
4. **References**: CWE/CVE link if applicable (one line, not a section)
Rules:
- No filler paragraphs or background explanations
- No restating the finding title in the body
- Code blocks should show the FIX, not the vulnerable code (the developer can see that in the diff)
- If the remediation is a one-liner, just say it — don't wrap it in a section header"#;
pub async fn generate_issue_description(
llm: &Arc<LlmClient>,
+74
View File
@@ -0,0 +1,74 @@
use serde::{Deserialize, Serialize};
use super::client::LlmClient;
use crate::error::AgentError;
// ── Embedding types ────────────────────────────────────────────
#[derive(Serialize)]
struct EmbeddingRequest {
model: String,
input: Vec<String>,
}
#[derive(Deserialize)]
struct EmbeddingResponse {
data: Vec<EmbeddingData>,
}
#[derive(Deserialize)]
struct EmbeddingData {
embedding: Vec<f64>,
index: usize,
}
// ── Embedding implementation ───────────────────────────────────
impl LlmClient {
pub fn embed_model(&self) -> &str {
&self.embed_model
}
/// Generate embeddings for a batch of texts
pub async fn embed(&self, texts: Vec<String>) -> Result<Vec<Vec<f64>>, AgentError> {
let url = format!("{}/v1/embeddings", self.base_url.trim_end_matches('/'));
let request_body = EmbeddingRequest {
model: self.embed_model.clone(),
input: texts,
};
let mut req = self
.http
.post(&url)
.header("content-type", "application/json")
.json(&request_body);
if let Some(auth) = self.auth_header() {
req = req.header("Authorization", auth);
}
let resp = req
.send()
.await
.map_err(|e| AgentError::Other(format!("Embedding request failed: {e}")))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(AgentError::Other(format!(
"Embedding API returned {status}: {body}"
)));
}
let body: EmbeddingResponse = resp
.json()
.await
.map_err(|e| AgentError::Other(format!("Failed to parse embedding response: {e}")))?;
let mut data = body.data;
data.sort_by_key(|d| d.index);
Ok(data.into_iter().map(|d| d.embedding).collect())
}
}
+18 -1
View File
@@ -5,7 +5,24 @@ use compliance_core::models::Finding;
use crate::error::AgentError;
use crate::llm::LlmClient;
const FIX_SYSTEM_PROMPT: &str = r#"You are a security engineer. Given a security finding with code context, suggest a concrete code fix. Return ONLY the fixed code snippet that can directly replace the vulnerable code. Include brief inline comments explaining the fix."#;
const FIX_SYSTEM_PROMPT: &str = r#"You are a security engineer suggesting a code fix. Return ONLY the corrected code that replaces the vulnerable snippet — no explanations, no markdown fences, no before/after comparison.
Rules:
- The fix must be a drop-in replacement for the vulnerable code
- Preserve the original code's style, indentation, and naming conventions
- Add at most one brief inline comment on the changed line explaining the security fix
- If the fix requires importing a new module, include the import on a separate line prefixed with the language's comment syntax + "Add import: "
- Do not refactor, rename variables, or "improve" unrelated code
- If the vulnerability is a false positive and the code is actually safe, return the original code unchanged with a comment explaining why no fix is needed
Language-specific fix guidance:
- Rust: use `?` for error propagation, prefer `SecretString` for secrets, use parameterized queries with `sqlx`/`diesel`
- Python: use parameterized queries (never f-strings in SQL), use `secrets` module not `random`, use `subprocess.run([...])` list form, use `markupsafe.escape()` for HTML
- Go: use `sql.Query` with `$1`/`?` placeholders, use `crypto/rand` not `math/rand`, use `html/template` not `text/template`, return errors don't panic
- Java/Kotlin: use `PreparedStatement` with `?` params, use `SecureRandom`, use `Jsoup.clean()` for HTML sanitization, use `@Valid` for input validation
- Ruby: use ActiveRecord parameterized finders, use `SecureRandom`, use `ERB::Util.html_escape`, use `strong_parameters`
- PHP: use PDO prepared statements with `:param` or `?`, use `random_bytes()`/`random_int()`, use `htmlspecialchars()` with `ENT_QUOTES`, use `password_hash(PASSWORD_BCRYPT)`
- C/C++: use `snprintf` not `sprintf`, use bounds-checked APIs, free resources in reverse allocation order, use `memset_s` for secret cleanup"#;
pub async fn suggest_fix(llm: &Arc<LlmClient>, finding: &Finding) -> Result<String, AgentError> {
let user_prompt = format!(
+5
View File
@@ -1,11 +1,16 @@
pub mod client;
#[allow(dead_code)]
pub mod descriptions;
pub mod embedding;
#[allow(dead_code)]
pub mod fixes;
#[allow(dead_code)]
pub mod pr_review;
pub mod review_prompts;
pub mod triage;
pub mod types;
pub use client::LlmClient;
pub use types::{
ChatMessage, LlmResponse, ToolCallRequest, ToolCallRequestFunction, ToolDefinition,
};
+108 -39
View File
@@ -1,69 +1,138 @@
// System prompts for multi-pass LLM code review.
// Each pass focuses on a different aspect to avoid overloading a single prompt.
pub const LOGIC_REVIEW_PROMPT: &str = r#"You are a senior software engineer reviewing code changes. Focus ONLY on logic and correctness issues.
pub const LOGIC_REVIEW_PROMPT: &str = r#"You are a senior software engineer reviewing a code diff. Report ONLY genuine logic bugs that would cause incorrect behavior at runtime.
Look for:
- Off-by-one errors, wrong comparisons, missing edge cases
- Incorrect control flow (unreachable code, missing returns, wrong loop conditions)
- Race conditions or concurrency bugs
- Resource leaks (unclosed handles, missing cleanup)
- Wrong variable used (copy-paste errors)
- Incorrect error handling (swallowed errors, wrong error type)
Report:
- Off-by-one errors, wrong comparisons, missing edge cases that cause wrong results
- Incorrect control flow that produces wrong output (not style preferences)
- Actual race conditions with concrete shared-state mutation (not theoretical ones)
- Resource leaks where cleanup is truly missing (not just "could be improved")
- Wrong variable used (copy-paste errors) — must be provably wrong, not just suspicious
- Swallowed errors that silently hide failures in a way that matters
Ignore: style, naming, formatting, documentation, minor improvements.
Do NOT report:
- Style, naming, formatting, documentation, or code organization preferences
- Theoretical issues without a concrete triggering scenario
- "Potential" problems that require assumptions not supported by the visible code
- Complexity or function length — that's a separate review pass
For each issue found, respond with a JSON array:
Language-idiomatic patterns that are NOT bugs (do not flag these):
- Rust: `||`/`&&` short-circuit evaluation, variable shadowing, `let` rebinding, `clone()`, `impl` blocks, `match` arms with guards, `?` operator chaining, `unsafe` blocks with safety comments
- Python: duck typing, EAFP pattern (try/except vs check-first), `*args`/`**kwargs`, walrus operator `:=`, truthiness checks on containers, bare `except:` in top-level handlers
- Go: multiple return values for errors, `if err != nil` patterns, goroutine + channel patterns, blank identifier `_`, named returns, `defer` for cleanup, `init()` functions
- Java/Kotlin: checked exception patterns, method overloading, `Optional` vs null checks, Kotlin `?.` safe calls, `!!` non-null assertions in tests, `when` exhaustive matching, companion objects, `lateinit`
- Ruby: monkey patching in libraries, method_missing, blocks/procs/lambdas, `rescue => e` patterns, `send`/`respond_to?` metaprogramming, `nil` checks via `&.` safe navigation
- PHP: loose comparisons with `==` (only flag if `===` was clearly intended), `@` error suppression in legacy code, `isset()`/`empty()` patterns, magic methods (`__get`, `__call`), array functions as callbacks
- C/C++: RAII patterns, move semantics, `const_cast`/`static_cast` in appropriate contexts, macro usage for platform compat, pointer arithmetic in low-level code, `goto` for cleanup in C
Severity guide:
- high: Will cause incorrect behavior in normal usage
- medium: Will cause incorrect behavior in edge cases
- low: Minor correctness concern with limited blast radius
Prefer returning [] over reporting low-confidence guesses. A false positive wastes more developer time than a missed low-severity issue.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "high|medium|low", "file": "...", "line": N, "suggestion": "..."}]
If no issues found, respond with: []"#;
pub const SECURITY_REVIEW_PROMPT: &str = r#"You are a security engineer reviewing code changes. Focus ONLY on security vulnerabilities.
pub const SECURITY_REVIEW_PROMPT: &str = r#"You are a security engineer reviewing a code diff. Report ONLY exploitable security vulnerabilities with a realistic attack scenario.
Look for:
- Injection vulnerabilities (SQL, command, XSS, template injection)
- Authentication/authorization bypasses
- Sensitive data exposure (logging secrets, hardcoded credentials)
- Insecure cryptography (weak algorithms, predictable randomness)
- Path traversal, SSRF, open redirects
- Unsafe deserialization
- Missing input validation at trust boundaries
Report:
- Injection vulnerabilities (SQL, command, XSS, template) where untrusted input reaches a sink
- Authentication/authorization bypasses with a concrete exploit path
- Sensitive data exposure: secrets in code, credentials in logs, PII leaks
- Insecure cryptography: weak algorithms, predictable randomness, hardcoded keys
- Path traversal, SSRF, open redirects — only where user input reaches the vulnerable API
- Unsafe deserialization of untrusted data
- Missing input validation at EXTERNAL trust boundaries (user input, API responses)
Ignore: code style, performance, general quality.
Do NOT report:
- Internal code that only handles trusted/validated data
- Hash functions used for non-security purposes (dedup fingerprints, cache keys, content addressing)
- Logging of non-sensitive operational data (finding titles, counts, performance metrics)
- "Information disclosure" for data that is already public or user-facing
- Code style, performance, or general quality issues
- Missing validation on internal function parameters (trust the caller within the same module/crate/package)
- Theoretical attacks that require preconditions not present in the code
For each issue found, respond with a JSON array:
Language-specific patterns that are NOT vulnerabilities (do not flag these):
- Python: `pickle` used on trusted internal data, `eval()`/`exec()` on hardcoded strings, `subprocess` with hardcoded commands, Django `mark_safe()` on static content, `assert` in non-security contexts
- Go: `crypto/rand` is secure (don't confuse with `math/rand`), `sql.DB` with parameterized queries is safe, `http.ListenAndServe` without TLS in dev/internal, error strings in responses (Go convention)
- Java/Kotlin: Spring Security annotations are sufficient auth checks, `@Transactional` provides atomicity, JPA parameterized queries are safe, Kotlin `require()`/`check()` are assertion patterns not vulnerabilities
- Ruby: Rails `params.permit()` is input validation, `render html:` with `html_safe` on generated content, ActiveRecord parameterized finders are safe, Devise/Warden patterns for auth
- PHP: PDO prepared statements are safe, Laravel Eloquent is parameterized, `htmlspecialchars()` is XSS mitigation, Symfony security voters are auth checks, `password_hash()`/`password_verify()` are correct bcrypt usage
- C/C++: `strncpy`/`snprintf` are bounds-checked (vs `strcpy`/`sprintf`), smart pointers manage memory, RAII handles cleanup, `static_assert` is compile-time only, OpenSSL with proper context setup
- Rust: `sha2`/`blake3` for fingerprinting is not "weak crypto", `unsafe` with documented invariants, `secrecy::SecretString` properly handles secrets
Severity guide:
- critical: Remote code execution, auth bypass, or data breach with no preconditions
- high: Exploitable vulnerability requiring minimal preconditions
- medium: Vulnerability requiring specific conditions or limited impact
Prefer returning [] over reporting speculative vulnerabilities. Every false positive erodes trust in the scanner.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "critical|high|medium", "file": "...", "line": N, "cwe": "CWE-XXX", "suggestion": "..."}]
If no issues found, respond with: []"#;
pub const CONVENTION_REVIEW_PROMPT: &str = r#"You are a code reviewer checking adherence to project conventions. Focus ONLY on patterns that indicate likely bugs or maintenance problems.
pub const CONVENTION_REVIEW_PROMPT: &str = r#"You are a code reviewer checking for convention violations that indicate likely bugs. Report ONLY deviations from the project's visible patterns that could cause real problems.
Look for:
- Inconsistent error handling patterns within the same module
- Public API that doesn't follow the project's established patterns
- Missing or incorrect type annotations that could cause runtime issues
- Anti-patterns specific to the language (e.g. unwrap in Rust library code, any in TypeScript)
Report:
- Inconsistent error handling within the same module where the inconsistency could hide failures
- Public API that breaks the module's established contract (not just different style)
- Anti-patterns that are bugs in this language: e.g. `unwrap()` in Rust library code where the CI enforces `clippy::unwrap_used`, `any` defeating TypeScript's type system
Do NOT report: minor style preferences, documentation gaps, formatting.
Only report issues with HIGH confidence that they deviate from the visible codebase conventions.
Do NOT report:
- Style preferences, formatting, naming conventions, or documentation
- Code organization suggestions ("this function should be split")
- Patterns that are valid in the language even if you'd write them differently
- "Missing type annotations" unless the code literally won't compile or causes a type inference bug
For each issue found, respond with a JSON array:
Language-specific patterns that are conventional (do not flag these):
- Rust: variable shadowing, `||`/`&&` short-circuit, `let` rebinding, builder patterns, `clone()`, `From`/`Into` impl chains, `#[allow(...)]` attributes
- Python: `**kwargs` forwarding, `@property` setters, `__dunder__` methods, list comprehensions with conditions, `if TYPE_CHECKING` imports, `noqa` comments
- Go: stuttering names (`http.HTTPClient`) discouraged but not a bug, `context.Context` as first param, init() functions, `//nolint` directives, returning concrete types vs interfaces in internal code
- Java/Kotlin: builder pattern boilerplate, Lombok annotations (`@Data`, `@Builder`), Kotlin data classes, `companion object` factories, `@Suppress` annotations, checked exception wrapping
- Ruby: `attr_accessor` usage, `Enumerable` mixin patterns, `module_function`, `class << self` syntax, DSL blocks (Rake, RSpec, Sinatra routes)
- PHP: `__construct` with property promotion, Laravel facades, static factory methods, nullable types with `?`, attribute syntax `#[...]`
- C/C++: header guards vs `#pragma once`, forward declarations, `const` correctness patterns, template specialization, `auto` type deduction
Severity guide:
- medium: Convention violation that will likely cause a bug or maintenance problem
- low: Convention violation that is a minor concern
Return at most 3 findings. Prefer [] over marginal findings.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "medium|low", "file": "...", "line": N, "suggestion": "..."}]
If no issues found, respond with: []"#;
pub const COMPLEXITY_REVIEW_PROMPT: &str = r#"You are reviewing code changes for excessive complexity that could lead to bugs.
pub const COMPLEXITY_REVIEW_PROMPT: &str = r#"You are reviewing code changes for complexity that is likely to cause bugs. Report ONLY complexity that makes the code demonstrably harder to reason about.
Look for:
- Functions over 50 lines that should be decomposed
- Deeply nested control flow (4+ levels)
- Complex boolean expressions that are hard to reason about
- Functions with 5+ parameters
- Code duplication within the changed files
Report:
- Functions over 80 lines with multiple interleaved responsibilities (not just long)
- Deeply nested control flow (5+ levels) where flattening would prevent bugs
- Complex boolean expressions that a reader would likely misinterpret
Only report complexity issues that are HIGH risk for future bugs. Ignore acceptable complexity in configuration, CLI argument parsing, or generated code.
Do NOT report:
- Functions that are long but linear and easy to follow
- Acceptable complexity: configuration setup, CLI parsing, test helpers, builder patterns
- Code that is complex because the problem is complex — only report if restructuring would reduce bug risk
- "This function does multiple things" unless you can identify a specific bug risk from the coupling
- Suggestions that would just move complexity elsewhere without reducing it
For each issue found, respond with a JSON array:
Severity guide:
- medium: Complexity that has a concrete risk of causing bugs during future changes
- low: Complexity that makes review harder but is unlikely to cause bugs
Return at most 2 findings. Prefer [] over reporting complexity that is justified.
Respond with a JSON array (no markdown fences):
[{"title": "...", "description": "...", "severity": "medium|low", "file": "...", "line": N, "suggestion": "..."}]
If no issues found, respond with: []"#;
+382 -99
View File
@@ -5,22 +5,49 @@ use compliance_core::models::{Finding, FindingStatus};
use crate::llm::LlmClient;
use crate::pipeline::orchestrator::GraphContext;
const TRIAGE_SYSTEM_PROMPT: &str = r#"You are a security finding triage expert. Analyze the following security finding with its code context and determine the appropriate action.
/// Maximum number of findings to include in a single LLM triage call.
const TRIAGE_CHUNK_SIZE: usize = 30;
const TRIAGE_SYSTEM_PROMPT: &str = r#"You are a pragmatic security triage expert. Your job is to filter out noise and keep only findings that a developer should actually fix. Be aggressive about dismissing false positives — a clean, high-signal list is more valuable than a comprehensive one.
Actions:
- "confirm": The finding is a true positive at the reported severity. Keep as-is.
- "downgrade": The finding is real but over-reported. Lower severity recommended.
- "upgrade": The finding is under-reported. Higher severity recommended.
- "dismiss": The finding is a false positive. Should be removed.
- "confirm": True positive with real impact. Keep severity as-is.
- "downgrade": Real issue but over-reported severity. Lower it.
- "upgrade": Under-reported — higher severity warranted.
- "dismiss": False positive, not exploitable, or not actionable. Remove it.
Consider:
- Is the code in a test, example, or generated file? (lower confidence for test code)
- Does the surrounding code context confirm or refute the finding?
- Is the finding actionable by a developer?
- Would a real attacker be able to exploit this?
Dismiss when:
- The scanner flagged a language idiom as a bug (see examples below)
- The finding is in test/example/generated/vendored code
- The "vulnerability" requires preconditions that don't exist in the code
- The finding is about code style, complexity, or theoretical concerns rather than actual bugs
- A hash function is used for non-security purposes (dedup, caching, content addressing)
- Internal logging of non-sensitive operational data is flagged as "information disclosure"
- The finding duplicates another finding already in the list
- Framework-provided security is already in place (e.g. ORM parameterized queries, CSRF middleware, auth decorators)
Respond in JSON format:
{"action": "confirm|downgrade|upgrade|dismiss", "confidence": 0-10, "rationale": "brief explanation", "remediation": "optional fix suggestion"}"#;
Common false positive patterns by language (dismiss these):
- Rust: short-circuit `||`/`&&`, variable shadowing, `clone()`, `unsafe` with safety docs, `sha2` for fingerprinting
- Python: EAFP try/except, `subprocess` with hardcoded args, `pickle` on trusted data, Django `mark_safe` on static content
- Go: `if err != nil` is not "swallowed error", `crypto/rand` is secure, returning errors is not "information disclosure"
- Java/Kotlin: Spring Security annotations are valid auth, JPA parameterized queries are safe, Kotlin `!!` in tests is fine
- Ruby: Rails `params.permit` is validation, ActiveRecord finders are parameterized, `html_safe` on generated content
- PHP: PDO prepared statements are safe, Laravel Eloquent is parameterized, `htmlspecialchars` is XSS mitigation
- C/C++: `strncpy`/`snprintf` are bounds-checked, smart pointers manage memory, RAII handles cleanup
Confirm only when:
- You can describe a concrete scenario where the bug manifests or the vulnerability is exploitable
- The fix is actionable (developer can change specific code to resolve it)
- The finding is in production code that handles external input or sensitive data
Confidence scoring (0-10):
- 8-10: Certain true positive with clear exploit/bug scenario
- 5-7: Likely true positive, some assumptions required
- 3-4: Uncertain, needs manual review
- 0-2: Almost certainly a false positive
Respond with a JSON array, one entry per finding in the same order presented (no markdown fences):
[{"id": "<fingerprint>", "action": "confirm|downgrade|upgrade|dismiss", "confidence": 0-10, "rationale": "1-2 sentences", "remediation": "optional fix"}, ...]"#;
pub async fn triage_findings(
llm: &Arc<LlmClient>,
@@ -29,60 +56,76 @@ pub async fn triage_findings(
) -> usize {
let mut passed = 0;
for finding in findings.iter_mut() {
let file_classification = classify_file_path(finding.file_path.as_deref());
// Process findings in chunks to avoid overflowing the LLM context window.
for chunk_start in (0..findings.len()).step_by(TRIAGE_CHUNK_SIZE) {
let chunk_end = (chunk_start + TRIAGE_CHUNK_SIZE).min(findings.len());
let chunk = &mut findings[chunk_start..chunk_end];
let mut user_prompt = format!(
"Scanner: {}\nRule: {}\nSeverity: {}\nTitle: {}\nDescription: {}\nFile: {}\nLine: {}\nCode: {}\nFile classification: {}",
finding.scanner,
finding.rule_id.as_deref().unwrap_or("N/A"),
finding.severity,
finding.title,
finding.description,
finding.file_path.as_deref().unwrap_or("N/A"),
finding.line_number.map(|n| n.to_string()).unwrap_or_else(|| "N/A".to_string()),
finding.code_snippet.as_deref().unwrap_or("N/A"),
file_classification,
);
// Build a combined prompt for the entire chunk.
let mut user_prompt = String::new();
let mut file_classifications: Vec<String> = Vec::new();
for (i, finding) in chunk.iter().enumerate() {
let file_classification = classify_file_path(finding.file_path.as_deref());
// Enrich with surrounding code context if possible
if let Some(context) = read_surrounding_context(finding) {
user_prompt.push_str(&format!(
"\n\n--- Surrounding Code (50 lines) ---\n{context}"
"\n--- Finding {} (id: {}) ---\nScanner: {}\nRule: {}\nSeverity: {}\nTitle: {}\nDescription: {}\nFile: {}\nLine: {}\nCode: {}\nFile classification: {}",
i + 1,
finding.fingerprint,
finding.scanner,
finding.rule_id.as_deref().unwrap_or("N/A"),
finding.severity,
finding.title,
finding.description,
finding.file_path.as_deref().unwrap_or("N/A"),
finding.line_number.map(|n| n.to_string()).unwrap_or_else(|| "N/A".to_string()),
finding.code_snippet.as_deref().unwrap_or("N/A"),
file_classification,
));
}
// Enrich with graph context if available
if let Some(ctx) = graph_context {
if let Some(impact) = ctx
.impacts
.iter()
.find(|i| i.finding_id == finding.fingerprint)
{
// Enrich with surrounding code context if possible
if let Some(context) = read_surrounding_context(finding) {
user_prompt.push_str(&format!(
"\n\n--- Code Graph Context ---\n\
Blast radius: {} nodes affected\n\
Entry points affected: {}\n\
Direct callers: {}\n\
Communities affected: {}\n\
Call chains: {}",
impact.blast_radius,
if impact.affected_entry_points.is_empty() {
"none".to_string()
} else {
impact.affected_entry_points.join(", ")
},
if impact.direct_callers.is_empty() {
"none".to_string()
} else {
impact.direct_callers.join(", ")
},
impact.affected_communities.len(),
impact.call_chains.len(),
"\n\n--- Surrounding Code (50 lines) ---\n{context}"
));
}
// Enrich with graph context if available
if let Some(ctx) = graph_context {
if let Some(impact) = ctx
.impacts
.iter()
.find(|im| im.finding_id == finding.fingerprint)
{
user_prompt.push_str(&format!(
"\n\n--- Code Graph Context ---\n\
Blast radius: {} nodes affected\n\
Entry points affected: {}\n\
Direct callers: {}\n\
Communities affected: {}\n\
Call chains: {}",
impact.blast_radius,
if impact.affected_entry_points.is_empty() {
"none".to_string()
} else {
impact.affected_entry_points.join(", ")
},
if impact.direct_callers.is_empty() {
"none".to_string()
} else {
impact.direct_callers.join(", ")
},
impact.affected_communities.len(),
impact.call_chains.len(),
));
}
}
user_prompt.push('\n');
file_classifications.push(file_classification);
}
// Send the batch to the LLM.
match llm
.chat(TRIAGE_SYSTEM_PROMPT, &user_prompt, Some(0.1))
.await
@@ -98,58 +141,77 @@ pub async fn triage_findings(
} else {
cleaned
};
if let Ok(result) = serde_json::from_str::<TriageResult>(cleaned) {
// Apply file-path confidence adjustment
let adjusted_confidence =
adjust_confidence(result.confidence, &file_classification);
finding.confidence = Some(adjusted_confidence);
finding.triage_action = Some(result.action.clone());
finding.triage_rationale = Some(result.rationale);
if let Some(remediation) = result.remediation {
finding.remediation = Some(remediation);
}
match result.action.as_str() {
"dismiss" => {
finding.status = FindingStatus::FalsePositive;
}
"downgrade" => {
// Downgrade severity by one level
finding.severity = downgrade_severity(&finding.severity);
finding.status = FindingStatus::Triaged;
passed += 1;
}
"upgrade" => {
finding.severity = upgrade_severity(&finding.severity);
finding.status = FindingStatus::Triaged;
passed += 1;
}
_ => {
// "confirm" or unknown — keep as-is
if adjusted_confidence >= 3.0 {
match serde_json::from_str::<Vec<TriageResult>>(cleaned) {
Ok(results) => {
for (idx, finding) in chunk.iter_mut().enumerate() {
// Match result by position; fall back to keeping the finding.
let Some(result) = results.get(idx) else {
finding.status = FindingStatus::Triaged;
passed += 1;
} else {
finding.status = FindingStatus::FalsePositive;
continue;
};
let file_classification = file_classifications
.get(idx)
.map(|s| s.as_str())
.unwrap_or("unknown");
let adjusted_confidence =
adjust_confidence(result.confidence, file_classification);
finding.confidence = Some(adjusted_confidence);
finding.triage_action = Some(result.action.clone());
finding.triage_rationale = Some(result.rationale.clone());
if let Some(ref remediation) = result.remediation {
finding.remediation = Some(remediation.clone());
}
match result.action.as_str() {
"dismiss" => {
finding.status = FindingStatus::FalsePositive;
}
"downgrade" => {
finding.severity = downgrade_severity(&finding.severity);
finding.status = FindingStatus::Triaged;
passed += 1;
}
"upgrade" => {
finding.severity = upgrade_severity(&finding.severity);
finding.status = FindingStatus::Triaged;
passed += 1;
}
_ => {
// "confirm" or unknown — keep as-is
if adjusted_confidence >= 3.0 {
finding.status = FindingStatus::Triaged;
passed += 1;
} else {
finding.status = FindingStatus::FalsePositive;
}
}
}
}
}
} else {
// Parse failure — keep the finding
finding.status = FindingStatus::Triaged;
passed += 1;
tracing::warn!(
"Failed to parse triage response for {}: {response}",
finding.fingerprint
);
Err(_) => {
// Batch parse failure — keep all findings in the chunk.
tracing::warn!(
"Failed to parse batch triage response for chunk starting at {chunk_start}: {cleaned}"
);
for finding in chunk.iter_mut() {
finding.status = FindingStatus::Triaged;
passed += 1;
}
}
}
}
Err(e) => {
// On LLM error, keep the finding
tracing::warn!("LLM triage failed for {}: {e}", finding.fingerprint);
finding.status = FindingStatus::Triaged;
passed += 1;
// On LLM error, keep all findings in the chunk.
tracing::warn!("LLM batch triage failed for chunk starting at {chunk_start}: {e}");
for finding in chunk.iter_mut() {
finding.status = FindingStatus::Triaged;
passed += 1;
}
}
}
}
@@ -266,6 +328,10 @@ fn upgrade_severity(
#[derive(serde::Deserialize)]
struct TriageResult {
/// Finding fingerprint echoed back by the LLM (optional).
#[serde(default)]
#[allow(dead_code)]
id: String,
#[serde(default = "default_action")]
action: String,
#[serde(default)]
@@ -278,3 +344,220 @@ struct TriageResult {
fn default_action() -> String {
"confirm".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::Severity;
// ── classify_file_path ───────────────────────────────────────
#[test]
fn classify_none_path() {
assert_eq!(classify_file_path(None), "unknown");
}
#[test]
fn classify_production_path() {
assert_eq!(classify_file_path(Some("src/main.rs")), "production");
assert_eq!(classify_file_path(Some("lib/core/engine.py")), "production");
}
#[test]
fn classify_test_paths() {
assert_eq!(classify_file_path(Some("src/test/helper.rs")), "test");
assert_eq!(classify_file_path(Some("src/tests/unit.rs")), "test");
assert_eq!(classify_file_path(Some("foo_test.go")), "test");
assert_eq!(classify_file_path(Some("bar.test.js")), "test");
assert_eq!(classify_file_path(Some("baz.spec.ts")), "test");
assert_eq!(
classify_file_path(Some("data/fixtures/sample.json")),
"test"
);
assert_eq!(classify_file_path(Some("src/testdata/input.txt")), "test");
}
#[test]
fn classify_example_paths() {
assert_eq!(
classify_file_path(Some("docs/examples/basic.rs")),
"example"
);
// /example matches because contains("/example")
assert_eq!(classify_file_path(Some("src/example/main.py")), "example");
assert_eq!(classify_file_path(Some("src/demo/run.sh")), "example");
assert_eq!(classify_file_path(Some("src/sample/lib.rs")), "example");
}
#[test]
fn classify_generated_paths() {
assert_eq!(
classify_file_path(Some("src/generated/api.rs")),
"generated"
);
assert_eq!(
classify_file_path(Some("proto/gen/service.go")),
"generated"
);
assert_eq!(classify_file_path(Some("api.generated.ts")), "generated");
assert_eq!(classify_file_path(Some("service.pb.go")), "generated");
assert_eq!(classify_file_path(Some("model_generated.rs")), "generated");
}
#[test]
fn classify_vendored_paths() {
// Implementation checks for /vendor/, /node_modules/, /third_party/ (with slashes)
assert_eq!(
classify_file_path(Some("src/vendor/lib/foo.go")),
"vendored"
);
assert_eq!(
classify_file_path(Some("src/node_modules/pkg/index.js")),
"vendored"
);
assert_eq!(
classify_file_path(Some("src/third_party/lib.c")),
"vendored"
);
}
#[test]
fn classify_is_case_insensitive() {
assert_eq!(classify_file_path(Some("src/TEST/Helper.rs")), "test");
assert_eq!(classify_file_path(Some("src/VENDOR/lib.go")), "vendored");
assert_eq!(
classify_file_path(Some("src/GENERATED/foo.ts")),
"generated"
);
}
// ── adjust_confidence ────────────────────────────────────────
#[test]
fn adjust_confidence_production() {
assert_eq!(adjust_confidence(8.0, "production"), 8.0);
}
#[test]
fn adjust_confidence_test() {
assert_eq!(adjust_confidence(10.0, "test"), 5.0);
}
#[test]
fn adjust_confidence_example() {
assert_eq!(adjust_confidence(10.0, "example"), 6.0);
}
#[test]
fn adjust_confidence_generated() {
assert_eq!(adjust_confidence(10.0, "generated"), 3.0);
}
#[test]
fn adjust_confidence_vendored() {
assert_eq!(adjust_confidence(10.0, "vendored"), 4.0);
}
#[test]
fn adjust_confidence_unknown_classification() {
assert_eq!(adjust_confidence(7.0, "unknown"), 7.0);
assert_eq!(adjust_confidence(7.0, "something_else"), 7.0);
}
#[test]
fn adjust_confidence_zero() {
assert_eq!(adjust_confidence(0.0, "test"), 0.0);
assert_eq!(adjust_confidence(0.0, "production"), 0.0);
}
// ── downgrade_severity ───────────────────────────────────────
#[test]
fn downgrade_severity_all_levels() {
assert_eq!(downgrade_severity(&Severity::Critical), Severity::High);
assert_eq!(downgrade_severity(&Severity::High), Severity::Medium);
assert_eq!(downgrade_severity(&Severity::Medium), Severity::Low);
assert_eq!(downgrade_severity(&Severity::Low), Severity::Info);
assert_eq!(downgrade_severity(&Severity::Info), Severity::Info);
}
#[test]
fn downgrade_severity_info_is_floor() {
// Downgrading Info twice should still be Info
let s = downgrade_severity(&Severity::Info);
assert_eq!(downgrade_severity(&s), Severity::Info);
}
// ── upgrade_severity ─────────────────────────────────────────
#[test]
fn upgrade_severity_all_levels() {
assert_eq!(upgrade_severity(&Severity::Info), Severity::Low);
assert_eq!(upgrade_severity(&Severity::Low), Severity::Medium);
assert_eq!(upgrade_severity(&Severity::Medium), Severity::High);
assert_eq!(upgrade_severity(&Severity::High), Severity::Critical);
assert_eq!(upgrade_severity(&Severity::Critical), Severity::Critical);
}
#[test]
fn upgrade_severity_critical_is_ceiling() {
let s = upgrade_severity(&Severity::Critical);
assert_eq!(upgrade_severity(&s), Severity::Critical);
}
// ── upgrade/downgrade roundtrip ──────────────────────────────
#[test]
fn upgrade_then_downgrade_is_identity_for_middle_values() {
for sev in [Severity::Low, Severity::Medium, Severity::High] {
assert_eq!(downgrade_severity(&upgrade_severity(&sev)), sev);
}
}
// ── TriageResult deserialization ─────────────────────────────
#[test]
fn triage_result_full() {
let json = r#"{"action":"dismiss","confidence":8.5,"rationale":"false positive","remediation":"remove code"}"#;
let r: TriageResult = serde_json::from_str(json).unwrap();
assert_eq!(r.action, "dismiss");
assert_eq!(r.confidence, 8.5);
assert_eq!(r.rationale, "false positive");
assert_eq!(r.remediation.as_deref(), Some("remove code"));
}
#[test]
fn triage_result_defaults() {
let json = r#"{}"#;
let r: TriageResult = serde_json::from_str(json).unwrap();
assert_eq!(r.action, "confirm");
assert_eq!(r.confidence, 0.0);
assert_eq!(r.rationale, "");
assert!(r.remediation.is_none());
}
#[test]
fn triage_result_partial() {
let json = r#"{"action":"downgrade","confidence":6.0}"#;
let r: TriageResult = serde_json::from_str(json).unwrap();
assert_eq!(r.action, "downgrade");
assert_eq!(r.confidence, 6.0);
assert_eq!(r.rationale, "");
assert!(r.remediation.is_none());
}
#[test]
fn triage_result_with_markdown_fences() {
// Simulate LLM wrapping response in markdown code fences
let raw = "```json\n{\"action\":\"upgrade\",\"confidence\":9,\"rationale\":\"critical\",\"remediation\":null}\n```";
let cleaned = raw
.trim()
.trim_start_matches("```json")
.trim_start_matches("```")
.trim_end_matches("```")
.trim();
let r: TriageResult = serde_json::from_str(cleaned).unwrap();
assert_eq!(r.action, "upgrade");
assert_eq!(r.confidence, 9.0);
}
}
+369
View File
@@ -0,0 +1,369 @@
use serde::{Deserialize, Serialize};
// ── Request types ──────────────────────────────────────────────
#[derive(Serialize, Clone, Debug)]
pub struct ChatMessage {
pub role: String,
pub content: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_calls: Option<Vec<ToolCallRequest>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_call_id: Option<String>,
}
#[derive(Serialize)]
pub(crate) struct ChatCompletionRequest {
pub(crate) model: String,
pub(crate) messages: Vec<ChatMessage>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) temperature: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) max_tokens: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) tools: Option<Vec<ToolDefinitionPayload>>,
}
#[derive(Serialize)]
pub(crate) struct ToolDefinitionPayload {
pub(crate) r#type: String,
pub(crate) function: ToolFunctionPayload,
}
#[derive(Serialize)]
pub(crate) struct ToolFunctionPayload {
pub(crate) name: String,
pub(crate) description: String,
pub(crate) parameters: serde_json::Value,
}
// ── Response types ─────────────────────────────────────────────
#[derive(Deserialize)]
pub(crate) struct ChatCompletionResponse {
pub(crate) choices: Vec<ChatChoice>,
}
#[derive(Deserialize)]
pub(crate) struct ChatChoice {
pub(crate) message: ChatResponseMessage,
}
#[derive(Deserialize)]
pub(crate) struct ChatResponseMessage {
#[serde(default)]
pub(crate) content: Option<String>,
#[serde(default)]
pub(crate) tool_calls: Option<Vec<ToolCallResponse>>,
}
#[derive(Deserialize)]
pub(crate) struct ToolCallResponse {
pub(crate) id: String,
pub(crate) function: ToolCallFunction,
}
#[derive(Deserialize)]
pub(crate) struct ToolCallFunction {
pub(crate) name: String,
pub(crate) arguments: String,
}
// ── Public types for tool calling ──────────────────────────────
/// Definition of a tool that the LLM can invoke
#[derive(Debug, Clone, Serialize)]
pub struct ToolDefinition {
pub name: String,
pub description: String,
pub parameters: serde_json::Value,
}
/// A tool call request from the LLM
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct LlmToolCall {
pub id: String,
pub name: String,
pub arguments: serde_json::Value,
}
/// A tool call in the request message format (for sending back tool_calls in assistant messages)
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ToolCallRequest {
pub id: String,
pub r#type: String,
pub function: ToolCallRequestFunction,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ToolCallRequestFunction {
pub name: String,
pub arguments: String,
}
/// Response from the LLM — either content or tool calls
#[derive(Debug, Clone)]
pub enum LlmResponse {
Content(String),
/// Tool calls with optional reasoning text from the LLM
ToolCalls {
calls: Vec<LlmToolCall>,
reasoning: String,
},
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
// ── ChatMessage ──────────────────────────────────────────────
#[test]
fn chat_message_serializes_minimal() {
let msg = ChatMessage {
role: "user".to_string(),
content: Some("hello".to_string()),
tool_calls: None,
tool_call_id: None,
};
let v = serde_json::to_value(&msg).unwrap();
assert_eq!(v["role"], "user");
assert_eq!(v["content"], "hello");
// None fields with skip_serializing_if should be absent
assert!(v.get("tool_calls").is_none());
assert!(v.get("tool_call_id").is_none());
}
#[test]
fn chat_message_serializes_with_tool_calls() {
let msg = ChatMessage {
role: "assistant".to_string(),
content: None,
tool_calls: Some(vec![ToolCallRequest {
id: "call_1".to_string(),
r#type: "function".to_string(),
function: ToolCallRequestFunction {
name: "get_weather".to_string(),
arguments: r#"{"city":"NYC"}"#.to_string(),
},
}]),
tool_call_id: None,
};
let v = serde_json::to_value(&msg).unwrap();
assert!(v["tool_calls"].is_array());
assert_eq!(v["tool_calls"][0]["function"]["name"], "get_weather");
}
#[test]
fn chat_message_content_null_when_none() {
let msg = ChatMessage {
role: "assistant".to_string(),
content: None,
tool_calls: None,
tool_call_id: None,
};
let v = serde_json::to_value(&msg).unwrap();
assert!(v["content"].is_null());
}
// ── ToolDefinition ───────────────────────────────────────────
#[test]
fn tool_definition_serializes() {
let td = ToolDefinition {
name: "search".to_string(),
description: "Search the web".to_string(),
parameters: json!({"type": "object", "properties": {"q": {"type": "string"}}}),
};
let v = serde_json::to_value(&td).unwrap();
assert_eq!(v["name"], "search");
assert_eq!(v["parameters"]["type"], "object");
}
#[test]
fn tool_definition_empty_parameters() {
let td = ToolDefinition {
name: "noop".to_string(),
description: "".to_string(),
parameters: json!({}),
};
let v = serde_json::to_value(&td).unwrap();
assert_eq!(v["parameters"], json!({}));
}
// ── LlmToolCall ──────────────────────────────────────────────
#[test]
fn llm_tool_call_roundtrip() {
let call = LlmToolCall {
id: "tc_42".to_string(),
name: "run_scan".to_string(),
arguments: json!({"path": "/tmp", "verbose": true}),
};
let serialized = serde_json::to_string(&call).unwrap();
let deserialized: LlmToolCall = serde_json::from_str(&serialized).unwrap();
assert_eq!(deserialized.id, "tc_42");
assert_eq!(deserialized.name, "run_scan");
assert_eq!(deserialized.arguments["path"], "/tmp");
assert_eq!(deserialized.arguments["verbose"], true);
}
#[test]
fn llm_tool_call_empty_arguments() {
let call = LlmToolCall {
id: "tc_0".to_string(),
name: "noop".to_string(),
arguments: json!({}),
};
let rt: LlmToolCall = serde_json::from_str(&serde_json::to_string(&call).unwrap()).unwrap();
assert!(rt.arguments.as_object().unwrap().is_empty());
}
// ── ToolCallRequest / ToolCallRequestFunction ────────────────
#[test]
fn tool_call_request_roundtrip() {
let req = ToolCallRequest {
id: "call_abc".to_string(),
r#type: "function".to_string(),
function: ToolCallRequestFunction {
name: "my_func".to_string(),
arguments: r#"{"x":1}"#.to_string(),
},
};
let json_str = serde_json::to_string(&req).unwrap();
let back: ToolCallRequest = serde_json::from_str(&json_str).unwrap();
assert_eq!(back.id, "call_abc");
assert_eq!(back.r#type, "function");
assert_eq!(back.function.name, "my_func");
assert_eq!(back.function.arguments, r#"{"x":1}"#);
}
#[test]
fn tool_call_request_type_field_serializes_as_type() {
let req = ToolCallRequest {
id: "id".to_string(),
r#type: "function".to_string(),
function: ToolCallRequestFunction {
name: "f".to_string(),
arguments: "{}".to_string(),
},
};
let v = serde_json::to_value(&req).unwrap();
// The field should be "type" in JSON, not "r#type"
assert!(v.get("type").is_some());
assert!(v.get("r#type").is_none());
}
// ── ChatCompletionRequest ────────────────────────────────────
#[test]
fn chat_completion_request_skips_none_fields() {
let req = ChatCompletionRequest {
model: "gpt-4".to_string(),
messages: vec![],
temperature: None,
max_tokens: None,
tools: None,
};
let v = serde_json::to_value(&req).unwrap();
assert_eq!(v["model"], "gpt-4");
assert!(v.get("temperature").is_none());
assert!(v.get("max_tokens").is_none());
assert!(v.get("tools").is_none());
}
#[test]
fn chat_completion_request_includes_set_fields() {
let req = ChatCompletionRequest {
model: "gpt-4".to_string(),
messages: vec![],
temperature: Some(0.7),
max_tokens: Some(1024),
tools: Some(vec![]),
};
let v = serde_json::to_value(&req).unwrap();
assert_eq!(v["temperature"], 0.7);
assert_eq!(v["max_tokens"], 1024);
assert!(v["tools"].is_array());
}
// ── ChatCompletionResponse deserialization ───────────────────
#[test]
fn chat_completion_response_deserializes_content() {
let json_str = r#"{"choices":[{"message":{"content":"Hello!"}}]}"#;
let resp: ChatCompletionResponse = serde_json::from_str(json_str).unwrap();
assert_eq!(resp.choices.len(), 1);
assert_eq!(resp.choices[0].message.content.as_deref(), Some("Hello!"));
assert!(resp.choices[0].message.tool_calls.is_none());
}
#[test]
fn chat_completion_response_deserializes_tool_calls() {
let json_str = r#"{
"choices": [{
"message": {
"tool_calls": [{
"id": "call_1",
"function": {"name": "search", "arguments": "{\"q\":\"rust\"}"}
}]
}
}]
}"#;
let resp: ChatCompletionResponse = serde_json::from_str(json_str).unwrap();
let tc = resp.choices[0].message.tool_calls.as_ref().unwrap();
assert_eq!(tc.len(), 1);
assert_eq!(tc[0].id, "call_1");
assert_eq!(tc[0].function.name, "search");
}
#[test]
fn chat_completion_response_defaults_missing_fields() {
// content and tool_calls are both missing — should default to None
let json_str = r#"{"choices":[{"message":{}}]}"#;
let resp: ChatCompletionResponse = serde_json::from_str(json_str).unwrap();
assert!(resp.choices[0].message.content.is_none());
assert!(resp.choices[0].message.tool_calls.is_none());
}
// ── LlmResponse ─────────────────────────────────────────────
#[test]
fn llm_response_content_variant() {
let resp = LlmResponse::Content("answer".to_string());
match resp {
LlmResponse::Content(s) => assert_eq!(s, "answer"),
_ => panic!("expected Content variant"),
}
}
#[test]
fn llm_response_tool_calls_variant() {
let resp = LlmResponse::ToolCalls {
calls: vec![LlmToolCall {
id: "1".to_string(),
name: "f".to_string(),
arguments: json!({}),
}],
reasoning: "because".to_string(),
};
match resp {
LlmResponse::ToolCalls { calls, reasoning } => {
assert_eq!(calls.len(), 1);
assert_eq!(reasoning, "because");
}
_ => panic!("expected ToolCalls variant"),
}
}
#[test]
fn llm_response_empty_content() {
let resp = LlmResponse::Content(String::new());
match resp {
LlmResponse::Content(s) => assert!(s.is_empty()),
_ => panic!("expected Content variant"),
}
}
}
+18 -18
View File
@@ -1,24 +1,21 @@
mod agent;
mod api;
mod config;
mod database;
mod error;
mod llm;
mod pipeline;
mod rag;
mod scheduler;
mod ssh;
#[allow(dead_code)]
mod trackers;
mod webhooks;
use compliance_agent::{agent, api, config, database, scheduler, ssh, webhooks};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
dotenvy::dotenv().ok();
match dotenvy::dotenv() {
Ok(path) => eprintln!("[dotenv] Loaded from: {}", path.display()),
Err(_) => eprintln!("[dotenv] No .env file found, using environment variables"),
}
let _telemetry_guard = compliance_core::telemetry::init_telemetry("compliance-agent");
tracing::info!("Loading configuration...");
// Log critical env vars at startup
tracing::info!(
chrome_ws_url = std::env::var("CHROME_WS_URL").ok().as_deref(),
pentest_email = std::env::var("PENTEST_VERIFICATION_EMAIL").ok().as_deref(),
encryption_key_set = std::env::var("PENTEST_ENCRYPTION_KEY").is_ok(),
"Loading configuration..."
);
let config = config::load_config()?;
// Ensure SSH key pair exists for cloning private repos
@@ -28,10 +25,13 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
}
tracing::info!("Connecting to MongoDB...");
let db = database::Database::connect(&config.mongodb_uri, &config.mongodb_database).await?;
db.ensure_indexes().await?;
// Per-tenant pool only — the agent has no shared "default" database
// after M7.2-D. `mongodb_database` is now the db-name prefix used
// for tenant databases (`<prefix>_<tenant_id>`).
let db_pool =
database::DatabasePool::connect(&config.mongodb_uri, &config.mongodb_database).await?;
let agent = agent::ComplianceAgent::new(config.clone(), db.clone());
let agent = agent::ComplianceAgent::new(config.clone(), db_pool);
tracing::info!("Starting scheduler...");
let scheduler_agent = agent.clone();
+487
View File
@@ -0,0 +1,487 @@
use compliance_core::models::pentest::{IdentityProvider, TestUserRecord};
use compliance_core::AgentConfig;
use secrecy::ExposeSecret;
use tracing::{info, warn};
/// Attempt to delete a test user created during a pentest session.
///
/// Routes to the appropriate identity provider based on `TestUserRecord.provider`.
/// Falls back to browser-based cleanup if no API credentials are available.
///
/// Returns `Ok(true)` if the user was deleted, `Ok(false)` if skipped, `Err` on failure.
pub async fn cleanup_test_user(
user: &TestUserRecord,
config: &AgentConfig,
http: &reqwest::Client,
) -> Result<bool, String> {
if user.cleaned_up {
return Ok(false);
}
let provider = user.provider.as_ref();
match provider {
Some(IdentityProvider::Keycloak) => cleanup_keycloak(user, config, http).await,
Some(IdentityProvider::Auth0) => cleanup_auth0(user, config, http).await,
Some(IdentityProvider::Okta) => cleanup_okta(user, config, http).await,
Some(IdentityProvider::Firebase) => {
warn!("Firebase user cleanup not yet implemented");
Ok(false)
}
Some(IdentityProvider::Custom) | None => {
// For custom/unknown providers, try Keycloak if configured, else skip
if config.keycloak_url.is_some() && config.keycloak_admin_username.is_some() {
cleanup_keycloak(user, config, http).await
} else {
warn!(
username = user.username.as_deref(),
"No identity provider configured for cleanup — skipping"
);
Ok(false)
}
}
}
}
/// Delete a user from Keycloak via the Admin REST API.
///
/// Flow: get admin token → search user by username → delete by ID.
async fn cleanup_keycloak(
user: &TestUserRecord,
config: &AgentConfig,
http: &reqwest::Client,
) -> Result<bool, String> {
let base_url = config
.keycloak_url
.as_deref()
.ok_or("KEYCLOAK_URL not configured")?;
let realm = config
.keycloak_realm
.as_deref()
.ok_or("KEYCLOAK_REALM not configured")?;
let admin_user = config
.keycloak_admin_username
.as_deref()
.ok_or("KEYCLOAK_ADMIN_USERNAME not configured")?;
let admin_pass = config
.keycloak_admin_password
.as_ref()
.ok_or("KEYCLOAK_ADMIN_PASSWORD not configured")?;
let username = user
.username
.as_deref()
.ok_or("No username in test user record")?;
info!(username, realm, "Cleaning up Keycloak test user");
// Step 1: Get admin access token
let token_url = format!("{base_url}/realms/master/protocol/openid-connect/token");
let token_resp = http
.post(&token_url)
.form(&[
("grant_type", "password"),
("client_id", "admin-cli"),
("username", admin_user),
("password", admin_pass.expose_secret()),
])
.send()
.await
.map_err(|e| format!("Keycloak token request failed: {e}"))?;
if !token_resp.status().is_success() {
let status = token_resp.status();
let body = token_resp.text().await.unwrap_or_default();
return Err(format!("Keycloak admin auth failed ({status}): {body}"));
}
let token_body: serde_json::Value = token_resp
.json()
.await
.map_err(|e| format!("Failed to parse Keycloak token: {e}"))?;
let access_token = token_body
.get("access_token")
.and_then(|v| v.as_str())
.ok_or("No access_token in Keycloak response")?;
// Step 2: Search for user by username
let search_url =
format!("{base_url}/admin/realms/{realm}/users?username={username}&exact=true");
let search_resp = http
.get(&search_url)
.bearer_auth(access_token)
.send()
.await
.map_err(|e| format!("Keycloak user search failed: {e}"))?;
if !search_resp.status().is_success() {
let status = search_resp.status();
let body = search_resp.text().await.unwrap_or_default();
return Err(format!("Keycloak user search failed ({status}): {body}"));
}
let users: Vec<serde_json::Value> = search_resp
.json()
.await
.map_err(|e| format!("Failed to parse Keycloak users: {e}"))?;
let user_id = users
.first()
.and_then(|u| u.get("id"))
.and_then(|v| v.as_str())
.ok_or_else(|| format!("User '{username}' not found in Keycloak realm '{realm}'"))?;
// Step 3: Delete the user
let delete_url = format!("{base_url}/admin/realms/{realm}/users/{user_id}");
let delete_resp = http
.delete(&delete_url)
.bearer_auth(access_token)
.send()
.await
.map_err(|e| format!("Keycloak user delete failed: {e}"))?;
if delete_resp.status().is_success() || delete_resp.status().as_u16() == 204 {
info!(username, user_id, "Keycloak test user deleted");
Ok(true)
} else {
let status = delete_resp.status();
let body = delete_resp.text().await.unwrap_or_default();
Err(format!("Keycloak delete failed ({status}): {body}"))
}
}
/// Delete a user from Auth0 via the Management API.
///
/// Requires `AUTH0_DOMAIN`, `AUTH0_CLIENT_ID`, `AUTH0_CLIENT_SECRET` env vars.
async fn cleanup_auth0(
user: &TestUserRecord,
_config: &AgentConfig,
http: &reqwest::Client,
) -> Result<bool, String> {
let domain = std::env::var("AUTH0_DOMAIN").map_err(|_| "AUTH0_DOMAIN not set")?;
let client_id = std::env::var("AUTH0_CLIENT_ID").map_err(|_| "AUTH0_CLIENT_ID not set")?;
let client_secret =
std::env::var("AUTH0_CLIENT_SECRET").map_err(|_| "AUTH0_CLIENT_SECRET not set")?;
let email = user
.email
.as_deref()
.ok_or("No email in test user record for Auth0 lookup")?;
info!(email, "Cleaning up Auth0 test user");
// Get management API token
let token_resp = http
.post(format!("https://{domain}/oauth/token"))
.json(&serde_json::json!({
"grant_type": "client_credentials",
"client_id": client_id,
"client_secret": client_secret,
"audience": format!("https://{domain}/api/v2/"),
}))
.send()
.await
.map_err(|e| format!("Auth0 token request failed: {e}"))?;
let token_body: serde_json::Value = token_resp
.json()
.await
.map_err(|e| format!("Failed to parse Auth0 token: {e}"))?;
let access_token = token_body
.get("access_token")
.and_then(|v| v.as_str())
.ok_or("No access_token in Auth0 response")?;
// Search for user by email
let encoded_email = urlencoding::encode(email);
let search_url = format!("https://{domain}/api/v2/users-by-email?email={encoded_email}");
let search_resp = http
.get(&search_url)
.bearer_auth(access_token)
.send()
.await
.map_err(|e| format!("Auth0 user search failed: {e}"))?;
let users: Vec<serde_json::Value> = search_resp
.json()
.await
.map_err(|e| format!("Failed to parse Auth0 users: {e}"))?;
let user_id = users
.first()
.and_then(|u| u.get("user_id"))
.and_then(|v| v.as_str())
.ok_or_else(|| format!("User with email '{email}' not found in Auth0"))?;
// Delete
let encoded_id = urlencoding::encode(user_id);
let delete_resp = http
.delete(format!("https://{domain}/api/v2/users/{encoded_id}"))
.bearer_auth(access_token)
.send()
.await
.map_err(|e| format!("Auth0 user delete failed: {e}"))?;
if delete_resp.status().is_success() || delete_resp.status().as_u16() == 204 {
info!(email, user_id, "Auth0 test user deleted");
Ok(true)
} else {
let status = delete_resp.status();
let body = delete_resp.text().await.unwrap_or_default();
Err(format!("Auth0 delete failed ({status}): {body}"))
}
}
/// Delete a user from Okta via the Users API.
///
/// Requires `OKTA_DOMAIN`, `OKTA_API_TOKEN` env vars.
async fn cleanup_okta(
user: &TestUserRecord,
_config: &AgentConfig,
http: &reqwest::Client,
) -> Result<bool, String> {
let domain = std::env::var("OKTA_DOMAIN").map_err(|_| "OKTA_DOMAIN not set")?;
let api_token = std::env::var("OKTA_API_TOKEN").map_err(|_| "OKTA_API_TOKEN not set")?;
let username = user
.username
.as_deref()
.or(user.email.as_deref())
.ok_or("No username/email in test user record for Okta lookup")?;
info!(username, "Cleaning up Okta test user");
// Search user
let encoded = urlencoding::encode(username);
let search_url = format!("https://{domain}/api/v1/users?search=profile.login+eq+\"{encoded}\"");
let search_resp = http
.get(&search_url)
.header("Authorization", format!("SSWS {api_token}"))
.send()
.await
.map_err(|e| format!("Okta user search failed: {e}"))?;
let users: Vec<serde_json::Value> = search_resp
.json()
.await
.map_err(|e| format!("Failed to parse Okta users: {e}"))?;
let user_id = users
.first()
.and_then(|u| u.get("id"))
.and_then(|v| v.as_str())
.ok_or_else(|| format!("User '{username}' not found in Okta"))?;
// Deactivate first (required by Okta before delete)
let _ = http
.post(format!(
"https://{domain}/api/v1/users/{user_id}/lifecycle/deactivate"
))
.header("Authorization", format!("SSWS {api_token}"))
.send()
.await;
// Delete
let delete_resp = http
.delete(format!("https://{domain}/api/v1/users/{user_id}"))
.header("Authorization", format!("SSWS {api_token}"))
.send()
.await
.map_err(|e| format!("Okta user delete failed: {e}"))?;
if delete_resp.status().is_success() || delete_resp.status().as_u16() == 204 {
info!(username, user_id, "Okta test user deleted");
Ok(true)
} else {
let status = delete_resp.status();
let body = delete_resp.text().await.unwrap_or_default();
Err(format!("Okta delete failed ({status}): {body}"))
}
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::pentest::{IdentityProvider, TestUserRecord};
use secrecy::SecretString;
fn make_config_no_keycloak() -> AgentConfig {
AgentConfig {
mongodb_uri: String::new(),
mongodb_database: String::new(),
litellm_url: String::new(),
litellm_api_key: SecretString::from(String::new()),
litellm_model: String::new(),
litellm_embed_model: String::new(),
github_token: None,
github_webhook_secret: None,
gitlab_url: None,
gitlab_token: None,
gitlab_webhook_secret: None,
jira_url: None,
jira_email: None,
jira_api_token: None,
jira_project_key: None,
searxng_url: None,
nvd_api_key: None,
agent_port: 3001,
scan_schedule: String::new(),
cve_monitor_schedule: String::new(),
git_clone_base_path: String::new(),
artifact_store_base_path: String::new(),
ssh_key_path: String::new(),
keycloak_url: None,
keycloak_realm: None,
keycloak_admin_username: None,
keycloak_admin_password: None,
pentest_verification_email: None,
pentest_imap_host: None,
pentest_imap_port: None,
pentest_imap_tls: true,
pentest_imap_username: None,
pentest_imap_password: None,
admin_api_token: None,
tenant_registry_url: None,
}
}
#[tokio::test]
async fn already_cleaned_up_returns_false() {
let user = TestUserRecord {
username: Some("test".into()),
email: None,
provider_user_id: None,
provider: Some(IdentityProvider::Keycloak),
cleaned_up: true,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert_eq!(result, Ok(false));
}
#[tokio::test]
async fn firebase_returns_false_not_implemented() {
let user = TestUserRecord {
username: Some("test".into()),
email: None,
provider_user_id: None,
provider: Some(IdentityProvider::Firebase),
cleaned_up: false,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert_eq!(result, Ok(false));
}
#[tokio::test]
async fn no_provider_no_keycloak_skips() {
let user = TestUserRecord {
username: Some("test".into()),
email: None,
provider_user_id: None,
provider: None,
cleaned_up: false,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert_eq!(result, Ok(false));
}
#[tokio::test]
async fn custom_provider_no_keycloak_skips() {
let user = TestUserRecord {
username: Some("test".into()),
email: None,
provider_user_id: None,
provider: Some(IdentityProvider::Custom),
cleaned_up: false,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert_eq!(result, Ok(false));
}
#[tokio::test]
async fn keycloak_missing_config_returns_error() {
let user = TestUserRecord {
username: Some("test".into()),
email: None,
provider_user_id: None,
provider: Some(IdentityProvider::Keycloak),
cleaned_up: false,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert!(result.is_err());
assert!(result
.as_ref()
.err()
.is_some_and(|e| e.contains("KEYCLOAK_URL")));
}
#[tokio::test]
async fn keycloak_missing_username_returns_error() {
let user = TestUserRecord {
username: None,
email: Some("test@example.com".into()),
provider_user_id: None,
provider: Some(IdentityProvider::Keycloak),
cleaned_up: false,
};
let mut config = make_config_no_keycloak();
config.keycloak_url = Some("http://localhost:8080".into());
config.keycloak_realm = Some("test".into());
config.keycloak_admin_username = Some("admin".into());
config.keycloak_admin_password = Some(SecretString::from("pass".to_string()));
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert!(result.is_err());
assert!(result
.as_ref()
.err()
.is_some_and(|e| e.contains("username")));
}
#[tokio::test]
async fn auth0_missing_env_returns_error() {
let user = TestUserRecord {
username: None,
email: Some("test@example.com".into()),
provider_user_id: None,
provider: Some(IdentityProvider::Auth0),
cleaned_up: false,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert!(result.is_err());
assert!(result
.as_ref()
.err()
.is_some_and(|e| e.contains("AUTH0_DOMAIN")));
}
#[tokio::test]
async fn okta_missing_env_returns_error() {
let user = TestUserRecord {
username: Some("test".into()),
email: None,
provider_user_id: None,
provider: Some(IdentityProvider::Okta),
cleaned_up: false,
};
let config = make_config_no_keycloak();
let http = reqwest::Client::new();
let result = cleanup_test_user(&user, &config, &http).await;
assert!(result.is_err());
assert!(result
.as_ref()
.err()
.is_some_and(|e| e.contains("OKTA_DOMAIN")));
}
}
+150
View File
@@ -0,0 +1,150 @@
use futures_util::StreamExt;
use mongodb::bson::doc;
use compliance_core::models::dast::DastTarget;
use compliance_core::models::finding::Finding;
use compliance_core::models::pentest::CodeContextHint;
use compliance_core::models::sbom::SbomEntry;
use super::orchestrator::PentestOrchestrator;
impl PentestOrchestrator {
/// Fetch SAST findings, SBOM entries (with CVEs), and code graph entry points
/// for the repo linked to this DAST target.
pub(crate) async fn gather_repo_context(
&self,
target: &DastTarget,
) -> (Vec<Finding>, Vec<SbomEntry>, Vec<CodeContextHint>) {
let Some(repo_id) = &target.repo_id else {
return (Vec::new(), Vec::new(), Vec::new());
};
let sast_findings = self.fetch_sast_findings(repo_id).await;
let sbom_entries = self.fetch_vulnerable_sbom(repo_id).await;
let code_context = self.fetch_code_context(repo_id, &sast_findings).await;
tracing::info!(
repo_id,
sast_findings = sast_findings.len(),
vulnerable_deps = sbom_entries.len(),
code_hints = code_context.len(),
"Gathered code-awareness context for pentest"
);
(sast_findings, sbom_entries, code_context)
}
/// Fetch open/triaged SAST findings for the repo (not false positives or resolved)
async fn fetch_sast_findings(&self, repo_id: &str) -> Vec<Finding> {
let cursor = self
.db
.findings()
.find(doc! {
"repo_id": repo_id,
"status": { "$in": ["open", "triaged"] },
})
.sort(doc! { "severity": -1 })
.limit(100)
.await;
match cursor {
Ok(mut c) => {
let mut results = Vec::new();
while let Some(Ok(f)) = c.next().await {
results.push(f);
}
results
}
Err(e) => {
tracing::warn!("Failed to fetch SAST findings for pentest: {e}");
Vec::new()
}
}
}
/// Fetch SBOM entries that have known vulnerabilities
async fn fetch_vulnerable_sbom(&self, repo_id: &str) -> Vec<SbomEntry> {
let cursor = self
.db
.sbom_entries()
.find(doc! {
"repo_id": repo_id,
"known_vulnerabilities": { "$exists": true, "$ne": [] },
})
.limit(50)
.await;
match cursor {
Ok(mut c) => {
let mut results = Vec::new();
while let Some(Ok(e)) = c.next().await {
results.push(e);
}
results
}
Err(e) => {
tracing::warn!("Failed to fetch vulnerable SBOM entries: {e}");
Vec::new()
}
}
}
/// Build CodeContextHint objects from the code knowledge graph.
/// Maps entry points to their source files and links SAST findings.
async fn fetch_code_context(
&self,
repo_id: &str,
sast_findings: &[Finding],
) -> Vec<CodeContextHint> {
// Get entry point nodes from the code graph
let cursor = self
.db
.graph_nodes()
.find(doc! {
"repo_id": repo_id,
"is_entry_point": true,
})
.limit(50)
.await;
let nodes = match cursor {
Ok(mut c) => {
let mut results = Vec::new();
while let Some(Ok(n)) = c.next().await {
results.push(n);
}
results
}
Err(_) => return Vec::new(),
};
// Build hints by matching graph nodes to SAST findings by file path
nodes
.into_iter()
.map(|node| {
// Find SAST findings in the same file
let linked_vulns: Vec<String> = sast_findings
.iter()
.filter(|f| f.file_path.as_deref() == Some(&node.file_path))
.map(|f| {
format!(
"[{}] {}: {} (line {})",
f.severity,
f.scanner,
f.title,
f.line_number.unwrap_or(0)
)
})
.collect();
CodeContextHint {
endpoint_pattern: node.qualified_name.clone(),
handler_function: node.name.clone(),
file_path: node.file_path.clone(),
code_snippet: String::new(), // Could fetch from embeddings
known_vulnerabilities: linked_vulns,
}
})
.collect()
}
}
+117
View File
@@ -0,0 +1,117 @@
use aes_gcm::aead::AeadCore;
use aes_gcm::{
aead::{Aead, KeyInit, OsRng},
Aes256Gcm, Nonce,
};
use base64::Engine;
/// Load the 32-byte encryption key from PENTEST_ENCRYPTION_KEY env var.
/// Returns None if not set or invalid length.
pub fn load_encryption_key() -> Option<[u8; 32]> {
let hex_key = std::env::var("PENTEST_ENCRYPTION_KEY").ok()?;
let bytes = hex::decode(hex_key).ok()?;
if bytes.len() != 32 {
return None;
}
let mut key = [0u8; 32];
key.copy_from_slice(&bytes);
Some(key)
}
/// Encrypt a plaintext string. Returns base64-encoded nonce+ciphertext.
/// Returns the original string if no encryption key is available.
pub fn encrypt(plaintext: &str) -> String {
let Some(key_bytes) = load_encryption_key() else {
return plaintext.to_string();
};
let Ok(cipher) = Aes256Gcm::new_from_slice(&key_bytes) else {
return plaintext.to_string();
};
let nonce = Aes256Gcm::generate_nonce(&mut OsRng);
let Ok(ciphertext) = cipher.encrypt(&nonce, plaintext.as_bytes()) else {
return plaintext.to_string();
};
let mut combined = nonce.to_vec();
combined.extend_from_slice(&ciphertext);
base64::engine::general_purpose::STANDARD.encode(&combined)
}
/// Decrypt a base64-encoded nonce+ciphertext string.
/// Returns None if decryption fails.
pub fn decrypt(encrypted: &str) -> Option<String> {
let key_bytes = load_encryption_key()?;
let cipher = Aes256Gcm::new_from_slice(&key_bytes).ok()?;
let combined = base64::engine::general_purpose::STANDARD
.decode(encrypted)
.ok()?;
if combined.len() < 12 {
return None;
}
let (nonce_bytes, ciphertext) = combined.split_at(12);
let nonce = Nonce::from_slice(nonce_bytes);
let plaintext = cipher.decrypt(nonce, ciphertext).ok()?;
String::from_utf8(plaintext).ok()
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
// Guard to serialize tests that touch env vars
static ENV_LOCK: Mutex<()> = Mutex::new(());
fn with_key<F: FnOnce()>(hex_key: &str, f: F) {
let _guard = ENV_LOCK.lock();
unsafe { std::env::set_var("PENTEST_ENCRYPTION_KEY", hex_key) };
f();
unsafe { std::env::remove_var("PENTEST_ENCRYPTION_KEY") };
}
#[test]
fn round_trip() {
let key = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
with_key(key, || {
let plaintext = "my_secret_password";
let encrypted = encrypt(plaintext);
assert_ne!(encrypted, plaintext);
let decrypted = decrypt(&encrypted);
assert_eq!(decrypted, Some(plaintext.to_string()));
});
}
#[test]
fn wrong_key_fails() {
let _guard = ENV_LOCK.lock();
let key1 = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
let key2 = "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789";
let encrypted = {
unsafe { std::env::set_var("PENTEST_ENCRYPTION_KEY", key1) };
let e = encrypt("secret");
unsafe { std::env::remove_var("PENTEST_ENCRYPTION_KEY") };
e
};
unsafe { std::env::set_var("PENTEST_ENCRYPTION_KEY", key2) };
assert!(decrypt(&encrypted).is_none());
unsafe { std::env::remove_var("PENTEST_ENCRYPTION_KEY") };
}
#[test]
fn no_key_passthrough() {
let _guard = ENV_LOCK.lock();
unsafe { std::env::remove_var("PENTEST_ENCRYPTION_KEY") };
let result = encrypt("plain");
assert_eq!(result, "plain");
}
#[test]
fn corrupted_ciphertext() {
let key = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
with_key(key, || {
assert!(decrypt("not-valid-base64!!!").is_none());
// Valid base64 but wrong content
let garbage = base64::engine::general_purpose::STANDARD.encode(b"tooshort");
assert!(decrypt(&garbage).is_none());
});
}
}
+9
View File
@@ -0,0 +1,9 @@
pub mod cleanup;
mod context;
pub mod crypto;
pub mod orchestrator;
mod prompt_builder;
pub mod report;
pub use orchestrator::PentestOrchestrator;
pub use report::generate_encrypted_report;
@@ -0,0 +1,735 @@
use std::sync::Arc;
use std::time::Duration;
use mongodb::bson::doc;
use tokio::sync::{broadcast, watch};
use compliance_core::models::dast::DastTarget;
use compliance_core::models::pentest::*;
use compliance_core::traits::pentest_tool::PentestToolContext;
use compliance_dast::ToolRegistry;
use crate::database::Database;
use crate::llm::{
ChatMessage, LlmClient, LlmResponse, ToolCallRequest, ToolCallRequestFunction, ToolDefinition,
};
/// Maximum duration for a single pentest session before timeout
const SESSION_TIMEOUT: Duration = Duration::from_secs(30 * 60); // 30 minutes
pub struct PentestOrchestrator {
pub(crate) tool_registry: ToolRegistry,
pub(crate) llm: Arc<LlmClient>,
pub(crate) db: Database,
pub(crate) event_tx: broadcast::Sender<PentestEvent>,
pub(crate) pause_rx: Option<watch::Receiver<bool>>,
}
impl PentestOrchestrator {
/// Create a new orchestrator with an externally-provided broadcast sender
/// and an optional pause receiver.
pub fn new(
llm: Arc<LlmClient>,
db: Database,
event_tx: broadcast::Sender<PentestEvent>,
pause_rx: Option<watch::Receiver<bool>>,
) -> Self {
Self {
tool_registry: ToolRegistry::new(),
llm,
db,
event_tx,
pause_rx,
}
}
/// Run a pentest session with timeout and automatic failure marking on errors.
pub async fn run_session_guarded(
&self,
session: &PentestSession,
target: &DastTarget,
initial_message: &str,
) {
let session_id = session.id;
// Use config-specified timeout or default
let timeout_duration = session
.config
.as_ref()
.and_then(|c| c.max_duration_minutes)
.map(|m| Duration::from_secs(m as u64 * 60))
.unwrap_or(SESSION_TIMEOUT);
let timeout_minutes = timeout_duration.as_secs() / 60;
match tokio::time::timeout(
timeout_duration,
self.run_session(session, target, initial_message),
)
.await
{
Ok(Ok(())) => {
tracing::info!(?session_id, "Pentest session completed successfully");
}
Ok(Err(e)) => {
tracing::error!(?session_id, error = %e, "Pentest session failed");
self.mark_session_failed(session_id, &format!("Error: {e}"))
.await;
let _ = self.event_tx.send(PentestEvent::Error {
message: format!("Session failed: {e}"),
});
}
Err(_) => {
let msg = format!("Session timed out after {timeout_minutes} minutes");
tracing::warn!(?session_id, "{msg}");
self.mark_session_failed(session_id, &msg).await;
let _ = self.event_tx.send(PentestEvent::Error { message: msg });
}
}
}
async fn mark_session_failed(
&self,
session_id: Option<mongodb::bson::oid::ObjectId>,
reason: &str,
) {
if let Some(sid) = session_id {
let _ = self
.db
.pentest_sessions()
.update_one(
doc! { "_id": sid },
doc! { "$set": {
"status": "failed",
"completed_at": mongodb::bson::DateTime::now(),
"error_message": reason,
}},
)
.await;
}
}
/// Check if the session is paused; if so, update DB status and wait until resumed.
async fn wait_if_paused(&self, session: &PentestSession) {
let Some(ref pause_rx) = self.pause_rx else {
return;
};
let mut rx = pause_rx.clone();
if !*rx.borrow() {
return;
}
// We are paused — update DB status
if let Some(sid) = session.id {
let _ = self
.db
.pentest_sessions()
.update_one(doc! { "_id": sid }, doc! { "$set": { "status": "paused" }})
.await;
}
let _ = self.event_tx.send(PentestEvent::Paused);
// Wait until unpaused
while *rx.borrow() {
if rx.changed().await.is_err() {
break;
}
}
// Resumed — update DB status back to running
if let Some(sid) = session.id {
let _ = self
.db
.pentest_sessions()
.update_one(doc! { "_id": sid }, doc! { "$set": { "status": "running" }})
.await;
}
let _ = self.event_tx.send(PentestEvent::Resumed);
}
async fn run_session(
&self,
session: &PentestSession,
target: &DastTarget,
initial_message: &str,
) -> Result<(), crate::error::AgentError> {
let session_id = session.id.map(|oid| oid.to_hex()).unwrap_or_default();
// Gather code-awareness context from linked repo
let (sast_findings, sbom_entries, code_context) = self.gather_repo_context(target).await;
// Build system prompt with code context
let system_prompt = self
.build_system_prompt(
session,
target,
&sast_findings,
&sbom_entries,
&code_context,
)
.await;
// Build tool definitions for LLM
let tool_defs: Vec<ToolDefinition> = self
.tool_registry
.all_definitions()
.into_iter()
.map(|td| ToolDefinition {
name: td.name,
description: td.description,
parameters: td.input_schema,
})
.collect();
// Initialize messages
let mut messages = vec![
ChatMessage {
role: "system".to_string(),
content: Some(system_prompt),
tool_calls: None,
tool_call_id: None,
},
ChatMessage {
role: "user".to_string(),
content: Some(initial_message.to_string()),
tool_calls: None,
tool_call_id: None,
},
];
// Store user message
let user_msg = PentestMessage::user(session_id.clone(), initial_message.to_string());
let _ = self.db.pentest_messages().insert_one(&user_msg).await;
// Build tool context with real data
let tool_context = PentestToolContext {
target: target.clone(),
session_id: session_id.clone(),
sast_findings,
sbom_entries,
code_context,
rate_limit: target.rate_limit,
allow_destructive: target.allow_destructive,
};
let max_iterations = 50;
let mut total_findings = 0u32;
let mut total_tool_calls = 0u32;
let mut total_successes = 0u32;
let mut prev_node_ids: Vec<String> = Vec::new();
for _iteration in 0..max_iterations {
// Check pause state at top of each iteration
self.wait_if_paused(session).await;
let response = self
.llm
.chat_with_tools(messages.clone(), &tool_defs, Some(0.2), Some(8192))
.await?;
match response {
LlmResponse::Content(content) => {
let msg = PentestMessage::assistant(session_id.clone(), content.clone());
let _ = self.db.pentest_messages().insert_one(&msg).await;
let _ = self.event_tx.send(PentestEvent::Message {
content: content.clone(),
});
messages.push(ChatMessage {
role: "assistant".to_string(),
content: Some(content.clone()),
tool_calls: None,
tool_call_id: None,
});
let done_indicators = [
"pentest complete",
"testing complete",
"scan complete",
"analysis complete",
"finished",
"that concludes",
];
let content_lower = content.to_lowercase();
if done_indicators
.iter()
.any(|ind| content_lower.contains(ind))
{
break;
}
break;
}
LlmResponse::ToolCalls {
calls: tool_calls,
reasoning,
} => {
let tc_requests: Vec<ToolCallRequest> = tool_calls
.iter()
.map(|tc| ToolCallRequest {
id: tc.id.clone(),
r#type: "function".to_string(),
function: ToolCallRequestFunction {
name: tc.name.clone(),
arguments: serde_json::to_string(&tc.arguments).unwrap_or_default(),
},
})
.collect();
messages.push(ChatMessage {
role: "assistant".to_string(),
content: if reasoning.is_empty() {
None
} else {
Some(reasoning.clone())
},
tool_calls: Some(tc_requests),
tool_call_id: None,
});
let mut current_batch_node_ids: Vec<String> = Vec::new();
for tc in &tool_calls {
total_tool_calls += 1;
let node_id = uuid::Uuid::new_v4().to_string();
let mut node = AttackChainNode::new(
session_id.clone(),
node_id.clone(),
tc.name.clone(),
tc.arguments.clone(),
reasoning.clone(),
);
// Link to previous iteration's nodes
node.parent_node_ids = prev_node_ids.clone();
node.status = AttackNodeStatus::Running;
node.started_at = Some(chrono::Utc::now());
let _ = self.db.attack_chain_nodes().insert_one(&node).await;
current_batch_node_ids.push(node_id.clone());
let _ = self.event_tx.send(PentestEvent::ToolStart {
node_id: node_id.clone(),
tool_name: tc.name.clone(),
input: tc.arguments.clone(),
});
let result = if let Some(tool) = self.tool_registry.get(&tc.name) {
match tool.execute(tc.arguments.clone(), &tool_context).await {
Ok(result) => {
total_successes += 1;
let findings_count = result.findings.len() as u32;
total_findings += findings_count;
let mut finding_ids: Vec<String> = Vec::new();
// Dedup findings within this tool result before inserting
let deduped_findings =
crate::pipeline::dedup::dedup_dast_findings(
result.findings,
);
for mut finding in deduped_findings {
finding.scan_run_id = session_id.clone();
finding.session_id = Some(session_id.clone());
// Check for existing duplicate in this session
let fp = crate::pipeline::dedup::compute_dast_fingerprint(
&finding,
);
let existing = self
.db
.dast_findings()
.find_one(doc! {
"session_id": &session_id,
"title": &finding.title,
"endpoint": &finding.endpoint,
"method": &finding.method,
})
.await;
if matches!(existing, Ok(Some(_))) {
tracing::debug!(
"Skipping duplicate DAST finding: {} (fp={:.12})",
finding.title,
fp,
);
continue;
}
let insert_result =
self.db.dast_findings().insert_one(&finding).await;
if let Ok(res) = &insert_result {
finding_ids.push(
res.inserted_id
.as_object_id()
.map(|oid| oid.to_hex())
.unwrap_or_default(),
);
}
let _ = self.event_tx.send(PentestEvent::Finding {
finding_id: finding
.id
.map(|oid| oid.to_hex())
.unwrap_or_default(),
title: finding.title.clone(),
severity: finding.severity.to_string(),
});
}
// Compute risk score based on findings severity
let risk_score: Option<u8> = if findings_count > 0 {
Some(std::cmp::min(
100,
(findings_count as u8)
.saturating_mul(15)
.saturating_add(20),
))
} else {
None
};
let _ = self.event_tx.send(PentestEvent::ToolComplete {
node_id: node_id.clone(),
summary: result.summary.clone(),
findings_count,
});
let finding_ids_bson: Vec<mongodb::bson::Bson> = finding_ids
.iter()
.map(|id| mongodb::bson::Bson::String(id.clone()))
.collect();
let mut update_doc = doc! {
"status": "completed",
"tool_output": mongodb::bson::to_bson(&result.data)
.unwrap_or(mongodb::bson::Bson::Null),
"completed_at": mongodb::bson::DateTime::now(),
"findings_produced": finding_ids_bson,
};
if let Some(rs) = risk_score {
update_doc.insert("risk_score", rs as i32);
}
let _ = self
.db
.attack_chain_nodes()
.update_one(
doc! {
"session_id": &session_id,
"node_id": &node_id,
},
doc! { "$set": update_doc },
)
.await;
// Build LLM-facing summary: strip large fields
// (screenshots, raw HTML) to save context window
let llm_data = summarize_tool_output(&result.data);
serde_json::json!({
"summary": result.summary,
"findings_count": findings_count,
"data": llm_data,
})
.to_string()
}
Err(e) => {
let _ = self
.db
.attack_chain_nodes()
.update_one(
doc! {
"session_id": &session_id,
"node_id": &node_id,
},
doc! { "$set": {
"status": "failed",
"completed_at": mongodb::bson::DateTime::now(),
}},
)
.await;
format!("Tool execution failed: {e}")
}
}
} else {
format!("Unknown tool: {}", tc.name)
};
messages.push(ChatMessage {
role: "tool".to_string(),
content: Some(result),
tool_calls: None,
tool_call_id: Some(tc.id.clone()),
});
}
// Advance parent links so next iteration's nodes connect to this batch
prev_node_ids = current_batch_node_ids;
if let Some(sid) = session.id {
let _ = self
.db
.pentest_sessions()
.update_one(
doc! { "_id": sid },
doc! { "$set": {
"tool_invocations": total_tool_calls as i64,
"tool_successes": total_successes as i64,
"findings_count": total_findings as i64,
}},
)
.await;
}
}
}
}
if let Some(sid) = session.id {
let _ = self
.db
.pentest_sessions()
.update_one(
doc! { "_id": sid },
doc! { "$set": {
"status": "completed",
"completed_at": mongodb::bson::DateTime::now(),
"tool_invocations": total_tool_calls as i64,
"tool_successes": total_successes as i64,
"findings_count": total_findings as i64,
}},
)
.await;
}
// Clean up test user via identity provider API if requested
if session
.config
.as_ref()
.is_some_and(|c| c.auth.cleanup_test_user)
{
if let Some(ref test_user) = session.test_user {
let http = reqwest::Client::new();
// We need the AgentConfig — read from env since orchestrator doesn't hold it
let config = crate::config::load_config();
match config {
Ok(cfg) => {
match crate::pentest::cleanup::cleanup_test_user(test_user, &cfg, &http)
.await
{
Ok(true) => {
tracing::info!(
username = test_user.username.as_deref(),
"Test user cleaned up via provider API"
);
// Mark as cleaned up in DB
if let Some(sid) = session.id {
let _ = self
.db
.pentest_sessions()
.update_one(
doc! { "_id": sid },
doc! { "$set": { "test_user.cleaned_up": true } },
)
.await;
}
}
Ok(false) => {
tracing::info!(
"Test user cleanup skipped (no provider configured)"
);
}
Err(e) => {
tracing::warn!(error = %e, "Test user cleanup failed");
let _ = self.event_tx.send(PentestEvent::Error {
message: format!("Test user cleanup failed: {e}"),
});
}
}
}
Err(e) => {
tracing::warn!(error = %e, "Could not load config for cleanup");
}
}
}
}
// Clean up the persistent browser session for this pentest
compliance_dast::tools::browser::cleanup_browser_session(&session_id).await;
let _ = self.event_tx.send(PentestEvent::Complete {
summary: format!(
"Pentest complete. {} findings from {} tool invocations.",
total_findings, total_tool_calls
),
});
Ok(())
}
}
/// Strip large fields from tool output before sending to the LLM.
/// Screenshots, raw HTML, and other bulky data are replaced with short summaries.
/// The full data is still stored in the DB for the report.
fn summarize_tool_output(data: &serde_json::Value) -> serde_json::Value {
let Some(obj) = data.as_object() else {
return data.clone();
};
let mut summarized = serde_json::Map::new();
for (key, value) in obj {
match key.as_str() {
// Replace screenshot base64 with a placeholder
"screenshot_base64" => {
if let Some(s) = value.as_str() {
if !s.is_empty() {
summarized.insert(
key.clone(),
serde_json::Value::String(
"[screenshot captured and saved to report]".to_string(),
),
);
continue;
}
}
summarized.insert(key.clone(), value.clone());
}
// Truncate raw HTML content
"html" => {
if let Some(s) = value.as_str() {
if s.len() > 2000 {
summarized.insert(
key.clone(),
serde_json::Value::String(format!(
"{}... [truncated, {} chars total]",
&s[..2000],
s.len()
)),
);
continue;
}
}
summarized.insert(key.clone(), value.clone());
}
// Truncate page text
"text" if value.as_str().is_some_and(|s| s.len() > 1500) => {
let s = value.as_str().unwrap_or_default();
summarized.insert(
key.clone(),
serde_json::Value::String(format!("{}... [truncated]", &s[..1500])),
);
}
// Trim large arrays (e.g., "elements", "links", "inputs")
"elements" | "links" | "inputs" => {
if let Some(arr) = value.as_array() {
if arr.len() > 15 {
let mut trimmed: Vec<serde_json::Value> = arr[..15].to_vec();
trimmed.push(serde_json::json!(format!(
"... and {} more",
arr.len() - 15
)));
summarized.insert(key.clone(), serde_json::Value::Array(trimmed));
continue;
}
}
summarized.insert(key.clone(), value.clone());
}
// Recursively summarize nested objects (e.g., "page" in get_content)
_ if value.is_object() => {
summarized.insert(key.clone(), summarize_tool_output(value));
}
// Keep everything else as-is
_ => {
summarized.insert(key.clone(), value.clone());
}
}
}
serde_json::Value::Object(summarized)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn test_summarize_strips_screenshot() {
let input = json!({
"screenshot_base64": "iVBOR...",
"url": "https://example.com"
});
let result = summarize_tool_output(&input);
assert_eq!(
result["screenshot_base64"],
"[screenshot captured and saved to report]"
);
assert_eq!(result["url"], "https://example.com");
}
#[test]
fn test_summarize_truncates_html() {
let long_html = "x".repeat(3000);
let input = json!({ "html": long_html });
let result = summarize_tool_output(&input);
let s = result["html"].as_str().unwrap_or_default();
assert!(s.contains("[truncated, 3000 chars total]"));
assert!(s.starts_with(&"x".repeat(2000)));
assert!(s.len() < 3000);
}
#[test]
fn test_summarize_truncates_text() {
let long_text = "a".repeat(2000);
let input = json!({ "text": long_text });
let result = summarize_tool_output(&input);
let s = result["text"].as_str().unwrap_or_default();
assert!(s.contains("[truncated]"));
assert!(s.starts_with(&"a".repeat(1500)));
assert!(s.len() < 2000);
}
#[test]
fn test_summarize_trims_large_arrays() {
let elements: Vec<serde_json::Value> = (0..20).map(|i| json!(format!("el-{i}"))).collect();
let input = json!({ "elements": elements });
let result = summarize_tool_output(&input);
let arr = result["elements"].as_array();
assert!(arr.is_some());
if let Some(arr) = arr {
// 15 kept + 1 summary entry
assert_eq!(arr.len(), 16);
assert_eq!(arr[15], json!("... and 5 more"));
}
}
#[test]
fn test_summarize_preserves_small_data() {
let input = json!({
"url": "https://example.com",
"status": 200,
"title": "Example"
});
let result = summarize_tool_output(&input);
assert_eq!(result, input);
}
#[test]
fn test_summarize_recursive() {
let input = json!({
"page": {
"screenshot_base64": "iVBORw0KGgoAAAA...",
"url": "https://example.com"
}
});
let result = summarize_tool_output(&input);
assert_eq!(
result["page"]["screenshot_base64"],
"[screenshot captured and saved to report]"
);
assert_eq!(result["page"]["url"], "https://example.com");
}
#[test]
fn test_summarize_non_object() {
let string_val = json!("just a string");
assert_eq!(summarize_tool_output(&string_val), string_val);
let num_val = json!(42);
assert_eq!(summarize_tool_output(&num_val), num_val);
}
}
@@ -0,0 +1,637 @@
use compliance_core::models::dast::DastTarget;
use compliance_core::models::finding::{Finding, FindingStatus, Severity};
use compliance_core::models::pentest::*;
use compliance_core::models::sbom::SbomEntry;
use super::orchestrator::PentestOrchestrator;
/// Attempt to decrypt a field; if decryption fails, return the original value
/// (which may be plaintext from before encryption was enabled).
fn decrypt_field(value: &str) -> String {
super::crypto::decrypt(value).unwrap_or_else(|| value.to_string())
}
/// Build additional prompt sections from PentestConfig when present.
fn build_config_sections(config: &PentestConfig) -> String {
let mut sections = String::new();
// Authentication section
match config.auth.mode {
AuthMode::Manual => {
sections.push_str("\n## Authentication\n");
sections.push_str("- **Mode**: Manual credentials\n");
if let Some(ref u) = config.auth.username {
let decrypted = decrypt_field(u);
sections.push_str(&format!("- **Username**: {decrypted}\n"));
}
if let Some(ref p) = config.auth.password {
let decrypted = decrypt_field(p);
sections.push_str(&format!("- **Password**: {decrypted}\n"));
}
sections.push_str(
"Use these credentials to log in before testing authenticated endpoints.\n",
);
}
AuthMode::AutoRegister => {
sections.push_str("\n## Authentication\n");
sections.push_str("- **Mode**: Auto-register\n");
if let Some(ref url) = config.auth.registration_url {
sections.push_str(&format!("- **Registration URL**: {url}\n"));
} else {
sections.push_str(
"- **Registration URL**: Not provided — use Playwright to discover the registration page.\n",
);
}
if let Some(ref email) = config.auth.verification_email {
sections.push_str(&format!(
"- **Verification Email**: Use plus-addressing from `{email}` \
(e.g. `{base}+{{session_id}}@{domain}`) for email verification. \
The system will poll the IMAP mailbox for verification links.\n",
base = email.split('@').next().unwrap_or(email),
domain = email.split('@').nth(1).unwrap_or("example.com"),
));
}
sections.push_str(
"Register a new test account using the registration page, then use it for testing.\n",
);
}
AuthMode::None => {}
}
// Custom headers
if !config.custom_headers.is_empty() {
sections.push_str("\n## Custom HTTP Headers\n");
sections.push_str("Include these headers in all HTTP requests:\n");
for (k, v) in &config.custom_headers {
sections.push_str(&format!("- `{k}: {v}`\n"));
}
}
// Scope exclusions
if !config.scope_exclusions.is_empty() {
sections.push_str("\n## Scope Exclusions\n");
sections.push_str("Do NOT test the following paths:\n");
for path in &config.scope_exclusions {
sections.push_str(&format!("- `{path}`\n"));
}
}
// Git context
if config.git_repo_url.is_some() || config.branch.is_some() || config.commit_hash.is_some() {
sections.push_str("\n## Git Context\n");
if let Some(ref url) = config.git_repo_url {
sections.push_str(&format!("- **Repository**: {url}\n"));
}
if let Some(ref branch) = config.branch {
sections.push_str(&format!("- **Branch**: {branch}\n"));
}
if let Some(ref commit) = config.commit_hash {
sections.push_str(&format!("- **Commit**: {commit}\n"));
}
}
// Environment
sections.push_str(&format!(
"\n## Environment\n- **Target environment**: {}\n",
config.environment
));
sections
}
/// Return strategy guidance text for the given strategy.
fn strategy_guidance(strategy: &PentestStrategy) -> &'static str {
match strategy {
PentestStrategy::Quick => {
"Focus on the most common and impactful vulnerabilities. Run a quick recon, then target the highest-risk areas."
}
PentestStrategy::Comprehensive => {
"Perform a thorough assessment covering all vulnerability types. Start with recon, then systematically test each attack surface."
}
PentestStrategy::Targeted => {
"Focus specifically on areas highlighted by SAST findings and known CVEs. Prioritize exploiting known weaknesses."
}
PentestStrategy::Aggressive => {
"Use all available tools aggressively. Test with maximum payloads and attempt full exploitation."
}
PentestStrategy::Stealth => {
"Minimize noise. Use fewer requests, avoid aggressive payloads. Focus on passive analysis and targeted probes."
}
}
}
/// Build the SAST findings section for the system prompt.
fn build_sast_section(sast_findings: &[Finding]) -> String {
if sast_findings.is_empty() {
return String::from("No SAST findings available for this target.");
}
let critical = sast_findings
.iter()
.filter(|f| f.severity == Severity::Critical)
.count();
let high = sast_findings
.iter()
.filter(|f| f.severity == Severity::High)
.count();
let mut section = format!(
"{} open findings ({} critical, {} high):\n",
sast_findings.len(),
critical,
high
);
// List the most important findings (critical/high first, up to 20)
for f in sast_findings.iter().take(20) {
let file_info = f
.file_path
.as_ref()
.map(|p| format!(" in {}:{}", p, f.line_number.unwrap_or(0)))
.unwrap_or_default();
let status_note = match f.status {
FindingStatus::Triaged => " [TRIAGED]",
_ => "",
};
section.push_str(&format!(
"- [{sev}] {title}{file}{status}\n",
sev = f.severity,
title = f.title,
file = file_info,
status = status_note,
));
if let Some(cwe) = &f.cwe {
section.push_str(&format!(" CWE: {cwe}\n"));
}
}
if sast_findings.len() > 20 {
section.push_str(&format!(
"... and {} more findings\n",
sast_findings.len() - 20
));
}
section
}
/// Build the SBOM/CVE section for the system prompt.
fn build_sbom_section(sbom_entries: &[SbomEntry]) -> String {
if sbom_entries.is_empty() {
return String::from("No vulnerable dependencies identified.");
}
let mut section = format!(
"{} dependencies with known vulnerabilities:\n",
sbom_entries.len()
);
for entry in sbom_entries.iter().take(15) {
let cve_ids: Vec<&str> = entry
.known_vulnerabilities
.iter()
.map(|v| v.id.as_str())
.collect();
section.push_str(&format!(
"- {} {} ({}): {}\n",
entry.name,
entry.version,
entry.package_manager,
cve_ids.join(", ")
));
}
if sbom_entries.len() > 15 {
section.push_str(&format!(
"... and {} more vulnerable dependencies\n",
sbom_entries.len() - 15
));
}
section
}
/// Build the code context section for the system prompt.
fn build_code_section(code_context: &[CodeContextHint]) -> String {
if code_context.is_empty() {
return String::from("No code knowledge graph available for this target.");
}
let with_vulns = code_context
.iter()
.filter(|c| !c.known_vulnerabilities.is_empty())
.count();
let mut section = format!(
"{} entry points identified ({} with linked SAST findings):\n",
code_context.len(),
with_vulns
);
for hint in code_context.iter().take(20) {
section.push_str(&format!(
"- {} ({})\n",
hint.endpoint_pattern, hint.file_path
));
for vuln in &hint.known_vulnerabilities {
section.push_str(&format!(" SAST: {vuln}\n"));
}
}
section
}
impl PentestOrchestrator {
pub(crate) async fn build_system_prompt(
&self,
session: &PentestSession,
target: &DastTarget,
sast_findings: &[Finding],
sbom_entries: &[SbomEntry],
code_context: &[CodeContextHint],
) -> String {
let tool_names = self.tool_registry.list_names().join(", ");
let guidance = strategy_guidance(&session.strategy);
let sast_section = build_sast_section(sast_findings);
let sbom_section = build_sbom_section(sbom_entries);
let code_section = build_code_section(code_context);
let config_sections = session
.config
.as_ref()
.map(build_config_sections)
.unwrap_or_default();
format!(
r#"You are an expert penetration tester conducting an authorized security assessment.
## Target
- **Name**: {target_name}
- **URL**: {base_url}
- **Type**: {target_type}
- **Rate Limit**: {rate_limit} req/s
- **Destructive Tests Allowed**: {allow_destructive}
- **Linked Repository**: {repo_linked}
## Strategy
{strategy_guidance}
## SAST Findings (Static Analysis)
{sast_section}
## Vulnerable Dependencies (SBOM)
{sbom_section}
## Code Entry Points (Knowledge Graph)
{code_section}
{config_sections}
## Available Tools
{tool_names}
## Instructions
1. Start by running reconnaissance (recon tool) to fingerprint the target and discover technologies.
2. Run the OpenAPI parser to discover API endpoints from specs.
3. Check infrastructure: DNS, DMARC, TLS, security headers, cookies, CSP, CORS.
4. If the target requires authentication (auto-register mode), use the browser tool to:
a. Navigate to the target — it will redirect to the login page.
b. Click the "Register" link to reach the registration form.
c. Fill all form fields (username, email with plus-addressing, password, name) one by one.
d. Click submit. If a Terms & Conditions page appears, accept it.
e. After registration, use the browser to navigate through the application pages.
f. **Take a screenshot after each major page** for evidence in the report.
5. Use the browser tool to explore the authenticated application — navigate to each section,
use get_content to understand the page structure, and take screenshots.
6. Based on SAST findings, prioritize testing endpoints where vulnerabilities were found in code.
7. For each vulnerability type found in SAST, use the corresponding DAST tool to verify exploitability.
8. If vulnerable dependencies are listed, try to trigger known CVE conditions against the running application.
9. Test rate limiting on critical endpoints (login, API).
10. Check for console.log leakage in frontend JavaScript.
11. Analyze tool results and chain findings — if one vulnerability enables others, explore the chain.
12. When testing is complete, provide a structured summary with severity and remediation.
13. Always explain your reasoning before invoking each tool.
14. When done, say "Testing complete" followed by a final summary.
## Browser Tool Usage
- The browser tab **persists** between calls — cookies and login state are preserved.
- After navigate, the response includes `elements` (links, inputs, buttons on the page).
- Use `get_content` to see forms, links, buttons, headings, and page text.
- Use `click` with CSS selectors to interact (e.g., `a:text('Register')`, `input[type='submit']`).
- Use `fill` with selector + value to fill form fields (e.g., `input[name='email']`).
- **Take screenshots** (`action: screenshot`) after important actions for evidence.
- For SPA apps: a 200 HTTP status does NOT mean the page is accessible — check the actual
page content with the browser tool to verify if it shows real data or a login redirect.
## Finding Quality Rules
- **Do not report the same issue twice.** If multiple tools detect the same missing header or
vulnerability on the same endpoint, report it ONCE with the most specific tool's output.
For example, if the recon tool and the header scanner both find missing HSTS, report it only
from the header scanner (more specific).
- **Group related findings.** Missing security headers on the same endpoint are ONE finding
("Missing security headers") listing all missing headers, not separate findings per header.
- **Severity must match real impact:**
- critical/high: Exploitable vulnerability (you can demonstrate the exploit)
- medium: Real misconfiguration with security implications but not directly exploitable
- low: Best-practice recommendation, defense-in-depth, or informational
- **Missing headers are medium at most** unless you can demonstrate a concrete exploit enabled
by the missing header (e.g., missing CSP + confirmed XSS = high for CSP finding).
- Console.log in third-party/vendored JS (node_modules, minified libraries) is informational only.
## Important
- This is an authorized penetration test. All testing is permitted within the target scope.
- Respect the rate limit of {rate_limit} requests per second.
- Only use destructive tests if explicitly allowed ({allow_destructive}).
- Use SAST findings to guide your testing — they tell you WHERE in the code vulnerabilities exist.
- Use SBOM data to understand what technologies and versions the target runs.
"#,
target_name = target.name,
base_url = target.base_url,
target_type = target.target_type,
rate_limit = target.rate_limit,
allow_destructive = target.allow_destructive,
repo_linked = target.repo_id.as_deref().unwrap_or("None"),
strategy_guidance = guidance,
)
}
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::finding::Severity;
use compliance_core::models::sbom::VulnRef;
use compliance_core::models::scan::ScanType;
fn make_finding(
severity: Severity,
title: &str,
file_path: Option<&str>,
line: Option<u32>,
status: FindingStatus,
cwe: Option<&str>,
) -> Finding {
let mut f = Finding::new(
"repo-1".into(),
format!("fp-{title}"),
"semgrep".into(),
ScanType::Sast,
title.into(),
"desc".into(),
severity,
);
f.file_path = file_path.map(|s| s.to_string());
f.line_number = line;
f.status = status;
f.cwe = cwe.map(|s| s.to_string());
f
}
fn make_sbom_entry(name: &str, version: &str, cves: &[&str]) -> SbomEntry {
let mut entry = SbomEntry::new("repo-1".into(), name.into(), version.into(), "npm".into());
entry.known_vulnerabilities = cves
.iter()
.map(|id| VulnRef {
id: id.to_string(),
source: "nvd".into(),
severity: None,
url: None,
})
.collect();
entry
}
fn make_code_hint(endpoint: &str, file: &str, vulns: Vec<String>) -> CodeContextHint {
CodeContextHint {
endpoint_pattern: endpoint.into(),
handler_function: "handler".into(),
file_path: file.into(),
code_snippet: String::new(),
known_vulnerabilities: vulns,
}
}
// ── strategy_guidance ────────────────────────────────────────────
#[test]
fn strategy_guidance_quick() {
let g = strategy_guidance(&PentestStrategy::Quick);
assert!(g.contains("most common"));
assert!(g.contains("quick recon"));
}
#[test]
fn strategy_guidance_comprehensive() {
let g = strategy_guidance(&PentestStrategy::Comprehensive);
assert!(g.contains("thorough assessment"));
}
#[test]
fn strategy_guidance_targeted() {
let g = strategy_guidance(&PentestStrategy::Targeted);
assert!(g.contains("SAST findings"));
assert!(g.contains("known CVEs"));
}
#[test]
fn strategy_guidance_aggressive() {
let g = strategy_guidance(&PentestStrategy::Aggressive);
assert!(g.contains("aggressively"));
assert!(g.contains("full exploitation"));
}
#[test]
fn strategy_guidance_stealth() {
let g = strategy_guidance(&PentestStrategy::Stealth);
assert!(g.contains("Minimize noise"));
assert!(g.contains("passive analysis"));
}
// ── build_sast_section ───────────────────────────────────────────
#[test]
fn sast_section_empty() {
let section = build_sast_section(&[]);
assert_eq!(section, "No SAST findings available for this target.");
}
#[test]
fn sast_section_single_critical() {
let findings = vec![make_finding(
Severity::Critical,
"SQL Injection",
Some("src/db.rs"),
Some(42),
FindingStatus::Open,
Some("CWE-89"),
)];
let section = build_sast_section(&findings);
assert!(section.contains("1 open findings (1 critical, 0 high)"));
assert!(section.contains("[critical] SQL Injection in src/db.rs:42"));
assert!(section.contains("CWE: CWE-89"));
}
#[test]
fn sast_section_triaged_finding_shows_marker() {
let findings = vec![make_finding(
Severity::High,
"XSS",
None,
None,
FindingStatus::Triaged,
None,
)];
let section = build_sast_section(&findings);
assert!(section.contains("[TRIAGED]"));
}
#[test]
fn sast_section_no_file_path_omits_location() {
let findings = vec![make_finding(
Severity::Medium,
"Open Redirect",
None,
None,
FindingStatus::Open,
None,
)];
let section = build_sast_section(&findings);
assert!(section.contains("- [medium] Open Redirect\n"));
assert!(!section.contains(" in "));
}
#[test]
fn sast_section_counts_critical_and_high() {
let findings = vec![
make_finding(
Severity::Critical,
"F1",
None,
None,
FindingStatus::Open,
None,
),
make_finding(
Severity::Critical,
"F2",
None,
None,
FindingStatus::Open,
None,
),
make_finding(Severity::High, "F3", None, None, FindingStatus::Open, None),
make_finding(
Severity::Medium,
"F4",
None,
None,
FindingStatus::Open,
None,
),
];
let section = build_sast_section(&findings);
assert!(section.contains("4 open findings (2 critical, 1 high)"));
}
#[test]
fn sast_section_truncates_at_20() {
let findings: Vec<Finding> = (0..25)
.map(|i| {
make_finding(
Severity::Low,
&format!("Finding {i}"),
None,
None,
FindingStatus::Open,
None,
)
})
.collect();
let section = build_sast_section(&findings);
assert!(section.contains("... and 5 more findings"));
// Should contain Finding 19 (the 20th) but not Finding 20 (the 21st)
assert!(section.contains("Finding 19"));
assert!(!section.contains("Finding 20"));
}
// ── build_sbom_section ───────────────────────────────────────────
#[test]
fn sbom_section_empty() {
let section = build_sbom_section(&[]);
assert_eq!(section, "No vulnerable dependencies identified.");
}
#[test]
fn sbom_section_single_entry() {
let entries = vec![make_sbom_entry("lodash", "4.17.20", &["CVE-2021-23337"])];
let section = build_sbom_section(&entries);
assert!(section.contains("1 dependencies with known vulnerabilities"));
assert!(section.contains("- lodash 4.17.20 (npm): CVE-2021-23337"));
}
#[test]
fn sbom_section_multiple_cves() {
let entries = vec![make_sbom_entry(
"openssl",
"1.1.1",
&["CVE-2022-0001", "CVE-2022-0002"],
)];
let section = build_sbom_section(&entries);
assert!(section.contains("CVE-2022-0001, CVE-2022-0002"));
}
#[test]
fn sbom_section_truncates_at_15() {
let entries: Vec<SbomEntry> = (0..18)
.map(|i| make_sbom_entry(&format!("pkg-{i}"), "1.0.0", &["CVE-2024-0001"]))
.collect();
let section = build_sbom_section(&entries);
assert!(section.contains("... and 3 more vulnerable dependencies"));
assert!(section.contains("pkg-14"));
assert!(!section.contains("pkg-15"));
}
// ── build_code_section ───────────────────────────────────────────
#[test]
fn code_section_empty() {
let section = build_code_section(&[]);
assert_eq!(
section,
"No code knowledge graph available for this target."
);
}
#[test]
fn code_section_single_entry_no_vulns() {
let hints = vec![make_code_hint("GET /api/users", "src/routes.rs", vec![])];
let section = build_code_section(&hints);
assert!(section.contains("1 entry points identified (0 with linked SAST findings)"));
assert!(section.contains("- GET /api/users (src/routes.rs)"));
}
#[test]
fn code_section_with_linked_vulns() {
let hints = vec![make_code_hint(
"POST /login",
"src/auth.rs",
vec!["[critical] semgrep: SQL Injection (line 15)".into()],
)];
let section = build_code_section(&hints);
assert!(section.contains("1 entry points identified (1 with linked SAST findings)"));
assert!(section.contains("SAST: [critical] semgrep: SQL Injection (line 15)"));
}
#[test]
fn code_section_counts_entries_with_vulns() {
let hints = vec![
make_code_hint("GET /a", "a.rs", vec!["vuln1".into()]),
make_code_hint("GET /b", "b.rs", vec![]),
make_code_hint("GET /c", "c.rs", vec!["vuln2".into(), "vuln3".into()]),
];
let section = build_code_section(&hints);
assert!(section.contains("3 entry points identified (2 with linked SAST findings)"));
}
#[test]
fn code_section_truncates_at_20() {
let hints: Vec<CodeContextHint> = (0..25)
.map(|i| make_code_hint(&format!("GET /ep{i}"), &format!("f{i}.rs"), vec![]))
.collect();
let section = build_code_section(&hints);
assert!(section.contains("GET /ep19"));
assert!(!section.contains("GET /ep20"));
}
}
@@ -0,0 +1,43 @@
use std::io::{Cursor, Write};
use zip::write::SimpleFileOptions;
use zip::AesMode;
use super::ReportContext;
pub(super) fn build_zip(
ctx: &ReportContext,
password: &str,
html: &str,
pdf: &[u8],
) -> Result<Vec<u8>, zip::result::ZipError> {
let buf = Cursor::new(Vec::new());
let mut zip = zip::ZipWriter::new(buf);
let options = SimpleFileOptions::default()
.compression_method(zip::CompressionMethod::Deflated)
.with_aes_encryption(AesMode::Aes256, password);
// report.pdf (primary)
zip.start_file("report.pdf", options)?;
zip.write_all(pdf)?;
// report.html (fallback)
zip.start_file("report.html", options)?;
zip.write_all(html.as_bytes())?;
// findings.json
let findings_json =
serde_json::to_string_pretty(&ctx.findings).unwrap_or_else(|_| "[]".to_string());
zip.start_file("findings.json", options)?;
zip.write_all(findings_json.as_bytes())?;
// attack-chain.json
let chain_json =
serde_json::to_string_pretty(&ctx.attack_chain).unwrap_or_else(|_| "[]".to_string());
zip.start_file("attack-chain.json", options)?;
zip.write_all(chain_json.as_bytes())?;
let cursor = zip.finish()?;
Ok(cursor.into_inner())
}
@@ -0,0 +1,40 @@
use super::html_escape;
pub(super) fn appendix(session_id: &str) -> String {
format!(
r##"<!-- ═══════════════ 5. APPENDIX ═══════════════ -->
<div class="page-break"></div>
<h2><span class="section-num">5.</span> Appendix</h2>
<h3>Severity Definitions</h3>
<table class="info">
<tr><td style="color: var(--sev-critical); font-weight: 700;">Critical</td><td>Vulnerabilities that can be exploited remotely without authentication to execute arbitrary code, exfiltrate sensitive data, or fully compromise the system.</td></tr>
<tr><td style="color: var(--sev-high); font-weight: 700;">High</td><td>Vulnerabilities that allow significant unauthorized access or data exposure, typically requiring minimal user interaction or privileges.</td></tr>
<tr><td style="color: var(--sev-medium); font-weight: 700;">Medium</td><td>Vulnerabilities that may lead to limited data exposure or require specific conditions to exploit, but still represent meaningful risk.</td></tr>
<tr><td style="color: var(--sev-low); font-weight: 700;">Low</td><td>Minor issues with limited direct impact. May contribute to broader attack chains or indicate defense-in-depth weaknesses.</td></tr>
<tr><td style="color: var(--sev-info); font-weight: 700;">Info</td><td>Observations and best-practice recommendations that do not represent direct security vulnerabilities.</td></tr>
</table>
<h3>Disclaimer</h3>
<p style="font-size: 9pt; color: var(--text-secondary);">
This report was generated by an automated AI-powered penetration testing engine. While the system
employs advanced techniques to identify vulnerabilities, no automated assessment can guarantee
complete coverage. The results should be reviewed by qualified security professionals and validated
in the context of the target application's threat model. Findings are point-in-time observations
and may change as the application evolves.
</p>
<!-- ═══════════════ FOOTER ═══════════════ -->
<div class="report-footer">
<div class="footer-company">Compliance Scanner</div>
<div>AI-Powered Security Assessment Platform</div>
<div style="margin-top: 6px;">This document is confidential and intended solely for the named recipient.</div>
<div>Report ID: {session_id}</div>
</div>
</div><!-- .report-body -->
</body>
</html>"##,
session_id = html_escape(session_id),
)
}
@@ -0,0 +1,193 @@
use super::html_escape;
use compliance_core::models::pentest::AttackChainNode;
pub(super) fn attack_chain(chain: &[AttackChainNode]) -> String {
let chain_section = if chain.is_empty() {
r#"<p style="color: var(--text-muted);">No attack chain steps recorded.</p>"#.to_string()
} else {
build_chain_html(chain)
};
format!(
r##"<!-- ═══════════════ 4. ATTACK CHAIN ═══════════════ -->
<div class="page-break"></div>
<h2><span class="section-num">4.</span> Attack Chain Timeline</h2>
<p>
The following sequence shows each tool invocation made by the AI orchestrator during the assessment,
grouped by phase. Each step includes the tool's name, execution status, and the AI's reasoning
for choosing that action.
</p>
<div style="margin-top: 16px;">
{chain_section}
</div>"##
)
}
fn build_chain_html(chain: &[AttackChainNode]) -> String {
let mut chain_html = String::new();
// Compute phases via BFS from root nodes
let mut phase_map: std::collections::HashMap<String, usize> = std::collections::HashMap::new();
let mut queue: std::collections::VecDeque<String> = std::collections::VecDeque::new();
for node in chain {
if node.parent_node_ids.is_empty() {
let nid = node.node_id.clone();
if !nid.is_empty() {
phase_map.insert(nid.clone(), 0);
queue.push_back(nid);
}
}
}
while let Some(nid) = queue.pop_front() {
let parent_phase = phase_map.get(&nid).copied().unwrap_or(0);
for node in chain {
if node.parent_node_ids.contains(&nid) {
let child_id = node.node_id.clone();
if !child_id.is_empty() && !phase_map.contains_key(&child_id) {
phase_map.insert(child_id.clone(), parent_phase + 1);
queue.push_back(child_id);
}
}
}
}
// Assign phase 0 to any unassigned nodes
for node in chain {
let nid = node.node_id.clone();
if !nid.is_empty() && !phase_map.contains_key(&nid) {
phase_map.insert(nid, 0);
}
}
// Group nodes by phase
let max_phase = phase_map.values().copied().max().unwrap_or(0);
let phase_labels = [
"Reconnaissance",
"Enumeration",
"Exploitation",
"Validation",
"Post-Exploitation",
];
for phase_idx in 0..=max_phase {
let phase_nodes: Vec<&AttackChainNode> = chain
.iter()
.filter(|n| {
let nid = n.node_id.clone();
phase_map.get(&nid).copied().unwrap_or(0) == phase_idx
})
.collect();
if phase_nodes.is_empty() {
continue;
}
let label = if phase_idx < phase_labels.len() {
phase_labels[phase_idx]
} else {
"Additional Testing"
};
chain_html.push_str(&format!(
r#"<div class="phase-block">
<div class="phase-header">
<span class="phase-num">Phase {}</span>
<span class="phase-label">{}</span>
<span class="phase-count">{} step{}</span>
</div>
<div class="phase-steps">"#,
phase_idx + 1,
label,
phase_nodes.len(),
if phase_nodes.len() == 1 { "" } else { "s" },
));
for (i, node) in phase_nodes.iter().enumerate() {
let status_label = format!("{:?}", node.status);
let status_class = match status_label.to_lowercase().as_str() {
"completed" => "step-completed",
"failed" => "step-failed",
_ => "step-running",
};
let findings_badge = if !node.findings_produced.is_empty() {
format!(
r#"<span class="step-findings">{} finding{}</span>"#,
node.findings_produced.len(),
if node.findings_produced.len() == 1 {
""
} else {
"s"
},
)
} else {
String::new()
};
let risk_badge = node
.risk_score
.map(|r| {
let risk_class = if r >= 70 {
"risk-high"
} else if r >= 40 {
"risk-med"
} else {
"risk-low"
};
format!(r#"<span class="step-risk {risk_class}">Risk: {r}</span>"#)
})
.unwrap_or_default();
let reasoning_html = if node.llm_reasoning.is_empty() {
String::new()
} else {
format!(
r#"<div class="step-reasoning">{}</div>"#,
html_escape(&node.llm_reasoning)
)
};
// Render inline screenshot if this is a browser screenshot action
let screenshot_html = if node.tool_name == "browser" {
node.tool_output
.as_ref()
.and_then(|out| out.get("screenshot_base64"))
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.map(|b64| {
format!(
r#"<div class="step-screenshot"><img src="data:image/png;base64,{b64}" alt="Browser screenshot" style="max-width:100%;border:1px solid #e2e8f0;border-radius:6px;margin-top:8px;"/></div>"#
)
})
.unwrap_or_default()
} else {
String::new()
};
chain_html.push_str(&format!(
r#"<div class="step-row">
<div class="step-num">{num}</div>
<div class="step-connector"></div>
<div class="step-content">
<div class="step-header">
<span class="step-tool">{tool_name}</span>
<span class="step-status {status_class}">{status_label}</span>
{findings_badge}
{risk_badge}
</div>
{reasoning_html}
{screenshot_html}
</div>
</div>"#,
num = i + 1,
tool_name = html_escape(&node.tool_name),
));
}
chain_html.push_str("</div></div>");
}
chain_html
}
@@ -0,0 +1,69 @@
use super::html_escape;
pub(super) fn cover(
target_name: &str,
session_id: &str,
date_short: &str,
target_url: &str,
requester_name: &str,
requester_email: &str,
app_screenshot_b64: Option<&str>,
) -> String {
let screenshot_html = app_screenshot_b64
.filter(|s| !s.is_empty())
.map(|b64| {
format!(
r#"<div style="margin: 20px auto; max-width: 560px; border: 1px solid #cbd5e1; border-radius: 8px; overflow: hidden; box-shadow: 0 4px 12px rgba(0,0,0,0.08);">
<img src="data:image/png;base64,{b64}" alt="Application screenshot" style="width:100%;display:block;"/>
</div>"#
)
})
.unwrap_or_default();
format!(
r##"<!-- ═══════════════ COVER PAGE ═══════════════ -->
<div class="cover">
<svg class="cover-shield" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96">
<defs>
<linearGradient id="sg" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" stop-color="#0d2137"/>
<stop offset="100%" stop-color="#1a56db"/>
</linearGradient>
</defs>
<path d="M48 6 L22 22 L22 48 C22 66 34 80 48 86 C62 80 74 66 74 48 L74 22 Z"
fill="none" stroke="url(#sg)" stroke-width="3.5" stroke-linejoin="round"/>
<path d="M48 12 L26 26 L26 47 C26 63 36 76 48 82 C60 76 70 63 70 47 L70 26 Z"
fill="url(#sg)" opacity="0.07"/>
<circle cx="44" cy="44" r="11" fill="none" stroke="#0d2137" stroke-width="2.5"/>
<line x1="52" y1="52" x2="62" y2="62" stroke="#0d2137" stroke-width="2.5" stroke-linecap="round"/>
<path d="M39 44 L42.5 47.5 L49 41" fill="none" stroke="#166534" stroke-width="2.5"
stroke-linecap="round" stroke-linejoin="round"/>
</svg>
<div class="cover-tag">CONFIDENTIAL</div>
<div class="cover-title">Penetration Test Report</div>
<div class="cover-subtitle">{target_name}</div>
<div class="cover-divider"></div>
<div class="cover-meta">
<strong>Report ID:</strong> {session_id}<br>
<strong>Date:</strong> {date_short}<br>
<strong>Target:</strong> {target_url}<br>
<strong>Prepared for:</strong> {requester_name} ({requester_email})
</div>
{screenshot_html}
<div class="cover-footer">
Compliance Scanner &mdash; AI-Powered Security Assessment Platform
</div>
</div>"##,
target_name = html_escape(target_name),
session_id = html_escape(session_id),
date_short = date_short,
target_url = html_escape(target_url),
requester_name = html_escape(requester_name),
requester_email = html_escape(requester_email),
)
}
@@ -0,0 +1,238 @@
use super::html_escape;
use compliance_core::models::dast::DastFinding;
pub(super) fn executive_summary(
findings: &[DastFinding],
target_name: &str,
target_url: &str,
tool_count: usize,
tool_invocations: u32,
success_rate: f64,
) -> String {
let critical = findings
.iter()
.filter(|f| f.severity.to_string() == "critical")
.count();
let high = findings
.iter()
.filter(|f| f.severity.to_string() == "high")
.count();
let medium = findings
.iter()
.filter(|f| f.severity.to_string() == "medium")
.count();
let low = findings
.iter()
.filter(|f| f.severity.to_string() == "low")
.count();
let info = findings
.iter()
.filter(|f| f.severity.to_string() == "info")
.count();
let exploitable = findings.iter().filter(|f| f.exploitable).count();
let total = findings.len();
let overall_risk = if critical > 0 {
"CRITICAL"
} else if high > 0 {
"HIGH"
} else if medium > 0 {
"MEDIUM"
} else if low > 0 {
"LOW"
} else {
"INFORMATIONAL"
};
let risk_color = match overall_risk {
"CRITICAL" => "#991b1b",
"HIGH" => "#c2410c",
"MEDIUM" => "#a16207",
"LOW" => "#1d4ed8",
_ => "#4b5563",
};
let risk_score: usize =
std::cmp::min(100, critical * 25 + high * 15 + medium * 8 + low * 3 + info);
let severity_bar = build_severity_bar(critical, high, medium, low, info, total);
// Table of contents finding sub-entries
let severity_order = ["critical", "high", "medium", "low", "info"];
let toc_findings_sub = if !findings.is_empty() {
let mut sub = String::new();
let mut fnum = 0usize;
for &sev_key in severity_order.iter() {
let count = findings
.iter()
.filter(|f| f.severity.to_string() == sev_key)
.count();
if count == 0 {
continue;
}
for f in findings
.iter()
.filter(|f| f.severity.to_string() == sev_key)
{
fnum += 1;
sub.push_str(&format!(
r#"<div class="toc-sub">F-{:03} — {}</div>"#,
fnum,
html_escape(&f.title),
));
}
}
sub
} else {
String::new()
};
let critical_high_str = format!("{} / {}", critical, high);
let escaped_target_name = html_escape(target_name);
let escaped_target_url = html_escape(target_url);
format!(
r##"<!-- ═══════════════ TABLE OF CONTENTS ═══════════════ -->
<div class="report-body">
<div class="toc">
<h2>Table of Contents</h2>
<div class="toc-entry"><span class="toc-num">1</span><span class="toc-label">Executive Summary</span></div>
<div class="toc-entry"><span class="toc-num">2</span><span class="toc-label">Scope &amp; Methodology</span></div>
<div class="toc-entry"><span class="toc-num">3</span><span class="toc-label">Findings ({total_findings})</span></div>
{toc_findings_sub}
<div class="toc-entry"><span class="toc-num">4</span><span class="toc-label">Attack Chain Timeline</span></div>
<div class="toc-entry"><span class="toc-num">5</span><span class="toc-label">Appendix</span></div>
</div>
<!-- ═══════════════ 1. EXECUTIVE SUMMARY ═══════════════ -->
<h2><span class="section-num">1.</span> Executive Summary</h2>
<div class="risk-gauge">
<div class="risk-gauge-meter">
<div class="risk-gauge-track">
<div class="risk-gauge-fill" style="width: {risk_score}%; background: {risk_color};"></div>
</div>
<div class="risk-gauge-score" style="color: {risk_color};">{risk_score} / 100</div>
</div>
<div class="risk-gauge-text">
<div class="risk-gauge-label" style="color: {risk_color};">Overall Risk: {overall_risk}</div>
<div class="risk-gauge-desc">
Based on {total_findings} finding{findings_plural} identified across the target application.
</div>
</div>
</div>
<div class="exec-grid">
<div class="kpi-card">
<div class="kpi-value">{total_findings}</div>
<div class="kpi-label">Total Findings</div>
</div>
<div class="kpi-card">
<div class="kpi-value" style="color: var(--sev-critical);">{critical_high}</div>
<div class="kpi-label">Critical / High</div>
</div>
<div class="kpi-card">
<div class="kpi-value" style="color: var(--sev-critical);">{exploitable_count}</div>
<div class="kpi-label">Exploitable</div>
</div>
<div class="kpi-card">
<div class="kpi-value">{tool_count}</div>
<div class="kpi-label">Tools Used</div>
</div>
</div>
<h3>Severity Distribution</h3>
{severity_bar}
<p>
This report presents the results of an automated penetration test conducted against
<strong>{target_name}</strong> (<code>{target_url}</code>) using the Compliance Scanner
AI-powered testing engine. A total of <strong>{total_findings} vulnerabilities</strong> were
identified, of which <strong>{exploitable_count}</strong> were confirmed exploitable with
working proof-of-concept payloads. The assessment employed <strong>{tool_count} security tools</strong>
across <strong>{tool_invocations} invocations</strong> ({success_rate:.0}% success rate).
</p>"##,
total_findings = total,
findings_plural = if total == 1 { "" } else { "s" },
critical_high = critical_high_str,
exploitable_count = exploitable,
target_name = escaped_target_name,
target_url = escaped_target_url,
)
}
fn build_severity_bar(
critical: usize,
high: usize,
medium: usize,
low: usize,
info: usize,
total: usize,
) -> String {
if total == 0 {
return String::new();
}
let crit_pct = (critical as f64 / total as f64 * 100.0) as usize;
let high_pct = (high as f64 / total as f64 * 100.0) as usize;
let med_pct = (medium as f64 / total as f64 * 100.0) as usize;
let low_pct = (low as f64 / total as f64 * 100.0) as usize;
let info_pct = 100_usize.saturating_sub(crit_pct + high_pct + med_pct + low_pct);
let mut bar = String::from(r#"<div class="sev-bar">"#);
if critical > 0 {
bar.push_str(&format!(
r#"<div class="sev-bar-seg sev-bar-critical" style="width:{}%"><span>{}</span></div>"#,
std::cmp::max(crit_pct, 4),
critical
));
}
if high > 0 {
bar.push_str(&format!(
r#"<div class="sev-bar-seg sev-bar-high" style="width:{}%"><span>{}</span></div>"#,
std::cmp::max(high_pct, 4),
high
));
}
if medium > 0 {
bar.push_str(&format!(
r#"<div class="sev-bar-seg sev-bar-medium" style="width:{}%"><span>{}</span></div>"#,
std::cmp::max(med_pct, 4),
medium
));
}
if low > 0 {
bar.push_str(&format!(
r#"<div class="sev-bar-seg sev-bar-low" style="width:{}%"><span>{}</span></div>"#,
std::cmp::max(low_pct, 4),
low
));
}
if info > 0 {
bar.push_str(&format!(
r#"<div class="sev-bar-seg sev-bar-info" style="width:{}%"><span>{}</span></div>"#,
std::cmp::max(info_pct, 4),
info
));
}
bar.push_str("</div>");
bar.push_str(r#"<div class="sev-bar-legend">"#);
if critical > 0 {
bar.push_str(r#"<span><i class="sev-dot" style="background:#991b1b"></i> Critical</span>"#);
}
if high > 0 {
bar.push_str(r#"<span><i class="sev-dot" style="background:#c2410c"></i> High</span>"#);
}
if medium > 0 {
bar.push_str(r#"<span><i class="sev-dot" style="background:#a16207"></i> Medium</span>"#);
}
if low > 0 {
bar.push_str(r#"<span><i class="sev-dot" style="background:#1d4ed8"></i> Low</span>"#);
}
if info > 0 {
bar.push_str(r#"<span><i class="sev-dot" style="background:#4b5563"></i> Info</span>"#);
}
bar.push_str("</div>");
bar
}
@@ -0,0 +1,522 @@
use super::html_escape;
use compliance_core::models::dast::DastFinding;
use compliance_core::models::finding::Finding;
use compliance_core::models::pentest::CodeContextHint;
use compliance_core::models::sbom::SbomEntry;
/// Render the findings section with code-level correlation.
///
/// For each DAST finding, if a linked SAST finding exists (via `linked_sast_finding_id`)
/// or if we can match the endpoint to a code entry point, we render a "Code-Level
/// Remediation" block showing the exact file, line, code snippet, and suggested fix.
pub(super) fn findings(
findings_list: &[DastFinding],
sast_findings: &[Finding],
code_context: &[CodeContextHint],
sbom_entries: &[SbomEntry],
) -> String {
if findings_list.is_empty() {
return r#"<!-- ═══════════════ 3. FINDINGS ═══════════════ -->
<div class="page-break"></div>
<h2><span class="section-num">3.</span> Findings</h2>
<p style="color: var(--text-muted);">No vulnerabilities were identified during this assessment.</p>"#.to_string();
}
let severity_order = ["critical", "high", "medium", "low", "info"];
let severity_labels = ["Critical", "High", "Medium", "Low", "Informational"];
let severity_colors = ["#991b1b", "#c2410c", "#a16207", "#1d4ed8", "#4b5563"];
// Build SAST lookup by ObjectId hex string
let sast_by_id: std::collections::HashMap<String, &Finding> = sast_findings
.iter()
.filter_map(|f| {
let id = f.id.as_ref()?.to_hex();
Some((id, f))
})
.collect();
let mut findings_html = String::new();
let mut finding_num = 0usize;
for (si, &sev_key) in severity_order.iter().enumerate() {
let sev_findings: Vec<&DastFinding> = findings_list
.iter()
.filter(|f| f.severity.to_string() == sev_key)
.collect();
if sev_findings.is_empty() {
continue;
}
findings_html.push_str(&format!(
r#"<h4 class="sev-group-title" style="border-color: {color}">{label} ({count})</h4>"#,
color = severity_colors[si],
label = severity_labels[si],
count = sev_findings.len(),
));
for f in sev_findings {
finding_num += 1;
let sev_color = severity_colors[si];
let exploitable_badge = if f.exploitable {
r#"<span class="badge badge-exploit">EXPLOITABLE</span>"#
} else {
""
};
let cwe_cell = f
.cwe
.as_deref()
.map(|c| format!("<tr><td>CWE</td><td>{}</td></tr>", html_escape(c)))
.unwrap_or_default();
let param_row = f
.parameter
.as_deref()
.map(|p| {
format!(
"<tr><td>Parameter</td><td><code>{}</code></td></tr>",
html_escape(p)
)
})
.unwrap_or_default();
let remediation = f
.remediation
.as_deref()
.unwrap_or("Refer to industry best practices for this vulnerability class.");
let evidence_html = build_evidence_html(f);
// ── Code-level correlation ──────────────────────────────
let code_correlation =
build_code_correlation(f, &sast_by_id, code_context, sbom_entries);
findings_html.push_str(&format!(
r#"
<div class="finding" style="border-left-color: {sev_color}">
<div class="finding-header">
<span class="finding-id">F-{num:03}</span>
<span class="finding-title">{title}</span>
{exploitable_badge}
</div>
<table class="finding-meta">
<tr><td>Type</td><td>{vuln_type}</td></tr>
<tr><td>Endpoint</td><td><code>{method} {endpoint}</code></td></tr>
{param_row}
{cwe_cell}
</table>
<div class="finding-desc">{description}</div>
{evidence_html}
{code_correlation}
<div class="remediation">
<div class="remediation-label">Recommendation</div>
{remediation}
</div>
</div>
"#,
num = finding_num,
title = html_escape(&f.title),
vuln_type = f.vuln_type,
method = f.method,
endpoint = html_escape(&f.endpoint),
description = html_escape(&f.description),
));
}
}
format!(
r##"<!-- ═══════════════ 3. FINDINGS ═══════════════ -->
<div class="page-break"></div>
<h2><span class="section-num">3.</span> Findings</h2>
{findings_html}"##
)
}
/// Build the evidence table HTML for a finding.
fn build_evidence_html(f: &DastFinding) -> String {
if f.evidence.is_empty() {
return String::new();
}
let mut eh = String::from(
r#"<div class="evidence-block"><div class="evidence-title">Evidence</div><table class="evidence-table"><thead><tr><th>Request</th><th>Status</th><th>Details</th></tr></thead><tbody>"#,
);
for ev in &f.evidence {
let payload_info = ev
.payload
.as_deref()
.map(|p| {
format!(
"<br><span class=\"evidence-payload\">Payload: <code>{}</code></span>",
html_escape(p)
)
})
.unwrap_or_default();
eh.push_str(&format!(
"<tr><td><code>{} {}</code></td><td>{}</td><td>{}{}</td></tr>",
html_escape(&ev.request_method),
html_escape(&ev.request_url),
ev.response_status,
ev.response_snippet
.as_deref()
.map(html_escape)
.unwrap_or_default(),
payload_info,
));
}
eh.push_str("</tbody></table></div>");
eh
}
/// Build the code-level correlation block for a DAST finding.
///
/// Attempts correlation in priority order:
/// 1. Direct link via `linked_sast_finding_id` → shows exact file, line, snippet, suggested fix
/// 2. Endpoint match via code context → shows handler function, file, known SAST vulns
/// 3. CWE/CVE match to SBOM → shows vulnerable dependency + version to upgrade
fn build_code_correlation(
dast_finding: &DastFinding,
sast_by_id: &std::collections::HashMap<String, &Finding>,
code_context: &[CodeContextHint],
sbom_entries: &[SbomEntry],
) -> String {
let mut sections: Vec<String> = Vec::new();
// 1. Direct SAST link
if let Some(ref sast_id) = dast_finding.linked_sast_finding_id {
if let Some(sast) = sast_by_id.get(sast_id) {
let mut s = String::new();
s.push_str(r#"<div class="code-correlation-item">"#);
s.push_str(r#"<div class="code-correlation-badge">SAST Correlation</div>"#);
s.push_str("<table class=\"code-meta\">");
if let Some(ref fp) = sast.file_path {
let line_info = sast
.line_number
.map(|l| format!(":{l}"))
.unwrap_or_default();
s.push_str(&format!(
"<tr><td>Location</td><td><code>{}{}</code></td></tr>",
html_escape(fp),
line_info,
));
}
s.push_str(&format!(
"<tr><td>Scanner</td><td>{} &mdash; {}</td></tr>",
html_escape(&sast.scanner),
html_escape(&sast.title),
));
if let Some(ref cwe) = sast.cwe {
s.push_str(&format!(
"<tr><td>CWE</td><td>{}</td></tr>",
html_escape(cwe)
));
}
if let Some(ref rule) = sast.rule_id {
s.push_str(&format!(
"<tr><td>Rule</td><td><code>{}</code></td></tr>",
html_escape(rule)
));
}
s.push_str("</table>");
// Code snippet
if let Some(ref snippet) = sast.code_snippet {
if !snippet.is_empty() {
s.push_str(&format!(
"<div class=\"code-snippet-block\"><div class=\"code-snippet-label\">Vulnerable Code</div><pre class=\"code-snippet\">{}</pre></div>",
html_escape(snippet)
));
}
}
// Suggested fix
if let Some(ref fix) = sast.suggested_fix {
if !fix.is_empty() {
s.push_str(&format!(
"<div class=\"code-fix-block\"><div class=\"code-fix-label\">Suggested Fix</div><pre class=\"code-fix\">{}</pre></div>",
html_escape(fix)
));
}
}
// Remediation from SAST
if let Some(ref rem) = sast.remediation {
if !rem.is_empty() {
s.push_str(&format!(
"<div class=\"code-remediation\">{}</div>",
html_escape(rem)
));
}
}
s.push_str("</div>");
sections.push(s);
}
}
// 2. Endpoint match via code context
let endpoint_lower = dast_finding.endpoint.to_lowercase();
let matching_hints: Vec<&CodeContextHint> = code_context
.iter()
.filter(|hint| {
// Match by endpoint pattern overlap
let pattern_lower = hint.endpoint_pattern.to_lowercase();
endpoint_lower.contains(&pattern_lower)
|| pattern_lower.contains(&endpoint_lower)
|| hint.file_path.to_lowercase().contains(
&endpoint_lower
.split('/')
.next_back()
.unwrap_or("")
.replace(".html", "")
.replace(".php", ""),
)
})
.collect();
for hint in &matching_hints {
let mut s = String::new();
s.push_str(r#"<div class="code-correlation-item">"#);
s.push_str(r#"<div class="code-correlation-badge">Code Entry Point</div>"#);
s.push_str("<table class=\"code-meta\">");
s.push_str(&format!(
"<tr><td>Handler</td><td><code>{}</code></td></tr>",
html_escape(&hint.handler_function),
));
s.push_str(&format!(
"<tr><td>File</td><td><code>{}</code></td></tr>",
html_escape(&hint.file_path),
));
s.push_str(&format!(
"<tr><td>Route</td><td><code>{}</code></td></tr>",
html_escape(&hint.endpoint_pattern),
));
s.push_str("</table>");
if !hint.known_vulnerabilities.is_empty() {
s.push_str("<div class=\"code-linked-vulns\"><strong>Known SAST issues in this file:</strong><ul>");
for vuln in &hint.known_vulnerabilities {
s.push_str(&format!("<li>{}</li>", html_escape(vuln)));
}
s.push_str("</ul></div>");
}
s.push_str("</div>");
sections.push(s);
}
// 3. SBOM match — if a linked SAST finding has a CVE, or we can match by CWE
let linked_cve = dast_finding
.linked_sast_finding_id
.as_deref()
.and_then(|id| sast_by_id.get(id))
.and_then(|f| f.cve.as_deref());
if let Some(cve_id) = linked_cve {
let matching_deps: Vec<&SbomEntry> = sbom_entries
.iter()
.filter(|e| e.known_vulnerabilities.iter().any(|v| v.id == cve_id))
.collect();
for dep in &matching_deps {
let mut s = String::new();
s.push_str(r#"<div class="code-correlation-item">"#);
s.push_str(r#"<div class="code-correlation-badge">Vulnerable Dependency</div>"#);
s.push_str("<table class=\"code-meta\">");
s.push_str(&format!(
"<tr><td>Package</td><td><code>{} {}</code> ({})</td></tr>",
html_escape(&dep.name),
html_escape(&dep.version),
html_escape(&dep.package_manager),
));
let cve_ids: Vec<&str> = dep
.known_vulnerabilities
.iter()
.map(|v| v.id.as_str())
.collect();
s.push_str(&format!(
"<tr><td>CVEs</td><td>{}</td></tr>",
cve_ids.join(", "),
));
if let Some(ref purl) = dep.purl {
s.push_str(&format!(
"<tr><td>PURL</td><td><code>{}</code></td></tr>",
html_escape(purl),
));
}
s.push_str("</table>");
s.push_str(&format!(
"<div class=\"code-remediation\">Upgrade <code>{}</code> to the latest patched version to resolve {}.</div>",
html_escape(&dep.name),
html_escape(cve_id),
));
s.push_str("</div>");
sections.push(s);
}
}
if sections.is_empty() {
return String::new();
}
format!(
r#"<div class="code-correlation">
<div class="code-correlation-title">Code-Level Remediation</div>
{}
</div>"#,
sections.join("\n")
)
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::dast::{DastEvidence, DastVulnType};
use compliance_core::models::finding::Severity;
use compliance_core::models::scan::ScanType;
/// Helper: create a minimal `DastFinding`.
fn make_dast(title: &str, severity: Severity, endpoint: &str) -> DastFinding {
DastFinding::new(
"run1".into(),
"target1".into(),
DastVulnType::Xss,
title.into(),
"desc".into(),
severity,
endpoint.into(),
"GET".into(),
)
}
/// Helper: create a minimal SAST `Finding` with an ObjectId.
fn make_sast(title: &str) -> Finding {
let mut f = Finding::new(
"repo1".into(),
"fp1".into(),
"semgrep".into(),
ScanType::Sast,
title.into(),
"sast desc".into(),
Severity::High,
);
f.id = Some(mongodb::bson::oid::ObjectId::new());
f
}
#[test]
fn test_findings_empty() {
let result = findings(&[], &[], &[], &[]);
assert!(
result.contains("No vulnerabilities were identified"),
"Empty findings should contain the no-vulns message"
);
}
#[test]
fn test_findings_grouped_by_severity() {
let f_high = make_dast("High vuln", Severity::High, "/a");
let f_low = make_dast("Low vuln", Severity::Low, "/b");
let f_critical = make_dast("Crit vuln", Severity::Critical, "/c");
let result = findings(&[f_high, f_low, f_critical], &[], &[], &[]);
// All severity group headers should appear
assert!(
result.contains("Critical (1)"),
"should have Critical header"
);
assert!(result.contains("High (1)"), "should have High header");
assert!(result.contains("Low (1)"), "should have Low header");
// Critical should appear before High, High before Low
let crit_pos = result.find("Critical (1)");
let high_pos = result.find("High (1)");
let low_pos = result.find("Low (1)");
assert!(crit_pos < high_pos, "Critical should come before High");
assert!(high_pos < low_pos, "High should come before Low");
}
#[test]
fn test_code_correlation_sast_link() {
let mut sast = make_sast("SQL Injection in query");
sast.file_path = Some("src/db/query.rs".into());
sast.line_number = Some(42);
sast.code_snippet =
Some("let q = format!(\"SELECT * FROM {} WHERE id={}\", table, id);".into());
let sast_id = sast.id.as_ref().map(|oid| oid.to_hex()).unwrap_or_default();
let mut dast = make_dast("SQLi on /api/users", Severity::High, "/api/users");
dast.linked_sast_finding_id = Some(sast_id);
let result = findings(&[dast], &[sast], &[], &[]);
assert!(
result.contains("SAST Correlation"),
"should render SAST Correlation badge"
);
assert!(
result.contains("src/db/query.rs"),
"should contain the file path"
);
assert!(result.contains(":42"), "should contain the line number");
assert!(
result.contains("Vulnerable Code"),
"should render code snippet block"
);
}
#[test]
fn test_code_correlation_no_match() {
let dast = make_dast("XSS in search", Severity::Medium, "/search");
// No linked_sast_finding_id, no code context, no sbom
let result = findings(&[dast], &[], &[], &[]);
assert!(
!result.contains("code-correlation"),
"should not contain any code-correlation div"
);
}
#[test]
fn test_evidence_html_empty() {
let f = make_dast("No evidence", Severity::Low, "/x");
let result = build_evidence_html(&f);
assert!(result.is_empty(), "no evidence should yield empty string");
}
#[test]
fn test_evidence_html_with_entries() {
let mut f = make_dast("Has evidence", Severity::High, "/y");
f.evidence.push(DastEvidence {
request_method: "POST".into(),
request_url: "https://example.com/login".into(),
request_headers: None,
request_body: None,
response_status: 200,
response_headers: None,
response_snippet: Some("OK".into()),
screenshot_path: None,
payload: Some("<script>alert(1)</script>".into()),
response_time_ms: None,
});
let result = build_evidence_html(&f);
assert!(
result.contains("evidence-table"),
"should render the evidence table"
);
assert!(result.contains("POST"), "should contain request method");
assert!(
result.contains("https://example.com/login"),
"should contain request URL"
);
assert!(result.contains("200"), "should contain response status");
assert!(
result.contains("&lt;script&gt;alert(1)&lt;/script&gt;"),
"payload should be HTML-escaped"
);
}
}
@@ -0,0 +1,518 @@
mod appendix;
mod attack_chain;
mod cover;
mod executive_summary;
mod findings;
mod scope;
mod styles;
use super::ReportContext;
#[allow(clippy::format_in_format_args)]
pub(super) fn build_html_report(ctx: &ReportContext) -> String {
let session = &ctx.session;
let session_id = session
.id
.map(|oid| oid.to_hex())
.unwrap_or_else(|| "-".to_string());
let date_str = session
.started_at
.format("%B %d, %Y at %H:%M UTC")
.to_string();
let date_short = session.started_at.format("%B %d, %Y").to_string();
let completed_str = session
.completed_at
.map(|d| d.format("%B %d, %Y at %H:%M UTC").to_string())
.unwrap_or_else(|| "In Progress".to_string());
// Collect unique tool names used
let tool_names: Vec<String> = {
let mut names: Vec<String> = ctx
.attack_chain
.iter()
.map(|n| n.tool_name.clone())
.collect();
names.sort();
names.dedup();
names
};
// Find the best app screenshot for the cover page:
// prefer the first navigate to the target URL that has a screenshot,
// falling back to any navigate with a screenshot
let app_screenshot: Option<String> = ctx
.attack_chain
.iter()
.filter(|n| n.tool_name == "browser")
.filter_map(|n| {
n.tool_output
.as_ref()?
.get("screenshot_base64")?
.as_str()
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
})
// Skip the Keycloak login page screenshots — prefer one that shows the actual app
.find(|_| {
ctx.attack_chain
.iter()
.filter(|n| n.tool_name == "browser")
.any(|n| {
n.tool_output
.as_ref()
.and_then(|o| o.get("title"))
.and_then(|t| t.as_str())
.is_some_and(|t| t.contains("Compliance") || t.contains("Dashboard"))
})
})
.or_else(|| {
// Fallback: any screenshot
ctx.attack_chain
.iter()
.filter(|n| n.tool_name == "browser")
.filter_map(|n| {
n.tool_output
.as_ref()?
.get("screenshot_base64")?
.as_str()
.filter(|s| !s.is_empty())
.map(|s| s.to_string())
})
.next()
});
let styles_html = styles::styles();
let cover_html = cover::cover(
&ctx.target_name,
&session_id,
&date_short,
&ctx.target_url,
&ctx.requester_name,
&ctx.requester_email,
app_screenshot.as_deref(),
);
let exec_html = executive_summary::executive_summary(
&ctx.findings,
&ctx.target_name,
&ctx.target_url,
tool_names.len(),
session.tool_invocations,
session.success_rate(),
);
let scope_html = scope::scope(
session,
&ctx.target_name,
&ctx.target_url,
&date_str,
&completed_str,
&tool_names,
ctx.config.as_ref(),
);
let findings_html = findings::findings(
&ctx.findings,
&ctx.sast_findings,
&ctx.code_context,
&ctx.sbom_entries,
);
let chain_html = attack_chain::attack_chain(&ctx.attack_chain);
let appendix_html = appendix::appendix(&session_id);
format!(
r#"<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Penetration Test Report — {target_name}</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Libre+Baskerville:ital,wght@0,400;0,700;1,400&family=Source+Sans+3:ital,wght@0,300;0,400;0,600;0,700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
{styles_html}
</head>
<body>
{cover_html}
{exec_html}
{scope_html}
{findings_html}
{chain_html}
{appendix_html}
"#,
target_name = html_escape(&ctx.target_name),
)
}
fn tool_category(tool_name: &str) -> &'static str {
let name = tool_name.to_lowercase();
if name.contains("nmap") || name.contains("port") {
return "Network Reconnaissance";
}
if name.contains("nikto") || name.contains("header") {
return "Web Server Analysis";
}
if name.contains("zap") || name.contains("spider") || name.contains("crawl") {
return "Web Application Scanning";
}
if name.contains("sqlmap") || name.contains("sqli") || name.contains("sql") {
return "SQL Injection Testing";
}
if name.contains("xss") || name.contains("cross-site") {
return "Cross-Site Scripting Testing";
}
if name.contains("dir")
|| name.contains("brute")
|| name.contains("fuzz")
|| name.contains("gobuster")
{
return "Directory Enumeration";
}
if name.contains("ssl") || name.contains("tls") || name.contains("cert") {
return "SSL/TLS Analysis";
}
if name.contains("api") || name.contains("endpoint") {
return "API Security Testing";
}
if name.contains("auth") || name.contains("login") || name.contains("credential") {
return "Authentication Testing";
}
if name.contains("cors") {
return "CORS Testing";
}
if name.contains("csrf") {
return "CSRF Testing";
}
if name.contains("nuclei") || name.contains("template") {
return "Vulnerability Scanning";
}
if name.contains("whatweb") || name.contains("tech") || name.contains("wappalyzer") {
return "Technology Fingerprinting";
}
"Security Testing"
}
fn html_escape(s: &str) -> String {
s.replace('&', "&amp;")
.replace('<', "&lt;")
.replace('>', "&gt;")
.replace('"', "&quot;")
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::dast::{DastFinding, DastVulnType};
use compliance_core::models::finding::Severity;
use compliance_core::models::pentest::{
AttackChainNode, AttackNodeStatus, PentestSession, PentestStrategy,
};
// ── html_escape ──────────────────────────────────────────────────
#[test]
fn html_escape_handles_ampersand() {
assert_eq!(html_escape("a & b"), "a &amp; b");
}
#[test]
fn html_escape_handles_angle_brackets() {
assert_eq!(html_escape("<script>"), "&lt;script&gt;");
}
#[test]
fn html_escape_handles_quotes() {
assert_eq!(html_escape(r#"key="val""#), "key=&quot;val&quot;");
}
#[test]
fn html_escape_handles_all_special_chars() {
assert_eq!(
html_escape(r#"<a href="x">&y</a>"#),
"&lt;a href=&quot;x&quot;&gt;&amp;y&lt;/a&gt;"
);
}
#[test]
fn html_escape_no_change_for_plain_text() {
assert_eq!(html_escape("hello world"), "hello world");
}
#[test]
fn html_escape_empty_string() {
assert_eq!(html_escape(""), "");
}
// ── tool_category ────────────────────────────────────────────────
#[test]
fn tool_category_nmap() {
assert_eq!(tool_category("nmap_scan"), "Network Reconnaissance");
}
#[test]
fn tool_category_port_scanner() {
assert_eq!(tool_category("port_scanner"), "Network Reconnaissance");
}
#[test]
fn tool_category_nikto() {
assert_eq!(tool_category("nikto"), "Web Server Analysis");
}
#[test]
fn tool_category_header_check() {
assert_eq!(
tool_category("security_header_check"),
"Web Server Analysis"
);
}
#[test]
fn tool_category_zap_spider() {
assert_eq!(tool_category("zap_spider"), "Web Application Scanning");
}
#[test]
fn tool_category_sqlmap() {
assert_eq!(tool_category("sqlmap"), "SQL Injection Testing");
}
#[test]
fn tool_category_xss_scanner() {
assert_eq!(tool_category("xss_scanner"), "Cross-Site Scripting Testing");
}
#[test]
fn tool_category_dir_bruteforce() {
assert_eq!(tool_category("dir_bruteforce"), "Directory Enumeration");
}
#[test]
fn tool_category_gobuster() {
assert_eq!(tool_category("gobuster"), "Directory Enumeration");
}
#[test]
fn tool_category_ssl_check() {
assert_eq!(tool_category("ssl_check"), "SSL/TLS Analysis");
}
#[test]
fn tool_category_tls_scan() {
assert_eq!(tool_category("tls_scan"), "SSL/TLS Analysis");
}
#[test]
fn tool_category_api_test() {
assert_eq!(tool_category("api_endpoint_test"), "API Security Testing");
}
#[test]
fn tool_category_auth_bypass() {
assert_eq!(tool_category("auth_bypass_check"), "Authentication Testing");
}
#[test]
fn tool_category_cors() {
assert_eq!(tool_category("cors_check"), "CORS Testing");
}
#[test]
fn tool_category_csrf() {
assert_eq!(tool_category("csrf_scanner"), "CSRF Testing");
}
#[test]
fn tool_category_nuclei() {
assert_eq!(tool_category("nuclei"), "Vulnerability Scanning");
}
#[test]
fn tool_category_whatweb() {
assert_eq!(tool_category("whatweb"), "Technology Fingerprinting");
}
#[test]
fn tool_category_unknown_defaults_to_security_testing() {
assert_eq!(tool_category("custom_tool"), "Security Testing");
}
#[test]
fn tool_category_is_case_insensitive() {
assert_eq!(tool_category("NMAP_Scanner"), "Network Reconnaissance");
assert_eq!(tool_category("SQLMap"), "SQL Injection Testing");
}
// ── build_html_report ────────────────────────────────────────────
fn make_session(strategy: PentestStrategy) -> PentestSession {
let mut s = PentestSession::new("target-1".into(), strategy);
s.tool_invocations = 5;
s.tool_successes = 4;
s.findings_count = 2;
s.exploitable_count = 1;
s
}
fn make_finding(severity: Severity, title: &str, exploitable: bool) -> DastFinding {
let mut f = DastFinding::new(
"run-1".into(),
"target-1".into(),
DastVulnType::Xss,
title.into(),
"description".into(),
severity,
"https://example.com/test".into(),
"GET".into(),
);
f.exploitable = exploitable;
f
}
fn make_attack_node(tool_name: &str) -> AttackChainNode {
let mut node = AttackChainNode::new(
"session-1".into(),
"node-1".into(),
tool_name.into(),
serde_json::json!({}),
"Testing this tool".into(),
);
node.status = AttackNodeStatus::Completed;
node
}
fn make_report_context(
findings: Vec<DastFinding>,
chain: Vec<AttackChainNode>,
) -> ReportContext {
ReportContext {
session: make_session(PentestStrategy::Comprehensive),
target_name: "Test App".into(),
target_url: "https://example.com".into(),
findings,
attack_chain: chain,
requester_name: "Alice".into(),
requester_email: "alice@example.com".into(),
config: None,
sast_findings: Vec::new(),
sbom_entries: Vec::new(),
code_context: Vec::new(),
}
}
#[test]
fn report_contains_target_info() {
let ctx = make_report_context(vec![], vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("Test App"));
assert!(html.contains("https://example.com"));
}
#[test]
fn report_contains_requester_info() {
let ctx = make_report_context(vec![], vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("Alice"));
assert!(html.contains("alice@example.com"));
}
#[test]
fn report_shows_informational_risk_when_no_findings() {
let ctx = make_report_context(vec![], vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("INFORMATIONAL"));
}
#[test]
fn report_shows_critical_risk_with_critical_finding() {
let findings = vec![make_finding(Severity::Critical, "Critical XSS", true)];
let ctx = make_report_context(findings, vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("CRITICAL"));
}
#[test]
fn report_shows_high_risk_without_critical() {
let findings = vec![make_finding(Severity::High, "High SQLi", false)];
let ctx = make_report_context(findings, vec![]);
let html = build_html_report(&ctx);
// Should show HIGH, not CRITICAL
assert!(html.contains("HIGH"));
}
#[test]
fn report_shows_medium_risk_level() {
let findings = vec![make_finding(Severity::Medium, "Medium Issue", false)];
let ctx = make_report_context(findings, vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("MEDIUM"));
}
#[test]
fn report_includes_finding_title() {
let findings = vec![make_finding(
Severity::High,
"Reflected XSS in /search",
true,
)];
let ctx = make_report_context(findings, vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("Reflected XSS in /search"));
}
#[test]
fn report_shows_exploitable_badge() {
let findings = vec![make_finding(Severity::Critical, "SQLi", true)];
let ctx = make_report_context(findings, vec![]);
let html = build_html_report(&ctx);
// The report should mark exploitable findings
assert!(html.contains("EXPLOITABLE"));
}
#[test]
fn report_includes_attack_chain_tool_names() {
let chain = vec![make_attack_node("nmap_scan"), make_attack_node("sqlmap")];
let ctx = make_report_context(vec![], chain);
let html = build_html_report(&ctx);
assert!(html.contains("nmap_scan"));
assert!(html.contains("sqlmap"));
}
#[test]
fn report_is_valid_html_structure() {
let ctx = make_report_context(vec![], vec![]);
let html = build_html_report(&ctx);
assert!(html.contains("<!DOCTYPE html>") || html.contains("<html"));
assert!(html.contains("</html>"));
}
#[test]
fn report_strategy_appears() {
let ctx = make_report_context(vec![], vec![]);
let html = build_html_report(&ctx);
// PentestStrategy::Comprehensive => "comprehensive"
assert!(html.contains("comprehensive") || html.contains("Comprehensive"));
}
#[test]
fn report_finding_count_is_correct() {
let findings = vec![
make_finding(Severity::Critical, "F1", true),
make_finding(Severity::High, "F2", false),
make_finding(Severity::Low, "F3", false),
];
let ctx = make_report_context(findings, vec![]);
let html = build_html_report(&ctx);
// The total count "3" should appear somewhere
assert!(
html.contains(">3<")
|| html.contains(">3 ")
|| html.contains("3 findings")
|| html.contains("3 Total")
);
}
}
@@ -0,0 +1,127 @@
use super::{html_escape, tool_category};
use compliance_core::models::pentest::{AuthMode, PentestConfig, PentestSession};
pub(super) fn scope(
session: &PentestSession,
target_name: &str,
target_url: &str,
date_str: &str,
completed_str: &str,
tool_names: &[String],
config: Option<&PentestConfig>,
) -> String {
let tools_table: String = tool_names
.iter()
.enumerate()
.map(|(i, t)| {
let category = tool_category(t);
format!(
"<tr><td>{}</td><td><code>{}</code></td><td>{}</td></tr>",
i + 1,
html_escape(t),
category,
)
})
.collect::<Vec<_>>()
.join("\n");
let engagement_config_section = if let Some(cfg) = config {
let mut rows = String::new();
rows.push_str(&format!(
"<tr><td>Environment</td><td>{}</td></tr>",
html_escape(&cfg.environment.to_string())
));
if let Some(ref app_type) = cfg.app_type {
rows.push_str(&format!(
"<tr><td>Application Type</td><td>{}</td></tr>",
html_escape(app_type)
));
}
let auth_mode = match cfg.auth.mode {
AuthMode::None => "No authentication",
AuthMode::Manual => "Manual credentials",
AuthMode::AutoRegister => "Auto-register",
};
rows.push_str(&format!("<tr><td>Auth Mode</td><td>{auth_mode}</td></tr>"));
if !cfg.scope_exclusions.is_empty() {
let excl = cfg
.scope_exclusions
.iter()
.map(|s| html_escape(s))
.collect::<Vec<_>>()
.join(", ");
rows.push_str(&format!(
"<tr><td>Scope Exclusions</td><td><code>{excl}</code></td></tr>"
));
}
if !cfg.tester.name.is_empty() {
rows.push_str(&format!(
"<tr><td>Tester</td><td>{} ({})</td></tr>",
html_escape(&cfg.tester.name),
html_escape(&cfg.tester.email)
));
}
if let Some(ref ts) = cfg.disclaimer_accepted_at {
rows.push_str(&format!(
"<tr><td>Disclaimer Accepted</td><td>{}</td></tr>",
ts.format("%B %d, %Y at %H:%M UTC")
));
}
if let Some(ref branch) = cfg.branch {
rows.push_str(&format!(
"<tr><td>Git Branch</td><td>{}</td></tr>",
html_escape(branch)
));
}
if let Some(ref commit) = cfg.commit_hash {
rows.push_str(&format!(
"<tr><td>Git Commit</td><td><code>{}</code></td></tr>",
html_escape(commit)
));
}
format!("<h3>Engagement Configuration</h3>\n<table class=\"info\">\n{rows}\n</table>")
} else {
String::new()
};
format!(
r##"
<!-- ═══════════════ 2. SCOPE & METHODOLOGY ═══════════════ -->
<div class="page-break"></div>
<h2><span class="section-num">2.</span> Scope &amp; Methodology</h2>
<p>
The assessment was performed using an AI-driven orchestrator that autonomously selects and
executes security testing tools based on the target's attack surface, technology stack, and
any available static analysis (SAST) findings and SBOM data.
</p>
<h3>Engagement Details</h3>
<table class="info">
<tr><td>Target</td><td><strong>{target_name}</strong></td></tr>
<tr><td>URL</td><td><code>{target_url}</code></td></tr>
<tr><td>Strategy</td><td>{strategy}</td></tr>
<tr><td>Status</td><td>{status}</td></tr>
<tr><td>Started</td><td>{date_str}</td></tr>
<tr><td>Completed</td><td>{completed_str}</td></tr>
<tr><td>Tool Invocations</td><td>{tool_invocations} ({tool_successes} successful, {success_rate:.1}% success rate)</td></tr>
</table>
{engagement_config_section}
<h3>Tools Employed</h3>
<table class="tools-table">
<thead><tr><th>#</th><th>Tool</th><th>Category</th></tr></thead>
<tbody>{tools_table}</tbody>
</table>"##,
target_name = html_escape(target_name),
target_url = html_escape(target_url),
strategy = session.strategy,
status = session.status,
date_str = date_str,
completed_str = completed_str,
tool_invocations = session.tool_invocations,
tool_successes = session.tool_successes,
success_rate = session.success_rate(),
)
}
@@ -0,0 +1,889 @@
pub(super) fn styles() -> String {
r##"<style>
/* ──────────────── Base / Print-first ──────────────── */
@page {
size: A4;
margin: 20mm 18mm 25mm 18mm;
}
@page :first {
margin: 0;
}
*, *::before, *::after { margin: 0; padding: 0; box-sizing: border-box; }
:root {
--text: #1a1a2e;
--text-secondary: #475569;
--text-muted: #64748b;
--heading: #0d2137;
--accent: #1a56db;
--accent-light: #dbeafe;
--border: #d1d5db;
--border-light: #e5e7eb;
--bg-subtle: #f8fafc;
--bg-section: #f1f5f9;
--sev-critical: #991b1b;
--sev-high: #c2410c;
--sev-medium: #a16207;
--sev-low: #1d4ed8;
--sev-info: #4b5563;
--font-serif: 'Libre Baskerville', 'Georgia', serif;
--font-sans: 'Source Sans 3', 'Helvetica Neue', sans-serif;
--font-mono: 'JetBrains Mono', 'Consolas', monospace;
}
body {
font-family: var(--font-sans);
color: var(--text);
background: #fff;
line-height: 1.65;
font-size: 10.5pt;
-webkit-print-color-adjust: exact;
print-color-adjust: exact;
}
.report-body {
max-width: 190mm;
margin: 0 auto;
padding: 0 16px;
}
/* ──────────────── Cover Page ──────────────── */
.cover {
height: 100vh;
min-height: 297mm;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
text-align: center;
padding: 40mm 30mm;
page-break-after: always;
break-after: page;
position: relative;
background: #fff;
}
.cover-shield {
width: 72px;
height: 72px;
margin-bottom: 32px;
}
.cover-tag {
display: inline-block;
background: var(--sev-critical);
color: #fff;
font-family: var(--font-sans);
font-size: 8pt;
font-weight: 700;
letter-spacing: 0.15em;
text-transform: uppercase;
padding: 4px 16px;
border-radius: 2px;
margin-bottom: 28px;
}
.cover-title {
font-family: var(--font-serif);
font-size: 28pt;
font-weight: 700;
color: var(--heading);
line-height: 1.2;
margin-bottom: 8px;
}
.cover-subtitle {
font-family: var(--font-serif);
font-size: 14pt;
color: var(--text-secondary);
font-weight: 400;
font-style: italic;
margin-bottom: 48px;
}
.cover-meta {
font-size: 10pt;
color: var(--text-secondary);
line-height: 2;
}
.cover-meta strong {
color: var(--text);
}
.cover-divider {
width: 60px;
height: 2px;
background: var(--accent);
margin: 24px auto;
}
.cover-footer {
position: absolute;
bottom: 30mm;
left: 0;
right: 0;
text-align: center;
font-size: 8pt;
color: var(--text-muted);
letter-spacing: 0.05em;
}
/* ──────────────── Typography ──────────────── */
h2 {
font-family: var(--font-serif);
font-size: 16pt;
font-weight: 700;
color: var(--heading);
margin: 36px 0 16px;
padding-bottom: 8px;
border-bottom: 2px solid var(--heading);
page-break-after: avoid;
break-after: avoid;
}
h3 {
font-family: var(--font-serif);
font-size: 12pt;
font-weight: 700;
color: var(--heading);
margin: 24px 0 10px;
page-break-after: avoid;
break-after: avoid;
}
h4 {
font-family: var(--font-sans);
font-size: 10pt;
font-weight: 700;
color: var(--text-secondary);
margin: 16px 0 8px;
}
p {
margin: 8px 0;
font-size: 10.5pt;
}
code {
font-family: var(--font-mono);
font-size: 9pt;
background: var(--bg-section);
padding: 1px 5px;
border-radius: 3px;
border: 1px solid var(--border-light);
word-break: break-all;
}
/* ──────────────── Section Numbers ──────────────── */
.section-num {
color: var(--accent);
margin-right: 8px;
}
/* ──────────────── Table of Contents ──────────────── */
.toc {
page-break-after: always;
break-after: page;
padding-top: 24px;
}
.toc h2 {
border-bottom-color: var(--accent);
margin-top: 0;
}
.toc-entry {
display: flex;
justify-content: space-between;
align-items: baseline;
padding: 6px 0;
border-bottom: 1px dotted var(--border);
font-size: 11pt;
}
.toc-entry .toc-num {
font-weight: 700;
color: var(--accent);
min-width: 24px;
margin-right: 10px;
}
.toc-entry .toc-label {
flex: 1;
font-weight: 600;
color: var(--heading);
}
.toc-sub {
padding: 3px 0 3px 34px;
font-size: 9.5pt;
color: var(--text-secondary);
}
/* ──────────────── Executive Summary ──────────────── */
.exec-grid {
display: grid;
grid-template-columns: 1fr 1fr 1fr 1fr;
gap: 12px;
margin: 16px 0 20px;
}
.kpi-card {
border: 1px solid var(--border);
border-radius: 6px;
padding: 14px 12px;
text-align: center;
background: var(--bg-subtle);
}
.kpi-value {
font-family: var(--font-serif);
font-size: 22pt;
font-weight: 700;
line-height: 1.1;
}
.kpi-label {
font-size: 8pt;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--text-muted);
margin-top: 4px;
font-weight: 600;
}
/* Risk gauge */
.risk-gauge {
display: flex;
align-items: center;
gap: 16px;
padding: 16px 20px;
border: 1px solid var(--border);
border-radius: 6px;
background: var(--bg-subtle);
margin: 16px 0;
}
.risk-gauge-meter {
width: 140px;
flex-shrink: 0;
}
.risk-gauge-track {
height: 10px;
background: var(--border-light);
border-radius: 5px;
overflow: hidden;
position: relative;
}
.risk-gauge-fill {
height: 100%;
border-radius: 5px;
transition: width 0.3s;
}
.risk-gauge-score {
font-family: var(--font-serif);
font-size: 9pt;
font-weight: 700;
text-align: center;
margin-top: 3px;
}
.risk-gauge-text {
flex: 1;
}
.risk-gauge-label {
font-family: var(--font-serif);
font-size: 14pt;
font-weight: 700;
}
.risk-gauge-desc {
font-size: 9.5pt;
color: var(--text-secondary);
margin-top: 2px;
}
/* Severity bar */
.sev-bar {
display: flex;
height: 28px;
border-radius: 4px;
overflow: hidden;
margin: 12px 0 6px;
border: 1px solid var(--border);
}
.sev-bar-seg {
display: flex;
align-items: center;
justify-content: center;
color: #fff;
font-size: 8.5pt;
font-weight: 700;
min-width: 24px;
}
.sev-bar-critical { background: var(--sev-critical); }
.sev-bar-high { background: var(--sev-high); }
.sev-bar-medium { background: var(--sev-medium); }
.sev-bar-low { background: var(--sev-low); }
.sev-bar-info { background: var(--sev-info); }
.sev-bar-legend {
display: flex;
gap: 16px;
font-size: 8.5pt;
color: var(--text-secondary);
margin-bottom: 16px;
}
.sev-dot {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 2px;
margin-right: 4px;
vertical-align: middle;
}
/* ──────────────── Info Tables ──────────────── */
table.info {
width: 100%;
border-collapse: collapse;
margin: 10px 0;
font-size: 10pt;
}
table.info td,
table.info th {
padding: 7px 12px;
border: 1px solid var(--border);
text-align: left;
vertical-align: top;
}
table.info td:first-child,
table.info th:first-child {
width: 160px;
font-weight: 600;
color: var(--text-secondary);
background: var(--bg-subtle);
}
/* Methodology tools table */
table.tools-table {
width: 100%;
border-collapse: collapse;
margin: 10px 0;
font-size: 10pt;
}
table.tools-table th {
background: var(--heading);
color: #fff;
padding: 8px 12px;
text-align: left;
font-weight: 600;
font-size: 9pt;
text-transform: uppercase;
letter-spacing: 0.04em;
}
table.tools-table td {
padding: 6px 12px;
border-bottom: 1px solid var(--border-light);
}
table.tools-table tr:nth-child(even) td {
background: var(--bg-subtle);
}
table.tools-table td:first-child {
width: 32px;
text-align: center;
color: var(--text-muted);
font-weight: 600;
}
/* ──────────────── Badges ──────────────── */
.badge {
display: inline-block;
padding: 2px 8px;
border-radius: 3px;
font-size: 7.5pt;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.03em;
vertical-align: middle;
}
.badge-exploit {
background: var(--sev-critical);
color: #fff;
}
/* ──────────────── Findings ──────────────── */
.sev-group-title {
font-family: var(--font-sans);
font-size: 11pt;
font-weight: 700;
color: var(--heading);
padding: 8px 0 6px 12px;
margin: 20px 0 8px;
border-left: 4px solid;
page-break-after: avoid;
break-after: avoid;
}
.finding {
border: 1px solid var(--border);
border-left: 4px solid;
border-radius: 0 4px 4px 0;
padding: 14px 16px;
margin-bottom: 12px;
background: #fff;
page-break-inside: avoid;
break-inside: avoid;
}
.finding-header {
display: flex;
align-items: center;
gap: 10px;
margin-bottom: 8px;
}
.finding-id {
font-family: var(--font-mono);
font-size: 9pt;
font-weight: 500;
color: var(--text-muted);
background: var(--bg-section);
padding: 2px 6px;
border-radius: 3px;
border: 1px solid var(--border-light);
}
.finding-title {
font-family: var(--font-serif);
font-weight: 700;
font-size: 11pt;
flex: 1;
color: var(--heading);
}
.finding-meta {
border-collapse: collapse;
margin: 6px 0;
font-size: 9.5pt;
width: 100%;
}
.finding-meta td {
padding: 3px 10px 3px 0;
vertical-align: top;
}
.finding-meta td:first-child {
color: var(--text-muted);
font-weight: 600;
width: 90px;
white-space: nowrap;
}
.finding-desc {
margin: 8px 0;
font-size: 10pt;
color: var(--text);
line-height: 1.6;
}
.remediation {
margin-top: 10px;
padding: 10px 14px;
background: var(--accent-light);
border-left: 3px solid var(--accent);
border-radius: 0 4px 4px 0;
font-size: 9.5pt;
line-height: 1.55;
}
.remediation-label {
font-weight: 700;
font-size: 8.5pt;
text-transform: uppercase;
letter-spacing: 0.06em;
color: var(--accent);
margin-bottom: 3px;
}
.evidence-block {
margin: 10px 0;
page-break-inside: avoid;
break-inside: avoid;
}
.evidence-title {
font-weight: 700;
font-size: 8.5pt;
text-transform: uppercase;
letter-spacing: 0.06em;
color: var(--text-muted);
margin-bottom: 4px;
}
.evidence-table {
width: 100%;
border-collapse: collapse;
font-size: 9pt;
}
.evidence-table th {
background: var(--bg-section);
padding: 5px 8px;
text-align: left;
font-weight: 600;
font-size: 8.5pt;
text-transform: uppercase;
letter-spacing: 0.03em;
color: var(--text-secondary);
border: 1px solid var(--border-light);
}
.evidence-table td {
padding: 5px 8px;
border: 1px solid var(--border-light);
vertical-align: top;
word-break: break-word;
}
.evidence-payload {
font-size: 8.5pt;
color: var(--sev-critical);
}
.linked-sast {
font-size: 9pt;
color: var(--text-muted);
margin: 6px 0;
font-style: italic;
}
/* ──────────────── Code-Level Correlation ──────────────── */
.code-correlation {
margin: 12px 0;
border: 1px solid #e2e8f0;
border-radius: 6px;
overflow: hidden;
}
.code-correlation-title {
background: #1e293b;
color: #f8fafc;
padding: 6px 12px;
font-size: 9pt;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.05em;
}
.code-correlation-item {
padding: 10px 12px;
border-bottom: 1px solid #e2e8f0;
}
.code-correlation-item:last-child { border-bottom: none; }
.code-correlation-badge {
display: inline-block;
background: #3b82f6;
color: #fff;
font-size: 7pt;
font-weight: 600;
padding: 2px 8px;
border-radius: 3px;
text-transform: uppercase;
letter-spacing: 0.04em;
margin-bottom: 6px;
}
.code-meta {
width: 100%;
font-size: 8.5pt;
border-collapse: collapse;
margin-bottom: 6px;
}
.code-meta td:first-child {
width: 80px;
font-weight: 600;
color: var(--text-muted);
padding: 2px 8px 2px 0;
vertical-align: top;
}
.code-meta td:last-child {
padding: 2px 0;
}
.code-snippet-block, .code-fix-block {
margin: 6px 0;
}
.code-snippet-label, .code-fix-label {
font-size: 7.5pt;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--text-muted);
margin-bottom: 3px;
}
.code-snippet-label { color: #dc2626; }
.code-fix-label { color: #16a34a; }
.code-snippet {
background: #fef2f2;
border: 1px solid #fecaca;
border-left: 3px solid #dc2626;
padding: 8px 10px;
font-family: 'JetBrains Mono', monospace;
font-size: 8pt;
line-height: 1.5;
white-space: pre-wrap;
word-break: break-word;
border-radius: 0 4px 4px 0;
margin: 0;
}
.code-fix {
background: #f0fdf4;
border: 1px solid #bbf7d0;
border-left: 3px solid #16a34a;
padding: 8px 10px;
font-family: 'JetBrains Mono', monospace;
font-size: 8pt;
line-height: 1.5;
white-space: pre-wrap;
word-break: break-word;
border-radius: 0 4px 4px 0;
margin: 0;
}
.code-remediation {
font-size: 8.5pt;
color: var(--text-secondary);
margin-top: 4px;
padding: 4px 0;
}
.code-linked-vulns {
font-size: 8.5pt;
margin-top: 4px;
}
.code-linked-vulns ul {
margin: 2px 0 0 16px;
padding: 0;
}
.code-linked-vulns li {
margin-bottom: 2px;
}
/* ──────────────── Attack Chain ──────────────── */
.phase-block {
margin-bottom: 20px;
page-break-inside: avoid;
break-inside: avoid;
}
.phase-header {
display: flex;
align-items: center;
gap: 10px;
padding: 8px 14px;
background: var(--heading);
color: #fff;
border-radius: 4px 4px 0 0;
font-size: 9.5pt;
}
.phase-num {
font-weight: 700;
font-size: 8pt;
text-transform: uppercase;
letter-spacing: 0.1em;
background: rgba(255,255,255,0.15);
padding: 2px 8px;
border-radius: 3px;
}
.phase-label {
font-weight: 600;
flex: 1;
}
.phase-count {
font-size: 8.5pt;
opacity: 0.7;
}
.phase-steps {
border: 1px solid var(--border);
border-top: none;
border-radius: 0 0 4px 4px;
}
.step-row {
display: flex;
align-items: flex-start;
gap: 10px;
padding: 8px 14px;
border-bottom: 1px solid var(--border-light);
position: relative;
}
.step-row:last-child {
border-bottom: none;
}
.step-num {
width: 22px;
height: 22px;
border-radius: 50%;
background: var(--bg-section);
border: 1px solid var(--border);
display: flex;
align-items: center;
justify-content: center;
font-size: 8pt;
font-weight: 700;
color: var(--text-secondary);
flex-shrink: 0;
margin-top: 1px;
}
.step-connector {
display: none;
}
.step-content {
flex: 1;
min-width: 0;
}
.step-header {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
}
.step-tool {
font-family: var(--font-mono);
font-size: 9.5pt;
font-weight: 500;
color: var(--heading);
}
.step-status {
font-size: 7.5pt;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.04em;
padding: 1px 7px;
border-radius: 3px;
}
.step-completed { background: #dcfce7; color: #166534; }
.step-failed { background: #fef2f2; color: #991b1b; }
.step-running { background: #fef9c3; color: #854d0e; }
.step-findings {
font-size: 8pt;
font-weight: 600;
color: var(--sev-high);
background: #fff7ed;
padding: 1px 7px;
border-radius: 3px;
border: 1px solid #fed7aa;
}
.step-risk {
font-size: 7.5pt;
font-weight: 700;
padding: 1px 6px;
border-radius: 3px;
}
.risk-high { background: #fef2f2; color: var(--sev-critical); border: 1px solid #fecaca; }
.risk-med { background: #fffbeb; color: var(--sev-medium); border: 1px solid #fde68a; }
.risk-low { background: #f0fdf4; color: #166534; border: 1px solid #bbf7d0; }
.step-reasoning {
font-size: 9pt;
color: var(--text-muted);
margin-top: 3px;
line-height: 1.5;
font-style: italic;
}
/* ──────────────── Footer ──────────────── */
.report-footer {
margin-top: 48px;
padding-top: 14px;
border-top: 2px solid var(--heading);
font-size: 8pt;
color: var(--text-muted);
text-align: center;
line-height: 1.8;
}
.report-footer .footer-company {
font-weight: 700;
color: var(--text-secondary);
}
/* ──────────────── Page Break Utilities ──────────────── */
.page-break {
page-break-before: always;
break-before: page;
}
.avoid-break {
page-break-inside: avoid;
break-inside: avoid;
}
/* ──────────────── Print Overrides ──────────────── */
@media print {
body {
font-size: 10pt;
}
.cover {
height: auto;
min-height: 250mm;
padding: 50mm 20mm;
}
.report-body {
padding: 0;
}
.no-print {
display: none !important;
}
a {
color: var(--accent);
text-decoration: none;
}
}
/* ──────────────── Screen Enhancements ──────────────── */
@media screen {
body {
background: #e2e8f0;
}
.cover {
background: #fff;
box-shadow: 0 1px 4px rgba(0,0,0,0.08);
}
.report-body {
background: #fff;
padding: 20px 32px 40px;
box-shadow: 0 1px 4px rgba(0,0,0,0.08);
margin-bottom: 40px;
}
}
</style>"##
.to_string()
}
@@ -0,0 +1,69 @@
mod archive;
mod html;
mod pdf;
use compliance_core::models::dast::DastFinding;
use compliance_core::models::finding::Finding;
use compliance_core::models::pentest::{
AttackChainNode, CodeContextHint, PentestConfig, PentestSession,
};
use compliance_core::models::sbom::SbomEntry;
use sha2::{Digest, Sha256};
/// Report archive with metadata
pub struct ReportArchive {
/// The password-protected ZIP bytes
pub archive: Vec<u8>,
/// SHA-256 hex digest of the archive
pub sha256: String,
}
/// Report context gathered from the database
pub struct ReportContext {
pub session: PentestSession,
pub target_name: String,
pub target_url: String,
pub findings: Vec<DastFinding>,
pub attack_chain: Vec<AttackChainNode>,
pub requester_name: String,
pub requester_email: String,
pub config: Option<PentestConfig>,
/// SAST findings for the linked repository (for code-level correlation)
pub sast_findings: Vec<Finding>,
/// Vulnerable dependencies from SBOM
pub sbom_entries: Vec<SbomEntry>,
/// Code knowledge graph entry points linked to SAST findings
pub code_context: Vec<CodeContextHint>,
}
/// Generate a password-protected ZIP archive containing the pentest report.
///
/// The archive contains:
/// - `report.pdf` — Professional pentest report (PDF)
/// - `report.html` — HTML source (fallback)
/// - `findings.json` — Raw findings data
/// - `attack-chain.json` — Attack chain timeline
///
/// Files are encrypted with AES-256 inside the ZIP (standard WinZip AES format,
/// supported by 7-Zip, WinRAR, macOS Archive Utility, etc.).
pub async fn generate_encrypted_report(
ctx: &ReportContext,
password: &str,
) -> Result<ReportArchive, String> {
let html = html::build_html_report(ctx);
// Convert HTML to PDF via headless Chrome
let pdf_bytes = pdf::html_to_pdf(&html).await?;
let zip_bytes = archive::build_zip(ctx, password, &html, &pdf_bytes)
.map_err(|e| format!("Failed to create archive: {e}"))?;
let mut hasher = Sha256::new();
hasher.update(&zip_bytes);
let sha256 = hex::encode(hasher.finalize());
Ok(ReportArchive {
archive: zip_bytes,
sha256,
})
}
+289
View File
@@ -0,0 +1,289 @@
use futures_util::SinkExt;
use tokio_tungstenite::tungstenite::Message;
type WsStream =
tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>>;
/// Convert HTML string to PDF bytes.
///
/// If `CHROME_WS_URL` is set (e.g. `ws://host:3000`), connects to a remote
/// headless Chrome via the Chrome DevTools Protocol over WebSocket.
/// Otherwise falls back to a local Chrome/Chromium binary.
pub(super) async fn html_to_pdf(html: &str) -> Result<Vec<u8>, String> {
if let Ok(ws_url) = std::env::var("CHROME_WS_URL") {
tracing::info!(url = %ws_url, "Generating PDF via remote Chrome (CDP)");
cdp_print_to_pdf(&ws_url, html).await
} else {
tracing::info!("Generating PDF via local Chrome binary");
local_chrome_pdf(html).await
}
}
/// Send a CDP command (no session) and return the response.
async fn cdp_send(
ws: &mut WsStream,
id: u64,
method: &str,
params: serde_json::Value,
) -> Result<serde_json::Value, String> {
let msg = serde_json::json!({ "id": id, "method": method, "params": params });
ws.send(Message::Text(msg.to_string().into()))
.await
.map_err(|e| format!("WS send failed: {e}"))?;
read_until_result(ws, id).await
}
/// Send a CDP command on a session and return the response.
async fn cdp_send_session(
ws: &mut WsStream,
id: u64,
session_id: &str,
method: &str,
params: serde_json::Value,
) -> Result<serde_json::Value, String> {
let msg = serde_json::json!({
"id": id,
"sessionId": session_id,
"method": method,
"params": params,
});
ws.send(Message::Text(msg.to_string().into()))
.await
.map_err(|e| format!("WS send failed: {e}"))?;
read_until_result(ws, id).await
}
/// Generate PDF by connecting to a remote Chrome instance over CDP WebSocket.
async fn cdp_print_to_pdf(base_ws_url: &str, html: &str) -> Result<Vec<u8>, String> {
use base64::Engine;
// Step 1: Discover browser WS endpoint via /json/version
let http_url = base_ws_url
.replace("ws://", "http://")
.replace("wss://", "https://");
let version_url = format!("{http_url}/json/version");
let version: serde_json::Value = reqwest::get(&version_url)
.await
.map_err(|e| format!("Failed to reach Chrome at {version_url}: {e}"))?
.json()
.await
.map_err(|e| format!("Invalid /json/version response: {e}"))?;
let browser_ws = version["webSocketDebuggerUrl"]
.as_str()
.ok_or_else(|| "No webSocketDebuggerUrl in /json/version".to_string())?;
// Step 2: Connect to browser WS endpoint
let (mut ws, _) = tokio_tungstenite::connect_async(browser_ws)
.await
.map_err(|e| format!("WebSocket connect failed: {e}"))?;
let mut id: u64 = 1;
// Step 3: Create a new target (tab)
let resp = cdp_send(
&mut ws,
id,
"Target.createTarget",
serde_json::json!({ "url": "about:blank" }),
)
.await?;
id += 1;
let target_id = resp
.get("result")
.and_then(|r| r.get("targetId"))
.and_then(|t| t.as_str())
.ok_or("No targetId in createTarget response")?
.to_string();
// Step 4: Attach to target
let resp = cdp_send(
&mut ws,
id,
"Target.attachToTarget",
serde_json::json!({ "targetId": target_id, "flatten": true }),
)
.await?;
id += 1;
let session_id = resp
.get("result")
.and_then(|r| r.get("sessionId"))
.and_then(|s| s.as_str())
.ok_or("No sessionId in attachToTarget response")?
.to_string();
// Step 5: Enable Page domain
cdp_send_session(
&mut ws,
id,
&session_id,
"Page.enable",
serde_json::json!({}),
)
.await?;
id += 1;
// Step 6: Set page content with the HTML
cdp_send_session(
&mut ws,
id,
&session_id,
"Page.setDocumentContent",
serde_json::json!({ "frameId": target_id, "html": html }),
)
.await?;
id += 1;
// Brief pause for rendering
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
// Step 7: Print to PDF
let pdf_response = cdp_send_session(
&mut ws,
id,
&session_id,
"Page.printToPDF",
serde_json::json!({
"printBackground": true,
"preferCSSPageSize": true,
"displayHeaderFooter": false,
}),
)
.await?;
id += 1;
let pdf_b64 = pdf_response
.get("result")
.and_then(|r| r.get("data"))
.and_then(|d| d.as_str())
.ok_or("No PDF data in printToPDF response")?;
let pdf_bytes = base64::engine::general_purpose::STANDARD
.decode(pdf_b64)
.map_err(|e| format!("Failed to decode PDF base64: {e}"))?;
// Step 8: Close the target
let _ = cdp_send(
&mut ws,
id,
"Target.closeTarget",
serde_json::json!({ "targetId": target_id }),
)
.await;
let _ = ws.close(None).await;
if pdf_bytes.is_empty() {
return Err("Chrome produced an empty PDF".to_string());
}
tracing::info!(
size_kb = pdf_bytes.len() / 1024,
"PDF report generated via CDP"
);
Ok(pdf_bytes)
}
/// Read WebSocket messages until we get a response matching the given id.
async fn read_until_result(ws: &mut WsStream, id: u64) -> Result<serde_json::Value, String> {
use futures_util::StreamExt;
let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(30);
loop {
let msg = tokio::time::timeout_at(deadline, ws.next())
.await
.map_err(|_| format!("Timeout waiting for CDP response id={id}"))?
.ok_or_else(|| "WebSocket closed unexpectedly".to_string())?
.map_err(|e| format!("WebSocket read error: {e}"))?;
if let Message::Text(text) = msg {
if let Ok(val) = serde_json::from_str::<serde_json::Value>(&text) {
if val.get("id").and_then(|i| i.as_u64()) == Some(id) {
if let Some(err) = val.get("error") {
return Err(format!("CDP error: {err}"));
}
return Ok(val);
}
}
}
}
}
/// Fallback: generate PDF using a local Chrome/Chromium binary.
async fn local_chrome_pdf(html: &str) -> Result<Vec<u8>, String> {
let tmp_dir = std::env::temp_dir();
let run_id = uuid::Uuid::new_v4().to_string();
let html_path = tmp_dir.join(format!("pentest-report-{run_id}.html"));
let pdf_path = tmp_dir.join(format!("pentest-report-{run_id}.pdf"));
std::fs::write(&html_path, html).map_err(|e| format!("Failed to write temp HTML: {e}"))?;
let chrome_bin = find_chrome_binary().ok_or_else(|| {
"Chrome/Chromium not found. Set CHROME_WS_URL for remote Chrome or install chromium locally."
.to_string()
})?;
tracing::info!(chrome = %chrome_bin, "Generating PDF report via headless Chrome");
let html_url = format!("file://{}", html_path.display());
let output = tokio::process::Command::new(&chrome_bin)
.args([
"--headless",
"--disable-gpu",
"--no-sandbox",
"--disable-software-rasterizer",
"--run-all-compositor-stages-before-draw",
"--disable-dev-shm-usage",
&format!("--print-to-pdf={}", pdf_path.display()),
"--no-pdf-header-footer",
&html_url,
])
.output()
.await
.map_err(|e| format!("Failed to run Chrome: {e}"))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
let _ = std::fs::remove_file(&html_path);
let _ = std::fs::remove_file(&pdf_path);
return Err(format!("Chrome PDF generation failed: {stderr}"));
}
let pdf_bytes =
std::fs::read(&pdf_path).map_err(|e| format!("Failed to read generated PDF: {e}"))?;
let _ = std::fs::remove_file(&html_path);
let _ = std::fs::remove_file(&pdf_path);
if pdf_bytes.is_empty() {
return Err("Chrome produced an empty PDF".to_string());
}
tracing::info!(size_kb = pdf_bytes.len() / 1024, "PDF report generated");
Ok(pdf_bytes)
}
/// Search for Chrome/Chromium binary on the system.
fn find_chrome_binary() -> Option<String> {
let candidates = [
"google-chrome-stable",
"google-chrome",
"chromium-browser",
"chromium",
];
for name in &candidates {
if let Ok(output) = std::process::Command::new("which").arg(name).output() {
if output.status.success() {
let path = String::from_utf8_lossy(&output.stdout).trim().to_string();
if !path.is_empty() {
return Some(path);
}
}
}
}
None
}
+51 -1
View File
@@ -66,8 +66,10 @@ impl CodeReviewScanner {
}
}
let deduped = dedup_cross_pass(all_findings);
ScanOutput {
findings: all_findings,
findings: deduped,
sbom_entries: Vec::new(),
}
}
@@ -184,3 +186,51 @@ struct ReviewIssue {
#[serde(default)]
suggestion: Option<String>,
}
/// Deduplicate findings across review passes.
///
/// Multiple passes often flag the same issue (e.g. SQL injection reported by
/// logic, security, and convention passes). We group by file + nearby line +
/// normalized title keywords and keep the highest-severity finding.
fn dedup_cross_pass(findings: Vec<Finding>) -> Vec<Finding> {
use std::collections::HashMap;
// Build a dedup key: (file, line bucket, normalized title words)
fn dedup_key(f: &Finding) -> String {
let file = f.file_path.as_deref().unwrap_or("");
// Group lines within 3 of each other
let line_bucket = f.line_number.unwrap_or(0) / 4;
// Normalize: lowercase, keep only alphanumeric, sort words for order-independence
let title_lower = f.title.to_lowercase();
let mut words: Vec<&str> = title_lower
.split(|c: char| !c.is_alphanumeric())
.filter(|w| w.len() > 2)
.collect();
words.sort();
format!("{file}:{line_bucket}:{}", words.join(","))
}
let mut groups: HashMap<String, Finding> = HashMap::new();
for finding in findings {
let key = dedup_key(&finding);
groups
.entry(key)
.and_modify(|existing| {
// Keep the higher severity; on tie, keep the one with more detail
if finding.severity > existing.severity
|| (finding.severity == existing.severity
&& finding.description.len() > existing.description.len())
{
*existing = finding.clone();
}
// Merge CWE if the existing one is missing it
if existing.cwe.is_none() {
existing.cwe = finding.cwe.clone();
}
})
.or_insert(finding);
}
groups.into_values().collect()
}
+283
View File
@@ -204,6 +204,202 @@ impl CveScanner {
Ok(results)
}
/// Match the CODESYS **runtime** component against NVD by CPE.
///
/// CODESYS advisories (the CoDe16 cluster and friends) are indexed in NVD by
/// CPE (`cpe:2.3:a:codesys:control*`) keyed off the *runtime* version — not by
/// the internal `Cmp*`/`Sys*` library names OSV-by-purl would look up. So we
/// find the runtime SBOM entry, pull every `cpe:2.3:a:codesys:*` CVE from NVD,
/// and keep the ones whose affected-version range covers our runtime version.
/// Best-effort: returns empty without an NVD key, on a network error, or when
/// no CODESYS runtime component is present.
pub async fn scan_codesys(&self, repo_id: &str, entries: &mut [SbomEntry]) -> Vec<CveAlert> {
let Some((name, version)) = codesys_runtime(entries) else {
return Vec::new();
};
let url = "https://services.nvd.nist.gov/rest/json/cves/2.0\
?virtualMatchString=cpe:2.3:a:codesys";
let mut req = self.http.get(url);
if let Some(key) = &self.nvd_api_key {
req = req.header("apiKey", key.as_str());
}
let body: serde_json::Value = match req.send().await {
Ok(r) if r.status().is_success() => match r.json().await {
Ok(b) => b,
Err(e) => {
tracing::warn!("CODESYS NVD parse failed: {e}");
return Vec::new();
}
},
Ok(r) => {
tracing::warn!("CODESYS NVD returned {}", r.status());
return Vec::new();
}
Err(e) => {
tracing::warn!("CODESYS NVD request failed: {e}");
return Vec::new();
}
};
let matched = parse_codesys_nvd(&body, &version);
let mut alerts = Vec::new();
for cve in matched {
if let Some(e) = entries
.iter_mut()
.find(|e| e.name == name && e.version == version)
{
e.known_vulnerabilities.push(VulnRef {
id: cve.id.clone(),
source: "nvd".to_string(),
severity: None,
url: Some(format!("https://nvd.nist.gov/vuln/detail/{}", cve.id)),
});
}
let mut alert = CveAlert::new(
cve.id,
repo_id.to_string(),
name.clone(),
version.clone(),
CveSource::Nvd,
);
alert.summary = cve.summary;
alert.cvss_score = cve.cvss;
alerts.push(alert);
}
tracing::info!(runtime = %name, version = %version, cves = alerts.len(), "CODESYS CVE match");
alerts
}
}
/// The CODESYS runtime component (name + version) from an SBOM, if present. The
/// runtime carries the version CODESYS advisories key off; the internal library
/// components do not.
fn codesys_runtime(entries: &[SbomEntry]) -> Option<(String, String)> {
entries
.iter()
.find(|e| e.package_manager == "codesys" && e.name.starts_with("CODESYS Control"))
.map(|e| (e.name.clone(), e.version.clone()))
}
/// A parsed NVD CVE that affects the CODESYS runtime.
struct CodesysCve {
id: String,
summary: Option<String>,
cvss: Option<f64>,
}
/// Version constraints from an NVD `cpeMatch` node.
#[derive(Default)]
struct CpeRange {
exact: Option<String>,
start_incl: Option<String>,
start_excl: Option<String>,
end_incl: Option<String>,
end_excl: Option<String>,
}
/// Parse an NVD CVE-list response and keep the CVEs whose CODESYS CPE match covers
/// `runtime_version`.
fn parse_codesys_nvd(body: &serde_json::Value, runtime_version: &str) -> Vec<CodesysCve> {
let mut out = Vec::new();
let Some(vulns) = body["vulnerabilities"].as_array() else {
return out;
};
for v in vulns {
let cve = &v["cve"];
let Some(id) = cve["id"].as_str() else {
continue;
};
let covered = cve["configurations"]
.as_array()
.into_iter()
.flatten()
.flat_map(|c| c["nodes"].as_array().into_iter().flatten())
.flat_map(|n| n["cpeMatch"].as_array().into_iter().flatten())
.any(|cm| {
cm["vulnerable"].as_bool() == Some(true)
&& cm["criteria"]
.as_str()
.is_some_and(|c| c.contains(":codesys:"))
&& version_matches(runtime_version, &cpe_range(cm))
});
if covered {
let summary = cve["descriptions"]
.as_array()
.and_then(|d| d.iter().find(|x| x["lang"].as_str() == Some("en")))
.and_then(|x| x["value"].as_str())
.map(String::from);
let cvss = cve["metrics"]["cvssMetricV31"]
.as_array()
.and_then(|m| m.first())
.and_then(|m| m["cvssData"]["baseScore"].as_f64());
out.push(CodesysCve {
id: id.to_string(),
summary,
cvss,
});
}
}
out
}
/// Build a [`CpeRange`] from an NVD `cpeMatch` object.
fn cpe_range(cm: &serde_json::Value) -> CpeRange {
let exact = cm["criteria"]
.as_str()
.and_then(cpe_version)
.filter(|v| v != "*" && v != "-" && !v.is_empty());
CpeRange {
exact,
start_incl: cm["versionStartIncluding"].as_str().map(String::from),
start_excl: cm["versionStartExcluding"].as_str().map(String::from),
end_incl: cm["versionEndIncluding"].as_str().map(String::from),
end_excl: cm["versionEndExcluding"].as_str().map(String::from),
}
}
/// The version field (6th component) of a CPE 2.3 string.
fn cpe_version(criteria: &str) -> Option<String> {
criteria.split(':').nth(5).map(String::from)
}
/// Whether `v` satisfies a CPE version range.
fn version_matches(v: &str, r: &CpeRange) -> bool {
use std::cmp::Ordering::{Equal, Greater, Less};
if let Some(exact) = &r.exact {
return cmp_dotted(v, exact) == Equal;
}
let mut ok = true;
if let Some(s) = &r.start_incl {
ok &= cmp_dotted(v, s) != Less;
}
if let Some(s) = &r.start_excl {
ok &= cmp_dotted(v, s) == Greater;
}
if let Some(e) = &r.end_incl {
ok &= cmp_dotted(v, e) != Greater;
}
if let Some(e) = &r.end_excl {
ok &= cmp_dotted(v, e) == Less;
}
ok
}
/// Compare two dotted numeric versions (`4.17.0.0` vs `4.9.0.0`); missing
/// components count as 0, non-numeric components as 0.
fn cmp_dotted(a: &str, b: &str) -> std::cmp::Ordering {
let pa: Vec<u64> = a.split('.').map(|x| x.parse().unwrap_or(0)).collect();
let pb: Vec<u64> = b.split('.').map(|x| x.parse().unwrap_or(0)).collect();
for i in 0..pa.len().max(pb.len()) {
let x = pa.get(i).copied().unwrap_or(0);
let y = pb.get(i).copied().unwrap_or(0);
match x.cmp(&y) {
std::cmp::Ordering::Equal => continue,
other => return other,
}
}
std::cmp::Ordering::Equal
}
#[derive(serde::Deserialize)]
@@ -228,3 +424,90 @@ struct OsvVuln {
summary: Option<String>,
severity: Option<String>,
}
#[cfg(test)]
mod tests {
use super::*;
use std::cmp::Ordering::{Equal, Greater, Less};
fn entry(name: &str, ver: &str, pm: &str) -> SbomEntry {
SbomEntry::new("t".into(), name.into(), ver.into(), pm.into())
}
#[test]
fn finds_the_codesys_runtime_component() {
let entries = vec![
entry("Standard", "3.5.18.0", "codesys"),
entry("CODESYS Control for Linux ARM SL", "4.17.0.0", "codesys"),
];
assert_eq!(
codesys_runtime(&entries),
Some(("CODESYS Control for Linux ARM SL".into(), "4.17.0.0".into()))
);
// Internal library components are not the runtime.
assert!(codesys_runtime(&[entry("Util", "3.5.21.0", "codesys")]).is_none());
}
#[test]
fn dotted_version_comparison() {
assert_eq!(cmp_dotted("4.17.0.0", "4.9.0.0"), Greater);
assert_eq!(cmp_dotted("4.9.0.0", "4.17.0.0"), Less);
assert_eq!(cmp_dotted("3.5.18.0", "3.5.18.0"), Equal);
assert_eq!(cmp_dotted("4.2", "4.2.0.0"), Equal); // missing components = 0
}
#[test]
fn version_range_matching() {
let end_excl = CpeRange {
end_excl: Some("4.9.0.0".into()),
..Default::default()
};
assert!(!version_matches("4.17.0.0", &end_excl)); // patched
assert!(version_matches("4.5.0.0", &end_excl)); // affected
let exact = CpeRange {
exact: Some("3.5.16.0".into()),
..Default::default()
};
assert!(version_matches("3.5.16.0", &exact));
assert!(!version_matches("3.5.17.0", &exact));
let span = CpeRange {
start_incl: Some("3.0.0.0".into()),
end_incl: Some("3.5.16.0".into()),
..Default::default()
};
assert!(version_matches("3.5.16.0", &span));
assert!(!version_matches("3.5.17.0", &span));
}
#[test]
fn parses_nvd_and_matches_by_runtime_version() {
// Two CODESYS CVEs: one affects < 4.9 (our 4.17 is patched), one affects
// <= 4.20 (our 4.17 is affected). Only the latter should match.
let body = serde_json::json!({
"vulnerabilities": [
{"cve": {"id":"CVE-2023-0001",
"descriptions":[{"lang":"en","value":"old CmpBlkDrvTcp bug"}],
"metrics":{"cvssMetricV31":[{"cvssData":{"baseScore":7.5}}]},
"configurations":[{"nodes":[{"cpeMatch":[
{"vulnerable":true,
"criteria":"cpe:2.3:a:codesys:control_for_linux_sl:*:*:*:*:*:*:*:*",
"versionEndExcluding":"4.9.0.0"}
]}]}]}},
{"cve": {"id":"CVE-2024-0002",
"descriptions":[{"lang":"en","value":"recent runtime bug"}],
"metrics":{"cvssMetricV31":[{"cvssData":{"baseScore":9.8}}]},
"configurations":[{"nodes":[{"cpeMatch":[
{"vulnerable":true,
"criteria":"cpe:2.3:a:codesys:control_for_linux_sl:*:*:*:*:*:*:*:*",
"versionEndIncluding":"4.20.0.0"}
]}]}]}}
]
});
let matched = parse_codesys_nvd(&body, "4.17.0.0");
let ids: Vec<&str> = matched.iter().map(|c| c.id.as_str()).collect();
assert_eq!(ids, vec!["CVE-2024-0002"]);
assert_eq!(matched[0].cvss, Some(9.8));
}
}
+405
View File
@@ -1,5 +1,7 @@
use sha2::{Digest, Sha256};
use compliance_core::models::dast::DastFinding;
pub fn compute_fingerprint(parts: &[&str]) -> String {
let mut hasher = Sha256::new();
for part in parts {
@@ -8,3 +10,406 @@ pub fn compute_fingerprint(parts: &[&str]) -> String {
}
hex::encode(hasher.finalize())
}
/// Compute a dedup fingerprint for a DAST finding.
///
/// The key is derived from the *canonicalized* title (lowercased, domain names
/// stripped, known synonyms resolved), endpoint, and HTTP method. This lets us
/// detect both exact duplicates (same tool reporting twice across passes) and
/// semantic duplicates (e.g., `security_header_missing` "Missing HSTS header"
/// vs `tls_misconfiguration` "Missing strict-transport-security header").
pub fn compute_dast_fingerprint(f: &DastFinding) -> String {
let canon = canonicalize_dast_title(&f.title);
let endpoint = f.endpoint.to_lowercase().trim_end_matches('/').to_string();
let method = f.method.to_uppercase();
let param = f.parameter.as_deref().unwrap_or("");
compute_fingerprint(&[&canon, &endpoint, &method, param])
}
/// Canonicalize a DAST finding title for dedup purposes.
///
/// 1. Lowercase
/// 2. Strip domain names / URLs (e.g. "for comp-dev.meghsakha.com")
/// 3. Resolve known header synonyms (hsts ↔ strict-transport-security, etc.)
/// 4. Strip extra whitespace
fn canonicalize_dast_title(title: &str) -> String {
let mut s = title.to_lowercase();
// Strip "for <domain>" or "on <domain>" suffixes
// Pattern: "for <word.word...>" or "on <method> <url>"
if let Some(idx) = s.find(" for ") {
// Check if what follows looks like a domain or URL
let rest = &s[idx + 5..];
if rest.contains('.') || rest.starts_with("http") {
s.truncate(idx);
}
}
if let Some(idx) = s.find(" on ") {
let rest = &s[idx + 4..];
if rest.contains("http") || rest.contains('/') {
s.truncate(idx);
}
}
// Resolve known header synonyms
let synonyms: &[(&str, &str)] = &[
("hsts", "strict-transport-security"),
("csp", "content-security-policy"),
("cors", "cross-origin-resource-sharing"),
("xfo", "x-frame-options"),
];
for &(short, canonical) in synonyms {
// Only replace whole words — check boundaries
if let Some(pos) = s.find(short) {
let before_ok = pos == 0 || !s.as_bytes()[pos - 1].is_ascii_alphanumeric();
let after_ok = pos + short.len() >= s.len()
|| !s.as_bytes()[pos + short.len()].is_ascii_alphanumeric();
if before_ok && after_ok {
s = format!("{}{}{}", &s[..pos], canonical, &s[pos + short.len()..]);
}
}
}
// Collapse whitespace
s.split_whitespace().collect::<Vec<_>>().join(" ")
}
/// Deduplicate a list of DAST findings, merging evidence from duplicates.
///
/// Two-phase approach:
/// 1. **Exact dedup** — group by canonicalized `(title, endpoint, method, parameter)`.
/// Merge evidence arrays, keep the highest severity, preserve exploitable flag.
/// 2. **CWE-based dedup** — within the same `(cwe, endpoint, method)` group, merge
/// findings whose canonicalized titles resolve to the same subject (e.g., HSTS
/// reported as both `security_header_missing` and `tls_misconfiguration`).
pub fn dedup_dast_findings(findings: Vec<DastFinding>) -> Vec<DastFinding> {
use std::collections::HashMap;
if findings.len() <= 1 {
return findings;
}
// Phase 1: exact fingerprint dedup
let mut seen: HashMap<String, usize> = HashMap::new();
let mut deduped: Vec<DastFinding> = Vec::new();
for finding in findings {
let fp = compute_dast_fingerprint(&finding);
if let Some(&idx) = seen.get(&fp) {
// Merge into existing
merge_dast_finding(&mut deduped[idx], &finding);
} else {
seen.insert(fp, deduped.len());
deduped.push(finding);
}
}
let before = deduped.len();
// Phase 2: CWE-based related dedup
// Group by (cwe, endpoint_normalized, method) — only when CWE is present
let mut cwe_groups: HashMap<String, Vec<usize>> = HashMap::new();
for (i, f) in deduped.iter().enumerate() {
if let Some(ref cwe) = f.cwe {
let key = format!(
"{}|{}|{}",
cwe,
f.endpoint.to_lowercase().trim_end_matches('/'),
f.method.to_uppercase(),
);
cwe_groups.entry(key).or_default().push(i);
}
}
// For each CWE group with multiple findings, keep the one with highest severity
// and most evidence, merge the rest into it
let mut merge_map: HashMap<usize, Vec<usize>> = HashMap::new();
let mut remove_indices: Vec<usize> = Vec::new();
for indices in cwe_groups.values() {
if indices.len() <= 1 {
continue;
}
// Find the "primary" finding: highest severity, then most evidence, then longest description
let Some(&primary_idx) = indices.iter().max_by(|&&a, &&b| {
deduped[a]
.severity
.cmp(&deduped[b].severity)
.then_with(|| deduped[a].evidence.len().cmp(&deduped[b].evidence.len()))
.then_with(|| {
deduped[a]
.description
.len()
.cmp(&deduped[b].description.len())
})
}) else {
continue;
};
for &idx in indices {
if idx != primary_idx {
remove_indices.push(idx);
merge_map.entry(primary_idx).or_default().push(idx);
}
}
}
if !remove_indices.is_empty() {
remove_indices.sort_unstable();
remove_indices.dedup();
// Merge evidence
for (&primary, secondaries) in &merge_map {
let extra_evidence: Vec<_> = secondaries
.iter()
.flat_map(|&i| deduped[i].evidence.clone())
.collect();
let any_exploitable = secondaries.iter().any(|&i| deduped[i].exploitable);
deduped[primary].evidence.extend(extra_evidence);
if any_exploitable {
deduped[primary].exploitable = true;
}
}
// Remove merged findings (iterate in reverse to preserve indices)
for &idx in remove_indices.iter().rev() {
deduped.remove(idx);
}
}
let after = deduped.len();
if before != after {
tracing::debug!(
"DAST CWE-based dedup: {before} → {after} findings ({} merged)",
before - after
);
}
deduped
}
/// Merge a duplicate DAST finding into a primary one.
fn merge_dast_finding(primary: &mut DastFinding, duplicate: &DastFinding) {
primary.evidence.extend(duplicate.evidence.clone());
if duplicate.severity > primary.severity {
primary.severity = duplicate.severity.clone();
}
if duplicate.exploitable {
primary.exploitable = true;
}
// Keep the longer/better description
if duplicate.description.len() > primary.description.len() {
primary.description.clone_from(&duplicate.description);
}
// Keep remediation if primary doesn't have one
if primary.remediation.is_none() && duplicate.remediation.is_some() {
primary.remediation.clone_from(&duplicate.remediation);
}
}
#[cfg(test)]
mod tests {
use super::*;
use compliance_core::models::dast::DastVulnType;
use compliance_core::models::finding::Severity;
#[test]
fn fingerprint_is_deterministic() {
let a = compute_fingerprint(&["repo1", "rule-x", "src/main.rs", "42"]);
let b = compute_fingerprint(&["repo1", "rule-x", "src/main.rs", "42"]);
assert_eq!(a, b);
}
#[test]
fn fingerprint_changes_with_different_input() {
let a = compute_fingerprint(&["repo1", "rule-x", "src/main.rs", "42"]);
let b = compute_fingerprint(&["repo1", "rule-x", "src/main.rs", "43"]);
assert_ne!(a, b);
}
#[test]
fn fingerprint_is_valid_hex_sha256() {
let fp = compute_fingerprint(&["hello"]);
assert_eq!(fp.len(), 64, "SHA-256 hex should be 64 chars");
assert!(fp.chars().all(|c| c.is_ascii_hexdigit()));
}
#[test]
fn fingerprint_empty_parts() {
let fp = compute_fingerprint(&[]);
// Should still produce a valid hash (of empty input)
assert_eq!(fp.len(), 64);
}
#[test]
fn fingerprint_order_matters() {
let a = compute_fingerprint(&["a", "b"]);
let b = compute_fingerprint(&["b", "a"]);
assert_ne!(a, b);
}
#[test]
fn fingerprint_separator_prevents_collision() {
// "ab" + "c" vs "a" + "bc" should differ because of the "|" separator
let a = compute_fingerprint(&["ab", "c"]);
let b = compute_fingerprint(&["a", "bc"]);
assert_ne!(a, b);
}
fn make_dast(title: &str, endpoint: &str, vuln_type: DastVulnType) -> DastFinding {
let mut f = DastFinding::new(
"run1".into(),
"target1".into(),
vuln_type,
title.into(),
format!("Description for {title}"),
Severity::Medium,
endpoint.into(),
"GET".into(),
);
f.cwe = Some("CWE-319".into());
f
}
#[test]
fn canonicalize_strips_domain_suffix() {
let canon = canonicalize_dast_title("Missing HSTS header for comp-dev.meghsakha.com");
assert!(!canon.contains("meghsakha"), "domain should be stripped");
assert!(
canon.contains("strict-transport-security"),
"hsts should be resolved: {canon}"
);
}
#[test]
fn canonicalize_resolves_synonyms() {
let a = canonicalize_dast_title("Missing HSTS header");
let b = canonicalize_dast_title("Missing strict-transport-security header");
assert_eq!(a, b);
}
#[test]
fn exact_dedup_merges_identical_findings() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let f2 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 1, "exact duplicates should be merged");
}
#[test]
fn synonym_dedup_merges_hsts_variants() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let f2 = make_dast(
"Missing HSTS header for example.com",
"https://example.com",
DastVulnType::TlsMisconfiguration,
);
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(
result.len(),
1,
"HSTS synonym variants should merge to 1 finding"
);
}
#[test]
fn different_headers_not_merged() {
let mut f1 = make_dast(
"Missing x-content-type-options header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f1.cwe = Some("CWE-16".into());
let mut f2 = make_dast(
"Missing permissions-policy header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f2.cwe = Some("CWE-16".into());
// These share CWE-16 but are different headers — phase 2 will merge them
// since they share the same CWE+endpoint. This is acceptable because they
// have the same root cause (missing security headers configuration).
let result = dedup_dast_findings(vec![f1, f2]);
// CWE-based dedup will merge these into 1
assert!(
result.len() <= 2,
"same CWE+endpoint findings may be merged"
);
}
#[test]
fn different_endpoints_not_merged() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let f2 = make_dast(
"Missing strict-transport-security header",
"https://other.com",
DastVulnType::SecurityHeaderMissing,
);
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 2, "different endpoints should not merge");
}
#[test]
fn dedup_preserves_highest_severity() {
let f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
let mut f2 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f2.severity = Severity::High;
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 1);
assert_eq!(result[0].severity, Severity::High);
}
#[test]
fn dedup_merges_evidence() {
let mut f1 = make_dast(
"Missing strict-transport-security header",
"https://example.com",
DastVulnType::SecurityHeaderMissing,
);
f1.evidence
.push(compliance_core::models::dast::DastEvidence {
request_method: "GET".into(),
request_url: "https://example.com".into(),
request_headers: None,
request_body: None,
response_status: 200,
response_headers: None,
response_snippet: Some("pass 1".into()),
screenshot_path: None,
payload: None,
response_time_ms: None,
});
let mut f2 = f1.clone();
f2.evidence[0].response_snippet = Some("pass 2".into());
let result = dedup_dast_findings(vec![f1, f2]);
assert_eq!(result.len(), 1);
assert_eq!(result[0].evidence.len(), 2, "evidence should be merged");
}
}
@@ -0,0 +1,152 @@
//! Firmware SBOM via tramiton.
//!
//! Phase 2 (full, the default): drive a **reproducible build** with tramiton's
//! `NixBackend` — `analyze` → `seal_and_build` → a sealed lock whose libraries
//! are pinned and whose firmware artifact carries a content hash — then render
//! the SBOM from the lock plus deep binary SCA of pre-compiled inputs. This is
//! the complete bill of materials (toolchain + every fetched library + the
//! firmware image), the same one `tramiton sbom` produces.
//!
//! Phase 1 fallback (analysis-only): when no nix backend is available or the
//! build fails, fall back to the resolvable libraries + toolchain from the build
//! plan alone (no build). A scan therefore always yields *something*, and a nix
//! that can't run in the deployment never breaks a scan.
use std::path::Path;
use compliance_core::models::{SbomEntry, TargetType};
use tramiton_repro::ReproBackend;
use tramiton_sbom::ComponentKind;
/// Whether firmware SBOM applies to this target family.
pub fn is_firmware_target(target_type: TargetType) -> bool {
matches!(
target_type,
TargetType::FirmwareBareMetal | TargetType::FirmwareRtos | TargetType::EmbeddedLinuxYocto
)
}
/// Build SBOM entries for a firmware target from its source tree. Prefers a full
/// reproducible build (sealed lock); falls back to analysis-only. Returns an
/// empty vector when tramiton cannot even form a build plan.
pub async fn firmware_sbom_entries(path: &Path, repo_id: &str) -> Vec<SbomEntry> {
let p = path.to_path_buf();
let repo = repo_id.to_string();
// The whole analyze → seal → build → render sequence is blocking (it shells
// out to nix), so keep it off the async runtime. Bound it: a firmware build
// that hangs must not wedge the scan (the orphaned task is abandoned).
let handle = tokio::task::spawn_blocking(move || build_sbom_blocking(&p, &repo));
match tokio::time::timeout(std::time::Duration::from_secs(900), handle).await {
Ok(Ok(entries)) => entries,
Ok(Err(e)) => {
tracing::warn!(repo_id, error = %e, "Firmware SBOM: task join error");
Vec::new()
}
Err(_) => {
tracing::warn!(repo_id, "Firmware SBOM: build exceeded 15m; skipping");
Vec::new()
}
}
}
fn build_sbom_blocking(path: &Path, repo_id: &str) -> Vec<SbomEntry> {
let repo = tramiton_core::Repo::new(path);
let plan = match tramiton_core::provider::analyze(&repo) {
Ok(Some(bp)) => bp,
Ok(None) => return Vec::new(),
Err(e) => {
tracing::warn!(repo_id, error = %e, "Firmware SBOM: tramiton analyze failed");
return Vec::new();
}
};
// Phase 2: reproducible build → sealed lock → complete SBOM.
if let Some(backend) = tramiton_repro::NixBackend::detect() {
match tramiton_repro::seal_and_build(&backend, &plan, path) {
Ok(lock) => {
let mut sbom = tramiton_sbom::Sbom::from_lock(&lock, repo_id);
// Deep binary SCA of any pre-compiled inputs in the tree.
sbom.components.extend(tramiton_sbom::binary::scan(path));
let entries = sbom_to_entries(&sbom, repo_id);
tracing::info!(
repo_id,
backend = backend.name(),
count = entries.len(),
"Firmware SBOM: sealed reproducible build"
);
return entries;
}
Err(e) => {
tracing::warn!(repo_id, error = %e, "Firmware SBOM: reproducible build failed; falling back to analysis-only")
}
}
} else {
tracing::info!(
repo_id,
"Firmware SBOM: no nix backend available; analysis-only SBOM"
);
}
// Phase 1 fallback: analysis-only (toolchain + resolvable libraries).
analysis_entries(&plan, repo_id)
}
/// Map a rendered [`tramiton_sbom::Sbom`] (primary firmware + components) into
/// our [`SbomEntry`] rows. Source-file (`File`) components are dropped — they are
/// build inputs, not a dependency inventory.
fn sbom_to_entries(sbom: &tramiton_sbom::Sbom, repo_id: &str) -> Vec<SbomEntry> {
let mut entries = Vec::new();
if let Some(primary) = &sbom.primary {
entries.push(component_to_entry(primary, repo_id));
}
for c in &sbom.components {
if matches!(c.kind, ComponentKind::File) {
continue;
}
entries.push(component_to_entry(c, repo_id));
}
entries
}
fn component_to_entry(c: &tramiton_sbom::Component, repo_id: &str) -> SbomEntry {
let manager = match c.kind {
ComponentKind::Firmware => "firmware",
ComponentKind::Library => "library",
ComponentKind::Toolchain => "toolchain",
ComponentKind::File => "file",
};
let mut entry = SbomEntry::new(
repo_id.to_string(),
c.name.clone(),
c.version.clone().unwrap_or_default(),
manager.to_string(),
);
entry.purl = c.source.clone();
entry
}
/// Analysis-only components from the build plan: the cross-toolchain plus the
/// resolvable fetched libraries, without a build.
fn analysis_entries(bp: &tramiton_core::BuildPlan, repo_id: &str) -> Vec<SbomEntry> {
let mut entries = Vec::new();
if let Some(id) = bp.toolchain.id.clone() {
let version = bp.toolchain.version.clone().unwrap_or_default();
entries.push(SbomEntry::new(
repo_id.to_string(),
id,
version,
"toolchain".to_string(),
));
}
for lib in tramiton_repro::lock::libraries_from_inputs(&bp.inputs) {
let mut entry = SbomEntry::new(
repo_id.to_string(),
lib.name,
lib.revision,
"library".to_string(),
);
entry.purl = lib.source;
entries.push(entry);
}
entries
}
+48 -2
View File
@@ -80,7 +80,10 @@ impl GitOps {
#[tracing::instrument(skip_all, fields(repo_name = %repo_name))]
pub fn clone_or_fetch(&self, git_url: &str, repo_name: &str) -> Result<PathBuf, AgentError> {
let repo_path = self.base_path.join(repo_name);
// Names can contain slashes or other path-hostile characters (a target
// named after a repo path, say); collapse to one safe directory segment
// so the clone path never nests or breaks.
let repo_path = self.base_path.join(sanitize_repo_dir(repo_name));
if repo_path.exists() {
tracing::info!("fetching updates for existing repo");
@@ -135,7 +138,7 @@ impl GitOps {
/// Build credentials from agent config + per-repo overrides
pub fn make_repo_credentials(
config: &compliance_core::AgentConfig,
repo: &compliance_core::models::TrackedRepository,
repo: &crate::pipeline::repo_view::RepoView,
) -> RepoCredentials {
RepoCredentials {
ssh_key_path: Some(config.ssh_key_path.clone()),
@@ -253,3 +256,46 @@ pub struct DiffFile {
pub path: String,
pub hunks: String,
}
/// Collapse a repository name into a single filesystem-safe directory segment.
/// Names may carry slashes or other path-hostile characters (a target named
/// after a repo path, for instance); those would otherwise nest or break the
/// clone path, so map anything outside `[A-Za-z0-9._-]` to `_`.
fn sanitize_repo_dir(name: &str) -> String {
let mapped: String = name
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.' {
c
} else {
'_'
}
})
.collect();
let trimmed = mapped.trim_matches(|c| c == '.' || c == '_');
if trimmed.is_empty() {
"repo".to_string()
} else {
trimmed.to_string()
}
}
#[cfg(test)]
mod tests {
use super::sanitize_repo_dir;
#[test]
fn sanitizes_path_hostile_names() {
assert_eq!(
sanitize_repo_dir("zephyr-example-app"),
"zephyr-example-app"
);
assert_eq!(
sanitize_repo_dir("ChristianRinn/bare_metal_stm32f411xe"),
"ChristianRinn_bare_metal_stm32f411xe"
);
assert_eq!(sanitize_repo_dir("../../etc/passwd"), "etc_passwd");
assert_eq!(sanitize_repo_dir("a b:c"), "a_b_c");
assert_eq!(sanitize_repo_dir("///"), "repo");
}
}
+134 -20
View File
@@ -19,26 +19,33 @@ impl Scanner for GitleaksScanner {
#[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let output = tokio::process::Command::new("gitleaks")
.args([
"detect",
"--source",
".",
"--report-format",
"json",
"--report-path",
"/dev/stdout",
"--no-banner",
"--exit-code",
"0",
])
.current_dir(repo_path)
.output()
.await
.map_err(|e| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: Box::new(e),
})?;
let output = tokio::time::timeout(
std::time::Duration::from_secs(300),
tokio::process::Command::new("gitleaks")
.args([
"detect",
"--source",
".",
"--report-format",
"json",
"--report-path",
"/dev/stdout",
"--no-banner",
"--exit-code",
"0",
])
.current_dir(repo_path)
.output(),
)
.await
.map_err(|_| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: "timed out after 5 minutes".into(),
})?
.map_err(|e| CoreError::Scanner {
scanner: "gitleaks".to_string(),
source: Box::new(e),
})?;
if output.stdout.is_empty() {
return Ok(ScanOutput::default());
@@ -129,3 +136,110 @@ struct GitleaksResult {
#[serde(rename = "Match")]
r#match: String,
}
#[cfg(test)]
mod tests {
use super::*;
// --- is_allowlisted tests ---
#[test]
fn allowlisted_env_example_files() {
assert!(is_allowlisted(".env.example"));
assert!(is_allowlisted("config/.env.sample"));
assert!(is_allowlisted("deploy/.ENV.TEMPLATE"));
}
#[test]
fn allowlisted_test_directories() {
assert!(is_allowlisted("src/test/config.json"));
assert!(is_allowlisted("src/tests/fixtures.rs"));
assert!(is_allowlisted("data/fixtures/secret.txt"));
assert!(is_allowlisted("pkg/testdata/key.pem"));
}
#[test]
fn allowlisted_mock_files() {
assert!(is_allowlisted("src/mock_service.py"));
assert!(is_allowlisted("lib/MockAuth.java"));
}
#[test]
fn allowlisted_test_suffixes() {
assert!(is_allowlisted("auth_test.go"));
assert!(is_allowlisted("auth.test.ts"));
assert!(is_allowlisted("auth.test.js"));
assert!(is_allowlisted("auth.spec.ts"));
assert!(is_allowlisted("auth.spec.js"));
}
#[test]
fn not_allowlisted_regular_files() {
assert!(!is_allowlisted("src/main.rs"));
assert!(!is_allowlisted("config/.env"));
assert!(!is_allowlisted("lib/auth.ts"));
assert!(!is_allowlisted("deploy/secrets.yaml"));
}
#[test]
fn not_allowlisted_partial_matches() {
// "test" as substring in a non-directory context should not match
assert!(!is_allowlisted("src/attestation.rs"));
assert!(!is_allowlisted("src/contest/data.json"));
}
// --- GitleaksResult deserialization tests ---
#[test]
fn deserialize_gitleaks_result() {
let json = r#"{
"Description": "AWS Access Key",
"RuleID": "aws-access-key",
"File": "src/config.rs",
"StartLine": 10,
"Match": "AKIAIOSFODNN7EXAMPLE"
}"#;
let result: GitleaksResult = serde_json::from_str(json).unwrap();
assert_eq!(result.description, "AWS Access Key");
assert_eq!(result.rule_id, "aws-access-key");
assert_eq!(result.file, "src/config.rs");
assert_eq!(result.start_line, 10);
assert_eq!(result.r#match, "AKIAIOSFODNN7EXAMPLE");
}
#[test]
fn deserialize_gitleaks_result_array() {
let json = r#"[
{
"Description": "Generic Secret",
"RuleID": "generic-secret",
"File": "app.py",
"StartLine": 5,
"Match": "password=hunter2"
}
]"#;
let results: Vec<GitleaksResult> = serde_json::from_str(json).unwrap();
assert_eq!(results.len(), 1);
assert_eq!(results[0].rule_id, "generic-secret");
}
#[test]
fn severity_mapping_private_key() {
// Verify the severity logic from the scan method
let rule_id = "some-private-key-rule";
assert!(rule_id.contains("private-key"));
}
#[test]
fn severity_mapping_token_password_secret() {
for keyword in &["token", "password", "secret"] {
let rule_id = format!("some-{}-rule", keyword);
assert!(
rule_id.contains("token")
|| rule_id.contains("password")
|| rule_id.contains("secret"),
"Expected '{rule_id}' to match token/password/secret"
);
}
}
}
@@ -0,0 +1,106 @@
use compliance_core::models::Finding;
use super::orchestrator::{GraphContext, PipelineOrchestrator};
use crate::error::AgentError;
impl PipelineOrchestrator {
/// Build the code knowledge graph for a repo and compute impact analyses
pub(super) async fn build_code_graph(
&self,
repo_path: &std::path::Path,
repo_id: &str,
findings: &[Finding],
) -> Result<GraphContext, AgentError> {
let graph_build_id = uuid::Uuid::new_v4().to_string();
let engine = compliance_graph::GraphEngine::new(50_000);
let (mut code_graph, build_run) =
engine
.build_graph(repo_path, repo_id, &graph_build_id)
.map_err(|e| AgentError::Other(format!("Graph build error: {e}")))?;
// Apply community detection
compliance_graph::graph::community::apply_communities(&mut code_graph);
// Store graph in MongoDB
let store = compliance_graph::graph::persistence::GraphStore::new(self.db.inner());
store
.delete_repo_graph(repo_id)
.await
.map_err(|e| AgentError::Other(format!("Graph cleanup error: {e}")))?;
store
.store_graph(&build_run, &code_graph.nodes, &code_graph.edges)
.await
.map_err(|e| AgentError::Other(format!("Graph store error: {e}")))?;
// Compute impact analysis for each finding
let analyzer = compliance_graph::GraphEngine::impact_analyzer(&code_graph);
let mut impacts = Vec::new();
for finding in findings {
if let Some(file_path) = &finding.file_path {
let impact = analyzer.analyze(
repo_id,
&finding.fingerprint,
&graph_build_id,
file_path,
finding.line_number,
);
store
.store_impact(&impact)
.await
.map_err(|e| AgentError::Other(format!("Impact store error: {e}")))?;
impacts.push(impact);
}
}
Ok(GraphContext {
node_count: build_run.node_count,
edge_count: build_run.edge_count,
community_count: build_run.community_count,
impacts,
})
}
/// Trigger DAST scan if a target is configured for this repo
pub(super) async fn maybe_trigger_dast(&self, repo_id: &str, scan_run_id: &str) {
use futures_util::TryStreamExt;
let filter = mongodb::bson::doc! { "repo_id": repo_id };
let targets: Vec<compliance_core::models::DastTarget> =
match self.db.dast_targets().find(filter).await {
Ok(cursor) => cursor.try_collect().await.unwrap_or_default(),
Err(_) => return,
};
if targets.is_empty() {
tracing::info!("[{repo_id}] No DAST targets configured, skipping");
return;
}
for target in targets {
let db = self.db.clone();
let scan_run_id = scan_run_id.to_string();
tokio::spawn(async move {
let orchestrator = compliance_dast::DastOrchestrator::new(100);
match orchestrator.run_scan(&target, Vec::new()).await {
Ok((mut scan_run, findings)) => {
scan_run.sast_scan_run_id = Some(scan_run_id);
if let Err(e) = db.dast_scan_runs().insert_one(&scan_run).await {
tracing::error!("Failed to store DAST scan run: {e}");
}
for finding in &findings {
if let Err(e) = db.dast_findings().insert_one(finding).await {
tracing::error!("Failed to store DAST finding: {e}");
}
}
tracing::info!("DAST scan complete: {} findings", findings.len());
}
Err(e) => {
tracing::error!("DAST scan failed: {e}");
}
}
});
}
}
}
@@ -0,0 +1,95 @@
//! Minimal EtherNet/IP (CIP) reachability probe.
//!
//! Sends an EtherNet/IP encapsulation **ListIdentity** command (0x0063) over TCP
//! 44818 and checks for a valid encapsulation reply — confirming a CIP device
//! without opening a session or writing anything.
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::timeout;
/// Outcome of an EtherNet/IP handshake probe.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct EnipProbe {
/// A TCP connection to the port was established.
pub reachable: bool,
/// The endpoint returned a valid EtherNet/IP encapsulation reply.
pub is_enip: bool,
}
/// Probe an EtherNet/IP endpoint with a ListIdentity request. Read-only.
pub async fn probe(host: &str, port: u16, budget: Duration) -> EnipProbe {
let mut out = EnipProbe::default();
let Ok(Ok(mut stream)) = timeout(budget, TcpStream::connect((host, port))).await else {
return out;
};
out.reachable = true;
// Encapsulation header (24 bytes): command(2) length(2) session(4) status(4)
// context(8) options(4). ListIdentity = command 0x0063, everything else zero.
let mut req = vec![0u8; 24];
req[0..2].copy_from_slice(&0x0063u16.to_le_bytes());
if timeout(budget, stream.write_all(&req))
.await
.ok()
.and_then(Result::ok)
.is_none()
{
return out;
}
let mut hdr = [0u8; 24];
if timeout(budget, stream.read_exact(&mut hdr))
.await
.ok()
.and_then(Result::ok)
.is_none()
{
return out;
}
let command = u16::from_le_bytes([hdr[0], hdr[1]]);
let status = u32::from_le_bytes([hdr[8], hdr[9], hdr[10], hdr[11]]);
// Echoed command + success status = a valid EtherNet/IP encapsulation reply.
if command == 0x0063 && status == 0 {
out.is_enip = true;
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::net::TcpListener;
async fn mock_server() -> std::net::SocketAddr {
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind");
let addr = listener.local_addr().expect("addr");
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.expect("accept");
let mut req = [0u8; 24];
if sock.read_exact(&mut req).await.is_err() {
return;
}
// Reply: echo command 0x0063, status 0, no data.
let mut hdr = vec![0u8; 24];
hdr[0..2].copy_from_slice(&0x0063u16.to_le_bytes());
let _ = sock.write_all(&hdr).await;
});
addr
}
#[tokio::test]
async fn probe_detects_an_ethernetip_device() {
let addr = mock_server().await;
let p = probe(&addr.ip().to_string(), addr.port(), Duration::from_secs(2)).await;
assert!(p.reachable && p.is_enip);
}
#[tokio::test]
async fn probe_reports_unreachable_for_a_closed_port() {
let p = probe("127.0.0.1", 1, Duration::from_millis(500)).await;
assert!(!p.reachable && !p.is_enip);
}
}
+273
View File
@@ -0,0 +1,273 @@
//! Dynamic ICS (industrial control system) probing for PLC/SPS targets.
//!
//! Where the control-logic scanner is static (over ST / PLCopen XML), this probes
//! the *running* device over industrial protocols and reports exposed /
//! unauthenticated control interfaces. It is read-only: it never writes to a live
//! process. Modbus/TCP and OPC UA are implemented; EtherNet-IP is a follow-on.
pub mod ethernetip;
pub mod modbus;
pub mod opcua;
pub mod portscan;
use std::time::Duration;
use compliance_core::models::{Finding, ScanType, Severity};
use crate::pipeline::dedup;
/// Well-known deep-probe ports (each independent of any WebVisu HTTP port).
const MODBUS_PORT: u16 = 502;
const OPCUA_PORT: u16 = 4840;
const ENIP_PORT: u16 = 44818;
/// Probe a PLC/SPS device's industrial-protocol surface and return findings.
/// Read-only. Deep-probes Modbus/TCP, OPC UA and EtherNet/IP, plus a service
/// discovery scan of the remaining OT / insecure-management ports. `endpoint` is
/// the target's live-URL / host reference.
pub async fn probe_target(endpoint: &str, repo_id: &str, budget: Duration) -> Vec<Finding> {
let (host, modbus_port) = parse_endpoint(endpoint);
let mut findings = modbus_findings(&host, modbus_port, repo_id, budget).await;
findings.extend(opcua_findings(&host, OPCUA_PORT, repo_id, budget).await);
findings.extend(enip_findings(&host, ENIP_PORT, repo_id, budget).await);
findings.extend(portscan_findings(&host, repo_id, budget).await);
findings
}
/// Findings from probing the Modbus/TCP surface.
async fn modbus_findings(host: &str, port: u16, repo_id: &str, budget: Duration) -> Vec<Finding> {
let probe = modbus::probe(host, port, budget).await;
let mut findings = Vec::new();
if !probe.speaks_modbus {
// Not reachable, or the port does not speak Modbus — nothing to report.
return findings;
}
let target = format!("{host}:{port}");
// Reachable Modbus/TCP = unauthenticated, cleartext control access by design.
let fp = dedup::compute_fingerprint(&[repo_id, "ics-modbus-exposed", &target]);
let mut f = Finding::new(
repo_id.to_string(),
fp,
"ics-probe".to_string(),
ScanType::IcsProbe,
"Modbus/TCP control interface exposed without authentication".to_string(),
format!(
"The device at {target} answers Modbus/TCP requests. Modbus/TCP has no \
authentication or encryption in the protocol, so any host that can reach this \
port can read and write process variables (coils/registers) and disrupt the \
controlled process."
),
Severity::Critical,
);
f.rule_id = Some("ics-modbus-exposed".to_string());
f.cwe = Some("CWE-306".to_string());
f.remediation = Some(
"Restrict the Modbus/TCP port to a trusted control network (segmentation / \
firewall / VPN), never expose it to IT or the internet, and prefer an authenticated \
transport (e.g. Modbus/TLS) or a secure protocol gateway where available."
.to_string(),
);
findings.push(f);
if let Some(dev) = &probe.device {
let details = [
dev.vendor.as_deref(),
dev.product.as_deref(),
dev.revision.as_deref(),
]
.into_iter()
.flatten()
.collect::<Vec<_>>()
.join(" / ");
let fp = dedup::compute_fingerprint(&[repo_id, "ics-device-disclosure", &target]);
let mut f = Finding::new(
repo_id.to_string(),
fp,
"ics-probe".to_string(),
ScanType::IcsProbe,
"PLC device identity disclosed over Modbus".to_string(),
format!(
"The device at {target} discloses its identity via Modbus Read Device \
Identification: {details}. This aids fingerprinting and targeting of \
known-vulnerable firmware/runtime versions."
),
Severity::Low,
);
f.rule_id = Some("ics-device-disclosure".to_string());
f.cwe = Some("CWE-200".to_string());
f.remediation = Some(
"Limit network reach to the device; Modbus device identification cannot be \
disabled, so exposure is bounded by network segmentation."
.to_string(),
);
findings.push(f);
}
findings
}
/// Findings from probing the OPC UA surface (default port 4840). A reachability
/// probe only: it flags an exposed OPC UA server for review of its security
/// policy / authentication (deep SecurityPolicy analysis is a follow-on).
async fn opcua_findings(host: &str, port: u16, repo_id: &str, budget: Duration) -> Vec<Finding> {
let probe = opcua::probe(host, port, budget).await;
let mut findings = Vec::new();
if !probe.is_opcua {
return findings;
}
let target = format!("{host}:{port}");
let fp = dedup::compute_fingerprint(&[repo_id, "ics-opcua-exposed", &target]);
let mut f = Finding::new(
repo_id.to_string(),
fp,
"ics-probe".to_string(),
ScanType::IcsProbe,
"OPC UA server exposed on the network".to_string(),
format!(
"An OPC UA server answers at {target}. Verify it enforces message security \
(a SecurityPolicy other than None) and rejects anonymous sessions — the common \
default of SecurityPolicy None + an Anonymous user token allows unauthenticated, \
unencrypted read/write of the server's address space."
),
Severity::Medium,
);
f.rule_id = Some("ics-opcua-exposed".to_string());
f.cwe = Some("CWE-319".to_string());
f.remediation = Some(
"Restrict OPC UA (4840) to a trusted network; require a signed & encrypted \
SecurityPolicy (Basic256Sha256 or better) with certificate / username \
authentication, and disable the Anonymous user token."
.to_string(),
);
findings.push(f);
findings
}
/// Findings from probing the EtherNet/IP (CIP) surface (default port 44818).
async fn enip_findings(host: &str, port: u16, repo_id: &str, budget: Duration) -> Vec<Finding> {
let probe = ethernetip::probe(host, port, budget).await;
if !probe.is_enip {
return Vec::new();
}
let target = format!("{host}:{port}");
let fp = dedup::compute_fingerprint(&[repo_id, "ics-ethernetip-exposed", &target]);
let mut f = Finding::new(
repo_id.to_string(),
fp,
"ics-probe".to_string(),
ScanType::IcsProbe,
"EtherNet/IP (CIP) interface exposed on the network".to_string(),
format!(
"The device at {target} answers EtherNet/IP (CIP) requests. EtherNet/IP has no \
authentication in the base protocol, so a host that can reach it can enumerate \
and interact with the device's control objects."
),
Severity::High,
);
f.rule_id = Some("ics-ethernetip-exposed".to_string());
f.cwe = Some("CWE-306".to_string());
f.remediation = Some(
"Restrict EtherNet/IP (44818/2222) to a trusted control network; use CIP Security \
(encryption + authentication) on devices that support it."
.to_string(),
);
vec![f]
}
/// Findings from the service-discovery port scan of the remaining OT /
/// insecure-management surface.
async fn portscan_findings(host: &str, repo_id: &str, budget: Duration) -> Vec<Finding> {
let open = portscan::scan(host, portscan::KNOWN_PORTS, budget).await;
open.into_iter()
.map(|kp| {
let target = format!("{host}:{}", kp.port);
let (title, severity, cwe, description) = match kp.kind {
portscan::PortKind::Ics => (
format!("ICS service exposed: {}", kp.service),
Severity::High,
"CWE-306",
format!(
"{target} exposes {} ({}). Industrial protocols are typically \
unauthenticated, so network reach implies control access.",
kp.service, kp.note
),
),
portscan::PortKind::InsecureMgmt => (
format!("Cleartext service exposed: {}", kp.service),
Severity::Medium,
"CWE-319",
format!(
"{target} exposes {} ({}), which transmits credentials and data in \
cleartext.",
kp.service, kp.note
),
),
};
let fp = dedup::compute_fingerprint(&[repo_id, "ics-service-exposed", &target]);
let mut f = Finding::new(
repo_id.to_string(),
fp,
"ics-probe".to_string(),
ScanType::IcsProbe,
title,
description,
severity,
);
f.rule_id = Some("ics-service-exposed".to_string());
f.cwe = Some(cwe.to_string());
f.remediation = Some(
"Restrict the service to a trusted network segment; disable it if unused; \
replace cleartext protocols (Telnet/FTP) with SSH/SFTP."
.to_string(),
);
f
})
.collect()
}
/// Extract `(host, port)` from a target reference. Modbus lives on its own port
/// (502 by default), independent of any HTTP/WebVisu URL, so unless the reference
/// explicitly carries `modbus://host:port` or a bare `host:port`, we probe 502.
fn parse_endpoint(endpoint: &str) -> (String, u16) {
let s = endpoint.trim();
let (scheme, rest) = match s.split_once("://") {
Some((sch, r)) => (Some(sch.to_ascii_lowercase()), r),
None => (None, s),
};
let hostport = rest.split(['/', '?']).next().unwrap_or(rest);
let (host, port) = match hostport.rsplit_once(':') {
Some((h, p)) => (h.to_string(), p.parse::<u16>().ok()),
None => (hostport.to_string(), None),
};
let port = match (scheme.as_deref(), port) {
// Explicit Modbus port, or a bare host:port the user chose.
(Some("modbus"), Some(p)) | (None, Some(p)) => p,
// An http(s)/WebVisu URL (or no port): Modbus is on its own port.
_ => MODBUS_PORT,
};
(host, port)
}
#[cfg(test)]
mod tests {
use super::parse_endpoint;
#[test]
fn endpoint_parsing_picks_the_modbus_port() {
assert_eq!(parse_endpoint("10.0.0.5"), ("10.0.0.5".into(), 502));
assert_eq!(parse_endpoint("10.0.0.5:1502"), ("10.0.0.5".into(), 1502));
assert_eq!(
parse_endpoint("modbus://plc.local:5020"),
("plc.local".into(), 5020)
);
// A WebVisu URL: the http port is ignored; Modbus is on 502.
assert_eq!(
parse_endpoint("http://plc.local:8080/webvisu"),
("plc.local".into(), 502)
);
assert_eq!(
parse_endpoint("https://plc.local/"),
("plc.local".into(), 502)
);
}
}
+205
View File
@@ -0,0 +1,205 @@
//! Minimal Modbus/TCP client for dynamic ICS probing.
//!
//! Modbus/TCP (port 502) has no authentication or encryption in the protocol, so
//! an endpoint that answers requests is, by design, open to any host that can
//! reach it. The probe only *reads* — a Read Holding Registers request and a Read
//! Device Identification request — and never writes to the live process.
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::timeout;
/// Outcome of probing a Modbus/TCP endpoint.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct ModbusProbe {
/// A TCP connection to the port was established.
pub reachable: bool,
/// The endpoint answered a Modbus request (a normal reply or a Modbus
/// exception) — i.e. it speaks Modbus, unauthenticated.
pub speaks_modbus: bool,
/// Device identity, if disclosed via Read Device Identification (FC 43 / 14).
pub device: Option<DeviceId>,
}
/// Vendor / product / revision from Read Device Identification.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct DeviceId {
pub vendor: Option<String>,
pub product: Option<String>,
pub revision: Option<String>,
}
/// Probe a Modbus/TCP endpoint. Read-only: issues a Read Holding Registers and a
/// Read Device Identification request; never writes to the device.
pub async fn probe(host: &str, port: u16, budget: Duration) -> ModbusProbe {
let mut out = ModbusProbe::default();
let Ok(Ok(mut stream)) = timeout(budget, TcpStream::connect((host, port))).await else {
return out; // unreachable
};
out.reachable = true;
// Read Holding Registers (FC 0x03), unit 1, addr 0, qty 1 — a benign read.
let rhr = [0x03u8, 0x00, 0x00, 0x00, 0x01];
if let Some(resp) = txn(&mut stream, 1, &rhr, budget).await {
// A normal reply (0x03) or an exception (0x83) both prove it speaks Modbus.
if matches!(resp.first(), Some(0x03) | Some(0x83)) {
out.speaks_modbus = true;
}
}
// Read Device Identification (FC 0x2B / MEI 0x0E), basic (0x01), object 0.
let rdi = [0x2Bu8, 0x0E, 0x01, 0x00];
if let Some(resp) = txn(&mut stream, 1, &rdi, budget).await {
if resp.first() == Some(&0x2B) {
out.speaks_modbus = true;
out.device = parse_device_id(&resp);
}
}
out
}
/// Send one Modbus PDU and return the response PDU (function code + data), or
/// `None` on timeout / malformed reply.
async fn txn(stream: &mut TcpStream, unit: u8, pdu: &[u8], budget: Duration) -> Option<Vec<u8>> {
// MBAP header: transaction id (2) + protocol id (2) = 0 + length (2) + unit (1),
// then the PDU. `length` counts the unit byte plus the PDU.
let len = (pdu.len() + 1) as u16;
let mut frame = Vec::with_capacity(7 + pdu.len());
frame.extend_from_slice(&[0x00, 0x01]); // transaction id
frame.extend_from_slice(&[0x00, 0x00]); // protocol id
frame.extend_from_slice(&len.to_be_bytes());
frame.push(unit);
frame.extend_from_slice(pdu);
timeout(budget, stream.write_all(&frame)).await.ok()?.ok()?;
let mut hdr = [0u8; 7];
timeout(budget, stream.read_exact(&mut hdr))
.await
.ok()?
.ok()?;
// Reject non-Modbus replies (protocol id must be 0).
if hdr[2] != 0 || hdr[3] != 0 {
return None;
}
let plen = u16::from_be_bytes([hdr[4], hdr[5]]) as usize;
if !(2..=260).contains(&plen) {
return None;
}
let mut body = vec![0u8; plen - 1]; // minus the unit id already in hdr[6]
timeout(budget, stream.read_exact(&mut body))
.await
.ok()?
.ok()?;
Some(body)
}
/// Parse vendor / product / revision from a Read Device Identification PDU:
/// `[0x2B, 0x0E, readDevIdCode, conformity, moreFollows, nextObjId, numObjects,
/// (objId, len, bytes…)…]`.
fn parse_device_id(pdu: &[u8]) -> Option<DeviceId> {
if pdu.len() < 7 {
return None;
}
let num = pdu[6] as usize;
let mut i = 7;
let mut dev = DeviceId::default();
for _ in 0..num {
if i + 2 > pdu.len() {
break;
}
let id = pdu[i];
let l = pdu[i + 1] as usize;
i += 2;
if i + l > pdu.len() {
break;
}
let val = String::from_utf8_lossy(&pdu[i..i + l]).trim().to_string();
i += l;
match id {
0x00 => dev.vendor = Some(val),
0x01 => dev.product = Some(val),
0x02 => dev.revision = Some(val),
_ => {}
}
}
if dev == DeviceId::default() {
None
} else {
Some(dev)
}
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::net::TcpListener;
/// A one-shot mock Modbus/TCP server that answers a Read Holding Registers
/// request and a Read Device Identification request on one connection.
async fn mock_server(with_device: bool) -> std::net::SocketAddr {
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind");
let addr = listener.local_addr().expect("addr");
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.expect("accept");
loop {
let mut hdr = [0u8; 7];
if sock.read_exact(&mut hdr).await.is_err() {
break;
}
let plen = u16::from_be_bytes([hdr[4], hdr[5]]) as usize;
let mut pdu = vec![0u8; plen - 1];
if sock.read_exact(&mut pdu).await.is_err() {
break;
}
let reply_pdu: Vec<u8> = match pdu.first() {
Some(0x03) => vec![0x03, 0x02, 0x00, 0x00], // 1 register = 0
Some(0x2B) if with_device => vec![
0x2B, 0x0E, 0x01, 0x81, 0x00, 0x00, 0x02, // 2 objects
0x00, 0x04, b'A', b'C', b'M', b'E', // vendor
0x01, 0x03, b'P', b'L', b'C', // product
],
_ => vec![pdu[0] | 0x80, 0x01], // exception
};
let len = (reply_pdu.len() + 1) as u16;
let mut frame = vec![hdr[0], hdr[1], 0x00, 0x00];
frame.extend_from_slice(&len.to_be_bytes());
frame.push(hdr[6]);
frame.extend_from_slice(&reply_pdu);
if sock.write_all(&frame).await.is_err() {
break;
}
}
});
addr
}
#[tokio::test]
async fn probe_detects_a_modbus_endpoint_and_reads_device_id() {
let addr = mock_server(true).await;
let p = probe(&addr.ip().to_string(), addr.port(), Duration::from_secs(2)).await;
assert!(p.reachable && p.speaks_modbus);
let dev = p.device.expect("device id");
assert_eq!(dev.vendor.as_deref(), Some("ACME"));
assert_eq!(dev.product.as_deref(), Some("PLC"));
}
#[tokio::test]
async fn probe_reports_unreachable_for_a_closed_port() {
// 127.0.0.1:1 is (almost certainly) closed.
let p = probe("127.0.0.1", 1, Duration::from_millis(500)).await;
assert!(!p.reachable && !p.speaks_modbus);
}
#[test]
fn parses_device_identification_objects() {
let pdu = [
0x2B, 0x0E, 0x01, 0x81, 0x00, 0x00, 0x01, // 1 object
0x02, 0x05, b'v', b'1', b'.', b'2', b'3', // revision
];
let dev = parse_device_id(&pdu).expect("device");
assert_eq!(dev.revision.as_deref(), Some("v1.23"));
assert!(dev.vendor.is_none());
}
}
+131
View File
@@ -0,0 +1,131 @@
//! Minimal OPC UA reachability probe.
//!
//! Speaks just the OPC UA Connection Protocol (UACP) handshake — a `HEL` (Hello)
//! message, expecting an `ACK` (or `ERR`) reply — to confirm an OPC UA server is
//! listening (default port 4840). It does **not** open a secure channel or make
//! service calls; deep analysis of the server's SecurityPolicy / user-token
//! policies (the common `None` + `Anonymous` misconfiguration) is a follow-on best
//! done with a full OPC UA stack.
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::time::timeout;
/// Outcome of an OPC UA handshake probe.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct OpcUaProbe {
/// A TCP connection to the port was established.
pub reachable: bool,
/// The endpoint replied to the UACP Hello (`ACK`) or rejected it (`ERR`) —
/// either way it speaks OPC UA.
pub is_opcua: bool,
}
/// Probe an OPC UA endpoint with a UACP Hello. Read-only handshake only.
pub async fn probe(host: &str, port: u16, budget: Duration) -> OpcUaProbe {
let mut out = OpcUaProbe::default();
let Ok(Ok(mut stream)) = timeout(budget, TcpStream::connect((host, port))).await else {
return out;
};
out.reachable = true;
let hello = hello_message(&format!("opc.tcp://{host}:{port}"));
if timeout(budget, stream.write_all(&hello))
.await
.ok()
.and_then(Result::ok)
.is_none()
{
return out;
}
// Read the 3-byte message type of the reply: ACK (accepted) or ERR (rejected
// our hello) both prove the peer speaks the OPC UA connection protocol.
let mut mt = [0u8; 3];
if timeout(budget, stream.read_exact(&mut mt))
.await
.ok()
.and_then(Result::ok)
.is_none()
{
return out;
}
if &mt == b"ACK" || &mt == b"ERR" {
out.is_opcua = true;
}
out
}
/// Build a UACP `HEL` (Hello) message advertising our buffer sizes + endpoint URL.
fn hello_message(endpoint_url: &str) -> Vec<u8> {
let url = endpoint_url.as_bytes();
let mut m = Vec::with_capacity(32 + url.len());
m.extend_from_slice(b"HELF");
m.extend_from_slice(&0u32.to_le_bytes()); // message size — patched below
m.extend_from_slice(&0u32.to_le_bytes()); // ProtocolVersion
m.extend_from_slice(&65536u32.to_le_bytes()); // ReceiveBufferSize
m.extend_from_slice(&65536u32.to_le_bytes()); // SendBufferSize
m.extend_from_slice(&0u32.to_le_bytes()); // MaxMessageSize (0 = no limit)
m.extend_from_slice(&0u32.to_le_bytes()); // MaxChunkCount
m.extend_from_slice(&(url.len() as i32).to_le_bytes()); // EndpointUrl length
m.extend_from_slice(url);
let size = m.len() as u32;
m[4..8].copy_from_slice(&size.to_le_bytes());
m
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::net::TcpListener;
/// A mock OPC UA server that reads the Hello and replies with an `ACK` frame.
async fn mock_server() -> std::net::SocketAddr {
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind");
let addr = listener.local_addr().expect("addr");
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.expect("accept");
// Read the Hello header (8 bytes) to learn the size, then drain it.
let mut hdr = [0u8; 8];
if sock.read_exact(&mut hdr).await.is_err() {
return;
}
let size = u32::from_le_bytes([hdr[4], hdr[5], hdr[6], hdr[7]]) as usize;
let mut rest = vec![0u8; size.saturating_sub(8)];
let _ = sock.read_exact(&mut rest).await;
// Reply: ACK + size + 5 u32 fields.
let mut ack = Vec::new();
ack.extend_from_slice(b"ACKF");
ack.extend_from_slice(&28u32.to_le_bytes());
for _ in 0..5 {
ack.extend_from_slice(&0u32.to_le_bytes());
}
let _ = sock.write_all(&ack).await;
});
addr
}
#[tokio::test]
async fn probe_detects_an_opcua_server() {
let addr = mock_server().await;
let p = probe(&addr.ip().to_string(), addr.port(), Duration::from_secs(2)).await;
assert!(p.reachable && p.is_opcua);
}
#[tokio::test]
async fn probe_reports_unreachable_for_a_closed_port() {
let p = probe("127.0.0.1", 1, Duration::from_millis(500)).await;
assert!(!p.reachable && !p.is_opcua);
}
#[test]
fn hello_message_is_well_formed() {
let m = hello_message("opc.tcp://h:4840");
assert_eq!(&m[0..4], b"HELF");
// The embedded size equals the actual length.
let size = u32::from_le_bytes([m[4], m[5], m[6], m[7]]) as usize;
assert_eq!(size, m.len());
}
}
@@ -0,0 +1,137 @@
//! TCP service discovery for a device.
//!
//! Connect-scans a curated set of OT/ICS and insecure-management ports and reports
//! the ones that are open. The deep protocol probes own Modbus (502), OPC UA
//! (4840) and EtherNet/IP (44818); this surfaces the *rest* of the industrial and
//! cleartext-management surface (Siemens S7, DNP3, CODESYS programming, Telnet, …).
use std::time::Duration;
use futures_util::future::join_all;
use tokio::net::TcpStream;
use tokio::time::timeout;
/// Whether an open port is an industrial protocol or an insecure management service.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PortKind {
/// An industrial control protocol (typically unauthenticated).
Ics,
/// A cleartext management service (credentials/data in the clear).
InsecureMgmt,
}
/// A well-known port worth flagging when open.
#[derive(Debug, Clone, Copy)]
pub struct KnownPort {
pub port: u16,
pub service: &'static str,
pub kind: PortKind,
pub note: &'static str,
}
/// The curated scan list. Excludes 502 / 4840 / 44818 — those have dedicated deep
/// probes (Modbus, OPC UA, EtherNet/IP) that report richer findings.
pub const KNOWN_PORTS: &[KnownPort] = &[
KnownPort {
port: 102,
service: "S7comm / ISO-TSAP",
kind: PortKind::Ics,
note: "Siemens S7 PLC communication",
},
KnownPort {
port: 20000,
service: "DNP3",
kind: PortKind::Ics,
note: "SCADA / DNP3",
},
KnownPort {
port: 1911,
service: "Niagara Fox",
kind: PortKind::Ics,
note: "Tridium Niagara building automation",
},
KnownPort {
port: 11740,
service: "CODESYS",
kind: PortKind::Ics,
note: "CODESYS programming protocol",
},
KnownPort {
port: 1962,
service: "PCWorx",
kind: PortKind::Ics,
note: "Phoenix Contact PCWorx",
},
KnownPort {
port: 9600,
service: "OMRON FINS",
kind: PortKind::Ics,
note: "Omron FINS",
},
KnownPort {
port: 789,
service: "Red Lion Crimson",
kind: PortKind::Ics,
note: "Red Lion controllers",
},
KnownPort {
port: 23,
service: "Telnet",
kind: PortKind::InsecureMgmt,
note: "cleartext remote shell",
},
KnownPort {
port: 21,
service: "FTP",
kind: PortKind::InsecureMgmt,
note: "cleartext file transfer",
},
];
/// Connect-scan `ports` on `host` (concurrently) and return those that accept a
/// TCP connection.
pub async fn scan<'a>(host: &str, ports: &'a [KnownPort], budget: Duration) -> Vec<&'a KnownPort> {
let checks = ports.iter().map(|kp| async move {
let open = timeout(budget, TcpStream::connect((host, kp.port)))
.await
.map(|r| r.is_ok())
.unwrap_or(false);
(kp, open)
});
join_all(checks)
.await
.into_iter()
.filter_map(|(kp, open)| open.then_some(kp))
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::net::TcpListener;
#[tokio::test]
async fn scan_reports_only_open_ports() {
// Bind one port (open) and pick another that is closed.
let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind");
let open_port = listener.local_addr().expect("addr").port();
let ports = [
KnownPort {
port: open_port,
service: "test-open",
kind: PortKind::Ics,
note: "",
},
KnownPort {
port: 1,
service: "test-closed",
kind: PortKind::InsecureMgmt,
note: "",
},
];
let found = scan("127.0.0.1", &ports, Duration::from_millis(500)).await;
let services: Vec<&str> = found.iter().map(|p| p.service).collect();
assert_eq!(services, vec!["test-open"]);
}
}
@@ -0,0 +1,260 @@
use mongodb::bson::doc;
use crate::pipeline::repo_view::RepoView;
use compliance_core::models::*;
use super::orchestrator::{extract_base_url, PipelineOrchestrator};
use super::tracker_dispatch::TrackerDispatch;
use crate::error::AgentError;
use crate::trackers;
impl PipelineOrchestrator {
/// Build an issue tracker client from a repository's tracker configuration.
/// Returns `None` if the repo has no tracker configured.
pub(super) fn build_tracker(&self, repo: &RepoView) -> Option<TrackerDispatch> {
let tracker_type = repo.tracker_type.as_ref()?;
// Per-repo token takes precedence, fall back to global config
match tracker_type {
TrackerType::GitHub => {
let token = repo.tracker_token.clone().or_else(|| {
self.config.github_token.as_ref().map(|t| {
use secrecy::ExposeSecret;
t.expose_secret().to_string()
})
})?;
let secret = secrecy::SecretString::from(token);
match trackers::github::GitHubTracker::new(&secret) {
Ok(t) => Some(TrackerDispatch::GitHub(t)),
Err(e) => {
tracing::warn!("Failed to build GitHub tracker: {e}");
None
}
}
}
TrackerType::GitLab => {
let base_url = self
.config
.gitlab_url
.clone()
.unwrap_or_else(|| "https://gitlab.com".to_string());
let token = repo.tracker_token.clone().or_else(|| {
self.config.gitlab_token.as_ref().map(|t| {
use secrecy::ExposeSecret;
t.expose_secret().to_string()
})
})?;
let secret = secrecy::SecretString::from(token);
Some(TrackerDispatch::GitLab(
trackers::gitlab::GitLabTracker::new(base_url, secret),
))
}
TrackerType::Gitea => {
let token = repo.tracker_token.clone()?;
let base_url = extract_base_url(&repo.git_url)?;
let secret = secrecy::SecretString::from(token);
Some(TrackerDispatch::Gitea(trackers::gitea::GiteaTracker::new(
base_url, secret,
)))
}
TrackerType::Jira => {
let base_url = self.config.jira_url.clone()?;
let email = self.config.jira_email.clone()?;
let project_key = self.config.jira_project_key.clone()?;
let token = repo.tracker_token.clone().or_else(|| {
self.config.jira_api_token.as_ref().map(|t| {
use secrecy::ExposeSecret;
t.expose_secret().to_string()
})
})?;
let secret = secrecy::SecretString::from(token);
Some(TrackerDispatch::Jira(trackers::jira::JiraTracker::new(
base_url,
email,
secret,
project_key,
)))
}
}
}
/// Create tracker issues for new findings (severity >= Medium).
/// Checks for duplicates via fingerprint search before creating.
#[tracing::instrument(skip_all, fields(repo_id = %repo_id))]
pub(super) async fn create_tracker_issues(
&self,
repo: &RepoView,
repo_id: &str,
new_findings: &[Finding],
) -> Result<(), AgentError> {
let tracker = match self.build_tracker(repo) {
Some(t) => t,
None => {
tracing::info!("[{repo_id}] No issue tracker configured, skipping");
return Ok(());
}
};
let owner = match repo.tracker_owner.as_deref() {
Some(o) => o,
None => {
tracing::warn!("[{repo_id}] tracker_owner not set, skipping issue creation");
return Ok(());
}
};
let tracker_repo_name = match repo.tracker_repo.as_deref() {
Some(r) => r,
None => {
tracing::warn!("[{repo_id}] tracker_repo not set, skipping issue creation");
return Ok(());
}
};
// Only create issues for medium+ severity findings
let actionable: Vec<&Finding> = new_findings
.iter()
.filter(|f| {
matches!(
f.severity,
Severity::Medium | Severity::High | Severity::Critical
)
})
.collect();
if actionable.is_empty() {
tracing::info!("[{repo_id}] No medium+ findings, skipping issue creation");
return Ok(());
}
tracing::info!(
"[{repo_id}] Creating issues for {} findings via {}",
actionable.len(),
tracker.name()
);
let mut created = 0u32;
for finding in actionable {
let title = format!(
"[{}] {}: {}",
finding.severity, finding.scanner, finding.title
);
// Check if an issue already exists by fingerprint first, then by title
let mut found_existing = false;
for search_term in [&finding.fingerprint, &title] {
match tracker
.find_existing_issue(owner, tracker_repo_name, search_term)
.await
{
Ok(Some(existing)) => {
tracing::debug!(
"[{repo_id}] Issue already exists for '{}': {}",
search_term,
existing.external_url
);
found_existing = true;
break;
}
Ok(None) => {}
Err(e) => {
tracing::warn!("[{repo_id}] Failed to search for existing issue: {e}");
}
}
}
if found_existing {
continue;
}
let body = format_issue_body(finding);
let labels = vec![
format!("severity:{}", finding.severity),
format!("scanner:{}", finding.scanner),
"compliance-scanner".to_string(),
];
match tracker
.create_issue(owner, tracker_repo_name, &title, &body, &labels)
.await
{
Ok(mut issue) => {
issue.finding_id = finding
.id
.as_ref()
.map(|id| id.to_hex())
.unwrap_or_default();
// Update the finding with the issue URL
if let Some(finding_id) = &finding.id {
let _ = self
.db
.findings()
.update_one(
doc! { "_id": finding_id },
doc! { "$set": { "tracker_issue_url": &issue.external_url } },
)
.await;
}
// Store the tracker issue record
if let Err(e) = self.db.tracker_issues().insert_one(&issue).await {
tracing::warn!("[{repo_id}] Failed to store tracker issue: {e}");
}
created += 1;
}
Err(e) => {
tracing::warn!(
"[{repo_id}] Failed to create issue for {}: {e}",
finding.fingerprint
);
}
}
}
tracing::info!("[{repo_id}] Created {created} tracker issues");
Ok(())
}
}
/// Format a finding into a markdown issue body for the tracker.
pub(super) fn format_issue_body(finding: &Finding) -> String {
let mut body = String::new();
body.push_str(&format!("## {} Finding\n\n", finding.severity));
body.push_str(&format!("**Scanner:** {}\n", finding.scanner));
body.push_str(&format!("**Severity:** {}\n", finding.severity));
if let Some(rule) = &finding.rule_id {
body.push_str(&format!("**Rule:** {}\n", rule));
}
if let Some(cwe) = &finding.cwe {
body.push_str(&format!("**CWE:** {}\n", cwe));
}
body.push_str(&format!("\n### Description\n\n{}\n", finding.description));
if let Some(file_path) = &finding.file_path {
body.push_str(&format!("\n### Location\n\n**File:** `{}`", file_path));
if let Some(line) = finding.line_number {
body.push_str(&format!(" (line {})", line));
}
body.push('\n');
}
if let Some(snippet) = &finding.code_snippet {
body.push_str(&format!("\n### Code\n\n```\n{}\n```\n", snippet));
}
if let Some(remediation) = &finding.remediation {
body.push_str(&format!("\n### Remediation\n\n{}\n", remediation));
}
if let Some(fix) = &finding.suggested_fix {
body.push_str(&format!("\n### Suggested Fix\n\n```\n{}\n```\n", fix));
}
body.push_str(&format!(
"\n---\n*Fingerprint:* `{}`\n*Generated by compliance-scanner*",
finding.fingerprint
));
body
}
-366
View File
@@ -1,366 +0,0 @@
use std::path::Path;
use std::time::Duration;
use compliance_core::models::{Finding, ScanType, Severity};
use compliance_core::traits::{ScanOutput, Scanner};
use compliance_core::CoreError;
use tokio::process::Command;
use crate::pipeline::dedup;
/// Timeout for each individual lint command
const LINT_TIMEOUT: Duration = Duration::from_secs(120);
pub struct LintScanner;
impl Scanner for LintScanner {
fn name(&self) -> &str {
"lint"
}
fn scan_type(&self) -> ScanType {
ScanType::Lint
}
#[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let mut all_findings = Vec::new();
// Detect which languages are present and run appropriate linters
if has_rust_project(repo_path) {
match run_clippy(repo_path, repo_id).await {
Ok(findings) => all_findings.extend(findings),
Err(e) => tracing::warn!("Clippy failed: {e}"),
}
}
if has_js_project(repo_path) {
match run_eslint(repo_path, repo_id).await {
Ok(findings) => all_findings.extend(findings),
Err(e) => tracing::warn!("ESLint failed: {e}"),
}
}
if has_python_project(repo_path) {
match run_ruff(repo_path, repo_id).await {
Ok(findings) => all_findings.extend(findings),
Err(e) => tracing::warn!("Ruff failed: {e}"),
}
}
Ok(ScanOutput {
findings: all_findings,
sbom_entries: Vec::new(),
})
}
}
fn has_rust_project(repo_path: &Path) -> bool {
repo_path.join("Cargo.toml").exists()
}
fn has_js_project(repo_path: &Path) -> bool {
// Only run if eslint is actually installed in the project
repo_path.join("package.json").exists() && repo_path.join("node_modules/.bin/eslint").exists()
}
fn has_python_project(repo_path: &Path) -> bool {
repo_path.join("pyproject.toml").exists()
|| repo_path.join("setup.py").exists()
|| repo_path.join("requirements.txt").exists()
}
/// Run a command with a timeout, returning its output or an error
async fn run_with_timeout(
child: tokio::process::Child,
scanner_name: &str,
) -> Result<std::process::Output, CoreError> {
let result = tokio::time::timeout(LINT_TIMEOUT, child.wait_with_output()).await;
match result {
Ok(Ok(output)) => Ok(output),
Ok(Err(e)) => Err(CoreError::Scanner {
scanner: scanner_name.to_string(),
source: Box::new(e),
}),
Err(_) => {
// Process is dropped here which sends SIGKILL on Unix
Err(CoreError::Scanner {
scanner: scanner_name.to_string(),
source: Box::new(std::io::Error::new(
std::io::ErrorKind::TimedOut,
format!("{scanner_name} timed out after {}s", LINT_TIMEOUT.as_secs()),
)),
})
}
}
}
// ── Clippy ──────────────────────────────────────────────
async fn run_clippy(repo_path: &Path, repo_id: &str) -> Result<Vec<Finding>, CoreError> {
let child = Command::new("cargo")
.args([
"clippy",
"--message-format=json",
"--quiet",
"--",
"-W",
"clippy::all",
])
.current_dir(repo_path)
.env("RUSTC_WRAPPER", "")
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| CoreError::Scanner {
scanner: "clippy".to_string(),
source: Box::new(e),
})?;
let output = run_with_timeout(child, "clippy").await?;
let stdout = String::from_utf8_lossy(&output.stdout);
let mut findings = Vec::new();
for line in stdout.lines() {
let msg: serde_json::Value = match serde_json::from_str(line) {
Ok(v) => v,
Err(_) => continue,
};
if msg.get("reason").and_then(|v| v.as_str()) != Some("compiler-message") {
continue;
}
let message = match msg.get("message") {
Some(m) => m,
None => continue,
};
let level = message.get("level").and_then(|v| v.as_str()).unwrap_or("");
if level != "warning" && level != "error" {
continue;
}
let text = message
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let code = message
.get("code")
.and_then(|v| v.get("code"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
if text.starts_with("aborting due to") || code.is_empty() {
continue;
}
let (file_path, line_number) = extract_primary_span(message);
let severity = if level == "error" {
Severity::High
} else {
Severity::Low
};
let fingerprint = dedup::compute_fingerprint(&[
repo_id,
"clippy",
&code,
&file_path,
&line_number.to_string(),
]);
let mut finding = Finding::new(
repo_id.to_string(),
fingerprint,
"clippy".to_string(),
ScanType::Lint,
format!("[clippy] {text}"),
text,
severity,
);
finding.rule_id = Some(code);
if !file_path.is_empty() {
finding.file_path = Some(file_path);
}
if line_number > 0 {
finding.line_number = Some(line_number);
}
findings.push(finding);
}
Ok(findings)
}
fn extract_primary_span(message: &serde_json::Value) -> (String, u32) {
let spans = match message.get("spans").and_then(|v| v.as_array()) {
Some(s) => s,
None => return (String::new(), 0),
};
for span in spans {
if span.get("is_primary").and_then(|v| v.as_bool()) == Some(true) {
let file = span
.get("file_name")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let line = span.get("line_start").and_then(|v| v.as_u64()).unwrap_or(0) as u32;
return (file, line);
}
}
(String::new(), 0)
}
// ── ESLint ──────────────────────────────────────────────
async fn run_eslint(repo_path: &Path, repo_id: &str) -> Result<Vec<Finding>, CoreError> {
// Use the project-local eslint binary directly, not npx (which can hang downloading)
let eslint_bin = repo_path.join("node_modules/.bin/eslint");
let child = Command::new(eslint_bin)
.args([".", "--format", "json", "--no-error-on-unmatched-pattern"])
.current_dir(repo_path)
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| CoreError::Scanner {
scanner: "eslint".to_string(),
source: Box::new(e),
})?;
let output = run_with_timeout(child, "eslint").await?;
if output.stdout.is_empty() {
return Ok(Vec::new());
}
let results: Vec<EslintFileResult> = serde_json::from_slice(&output.stdout).unwrap_or_default();
let mut findings = Vec::new();
for file_result in results {
for msg in file_result.messages {
let severity = match msg.severity {
2 => Severity::Medium,
_ => Severity::Low,
};
let rule_id = msg.rule_id.unwrap_or_default();
let fingerprint = dedup::compute_fingerprint(&[
repo_id,
"eslint",
&rule_id,
&file_result.file_path,
&msg.line.to_string(),
]);
let mut finding = Finding::new(
repo_id.to_string(),
fingerprint,
"eslint".to_string(),
ScanType::Lint,
format!("[eslint] {}", msg.message),
msg.message,
severity,
);
finding.rule_id = Some(rule_id);
finding.file_path = Some(file_result.file_path.clone());
finding.line_number = Some(msg.line);
findings.push(finding);
}
}
Ok(findings)
}
#[derive(serde::Deserialize)]
struct EslintFileResult {
#[serde(rename = "filePath")]
file_path: String,
messages: Vec<EslintMessage>,
}
#[derive(serde::Deserialize)]
struct EslintMessage {
#[serde(rename = "ruleId")]
rule_id: Option<String>,
severity: u8,
message: String,
line: u32,
}
// ── Ruff ────────────────────────────────────────────────
async fn run_ruff(repo_path: &Path, repo_id: &str) -> Result<Vec<Finding>, CoreError> {
let child = Command::new("ruff")
.args(["check", ".", "--output-format", "json", "--exit-zero"])
.current_dir(repo_path)
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| CoreError::Scanner {
scanner: "ruff".to_string(),
source: Box::new(e),
})?;
let output = run_with_timeout(child, "ruff").await?;
if output.stdout.is_empty() {
return Ok(Vec::new());
}
let results: Vec<RuffResult> = serde_json::from_slice(&output.stdout).unwrap_or_default();
let findings = results
.into_iter()
.map(|r| {
let severity = if r.code.starts_with('E') || r.code.starts_with('F') {
Severity::Medium
} else {
Severity::Low
};
let fingerprint = dedup::compute_fingerprint(&[
repo_id,
"ruff",
&r.code,
&r.filename,
&r.location.row.to_string(),
]);
let mut finding = Finding::new(
repo_id.to_string(),
fingerprint,
"ruff".to_string(),
ScanType::Lint,
format!("[ruff] {}: {}", r.code, r.message),
r.message,
severity,
);
finding.rule_id = Some(r.code);
finding.file_path = Some(r.filename);
finding.line_number = Some(r.location.row);
finding
})
.collect();
Ok(findings)
}
#[derive(serde::Deserialize)]
struct RuffResult {
code: String,
message: String,
filename: String,
location: RuffLocation,
}
#[derive(serde::Deserialize)]
struct RuffLocation {
row: u32,
}
@@ -0,0 +1,251 @@
use std::path::Path;
use compliance_core::models::{Finding, ScanType, Severity};
use compliance_core::CoreError;
use tokio::process::Command;
use crate::pipeline::dedup;
use super::run_with_timeout;
pub(super) async fn run_clippy(repo_path: &Path, repo_id: &str) -> Result<Vec<Finding>, CoreError> {
let child = Command::new("cargo")
.args([
"clippy",
"--message-format=json",
"--quiet",
"--",
"-W",
"clippy::all",
])
.current_dir(repo_path)
.env("RUSTC_WRAPPER", "")
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| CoreError::Scanner {
scanner: "clippy".to_string(),
source: Box::new(e),
})?;
let output = run_with_timeout(child, "clippy").await?;
let stdout = String::from_utf8_lossy(&output.stdout);
let mut findings = Vec::new();
for line in stdout.lines() {
let msg: serde_json::Value = match serde_json::from_str(line) {
Ok(v) => v,
Err(_) => continue,
};
if msg.get("reason").and_then(|v| v.as_str()) != Some("compiler-message") {
continue;
}
let message = match msg.get("message") {
Some(m) => m,
None => continue,
};
let level = message.get("level").and_then(|v| v.as_str()).unwrap_or("");
if level != "warning" && level != "error" {
continue;
}
let text = message
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let code = message
.get("code")
.and_then(|v| v.get("code"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
if text.starts_with("aborting due to") || code.is_empty() {
continue;
}
let (file_path, line_number) = extract_primary_span(message);
let severity = if level == "error" {
Severity::High
} else {
Severity::Low
};
let fingerprint = dedup::compute_fingerprint(&[
repo_id,
"clippy",
&code,
&file_path,
&line_number.to_string(),
]);
let mut finding = Finding::new(
repo_id.to_string(),
fingerprint,
"clippy".to_string(),
ScanType::Lint,
format!("[clippy] {text}"),
text,
severity,
);
finding.rule_id = Some(code);
if !file_path.is_empty() {
finding.file_path = Some(file_path);
}
if line_number > 0 {
finding.line_number = Some(line_number);
}
findings.push(finding);
}
Ok(findings)
}
fn extract_primary_span(message: &serde_json::Value) -> (String, u32) {
let spans = match message.get("spans").and_then(|v| v.as_array()) {
Some(s) => s,
None => return (String::new(), 0),
};
for span in spans {
if span.get("is_primary").and_then(|v| v.as_bool()) == Some(true) {
let file = span
.get("file_name")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let line = span.get("line_start").and_then(|v| v.as_u64()).unwrap_or(0) as u32;
return (file, line);
}
}
(String::new(), 0)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn extract_primary_span_with_primary() {
let msg = serde_json::json!({
"spans": [
{
"file_name": "src/lib.rs",
"line_start": 42,
"is_primary": true
}
]
});
let (file, line) = extract_primary_span(&msg);
assert_eq!(file, "src/lib.rs");
assert_eq!(line, 42);
}
#[test]
fn extract_primary_span_no_primary() {
let msg = serde_json::json!({
"spans": [
{
"file_name": "src/lib.rs",
"line_start": 42,
"is_primary": false
}
]
});
let (file, line) = extract_primary_span(&msg);
assert_eq!(file, "");
assert_eq!(line, 0);
}
#[test]
fn extract_primary_span_multiple_spans() {
let msg = serde_json::json!({
"spans": [
{
"file_name": "src/other.rs",
"line_start": 10,
"is_primary": false
},
{
"file_name": "src/main.rs",
"line_start": 99,
"is_primary": true
}
]
});
let (file, line) = extract_primary_span(&msg);
assert_eq!(file, "src/main.rs");
assert_eq!(line, 99);
}
#[test]
fn extract_primary_span_no_spans() {
let msg = serde_json::json!({});
let (file, line) = extract_primary_span(&msg);
assert_eq!(file, "");
assert_eq!(line, 0);
}
#[test]
fn extract_primary_span_empty_spans() {
let msg = serde_json::json!({ "spans": [] });
let (file, line) = extract_primary_span(&msg);
assert_eq!(file, "");
assert_eq!(line, 0);
}
#[test]
fn parse_clippy_compiler_message_line() {
let line = r#"{"reason":"compiler-message","message":{"level":"warning","message":"unused variable","code":{"code":"unused_variables"},"spans":[{"file_name":"src/main.rs","line_start":5,"is_primary":true}]}}"#;
let msg: serde_json::Value = serde_json::from_str(line).unwrap();
assert_eq!(
msg.get("reason").and_then(|v| v.as_str()),
Some("compiler-message")
);
let message = msg.get("message").unwrap();
assert_eq!(
message.get("level").and_then(|v| v.as_str()),
Some("warning")
);
assert_eq!(
message.get("message").and_then(|v| v.as_str()),
Some("unused variable")
);
assert_eq!(
message
.get("code")
.and_then(|v| v.get("code"))
.and_then(|v| v.as_str()),
Some("unused_variables")
);
let (file, line_num) = extract_primary_span(message);
assert_eq!(file, "src/main.rs");
assert_eq!(line_num, 5);
}
#[test]
fn skip_non_compiler_message() {
let line = r#"{"reason":"build-script-executed","package_id":"foo 0.1.0"}"#;
let msg: serde_json::Value = serde_json::from_str(line).unwrap();
assert_ne!(
msg.get("reason").and_then(|v| v.as_str()),
Some("compiler-message")
);
}
#[test]
fn skip_aborting_message() {
let text = "aborting due to 3 previous errors";
assert!(text.starts_with("aborting due to"));
}
}
@@ -0,0 +1,183 @@
use std::path::Path;
use compliance_core::models::{Finding, ScanType, Severity};
use compliance_core::CoreError;
use tokio::process::Command;
use crate::pipeline::dedup;
use super::run_with_timeout;
pub(super) async fn run_eslint(repo_path: &Path, repo_id: &str) -> Result<Vec<Finding>, CoreError> {
// Use the project-local eslint binary directly, not npx (which can hang downloading)
let eslint_bin = repo_path.join("node_modules/.bin/eslint");
let child = Command::new(eslint_bin)
.args([".", "--format", "json", "--no-error-on-unmatched-pattern"])
.current_dir(repo_path)
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| CoreError::Scanner {
scanner: "eslint".to_string(),
source: Box::new(e),
})?;
let output = run_with_timeout(child, "eslint").await?;
if output.stdout.is_empty() {
return Ok(Vec::new());
}
let results: Vec<EslintFileResult> = serde_json::from_slice(&output.stdout).unwrap_or_default();
let mut findings = Vec::new();
for file_result in results {
for msg in file_result.messages {
let severity = match msg.severity {
2 => Severity::Medium,
_ => Severity::Low,
};
let rule_id = msg.rule_id.unwrap_or_default();
let fingerprint = dedup::compute_fingerprint(&[
repo_id,
"eslint",
&rule_id,
&file_result.file_path,
&msg.line.to_string(),
]);
let mut finding = Finding::new(
repo_id.to_string(),
fingerprint,
"eslint".to_string(),
ScanType::Lint,
format!("[eslint] {}", msg.message),
msg.message,
severity,
);
finding.rule_id = Some(rule_id);
finding.file_path = Some(file_result.file_path.clone());
finding.line_number = Some(msg.line);
findings.push(finding);
}
}
Ok(findings)
}
#[derive(serde::Deserialize)]
struct EslintFileResult {
#[serde(rename = "filePath")]
file_path: String,
messages: Vec<EslintMessage>,
}
#[derive(serde::Deserialize)]
struct EslintMessage {
#[serde(rename = "ruleId")]
rule_id: Option<String>,
severity: u8,
message: String,
line: u32,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn deserialize_eslint_output() {
let json = r#"[
{
"filePath": "/home/user/project/src/app.js",
"messages": [
{
"ruleId": "no-unused-vars",
"severity": 2,
"message": "'x' is defined but never used.",
"line": 10
},
{
"ruleId": "semi",
"severity": 1,
"message": "Missing semicolon.",
"line": 15
}
]
}
]"#;
let results: Vec<EslintFileResult> = serde_json::from_str(json).unwrap();
assert_eq!(results.len(), 1);
assert_eq!(results[0].file_path, "/home/user/project/src/app.js");
assert_eq!(results[0].messages.len(), 2);
assert_eq!(
results[0].messages[0].rule_id,
Some("no-unused-vars".to_string())
);
assert_eq!(results[0].messages[0].severity, 2);
assert_eq!(results[0].messages[0].line, 10);
assert_eq!(results[0].messages[1].severity, 1);
}
#[test]
fn deserialize_eslint_null_rule_id() {
let json = r#"[
{
"filePath": "src/index.js",
"messages": [
{
"ruleId": null,
"severity": 2,
"message": "Parsing error: Unexpected token",
"line": 1
}
]
}
]"#;
let results: Vec<EslintFileResult> = serde_json::from_str(json).unwrap();
assert_eq!(results[0].messages[0].rule_id, None);
}
#[test]
fn deserialize_eslint_empty_messages() {
let json = r#"[{"filePath": "src/clean.js", "messages": []}]"#;
let results: Vec<EslintFileResult> = serde_json::from_str(json).unwrap();
assert_eq!(results[0].messages.len(), 0);
}
#[test]
fn deserialize_eslint_empty_array() {
let json = "[]";
let results: Vec<EslintFileResult> = serde_json::from_str(json).unwrap();
assert!(results.is_empty());
}
#[test]
fn eslint_severity_mapping() {
// severity 2 = error -> Medium, anything else -> Low
assert_eq!(
match 2u8 {
2 => "Medium",
_ => "Low",
},
"Medium"
);
assert_eq!(
match 1u8 {
2 => "Medium",
_ => "Low",
},
"Low"
);
assert_eq!(
match 0u8 {
2 => "Medium",
_ => "Low",
},
"Low"
);
}
}
+97
View File
@@ -0,0 +1,97 @@
mod clippy;
mod eslint;
mod ruff;
use std::path::Path;
use std::time::Duration;
use compliance_core::models::ScanType;
use compliance_core::traits::{ScanOutput, Scanner};
use compliance_core::CoreError;
/// Timeout for each individual lint command
pub(crate) const LINT_TIMEOUT: Duration = Duration::from_secs(120);
pub struct LintScanner;
impl Scanner for LintScanner {
fn name(&self) -> &str {
"lint"
}
fn scan_type(&self) -> ScanType {
ScanType::Lint
}
#[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let mut all_findings = Vec::new();
// Detect which languages are present and run appropriate linters
if has_rust_project(repo_path) {
match clippy::run_clippy(repo_path, repo_id).await {
Ok(findings) => all_findings.extend(findings),
Err(e) => tracing::warn!("Clippy failed: {e}"),
}
}
if has_js_project(repo_path) {
match eslint::run_eslint(repo_path, repo_id).await {
Ok(findings) => all_findings.extend(findings),
Err(e) => tracing::warn!("ESLint failed: {e}"),
}
}
if has_python_project(repo_path) {
match ruff::run_ruff(repo_path, repo_id).await {
Ok(findings) => all_findings.extend(findings),
Err(e) => tracing::warn!("Ruff failed: {e}"),
}
}
Ok(ScanOutput {
findings: all_findings,
sbom_entries: Vec::new(),
})
}
}
fn has_rust_project(repo_path: &Path) -> bool {
repo_path.join("Cargo.toml").exists()
}
fn has_js_project(repo_path: &Path) -> bool {
// Only run if eslint is actually installed in the project
repo_path.join("package.json").exists() && repo_path.join("node_modules/.bin/eslint").exists()
}
fn has_python_project(repo_path: &Path) -> bool {
repo_path.join("pyproject.toml").exists()
|| repo_path.join("setup.py").exists()
|| repo_path.join("requirements.txt").exists()
}
/// Run a command with a timeout, returning its output or an error
pub(crate) async fn run_with_timeout(
child: tokio::process::Child,
scanner_name: &str,
) -> Result<std::process::Output, CoreError> {
let result = tokio::time::timeout(LINT_TIMEOUT, child.wait_with_output()).await;
match result {
Ok(Ok(output)) => Ok(output),
Ok(Err(e)) => Err(CoreError::Scanner {
scanner: scanner_name.to_string(),
source: Box::new(e),
}),
Err(_) => {
// Process is dropped here which sends SIGKILL on Unix
Err(CoreError::Scanner {
scanner: scanner_name.to_string(),
source: Box::new(std::io::Error::new(
std::io::ErrorKind::TimedOut,
format!("{scanner_name} timed out after {}s", LINT_TIMEOUT.as_secs()),
)),
})
}
}
}
+150
View File
@@ -0,0 +1,150 @@
use std::path::Path;
use compliance_core::models::{Finding, ScanType, Severity};
use compliance_core::CoreError;
use tokio::process::Command;
use crate::pipeline::dedup;
use super::run_with_timeout;
pub(super) async fn run_ruff(repo_path: &Path, repo_id: &str) -> Result<Vec<Finding>, CoreError> {
let child = Command::new("ruff")
.args(["check", ".", "--output-format", "json", "--exit-zero"])
.current_dir(repo_path)
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.map_err(|e| CoreError::Scanner {
scanner: "ruff".to_string(),
source: Box::new(e),
})?;
let output = run_with_timeout(child, "ruff").await?;
if output.stdout.is_empty() {
return Ok(Vec::new());
}
let results: Vec<RuffResult> = serde_json::from_slice(&output.stdout).unwrap_or_default();
let findings = results
.into_iter()
.map(|r| {
let severity = if r.code.starts_with('E') || r.code.starts_with('F') {
Severity::Medium
} else {
Severity::Low
};
let fingerprint = dedup::compute_fingerprint(&[
repo_id,
"ruff",
&r.code,
&r.filename,
&r.location.row.to_string(),
]);
let mut finding = Finding::new(
repo_id.to_string(),
fingerprint,
"ruff".to_string(),
ScanType::Lint,
format!("[ruff] {}: {}", r.code, r.message),
r.message,
severity,
);
finding.rule_id = Some(r.code);
finding.file_path = Some(r.filename);
finding.line_number = Some(r.location.row);
finding
})
.collect();
Ok(findings)
}
#[derive(serde::Deserialize)]
struct RuffResult {
code: String,
message: String,
filename: String,
location: RuffLocation,
}
#[derive(serde::Deserialize)]
struct RuffLocation {
row: u32,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn deserialize_ruff_output() {
let json = r#"[
{
"code": "E501",
"message": "Line too long (120 > 79 characters)",
"filename": "src/main.py",
"location": {"row": 42}
},
{
"code": "F401",
"message": "`os` imported but unused",
"filename": "src/utils.py",
"location": {"row": 1}
}
]"#;
let results: Vec<RuffResult> = serde_json::from_str(json).unwrap();
assert_eq!(results.len(), 2);
assert_eq!(results[0].code, "E501");
assert_eq!(results[0].filename, "src/main.py");
assert_eq!(results[0].location.row, 42);
assert_eq!(results[1].code, "F401");
assert_eq!(results[1].location.row, 1);
}
#[test]
fn deserialize_ruff_empty() {
let json = "[]";
let results: Vec<RuffResult> = serde_json::from_str(json).unwrap();
assert!(results.is_empty());
}
#[test]
fn ruff_severity_e_and_f_are_medium() {
for code in &["E501", "E302", "F401", "F811"] {
let is_medium = code.starts_with('E') || code.starts_with('F');
assert!(is_medium, "Expected {code} to be Medium severity");
}
}
#[test]
fn ruff_severity_others_are_low() {
for code in &["W291", "I001", "D100", "C901", "N801"] {
let is_medium = code.starts_with('E') || code.starts_with('F');
assert!(!is_medium, "Expected {code} to be Low severity");
}
}
#[test]
fn deserialize_ruff_with_extra_fields() {
// Ruff output may contain additional fields we don't use
let json = r#"[{
"code": "W291",
"message": "Trailing whitespace",
"filename": "app.py",
"location": {"row": 3, "column": 10},
"end_location": {"row": 3, "column": 11},
"fix": null,
"noqa_row": 3
}]"#;
let results: Vec<RuffResult> = serde_json::from_str(json).unwrap();
assert_eq!(results.len(), 1);
assert_eq!(results[0].code, "W291");
}
}
+9
View File
@@ -1,10 +1,19 @@
pub mod code_review;
pub mod cve;
pub mod dedup;
pub mod firmware_sbom;
pub mod git;
pub mod gitleaks;
mod graph_build;
pub mod ics;
mod issue_creation;
pub mod lint;
pub mod orchestrator;
pub mod patterns;
pub mod plan;
pub mod plc;
mod pr_review;
pub mod repo_view;
pub mod sbom;
pub mod semgrep;
mod tracker_dispatch;
File diff suppressed because it is too large Load Diff
+159 -1
View File
@@ -33,6 +33,7 @@ struct PatternRule {
file_extensions: Vec<String>,
}
#[allow(clippy::new_without_default)]
impl GdprPatternScanner {
pub fn new() -> Self {
let patterns = vec![
@@ -98,6 +99,7 @@ impl Scanner for GdprPatternScanner {
}
}
#[allow(clippy::new_without_default)]
impl OAuthPatternScanner {
pub fn new() -> Self {
let patterns = vec![
@@ -213,7 +215,7 @@ fn scan_with_patterns(
repo_id.to_string(),
fingerprint,
scanner_name.to_string(),
scan_type.clone(),
scan_type,
pattern.title.clone(),
pattern.description.clone(),
pattern.severity.clone(),
@@ -256,3 +258,159 @@ fn walkdir(path: &Path) -> Result<Vec<walkdir::DirEntry>, CoreError> {
Ok(entries)
}
#[cfg(test)]
mod tests {
use super::*;
// --- compile_regex tests ---
#[test]
fn compile_regex_valid_pattern() {
let re = compile_regex(r"\bfoo\b");
assert!(re.is_match("hello foo bar"));
assert!(!re.is_match("foobar"));
}
#[test]
fn compile_regex_invalid_pattern_returns_fallback() {
// An invalid regex should return the fallback "^$" that only matches empty strings
let re = compile_regex(r"[invalid");
assert!(re.is_match(""));
assert!(!re.is_match("anything"));
}
// --- GDPR pattern tests ---
#[test]
fn gdpr_pii_logging_matches() {
let scanner = GdprPatternScanner::new();
let pattern = &scanner.patterns[0]; // gdpr-pii-logging
// Regex: (log|print|console\.|logger\.|tracing::)\s*[\.(].*\b(pii_keyword)\b
assert!(pattern.pattern.is_match("console.log(email)"));
assert!(pattern.pattern.is_match("console.log(user.ssn)"));
assert!(pattern.pattern.is_match("print(phone_number)"));
assert!(pattern.pattern.is_match("tracing::(ip_addr)"));
assert!(pattern.pattern.is_match("log.debug(credit_card)"));
}
#[test]
fn gdpr_pii_logging_no_false_positive() {
let scanner = GdprPatternScanner::new();
let pattern = &scanner.patterns[0];
// Regular logging without PII fields should not match
assert!(!pattern
.pattern
.is_match("logger.info(\"request completed\")"));
assert!(!pattern.pattern.is_match("let email = user.email;"));
}
#[test]
fn gdpr_no_consent_matches() {
let scanner = GdprPatternScanner::new();
let pattern = &scanner.patterns[1]; // gdpr-no-consent
assert!(pattern.pattern.is_match("collect personal data"));
assert!(pattern.pattern.is_match("store user_data in db"));
assert!(pattern.pattern.is_match("save pii to disk"));
}
#[test]
fn gdpr_user_model_matches() {
let scanner = GdprPatternScanner::new();
let pattern = &scanner.patterns[2]; // gdpr-no-delete-endpoint
assert!(pattern.pattern.is_match("struct User {"));
assert!(pattern.pattern.is_match("class User(Model):"));
}
#[test]
fn gdpr_hardcoded_retention_matches() {
let scanner = GdprPatternScanner::new();
let pattern = &scanner.patterns[3]; // gdpr-hardcoded-retention
assert!(pattern.pattern.is_match("retention = 30"));
assert!(pattern.pattern.is_match("ttl: 3600"));
assert!(pattern.pattern.is_match("expire = 86400"));
}
// --- OAuth pattern tests ---
#[test]
fn oauth_implicit_grant_matches() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[0]; // oauth-implicit-grant
assert!(pattern.pattern.is_match("response_type = \"token\""));
assert!(pattern.pattern.is_match("grant_type: implicit"));
assert!(pattern.pattern.is_match("response_type='token'"));
}
#[test]
fn oauth_implicit_grant_no_false_positive() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[0];
assert!(!pattern.pattern.is_match("response_type = \"code\""));
assert!(!pattern.pattern.is_match("grant_type: authorization_code"));
}
#[test]
fn oauth_authorization_code_matches() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[1]; // oauth-missing-pkce
assert!(pattern.pattern.is_match("uses authorization_code flow"));
assert!(pattern.pattern.is_match("authorization code grant"));
}
#[test]
fn oauth_token_localstorage_matches() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[2]; // oauth-token-localstorage
assert!(pattern
.pattern
.is_match("localStorage.setItem('access_token', tok)"));
assert!(pattern
.pattern
.is_match("localStorage.getItem(\"refresh_token\")"));
}
#[test]
fn oauth_token_localstorage_no_false_positive() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[2];
assert!(!pattern
.pattern
.is_match("localStorage.setItem('theme', 'dark')"));
assert!(!pattern
.pattern
.is_match("sessionStorage.setItem('token', t)"));
}
#[test]
fn oauth_token_url_matches() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[3]; // oauth-token-url
assert!(pattern.pattern.is_match("access_token = build_url(query)"));
assert!(pattern.pattern.is_match("bearer = url.param"));
}
// --- Pattern rule file extension filtering ---
#[test]
fn gdpr_patterns_cover_common_languages() {
let scanner = GdprPatternScanner::new();
for pattern in &scanner.patterns {
assert!(
pattern.file_extensions.contains(&"rs".to_string()),
"Pattern {} should cover .rs files",
pattern.id
);
}
}
#[test]
fn oauth_localstorage_only_js_ts() {
let scanner = OAuthPatternScanner::new();
let pattern = &scanner.patterns[2]; // oauth-token-localstorage
assert!(pattern.file_extensions.contains(&"js".to_string()));
assert!(pattern.file_extensions.contains(&"ts".to_string()));
assert!(!pattern.file_extensions.contains(&"rs".to_string()));
assert!(!pattern.file_extensions.contains(&"py".to_string()));
}
}
+214
View File
@@ -0,0 +1,214 @@
//! The scan plan.
//!
//! [`build_scan_plan`] turns an [`OnboardedTarget`] into the concrete ordered
//! list of scans to run, each bound to the artifact it consumes. It intersects
//! the scan-applicability matrix ([`applicable_scans`]) with the target's
//! `scan_config` overrides: a scan runs when its required artifact is present and
//! it is either on by default or explicitly enabled, and is not explicitly
//! disabled. This is the decision engine the unified pipeline (`run_target`)
//! executes.
use compliance_core::models::{Artifact, ArtifactKind, OnboardedTarget, ScanPhase, ScanType};
use compliance_core::scan_matrix::applicable_scans;
/// One scan to run, bound to the artifact it operates on.
#[derive(Debug, Clone, PartialEq)]
pub struct ScanStep {
/// The scan to run.
pub scan_type: ScanType,
/// The pipeline phase to report while it runs.
pub phase: ScanPhase,
/// The id of the artifact this scan consumes ([`Artifact::id`]).
pub artifact_id: String,
}
/// The ordered set of scans to run for a target.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct ScanPlan {
/// The scans, in matrix order.
pub steps: Vec<ScanStep>,
}
impl ScanPlan {
/// Whether the plan contains a step for the given scan type.
pub fn has(&self, scan: ScanType) -> bool {
self.steps.iter().any(|s| s.scan_type == scan)
}
/// Whether the plan is empty (nothing to run).
pub fn is_empty(&self) -> bool {
self.steps.is_empty()
}
}
/// Build the scan plan for a target: matrix defaults ∩ `scan_config`, each scan
/// bound to the artifact it consumes. Scans whose required artifact is absent, or
/// that are disabled, or off-by-default and not explicitly enabled, are dropped.
pub fn build_scan_plan(target: &OnboardedTarget) -> ScanPlan {
let enabled = &target.scan_config.enabled_scans;
let disabled = &target.scan_config.disabled_scans;
let mut steps = Vec::new();
for option in applicable_scans(target) {
// Required artifact missing → not runnable.
if option.blocked_reason.is_some() {
continue;
}
// Explicit opt-out wins.
if disabled.contains(&option.scan) {
continue;
}
// Run if on by default, or explicitly enabled.
if !option.default_on && !enabled.contains(&option.scan) {
continue;
}
let Some(artifact) = resolve_artifact(target, option.required_artifact) else {
continue;
};
steps.push(ScanStep {
scan_type: option.scan,
phase: phase_for(option.scan),
artifact_id: artifact.id.clone(),
});
}
ScanPlan { steps }
}
/// Resolve the artifact a scan consumes. A "code" requirement (represented by
/// `GitRepo`) is satisfied by a git repo *or* a source archive. The PLC
/// control-logic requirement (represented by `PlcProject`) prefers an uploaded
/// PLC project but also accepts a code artifact — a git repo / source archive
/// holding PLCopen XML / ST exports.
fn resolve_artifact(target: &OnboardedTarget, required: Option<ArtifactKind>) -> Option<&Artifact> {
match required {
Some(ArtifactKind::GitRepo) => target.code_artifact(),
Some(ArtifactKind::PlcProject) => target
.first_of(ArtifactKind::PlcProject)
.or_else(|| target.code_artifact()),
Some(kind) => target.first_of(kind),
None => target.code_artifact().or_else(|| target.artifacts.first()),
}
}
/// The pipeline phase reported while a given scan runs.
fn phase_for(scan: ScanType) -> ScanPhase {
match scan {
ScanType::Sast => ScanPhase::Sast,
ScanType::Sbom => ScanPhase::SbomGeneration,
ScanType::Cve => ScanPhase::CveScanning,
ScanType::Gdpr | ScanType::OAuth => ScanPhase::PatternScanning,
ScanType::SecretDetection => ScanPhase::SecretDetection,
ScanType::Lint => ScanPhase::LintScanning,
ScanType::CodeReview => ScanPhase::CodeReview,
ScanType::Graph => ScanPhase::GraphBuilding,
ScanType::Dast => ScanPhase::DastScanning,
ScanType::FirmwareStatic => ScanPhase::FirmwareStatic,
ScanType::PlcControlLogic => ScanPhase::PlcAnalysis,
ScanType::MobileStatic => ScanPhase::MobileStatic,
ScanType::ContainerScan => ScanPhase::ContainerScan,
ScanType::IcsProbe => ScanPhase::IcsProbe,
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{PlcFormat, TargetType};
fn target(target_type: TargetType, artifacts: Vec<Artifact>) -> OnboardedTarget {
let mut t = OnboardedTarget::new("t".to_string(), target_type);
t.artifacts = artifacts;
t
}
fn step_for<'a>(plan: &'a ScanPlan, scan: ScanType) -> Option<&'a ScanStep> {
plan.steps.iter().find(|s| s.scan_type == scan)
}
#[test]
fn webapp_with_code_and_url_runs_sast_and_dast_bound_to_the_right_artifacts() {
let code = Artifact::git_repo("https://git/x", "main");
let url = Artifact::live_url("https://x");
let (code_id, url_id) = (code.id.clone(), url.id.clone());
let t = target(TargetType::WebApp, vec![code, url]);
let plan = build_scan_plan(&t);
let sast = step_for(&plan, ScanType::Sast).expect("sast planned");
assert_eq!(sast.artifact_id, code_id);
assert_eq!(sast.phase, ScanPhase::Sast);
let dast = step_for(&plan, ScanType::Dast).expect("dast planned");
assert_eq!(dast.artifact_id, url_id);
}
#[test]
fn webapp_without_url_omits_dast() {
let t = target(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
let plan = build_scan_plan(&t);
assert!(plan.has(ScanType::Sast));
assert!(!plan.has(ScanType::Dast));
}
#[test]
fn code_scan_binds_to_source_archive_when_no_git_repo() {
let arc = Artifact::source_archive("src.zip");
let arc_id = arc.id.clone();
let t = target(TargetType::BackendService, vec![arc]);
let plan = build_scan_plan(&t);
let sast = step_for(&plan, ScanType::Sast).expect("sast planned");
assert_eq!(sast.artifact_id, arc_id);
}
#[test]
fn firmware_sbom_and_cve_bind_to_the_firmware_image() {
let fw = Artifact::firmware_image("fw.bin");
let fw_id = fw.id.clone();
let t = target(TargetType::FirmwareBareMetal, vec![fw]);
let plan = build_scan_plan(&t);
let sbom = step_for(&plan, ScanType::Sbom).expect("sbom planned");
assert_eq!(sbom.artifact_id, fw_id);
assert!(step_for(&plan, ScanType::FirmwareStatic).is_some());
assert!(!plan.has(ScanType::Dast));
}
#[test]
fn plc_plans_only_control_logic() {
let plc = Artifact::plc_project("p.xml", PlcFormat::PlcopenXml);
let t = target(TargetType::PlcSps, vec![plc]);
let plan = build_scan_plan(&t);
assert_eq!(plan.steps.len(), 1);
assert_eq!(plan.steps[0].scan_type, ScanType::PlcControlLogic);
assert_eq!(plan.steps[0].phase, ScanPhase::PlcAnalysis);
}
#[test]
fn plc_control_logic_binds_to_a_git_repo() {
// A CODESYS project in git (PLCopen XML / ST exports) with no uploaded
// PlcProject: control-logic still plans, bound to the git artifact.
let git = Artifact::git_repo("https://git/plc", "main");
let git_id = git.id.clone();
let t = target(TargetType::PlcSps, vec![git]);
let plan = build_scan_plan(&t);
let step = step_for(&plan, ScanType::PlcControlLogic).expect("control-logic planned");
assert_eq!(step.artifact_id, git_id, "PLC scan binds to the git repo");
}
#[test]
fn disabled_scan_is_dropped_and_off_by_default_can_be_enabled() {
let mut t = target(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
t.scan_config.disabled_scans = vec![ScanType::Lint];
// CodeReview is off by default for web; enable it explicitly.
t.scan_config.enabled_scans = vec![ScanType::CodeReview];
let plan = build_scan_plan(&t);
assert!(!plan.has(ScanType::Lint));
assert!(plan.has(ScanType::CodeReview));
assert!(plan.has(ScanType::Sast));
}
#[test]
fn no_code_artifact_yields_empty_plan_for_web() {
let t = target(TargetType::WebApp, vec![]);
let plan = build_scan_plan(&t);
assert!(plan.is_empty());
}
}
+226
View File
@@ -0,0 +1,226 @@
//! Abstract syntax tree for IEC 61131-3 Structured Text (ST).
//!
//! This is the security-relevant subset: POUs with their variable declarations
//! and statement bodies, enough to run semantic control-logic rules over. It is
//! deliberately not a full language model — declarations we don't reason about
//! (e.g. exotic type definitions) are parsed loosely and kept as raw text.
/// A Program Organization Unit: a PROGRAM, FUNCTION, or FUNCTION_BLOCK.
#[derive(Debug, Clone)]
pub struct Pou {
pub name: String,
pub kind: PouKind,
/// The declared variables, across all VAR_* sections.
pub vars: Vec<VarDecl>,
/// The statement body.
pub body: Vec<Stmt>,
/// 1-based line where the POU header appears (in the source that was parsed).
pub line: u32,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PouKind {
Program,
Function,
FunctionBlock,
}
impl PouKind {
pub fn label(self) -> &'static str {
match self {
PouKind::Program => "PROGRAM",
PouKind::Function => "FUNCTION",
PouKind::FunctionBlock => "FUNCTION_BLOCK",
}
}
}
/// A single declared variable.
#[derive(Debug, Clone)]
pub struct VarDecl {
pub name: String,
pub section: VarSection,
/// The declared type as written (e.g. `BOOL`, `INT`, `ARRAY[0..9] OF INT`).
pub type_name: String,
/// Whether the type is an ARRAY, and its declared bounds `(lo, hi)` when
/// they are literal integers — used by the array-bounds rule.
pub array_bounds: Option<(i64, i64)>,
/// The initializer expression, if any (`:= <expr>`).
pub init: Option<Expr>,
pub line: u32,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum VarSection {
Var,
Input,
Output,
InOut,
Global,
Temp,
External,
}
/// A statement.
#[derive(Debug, Clone)]
pub enum Stmt {
Assign {
target: Expr,
value: Expr,
line: u32,
},
If {
/// (condition, body) for IF and each ELSIF, in order.
branches: Vec<(Expr, Vec<Stmt>)>,
else_body: Option<Vec<Stmt>>,
line: u32,
},
Case {
selector: Expr,
/// (label expressions, body) per CASE arm.
arms: Vec<(Vec<Expr>, Vec<Stmt>)>,
else_body: Option<Vec<Stmt>>,
line: u32,
},
For {
var: String,
from: Expr,
to: Expr,
by: Option<Expr>,
body: Vec<Stmt>,
line: u32,
},
While {
cond: Expr,
body: Vec<Stmt>,
line: u32,
},
Repeat {
body: Vec<Stmt>,
until: Expr,
line: u32,
},
/// A bare call statement, e.g. `TON1(IN := x, PT := T#5s);`.
Call {
callee: String,
args: Vec<CallArg>,
line: u32,
},
Return {
line: u32,
},
Exit {
line: u32,
},
/// `JMP label;` — an unstructured jump.
Jump {
label: String,
line: u32,
},
/// `label:` — a jump target.
Label {
name: String,
line: u32,
},
}
/// One argument in a call: positional (`name: None`) or named (`X := expr`).
#[derive(Debug, Clone)]
pub struct CallArg {
pub name: Option<String>,
pub value: Expr,
}
/// An expression.
#[derive(Debug, Clone)]
pub enum Expr {
Int(i64, u32),
Real(f64, u32),
Bool(bool, u32),
/// A string literal, with the unquoted contents.
Str(String, u32),
/// A duration / date / time literal, kept as raw text (`T#5s`, `DT#...`).
Time(String, u32),
Ident(String, u32),
/// `base[index]`.
Index {
base: Box<Expr>,
index: Box<Expr>,
line: u32,
},
/// `base.field`.
Member {
base: Box<Expr>,
field: String,
line: u32,
},
Unary {
op: UnOp,
expr: Box<Expr>,
line: u32,
},
Binary {
op: BinOp,
lhs: Box<Expr>,
rhs: Box<Expr>,
line: u32,
},
/// A function call used as an expression, e.g. `LIMIT(a, b, c)`.
Call {
callee: String,
args: Vec<CallArg>,
line: u32,
},
}
impl Expr {
/// The 1-based source line this expression starts on.
pub fn line(&self) -> u32 {
match self {
Expr::Int(_, l)
| Expr::Real(_, l)
| Expr::Bool(_, l)
| Expr::Str(_, l)
| Expr::Time(_, l)
| Expr::Ident(_, l)
| Expr::Index { line: l, .. }
| Expr::Member { line: l, .. }
| Expr::Unary { line: l, .. }
| Expr::Binary { line: l, .. }
| Expr::Call { line: l, .. } => *l,
}
}
/// If this expression is a plain identifier, its name.
pub fn as_ident(&self) -> Option<&str> {
match self {
Expr::Ident(name, _) => Some(name.as_str()),
_ => None,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UnOp {
Not,
Neg,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BinOp {
Add,
Sub,
Mul,
Div,
Mod,
Pow,
Eq,
Ne,
Lt,
Le,
Gt,
Ge,
And,
Or,
Xor,
}
+372
View File
@@ -0,0 +1,372 @@
//! Lexer for IEC 61131-3 Structured Text.
//!
//! Tokenizes ST source into a flat token stream with 1-based line numbers.
//! Keywords are case-insensitive. Handles `(* *)` and `//` comments, `'..'` and
//! `".."` strings (with `''`/`""` escapes), based integers (`16#FF`, `2#1010`),
//! and duration/date literals (`T#5s`, `DT#...`) kept as raw text.
/// A lexed token with its source line.
#[derive(Debug, Clone)]
pub struct Token {
pub kind: Tok,
pub line: u32,
}
#[derive(Debug, Clone, PartialEq)]
pub enum Tok {
Int(i64),
Real(f64),
Str(String),
Time(String),
Bool(bool),
Ident(String),
Kw(Keyword),
Assign, // :=
Plus, // +
Minus, // -
Star, // *
Slash, // /
Power, // **
LParen, // (
RParen, // )
LBrack, // [
RBrack, // ]
Dot, // .
DotDot, // ..
Comma, // ,
Semi, // ;
Colon, // :
Lt, // <
Le, // <=
Gt, // >
Ge, // >=
Eq, // =
Ne, // <>
Amp, // &
Eof,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Keyword {
Program,
EndProgram,
Function,
EndFunction,
FunctionBlock,
EndFunctionBlock,
Var,
VarInput,
VarOutput,
VarInOut,
VarGlobal,
VarTemp,
VarExternal,
Constant,
EndVar,
Array,
Of,
If,
Then,
Elsif,
Else,
EndIf,
Case,
EndCase,
For,
To,
By,
Do,
EndFor,
While,
EndWhile,
Repeat,
Until,
EndRepeat,
Return,
Exit,
Jmp,
Not,
And,
Or,
Xor,
Mod,
Type,
EndType,
Struct,
EndStruct,
}
fn keyword_from(word: &str) -> Option<Keyword> {
use Keyword::*;
Some(match word.to_ascii_uppercase().as_str() {
"PROGRAM" => Program,
"END_PROGRAM" => EndProgram,
"FUNCTION" => Function,
"END_FUNCTION" => EndFunction,
"FUNCTION_BLOCK" => FunctionBlock,
"END_FUNCTION_BLOCK" => EndFunctionBlock,
"VAR" => Var,
"VAR_INPUT" => VarInput,
"VAR_OUTPUT" => VarOutput,
"VAR_IN_OUT" => VarInOut,
"VAR_GLOBAL" => VarGlobal,
"VAR_TEMP" => VarTemp,
"VAR_EXTERNAL" => VarExternal,
"CONSTANT" => Constant,
"END_VAR" => EndVar,
"ARRAY" => Array,
"OF" => Of,
"IF" => If,
"THEN" => Then,
"ELSIF" => Elsif,
"ELSE" => Else,
"END_IF" => EndIf,
"CASE" => Case,
"END_CASE" => EndCase,
"FOR" => For,
"TO" => To,
"BY" => By,
"DO" => Do,
"END_FOR" => EndFor,
"WHILE" => While,
"END_WHILE" => EndWhile,
"REPEAT" => Repeat,
"UNTIL" => Until,
"END_REPEAT" => EndRepeat,
"RETURN" => Return,
"EXIT" => Exit,
"JMP" => Jmp,
"NOT" => Not,
"AND" => And,
"OR" => Or,
"XOR" => Xor,
"MOD" => Mod,
"TYPE" => Type,
"END_TYPE" => EndType,
"STRUCT" => Struct,
"END_STRUCT" => EndStruct,
_ => return None,
})
}
/// Tokenize `src`. Unknown characters are skipped (best-effort — a scanner must
/// not die on odd input).
pub fn lex(src: &str) -> Vec<Token> {
let chars: Vec<char> = src.chars().collect();
let mut i = 0usize;
let mut line = 1u32;
let mut out = Vec::new();
let bump_line = |c: char, line: &mut u32| {
if c == '\n' {
*line += 1;
}
};
while i < chars.len() {
let c = chars[i];
// Whitespace.
if c.is_whitespace() {
bump_line(c, &mut line);
i += 1;
continue;
}
// Line comment: //
if c == '/' && i + 1 < chars.len() && chars[i + 1] == '/' {
while i < chars.len() && chars[i] != '\n' {
i += 1;
}
continue;
}
// Block comment: (* ... *)
if c == '(' && i + 1 < chars.len() && chars[i + 1] == '*' {
i += 2;
while i + 1 < chars.len() && !(chars[i] == '*' && chars[i + 1] == ')') {
bump_line(chars[i], &mut line);
i += 1;
}
i = (i + 2).min(chars.len());
continue;
}
let tok_line = line;
// String literal: '...' or "..."
if c == '\'' || c == '"' {
let quote = c;
i += 1;
let mut s = String::new();
while i < chars.len() {
let ch = chars[i];
if ch == quote {
// Doubled quote is an escaped quote.
if i + 1 < chars.len() && chars[i + 1] == quote {
s.push(quote);
i += 2;
continue;
}
i += 1;
break;
}
bump_line(ch, &mut line);
s.push(ch);
i += 1;
}
out.push(Token {
kind: Tok::Str(s),
line: tok_line,
});
continue;
}
// Identifier / keyword / time literal / boolean.
if c.is_ascii_alphabetic() || c == '_' {
let start = i;
while i < chars.len() && (chars[i].is_ascii_alphanumeric() || chars[i] == '_') {
i += 1;
}
let word: String = chars[start..i].iter().collect();
// Duration/date/time literal prefix: T#, TIME#, DT#, D#, TOD#, LT# ...
if i < chars.len() && chars[i] == '#' {
let up = word.to_ascii_uppercase();
if matches!(
up.as_str(),
"T" | "TIME" | "DT" | "D" | "TOD" | "LT" | "DATE"
) {
let lit_start = start;
i += 1; // consume '#'
while i < chars.len()
&& (chars[i].is_ascii_alphanumeric()
|| chars[i] == '.'
|| chars[i] == '_'
|| chars[i] == ':')
{
i += 1;
}
let lit: String = chars[lit_start..i].iter().collect();
out.push(Token {
kind: Tok::Time(lit),
line: tok_line,
});
continue;
}
}
let kind = match word.to_ascii_uppercase().as_str() {
"TRUE" => Tok::Bool(true),
"FALSE" => Tok::Bool(false),
_ => match keyword_from(&word) {
Some(kw) => Tok::Kw(kw),
None => Tok::Ident(word),
},
};
out.push(Token {
kind,
line: tok_line,
});
continue;
}
// Number: decimal, real, or based (16#..., 2#...).
if c.is_ascii_digit() {
let start = i;
while i < chars.len() && (chars[i].is_ascii_digit() || chars[i] == '_') {
i += 1;
}
// Based literal: <base>#<digits>
if i < chars.len() && chars[i] == '#' {
let base_str: String = chars[start..i].iter().filter(|c| **c != '_').collect();
i += 1;
let dstart = i;
while i < chars.len() && (chars[i].is_ascii_alphanumeric() || chars[i] == '_') {
i += 1;
}
let digits: String = chars[dstart..i].iter().filter(|c| **c != '_').collect();
let radix = base_str.parse::<u32>().unwrap_or(10);
let val = i64::from_str_radix(&digits, radix.clamp(2, 36)).unwrap_or(0);
out.push(Token {
kind: Tok::Int(val),
line: tok_line,
});
continue;
}
// Real: has a '.' (not '..') or exponent.
let is_real =
i < chars.len() && chars[i] == '.' && !(i + 1 < chars.len() && chars[i + 1] == '.');
if is_real {
i += 1;
while i < chars.len() && (chars[i].is_ascii_digit() || chars[i] == '_') {
i += 1;
}
let raw: String = chars[start..i].iter().filter(|c| **c != '_').collect();
out.push(Token {
kind: Tok::Real(raw.parse().unwrap_or(0.0)),
line: tok_line,
});
continue;
}
let raw: String = chars[start..i].iter().filter(|c| **c != '_').collect();
out.push(Token {
kind: Tok::Int(raw.parse().unwrap_or(0)),
line: tok_line,
});
continue;
}
// Operators / punctuation (longest match first).
let two: String = chars[i..(i + 2).min(chars.len())].iter().collect();
let kind = match two.as_str() {
":=" => Some(Tok::Assign),
"<=" => Some(Tok::Le),
">=" => Some(Tok::Ge),
"<>" => Some(Tok::Ne),
".." => Some(Tok::DotDot),
"**" => Some(Tok::Power),
_ => None,
};
if let Some(k) = kind {
out.push(Token {
kind: k,
line: tok_line,
});
i += 2;
continue;
}
let one = match c {
'+' => Some(Tok::Plus),
'-' => Some(Tok::Minus),
'*' => Some(Tok::Star),
'/' => Some(Tok::Slash),
'(' => Some(Tok::LParen),
')' => Some(Tok::RParen),
'[' => Some(Tok::LBrack),
']' => Some(Tok::RBrack),
'.' => Some(Tok::Dot),
',' => Some(Tok::Comma),
';' => Some(Tok::Semi),
':' => Some(Tok::Colon),
'<' => Some(Tok::Lt),
'>' => Some(Tok::Gt),
'=' => Some(Tok::Eq),
'&' => Some(Tok::Amp),
_ => None,
};
if let Some(k) = one {
out.push(Token {
kind: k,
line: tok_line,
});
}
i += 1;
}
out.push(Token {
kind: Tok::Eof,
line,
});
out
}
+234
View File
@@ -0,0 +1,234 @@
//! PLC control-logic security scanner for IEC 61131-3 targets.
//!
//! Parses Structured Text (raw `.st`/`.scl`/`.exp` files and PLCopen-XML
//! projects) into an AST and runs semantic control-logic security rules over it.
//! Implements [`ScanType::PlcControlLogic`].
pub mod ast;
pub mod lexer;
pub mod parser;
pub mod plcopen;
pub mod rules;
pub mod sbom;
use std::path::Path;
use compliance_core::error::CoreError;
use compliance_core::models::{Finding, ScanType};
use compliance_core::traits::{ScanOutput, Scanner};
use crate::pipeline::dedup;
/// Scanner for `ScanType::PlcControlLogic`.
pub struct PlcControlLogicScanner;
impl Scanner for PlcControlLogicScanner {
fn name(&self) -> &str {
"plc-control-logic"
}
fn scan_type(&self) -> ScanType {
ScanType::PlcControlLogic
}
#[tracing::instrument(skip_all)]
async fn scan(&self, repo_path: &Path, repo_id: &str) -> Result<ScanOutput, CoreError> {
let findings = analyze_tree(repo_path, repo_id);
Ok(ScanOutput {
findings,
sbom_entries: Vec::new(),
})
}
}
/// Walk a PLC project tree and produce findings.
pub(crate) fn analyze_tree(root: &Path, repo_id: &str) -> Vec<Finding> {
let mut findings = Vec::new();
for entry in walkdir::WalkDir::new(root)
.into_iter()
.filter_map(|e| e.ok())
{
if !entry.file_type().is_file() {
continue;
}
let path = entry.path();
let ext = path
.extension()
.and_then(|e| e.to_str())
.unwrap_or("")
.to_ascii_lowercase();
let is_st = matches!(ext.as_str(), "st" | "iecst" | "scl" | "exp" | "il");
let is_xml = matches!(ext.as_str(), "xml" | "plcopen" | "project");
if !is_st && !is_xml {
continue;
}
let Ok(content) = std::fs::read_to_string(path) else {
continue;
};
let pous = if is_xml {
plcopen::parse_plcopen(&content)
} else {
parser::parse(&content)
};
if pous.is_empty() {
continue;
}
let rel = path
.strip_prefix(root)
.unwrap_or(path)
.to_string_lossy()
.to_string();
for pou in &pous {
for hit in rules::analyze(pou) {
let line_s = hit.line.to_string();
let fingerprint =
dedup::compute_fingerprint(&[repo_id, &rel, hit.rule_id, &pou.name, &line_s]);
let mut f = Finding::new(
repo_id.to_string(),
fingerprint,
"plc-control-logic".to_string(),
ScanType::PlcControlLogic,
hit.title,
hit.description,
hit.severity,
);
f.file_path = Some(rel.clone());
f.line_number = Some(hit.line);
f.rule_id = Some(hit.rule_id.to_string());
f.cwe = hit.cwe.map(String::from);
f.remediation = Some(hit.remediation.to_string());
findings.push(f);
}
}
}
findings
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashSet;
use std::path::PathBuf;
fn demo_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.parent()
.expect("workspace root")
.join("examples/plc-demo")
}
#[test]
fn scans_demo_project_end_to_end() {
let findings = analyze_tree(&demo_dir(), "demo-target");
assert!(!findings.is_empty(), "demo project should produce findings");
let rules: HashSet<&str> = findings
.iter()
.filter_map(|f| f.rule_id.as_deref())
.collect();
for r in [
"plc-hardcoded-credential",
"plc-default-password",
"plc-safety-bypass",
"plc-array-unchecked-index",
"plc-insecure-comm",
"plc-insecure-protocol-port",
"plc-unstructured-jump",
"plc-division-by-zero",
] {
assert!(rules.contains(r), "expected rule {r}; got {rules:?}");
}
// Every finding is well-formed for storage.
for f in &findings {
assert_eq!(f.repo_id, "demo-target");
assert!(f.file_path.is_some(), "finding needs a file");
assert!(f.line_number.is_some(), "finding needs a line");
}
// The guarded division (IF ScaleFactor <> 0.0) must not be double-counted:
// exactly one division-by-zero (the unguarded MeasuredFlow divide).
let div0 = findings
.iter()
.filter(|f| f.rule_id.as_deref() == Some("plc-division-by-zero"))
.count();
assert_eq!(div0, 1, "only the unguarded division should be flagged");
}
/// The realistic OpenPLC-style traffic-light sample is mostly sound control
/// logic: the scanner must surface its few genuine defects and stay quiet on
/// the timed state machine and the guarded duty-cycle division.
#[test]
fn realistic_sample_flags_only_real_issues() {
let all = analyze_tree(&demo_dir(), "demo-target");
let tl: Vec<_> = all
.iter()
.filter(|f| {
f.file_path
.as_deref()
.is_some_and(|p| p.ends_with("traffic_light.st"))
})
.collect();
assert!(!tl.is_empty(), "traffic_light.st should produce findings");
let rules: HashSet<&str> = tl.iter().filter_map(|f| f.rule_id.as_deref()).collect();
// The three planted defects: hardcoded SCADA password, cleartext Modbus
// master (no auth), and a maintenance mode that drops the PedPermit.
for r in [
"plc-hardcoded-credential",
"plc-insecure-comm",
"plc-safety-bypass",
] {
assert!(rules.contains(r), "expected rule {r}; got {rules:?}");
}
// Modbus/TCP on 502 is also an insecure-protocol port.
assert!(rules.contains("plc-insecure-protocol-port"));
// Low false positives: the guarded `IF LampCount <> 0` division and the
// JMP-free state machine must not trip anything.
assert_eq!(
tl.iter()
.filter(|f| f.rule_id.as_deref() == Some("plc-division-by-zero"))
.count(),
0,
"the guarded duty-cycle division must not be flagged"
);
assert!(
!rules.contains("plc-unstructured-jump"),
"the CASE state machine uses no JMP"
);
}
/// Graphical logic must be analysed too: an FBD POU (blocks + in/out
/// variables) is translated to synthetic ST, so the same rules fire on the
/// cleartext Modbus block, the hardcoded HMI password and the safety write.
#[test]
fn fbd_graphical_body_is_analysed() {
let all = analyze_tree(&demo_dir(), "demo-target");
let fbd: Vec<_> = all
.iter()
.filter(|f| {
f.file_path
.as_deref()
.is_some_and(|p| p.ends_with("pump_fbd.xml"))
})
.collect();
assert!(
!fbd.is_empty(),
"pump_fbd.xml (FBD) should produce findings"
);
let rules: HashSet<&str> = fbd.iter().filter_map(|f| f.rule_id.as_deref()).collect();
for r in [
"plc-insecure-comm", // Modbus_TCP_Master(AUTH := FALSE)
"plc-insecure-protocol-port", // PORT := 502
"plc-hardcoded-credential", // HmiPassword := 'admin123'
"plc-safety-bypass", // Safety_Enable := FALSE
] {
assert!(
rules.contains(r),
"expected rule {r} from FBD; got {rules:?}"
);
}
}
}
+766
View File
@@ -0,0 +1,766 @@
//! Recursive-descent parser for the security-relevant subset of Structured Text.
//!
//! Tolerant by design: it parses the POUs, variable sections, and statement
//! bodies it understands, and skips (with statement/POU-level recovery) anything
//! it does not, so a single odd construct never sinks the whole file.
use super::ast::*;
use super::lexer::{Keyword as K, Tok, Token};
pub struct Parser {
toks: Vec<Token>,
pos: usize,
}
impl Parser {
pub fn new(toks: Vec<Token>) -> Self {
Self { toks, pos: 0 }
}
// ── token helpers ──────────────────────────────────────────────
fn peek(&self) -> &Tok {
&self.toks[self.pos.min(self.toks.len() - 1)].kind
}
fn line(&self) -> u32 {
self.toks[self.pos.min(self.toks.len() - 1)].line
}
fn at_end(&self) -> bool {
matches!(self.peek(), Tok::Eof)
}
fn advance(&mut self) -> Tok {
let t = self.toks[self.pos.min(self.toks.len() - 1)].kind.clone();
if self.pos < self.toks.len() - 1 {
self.pos += 1;
}
t
}
fn eat(&mut self, t: &Tok) -> bool {
if self.peek() == t {
self.advance();
true
} else {
false
}
}
fn eat_kw(&mut self, k: K) -> bool {
if matches!(self.peek(), Tok::Kw(x) if *x == k) {
self.advance();
true
} else {
false
}
}
fn at_kw(&self, k: K) -> bool {
matches!(self.peek(), Tok::Kw(x) if *x == k)
}
fn ident(&mut self) -> Option<String> {
if let Tok::Ident(s) = self.peek() {
let s = s.clone();
self.advance();
Some(s)
} else {
None
}
}
// ── top level ──────────────────────────────────────────────────
/// Parse every POU in the token stream.
pub fn parse_units(&mut self) -> Vec<Pou> {
let mut pous = Vec::new();
while !self.at_end() {
match self.peek() {
Tok::Kw(K::Program) => {
self.advance();
if let Some(p) = self.parse_pou(PouKind::Program, K::EndProgram) {
pous.push(p);
}
}
Tok::Kw(K::Function) => {
self.advance();
if let Some(p) = self.parse_pou(PouKind::Function, K::EndFunction) {
pous.push(p);
}
}
Tok::Kw(K::FunctionBlock) => {
self.advance();
if let Some(p) = self.parse_pou(PouKind::FunctionBlock, K::EndFunctionBlock) {
pous.push(p);
}
}
// Skip TYPE...END_TYPE and anything else at top level.
_ => {
self.advance();
}
}
}
pous
}
fn parse_pou(&mut self, kind: PouKind, end: K) -> Option<Pou> {
let line = self.line();
let name = self.ident().unwrap_or_else(|| "<anonymous>".to_string());
// Optional `: return_type` for functions.
if self.eat(&Tok::Colon) {
let _ = self.advance(); // return type token
}
let mut vars = Vec::new();
// Variable sections precede the body.
while let Some(section) = self.var_section_kw() {
self.advance();
let _ = self.eat_kw(K::Constant); // CONSTANT is informational for our rules
self.parse_var_decls(section, &mut vars);
}
// Body statements until END_<kind>.
let mut body = Vec::new();
while !self.at_end() && !self.at_kw(end) {
if let Some(s) = self.parse_stmt() {
body.push(s);
}
}
self.eat_kw(end);
Some(Pou {
name,
kind,
vars,
body,
line,
})
}
fn var_section_kw(&self) -> Option<VarSection> {
match self.peek() {
Tok::Kw(K::Var) => Some(VarSection::Var),
Tok::Kw(K::VarInput) => Some(VarSection::Input),
Tok::Kw(K::VarOutput) => Some(VarSection::Output),
Tok::Kw(K::VarInOut) => Some(VarSection::InOut),
Tok::Kw(K::VarGlobal) => Some(VarSection::Global),
Tok::Kw(K::VarTemp) => Some(VarSection::Temp),
Tok::Kw(K::VarExternal) => Some(VarSection::External),
_ => None,
}
}
fn parse_var_decls(&mut self, section: VarSection, out: &mut Vec<VarDecl>) {
while !self.at_end() && !self.at_kw(K::EndVar) {
let line = self.line();
// names: a, b, c
let mut names = Vec::new();
match self.ident() {
Some(n) => names.push(n),
None => {
// Not a declaration we understand — skip to next ; or END_VAR.
self.sync_decl();
continue;
}
}
while self.eat(&Tok::Comma) {
if let Some(n) = self.ident() {
names.push(n);
}
}
if !self.eat(&Tok::Colon) {
self.sync_decl();
continue;
}
let (type_name, array_bounds) = self.parse_type();
let init = if self.eat(&Tok::Assign) {
Some(self.parse_expr())
} else {
None
};
self.eat(&Tok::Semi);
for n in names {
out.push(VarDecl {
name: n,
section,
type_name: type_name.clone(),
array_bounds,
init: init.clone(),
line,
});
}
}
self.eat_kw(K::EndVar);
}
/// Parse a (possibly ARRAY) type, returning its rendered name and literal
/// bounds when present.
fn parse_type(&mut self) -> (String, Option<(i64, i64)>) {
if self.eat_kw(K::Array) {
let mut bounds = None;
if self.eat(&Tok::LBrack) {
let lo = self.int_lit();
self.eat(&Tok::DotDot);
let hi = self.int_lit();
if let (Some(lo), Some(hi)) = (lo, hi) {
bounds = Some((lo, hi));
}
// Skip any further dimensions / tokens to the closing bracket.
while !self.at_end() && !self.eat(&Tok::RBrack) {
self.advance();
}
}
self.eat_kw(K::Of);
let elem = self.type_ident();
(format!("ARRAY OF {elem}"), bounds)
} else {
(self.type_ident(), None)
}
}
fn type_ident(&mut self) -> String {
// Types can be qualified idents; keep it simple: one token, plus any
// string-length suffix like STRING[80].
let base = match self.advance() {
Tok::Ident(s) => s,
Tok::Kw(_) => "TYPE".to_string(),
other => format!("{other:?}"),
};
if self.eat(&Tok::LBrack) {
while !self.at_end() && !self.eat(&Tok::RBrack) {
self.advance();
}
}
base
}
fn int_lit(&mut self) -> Option<i64> {
match self.peek() {
Tok::Int(n) => {
let n = *n;
self.advance();
Some(n)
}
Tok::Minus => {
self.advance();
if let Tok::Int(n) = self.peek() {
let n = -*n;
self.advance();
Some(n)
} else {
None
}
}
_ => None,
}
}
fn sync_decl(&mut self) {
while !self.at_end() && !self.eat(&Tok::Semi) && !self.at_kw(K::EndVar) {
self.advance();
}
}
fn sync_stmt(&mut self) {
while !self.at_end() && !self.eat(&Tok::Semi) {
// Stop at block terminators so recovery doesn't swallow structure.
if matches!(
self.peek(),
Tok::Kw(
K::EndIf
| K::EndFor
| K::EndWhile
| K::EndCase
| K::EndRepeat
| K::EndProgram
| K::EndFunction
| K::EndFunctionBlock
| K::Else
| K::Elsif
)
) {
return;
}
self.advance();
}
}
// ── statements ─────────────────────────────────────────────────
fn parse_stmt(&mut self) -> Option<Stmt> {
let line = self.line();
match self.peek().clone() {
Tok::Semi => {
self.advance();
None
}
Tok::Kw(K::If) => self.parse_if(),
Tok::Kw(K::Case) => self.parse_case(),
Tok::Kw(K::For) => self.parse_for(),
Tok::Kw(K::While) => self.parse_while(),
Tok::Kw(K::Repeat) => self.parse_repeat(),
Tok::Kw(K::Return) => {
self.advance();
self.eat(&Tok::Semi);
Some(Stmt::Return { line })
}
Tok::Kw(K::Exit) => {
self.advance();
self.eat(&Tok::Semi);
Some(Stmt::Exit { line })
}
Tok::Kw(K::Jmp) => {
self.advance();
let label = self.ident().unwrap_or_default();
self.eat(&Tok::Semi);
Some(Stmt::Jump { label, line })
}
Tok::Ident(name) => {
// Could be `label:`, `call(...)`, or an assignment.
// Lookahead: ident ':' (not ':=') → label.
if matches!(
self.toks.get(self.pos + 1).map(|t| &t.kind),
Some(Tok::Colon)
) && !matches!(self.toks.get(self.pos + 2).map(|t| &t.kind), Some(Tok::Eq))
{
self.advance(); // ident
self.advance(); // ':'
return Some(Stmt::Label { name, line });
}
let lhs = self.parse_expr();
if self.eat(&Tok::Assign) {
let value = self.parse_expr();
self.eat(&Tok::Semi);
Some(Stmt::Assign {
target: lhs,
value,
line,
})
} else if let Expr::Call { callee, args, .. } = lhs {
self.eat(&Tok::Semi);
Some(Stmt::Call { callee, args, line })
} else {
// Bare expression / FB invocation without args recognized —
// skip to the terminator.
self.sync_stmt();
None
}
}
_ => {
self.sync_stmt();
None
}
}
}
fn parse_block_until(&mut self, terms: &[K]) -> Vec<Stmt> {
let mut body = Vec::new();
while !self.at_end() && !terms.iter().any(|k| self.at_kw(*k)) {
if let Some(s) = self.parse_stmt() {
body.push(s);
}
}
body
}
fn parse_if(&mut self) -> Option<Stmt> {
let line = self.line();
self.eat_kw(K::If);
let mut branches = Vec::new();
let cond = self.parse_expr();
self.eat_kw(K::Then);
let body = self.parse_block_until(&[K::Elsif, K::Else, K::EndIf]);
branches.push((cond, body));
while self.eat_kw(K::Elsif) {
let c = self.parse_expr();
self.eat_kw(K::Then);
let b = self.parse_block_until(&[K::Elsif, K::Else, K::EndIf]);
branches.push((c, b));
}
let else_body = if self.eat_kw(K::Else) {
Some(self.parse_block_until(&[K::EndIf]))
} else {
None
};
self.eat_kw(K::EndIf);
self.eat(&Tok::Semi);
Some(Stmt::If {
branches,
else_body,
line,
})
}
fn parse_case(&mut self) -> Option<Stmt> {
let line = self.line();
self.eat_kw(K::Case);
let selector = self.parse_expr();
self.eat_kw(K::Of);
let mut arms = Vec::new();
let mut else_body = None;
while !self.at_end() && !self.at_kw(K::EndCase) {
if self.eat_kw(K::Else) {
else_body = Some(self.parse_block_until(&[K::EndCase]));
break;
}
// labels: expr {, expr} :
let mut labels = vec![self.parse_expr()];
while self.eat(&Tok::Comma) {
labels.push(self.parse_expr());
}
self.eat(&Tok::Colon);
let body = self.parse_block_until(&[K::EndCase, K::Else]);
arms.push((labels, body));
}
self.eat_kw(K::EndCase);
self.eat(&Tok::Semi);
Some(Stmt::Case {
selector,
arms,
else_body,
line,
})
}
fn parse_for(&mut self) -> Option<Stmt> {
let line = self.line();
self.eat_kw(K::For);
let var = self.ident().unwrap_or_default();
self.eat(&Tok::Assign);
let from = self.parse_expr();
self.eat_kw(K::To);
let to = self.parse_expr();
let by = if self.eat_kw(K::By) {
Some(self.parse_expr())
} else {
None
};
self.eat_kw(K::Do);
let body = self.parse_block_until(&[K::EndFor]);
self.eat_kw(K::EndFor);
self.eat(&Tok::Semi);
Some(Stmt::For {
var,
from,
to,
by,
body,
line,
})
}
fn parse_while(&mut self) -> Option<Stmt> {
let line = self.line();
self.eat_kw(K::While);
let cond = self.parse_expr();
self.eat_kw(K::Do);
let body = self.parse_block_until(&[K::EndWhile]);
self.eat_kw(K::EndWhile);
self.eat(&Tok::Semi);
Some(Stmt::While { cond, body, line })
}
fn parse_repeat(&mut self) -> Option<Stmt> {
let line = self.line();
self.eat_kw(K::Repeat);
let body = self.parse_block_until(&[K::Until, K::EndRepeat]);
self.eat_kw(K::Until);
let until = self.parse_expr();
self.eat_kw(K::EndRepeat);
self.eat(&Tok::Semi);
Some(Stmt::Repeat { body, until, line })
}
// ── expressions (precedence climbing) ──────────────────────────
pub fn parse_expr(&mut self) -> Expr {
self.parse_or()
}
fn parse_or(&mut self) -> Expr {
let mut lhs = self.parse_and();
loop {
let op = match self.peek() {
Tok::Kw(K::Or) => BinOp::Or,
Tok::Kw(K::Xor) => BinOp::Xor,
_ => break,
};
let line = self.line();
self.advance();
let rhs = self.parse_and();
lhs = Expr::Binary {
op,
lhs: Box::new(lhs),
rhs: Box::new(rhs),
line,
};
}
lhs
}
fn parse_and(&mut self) -> Expr {
let mut lhs = self.parse_cmp();
while matches!(self.peek(), Tok::Kw(K::And) | Tok::Amp) {
let op = BinOp::And;
let line = self.line();
self.advance();
let rhs = self.parse_cmp();
lhs = Expr::Binary {
op,
lhs: Box::new(lhs),
rhs: Box::new(rhs),
line,
};
}
lhs
}
fn parse_cmp(&mut self) -> Expr {
let mut lhs = self.parse_add();
loop {
let op = match self.peek() {
Tok::Eq => BinOp::Eq,
Tok::Ne => BinOp::Ne,
Tok::Lt => BinOp::Lt,
Tok::Le => BinOp::Le,
Tok::Gt => BinOp::Gt,
Tok::Ge => BinOp::Ge,
_ => break,
};
let line = self.line();
self.advance();
let rhs = self.parse_add();
lhs = Expr::Binary {
op,
lhs: Box::new(lhs),
rhs: Box::new(rhs),
line,
};
}
lhs
}
fn parse_add(&mut self) -> Expr {
let mut lhs = self.parse_mul();
loop {
let op = match self.peek() {
Tok::Plus => BinOp::Add,
Tok::Minus => BinOp::Sub,
_ => break,
};
let line = self.line();
self.advance();
let rhs = self.parse_mul();
lhs = Expr::Binary {
op,
lhs: Box::new(lhs),
rhs: Box::new(rhs),
line,
};
}
lhs
}
fn parse_mul(&mut self) -> Expr {
let mut lhs = self.parse_unary();
loop {
let op = match self.peek() {
Tok::Star => BinOp::Mul,
Tok::Slash => BinOp::Div,
Tok::Kw(K::Mod) => BinOp::Mod,
Tok::Power => BinOp::Pow,
_ => break,
};
let line = self.line();
self.advance();
let rhs = self.parse_unary();
lhs = Expr::Binary {
op,
lhs: Box::new(lhs),
rhs: Box::new(rhs),
line,
};
}
lhs
}
fn parse_unary(&mut self) -> Expr {
let line = self.line();
match self.peek() {
Tok::Kw(K::Not) => {
self.advance();
Expr::Unary {
op: UnOp::Not,
expr: Box::new(self.parse_unary()),
line,
}
}
Tok::Minus => {
self.advance();
Expr::Unary {
op: UnOp::Neg,
expr: Box::new(self.parse_unary()),
line,
}
}
_ => self.parse_postfix(),
}
}
fn parse_postfix(&mut self) -> Expr {
let mut e = self.parse_primary();
loop {
let line = self.line();
match self.peek() {
Tok::LBrack => {
self.advance();
let index = self.parse_expr();
self.eat(&Tok::RBrack);
e = Expr::Index {
base: Box::new(e),
index: Box::new(index),
line,
};
}
Tok::Dot => {
self.advance();
let field = self.ident().unwrap_or_default();
e = Expr::Member {
base: Box::new(e),
field,
line,
};
}
_ => break,
}
}
e
}
fn parse_primary(&mut self) -> Expr {
let line = self.line();
match self.advance() {
Tok::Int(n) => Expr::Int(n, line),
Tok::Real(r) => Expr::Real(r, line),
Tok::Bool(b) => Expr::Bool(b, line),
Tok::Str(s) => Expr::Str(s, line),
Tok::Time(t) => Expr::Time(t, line),
Tok::LParen => {
let e = self.parse_expr();
self.eat(&Tok::RParen);
e
}
Tok::Ident(name) => {
if self.eat(&Tok::LParen) {
let args = self.parse_call_args();
Expr::Call {
callee: name,
args,
line,
}
} else {
Expr::Ident(name, line)
}
}
// Unrecognized start of expression — yield a placeholder identifier.
_ => Expr::Ident(String::new(), line),
}
}
fn parse_call_args(&mut self) -> Vec<CallArg> {
let mut args = Vec::new();
if self.eat(&Tok::RParen) {
return args;
}
loop {
// Named arg: ident := expr (peek two tokens).
if let Tok::Ident(name) = self.peek().clone() {
if matches!(
self.toks.get(self.pos + 1).map(|t| &t.kind),
Some(Tok::Assign)
) {
self.advance(); // ident
self.advance(); // :=
let value = self.parse_expr();
args.push(CallArg {
name: Some(name),
value,
});
if self.eat(&Tok::Comma) {
continue;
}
break;
}
}
let value = self.parse_expr();
args.push(CallArg { name: None, value });
if self.eat(&Tok::Comma) {
continue;
}
break;
}
self.eat(&Tok::RParen);
args
}
}
/// Parse ST source into its POUs.
pub fn parse(src: &str) -> Vec<Pou> {
let toks = super::lexer::lex(src);
Parser::new(toks).parse_units()
}
#[cfg(test)]
mod tests {
use super::*;
const SAMPLE: &str = r#"
PROGRAM Main
VAR
idx : INT;
pw : STRING := 'admin123';
buf : ARRAY[0..9] OF INT;
ok : BOOL := FALSE;
END_VAR
// a comment
IF idx > 0 THEN
buf[idx] := idx * 2;
ELSE
JMP done;
END_IF;
Comm(IP := '10.0.0.1', PORT := 502);
done:
ok := TRUE;
END_PROGRAM
"#;
#[test]
fn parses_program_vars_and_body() {
let pous = parse(SAMPLE);
assert_eq!(pous.len(), 1, "one POU");
let p = &pous[0];
assert_eq!(p.name, "Main");
assert_eq!(p.kind, PouKind::Program);
// vars: idx, pw, buf, ok
assert_eq!(p.vars.len(), 4);
let pw = p.vars.iter().find(|v| v.name == "pw").expect("pw");
assert!(matches!(&pw.init, Some(Expr::Str(s, _)) if s == "admin123"));
let buf = p.vars.iter().find(|v| v.name == "buf").expect("buf");
assert_eq!(buf.array_bounds, Some((0, 9)));
// body has an IF, a Call, a Label, and an Assign
assert!(p.body.iter().any(|s| matches!(s, Stmt::If { .. })));
assert!(p
.body
.iter()
.any(|s| matches!(s, Stmt::Call { callee, .. } if callee == "Comm")));
assert!(p
.body
.iter()
.any(|s| matches!(s, Stmt::Label { name, .. } if name == "done")));
}
#[test]
fn jmp_inside_if_is_captured() {
let pous = parse(SAMPLE);
let p = &pous[0];
// find the IF, check its else branch has a JMP
let has_jmp = p.body.iter().any(|s| match s {
Stmt::If { else_body, .. } => else_body
.as_ref()
.map(|b| b.iter().any(|s| matches!(s, Stmt::Jump { .. })))
.unwrap_or(false),
_ => false,
});
assert!(has_jmp, "JMP should be parsed inside the ELSE branch");
}
}
@@ -0,0 +1,418 @@
//! PLCopen XML → Structured Text POUs.
//!
//! A PLCopen project stores each POU as `<pou name=".." pouType="..">` with an
//! `<interface>` (typed variable sections) and a `<body>` in one of the IEC
//! 61131-3 languages. We reconstruct an equivalent Structured-Text source for
//! each POU (a `VAR` block from the interface + statements from the body) and run
//! it through the ST parser, so raw `.st` files and PLCopen projects — textual or
//! graphical — flow through one analysis path.
//!
//! Body languages:
//! - **ST** — taken verbatim.
//! - **FBD / LD** — the graphical network is translated to synthetic ST: blocks
//! become calls (`TypeName(pin := arg, …)`), out-variables / coils become
//! assignments, with input pins resolved by tracing connections. This lets the
//! semantic rules see comm calls, hardcoded arguments and safety writes that
//! live in graphical logic, not just in text.
//! - **SFC** — the step/transition graph itself is skipped; the ST/FBD/LD bodies
//! embedded in its actions and transitions are still translated.
use std::collections::HashMap;
use roxmltree::Node;
use super::ast::Pou;
use super::parser;
/// Parse every POU out of a PLCopen XML document (ST, FBD or LD bodies).
pub fn parse_plcopen(xml: &str) -> Vec<Pou> {
let doc = match roxmltree::Document::parse(xml) {
Ok(d) => d,
Err(_) => return Vec::new(),
};
let mut pous = Vec::new();
for pou in doc.descendants().filter(|n| n.has_tag_name("pou")) {
let name = pou.attribute("name").unwrap_or("pou").to_string();
let pou_type = pou.attribute("pouType").unwrap_or("program");
let Some(body) = reconstruct_body(pou) else {
continue;
};
if body.trim().is_empty() {
continue;
}
let var_block = build_var_block(pou);
let kw = match pou_type.to_ascii_lowercase().as_str() {
"function" => "FUNCTION",
"functionblock" | "functionblocktype" => "FUNCTION_BLOCK",
_ => "PROGRAM",
};
let synthetic = format!("{kw} {name}\n{var_block}{body}\nEND_{kw}\n");
pous.extend(parser::parse(&synthetic));
}
pous
}
/// Case-insensitive tag match (PLCopen uses `FBD`/`LD`/`ST`, CODESYS may vary).
fn tag_is(n: &Node, name: &str) -> bool {
n.tag_name().name().eq_ignore_ascii_case(name)
}
/// Reconstruct a POU's body as Structured Text, whatever language it is written
/// in. Concatenates every language body found under `<body>` (SFC actions and
/// transitions carry their own ST/FBD/LD sub-bodies).
fn reconstruct_body(pou: Node) -> Option<String> {
let mut out = String::new();
for body in pou.descendants().filter(|n| tag_is(n, "body")) {
for lang in body.children().filter(|n| n.is_element()) {
let piece = match lang.tag_name().name().to_ascii_uppercase().as_str() {
"ST" | "IL" => collect_text(lang),
"FBD" | "LD" => translate_network(lang),
_ => continue,
};
if !piece.trim().is_empty() {
out.push_str(&piece);
if !piece.ends_with('\n') {
out.push('\n');
}
}
}
}
if out.trim().is_empty() {
None
} else {
Some(out)
}
}
// ── graphical (FBD / LD) → synthetic ST ────────────────────────────────
/// Translate one FBD/LD network into ST statements: blocks → calls,
/// out-variables and coils → assignments.
fn translate_network(net: Node) -> String {
let by_id = index_local_ids(net);
let mut out = String::new();
for el in net.children().filter(|n| n.is_element()) {
let stmt = match el.tag_name().name().to_ascii_lowercase().as_str() {
"block" => block_call(el, &by_id).map(|c| format!("{c};")),
"outvariable" => out_assignment(el, &by_id),
"coil" => coil_assignment(el, &by_id),
_ => None,
};
if let Some(s) = stmt {
out.push_str(&s);
out.push('\n');
}
}
out
}
/// Index every element in a network by its `localId` so connections resolve.
fn index_local_ids<'a, 'input>(net: Node<'a, 'input>) -> HashMap<String, Node<'a, 'input>> {
net.descendants()
.filter(|n| n.is_element())
.filter_map(|n| n.attribute("localId").map(|id| (id.to_string(), n)))
.collect()
}
/// Build a call expression for a block: `TypeName(pin := arg, …)`.
fn block_call(block: Node, by_id: &HashMap<String, Node>) -> Option<String> {
let ty = block.attribute("typeName")?;
let mut args = Vec::new();
if let Some(inputs) = block.children().find(|n| tag_is(n, "inputVariables")) {
for v in inputs.children().filter(|n| tag_is(n, "variable")) {
let Some(expr) = input_expr(v, by_id, 0) else {
continue;
};
match v.attribute("formalParameter") {
Some(pin) if !pin.is_empty() => args.push(format!("{pin} := {expr}")),
_ => args.push(expr),
}
}
}
Some(format!("{ty}({})", args.join(", ")))
}
/// `target := <traced expression>;` for an FBD out-variable.
fn out_assignment(outvar: Node, by_id: &HashMap<String, Node>) -> Option<String> {
let target = expression_text(outvar)?;
let value = input_expr(outvar, by_id, 0).unwrap_or_else(|| "0".to_string());
Some(format!("{target} := {value};"))
}
/// `coil := <traced rung expression>;` for an LD coil (negated → `NOT (…)`).
fn coil_assignment(coil: Node, by_id: &HashMap<String, Node>) -> Option<String> {
let target = child_text(coil, "variable")?;
let rung = input_expr(coil, by_id, 0).unwrap_or_else(|| "TRUE".to_string());
let negated = matches!(coil.attribute("negated"), Some(v) if v.eq_ignore_ascii_case("true"));
let rhs = if negated {
format!("NOT ({rung})")
} else {
rung
};
Some(format!("{target} := {rhs};"))
}
/// Resolve the expression feeding `node`'s single input connection.
fn input_expr(node: Node, by_id: &HashMap<String, Node>, depth: u8) -> Option<String> {
let refid = ref_local_id(node)?;
Some(expr_for(&refid, by_id, depth))
}
/// Build the ST expression produced by the element with this `localId`.
fn expr_for(local_id: &str, by_id: &HashMap<String, Node>, depth: u8) -> String {
if depth > 24 {
return "0".to_string();
}
let Some(node) = by_id.get(local_id) else {
return format!("__net{local_id}");
};
match node.tag_name().name().to_ascii_lowercase().as_str() {
"invariable" | "inoutvariable" => {
expression_text(*node).unwrap_or_else(|| format!("__net{local_id}"))
}
// A block feeding another element: reference it by a synthetic result
// name; the block is emitted as its own call statement, so we neither
// duplicate the call nor lose it.
"block" => format!("__blk{local_id}"),
"contact" => {
let var = child_text(*node, "variable").unwrap_or_else(|| "TRUE".to_string());
let negated =
matches!(node.attribute("negated"), Some(v) if v.eq_ignore_ascii_case("true"));
let term = if negated { format!("NOT {var}") } else { var };
match ref_local_id(*node) {
Some(up) => {
let upstream = expr_for(&up, by_id, depth + 1);
if upstream == "TRUE" {
term
} else {
format!("({upstream} AND {term})")
}
}
None => term,
}
}
"leftpowerrail" => "TRUE".to_string(),
_ => format!("__net{local_id}"),
}
}
/// The `refLocalId` of `node`'s first input connection, if any.
fn ref_local_id(node: Node) -> Option<String> {
node.descendants()
.find(|n| tag_is(n, "connectionPointIn"))
.and_then(|cpi| cpi.descendants().find(|n| tag_is(n, "connection")))
.and_then(|c| c.attribute("refLocalId"))
.map(|s| s.to_string())
}
/// Text of a node's `<expression>` child (variable name or literal).
fn expression_text(node: Node) -> Option<String> {
let e = node.children().find(|n| tag_is(n, "expression"))?;
let t = collect_text(e).trim().to_string();
if t.is_empty() {
None
} else {
Some(t)
}
}
/// Text of a named child element (e.g. `<variable>` of a contact/coil).
fn child_text(node: Node, name: &str) -> Option<String> {
let c = node.children().find(|n| tag_is(n, name))?;
let t = collect_text(c).trim().to_string();
if t.is_empty() {
None
} else {
Some(t)
}
}
/// Concatenate the text of a node's descendant text nodes (bodies are often
/// wrapped in `<xhtml>` and may contain multiple text runs). Only text nodes are
/// gathered: an element's `.text()` would re-yield its first child's text, which
/// (with the text node itself) would duplicate every value.
fn collect_text(node: Node) -> String {
node.descendants()
.filter(|n| n.is_text())
.filter_map(|n| n.text())
.collect::<String>()
}
/// Build an ST `VAR … END_VAR` block from a POU's `<interface>` variable
/// sections, so declarations (types, initial values) reach the rules.
fn build_var_block(pou: Node) -> String {
let Some(interface) = pou.children().find(|n| n.has_tag_name("interface")) else {
return String::new();
};
let mut out = String::from("VAR\n");
let mut any = false;
for container in interface.children().filter(|n| n.is_element()) {
// localVars / inputVars / outputVars / inOutVars / tempVars / globalVars / externalVars
if !container.tag_name().name().ends_with("Vars") {
continue;
}
for var in container.children().filter(|n| n.has_tag_name("variable")) {
let Some(vname) = var.attribute("name") else {
continue;
};
let ty = var
.children()
.find(|n| n.has_tag_name("type"))
.map(type_name)
.unwrap_or_else(|| "BOOL".to_string());
let init = var
.children()
.find(|n| n.has_tag_name("initialValue"))
.and_then(initial_value);
match init {
Some(v) => out.push_str(&format!(" {vname} : {ty} := {v};\n")),
None => out.push_str(&format!(" {vname} : {ty};\n")),
}
any = true;
}
}
out.push_str("END_VAR\n");
if any {
out
} else {
String::new()
}
}
/// Render a PLCopen `<type>` element as an ST type string.
fn type_name(type_node: Node) -> String {
let Some(inner) = type_node.children().find(|n| n.is_element()) else {
return "BOOL".to_string();
};
let tag = inner.tag_name().name();
match tag {
"derived" => inner.attribute("name").unwrap_or("DERIVED").to_string(),
"array" => {
let dim = inner.children().find(|n| n.has_tag_name("dimension"));
let (lo, hi) = dim
.map(|d| {
(
d.attribute("lower").unwrap_or("0").to_string(),
d.attribute("upper").unwrap_or("0").to_string(),
)
})
.unwrap_or_else(|| ("0".to_string(), "0".to_string()));
let base = inner
.children()
.find(|n| n.has_tag_name("baseType"))
.map(type_name)
.unwrap_or_else(|| "INT".to_string());
format!("ARRAY[{lo}..{hi}] OF {base}")
}
"string" | "wstring" => "STRING".to_string(),
// BOOL, INT, DINT, REAL, TIME, ... — the tag name is the ST type.
other => other.to_ascii_uppercase(),
}
}
/// Extract an initial value as an ST literal (quoting strings).
fn initial_value(iv: Node) -> Option<String> {
let simple = iv.descendants().find(|n| n.has_tag_name("simpleValue"))?;
let raw = simple.attribute("value")?.trim().to_string();
if raw.is_empty() {
return None;
}
// Numbers / booleans / time literals pass through; everything else is a
// string literal.
let is_scalar = raw.eq_ignore_ascii_case("true")
|| raw.eq_ignore_ascii_case("false")
|| raw.starts_with(['T', 't', 'D', 'd']) && raw.contains('#')
|| raw
.chars()
.all(|c| c.is_ascii_digit() || c == '.' || c == '-' || c == '+');
if is_scalar || raw.starts_with('\'') || raw.starts_with('"') {
Some(raw)
} else {
Some(format!("'{}'", raw.replace('\'', "''")))
}
}
#[cfg(test)]
mod tests {
use super::parse_plcopen;
use crate::pipeline::plc::rules;
use std::collections::HashSet;
fn rule_ids(xml: &str) -> HashSet<&'static str> {
parse_plcopen(xml)
.iter()
.flat_map(rules::analyze)
.map(|h| h.rule_id)
.collect()
}
/// A Ladder Diagram network: a rung (power rail → contact → coil) plus an
/// insecure comm block. Coils/contacts translate to assignments; the block
/// translates to a call so the port rule fires.
#[test]
fn ld_coil_and_block_translate_and_are_analysed() {
let xml = r#"<?xml version="1.0"?>
<project xmlns="http://www.plcopen.org/xml/tc6_0201">
<types><pous>
<pou name="Rung" pouType="program">
<interface><localVars>
<variable name="Motor"><type><BOOL/></type></variable>
</localVars></interface>
<body><LD>
<leftPowerRail localId="0"/>
<contact localId="1"><variable>Start</variable>
<connectionPointIn><connection refLocalId="0"/></connectionPointIn></contact>
<coil localId="2"><variable>Motor</variable>
<connectionPointIn><connection refLocalId="1"/></connectionPointIn></coil>
<inVariable localId="3"><expression>21</expression></inVariable>
<inVariable localId="4"><expression>FALSE</expression></inVariable>
<block localId="10" typeName="Ftp_Send">
<inputVariables>
<variable formalParameter="PORT">
<connectionPointIn><connection refLocalId="3"/></connectionPointIn></variable>
<variable formalParameter="ENCRYPT">
<connectionPointIn><connection refLocalId="4"/></connectionPointIn></variable>
</inputVariables>
</block>
</LD></body>
</pou>
</pous></types>
</project>"#;
let ids = rule_ids(xml);
// Ftp_Send(PORT := 21, ENCRYPT := FALSE) — port 21 is an insecure protocol.
assert!(
ids.contains("plc-insecure-protocol-port"),
"LD block should flag port 21; got {ids:?}"
);
}
/// Doubled-text regression: a graphical expression must be extracted once,
/// so literals like `502` and `FALSE` stay intact (not `502502`/`FALSEFALSE`).
#[test]
fn graphical_expression_text_is_not_duplicated() {
let xml = r#"<?xml version="1.0"?>
<project xmlns="http://www.plcopen.org/xml/tc6_0201">
<types><pous>
<pou name="Comm" pouType="program">
<body><FBD>
<inVariable localId="1"><expression>502</expression></inVariable>
<inVariable localId="2"><expression>FALSE</expression></inVariable>
<block localId="10" typeName="Modbus_TCP_Master">
<inputVariables>
<variable formalParameter="PORT">
<connectionPointIn><connection refLocalId="1"/></connectionPointIn></variable>
<variable formalParameter="AUTH">
<connectionPointIn><connection refLocalId="2"/></connectionPointIn></variable>
</inputVariables>
</block>
</FBD></body>
</pou>
</pous></types>
</project>"#;
let ids = rule_ids(xml);
assert!(ids.contains("plc-insecure-protocol-port")); // PORT := 502 (not 502502)
assert!(ids.contains("plc-insecure-comm")); // AUTH := FALSE (not FALSEFALSE)
}
}
+632
View File
@@ -0,0 +1,632 @@
//! Semantic control-logic security rules over the Structured Text AST.
//!
//! Each rule walks the parsed [`Pou`] and yields [`RuleHit`]s the scanner turns
//! into findings. Rules reason over structure (declarations, assignments, calls,
//! array accesses, division, jumps) rather than raw text, so they see through
//! formatting and comments.
use std::collections::{HashMap, HashSet};
use compliance_core::models::Severity;
use super::ast::*;
/// One rule match within a POU.
pub struct RuleHit {
pub line: u32,
pub severity: Severity,
pub rule_id: &'static str,
pub title: String,
pub description: String,
pub cwe: Option<&'static str>,
pub remediation: &'static str,
}
/// Run every rule over a POU.
pub fn analyze(pou: &Pou) -> Vec<RuleHit> {
let mut hits = Vec::new();
let ctx = Ctx::build(pou);
// Declaration-level rules.
for v in &pou.vars {
if let Some(init) = &v.init {
check_credential_binding(&v.name, init, &pou.name, &mut hits);
check_default_password(init, &v.name, &pou.name, &mut hits);
}
}
// Body walk.
walk(&pou.body, pou, &ctx, &GuardSet::default(), &mut hits);
hits
}
/// Per-POU context precomputed once.
struct Ctx {
/// Names declared in VAR_INPUT (untrusted / externally driven).
input_vars: HashSet<String>,
/// Array variable name → declared (lo, hi) bounds.
arrays: HashMap<String, (i64, i64)>,
}
impl Ctx {
fn build(pou: &Pou) -> Self {
let mut input_vars = HashSet::new();
let mut arrays = HashMap::new();
for v in &pou.vars {
if v.section == VarSection::Input {
input_vars.insert(v.name.to_ascii_lowercase());
}
if let Some(b) = v.array_bounds {
arrays.insert(v.name.to_ascii_lowercase(), b);
}
}
Self { input_vars, arrays }
}
}
/// Variables proven non-zero on the current control-flow path (from enclosing
/// `IF`/`WHILE` conditions), so guarded divisions aren't false-flagged.
#[derive(Default, Clone)]
struct GuardSet {
nonzero: HashSet<String>,
}
impl GuardSet {
fn with(&self, names: Vec<String>) -> Self {
let mut g = self.clone();
g.nonzero.extend(names);
g
}
fn is_nonzero(&self, name: &str) -> bool {
self.nonzero.contains(name)
}
}
/// Variable names a condition proves non-zero (`v <> 0`, `v > 0`, `v >= 1`,
/// `v < 0`, and conjunctions thereof).
fn guards_from_cond(cond: &Expr) -> Vec<String> {
let mut out = Vec::new();
collect_nonzero(cond, &mut out);
out
}
fn collect_nonzero(e: &Expr, out: &mut Vec<String>) {
let Expr::Binary { op, lhs, rhs, .. } = e else {
return;
};
let is_zero = |x: &Expr| {
matches!(x, Expr::Int(0, _)) || matches!(x, Expr::Real(r, _) if r.abs() < f64::EPSILON)
};
let int_of = |x: &Expr| match x {
Expr::Int(n, _) => Some(*n),
_ => None,
};
match op {
BinOp::And => {
collect_nonzero(lhs, out);
collect_nonzero(rhs, out);
}
BinOp::Ne => {
if let (Some(v), true) = (lhs.as_ident(), is_zero(rhs)) {
out.push(v.to_ascii_lowercase());
}
if let (true, Some(v)) = (is_zero(lhs), rhs.as_ident()) {
out.push(v.to_ascii_lowercase());
}
}
BinOp::Gt | BinOp::Lt => {
// v > 0 or v < 0
if let (Some(v), true) = (lhs.as_ident(), is_zero(rhs)) {
out.push(v.to_ascii_lowercase());
}
}
BinOp::Ge => {
// v >= n, n >= 1
if let (Some(v), Some(n)) = (lhs.as_ident(), int_of(rhs)) {
if n >= 1 {
out.push(v.to_ascii_lowercase());
}
}
}
_ => {}
}
}
// ── the walker ─────────────────────────────────────────────────────
fn walk(stmts: &[Stmt], pou: &Pou, ctx: &Ctx, guards: &GuardSet, hits: &mut Vec<RuleHit>) {
for s in stmts {
match s {
Stmt::Assign {
target,
value,
line,
} => {
check_safety_bypass(target, value, *line, &pou.name, hits);
// A string bound to a secret-looking target is a credential.
if let Some(name) = flatten_ident(target) {
check_credential_binding(&name, value, &pou.name, hits);
check_default_password(value, &name, &pou.name, hits);
}
walk_expr(target, pou, ctx, guards, hits);
walk_expr(value, pou, ctx, guards, hits);
}
Stmt::Call { callee, args, line } => {
check_insecure_comm(callee, args, *line, &pou.name, hits);
check_credentials_in_call(callee, args, *line, &pou.name, hits);
for a in args {
walk_expr(&a.value, pou, ctx, guards, hits);
}
}
Stmt::Jump { label, line } => hits.push(RuleHit {
line: *line,
severity: Severity::Medium,
rule_id: "plc-unstructured-jump",
title: "Unstructured jump (JMP) in control logic".to_string(),
description: format!(
"POU `{}` uses `JMP {label}`. Unstructured jumps make control flow hard to \
verify and can bypass safety interlocks or leave outputs in an undefined \
state on unexpected paths.",
pou.name
),
cwe: Some("CWE-691"),
remediation: "Replace JMP with structured constructs (IF/CASE/loops); reserve \
jumps for well-reviewed state machines only.",
}),
Stmt::If {
branches,
else_body,
..
} => {
for (cond, body) in branches {
walk_expr(cond, pou, ctx, guards, hits);
let child = guards.with(guards_from_cond(cond));
walk(body, pou, ctx, &child, hits);
}
if let Some(b) = else_body {
walk(b, pou, ctx, guards, hits);
}
}
Stmt::Case {
selector,
arms,
else_body,
..
} => {
walk_expr(selector, pou, ctx, guards, hits);
for (labels, body) in arms {
for l in labels {
walk_expr(l, pou, ctx, guards, hits);
}
walk(body, pou, ctx, guards, hits);
}
if let Some(b) = else_body {
walk(b, pou, ctx, guards, hits);
}
}
Stmt::For {
from, to, by, body, ..
} => {
walk_expr(from, pou, ctx, guards, hits);
walk_expr(to, pou, ctx, guards, hits);
if let Some(b) = by {
walk_expr(b, pou, ctx, guards, hits);
}
walk(body, pou, ctx, guards, hits);
}
Stmt::While { cond, body, .. } => {
walk_expr(cond, pou, ctx, guards, hits);
let child = guards.with(guards_from_cond(cond));
walk(body, pou, ctx, &child, hits);
}
Stmt::Repeat { body, until, .. } => {
walk(body, pou, ctx, guards, hits);
walk_expr(until, pou, ctx, guards, hits);
}
Stmt::Return { .. } | Stmt::Exit { .. } | Stmt::Label { .. } => {}
}
}
}
fn walk_expr(e: &Expr, pou: &Pou, ctx: &Ctx, guards: &GuardSet, hits: &mut Vec<RuleHit>) {
match e {
Expr::Index { base, index, line } => {
check_array_bounds(base, index, *line, ctx, &pou.name, hits);
walk_expr(base, pou, ctx, guards, hits);
walk_expr(index, pou, ctx, guards, hits);
}
Expr::Binary { op, lhs, rhs, line } => {
if matches!(op, BinOp::Div | BinOp::Mod) {
check_division(rhs, *line, &pou.name, guards, hits);
}
walk_expr(lhs, pou, ctx, guards, hits);
walk_expr(rhs, pou, ctx, guards, hits);
}
Expr::Unary { expr, .. } => walk_expr(expr, pou, ctx, guards, hits),
Expr::Member { base, .. } => walk_expr(base, pou, ctx, guards, hits),
Expr::Call { args, .. } => {
for a in args {
walk_expr(&a.value, pou, ctx, guards, hits);
}
}
_ => {}
}
}
// ── individual rules ───────────────────────────────────────────────
const SECRET_HINTS: &[&str] = &[
"password",
"passwd",
"pwd",
"secret",
"apikey",
"api_key",
"token",
"credential",
"privkey",
"private_key",
"passphrase",
];
const DEFAULT_PASSWORDS: &[&str] = &[
"admin",
"administrator",
"password",
"passwd",
"1234",
"12345",
"123456",
"0000",
"1111",
"root",
"default",
"admin123",
"changeme",
"letmein",
"guest",
"user",
"system",
"plc",
"codesys",
];
const COMM_FB_HINTS: &[&str] = &[
"modbus", "tcp", "udp", "socket", "mqtt", "opcua", "opc_ua", "ethernet", "ethip", "enip",
"dnp3", "ftp", "telnet", "http", "send", "connect", "sock", "comm", "profinet", "s7",
];
/// Insecure cleartext service ports.
const INSECURE_PORTS: &[i64] = &[21, 23, 80, 502, 20000, 44818, 102];
fn check_credential_binding(var_name: &str, value: &Expr, pou: &str, hits: &mut Vec<RuleHit>) {
let name = var_name.to_ascii_lowercase();
let looks_secret = SECRET_HINTS.iter().any(|h| name.contains(h));
if looks_secret {
if let Expr::Str(s, line) = value {
if !s.is_empty() {
hits.push(RuleHit {
line: *line,
severity: Severity::High,
rule_id: "plc-hardcoded-credential",
title: "Hardcoded credential in PLC program".to_string(),
description: format!(
"POU `{pou}` binds a hardcoded secret to `{var_name}`. Credentials \
embedded in control logic are extracted trivially from a project export \
or a firmware dump and cannot be rotated without a redeploy."
),
cwe: Some("CWE-798"),
remediation: "Store secrets outside the program (secure parameter store / \
operator-entered, retained-but-protected memory); never commit \
them to the POU.",
});
}
}
}
}
fn check_default_password(value: &Expr, var_name: &str, pou: &str, hits: &mut Vec<RuleHit>) {
if let Expr::Str(s, line) = value {
let lower = s.to_ascii_lowercase();
if DEFAULT_PASSWORDS.contains(&lower.as_str()) {
hits.push(RuleHit {
line: *line,
severity: Severity::Critical,
rule_id: "plc-default-password",
title: "Default/weak password in PLC program".to_string(),
description: format!(
"POU `{pou}` uses the well-known default/weak password `{s}` (bound to \
`{var_name}`). Default PLC credentials are the first thing an attacker tries."
),
cwe: Some("CWE-1393"),
remediation:
"Require a strong, unique, operator-set password; block commissioning \
until the default is changed.",
});
}
}
}
fn check_credentials_in_call(
callee: &str,
args: &[CallArg],
line: u32,
pou: &str,
hits: &mut Vec<RuleHit>,
) {
for a in args {
if let Some(name) = &a.name {
let n = name.to_ascii_lowercase();
if SECRET_HINTS.iter().any(|h| n.contains(h)) {
if let Expr::Str(s, l) = &a.value {
if !s.is_empty() {
hits.push(RuleHit {
line: *l,
severity: Severity::High,
rule_id: "plc-hardcoded-credential",
title: "Hardcoded credential passed to a function block".to_string(),
description: format!(
"POU `{pou}` passes a hardcoded secret as `{name}` to `{callee}`."
),
cwe: Some("CWE-798"),
remediation: "Supply credentials from protected configuration at \
runtime, not as a literal argument.",
});
}
}
}
}
}
let _ = line;
}
fn check_safety_bypass(target: &Expr, value: &Expr, line: u32, pou: &str, hits: &mut Vec<RuleHit>) {
let Some(name) = flatten_ident(target) else {
return;
};
let n = name.to_ascii_lowercase();
let safety = [
"safety",
"estop",
"e_stop",
"emergency",
"interlock",
"guard",
"permit",
]
.iter()
.any(|h| n.contains(h));
let watchdog = n.contains("watchdog") || n.contains("wdt");
// A safety enable / interlock / watchdog signal driven to FALSE or 0 in
// application logic is a bypass (e.g. `Safety_Enable := FALSE`, `Watchdog_Kick := 0`).
let disabling = matches!(value, Expr::Bool(false, _)) || matches!(value, Expr::Int(0, _));
if (safety || watchdog) && disabling {
hits.push(RuleHit {
line,
severity: Severity::Critical,
rule_id: "plc-safety-bypass",
title: "Safety interlock / watchdog disabled in logic".to_string(),
description: format!(
"POU `{pou}` disables a safety-related signal (`{name}`) in program logic. \
Bypassing interlocks or watchdogs in code defeats the plant's protective \
functions and is a direct hazard."
),
cwe: Some("CWE-1384"),
remediation: "Never disable safety functions from application logic; safety must be \
handled by a certified safety controller / hard-wired circuit.",
});
}
}
fn check_array_bounds(
base: &Expr,
index: &Expr,
line: u32,
ctx: &Ctx,
pou: &str,
hits: &mut Vec<RuleHit>,
) {
// Only reason about arrays we know the bounds of.
let Some(arr_name) = base.as_ident() else {
return;
};
if !ctx.arrays.contains_key(&arr_name.to_ascii_lowercase()) {
return;
}
// Index by an untrusted input variable → potential out-of-bounds access.
if let Some(idx_name) = index.as_ident() {
if ctx.input_vars.contains(&idx_name.to_ascii_lowercase()) {
hits.push(RuleHit {
line,
severity: Severity::High,
rule_id: "plc-array-unchecked-index",
title: "Array indexed by unvalidated input".to_string(),
description: format!(
"POU `{pou}` indexes array `{arr_name}` with the input variable `{idx_name}` \
without a validated bounds check. An out-of-range index corrupts adjacent \
memory or faults the PLC (loss of control)."
),
cwe: Some("CWE-129"),
remediation: "Clamp or validate the index against the array bounds (e.g. \
`LIMIT`/explicit `IF idx >= lo AND idx <= hi`) before the access.",
});
}
}
}
fn check_division(
divisor: &Expr,
line: u32,
pou: &str,
guards: &GuardSet,
hits: &mut Vec<RuleHit>,
) {
// A divisor proven non-zero by an enclosing guard is safe.
if let Expr::Ident(name, _) = divisor {
if guards.is_nonzero(&name.to_ascii_lowercase()) {
return;
}
}
// Flag division by a variable (could be zero); nonzero literals are fine.
let risky = matches!(
divisor,
Expr::Ident(_, _) | Expr::Member { .. } | Expr::Index { .. } | Expr::Int(0, _)
);
if risky {
hits.push(RuleHit {
line,
severity: Severity::Medium,
rule_id: "plc-division-by-zero",
title: "Division by a variable without a zero-guard".to_string(),
description: format!(
"POU `{pou}` divides by a variable that is not proven non-zero. A zero divisor \
raises a PLC exception and can halt the scan cycle (denial of control)."
),
cwe: Some("CWE-369"),
remediation: "Guard the divisor (`IF d <> 0 THEN …`) or use a safe-divide helper that \
returns a defined value for a zero denominator.",
});
}
}
fn check_insecure_comm(
callee: &str,
args: &[CallArg],
line: u32,
pou: &str,
hits: &mut Vec<RuleHit>,
) {
let c = callee.to_ascii_lowercase();
let is_comm = COMM_FB_HINTS.iter().any(|h| c.contains(h));
if !is_comm {
return;
}
// Auth/encryption explicitly disabled.
for a in args {
if let Some(name) = &a.name {
let n = name.to_ascii_lowercase();
let security_flag = ["auth", "secure", "encrypt", "tls", "ssl", "authentication"]
.iter()
.any(|h| n.contains(h));
if security_flag && matches!(a.value, Expr::Bool(false, _)) {
hits.push(RuleHit {
line,
severity: Severity::High,
rule_id: "plc-insecure-comm",
title: "Network communication with security disabled".to_string(),
description: format!(
"POU `{pou}` calls `{callee}` with `{name} := FALSE`, disabling \
authentication/encryption on an industrial network link."
),
cwe: Some("CWE-319"),
remediation: "Enable authentication + transport encryption; segment OT \
networks and restrict the endpoint to trusted peers.",
});
}
}
// Well-known cleartext port literal.
if let Expr::Int(p, _) = &a.value {
if INSECURE_PORTS.contains(p) {
hits.push(RuleHit {
line,
severity: Severity::Medium,
rule_id: "plc-insecure-protocol-port",
title: "Cleartext industrial protocol port".to_string(),
description: format!(
"POU `{pou}` opens `{callee}` on port {p}, a well-known cleartext OT \
protocol port with no built-in authentication or encryption."
),
cwe: Some("CWE-319"),
remediation: "Front the protocol with a secure gateway/VPN, or use the \
authenticated/encrypted variant; never expose it to untrusted \
networks.",
});
}
}
}
let _ = line;
}
/// The dotted/base identifier of an lvalue expression (`a`, `a.b` → `a.b`,
/// `a[i]` → `a`), for name-based rules.
fn flatten_ident(e: &Expr) -> Option<String> {
match e {
Expr::Ident(n, _) => Some(n.clone()),
Expr::Member { base, field, .. } => flatten_ident(base).map(|b| format!("{b}.{field}")),
Expr::Index { base, .. } => flatten_ident(base),
_ => None,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::pipeline::plc::parser;
const VULN: &str = r#"
FUNCTION_BLOCK CommCtrl
VAR_INPUT
cmdIndex : INT;
END_VAR
VAR
Password : STRING := 'admin123';
buffer : ARRAY[0..15] OF INT;
Safety_Enable : BOOL := TRUE;
divisor : INT;
result : INT;
END_VAR
Safety_Enable := FALSE;
result := 100 / divisor;
buffer[cmdIndex] := 1;
Modbus_Connect(IP := '192.168.0.10', PORT := 502, AUTH := FALSE);
IF cmdIndex > 100 THEN
JMP fault;
END_IF;
fault:
result := 0;
END_FUNCTION_BLOCK
"#;
fn rule_ids(src: &str) -> Vec<&'static str> {
parser::parse(src)
.iter()
.flat_map(analyze)
.map(|h| h.rule_id)
.collect()
}
#[test]
fn vulnerable_program_triggers_every_rule() {
let ids = rule_ids(VULN);
for expected in [
"plc-hardcoded-credential",
"plc-default-password",
"plc-safety-bypass",
"plc-division-by-zero",
"plc-array-unchecked-index",
"plc-insecure-comm",
"plc-insecure-protocol-port",
"plc-unstructured-jump",
] {
assert!(
ids.contains(&expected),
"expected rule {expected}, got {ids:?}"
);
}
}
#[test]
fn clean_program_has_no_findings() {
let clean = r#"
PROGRAM Clean
VAR
a : INT := 5;
b : INT := 3;
total : INT;
END_VAR
IF b <> 0 THEN
total := a / b;
END_IF;
END_PROGRAM
"#;
assert!(rule_ids(clean).is_empty(), "clean program should be quiet");
}
}
+253
View File
@@ -0,0 +1,253 @@
//! Control-application dependency SBOM from a CODESYS `.projectarchive`.
//!
//! A `.projectarchive` is a ZIP that bundles the project plus its referenced
//! libraries and the target runtime. Each referenced library is an entry whose
//! path segment follows the CODESYS convention
//! `Name, Major.Minor.Patch.Build (Company)` (e.g. `Standard, 3.5.18.0 (System)`,
//! `CSV Utility SL, 1.9.0.0 (CODESYS)`); the runtime appears as a device-descriptor
//! entry `CODESYS Control … <version> …`. We enumerate those entries — no binary
//! parsing — and emit SBOM components tagged `pkg:codesys/…`, so the CVE pipeline
//! can match them (the runtime `Cmp*` / `3SLicense` components carry real CODESYS
//! CVEs).
use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
use compliance_core::models::SbomEntry;
/// Collect the control-application SBOM from every `.projectarchive` reachable for
/// a target: the ingested artifact file itself (an uploaded archive), plus any
/// `*.projectarchive` committed inside the working tree — e.g. a git repo or an
/// extracted source archive that ships the archive alongside its PLCopen XML / ST
/// exports. Deduplicated by (name, version).
pub fn collect_sbom(artifact_file: &Path, working_path: &Path, repo_id: &str) -> Vec<SbomEntry> {
let mut archives: Vec<PathBuf> = Vec::new();
if artifact_file.is_file() {
archives.push(artifact_file.to_path_buf());
}
for entry in walkdir::WalkDir::new(working_path)
.max_depth(8)
.into_iter()
.filter_map(|e| e.ok())
{
let p = entry.path();
if entry.file_type().is_file()
&& p.extension()
.and_then(|x| x.to_str())
.is_some_and(|x| x.eq_ignore_ascii_case("projectarchive"))
{
archives.push(p.to_path_buf());
}
}
let mut seen: BTreeSet<(String, String)> = BTreeSet::new();
let mut out = Vec::new();
for a in archives {
for e in projectarchive_sbom(&a, repo_id) {
if seen.insert((e.name.clone(), e.version.clone())) {
out.push(e);
}
}
}
out
}
/// Extract CODESYS library + runtime components from a `.projectarchive` (a zip).
/// Best-effort: returns empty if the file is not a readable zip (e.g. a bare
/// `.st`/`.xml` project, which carries no library manifest).
pub fn projectarchive_sbom(archive: &Path, repo_id: &str) -> Vec<SbomEntry> {
let Ok(file) = std::fs::File::open(archive) else {
return Vec::new();
};
let Ok(mut zip) = zip::ZipArchive::new(file) else {
return Vec::new();
};
let mut seen: BTreeSet<(String, String)> = BTreeSet::new();
let mut entries = Vec::new();
for i in 0..zip.len() {
let Ok(entry) = zip.by_index(i) else {
continue;
};
// Entry paths use `\` (Windows-authored) and/or `/` separators; the
// component id is one path segment.
for seg in entry.name().split(['/', '\\']) {
if let Some((name, version)) = parse_library(seg).or_else(|| parse_runtime(seg)) {
if seen.insert((name.clone(), version.clone())) {
let mut e = SbomEntry::new(
repo_id.to_string(),
name.clone(),
version.clone(),
"codesys".to_string(),
);
e.purl = Some(format!(
"pkg:codesys/{}@{version}",
name.replace(' ', "%20")
));
entries.push(e);
}
}
}
}
entries
}
/// `Name, X.Y.Z.W (Company)` → (name, version).
fn parse_library(seg: &str) -> Option<(String, String)> {
let seg = seg.trim();
// Company is the trailing "(…)".
let open = seg.rfind(" (")?;
let rest = &seg[open + 2..];
let close = rest.find(')')?;
if rest[..close].trim().is_empty() {
return None;
}
let head = seg[..open].trim(); // "Name, X.Y.Z.W"
let comma = head.rfind(", ")?;
let name = head[..comma].trim().to_string();
let version = head[comma + 2..].trim().to_string();
if name.is_empty() || !is_dotted_version(&version) {
return None;
}
Some((name, version))
}
/// Device-descriptor entry `CODESYS Control … X.Y.Z.W …` → (runtime name, version).
fn parse_runtime(seg: &str) -> Option<(String, String)> {
let seg = seg.trim();
if !seg.starts_with("CODESYS Control") {
return None;
}
let version = seg
.split_whitespace()
.find(|t| is_dotted_version(t))?
.to_string();
// The runtime name is the first field, before the run of padding spaces that
// precede the descriptor's numeric columns.
let name = seg.split(" ").next().unwrap_or(seg).trim().to_string();
if name.is_empty() {
return None;
}
Some((name, version))
}
/// A dotted numeric version with at least 3 components (`3.5.18.0`, `4.17.0.0`).
fn is_dotted_version(s: &str) -> bool {
let parts: Vec<&str> = s.split('.').collect();
parts.len() >= 3
&& parts
.iter()
.all(|p| !p.is_empty() && p.chars().all(|c| c.is_ascii_digit()))
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use std::io::Write;
/// Build a synthetic `.projectarchive` (zip) mirroring the real CODESYS entry
/// naming (verified against Proemion/codesys-examples): a native `.project`,
/// referenced libraries as `Name, Version (Company)` segments, and a runtime
/// device descriptor.
fn synthetic_archive(dir: &Path) -> std::path::PathBuf {
let path = dir.join("App.projectarchive");
write_synthetic_archive(&path);
path
}
fn write_synthetic_archive(path: &Path) {
let file = std::fs::File::create(path).expect("create");
let mut zip = zip::ZipWriter::new(file);
let opts: zip::write::SimpleFileOptions = Default::default();
let names = [
"App.project",
r"{b0b5}\App.Device.Plc.compileinfo",
r"{e179}\Standard, 3.5.18.0 (System) standard.compiled-library-v3",
r"{e179}\Util, 3.5.21.0 (System) util.compiled-library-v3",
r"{e179}\CSV Utility SL, 1.9.0.0 (CODESYS) csv utility sl.compiled-library-v3",
r"{e179}\3SLicense, 3.5.20.0 (CODESYS) 3slicense.compiled-library-v3",
r"{0c63}\CODESYS Control for Linux ARM SL 0000 0006 4.17.0.0 4096 .zip",
];
for n in names {
zip.start_file(n, opts).expect("start");
zip.write_all(b"x").expect("write");
}
zip.finish().expect("finish");
}
#[test]
fn extracts_libraries_and_runtime_from_projectarchive() {
let tmp = std::env::temp_dir().join(format!("cs-plc-sbom-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&tmp).expect("mkdir");
let archive = synthetic_archive(&tmp);
let entries = projectarchive_sbom(&archive, "plc-target");
let by_name: HashMap<&str, &SbomEntry> =
entries.iter().map(|e| (e.name.as_str(), e)).collect();
// Libraries with their versions.
assert_eq!(
by_name.get("Standard").map(|e| e.version.as_str()),
Some("3.5.18.0")
);
assert_eq!(
by_name.get("Util").map(|e| e.version.as_str()),
Some("3.5.21.0")
);
assert_eq!(
by_name.get("CSV Utility SL").map(|e| e.version.as_str()),
Some("1.9.0.0"),
"multi-word library names must parse"
);
assert!(by_name.contains_key("3SLicense"));
// The runtime, from the device descriptor.
assert_eq!(
by_name
.get("CODESYS Control for Linux ARM SL")
.map(|e| e.version.as_str()),
Some("4.17.0.0")
);
// Every component is CODESYS-tagged with a purl the CVE pipeline can match,
// and the native `.project` / compileinfo are not mistaken for components.
for e in &entries {
assert_eq!(e.package_manager, "codesys");
assert!(e.purl.as_deref().unwrap_or("").starts_with("pkg:codesys/"));
}
assert!(!by_name.contains_key("App"));
let _ = std::fs::remove_dir_all(&tmp);
}
#[test]
fn collect_sbom_finds_a_projectarchive_committed_in_a_git_tree() {
let tmp = std::env::temp_dir().join(format!("cs-plc-collect-{}", uuid::Uuid::new_v4()));
let src = tmp.join("clone/src");
std::fs::create_dir_all(&src).expect("mkdir");
// Simulate a git clone that commits the archive alongside its exports.
write_synthetic_archive(&src.join("PumpStation.projectarchive"));
// The artifact "file" is a git URL (not a real file), so the SBOM must
// come from walking the cloned tree.
let entries = collect_sbom(Path::new("https://git.example/plc.git"), &tmp, "t");
let names: std::collections::HashSet<&str> =
entries.iter().map(|e| e.name.as_str()).collect();
assert!(
names.contains("Standard"),
"found libs in the committed archive"
);
assert!(names.contains("CODESYS Control for Linux ARM SL"));
let _ = std::fs::remove_dir_all(&tmp);
}
#[test]
fn non_zip_file_yields_no_sbom() {
let tmp = std::env::temp_dir().join(format!("cs-plc-sbom-st-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&tmp).expect("mkdir");
let st = tmp.join("prog.st");
std::fs::write(&st, "PROGRAM P\nVAR x : INT; END_VAR\nEND_PROGRAM\n").expect("write");
assert!(projectarchive_sbom(&st, "t").is_empty());
let _ = std::fs::remove_dir_all(&tmp);
}
}

Some files were not shown because too many files have changed in this diff Show More