Compare commits

..
Author SHA1 Message Date
Sharang ParnerkarandClaude Opus 4.8 e3b918b365 fix(dashboard): Findings/SBOM filter by onboarded targets; accurate target findings_count
CI / Check (pull_request) Successful in 7m4s
CI / Detect Changes (pull_request) Has been skipped
CI / Deploy Agent (pull_request) Has been skipped
CI / Deploy Dashboard (pull_request) Has been skipped
CI / Deploy Docs (pull_request) Has been skipped
CI / Deploy MCP (pull_request) Has been skipped
The Findings and SBOM pages populated their target dropdown from the legacy
`repositories` collection, so onboarded targets never appeared and their
findings/SBOM couldn't be filtered by name (the data was there, keyed by the
target id). Point both dropdowns at `onboarded_targets` via `fetch_targets`.

Also refresh `OnboardedTarget.findings_count` at the end of `run_target`: the
shared pipeline (Stage 7) increments the legacy `repositories` doc, which the
unified path has none of, so the Targets page always showed 0. Set the accurate
total (count of findings keyed by the target id) on the target itself.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 09:52:09 +02:00
sharang bf32b9939a fix(onboarding): targets visibility + unified pipeline by default (#154)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Successful in 3m41s
CI / Deploy Dashboard (push) Successful in 2m46s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m46s
2026-07-13 07:29:04 +00:00
sharang 669e1f1b03 feat(onboarding): scan-trigger endpoint + wizard Run-Scan button (#153)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 4s
CI / Deploy Agent (push) Successful in 3m34s
CI / Deploy Dashboard (push) Successful in 2m56s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 22:19:32 +00:00
sharang 9e70bd1c8e ci: don't cancel-in-progress for main-branch runs (only pull_request) (#152)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 22:08:58 +00:00
sharang 9ec07ff7a1 fix(ci): authenticate tramiton fetch in dashboard + mcp image builds (#147)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m47s
CI / Deploy Dashboard (push) Successful in 2m37s
CI / Deploy Docs (push) Has been skipped
2026-07-12 21:52:38 +00:00
sharang 0e57c2d7a7 feat(pipeline): run tramiton classification + provision DAST in run_target (#146)
CI / Deploy Agent (push) Successful in 3m40s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy MCP (push) Has been skipped
2026-07-12 21:49:22 +00:00
sharang c6a02f0ac3 feat(pipeline): unified run_target execution behind UNIFIED_PIPELINE (#145)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Successful in 3m47s
CI / Deploy Dashboard (push) Failing after 4m28s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Failing after 5s
2026-07-12 21:31:21 +00:00
sharang 17f788858d feat(dashboard): onboarding wizard UI (#144)
CI / Detect Changes (push) Successful in 3s
CI / Check (push) Has been skipped
CI / Deploy Docs (push) Has been cancelled
CI / Deploy MCP (push) Has been cancelled
CI / Deploy Dashboard (push) Has been cancelled
CI / Deploy Agent (push) Has been skipped
2026-07-12 21:28:52 +00:00
sharangandSharang Parnerkar c6b5622113 fix(ci): authenticate tramiton-core fetch in the agent image build (main deploys) (#143)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m46s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
The PR `check` job authenticates the private tramiton-core git fetch, but the main-branch **`deploy-agent`** job builds `Dockerfile.agent` where cargo also fetches tramiton-core — inside the image build, with no credentials — so **agent image builds on main fail**.

Fix: inject the PAT as a **BuildKit secret** (never baked into an image layer).
- `Dockerfile.agent`: `RUN --mount=type=secret,id=tramiton_token …` applies the same `https`-insteadOf rewrite + `CARGO_NET_GIT_FETCH_WITH_CLI=true` before `cargo build`.
- `deploy-agent`: `DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN …`, reusing the existing `TRAMITON_FETCH_TOKEN` secret.

Only the agent image depends on tramiton-core (dashboard/mcp/docs unaffected). **Self-tests on merge** — changing `Dockerfile.agent` makes `detect-changes` run `deploy-agent`.

Note: couldn't fully run the image build locally (no PAT value on hand + no `.dockerignore` so the context is large), but this mirrors the working PR-stage auth and uses standard BuildKit secret injection. Assumes the deploy runner's Docker daemon supports BuildKit (docker:27-cli → yes).
---------

Co-authored-by: Sharang Parnerkar <30073382+mighty840@users.noreply.github.com>
Reviewed-on: #143
2026-07-12 20:25:39 +00:00
sharang a204f0c59c feat(api): onboarding endpoints for unified targets (#142)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 20:15:22 +00:00
sharang cf4afdda1b feat(migrate): onboarding backfill (repositories + dast_targets -> onboarded_targets) (#141)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-12 20:11:16 +00:00
sharang a074efd0b4 ci: Kellnr crates.io mirror + persistent S3-backed sccache (#140)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Has been skipped
CI / Deploy Dashboard (push) Has been skipped
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Has been skipped
2026-07-10 16:30:51 +00:00
sharang ef6ee3dcd1 feat(onboarding): artifact ingest + classifier + native tramiton + suite seams (#138)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 3s
CI / Deploy Agent (push) Failing after 5s
CI / Deploy Dashboard (push) Failing after 5s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Failing after 4s
2026-07-10 16:00:51 +00:00
sharang 675c4ef699 feat(onboarding): unified multi-target model + scan matrix foundation (#134)
CI / Check (push) Has been skipped
CI / Detect Changes (push) Successful in 5s
CI / Deploy Agent (push) Successful in 7m36s
CI / Deploy Dashboard (push) Successful in 7m9s
CI / Deploy Docs (push) Has been skipped
CI / Deploy MCP (push) Successful in 1m44s
2026-07-10 13:41:25 +00:00
39 changed files with 2943 additions and 384 deletions
+67 -10
View File
@@ -9,15 +9,33 @@ on:
env:
CARGO_TERM_COLOR: always
RUSTFLAGS: "-D warnings"
# sccache caches compilation artifacts within a job so that compiling
# both --features server and --features web shares common crate work.
# Compile cache: sccache -> Hetzner S3 (breakpilot-sccache), runner-independent
# and persistent across CI runs (own key prefix). Reuses the shared cluster S3
# creds (same bucket as werkpilot). Requires repo secrets HETZNER_S3_ACCESS_KEY
# and HETZNER_S3_SECRET_KEY.
RUSTC_WRAPPER: /usr/local/bin/sccache
SCCACHE_DIR: /tmp/sccache
SCCACHE_BUCKET: breakpilot-sccache
SCCACHE_ENDPOINT: https://nbg1.your-objectstorage.com
SCCACHE_REGION: auto
SCCACHE_S3_USE_SSL: "true"
SCCACHE_S3_KEY_PREFIX: compliance-scanner
AWS_ACCESS_KEY_ID: ${{ secrets.HETZNER_S3_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.HETZNER_S3_SECRET_KEY }}
# compliance-agent depends on tramiton-core via git; use the system git so the
# credential rewrite below (see "Configure git auth ...") is honored on fetch.
CARGO_NET_GIT_FETCH_WITH_CLI: "true"
# Throttle cargo so a ~670-crate concurrent download burst doesn't 429 the
# Kellnr mirror: fewer concurrent connections (HTTP/1.1) + more retries.
CARGO_NET_RETRY: "10"
CARGO_HTTP_MULTIPLEXING: "false"
# Cancel in-progress runs for the same branch/PR
# Cancel superseded PR runs, but NEVER cancel main-branch runs — those build and
# deploy per-service images, and cancelling one merge's deploy when the next
# merge lands leaves a service un-deployed (as happened between two back-to-back
# merges). So cancel-in-progress only for pull_request events.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
# ---------------------------------------------------------------------------
@@ -36,16 +54,44 @@ jobs:
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}"
git checkout FETCH_HEAD
# Resolve crates.io deps through the self-hosted Kellnr mirror (cached,
# crates.io-independent). Git deps (tramiton-core) are unaffected — source
# replacement only applies to crates.io-sourced crates.
- name: Use Kellnr crates.io mirror
run: |
: "${CARGO_HOME:=/usr/local/cargo}"
mkdir -p "$CARGO_HOME"
{
echo '[source.crates-io]'
echo 'replace-with = "kellnr"'
echo '[registries.kellnr]'
echo 'index = "sparse+https://crates.meghsakha.com/api/v1/cratesio/"'
} >> "$CARGO_HOME/config.toml"
env:
RUSTC_WRAPPER: ""
- name: Install tools
run: |
rustup component add rustfmt clippy
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.9.1/sccache-v0.9.1-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.9.1-x86_64-unknown-linux-musl/sccache
curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
| tar xz --strip-components=1 -C /usr/local/bin/ sccache-v0.10.0-x86_64-unknown-linux-musl/sccache
chmod +x /usr/local/bin/sccache
cargo install cargo-audit --locked
env:
RUSTC_WRAPPER: ""
# compliance-agent has a git dependency on tramiton-core (a private repo on
# this Gitea instance). Rewrite its SSH URL to HTTPS + a PAT so the runner
# can fetch it. Requires the repo secret TRAMITON_FETCH_TOKEN (a Gitea PAT
# with read:repository, owned by a user with access to sharang/tramiton).
# (Honored on fetch because CARGO_NET_GIT_FETCH_WITH_CLI=true uses system git.)
- name: Configure git auth for private tramiton dependency
run: |
git config --global \
url."https://sharang:${{ secrets.TRAMITON_FETCH_TOKEN }}@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"
env:
RUSTC_WRAPPER: ""
# Format (no compilation needed)
- name: Format
run: cargo fmt --all --check
@@ -148,13 +194,18 @@ jobs:
image: docker:27-cli
steps:
- name: Build, push and trigger orca redeploy
env:
# PAT for fetching the private tramiton-core git dependency during the
# image build (injected as a BuildKit secret, never baked into a layer).
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-agent
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.agent -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy agent"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
@@ -169,13 +220,16 @@ jobs:
image: docker:27-cli
steps:
- name: Build, push and trigger orca redeploy
env:
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-dashboard
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.dashboard -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy dashboard"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
@@ -211,13 +265,16 @@ jobs:
image: docker:27-cli
steps:
- name: Build, push and trigger orca redeploy
env:
TRAMITON_FETCH_TOKEN: ${{ secrets.TRAMITON_FETCH_TOKEN }}
run: |
apk add --no-cache git curl openssl
git init && git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git fetch --depth=1 origin "${GITHUB_SHA}" && git checkout FETCH_HEAD
IMAGE=registry.meghsakha.com/compliance-mcp
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login registry.meghsakha.com -u "${{ secrets.REGISTRY_USERNAME }}" --password-stdin
docker build -f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN \
-f Dockerfile.mcp -t "$IMAGE:latest" -t "$IMAGE:${GITHUB_SHA}" .
docker push "$IMAGE:latest" && docker push "$IMAGE:${GITHUB_SHA}"
PAYLOAD=$(printf '{"ref":"refs/heads/main","repository":{"full_name":"sharang/compliance-scanner-agent"},"head_commit":{"id":"%s","message":"deploy mcp"}}' "${GITHUB_SHA}")
SIG=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hmac "${{ secrets.ORCA_WEBHOOK_SECRET }}" | awk '{print $2}')
Generated
+65 -2
View File
@@ -692,6 +692,7 @@ dependencies = [
"tower-http",
"tracing",
"tracing-subscriber",
"tramiton-core",
"urlencoding",
"uuid",
"walkdir",
@@ -4197,7 +4198,7 @@ version = "3.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "219cb19e96be00ab2e37d6e299658a0cfa83e52429179969b0f0121b4ac46983"
dependencies = [
"toml_edit",
"toml_edit 0.23.10+spec-1.0.0",
]
[[package]]
@@ -4996,6 +4997,15 @@ dependencies = [
"syn",
]
[[package]]
name = "serde_spanned"
version = "0.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3"
dependencies = [
"serde",
]
[[package]]
name = "serde_urlencoded"
version = "0.7.1"
@@ -5810,6 +5820,27 @@ dependencies = [
"tokio",
]
[[package]]
name = "toml"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362"
dependencies = [
"serde",
"serde_spanned",
"toml_datetime 0.6.11",
"toml_edit 0.22.27",
]
[[package]]
name = "toml_datetime"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
dependencies = [
"serde",
]
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
@@ -5819,6 +5850,20 @@ dependencies = [
"serde_core",
]
[[package]]
name = "toml_edit"
version = "0.22.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a"
dependencies = [
"indexmap 2.13.0",
"serde",
"serde_spanned",
"toml_datetime 0.6.11",
"toml_write",
"winnow",
]
[[package]]
name = "toml_edit"
version = "0.23.10+spec-1.0.0"
@@ -5826,7 +5871,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84c8b9f757e028cee9fa244aea147aab2a9ec09d5325a9b01e0a49730c2b5269"
dependencies = [
"indexmap 2.13.0",
"toml_datetime",
"toml_datetime 0.7.5+spec-1.1.0",
"toml_parser",
"winnow",
]
@@ -5840,6 +5885,12 @@ dependencies = [
"winnow",
]
[[package]]
name = "toml_write"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "tonic"
version = "0.12.3"
@@ -6086,6 +6137,18 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "tramiton-core"
version = "0.4.0"
source = "git+ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git?tag=v0.4.0#e3dc1bf7027a2f6d7b1fe43043d6dfa887ce4af3"
dependencies = [
"serde",
"tempfile",
"thiserror 1.0.69",
"toml",
"walkdir",
]
[[package]]
name = "tree-sitter"
version = "0.24.7"
+10 -1
View File
@@ -2,7 +2,16 @@ FROM rust:1.94-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release -p compliance-agent
# compliance-agent depends on the private tramiton-core git repo. Authenticate
# the fetch with a PAT passed as a BuildKit secret (never baked into a layer).
# Build with: DOCKER_BUILDKIT=1 docker build --secret id=tramiton_token,env=TRAMITON_FETCH_TOKEN ...
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release -p compliance-agent
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 git curl python3 python3-pip npm golang-go php-cli && rm -rf /var/lib/apt/lists/*
+10 -1
View File
@@ -7,7 +7,16 @@ ARG DOCS_URL=/docs
WORKDIR /app
COPY . .
ENV DOCS_URL=${DOCS_URL}
RUN dx build --release --package compliance-dashboard
# compliance-agent (a workspace member) depends on the private tramiton-core git
# repo, so the workspace resolve needs it even to build the dashboard.
# Authenticate the fetch with a PAT passed as a BuildKit secret.
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true dx build --release --package compliance-dashboard
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 && rm -rf /var/lib/apt/lists/*
+10 -1
View File
@@ -2,7 +2,16 @@ FROM rust:1.94-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release -p compliance-mcp
# compliance-agent (a workspace member) depends on the private tramiton-core git
# repo, so the workspace resolve needs it even to build the mcp binary.
# Authenticate the fetch with a PAT passed as a BuildKit secret.
RUN --mount=type=secret,id=tramiton_token \
if [ -s /run/secrets/tramiton_token ]; then \
git config --global \
url."https://sharang:$(cat /run/secrets/tramiton_token)@gitea.meghsakha.com/".insteadOf \
"ssh://git@gitea.meghsakha.com:22222/"; \
fi && \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release -p compliance-mcp
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates libssl3 && rm -rf /var/lib/apt/lists/*
+5
View File
@@ -10,6 +10,11 @@ workspace = true
compliance-core = { workspace = true, features = ["mongodb", "telemetry", "axum"] }
compliance-graph = { path = "../compliance-graph" }
compliance-dast = { path = "../compliance-dast" }
# Native firmware build/target detection for bare-metal & RTOS artifacts.
# Same-company IP, used directly (not via CLI) so the whole tramiton suite is
# available to the onboarding classifier. NOTE: CI must be able to fetch this
# private repo (see the git-auth step in .gitea/workflows/ci.yml).
tramiton-core = { git = "ssh://git@gitea.meghsakha.com:22222/sharang/tramiton.git", tag = "v0.4.0" }
serde = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true }
+25 -1
View File
@@ -63,7 +63,31 @@ impl ComplianceAgent {
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator.run(repo_id, trigger).await
if self.config.unified_pipeline {
orchestrator.run_target(repo_id, trigger).await
} else {
orchestrator.run(repo_id, trigger).await
}
}
/// Run a scan for an onboarded target through the unified pipeline,
/// unconditionally.
///
/// Unlike [`Self::run_scan`], this does *not* consult the
/// `unified_pipeline` transition flag: the caller (the `/targets/{id}/scan`
/// endpoint) operates on `onboarded_targets` by construction, so it must
/// always dispatch to `run_target` regardless of how the legacy paths
/// (scheduler, webhooks, `/repositories/{id}/scan`) are configured.
pub async fn run_target_scan(
&self,
tenant_id: &str,
target_id: &str,
trigger: compliance_core::models::ScanTrigger,
) -> Result<(), crate::error::AgentError> {
let db = self.db_pool.for_tenant_id(tenant_id).await?;
let orchestrator =
PipelineOrchestrator::new(self.config.clone(), db, self.llm.clone(), self.http.clone());
orchestrator.run_target(target_id, trigger).await
}
/// Run a PR review: scan the diff and post review comments.
+1
View File
@@ -9,6 +9,7 @@ pub mod help_chat;
pub mod issues;
pub mod mcp_tokens;
pub mod notifications;
pub mod onboarding;
pub mod pentest_handlers;
pub use pentest_handlers as pentest;
pub mod repos;
@@ -0,0 +1,393 @@
//! Onboarding API — CRUD for unified targets, artifact add, classification, and
//! the scan-applicability matrix. The wizard (and future integrations) drive
//! onboarding through these endpoints.
use std::collections::HashMap;
use std::sync::Arc;
use axum::extract::{Extension, Path, Query};
use axum::http::StatusCode;
use axum::Json;
use mongodb::bson::{doc, oid::ObjectId, to_bson};
use serde::{Deserialize, Serialize};
use compliance_core::models::{
Artifact, ArtifactKind, ComplianceProfile, OnboardedTarget, PlcFormat, TargetScanConfig,
TargetType,
};
use compliance_core::scan_matrix::{applicable_scans, supports_pentest};
use compliance_core::tenant_ctx::TenantCtx;
use crate::agent::ComplianceAgent;
use crate::classify::{classify_target, MockFirmwareDetector};
use super::dto::tenant_db;
use super::{collect_cursor_async, ApiResponse, PaginationParams};
type AgentExt = Extension<Arc<ComplianceAgent>>;
/// A client-supplied artifact spec. The server builds the [`Artifact`] (and its
/// id) from it, so clients never set internal fields.
#[derive(Deserialize)]
pub struct ArtifactInput {
pub kind: ArtifactKind,
pub source_ref: String,
#[serde(default)]
pub branch: Option<String>,
#[serde(default)]
pub plc_format: Option<PlcFormat>,
}
impl ArtifactInput {
fn build(&self) -> Artifact {
let s = self.source_ref.clone();
match self.kind {
ArtifactKind::GitRepo => {
Artifact::git_repo(s, self.branch.clone().unwrap_or_else(|| "main".to_string()))
}
ArtifactKind::LiveUrl => Artifact::live_url(s),
ArtifactKind::FirmwareImage => Artifact::firmware_image(s),
ArtifactKind::SourceArchive => Artifact::source_archive(s),
ArtifactKind::MobilePackage => Artifact::mobile_package(s),
ArtifactKind::ContainerImage => Artifact::container_image(s),
ArtifactKind::PlcProject => {
Artifact::plc_project(s, self.plc_format.unwrap_or(PlcFormat::PlcopenXml))
}
ArtifactKind::PlaintextDescription => Artifact::plaintext(s),
}
}
}
#[derive(Deserialize)]
pub struct CreateTargetRequest {
pub name: String,
pub target_type: TargetType,
#[serde(default)]
pub description: Option<String>,
#[serde(default)]
pub artifacts: Vec<ArtifactInput>,
}
#[derive(Deserialize)]
pub struct UpdateTargetRequest {
pub name: Option<String>,
pub target_type: Option<TargetType>,
pub scan_config: Option<TargetScanConfig>,
pub compliance_profile: Option<ComplianceProfile>,
pub scan_schedule: Option<String>,
}
/// One applicable-scan option, serialized for the wizard.
#[derive(Serialize)]
pub struct ScanOptionDto {
pub scan: String,
pub default_on: bool,
pub rationale: String,
pub required_artifact: Option<String>,
pub blocked_reason: Option<String>,
}
#[derive(Serialize)]
pub struct ApplicableScansResponse {
pub scans: Vec<ScanOptionDto>,
pub pentest_supported: bool,
}
fn parse_oid(id: &str) -> Result<ObjectId, StatusCode> {
ObjectId::parse_str(id).map_err(|_| StatusCode::BAD_REQUEST)
}
/// GET /api/v1/targets — list onboarded targets (paginated).
#[tracing::instrument(skip_all)]
pub async fn list_targets(
Extension(agent): AgentExt,
tenant: TenantCtx,
Query(params): Query<PaginationParams>,
) -> Result<Json<ApiResponse<Vec<OnboardedTarget>>>, StatusCode> {
let db = tenant_db(&agent, &tenant).await?;
let skip = (params.page.saturating_sub(1)) * params.limit as u64;
let total = db
.onboarded_targets()
.count_documents(doc! {})
.await
.unwrap_or(0);
let targets = match db
.onboarded_targets()
.find(doc! {})
.skip(skip)
.limit(params.limit)
.await
{
Ok(cursor) => collect_cursor_async(cursor).await,
Err(e) => {
tracing::warn!("Failed to fetch onboarded targets: {e}");
Vec::new()
}
};
Ok(Json(ApiResponse {
data: targets,
total: Some(total),
page: Some(params.page),
}))
}
/// POST /api/v1/targets — create an onboarded target.
#[tracing::instrument(skip_all)]
pub async fn create_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Json(req): Json<CreateTargetRequest>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let mut target = OnboardedTarget::new(req.name, req.target_type);
target.description = req.description;
target.artifacts = req.artifacts.iter().map(ArtifactInput::build).collect();
let db = tenant_db(&agent, &tenant).await?;
let res = db
.onboarded_targets()
.insert_one(&target)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
target.id = res.inserted_id.as_object_id();
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// GET /api/v1/targets/{id} — fetch one target.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn get_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// PATCH /api/v1/targets/{id} — update mutable fields.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn update_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(req): Json<UpdateTargetRequest>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let mut set = doc! { "updated_at": mongodb::bson::DateTime::now() };
if let Some(name) = req.name {
set.insert("name", name);
}
if let Some(tt) = req.target_type {
set.insert(
"target_type",
to_bson(&tt).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(sc) = req.scan_config {
set.insert(
"scan_config",
to_bson(&sc).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(cp) = req.compliance_profile {
set.insert(
"compliance_profile",
to_bson(&cp).map_err(|_| StatusCode::BAD_REQUEST)?,
);
}
if let Some(ss) = req.scan_schedule {
set.insert("scan_schedule", ss);
}
db.onboarded_targets()
.update_one(doc! { "_id": oid }, doc! { "$set": set })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// DELETE /api/v1/targets/{id} — remove the target and its findings/scans.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn delete_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
db.onboarded_targets()
.delete_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
// Cascade the collections keyed by repo_id == target id (best-effort).
let by_repo = doc! { "repo_id": &id };
let _ = db.findings().delete_many(by_repo.clone()).await;
let _ = db.scan_runs().delete_many(by_repo.clone()).await;
let _ = db.sbom_entries().delete_many(by_repo.clone()).await;
let _ = db.cve_alerts().delete_many(by_repo).await;
Ok(Json(serde_json::json!({ "status": "deleted" })))
}
/// POST /api/v1/targets/{id}/artifacts — attach an artifact (by reference).
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn add_artifact(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
Json(input): Json<ArtifactInput>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let artifact = to_bson(&input.build()).map_err(|_| StatusCode::BAD_REQUEST)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$push": { "artifacts": artifact }, "$set": { "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
get_target(Extension(agent), tenant, Path(id)).await
}
/// GET /api/v1/targets/{id}/applicable-scans — the scan-applicability matrix.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn applicable_scans_for_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<ApplicableScansResponse>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
let scans = applicable_scans(&target)
.into_iter()
.map(|o| ScanOptionDto {
scan: o.scan.to_string(),
default_on: o.default_on,
rationale: o.rationale,
required_artifact: o.required_artifact.map(|k| k.to_string()),
blocked_reason: o.blocked_reason,
})
.collect();
Ok(Json(ApiResponse {
data: ApplicableScansResponse {
scans,
pentest_supported: supports_pentest(target.target_type),
},
total: None,
page: None,
}))
}
/// POST /api/v1/targets/{id}/detect — classify the target from its artifacts.
///
/// This is the lightweight pass: it classifies from artifact kinds without
/// ingesting (cloning) sources, so it returns immediately. Deep detection (after
/// ingest, with tramiton firmware analysis) is a follow-up background step.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn detect_target(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<ApiResponse<OnboardedTarget>>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
let mut target = db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.ok_or(StatusCode::NOT_FOUND)?;
// No ingested working paths here → kind-based classification only; the mock
// firmware detector is never invoked (no firmware working path present).
let empty = HashMap::new();
let detector = MockFirmwareDetector { detection: None };
let classification = classify_target(&target, &empty, &detector)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
let classification_bson =
to_bson(&classification).map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
db.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "classification": classification_bson, "updated_at": mongodb::bson::DateTime::now() } },
)
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
target.classification = Some(classification);
Ok(Json(ApiResponse {
data: target,
total: None,
page: None,
}))
}
/// POST /api/v1/targets/{id}/scan — trigger a scan for the target.
///
/// Dispatches to the unified pipeline when `UNIFIED_PIPELINE` is set (else the
/// legacy path). Runs in the background and returns immediately.
#[tracing::instrument(skip_all, fields(target_id = %id))]
pub async fn trigger_target_scan(
Extension(agent): AgentExt,
tenant: TenantCtx,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, StatusCode> {
let oid = parse_oid(&id)?;
let db = tenant_db(&agent, &tenant).await?;
// 404 if the target doesn't exist for this tenant.
if db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?
.is_none()
{
return Err(StatusCode::NOT_FOUND);
}
let agent_clone = (*agent).clone();
let tenant_id = tenant.0.tenant_id.clone();
tokio::spawn(async move {
// Always the unified target pipeline — this endpoint is about an
// onboarded target by construction, independent of the global
// `unified_pipeline` transition flag used by the legacy paths.
if let Err(e) = agent_clone
.run_target_scan(
&tenant_id,
&id,
compliance_core::models::ScanTrigger::Manual,
)
.await
{
tracing::error!("Manual target scan failed for {id}: {e}");
}
});
Ok(Json(serde_json::json!({ "status": "scan_triggered" })))
}
+27
View File
@@ -25,6 +25,33 @@ pub fn build_router() -> Router {
"/api/v1/repositories/{id}/webhook-config",
get(handlers::get_webhook_config),
)
// Unified onboarding targets (#131).
.route(
"/api/v1/targets",
get(handlers::onboarding::list_targets).post(handlers::onboarding::create_target),
)
.route(
"/api/v1/targets/{id}",
get(handlers::onboarding::get_target)
.patch(handlers::onboarding::update_target)
.delete(handlers::onboarding::delete_target),
)
.route(
"/api/v1/targets/{id}/artifacts",
post(handlers::onboarding::add_artifact),
)
.route(
"/api/v1/targets/{id}/applicable-scans",
get(handlers::onboarding::applicable_scans_for_target),
)
.route(
"/api/v1/targets/{id}/detect",
post(handlers::onboarding::detect_target),
)
.route(
"/api/v1/targets/{id}/scan",
post(handlers::onboarding::trigger_target_scan),
)
.route("/api/v1/findings", get(handlers::list_findings))
.route("/api/v1/findings/{id}", get(handlers::get_finding))
.route(
+75 -104
View File
@@ -1,55 +1,40 @@
//! Firmware classification via tramiton.
//!
//! tramiton is the company's firmware build/repro engine; we do not re-implement
//! its detection. This module shells out to `tramiton detect --json` behind a
//! [`FirmwareDetector`] port (so a future in-process or cloud impl can slot in)
//! and maps the resulting build plan onto a [`TargetType`]. A deterministic
//! [`MockFirmwareDetector`] backs the tests so CI never needs the binary.
//!
//! The parsed structs mirror a *subset* of tramiton's `BuildPlan` JSON — we
//! deliberately do not depend on the proprietary `tramiton-core` crate.
//! its detection. We depend on `tramiton-core` directly (same-company IP) and run
//! its provider analysis in-process behind a [`FirmwareDetector`] port, mapping
//! tramiton's `BuildPlan` onto a [`TargetType`]. A deterministic
//! [`MockFirmwareDetector`] backs the tests so CI unit tests need neither the
//! tramiton sources nor a real firmware tree.
use std::path::Path;
use serde::Deserialize;
use compliance_core::error::CoreError;
use compliance_core::models::{DetectedFact, TargetType};
use compliance_core::traits::ClassifierVerdict;
/// The top-level `tramiton detect --json` document (fields we use).
#[derive(Debug, Clone, Deserialize)]
pub struct TramitonDetect {
/// The selected build plan, if tramiton could form one.
#[serde(default)]
pub plan: Option<TramitonBuildPlan>,
}
/// The subset of tramiton's `BuildPlan` we consume for classification.
#[derive(Debug, Clone, Default, Deserialize)]
pub struct TramitonBuildPlan {
/// A minimal firmware-detection summary, mapped from tramiton's `BuildPlan`.
/// Kept small and tramiton-independent so the classifier and the test mock don't
/// need to construct a full tramiton plan.
#[derive(Debug, Clone, Default)]
pub struct FirmwareDetection {
/// The detecting provider (e.g. `zephyr`, `cmake`, `source-archaeology`).
#[serde(default)]
pub provider: String,
/// Detection confidence: `low` | `medium` | `high`.
#[serde(default)]
pub confidence: String,
/// Build system (kebab-case: `zephyr`, `esp-idf`, `cmake`, `make`, ...).
#[serde(default)]
/// Build-system label (e.g. `Zephyr`, `ESP-IDF`, `CMake`).
pub build_system: String,
/// Framework, when known (`zephyr`, `esp-idf`, `mbed`, `bare-metal`, ...).
/// Framework, when known (`zephyr`, `esp-idf`, `bare-metal`, ...).
pub framework: Option<String>,
/// Target board / MCU / arch.
#[serde(default)]
pub target: TramitonTarget,
pub target: FirmwareTarget,
/// Unresolved gaps in the plan.
#[serde(default)]
pub gaps: Vec<String>,
}
/// tramiton's target descriptor.
#[derive(Debug, Clone, Default, Deserialize)]
pub struct TramitonTarget {
/// The detected firmware target (board / MCU / arch).
#[derive(Debug, Clone, Default)]
pub struct FirmwareTarget {
/// Board name.
pub board: Option<String>,
/// MCU part.
@@ -61,57 +46,52 @@ pub struct TramitonTarget {
/// A source of tramiton firmware detection.
#[allow(async_fn_in_trait)]
pub trait FirmwareDetector: Send + Sync {
/// Run detection over a path, returning tramiton's build plan if any.
async fn detect(&self, path: &Path) -> Result<Option<TramitonBuildPlan>, CoreError>;
/// Run detection over a path, returning a firmware detection if tramiton
/// could form a build plan.
async fn detect(&self, path: &Path) -> Result<Option<FirmwareDetection>, CoreError>;
}
/// Shells out to the `tramiton` CLI. A missing binary or a non-zero exit is
/// treated as "no detection" rather than an error, so firmware classification
/// degrades gracefully when tramiton is not installed.
pub struct TramitonCli {
/// The `tramiton` binary to invoke.
pub bin: String,
}
/// Uses `tramiton-core` in-process. The analysis is blocking (filesystem walk),
/// so it runs on a blocking thread to avoid stalling the async runtime. A path
/// with no recognizable build system yields `Ok(None)`.
pub struct TramitonNative;
impl TramitonCli {
/// Construct from `TRAMITON_BIN` (default `tramiton`).
pub fn from_env() -> Self {
Self {
bin: std::env::var("TRAMITON_BIN").unwrap_or_else(|_| "tramiton".to_string()),
}
impl FirmwareDetector for TramitonNative {
async fn detect(&self, path: &Path) -> Result<Option<FirmwareDetection>, CoreError> {
let path = path.to_path_buf();
let plan = tokio::task::spawn_blocking(move || {
let repo = tramiton_core::Repo::new(&path);
tramiton_core::provider::analyze(&repo)
})
.await
.map_err(|e| CoreError::Other(format!("tramiton detect task join error: {e}")))?
.map_err(|e| CoreError::Other(format!("tramiton analyze error: {e}")))?;
Ok(plan.map(|bp| detection_from_build_plan(&bp)))
}
}
impl FirmwareDetector for TramitonCli {
async fn detect(&self, path: &Path) -> Result<Option<TramitonBuildPlan>, CoreError> {
let output = tokio::process::Command::new(&self.bin)
.arg("detect")
.arg("--json")
.arg(path)
.output()
.await;
match output {
Ok(o) if o.status.success() => {
let parsed: TramitonDetect = serde_json::from_slice(&o.stdout)?;
Ok(parsed.plan)
}
// Non-zero exit: tramiton ran but formed no plan.
Ok(_) => Ok(None),
// Binary not found / not executable: degrade gracefully.
Err(_) => Ok(None),
}
/// Map tramiton's `BuildPlan` onto our minimal detection summary.
fn detection_from_build_plan(bp: &tramiton_core::BuildPlan) -> FirmwareDetection {
FirmwareDetection {
provider: bp.provider.clone(),
confidence: bp.confidence.to_string(),
build_system: bp.build_system.label().to_string(),
framework: bp.framework.clone(),
target: FirmwareTarget {
board: bp.target.board.clone(),
mcu: bp.target.mcu.clone(),
arch: bp.target.arch.clone(),
},
gaps: bp.gaps.clone(),
}
}
/// Map a tramiton build plan to a target type. Framework/build-system signals
/// Map a firmware detection to a target type. Framework/build-system signals
/// distinguish RTOS from bare-metal from Yocto.
pub fn plan_to_target_type(plan: &TramitonBuildPlan) -> TargetType {
let framework = plan.framework.as_deref().unwrap_or("").to_lowercase();
let build_system = plan.build_system.to_lowercase();
let signal = format!(
"{framework} {build_system} {}",
plan.provider.to_lowercase()
);
pub fn detection_to_target_type(det: &FirmwareDetection) -> TargetType {
let framework = det.framework.as_deref().unwrap_or("").to_lowercase();
let build_system = det.build_system.to_lowercase();
let signal = format!("{framework} {build_system} {}", det.provider.to_lowercase());
const RTOS: [&str; 6] = ["zephyr", "esp-idf", "freertos", "nuttx", "riot", "chibios"];
if signal.contains("bitbake") || signal.contains("yocto") || signal.contains("openembedded") {
@@ -133,35 +113,35 @@ fn confidence_score(label: &str) -> f32 {
}
}
/// Turn a tramiton build plan into a classifier verdict, carrying the MCU / board
/// Turn a firmware detection into a classifier verdict, carrying the MCU / board
/// / build-system as facts.
pub fn plan_to_verdict(plan: &TramitonBuildPlan) -> ClassifierVerdict {
let target_type = plan_to_target_type(plan);
pub fn detection_to_verdict(det: &FirmwareDetection) -> ClassifierVerdict {
let target_type = detection_to_target_type(det);
let mut facts = vec![DetectedFact::new(
"build_system",
plan.build_system.clone(),
det.build_system.clone(),
"tramiton",
)];
if let Some(fw) = &plan.framework {
if let Some(fw) = &det.framework {
facts.push(DetectedFact::new("framework", fw.clone(), "tramiton"));
}
if let Some(mcu) = &plan.target.mcu {
if let Some(mcu) = &det.target.mcu {
facts.push(DetectedFact::new("mcu", mcu.clone(), "tramiton"));
}
if let Some(board) = &plan.target.board {
if let Some(board) = &det.target.board {
facts.push(DetectedFact::new("board", board.clone(), "tramiton"));
}
if let Some(arch) = &plan.target.arch {
if let Some(arch) = &det.target.arch {
facts.push(DetectedFact::new("arch", arch.clone(), "tramiton"));
}
ClassifierVerdict {
target_type,
confidence: confidence_score(&plan.confidence),
confidence: confidence_score(&det.confidence),
facts,
rationale: format!(
"tramiton detected build system '{}'{}",
plan.build_system,
plan.framework
det.build_system,
det.framework
.as_ref()
.map(|f| format!(" (framework {f})"))
.unwrap_or_default()
@@ -169,15 +149,15 @@ pub fn plan_to_verdict(plan: &TramitonBuildPlan) -> ClassifierVerdict {
}
}
/// A deterministic [`FirmwareDetector`] for tests — returns a preset plan.
/// A deterministic [`FirmwareDetector`] for tests — returns a preset detection.
pub struct MockFirmwareDetector {
/// The plan to return (or `None` for "no detection").
pub plan: Option<TramitonBuildPlan>,
/// The detection to return (or `None` for "no detection").
pub detection: Option<FirmwareDetection>,
}
impl FirmwareDetector for MockFirmwareDetector {
async fn detect(&self, _path: &Path) -> Result<Option<TramitonBuildPlan>, CoreError> {
Ok(self.plan.clone())
async fn detect(&self, _path: &Path) -> Result<Option<FirmwareDetection>, CoreError> {
Ok(self.detection.clone())
}
}
@@ -186,13 +166,13 @@ impl FirmwareDetector for MockFirmwareDetector {
mod tests {
use super::*;
fn plan(build_system: &str, framework: Option<&str>) -> TramitonBuildPlan {
TramitonBuildPlan {
fn detection(build_system: &str, framework: Option<&str>) -> FirmwareDetection {
FirmwareDetection {
provider: build_system.to_string(),
confidence: "high".to_string(),
build_system: build_system.to_string(),
framework: framework.map(|s| s.to_string()),
target: TramitonTarget {
target: FirmwareTarget {
mcu: Some("stm32f429".to_string()),
..Default::default()
},
@@ -203,7 +183,7 @@ mod tests {
#[test]
fn zephyr_maps_to_rtos() {
assert_eq!(
plan_to_target_type(&plan("zephyr", Some("zephyr"))),
detection_to_target_type(&detection("zephyr", Some("zephyr"))),
TargetType::FirmwareRtos
);
}
@@ -211,7 +191,7 @@ mod tests {
#[test]
fn bare_cmake_maps_to_bare_metal() {
assert_eq!(
plan_to_target_type(&plan("cmake", Some("bare-metal"))),
detection_to_target_type(&detection("cmake", Some("bare-metal"))),
TargetType::FirmwareBareMetal
);
}
@@ -219,14 +199,14 @@ mod tests {
#[test]
fn bitbake_maps_to_yocto() {
assert_eq!(
plan_to_target_type(&plan("bitbake", None)),
detection_to_target_type(&detection("bitbake", None)),
TargetType::EmbeddedLinuxYocto
);
}
#[test]
fn verdict_carries_mcu_fact_and_confidence() {
let v = plan_to_verdict(&plan("esp-idf", Some("esp-idf")));
let v = detection_to_verdict(&detection("esp-idf", Some("esp-idf")));
assert_eq!(v.target_type, TargetType::FirmwareRtos);
assert!((v.confidence - 0.9).abs() < f32::EPSILON);
assert!(v
@@ -234,13 +214,4 @@ mod tests {
.iter()
.any(|f| f.key == "mcu" && f.value == "stm32f429"));
}
#[test]
fn detect_json_parses() {
let json = r#"{"repo":"/x","detections":[],"plan":{"provider":"zephyr","confidence":"high","build_system":"zephyr","framework":"zephyr","target":{"mcu":"nrf52840","board":"nrf52840dk","arch":"arm"},"gaps":[]}}"#;
let parsed: TramitonDetect = serde_json::from_str(json).expect("parse");
let plan = parsed.plan.expect("plan present");
assert_eq!(plan.target.mcu.as_deref(), Some("nrf52840"));
assert_eq!(plan_to_target_type(&plan), TargetType::FirmwareRtos);
}
}
+7 -8
View File
@@ -9,8 +9,7 @@ mod firmware;
mod language;
pub use firmware::{
FirmwareDetector, MockFirmwareDetector, TramitonBuildPlan, TramitonCli, TramitonDetect,
TramitonTarget,
FirmwareDetection, FirmwareDetector, FirmwareTarget, MockFirmwareDetector, TramitonNative,
};
pub use language::HeuristicClassifier;
@@ -23,7 +22,7 @@ use compliance_core::models::{
};
use compliance_core::traits::{ClassificationInput, ClassifierVerdict, TargetClassifier};
use firmware::plan_to_verdict;
use firmware::detection_to_verdict;
/// Classify a target from its artifacts and their ingested working paths, using
/// the heuristic classifier plus the tramiton firmware detector. Verdicts are
@@ -60,8 +59,8 @@ pub async fn classify_target<D: FirmwareDetector>(
let Some(path) = working_paths.get(&artifact.id) else {
continue;
};
if let Some(plan) = firmware_detector.detect(path).await? {
verdicts.push(plan_to_verdict(&plan));
if let Some(detection) = firmware_detector.detect(path).await? {
verdicts.push(detection_to_verdict(&detection));
tramiton_used = true;
}
}
@@ -155,7 +154,7 @@ mod tests {
}
fn no_firmware() -> MockFirmwareDetector {
MockFirmwareDetector { plan: None }
MockFirmwareDetector { detection: None }
}
#[tokio::test]
@@ -190,12 +189,12 @@ mod tests {
target.artifacts.push(artifact);
let detector = MockFirmwareDetector {
plan: Some(TramitonBuildPlan {
detection: Some(FirmwareDetection {
provider: "zephyr".to_string(),
confidence: "high".to_string(),
build_system: "zephyr".to_string(),
framework: Some("zephyr".to_string()),
target: TramitonTarget {
target: FirmwareTarget {
mcu: Some("nrf52840".to_string()),
..Default::default()
},
+6
View File
@@ -47,6 +47,12 @@ pub fn load_config() -> Result<AgentConfig, AgentError> {
.unwrap_or_else(|| "/tmp/compliance-scanner/repos".to_string()),
artifact_store_base_path: env_var_opt("ARTIFACT_STORE_BASE_PATH")
.unwrap_or_else(|| "/data/compliance-scanner/artifacts".to_string()),
// Defaults ON: the unified onboarded-target pipeline is now the primary
// path (no legacy `repositories` data in production). Set
// `UNIFIED_PIPELINE=0` to fall back to the legacy repository pipeline.
unified_pipeline: env_var_opt("UNIFIED_PIPELINE")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(true),
ssh_key_path: env_var_opt("SSH_KEY_PATH")
.unwrap_or_else(|| "/data/compliance-scanner/ssh/id_ed25519".to_string()),
keycloak_url: env_var_opt("KEYCLOAK_URL"),
+24
View File
@@ -179,6 +179,23 @@ impl DatabasePool {
.collect())
}
/// Tenant ids for every provisioned tenant database, derived by stripping
/// the `<prefix>_` from the database names. Skips the admin database
/// (`<prefix>__admin`). Hash-fallback names (very long tenant_ids) are lost
/// at the cluster level and cannot be recovered here — in practice tenant
/// ids are UUIDs and never hit that path. Used by the migration CLI's
/// `--all` mode.
pub async fn list_tenant_ids(&self) -> Result<Vec<String>, AgentError> {
let prefix = format!("{}_", self.db_prefix);
Ok(self
.list_tenant_db_names()
.await?
.into_iter()
.filter_map(|n| n.strip_prefix(&prefix).map(str::to_string))
.filter(|id| !id.starts_with('_'))
.collect())
}
/// Drop the database for a specific tenant. Used by GDPR delete
/// and tenant offboarding. Idempotent — dropping a non-existent
/// database is a no-op at the driver level.
@@ -521,6 +538,13 @@ impl Database {
self.inner.collection("onboarded_targets")
}
/// A typed handle to an arbitrary collection by name. For bookkeeping
/// collections without a dedicated model (e.g. `schema_migrations`,
/// `onboarding_migration_log`).
pub fn collection_named<T: Send + Sync>(&self, name: &str) -> Collection<T> {
self.inner.collection(name)
}
pub fn dast_scan_runs(&self) -> Collection<DastScanRun> {
self.inner.collection("dast_scan_runs")
}
+1
View File
@@ -8,6 +8,7 @@ pub mod database;
pub mod error;
pub mod ingest;
pub mod llm;
pub mod migrate;
pub mod pentest;
pub mod pipeline;
pub mod rag;
+54 -1
View File
@@ -1,4 +1,50 @@
use compliance_agent::{agent, api, config, database, scheduler, ssh, webhooks};
use compliance_agent::{agent, api, config, database, migrate, scheduler, ssh, webhooks};
/// Run the `migrate onboarding` subcommand and exit. Backfills (or reverts) the
/// unified `onboarded_targets` collection per tenant.
///
/// Usage: `compliance-agent migrate onboarding [--all | --tenant <id>] [--dry-run] [--revert]`
async fn run_migration(
args: &[String],
pool: &database::DatabasePool,
) -> Result<(), compliance_agent::error::AgentError> {
if args.get(2).map(String::as_str) != Some("onboarding") {
eprintln!(
"usage: compliance-agent migrate onboarding [--all | --tenant <id>] [--dry-run] [--revert]"
);
std::process::exit(2);
}
let has = |flag: &str| args.iter().any(|a| a == flag);
let dry_run = has("--dry-run");
let revert = has("--revert");
let tenant = args
.iter()
.position(|a| a == "--tenant")
.and_then(|i| args.get(i + 1))
.cloned();
let tenants: Vec<String> = if has("--all") {
pool.list_tenant_ids().await?
} else if let Some(t) = tenant {
vec![t]
} else {
eprintln!("specify --all or --tenant <id>");
std::process::exit(2);
};
for tenant_id in tenants {
let db = pool.for_tenant_id(&tenant_id).await?;
if revert {
migrate::onboarding::revert(&db).await?;
println!("[{tenant_id}] reverted onboarding backfill");
} else {
let report = migrate::onboarding::backfill_onboarded_targets(&db, dry_run).await?;
let prefix = if dry_run { "(dry-run) " } else { "" };
println!("[{tenant_id}] {prefix}{report:?}");
}
}
Ok(())
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
@@ -31,6 +77,13 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
let db_pool =
database::DatabasePool::connect(&config.mongodb_uri, &config.mongodb_database).await?;
// One-shot subcommands run and exit without starting the servers.
let args: Vec<String> = std::env::args().collect();
if args.get(1).map(String::as_str) == Some("migrate") {
run_migration(&args, &db_pool).await?;
return Ok(());
}
let agent = agent::ComplianceAgent::new(config.clone(), db_pool);
tracing::info!("Starting scheduler...");
+8
View File
@@ -0,0 +1,8 @@
//! One-time data migrations.
//!
//! Currently just the onboarding backfill ([`onboarding`]), which folds the
//! legacy `repositories` and `dast_targets` collections into the unified
//! `onboarded_targets` collection, preserving `_id` so every downstream record
//! keyed by `repo_id` / `target_id` keeps resolving.
pub mod onboarding;
+406
View File
@@ -0,0 +1,406 @@
//! Backfill: legacy `repositories` + `dast_targets` → `onboarded_targets`.
//!
//! The transforms here are **id-preserving**: an [`OnboardedTarget`] keeps the
//! same `_id` as the `TrackedRepository` / `DastTarget` it came from, so every
//! downstream collection keyed by that hex id (findings, sbom, scan_runs,
//! graph, dast_*, pentest_*) keeps resolving with zero row rewrites, and
//! existing webhook URLs keep working. The mapping functions are pure and unit
//! tested; the DB orchestration (idempotent per-tenant backfill + revert) is a
//! thin driver over them.
use compliance_core::models::{
Artifact, ArtifactKind, DastTarget, DastTargetType, GitArtifactConfig, IssueTrackerConfig,
OnboardedTarget, TargetType, TrackedRepository, WebArtifactConfig,
};
use futures_util::TryStreamExt;
use mongodb::bson::{doc, Document};
use crate::database::Database;
use crate::error::AgentError;
/// Marker id in `schema_migrations` recording that the backfill has run.
const MIGRATION_MARKER: &str = "onboarding_backfill_v1";
/// Summary of a backfill run.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct MigrationReport {
/// Repositories turned into onboarded targets.
pub repos_migrated: u64,
/// DAST targets folded into an existing (repo-linked) target as a LiveUrl.
pub dast_targets_folded: u64,
/// DAST targets with no repo link, migrated as standalone targets.
pub dast_targets_standalone: u64,
/// Records skipped because a target with that `_id` already existed.
pub skipped_existing: u64,
}
/// Map a legacy `DastTargetType` to a unified [`TargetType`]. REST/GraphQL APIs
/// are backend services; a browser app is a web app.
fn target_type_for_dast(kind: &DastTargetType) -> TargetType {
match kind {
DastTargetType::WebApp => TargetType::WebApp,
DastTargetType::RestApi | DastTargetType::GraphQl => TargetType::BackendService,
}
}
/// Build the LiveUrl artifact for a DAST target (its base URL + crawl config +
/// auth). Shared by fold-in and standalone migration.
pub fn dast_to_artifact(dast: &DastTarget) -> Artifact {
let mut artifact = Artifact::live_url(dast.base_url.clone());
artifact.web = Some(WebArtifactConfig {
target_kind: dast.target_type.clone(),
excluded_paths: dast.excluded_paths.clone(),
max_crawl_depth: dast.max_crawl_depth,
rate_limit: dast.rate_limit,
allow_destructive: dast.allow_destructive,
});
artifact.auth = dast.auth_config.clone().map(Into::into);
artifact
}
/// Map a `TrackedRepository` to an onboarded target, preserving `_id`. The git
/// remote becomes a `GitRepo` artifact carrying the repo's branch, watermark,
/// and auth; tracker config folds into `scan_config`.
///
/// `target_type` is a safe default (`BackendService`) — the classifier can
/// refine it later; `classification` is left `None` (unconfirmed).
pub fn repo_to_target(repo: &TrackedRepository) -> OnboardedTarget {
let mut target = OnboardedTarget::new(repo.name.clone(), TargetType::BackendService);
target.id = repo.id;
let mut artifact = Artifact::git_repo(repo.git_url.clone(), repo.default_branch.clone());
artifact.git = Some(GitArtifactConfig {
default_branch: repo.default_branch.clone(),
last_scanned_commit: repo.last_scanned_commit.clone(),
local_path: repo.local_path.clone(),
});
if repo.auth_token.is_some() || repo.auth_username.is_some() {
artifact.auth = Some(compliance_core::models::ArtifactAuth {
method: "token".to_string(),
username: repo.auth_username.clone(),
secret: repo.auth_token.clone(),
..Default::default()
});
}
target.artifacts.push(artifact);
if repo.tracker_type.is_some() {
target.scan_config.issue_tracker = Some(IssueTrackerConfig {
tracker_type: repo.tracker_type.clone(),
owner: repo.tracker_owner.clone(),
repo: repo.tracker_repo.clone(),
token: repo.tracker_token.clone(),
});
}
target.scan_schedule = repo.scan_schedule.clone();
target.webhook_enabled = repo.webhook_enabled;
target.webhook_secret = repo.webhook_secret.clone();
target.findings_count = repo.findings_count;
target.created_at = repo.created_at;
target.updated_at = repo.updated_at;
target
}
/// Append a DAST target's LiveUrl artifact onto an existing (repo-derived)
/// target. If the repo default was `BackendService` but the DAST target is a
/// browser web app, promote the type to `WebApp`.
pub fn fold_dast_into_target(target: &mut OnboardedTarget, dast: &DastTarget) {
if matches!(dast.target_type, DastTargetType::WebApp)
&& target.target_type == TargetType::BackendService
{
target.target_type = TargetType::WebApp;
}
if !target.has(ArtifactKind::LiveUrl) {
target.artifacts.push(dast_to_artifact(dast));
}
}
/// Map a repo-less DAST target to a standalone onboarded target, preserving `_id`.
pub fn dast_to_standalone_target(dast: &DastTarget) -> OnboardedTarget {
let mut target =
OnboardedTarget::new(dast.name.clone(), target_type_for_dast(&dast.target_type));
target.id = dast.id;
target.artifacts.push(dast_to_artifact(dast));
target.created_at = dast.created_at;
target.updated_at = dast.updated_at;
target
}
/// Whether the onboarding backfill has already been applied to this database.
pub async fn already_applied(db: &Database) -> Result<bool, AgentError> {
let found = db
.collection_named::<Document>("schema_migrations")
.find_one(doc! { "_id": MIGRATION_MARKER })
.await?;
Ok(found.is_some())
}
/// Backfill `onboarded_targets` from `repositories` + `dast_targets` for one
/// tenant database.
///
/// Id-preserving and **idempotent**: targets that already exist (by `_id`) are
/// skipped, so re-running is safe. With `dry_run`, computes the report without
/// writing. The legacy collections are never deleted; the only mutation outside
/// `onboarded_targets` is the history relink of folded DAST targets, which is
/// logged so [`revert`] can undo it.
pub async fn backfill_onboarded_targets(
db: &Database,
dry_run: bool,
) -> Result<MigrationReport, AgentError> {
let mut report = MigrationReport::default();
// 1. repositories -> onboarded_targets (preserve _id, skip existing).
let mut repos = db.repositories().find(doc! {}).await?;
while let Some(repo) = repos.try_next().await? {
let Some(id) = repo.id else { continue };
if db
.onboarded_targets()
.find_one(doc! { "_id": id })
.await?
.is_some()
{
report.skipped_existing += 1;
continue;
}
if !dry_run {
db.onboarded_targets()
.insert_one(repo_to_target(&repo))
.await?;
}
report.repos_migrated += 1;
}
// 2. dast_targets -> fold into the linked repo target, or migrate standalone.
let mut dasts = db.dast_targets().find(doc! {}).await?;
while let Some(dast) = dasts.try_next().await? {
let Some(dast_id) = dast.id else { continue };
let repo_oid = dast
.repo_id
.as_deref()
.and_then(|r| mongodb::bson::oid::ObjectId::parse_str(r).ok());
let linked = match repo_oid {
Some(oid) => db.onboarded_targets().find_one(doc! { "_id": oid }).await?,
None => None,
};
match (linked, repo_oid) {
// Fold into an existing repo-derived target.
(Some(mut target), Some(oid)) => {
if target.has(ArtifactKind::LiveUrl) {
report.skipped_existing += 1; // already folded on a prior run
continue;
}
fold_dast_into_target(&mut target, &dast);
if !dry_run {
db.onboarded_targets()
.replace_one(doc! { "_id": oid }, &target)
.await?;
relink_history(db, &dast_id.to_hex(), &oid.to_hex()).await?;
}
report.dast_targets_folded += 1;
}
// No linked repo target: migrate as a standalone target (keeps _id).
_ => {
if db
.onboarded_targets()
.find_one(doc! { "_id": dast_id })
.await?
.is_some()
{
report.skipped_existing += 1;
continue;
}
if !dry_run {
db.onboarded_targets()
.insert_one(dast_to_standalone_target(&dast))
.await?;
}
report.dast_targets_standalone += 1;
}
}
}
if !dry_run {
db.collection_named::<Document>("schema_migrations")
.update_one(
doc! { "_id": MIGRATION_MARKER },
doc! { "$set": { "applied_at": mongodb::bson::DateTime::now() } },
)
.upsert(true)
.await?;
}
Ok(report)
}
/// Relink DAST scan runs and pentest sessions from the old DAST target id to the
/// unified target id, logging each move so [`revert`] can undo it.
///
/// Note: if multiple DAST targets fold into the same repo target, revert
/// restores only the last-logged mapping — a rare edge. The source collections
/// (`repositories`, `dast_targets`) are never deleted, so no data is lost.
async fn relink_history(db: &Database, old_id: &str, new_id: &str) -> Result<(), AgentError> {
db.dast_scan_runs()
.update_many(
doc! { "target_id": old_id },
doc! { "$set": { "target_id": new_id } },
)
.await?;
db.pentest_sessions()
.update_many(
doc! { "target_id": old_id },
doc! { "$set": { "target_id": new_id } },
)
.await?;
db.collection_named::<Document>("onboarding_migration_log")
.insert_one(doc! { "old_target_id": old_id, "new_target_id": new_id })
.await?;
Ok(())
}
/// Undo the backfill: replay the relink log in reverse, drop `onboarded_targets`
/// and the log, and clear the marker. The legacy collections are untouched, so
/// this restores the pre-migration state.
pub async fn revert(db: &Database) -> Result<(), AgentError> {
let log = db.collection_named::<Document>("onboarding_migration_log");
let mut cursor = log.find(doc! {}).await?;
while let Some(entry) = cursor.try_next().await? {
if let (Ok(old), Ok(new)) = (
entry.get_str("old_target_id"),
entry.get_str("new_target_id"),
) {
db.dast_scan_runs()
.update_many(
doc! { "target_id": new },
doc! { "$set": { "target_id": old } },
)
.await?;
db.pentest_sessions()
.update_many(
doc! { "target_id": new },
doc! { "$set": { "target_id": old } },
)
.await?;
}
}
db.onboarded_targets().drop().await?;
log.drop().await?;
db.collection_named::<Document>("schema_migrations")
.delete_one(doc! { "_id": MIGRATION_MARKER })
.await?;
Ok(())
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{DastAuthConfig, TrackerType};
fn repo() -> TrackedRepository {
let mut r = TrackedRepository::new("acme".to_string(), "https://git/acme.git".to_string());
r.id = Some(mongodb::bson::oid::ObjectId::new());
r.default_branch = "develop".to_string();
r.last_scanned_commit = Some("abc123".to_string());
r.auth_token = Some("pat".to_string());
r.auth_username = Some("bob".to_string());
r.tracker_type = Some(TrackerType::Gitea);
r.tracker_owner = Some("acme".to_string());
r.findings_count = 7;
r
}
fn dast(repo_id: Option<String>, kind: DastTargetType) -> DastTarget {
let mut d = DastTarget::new(
"acme-web".to_string(),
"https://acme.example.com".to_string(),
kind,
);
d.id = Some(mongodb::bson::oid::ObjectId::new());
d.repo_id = repo_id;
d.max_crawl_depth = 5;
d.auth_config = Some(DastAuthConfig {
method: "bearer".to_string(),
login_url: None,
username: None,
password: None,
token: Some("tok".to_string()),
headers: None,
});
d
}
#[test]
fn repo_maps_preserving_id_and_git_artifact() {
let r = repo();
let t = repo_to_target(&r);
assert_eq!(t.id, r.id); // id preserved
assert_eq!(t.findings_count, 7);
assert_eq!(t.scan_schedule, r.scan_schedule);
let git = t.code_artifact().expect("git artifact");
assert_eq!(git.kind, ArtifactKind::GitRepo);
assert_eq!(git.source_ref, "https://git/acme.git");
let gc = git.git.as_ref().expect("git config");
assert_eq!(gc.default_branch, "develop");
assert_eq!(gc.last_scanned_commit.as_deref(), Some("abc123"));
let auth = git.auth.as_ref().expect("auth");
assert_eq!(auth.secret.as_deref(), Some("pat"));
assert_eq!(auth.username.as_deref(), Some("bob"));
assert_eq!(
t.scan_config
.issue_tracker
.as_ref()
.and_then(|it| it.tracker_type.clone()),
Some(TrackerType::Gitea)
);
}
#[test]
fn standalone_dast_maps_preserving_id_and_live_url() {
let d = dast(None, DastTargetType::WebApp);
let t = dast_to_standalone_target(&d);
assert_eq!(t.id, d.id);
assert_eq!(t.target_type, TargetType::WebApp);
let url = t.live_url().expect("live url");
assert_eq!(url.source_ref, "https://acme.example.com");
let web = url.web.as_ref().expect("web config");
assert_eq!(web.max_crawl_depth, 5);
assert_eq!(
url.auth.as_ref().and_then(|a| a.secret.clone()),
Some("tok".to_string())
);
}
#[test]
fn rest_api_dast_maps_to_backend_service() {
let d = dast(None, DastTargetType::RestApi);
assert_eq!(
dast_to_standalone_target(&d).target_type,
TargetType::BackendService
);
}
#[test]
fn fold_adds_live_url_and_promotes_webapp() {
let mut t = repo_to_target(&repo());
assert_eq!(t.target_type, TargetType::BackendService);
fold_dast_into_target(&mut t, &dast(Some("x".to_string()), DastTargetType::WebApp));
assert_eq!(t.target_type, TargetType::WebApp); // promoted
assert!(t.has(ArtifactKind::LiveUrl));
assert!(t.has(ArtifactKind::GitRepo));
}
#[test]
fn fold_is_idempotent_on_live_url() {
let mut t = repo_to_target(&repo());
let d = dast(Some("x".to_string()), DastTargetType::WebApp);
fold_dast_into_target(&mut t, &d);
fold_dast_into_target(&mut t, &d);
let live_urls = t
.artifacts
.iter()
.filter(|a| a.kind == ArtifactKind::LiveUrl)
.count();
assert_eq!(live_urls, 1);
}
}
+1
View File
@@ -342,6 +342,7 @@ mod tests {
pentest_imap_password: None,
admin_api_token: None,
tenant_registry_url: None,
unified_pipeline: false,
}
}
+1
View File
@@ -8,6 +8,7 @@ mod issue_creation;
pub mod lint;
pub mod orchestrator;
pub mod patterns;
pub mod plan;
mod pr_review;
pub mod sbom;
pub mod semgrep;
@@ -15,6 +15,7 @@ use crate::pipeline::git::GitOps;
use crate::pipeline::gitleaks::GitleaksScanner;
use crate::pipeline::lint::LintScanner;
use crate::pipeline::patterns::{GdprPatternScanner, OAuthPatternScanner};
use crate::pipeline::plan::build_scan_plan;
use crate::pipeline::sbom::SbomScanner;
use crate::pipeline::semgrep::SemgrepScanner;
@@ -419,6 +420,279 @@ impl PipelineOrchestrator {
Ok(new_count)
}
/// Unified entry point (behind `UNIFIED_PIPELINE`): run a scan for an
/// `OnboardedTarget`. Mirrors [`Self::run`] but sources the target from
/// `onboarded_targets` and dispatches by the scan plan.
#[tracing::instrument(skip_all, fields(target_id = %target_id, trigger = ?trigger))]
pub async fn run_target(
&self,
target_id: &str,
trigger: ScanTrigger,
) -> Result<(), AgentError> {
let oid = mongodb::bson::oid::ObjectId::parse_str(target_id)
.map_err(|e| AgentError::Other(e.to_string()))?;
let target = self
.db
.onboarded_targets()
.find_one(doc! { "_id": oid })
.await?
.ok_or_else(|| AgentError::Other(format!("Onboarded target {target_id} not found")))?;
let scan_run = ScanRun::new(target_id.to_string(), trigger);
let insert = self.db.scan_runs().insert_one(&scan_run).await?;
let scan_run_id = insert
.inserted_id
.as_object_id()
.map(|id| id.to_hex())
.unwrap_or_default();
let result = self.run_target_pipeline(&target, &scan_run_id).await;
match &result {
Ok(count) => {
self.db
.scan_runs()
.update_one(
doc! { "_id": &insert.inserted_id },
doc! { "$set": {
"status": "completed",
"current_phase": "completed",
"new_findings_count": *count as i64,
"completed_at": mongodb::bson::DateTime::now(),
} },
)
.await?;
// Refresh the target's cached findings count. The shared pipeline
// (Stage 7) increments `repositories`, which the unified path does
// not use, so set the accurate total on the target itself.
let total = self
.db
.findings()
.count_documents(doc! { "repo_id": target_id })
.await
.unwrap_or(*count as u64);
self.db
.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$set": {
"findings_count": total as i64,
"updated_at": mongodb::bson::DateTime::now(),
} },
)
.await?;
}
Err(e) => {
tracing::error!(target_id, error = %e, "Unified scan pipeline failed");
self.db
.scan_runs()
.update_one(
doc! { "_id": &insert.inserted_id },
doc! { "$set": {
"status": "failed",
"error_message": e.to_string(),
"completed_at": mongodb::bson::DateTime::now(),
} },
)
.await?;
}
}
result.map(|_| ())
}
/// Run the applicable scans for a target. For a code target this reuses the
/// full legacy pipeline over the code artifact (clone → SAST umbrella →
/// triage → persist → issues → DAST); firmware/PLC/mobile scanners are
/// follow-ups (#128/#129/#130). Returns the number of new findings.
async fn run_target_pipeline(
&self,
target: &OnboardedTarget,
scan_run_id: &str,
) -> Result<u32, AgentError> {
let target_id = target.id.map(|id| id.to_hex()).unwrap_or_default();
let plan = build_scan_plan(target);
tracing::info!(
target_id = %target_id,
target_type = %target.target_type,
planned_steps = plan.steps.len(),
"Unified pipeline: scan plan built"
);
// Ingest + classify (tramiton for firmware) and store the detected type.
self.classify_and_store(target, &target_id, scan_run_id)
.await;
// Provision a DAST target from a LiveUrl artifact so DAST fires for
// wizard-created targets, not just migrated ones.
self.ensure_dast_target(target, &plan).await;
match target.code_artifact() {
Some(code) if code.kind == ArtifactKind::GitRepo => {
let repo = repo_view_from_target(target, code);
let new_count = self.run_pipeline(&repo, scan_run_id).await?;
self.finalize_target(target, &repo, new_count).await?;
Ok(new_count)
}
Some(_) => {
tracing::warn!(
target_id = %target_id,
"Unified pipeline: source-archive scanning not yet wired; skipping"
);
Ok(0)
}
None => {
// No code to scan. Firmware/PLC/mobile static scanners land in
// #128/#129/#130; DAST for a running URL still works when a
// DastTarget row exists (migrated targets).
tracing::info!(
target_id = %target_id,
"Unified pipeline: no code artifact; attempting DAST only"
);
self.update_phase(scan_run_id, "dast_scanning").await;
self.maybe_trigger_dast(&target_id, scan_run_id).await;
Ok(0)
}
}
}
/// Ingest the target's artifacts, classify (tramiton for firmware/RTOS/Yocto,
/// heuristics otherwise), and store the detected classification on the target.
/// Best-effort — never fails the scan.
async fn classify_and_store(
&self,
target: &OnboardedTarget,
target_id: &str,
scan_run_id: &str,
) {
self.update_phase(scan_run_id, "classification").await;
let ctx = crate::ingest::IngestContext::from_config(&self.config, target_id);
let ingest_set = match crate::ingest::ingest_all(target, &ctx) {
Ok(set) => set,
Err(e) => {
tracing::warn!(target_id, error = %e, "Unified pipeline: ingest for classification failed");
return;
}
};
let working_paths = ingest_set.working_paths();
match crate::classify::classify_target(
target,
&working_paths,
&crate::classify::TramitonNative,
)
.await
{
Ok(classification) => {
tracing::info!(
target_id,
suggested = %classification.suggested,
"Unified pipeline: classified target"
);
if let (Some(oid), Ok(bson)) = (target.id, mongodb::bson::to_bson(&classification))
{
let _ = self
.db
.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! { "$set": { "classification": bson } },
)
.await;
}
}
Err(e) => {
tracing::warn!(target_id, error = %e, "Unified pipeline: classification failed")
}
}
}
/// If the target has a `LiveUrl` artifact and DAST is planned, provision a
/// `DastTarget` (keyed by `repo_id` = target id) so the existing DAST trigger
/// fires for wizard-created targets. Idempotent.
async fn ensure_dast_target(
&self,
target: &OnboardedTarget,
plan: &crate::pipeline::plan::ScanPlan,
) {
if !plan.has(ScanType::Dast) {
return;
}
let (Some(url), Some(oid)) = (target.live_url(), target.id) else {
return;
};
let target_id = oid.to_hex();
if self
.db
.dast_targets()
.find_one(doc! { "repo_id": &target_id })
.await
.ok()
.flatten()
.is_some()
{
return; // already provisioned
}
let kind = url
.web
.as_ref()
.map(|w| w.target_kind.clone())
.unwrap_or(DastTargetType::WebApp);
let mut dast = DastTarget::new(target.name.clone(), url.source_ref.clone(), kind);
dast.repo_id = Some(target_id);
if let Some(web) = &url.web {
dast.excluded_paths = web.excluded_paths.clone();
dast.max_crawl_depth = web.max_crawl_depth;
dast.rate_limit = web.rate_limit;
dast.allow_destructive = web.allow_destructive;
}
if let Some(auth) = &url.auth {
dast.auth_config = Some(DastAuthConfig {
method: auth.method.clone(),
login_url: auth.login_url.clone(),
username: auth.username.clone(),
password: None,
token: auth.secret.clone(),
headers: auth.headers.clone(),
});
}
if let Err(e) = self.db.dast_targets().insert_one(&dast).await {
tracing::warn!(error = %e, "Unified pipeline: failed to provision DAST target");
}
}
/// Sync the onboarded-target document after a scan: bump `findings_count`
/// and advance the git artifact's `last_scanned_commit` watermark.
async fn finalize_target(
&self,
target: &OnboardedTarget,
repo: &TrackedRepository,
new_count: u32,
) -> Result<(), AgentError> {
let oid = match target.id {
Some(id) => id,
None => return Ok(()),
};
self.db
.onboarded_targets()
.update_one(
doc! { "_id": oid },
doc! {
"$inc": { "findings_count": new_count as i64 },
"$set": { "updated_at": mongodb::bson::DateTime::now() },
},
)
.await?;
let repo_path = std::path::Path::new(&self.config.git_clone_base_path).join(&repo.name);
if let (Ok(sha), Some(code)) = (GitOps::get_head_sha(&repo_path), target.code_artifact()) {
self.db
.onboarded_targets()
.update_one(
doc! { "_id": oid, "artifacts.id": &code.id },
doc! { "$set": { "artifacts.$.git.last_scanned_commit": sha } },
)
.await?;
}
Ok(())
}
pub(super) async fn update_phase(&self, scan_run_id: &str, phase: &str) {
if let Ok(oid) = mongodb::bson::oid::ObjectId::parse_str(scan_run_id) {
let _ = self
@@ -436,6 +710,37 @@ impl PipelineOrchestrator {
}
}
/// Build a legacy `TrackedRepository` view from an onboarded target's code
/// artifact, so the unified pipeline can reuse the existing repo pipeline. The
/// inverse of the migration's `repo_to_target`. `_id` is preserved so findings
/// and DAST lookups resolve against the same key.
fn repo_view_from_target(target: &OnboardedTarget, code: &Artifact) -> TrackedRepository {
let mut repo = TrackedRepository::new(target.name.clone(), code.source_ref.clone());
repo.id = target.id;
if let Some(git) = &code.git {
repo.default_branch = git.default_branch.clone();
repo.last_scanned_commit = git.last_scanned_commit.clone();
repo.local_path = git.local_path.clone();
}
if let Some(auth) = &code.auth {
repo.auth_token = auth.secret.clone();
repo.auth_username = auth.username.clone();
}
if let Some(it) = &target.scan_config.issue_tracker {
repo.tracker_type = it.tracker_type.clone();
repo.tracker_owner = it.owner.clone();
repo.tracker_repo = it.repo.clone();
repo.tracker_token = it.token.clone();
}
repo.scan_schedule = target.scan_schedule.clone();
repo.webhook_enabled = target.webhook_enabled;
repo.webhook_secret = target.webhook_secret.clone();
repo.findings_count = target.findings_count;
repo.created_at = target.created_at;
repo.updated_at = target.updated_at;
repo
}
/// Extract the scheme + host from a git URL.
/// e.g. "https://gitea.example.com/owner/repo.git" -> "https://gitea.example.com"
/// e.g. "ssh://git@gitea.example.com:22/owner/repo.git" -> "https://gitea.example.com"
@@ -460,3 +765,50 @@ pub(super) fn extract_base_url(git_url: &str) -> Option<String> {
None
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{
Artifact, ArtifactAuth, IssueTrackerConfig, TargetType, TrackerType,
};
#[test]
fn repo_view_preserves_id_git_auth_and_tracker() {
let mut target = OnboardedTarget::new("acme".to_string(), TargetType::WebApp);
target.id = Some(mongodb::bson::oid::ObjectId::new());
target.findings_count = 3;
target.scan_config.issue_tracker = Some(IssueTrackerConfig {
tracker_type: Some(TrackerType::Gitea),
owner: Some("acme".to_string()),
repo: Some("web".to_string()),
token: Some("tt".to_string()),
});
let mut artifact = Artifact::git_repo("https://git/acme.git", "develop");
if let Some(git) = artifact.git.as_mut() {
git.last_scanned_commit = Some("abc123".to_string());
}
artifact.auth = Some(ArtifactAuth {
method: "token".to_string(),
username: Some("bob".to_string()),
secret: Some("pat".to_string()),
..Default::default()
});
target.artifacts.push(artifact);
let code = target.code_artifact().expect("code artifact");
let repo = repo_view_from_target(&target, code);
assert_eq!(repo.id, target.id); // preserved
assert_eq!(repo.git_url, "https://git/acme.git");
assert_eq!(repo.default_branch, "develop");
assert_eq!(repo.last_scanned_commit.as_deref(), Some("abc123"));
assert_eq!(repo.auth_token.as_deref(), Some("pat"));
assert_eq!(repo.auth_username.as_deref(), Some("bob"));
assert_eq!(repo.tracker_type, Some(TrackerType::Gitea));
assert_eq!(repo.tracker_owner.as_deref(), Some("acme"));
assert_eq!(repo.findings_count, 3);
}
}
+195
View File
@@ -0,0 +1,195 @@
//! The scan plan.
//!
//! [`build_scan_plan`] turns an [`OnboardedTarget`] into the concrete ordered
//! list of scans to run, each bound to the artifact it consumes. It intersects
//! the scan-applicability matrix ([`applicable_scans`]) with the target's
//! `scan_config` overrides: a scan runs when its required artifact is present and
//! it is either on by default or explicitly enabled, and is not explicitly
//! disabled. This is the decision engine the unified pipeline (`run_target`)
//! executes.
use compliance_core::models::{Artifact, ArtifactKind, OnboardedTarget, ScanPhase, ScanType};
use compliance_core::scan_matrix::applicable_scans;
/// One scan to run, bound to the artifact it operates on.
#[derive(Debug, Clone, PartialEq)]
pub struct ScanStep {
/// The scan to run.
pub scan_type: ScanType,
/// The pipeline phase to report while it runs.
pub phase: ScanPhase,
/// The id of the artifact this scan consumes ([`Artifact::id`]).
pub artifact_id: String,
}
/// The ordered set of scans to run for a target.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct ScanPlan {
/// The scans, in matrix order.
pub steps: Vec<ScanStep>,
}
impl ScanPlan {
/// Whether the plan contains a step for the given scan type.
pub fn has(&self, scan: ScanType) -> bool {
self.steps.iter().any(|s| s.scan_type == scan)
}
/// Whether the plan is empty (nothing to run).
pub fn is_empty(&self) -> bool {
self.steps.is_empty()
}
}
/// Build the scan plan for a target: matrix defaults ∩ `scan_config`, each scan
/// bound to the artifact it consumes. Scans whose required artifact is absent, or
/// that are disabled, or off-by-default and not explicitly enabled, are dropped.
pub fn build_scan_plan(target: &OnboardedTarget) -> ScanPlan {
let enabled = &target.scan_config.enabled_scans;
let disabled = &target.scan_config.disabled_scans;
let mut steps = Vec::new();
for option in applicable_scans(target) {
// Required artifact missing → not runnable.
if option.blocked_reason.is_some() {
continue;
}
// Explicit opt-out wins.
if disabled.contains(&option.scan) {
continue;
}
// Run if on by default, or explicitly enabled.
if !option.default_on && !enabled.contains(&option.scan) {
continue;
}
let Some(artifact) = resolve_artifact(target, option.required_artifact) else {
continue;
};
steps.push(ScanStep {
scan_type: option.scan,
phase: phase_for(option.scan),
artifact_id: artifact.id.clone(),
});
}
ScanPlan { steps }
}
/// Resolve the artifact a scan consumes. A "code" requirement (represented by
/// `GitRepo`) is satisfied by a git repo *or* a source archive.
fn resolve_artifact(target: &OnboardedTarget, required: Option<ArtifactKind>) -> Option<&Artifact> {
match required {
Some(ArtifactKind::GitRepo) => target.code_artifact(),
Some(kind) => target.first_of(kind),
None => target.code_artifact().or_else(|| target.artifacts.first()),
}
}
/// The pipeline phase reported while a given scan runs.
fn phase_for(scan: ScanType) -> ScanPhase {
match scan {
ScanType::Sast => ScanPhase::Sast,
ScanType::Sbom => ScanPhase::SbomGeneration,
ScanType::Cve => ScanPhase::CveScanning,
ScanType::Gdpr | ScanType::OAuth => ScanPhase::PatternScanning,
ScanType::SecretDetection => ScanPhase::SecretDetection,
ScanType::Lint => ScanPhase::LintScanning,
ScanType::CodeReview => ScanPhase::CodeReview,
ScanType::Graph => ScanPhase::GraphBuilding,
ScanType::Dast => ScanPhase::DastScanning,
ScanType::FirmwareStatic => ScanPhase::FirmwareStatic,
ScanType::PlcControlLogic => ScanPhase::PlcAnalysis,
ScanType::MobileStatic => ScanPhase::MobileStatic,
ScanType::ContainerScan => ScanPhase::ContainerScan,
}
}
#[cfg(test)]
#[allow(clippy::expect_used, clippy::unwrap_used)]
mod tests {
use super::*;
use compliance_core::models::{PlcFormat, TargetType};
fn target(target_type: TargetType, artifacts: Vec<Artifact>) -> OnboardedTarget {
let mut t = OnboardedTarget::new("t".to_string(), target_type);
t.artifacts = artifacts;
t
}
fn step_for<'a>(plan: &'a ScanPlan, scan: ScanType) -> Option<&'a ScanStep> {
plan.steps.iter().find(|s| s.scan_type == scan)
}
#[test]
fn webapp_with_code_and_url_runs_sast_and_dast_bound_to_the_right_artifacts() {
let code = Artifact::git_repo("https://git/x", "main");
let url = Artifact::live_url("https://x");
let (code_id, url_id) = (code.id.clone(), url.id.clone());
let t = target(TargetType::WebApp, vec![code, url]);
let plan = build_scan_plan(&t);
let sast = step_for(&plan, ScanType::Sast).expect("sast planned");
assert_eq!(sast.artifact_id, code_id);
assert_eq!(sast.phase, ScanPhase::Sast);
let dast = step_for(&plan, ScanType::Dast).expect("dast planned");
assert_eq!(dast.artifact_id, url_id);
}
#[test]
fn webapp_without_url_omits_dast() {
let t = target(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
let plan = build_scan_plan(&t);
assert!(plan.has(ScanType::Sast));
assert!(!plan.has(ScanType::Dast));
}
#[test]
fn code_scan_binds_to_source_archive_when_no_git_repo() {
let arc = Artifact::source_archive("src.zip");
let arc_id = arc.id.clone();
let t = target(TargetType::BackendService, vec![arc]);
let plan = build_scan_plan(&t);
let sast = step_for(&plan, ScanType::Sast).expect("sast planned");
assert_eq!(sast.artifact_id, arc_id);
}
#[test]
fn firmware_sbom_and_cve_bind_to_the_firmware_image() {
let fw = Artifact::firmware_image("fw.bin");
let fw_id = fw.id.clone();
let t = target(TargetType::FirmwareBareMetal, vec![fw]);
let plan = build_scan_plan(&t);
let sbom = step_for(&plan, ScanType::Sbom).expect("sbom planned");
assert_eq!(sbom.artifact_id, fw_id);
assert!(step_for(&plan, ScanType::FirmwareStatic).is_some());
assert!(!plan.has(ScanType::Dast));
}
#[test]
fn plc_plans_only_control_logic() {
let plc = Artifact::plc_project("p.xml", PlcFormat::PlcopenXml);
let t = target(TargetType::PlcSps, vec![plc]);
let plan = build_scan_plan(&t);
assert_eq!(plan.steps.len(), 1);
assert_eq!(plan.steps[0].scan_type, ScanType::PlcControlLogic);
assert_eq!(plan.steps[0].phase, ScanPhase::PlcAnalysis);
}
#[test]
fn disabled_scan_is_dropped_and_off_by_default_can_be_enabled() {
let mut t = target(TargetType::WebApp, vec![Artifact::git_repo("u", "main")]);
t.scan_config.disabled_scans = vec![ScanType::Lint];
// CodeReview is off by default for web; enable it explicitly.
t.scan_config.enabled_scans = vec![ScanType::CodeReview];
let plan = build_scan_plan(&t);
assert!(!plan.has(ScanType::Lint));
assert!(plan.has(ScanType::CodeReview));
assert!(plan.has(ScanType::Sast));
}
#[test]
fn no_code_artifact_yields_empty_plan_for_web() {
let t = target(TargetType::WebApp, vec![]);
let plan = build_scan_plan(&t);
assert!(plan.is_empty());
}
}
+7 -7
View File
@@ -288,25 +288,25 @@ async fn scan_all_repos(agent: &ComplianceAgent, tenant_id: &str) {
None => return,
};
let cursor = match db.repositories().find(doc! {}).await {
let cursor = match db.onboarded_targets().find(doc! {}).await {
Ok(c) => c,
Err(e) => {
tracing::error!("Failed to list repos for tenant '{tenant_id}': {e}");
tracing::error!("Failed to list targets for tenant '{tenant_id}': {e}");
return;
}
};
let repos: Vec<_> = cursor.filter_map(|r| async { r.ok() }).collect().await;
let targets: Vec<_> = cursor.filter_map(|r| async { r.ok() }).collect().await;
for repo in repos {
let repo_id = repo.id.map(|id| id.to_hex()).unwrap_or_default();
for target in targets {
let target_id = target.id.map(|id| id.to_hex()).unwrap_or_default();
if let Err(e) = agent
.run_scan(tenant_id, &repo_id, ScanTrigger::Scheduled)
.run_target_scan(tenant_id, &target_id, ScanTrigger::Scheduled)
.await
{
tracing::error!(
"Scheduled scan failed for {} (tenant '{tenant_id}'): {e}",
repo.name
target.name
);
}
}
+4 -2
View File
@@ -25,8 +25,9 @@ impl TestServer {
let mongodb_uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// Unique database name per test run to avoid collisions
let db_name = format!("test_{}", uuid::Uuid::new_v4().simple());
// Unique db-name prefix per run. Must fit the pool's 30-char cap
// (`<prefix>_<32 hex>` <= 63), so use a 16-hex-char suffix.
let db_name = format!("t_{}", &uuid::Uuid::new_v4().simple().to_string()[..16]);
let db_pool = DatabasePool::connect(&mongodb_uri, &db_name)
.await
@@ -69,6 +70,7 @@ impl TestServer {
pentest_imap_password: None,
admin_api_token: None,
tenant_registry_url: None,
unified_pipeline: false,
};
let agent = ComplianceAgent::new(config, db_pool);
@@ -2,5 +2,6 @@ mod cascade_delete;
mod dast;
mod findings;
mod health;
mod onboarding;
mod repositories;
mod stats;
@@ -0,0 +1,115 @@
use crate::common::TestServer;
use serde_json::json;
#[tokio::test]
async fn create_list_and_applicable_scans() {
let server = TestServer::start().await;
// Initially empty.
let resp = server.get("/api/v1/targets").await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 0);
// Create a web-app target with a git repo + a live URL.
let resp = server
.post(
"/api/v1/targets",
&json!({
"name": "acme-web",
"target_type": "web_app",
"artifacts": [
{ "kind": "git_repo", "source_ref": "https://git/acme.git", "branch": "main" },
{ "kind": "live_url", "source_ref": "https://acme.example.com" }
]
}),
)
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let id = body["data"]["_id"]["$oid"].as_str().unwrap().to_string();
assert!(!id.is_empty());
assert_eq!(body["data"]["artifacts"].as_array().unwrap().len(), 2);
// List returns it.
let resp = server.get("/api/v1/targets").await;
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"].as_array().unwrap().len(), 1);
// Applicable scans: SAST present + DAST offered (live URL present), pentest supported.
let resp = server
.get(&format!("/api/v1/targets/{id}/applicable-scans"))
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
let scans = body["data"]["scans"].as_array().unwrap();
let names: Vec<&str> = scans.iter().filter_map(|s| s["scan"].as_str()).collect();
assert!(names.contains(&"sast"));
assert!(names.contains(&"dast"));
assert_eq!(body["data"]["pentest_supported"], true);
server.cleanup().await;
}
#[tokio::test]
async fn detect_classifies_a_plc_target() {
let server = TestServer::start().await;
// A PLC project artifact is a strong kind-based signal.
let resp = server
.post(
"/api/v1/targets",
&json!({
"name": "line-controller",
"target_type": "backend_service", // deliberately wrong; detect should suggest PLC
"artifacts": [
{ "kind": "plc_project", "source_ref": "line.xml", "plc_format": "plcopen_xml" }
]
}),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let id = body["data"]["_id"]["$oid"].as_str().unwrap().to_string();
let resp = server
.post(&format!("/api/v1/targets/{id}/detect"), &json!({}))
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["classification"]["suggested"], "plc_sps");
server.cleanup().await;
}
#[tokio::test]
async fn add_artifact_and_delete_target() {
let server = TestServer::start().await;
let resp = server
.post(
"/api/v1/targets",
&json!({ "name": "svc", "target_type": "backend_service" }),
)
.await;
let body: serde_json::Value = resp.json().await.unwrap();
let id = body["data"]["_id"]["$oid"].as_str().unwrap().to_string();
// Attach a git repo.
let resp = server
.post(
&format!("/api/v1/targets/{id}/artifacts"),
&json!({ "kind": "git_repo", "source_ref": "https://git/svc.git" }),
)
.await;
assert_eq!(resp.status(), 200);
let body: serde_json::Value = resp.json().await.unwrap();
assert_eq!(body["data"]["artifacts"].as_array().unwrap().len(), 1);
// Delete it.
let resp = server.delete(&format!("/api/v1/targets/{id}")).await;
assert_eq!(resp.status(), 200);
let resp = server.get(&format!("/api/v1/targets/{id}")).await;
assert_eq!(resp.status(), 404);
server.cleanup().await;
}
@@ -0,0 +1,156 @@
// Integration tests for the onboarding backfill migration.
//
// Requires MongoDB (set TEST_MONGODB_URI if not at the default).
// Not run in CI (which is `--lib` only) — run locally:
// cargo test -p compliance-agent --test e2e migration
use compliance_agent::database::{Database, DatabasePool};
use compliance_agent::migrate::onboarding;
use compliance_core::models::{
ArtifactKind, DastTarget, DastTargetType, TargetType, TrackedRepository,
};
use mongodb::bson::{doc, Document};
async fn fresh_db() -> (DatabasePool, String, Database) {
let uri = std::env::var("TEST_MONGODB_URI")
.unwrap_or_else(|_| "mongodb://root:example@localhost:27017/?authSource=admin".into());
// Prefix must fit the pool's 30-char cap (`<prefix>_<32 hex>` <= 63).
let prefix = format!("t_{}", &uuid::Uuid::new_v4().simple().to_string()[..16]);
let pool = DatabasePool::connect(&uri, &prefix)
.await
.expect("connect mongo");
let db = pool.for_tenant_id("t1").await.expect("tenant db");
(pool, prefix, db)
}
async fn cleanup(pool: &DatabasePool, prefix: &str) {
if let Ok(names) = pool.client().list_database_names().await {
for n in names {
if n.starts_with(prefix) {
pool.client().database(&n).drop().await.ok();
}
}
}
}
#[tokio::test]
async fn backfill_folds_relinks_is_idempotent_and_reversible() {
let (pool, prefix, db) = fresh_db().await;
// Seed a repo.
let repo = TrackedRepository::new("acme".into(), "https://git/acme.git".into());
let repo_id = db
.repositories()
.insert_one(repo)
.await
.expect("insert repo")
.inserted_id
.as_object_id()
.expect("repo oid");
// A DAST target linked to the repo (folds + promotes to WebApp + relinks).
let mut linked = DastTarget::new(
"acme-web".into(),
"https://acme.example.com".into(),
DastTargetType::WebApp,
);
linked.repo_id = Some(repo_id.to_hex());
let linked_id = db
.dast_targets()
.insert_one(linked)
.await
.expect("insert linked dast")
.inserted_id
.as_object_id()
.expect("linked oid");
// A repo-less DAST target (standalone).
let standalone = DastTarget::new(
"acme-api".into(),
"https://api.acme.com".into(),
DastTargetType::RestApi,
);
let standalone_id = db
.dast_targets()
.insert_one(standalone)
.await
.expect("insert standalone dast")
.inserted_id
.as_object_id()
.expect("standalone oid");
// A DAST scan run pointing at the linked target — should be relinked to the repo.
db.collection_named::<Document>("dast_scan_runs")
.insert_one(doc! { "target_id": linked_id.to_hex(), "status": "completed" })
.await
.expect("insert dast run");
// --- Backfill ---
assert!(!onboarding::already_applied(&db).await.unwrap());
let report = onboarding::backfill_onboarded_targets(&db, false)
.await
.expect("backfill");
assert_eq!(report.repos_migrated, 1);
assert_eq!(report.dast_targets_folded, 1);
assert_eq!(report.dast_targets_standalone, 1);
assert!(onboarding::already_applied(&db).await.unwrap());
// Repo target: preserved _id, has git + folded live-url, promoted to WebApp.
let repo_target = db
.onboarded_targets()
.find_one(doc! { "_id": repo_id })
.await
.unwrap()
.expect("repo target");
assert!(repo_target.has(ArtifactKind::GitRepo));
assert!(repo_target.has(ArtifactKind::LiveUrl));
assert_eq!(repo_target.target_type, TargetType::WebApp);
// Standalone target: preserved _id, live-url, backend service.
let standalone_target = db
.onboarded_targets()
.find_one(doc! { "_id": standalone_id })
.await
.unwrap()
.expect("standalone target");
assert!(standalone_target.has(ArtifactKind::LiveUrl));
assert_eq!(standalone_target.target_type, TargetType::BackendService);
// The DAST run was relinked from the old dast id to the repo (unified) id.
let run = db
.collection_named::<Document>("dast_scan_runs")
.find_one(doc! {})
.await
.unwrap()
.expect("run");
assert_eq!(run.get_str("target_id").unwrap(), repo_id.to_hex());
// --- Idempotent: re-run migrates nothing new ---
let again = onboarding::backfill_onboarded_targets(&db, false)
.await
.expect("backfill again");
assert_eq!(again.repos_migrated, 0);
assert_eq!(again.dast_targets_folded, 0);
assert_eq!(again.dast_targets_standalone, 0);
assert!(again.skipped_existing >= 2);
// --- Revert: onboarded targets gone, relink undone, marker cleared ---
onboarding::revert(&db).await.expect("revert");
assert_eq!(
db.onboarded_targets()
.count_documents(doc! {})
.await
.unwrap(),
0
);
let run_after = db
.collection_named::<Document>("dast_scan_runs")
.find_one(doc! {})
.await
.unwrap()
.expect("run");
assert_eq!(run_after.get_str("target_id").unwrap(), linked_id.to_hex());
assert!(!onboarding::already_applied(&db).await.unwrap());
cleanup(&pool, &prefix).await;
}
@@ -7,3 +7,4 @@
// Or nightly: (via CI with MongoDB service container)
mod api;
mod migration;
+5
View File
@@ -49,6 +49,11 @@ pub struct AgentConfig {
/// of tenants to iterate. When `None` or unreachable, scheduler
/// falls back to `SCHEDULER_TENANT_IDS` env (M7.2-C).
pub tenant_registry_url: Option<String>,
/// When true, `run_scan` dispatches to the unified `run_target` pipeline
/// (reads `onboarded_targets`) instead of the legacy repository pipeline.
/// Env `UNIFIED_PIPELINE`. Defaults on; set `UNIFIED_PIPELINE=0` to use the
/// legacy repository pipeline.
pub unified_pipeline: bool,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
+4
View File
@@ -12,6 +12,10 @@ pub enum Route {
OverviewPage {},
#[route("/repositories")]
RepositoriesPage {},
#[route("/targets")]
TargetsPage {},
#[route("/onboard")]
OnboardingPage {},
#[route("/findings")]
FindingsPage {},
#[route("/findings/:id")]
@@ -24,10 +24,15 @@ pub fn Sidebar() -> Element {
icon: rsx! { Icon { icon: BsSpeedometer2, width: 18, height: 18 } },
},
NavItem {
label: "Repositories",
route: Route::RepositoriesPage {},
label: "Targets",
route: Route::TargetsPage {},
icon: rsx! { Icon { icon: BsFolder2Open, width: 18, height: 18 } },
},
NavItem {
label: "Onboard",
route: Route::OnboardingPage {},
icon: rsx! { Icon { icon: BsPlusCircle, width: 18, height: 18 } },
},
NavItem {
label: "Findings",
route: Route::FindingsPage {},
@@ -10,6 +10,7 @@ pub mod issues;
pub mod mcp;
pub mod mcp_tokens;
pub mod notifications;
pub mod onboarding;
pub mod pentest;
#[allow(clippy::too_many_arguments)]
pub mod repositories;
@@ -0,0 +1,139 @@
//! Server functions for the onboarding wizard — proxy to the agent's
//! `/api/v1/targets` endpoints.
use dioxus::prelude::*;
use serde::{Deserialize, Serialize};
/// One artifact the wizard collects for a target.
#[derive(Debug, Clone, Serialize, Deserialize, Default, PartialEq)]
pub struct ArtifactInputDto {
pub kind: String,
pub source_ref: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub branch: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub plc_format: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct TargetsResponse {
pub data: Vec<serde_json::Value>,
pub total: Option<u64>,
}
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct TargetResponse {
pub data: serde_json::Value,
}
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct ApplicableScansData {
#[serde(default)]
pub scans: Vec<serde_json::Value>,
#[serde(default)]
pub pentest_supported: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct ApplicableScansResponse {
pub data: ApplicableScansData,
}
/// List onboarded targets.
#[server]
pub async fn fetch_targets() -> Result<TargetsResponse, ServerFnError> {
let resp = super::agent_client::agent_get("/api/v1/targets")
.await?
.send()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
resp.json()
.await
.map_err(|e| ServerFnError::new(e.to_string()))
}
/// Create a target with the collected artifacts.
#[server]
pub async fn create_target(
name: String,
target_type: String,
description: Option<String>,
artifacts: Vec<ArtifactInputDto>,
) -> Result<TargetResponse, ServerFnError> {
let body = serde_json::json!({
"name": name,
"target_type": target_type,
"description": description,
"artifacts": artifacts,
});
let resp = super::agent_client::agent_request(reqwest::Method::POST, "/api/v1/targets")
.await?
.json(&body)
.send()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
resp.json()
.await
.map_err(|e| ServerFnError::new(e.to_string()))
}
/// Run kind-based classification on a target.
#[server]
pub async fn detect_target(id: String) -> Result<TargetResponse, ServerFnError> {
let resp = super::agent_client::agent_request(
reqwest::Method::POST,
&format!("/api/v1/targets/{id}/detect"),
)
.await?
.send()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
resp.json()
.await
.map_err(|e| ServerFnError::new(e.to_string()))
}
/// Fetch the scan-applicability matrix for a target.
#[server]
pub async fn fetch_applicable_scans(id: String) -> Result<ApplicableScansResponse, ServerFnError> {
let resp = super::agent_client::agent_get(&format!("/api/v1/targets/{id}/applicable-scans"))
.await?
.send()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
resp.json()
.await
.map_err(|e| ServerFnError::new(e.to_string()))
}
/// Delete a target (and cascade its findings / SBOM / scan runs / CVE alerts).
#[server]
pub async fn delete_target(id: String) -> Result<serde_json::Value, ServerFnError> {
let resp = super::agent_client::agent_request(
reqwest::Method::DELETE,
&format!("/api/v1/targets/{id}"),
)
.await?
.send()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
resp.json()
.await
.map_err(|e| ServerFnError::new(e.to_string()))
}
/// Trigger a scan for a target.
#[server]
pub async fn trigger_target_scan(id: String) -> Result<serde_json::Value, ServerFnError> {
let resp = super::agent_client::agent_request(
reqwest::Method::POST,
&format!("/api/v1/targets/{id}/scan"),
)
.await?
.send()
.await
.map_err(|e| ServerFnError::new(e.to_string()))?;
resp.json()
.await
.map_err(|e| ServerFnError::new(e.to_string()))
}
+5 -5
View File
@@ -20,7 +20,7 @@ pub fn FindingsPage() -> Element {
let mut selected_ids = use_signal(Vec::<String>::new);
let repos = use_resource(|| async {
crate::infrastructure::repositories::fetch_repositories(1)
crate::infrastructure::onboarding::fetch_targets()
.await
.ok()
});
@@ -86,14 +86,14 @@ pub fn FindingsPage() -> Element {
}
select {
onchange: move |e| { repo_filter.set(e.value()); page.set(1); },
option { value: "", "All Repositories" }
option { value: "", "All Targets" }
{
match &*repos.read() {
Some(Some(resp)) => rsx! {
for repo in &resp.data {
for t in &resp.data {
{
let id = repo.id.as_ref().map(|id| id.to_hex()).unwrap_or_default();
let name = repo.name.clone();
let id = t.get("_id").and_then(|o| o.get("$oid")).and_then(|s| s.as_str()).unwrap_or_default().to_string();
let name = t.get("name").and_then(|n| n.as_str()).unwrap_or_default().to_string();
rsx! {
option { value: "{id}", "{name}" }
}
+4
View File
@@ -12,11 +12,13 @@ pub mod impact_analysis;
pub mod issues;
pub mod mcp_servers;
pub mod mcp_tokens;
pub mod onboarding;
pub mod overview;
pub mod pentest_dashboard;
pub mod pentest_session;
pub mod repositories;
pub mod sbom;
pub mod targets;
pub use chat::ChatPage;
pub use chat_index::ChatIndexPage;
@@ -32,8 +34,10 @@ pub use impact_analysis::ImpactAnalysisPage;
pub use issues::IssuesPage;
pub use mcp_servers::McpServersPage;
pub use mcp_tokens::McpTokensPage;
pub use onboarding::OnboardingPage;
pub use overview::OverviewPage;
pub use pentest_dashboard::PentestDashboardPage;
pub use pentest_session::PentestSessionPage;
pub use repositories::RepositoriesPage;
pub use sbom::SbomPage;
pub use targets::TargetsPage;
@@ -0,0 +1,402 @@
use dioxus::prelude::*;
use crate::components::page_header::PageHeader;
use crate::infrastructure::onboarding::{
create_target, detect_target, fetch_applicable_scans, trigger_target_scan, ArtifactInputDto,
};
/// (value, label, one-line description) for the 9 target families.
const TARGET_TYPES: &[(&str, &str, &str)] = &[
("web_app", "Web Application", "Front end + server"),
("backend_service", "Backend / API", "REST, GraphQL, gRPC"),
("desktop_app", "Desktop App", "Windows / macOS / Linux"),
("android_app", "Android App", "APK / AAB"),
("ios_app", "iOS App", "IPA"),
(
"firmware_bare_metal",
"Firmware — bare metal",
"No operating system",
),
("firmware_rtos", "Firmware — RTOS", "Zephyr, FreeRTOS, ..."),
(
"embedded_linux_yocto",
"Embedded Linux / Yocto",
"BSP + image",
),
("plc_sps", "PLC / SPS", "IEC 61131-3"),
];
/// (value, label) for the artifact kinds a user can attach.
const ARTIFACT_KINDS: &[(&str, &str)] = &[
("git_repo", "Git repository"),
("source_archive", "Source archive (zip)"),
("firmware_image", "Firmware image"),
("mobile_package", "Mobile package (APK/IPA)"),
("container_image", "Container image"),
("live_url", "Live URL"),
("plc_project", "PLC project"),
("plaintext_description", "Description (text)"),
];
const STEP_LABELS: &[&str] = &["Target type", "Artifacts", "Review", "Done"];
/// One row in the applicable-scans list on the success step.
#[component]
fn ScanRow(scan: serde_json::Value) -> Element {
let name = scan
.get("scan")
.and_then(|v| v.as_str())
.unwrap_or("?")
.to_string();
let rationale = scan
.get("rationale")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let blocked = scan
.get("blocked_reason")
.and_then(|v| v.as_str())
.map(String::from);
let default_on = scan
.get("default_on")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let badge_class = if blocked.is_some() {
"badge badge-info"
} else if default_on {
"badge badge-success"
} else {
"badge"
};
rsx! {
div { style: "display: flex; gap: 8px; align-items: center; padding: 6px 0;",
span { class: "{badge_class}", "{name}" }
span { style: "opacity: 0.8;", "{rationale}" }
if let Some(b) = blocked {
span { style: "opacity: 0.6; font-style: italic;", "— {b}" }
}
}
}
}
fn kind_label(kind: &str) -> &str {
ARTIFACT_KINDS
.iter()
.find(|(v, _)| *v == kind)
.map(|(_, l)| *l)
.unwrap_or(kind)
}
fn type_label(value: &str) -> &str {
TARGET_TYPES
.iter()
.find(|(v, _, _)| *v == value)
.map(|(_, l, _)| *l)
.unwrap_or(value)
}
#[component]
pub fn OnboardingPage() -> Element {
let mut step = use_signal(|| 0usize);
let mut name = use_signal(String::new);
let mut target_type = use_signal(String::new);
let mut description = use_signal(String::new);
let mut artifacts = use_signal(Vec::<ArtifactInputDto>::new);
// "Add artifact" mini-form.
let mut new_kind = use_signal(|| "git_repo".to_string());
let mut new_source = use_signal(String::new);
let mut new_branch = use_signal(|| "main".to_string());
// Create + result state.
let mut creating = use_signal(|| false);
let mut error = use_signal(|| Option::<String>::None);
let mut scans = use_signal(Vec::<serde_json::Value>::new);
let mut suggested = use_signal(|| Option::<String>::None);
let mut created_id = use_signal(|| Option::<String>::None);
let mut scan_msg = use_signal(|| Option::<String>::None);
let step_now = step();
let can_advance_type = !name().trim().is_empty() && !target_type().trim().is_empty();
let has_artifacts = !artifacts().is_empty();
rsx! {
PageHeader {
title: "Onboard a target",
description: "Add a target, attach its artifacts, and see which scans apply.",
}
// Stepper.
div { class: "wizard-steps",
for (i, label) in STEP_LABELS.iter().enumerate() {
div {
class: if i == step_now { "wizard-step wizard-step-active" } else { "wizard-step" },
span { class: "wizard-step-dot", "{i + 1}" }
span { class: "wizard-step-label", "{label}" }
}
}
}
if let Some(err) = error() {
div { class: "card", style: "border-color: var(--danger, #d33); margin-bottom: 12px;",
div { class: "card-header", "Error" }
div { style: "padding: 12px;", "{err}" }
}
}
div { class: "card",
// ---- Step 0: target type + name ----
if step_now == 0 {
div { class: "card-header", "What kind of software is this?" }
div { style: "padding: 16px;",
div { class: "form-group",
label { "Name" }
input {
r#type: "text",
placeholder: "acme-web",
value: "{name}",
oninput: move |e| name.set(e.value()),
}
}
div {
style: "display: grid; grid-template-columns: repeat(auto-fill, minmax(200px, 1fr)); gap: 12px; margin-top: 12px;",
for (value, tlabel, tdesc) in TARGET_TYPES.iter().copied() {
div {
class: "card",
style: if target_type() == value {
"padding: 12px; cursor: pointer; border: 2px solid var(--accent, #3b82f6);"
} else {
"padding: 12px; cursor: pointer;"
},
onclick: move |_| target_type.set(value.to_string()),
div { style: "font-weight: 600;", "{tlabel}" }
div { style: "font-size: 0.85em; opacity: 0.7;", "{tdesc}" }
}
}
}
}
}
// ---- Step 1: artifacts ----
if step_now == 1 {
div { class: "card-header", "Attach artifacts" }
div { style: "padding: 16px;",
div { style: "display: flex; gap: 8px; flex-wrap: wrap; align-items: flex-end;",
div { class: "form-group", style: "margin: 0;",
label { "Kind" }
select {
value: "{new_kind}",
oninput: move |e| new_kind.set(e.value()),
for (value, klabel) in ARTIFACT_KINDS.iter().copied() {
option { value: "{value}", "{klabel}" }
}
}
}
div { class: "form-group", style: "margin: 0; flex: 1; min-width: 240px;",
label { "Reference (URL / path / text)" }
input {
r#type: "text",
placeholder: "https://git.example.com/acme.git",
value: "{new_source}",
oninput: move |e| new_source.set(e.value()),
}
}
if new_kind() == "git_repo" {
div { class: "form-group", style: "margin: 0;",
label { "Branch" }
input {
r#type: "text",
value: "{new_branch}",
oninput: move |e| new_branch.set(e.value()),
}
}
}
button {
class: "btn btn-secondary",
onclick: move |_| {
let kind = new_kind();
if !new_source().trim().is_empty() {
let branch = if kind == "git_repo" { Some(new_branch()) } else { None };
artifacts.write().push(ArtifactInputDto {
kind,
source_ref: new_source(),
branch,
plc_format: None,
});
new_source.set(String::new());
}
},
"+ Add"
}
}
div { style: "margin-top: 16px;",
if has_artifacts {
for (i, a) in artifacts().iter().enumerate() {
div {
style: "display: flex; justify-content: space-between; align-items: center; padding: 8px 12px; border: 1px solid var(--border, #333); border-radius: 6px; margin-bottom: 6px;",
span {
span { style: "opacity: 0.7;", "{kind_label(&a.kind)}: " }
"{a.source_ref}"
}
button {
class: "btn btn-ghost-danger btn-sm",
onclick: move |_| { artifacts.write().remove(i); },
"Remove"
}
}
}
} else {
div { style: "opacity: 0.6;", "No artifacts yet. Add at least one." }
}
}
}
}
// ---- Step 2: review + create ----
if step_now == 2 {
div { class: "card-header", "Review" }
div { style: "padding: 16px;",
div { class: "wizard-summary",
div { strong { "Name: " } "{name()}" }
div { strong { "Type: " } "{type_label(&target_type())}" }
div { strong { "Artifacts:" } }
ul {
for a in artifacts() {
li { "{kind_label(&a.kind)}: {a.source_ref}" }
}
}
}
div { style: "margin-top: 12px; opacity: 0.7; font-size: 0.9em;",
"The applicable scans (SAST / DAST / firmware / PLC) are shown after the target is created."
}
}
}
// ---- Step 3: created ----
if step_now == 3 {
div { class: "card-header", "Target onboarded" }
div { style: "padding: 16px;",
p {
strong { "{name()}" }
" was created."
if let Some(s) = suggested() {
span { " Suggested type from detection: " strong { "{type_label(&s)}" } "." }
}
}
h4 { style: "margin-top: 16px;", "Applicable scans" }
if scans().is_empty() {
div { style: "opacity: 0.6;", "No scans available (no code / URL / firmware artifact present)." }
} else {
for s in scans() {
ScanRow { scan: s }
}
}
if let Some(msg) = scan_msg() {
div { style: "margin-top: 8px; color: var(--success, #2a2);", "{msg}" }
}
div { style: "margin-top: 16px; display: flex; gap: 8px;",
button {
class: "btn btn-primary",
onclick: move |_| {
if let Some(id) = created_id() {
scan_msg.set(Some("Scan triggered...".to_string()));
spawn(async move {
match trigger_target_scan(id).await {
Ok(_) => scan_msg.set(Some(
"Scan started — findings will appear as it runs.".to_string(),
)),
Err(e) => scan_msg.set(Some(format!("Failed to start scan: {e}"))),
}
});
}
},
"Run scan"
}
button {
class: "btn btn-secondary",
onclick: move |_| {
step.set(0);
name.set(String::new());
target_type.set(String::new());
description.set(String::new());
artifacts.write().clear();
scans.write().clear();
suggested.set(None);
created_id.set(None);
scan_msg.set(None);
error.set(None);
},
"Onboard another"
}
}
}
}
}
// ---- Footer navigation ----
if step_now < 3 {
div { style: "display: flex; justify-content: space-between; margin-top: 16px;",
button {
class: "btn btn-back",
disabled: step_now == 0,
onclick: move |_| { if step() > 0 { step.set(step() - 1); } },
"Back"
}
if step_now < 2 {
button {
class: "btn btn-primary",
disabled: (step_now == 0 && !can_advance_type) || (step_now == 1 && !has_artifacts),
onclick: move |_| step.set(step() + 1),
"Next"
}
} else {
button {
class: "btn btn-primary",
disabled: creating(),
onclick: move |_| {
let n = name();
let tt = target_type();
let desc = description();
let arts = artifacts();
let d = if desc.trim().is_empty() { None } else { Some(desc) };
creating.set(true);
error.set(None);
spawn(async move {
match create_target(n, tt, d, arts).await {
Ok(resp) => {
let id = resp
.data
.get("_id")
.and_then(|o| o.get("$oid"))
.and_then(|s| s.as_str())
.map(String::from);
if let Some(id) = id {
created_id.set(Some(id.clone()));
if let Ok(sc) = fetch_applicable_scans(id.clone()).await {
scans.set(sc.data.scans);
}
if let Ok(det) = detect_target(id).await {
suggested.set(
det.data
.get("classification")
.and_then(|c| c.get("suggested"))
.and_then(|s| s.as_str())
.map(String::from),
);
}
}
step.set(3);
}
Err(e) => error.set(Some(e.to_string())),
}
creating.set(false);
});
},
if creating() { "Creating..." } else { "Create target" }
}
}
}
}
}
}
+1 -230
View File
@@ -23,20 +23,6 @@ async fn async_sleep_5s() {
#[component]
pub fn RepositoriesPage() -> Element {
let mut page = use_signal(|| 1u64);
let mut show_add_form = use_signal(|| false);
let mut name = use_signal(String::new);
let mut git_url = use_signal(String::new);
let mut branch = use_signal(|| "main".to_string());
let mut auth_token = use_signal(String::new);
let mut auth_username = use_signal(String::new);
let mut show_auth = use_signal(|| false);
let mut ssh_public_key = use_signal(String::new);
let mut show_tracker = use_signal(|| false);
let mut tracker_type_val = use_signal(String::new);
let mut tracker_owner_val = use_signal(String::new);
let mut tracker_repo_val = use_signal(String::new);
let mut tracker_token_val = use_signal(String::new);
let mut adding = use_signal(|| false);
let mut toasts = use_context::<Toasts>();
let mut confirm_delete = use_signal(|| Option::<(String, String)>::None); // (id, name)
let mut edit_repo_id = use_signal(|| Option::<String>::None);
@@ -64,222 +50,7 @@ pub fn RepositoriesPage() -> Element {
rsx! {
PageHeader {
title: "Repositories",
description: "Tracked git repositories",
}
div { style: "margin-bottom: 16px;",
button {
class: "btn btn-primary",
onclick: move |_| show_add_form.toggle(),
if show_add_form() { "Cancel" } else { "+ Add Repository" }
}
}
if show_add_form() {
div { class: "card",
div { class: "card-header", "Add Repository" }
div { class: "form-group",
label { "Name" }
input {
r#type: "text",
placeholder: "my-project",
value: "{name}",
oninput: move |e| name.set(e.value()),
}
}
div { class: "form-group",
label { "Git URL" }
input {
r#type: "text",
placeholder: "https://github.com/org/repo.git or git@github.com:org/repo.git",
value: "{git_url}",
oninput: move |e| git_url.set(e.value()),
}
}
div { class: "form-group",
label { "Default Branch" }
input {
r#type: "text",
placeholder: "main",
value: "{branch}",
oninput: move |e| branch.set(e.value()),
}
}
// Private repo auth section
div { style: "margin-top: 8px;",
button {
class: "btn btn-ghost",
style: "font-size: 12px; padding: 4px 8px;",
onclick: move |_| {
let opening = !show_auth();
show_auth.toggle();
if opening {
// Fetch SSH key every time the section opens
ssh_public_key.set(String::new());
spawn(async move {
match crate::infrastructure::repositories::fetch_ssh_public_key().await {
Ok(key) => ssh_public_key.set(key),
Err(_) => ssh_public_key.set("(not available)".to_string()),
}
});
}
},
if show_auth() { "Hide auth options" } else { "Private repository?" }
}
}
if show_auth() {
div { class: "auth-section", style: "margin-top: 12px; padding: 12px; border: 1px solid var(--border-subtle); border-radius: 8px;",
// SSH deploy key display
div { style: "margin-bottom: 12px;",
label { style: "font-size: 12px; color: var(--text-secondary);",
"For SSH URLs: add this deploy key (read-only) to your repository"
}
div {
class: "copyable",
style: "margin-top: 4px; padding: 8px; background: var(--bg-secondary); border-radius: 4px;",
code {
style: "font-size: 11px; word-break: break-all; user-select: all;",
if ssh_public_key().is_empty() {
"Loading..."
} else {
"{ssh_public_key}"
}
}
if !ssh_public_key().is_empty() {
crate::components::copy_button::CopyButton { value: ssh_public_key(), small: true }
}
}
}
// HTTPS auth fields
p { style: "font-size: 12px; color: var(--text-secondary); margin-bottom: 8px;",
"For HTTPS URLs: provide an access token (PAT) or username/password"
}
div { class: "form-group",
label { "Auth Token / Password" }
input {
r#type: "password",
placeholder: "ghp_xxxx or personal access token",
value: "{auth_token}",
oninput: move |e| auth_token.set(e.value()),
}
}
div { class: "form-group",
label { "Username (optional, defaults to x-access-token)" }
input {
r#type: "text",
placeholder: "x-access-token",
value: "{auth_username}",
oninput: move |e| auth_username.set(e.value()),
}
}
}
}
// Issue tracker config section
div { style: "margin-top: 8px;",
button {
class: "btn btn-ghost",
style: "font-size: 12px; padding: 4px 8px;",
onclick: move |_| show_tracker.toggle(),
if show_tracker() { "Hide tracker options" } else { "Issue tracker?" }
}
}
if show_tracker() {
div { class: "auth-section", style: "margin-top: 12px; padding: 12px; border: 1px solid var(--border-subtle); border-radius: 8px;",
p { style: "font-size: 12px; color: var(--text-secondary); margin-bottom: 8px;",
"Configure an issue tracker to auto-create issues from findings"
}
div { class: "form-group",
label { "Tracker Type" }
select {
value: "{tracker_type_val}",
onchange: move |e| tracker_type_val.set(e.value()),
option { value: "", "None" }
option { value: "github", "GitHub" }
option { value: "gitlab", "GitLab" }
option { value: "gitea", "Gitea" }
option { value: "jira", "Jira" }
}
}
div { class: "form-group",
label { "Owner / Namespace" }
input {
r#type: "text",
placeholder: "org-name",
value: "{tracker_owner_val}",
oninput: move |e| tracker_owner_val.set(e.value()),
}
}
div { class: "form-group",
label { "Repository / Project" }
input {
r#type: "text",
placeholder: "repo-name",
value: "{tracker_repo_val}",
oninput: move |e| tracker_repo_val.set(e.value()),
}
}
div { class: "form-group",
label { "Tracker Token (PAT)" }
input {
r#type: "password",
placeholder: "ghp_xxxx / glpat-xxxx",
value: "{tracker_token_val}",
oninput: move |e| tracker_token_val.set(e.value()),
}
}
}
}
button {
class: "btn btn-primary",
disabled: adding(),
onclick: move |_| {
let n = name();
let u = git_url();
let b = branch();
let tok = {
let v = auth_token();
if v.is_empty() { None } else { Some(v) }
};
let usr = {
let v = auth_username();
if v.is_empty() { None } else { Some(v) }
};
let tt = { let v = tracker_type_val(); if v.is_empty() { None } else { Some(v) } };
let t_owner = { let v = tracker_owner_val(); if v.is_empty() { None } else { Some(v) } };
let t_repo = { let v = tracker_repo_val(); if v.is_empty() { None } else { Some(v) } };
let t_tok = { let v = tracker_token_val(); if v.is_empty() { None } else { Some(v) } };
adding.set(true);
spawn(async move {
match crate::infrastructure::repositories::add_repository(n, u, b, tok, usr, tt, t_owner, t_repo, t_tok).await {
Ok(_) => {
toasts.push(ToastType::Success, "Repository added");
repos.restart();
}
Err(e) => toasts.push(ToastType::Error, e.to_string()),
}
adding.set(false);
});
show_add_form.set(false);
show_auth.set(false);
show_tracker.set(false);
name.set(String::new());
git_url.set(String::new());
auth_token.set(String::new());
auth_username.set(String::new());
tracker_type_val.set(String::new());
tracker_owner_val.set(String::new());
tracker_repo_val.set(String::new());
tracker_token_val.set(String::new());
},
if adding() { "Validating..." } else { "Add" }
}
}
description: "Legacy git repositories. Onboard new targets from Targets / Onboard.",
}
// ── Delete confirmation dialog ──
+9 -9
View File
@@ -28,9 +28,9 @@ pub fn SbomPage() -> Element {
let mut diff_repo_a = use_signal(String::new);
let mut diff_repo_b = use_signal(String::new);
// ── Repos for dropdowns ──
// ── Targets for dropdowns ──
let repos = use_resource(|| async {
crate::infrastructure::repositories::fetch_repositories(1)
crate::infrastructure::onboarding::fetch_targets()
.await
.ok()
});
@@ -114,14 +114,14 @@ pub fn SbomPage() -> Element {
select {
class: "sbom-filter-select",
onchange: move |e| { repo_filter.set(e.value()); page.set(1); },
option { value: "", "All Repositories" }
option { value: "", "All Targets" }
{
match &*repos.read() {
Some(Some(resp)) => rsx! {
for repo in &resp.data {
{
let id = repo.id.as_ref().map(|id| id.to_hex()).unwrap_or_default();
let name = repo.name.clone();
let id = repo.get("_id").and_then(|o| o.get("$oid")).and_then(|s| s.as_str()).unwrap_or_default().to_string();
let name = repo.get("name").and_then(|n| n.as_str()).unwrap_or_default().to_string();
rsx! { option { value: "{id}", "{name}" } }
}
}
@@ -476,8 +476,8 @@ pub fn SbomPage() -> Element {
Some(Some(resp)) => rsx! {
for repo in &resp.data {
{
let id = repo.id.as_ref().map(|id| id.to_hex()).unwrap_or_default();
let name = repo.name.clone();
let id = repo.get("_id").and_then(|o| o.get("$oid")).and_then(|s| s.as_str()).unwrap_or_default().to_string();
let name = repo.get("name").and_then(|n| n.as_str()).unwrap_or_default().to_string();
rsx! { option { value: "{id}", "{name}" } }
}
}
@@ -498,8 +498,8 @@ pub fn SbomPage() -> Element {
Some(Some(resp)) => rsx! {
for repo in &resp.data {
{
let id = repo.id.as_ref().map(|id| id.to_hex()).unwrap_or_default();
let name = repo.name.clone();
let id = repo.get("_id").and_then(|o| o.get("$oid")).and_then(|s| s.as_str()).unwrap_or_default().to_string();
let name = repo.get("name").and_then(|n| n.as_str()).unwrap_or_default().to_string();
rsx! { option { value: "{id}", "{name}" } }
}
}
+339
View File
@@ -0,0 +1,339 @@
//! Targets page — lists onboarded targets (the unified `OnboardedTarget`
//! records the wizard creates) and lets the user run a scan, inspect the
//! classification / applicable scans, or delete a target.
//!
//! Creation lives in the Onboard wizard (`/onboard`); this page is the
//! "where is my target, and what did the scan find" surface.
use dioxus::prelude::*;
use dioxus_free_icons::icons::bs_icons::*;
use dioxus_free_icons::Icon;
use crate::components::page_header::PageHeader;
use crate::components::toast::{ToastType, Toasts};
use crate::infrastructure::onboarding::{
delete_target, fetch_applicable_scans, fetch_targets, trigger_target_scan,
};
/// Prettify a snake_case target-type value into a human label.
fn pretty_type(v: &str) -> String {
match v {
"web_app" => "Web Application".into(),
"backend_service" => "Backend / API".into(),
"desktop_app" => "Desktop App".into(),
"android_app" => "Android App".into(),
"ios_app" => "iOS App".into(),
"firmware_bare_metal" => "Firmware — bare metal".into(),
"firmware_rtos" => "Firmware — RTOS".into(),
"embedded_linux_yocto" => "Embedded Linux / Yocto".into(),
"plc_sps" => "PLC / SPS".into(),
other => other.replace('_', " "),
}
}
fn str_at<'a>(v: &'a serde_json::Value, key: &str) -> &'a str {
v.get(key).and_then(|x| x.as_str()).unwrap_or("")
}
fn target_id(t: &serde_json::Value) -> String {
t.get("_id")
.and_then(|o| o.get("$oid"))
.and_then(|s| s.as_str())
.unwrap_or_default()
.to_string()
}
/// The applicable-scans matrix for one target, fetched on expand.
#[component]
fn TargetScans(id: String) -> Element {
let scan_id = id.clone();
let scans = use_resource(move || {
let id = scan_id.clone();
async move { fetch_applicable_scans(id).await.ok() }
});
let snapshot = scans.read().clone();
match &snapshot {
Some(Some(resp)) => {
let rows = resp.data.scans.clone();
let pentest = resp.data.pentest_supported;
rsx! {
div { style: "margin-top: 8px;",
if rows.is_empty() {
div { style: "opacity: 0.6;", "No scans available (no code / URL / firmware artifact present)." }
}
for s in rows {
{
let name = str_at(&s, "scan").to_string();
let rationale = str_at(&s, "rationale").to_string();
let blocked = s.get("blocked_reason").and_then(|b| b.as_str()).map(String::from);
let default_on = s.get("default_on").and_then(|b| b.as_bool()).unwrap_or(false);
let badge = if blocked.is_some() {
"badge badge-info"
} else if default_on {
"badge badge-success"
} else {
"badge"
};
rsx! {
div { style: "display: flex; gap: 8px; align-items: center; padding: 4px 0;",
span { class: "{badge}", "{name}" }
span { style: "opacity: 0.8; font-size: 0.9em;", "{rationale}" }
if let Some(b) = blocked {
span { style: "opacity: 0.6; font-style: italic; font-size: 0.9em;", "— {b}" }
}
}
}
}
}
if pentest {
div { style: "margin-top: 6px; opacity: 0.75; font-size: 0.85em;",
"Active pentest is supported for this target type."
}
}
}
}
}
Some(None) => rsx! { div { style: "opacity: 0.6;", "Failed to load applicable scans." } },
None => rsx! { div { style: "opacity: 0.6;", "Loading scans..." } },
}
}
#[component]
pub fn TargetsPage() -> Element {
let mut toasts = use_context::<Toasts>();
let mut scanning_ids = use_signal(Vec::<String>::new);
let mut expanded_ids = use_signal(Vec::<String>::new);
let mut confirm_delete = use_signal(|| Option::<(String, String)>::None);
let mut targets = use_resource(move || async move { fetch_targets().await.ok() });
rsx! {
PageHeader {
title: "Targets",
description: "Onboarded targets and what their scans found. Add new targets from Onboard.",
}
div { style: "margin-bottom: 16px; display: flex; gap: 8px;",
Link { to: crate::app::Route::OnboardingPage {}, class: "btn btn-primary",
"+ Onboard a target"
}
button {
class: "btn btn-secondary",
onclick: move |_| targets.restart(),
"Refresh"
}
}
// ── Delete confirmation ──
if let Some((del_id, del_name)) = confirm_delete() {
div { class: "modal-overlay",
div { class: "modal-dialog",
h3 { "Delete Target" }
p { "Delete " strong { "{del_name}" } "?" }
p { class: "modal-warning",
"This permanently removes the target and its findings, SBOM entries, scan runs, and CVE alerts."
}
div { class: "modal-actions",
button {
class: "btn btn-secondary",
onclick: move |_| confirm_delete.set(None),
"Cancel"
}
button {
class: "btn btn-danger",
onclick: move |_| {
let id = del_id.clone();
let name = del_name.clone();
confirm_delete.set(None);
spawn(async move {
match delete_target(id).await {
Ok(_) => {
toasts.push(ToastType::Success, format!("{name} deleted"));
targets.restart();
}
Err(e) => toasts.push(ToastType::Error, e.to_string()),
}
});
},
"Delete"
}
}
}
}
}
{
let targets_snapshot = targets.read().clone();
match &targets_snapshot {
Some(Some(resp)) => {
let rows = resp.data.clone();
if rows.is_empty() {
rsx! {
div { class: "card", style: "padding: 24px; text-align: center;",
p { style: "opacity: 0.7;", "No targets yet." }
Link { to: crate::app::Route::OnboardingPage {}, class: "btn btn-primary",
"Onboard your first target"
}
}
}
} else {
rsx! {
div { class: "card",
div { class: "table-wrapper",
table {
thead {
tr {
th { "Name" }
th { "Type" }
th { "Detected" }
th { "Artifacts" }
th { "Findings" }
th { "Actions" }
}
}
tbody {
for t in rows {
{
let id = target_id(&t);
let name = str_at(&t, "name").to_string();
let ttype = pretty_type(str_at(&t, "target_type"));
let suggested = t
.get("classification")
.and_then(|c| c.get("suggested"))
.and_then(|s| s.as_str())
.map(pretty_type);
let artifacts = t
.get("artifacts")
.and_then(|a| a.as_array())
.cloned()
.unwrap_or_default();
let findings = t
.get("findings_count")
.and_then(|n| n.as_u64())
.unwrap_or(0);
let facts = t
.get("classification")
.and_then(|c| c.get("facts"))
.and_then(|f| f.as_array())
.cloned()
.unwrap_or_default();
let is_scanning = scanning_ids().contains(&id);
let is_expanded = expanded_ids().contains(&id);
let id_scan = id.clone();
let id_exp = id.clone();
let id_del = id.clone();
let name_del = name.clone();
let artifacts_detail = artifacts.clone();
rsx! {
tr {
td { strong { "{name}" } }
td { "{ttype}" }
td {
if let Some(sug) = suggested.clone() {
span { class: "badge badge-success", "{sug}" }
} else {
span { style: "opacity: 0.5;", "" }
}
}
td { "{artifacts.len()}" }
td { "{findings}" }
td { style: "display: flex; gap: 4px;",
button {
class: "btn btn-ghost",
title: "Details",
onclick: move |_| {
let mut ids = expanded_ids();
if ids.contains(&id_exp) {
ids.retain(|i| i != &id_exp);
} else {
ids.push(id_exp.clone());
}
expanded_ids.set(ids);
},
Icon { icon: BsInfoCircle, width: 16, height: 16 }
}
button {
class: if is_scanning { "btn btn-ghost btn-scanning" } else { "btn btn-ghost" },
title: "Run scan",
disabled: is_scanning,
onclick: move |_| {
let id = id_scan.clone();
let mut ids = scanning_ids();
ids.push(id.clone());
scanning_ids.set(ids);
spawn(async move {
match trigger_target_scan(id.clone()).await {
Ok(_) => toasts.push(ToastType::Success, "Scan triggered — findings appear as it runs. Use Refresh."),
Err(e) => toasts.push(ToastType::Error, e.to_string()),
}
let mut ids = scanning_ids();
ids.retain(|i| i != &id);
scanning_ids.set(ids);
});
},
if is_scanning {
span { class: "spinner" }
} else {
Icon { icon: BsPlayCircle, width: 16, height: 16 }
}
}
button {
class: "btn btn-ghost btn-ghost-danger",
title: "Delete target",
onclick: move |_| {
confirm_delete.set(Some((id_del.clone(), name_del.clone())));
},
Icon { icon: BsTrash, width: 16, height: 16 }
}
}
}
if is_expanded {
tr {
td { colspan: "6",
div { style: "padding: 12px 8px;",
h4 { style: "margin: 0 0 6px;", "Artifacts" }
if artifacts_detail.is_empty() {
div { style: "opacity: 0.6;", "No artifacts." }
}
for a in artifacts_detail {
div { style: "font-size: 0.9em; padding: 2px 0;",
span { style: "opacity: 0.7;", "{str_at(&a, \"kind\")}: " }
span { style: "font-family: monospace;", "{str_at(&a, \"source_ref\")}" }
}
}
if !facts.is_empty() {
h4 { style: "margin: 12px 0 6px;", "Detected facts" }
for f in facts {
div { style: "font-size: 0.9em; padding: 2px 0;",
span { style: "font-family: monospace;", "{str_at(&f, \"key\")}={str_at(&f, \"value\")}" }
span { style: "opacity: 0.5;", " ({str_at(&f, \"source\")})" }
}
}
}
h4 { style: "margin: 12px 0 6px;", "Applicable scans" }
TargetScans { id: id.clone() }
}
}
}
}
}
}
}
}
}
}
}
}
}
}
Some(None) => rsx! {
div { class: "card", p { "Failed to load targets." } }
},
None => rsx! {
div { class: "loading", "Loading targets..." }
},
}
}
}
}