docs: added vite-press docs (#4)
Co-authored-by: Sharang Parnerkar <parnerkarsharang@gmail.com> Reviewed-on: #4
This commit was merged in pull request #4.
This commit is contained in:
125
docs/deployment/docker.md
Normal file
125
docs/deployment/docker.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# Docker Compose Deployment
|
||||
|
||||
The recommended way to deploy Compliance Scanner is with Docker Compose.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- At least 4 GB of available RAM
|
||||
- Git repository access (tokens configured in `.env`)
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone <repo-url> compliance-scanner
|
||||
cd compliance-scanner
|
||||
|
||||
# Configure environment
|
||||
cp .env.example .env
|
||||
# Edit .env with your MongoDB credentials, tokens, etc.
|
||||
|
||||
# Start all services
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Services
|
||||
|
||||
The `docker-compose.yml` includes these services:
|
||||
|
||||
| Service | Port | Description |
|
||||
|---------|------|-------------|
|
||||
| `mongo` | 27017 | MongoDB database |
|
||||
| `agent` | 3001, 3002 | Compliance agent (REST API + webhooks) |
|
||||
| `dashboard` | 8080 | Web dashboard |
|
||||
| `chromium` | 3003 | Headless browser for DAST crawling |
|
||||
| `otel-collector` | 4317, 4318 | OpenTelemetry collector (optional) |
|
||||
|
||||
## Volumes
|
||||
|
||||
| Volume | Purpose |
|
||||
|--------|---------|
|
||||
| `mongo_data` | Persistent MongoDB data |
|
||||
| `repos_data` | Cloned repository files |
|
||||
|
||||
## Checking Status
|
||||
|
||||
```bash
|
||||
# View running services
|
||||
docker-compose ps
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f agent
|
||||
docker-compose logs -f dashboard
|
||||
|
||||
# Restart a service
|
||||
docker-compose restart agent
|
||||
```
|
||||
|
||||
## Accessing the Dashboard
|
||||
|
||||
Once running, open [http://localhost:8080](http://localhost:8080) in your browser.
|
||||
|
||||
If Keycloak authentication is configured, you'll be redirected to sign in. Otherwise, the dashboard is accessible directly.
|
||||
|
||||
## Updating
|
||||
|
||||
```bash
|
||||
# Pull latest changes
|
||||
git pull
|
||||
|
||||
# Rebuild and restart
|
||||
docker-compose up -d --build
|
||||
```
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### MongoDB
|
||||
|
||||
For production, use a managed MongoDB instance or configure replication:
|
||||
|
||||
```bash
|
||||
MONGODB_URI=mongodb+srv://user:pass@cluster.mongodb.net/compliance_scanner
|
||||
```
|
||||
|
||||
### Reverse Proxy
|
||||
|
||||
Place the dashboard behind a reverse proxy (nginx, Caddy, Traefik) with TLS:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name compliance.example.com;
|
||||
|
||||
ssl_certificate /path/to/cert.pem;
|
||||
ssl_certificate_key /path/to/key.pem;
|
||||
|
||||
location / {
|
||||
proxy_pass http://localhost:8080;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Resource Limits
|
||||
|
||||
Add resource limits to Docker Compose for production:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
agent:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 2G
|
||||
cpus: '2.0'
|
||||
dashboard:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 512M
|
||||
cpus: '1.0'
|
||||
```
|
||||
84
docs/deployment/environment.md
Normal file
84
docs/deployment/environment.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# Environment Variables
|
||||
|
||||
Complete reference for all environment variables. See [Configuration](/guide/configuration) for detailed descriptions of each variable.
|
||||
|
||||
## Required
|
||||
|
||||
```bash
|
||||
# MongoDB connection
|
||||
MONGODB_URI=mongodb://root:example@localhost:27017/compliance_scanner?authSource=admin
|
||||
```
|
||||
|
||||
## Agent
|
||||
|
||||
```bash
|
||||
AGENT_PORT=3001
|
||||
SCAN_SCHEDULE=0 0 */6 * * *
|
||||
CVE_MONITOR_SCHEDULE=0 0 0 * * *
|
||||
GIT_CLONE_BASE_PATH=/tmp/compliance-scanner/repos
|
||||
MONGODB_DATABASE=compliance_scanner
|
||||
```
|
||||
|
||||
## Dashboard
|
||||
|
||||
```bash
|
||||
DASHBOARD_PORT=8080
|
||||
AGENT_API_URL=http://localhost:3001
|
||||
```
|
||||
|
||||
## LLM / AI
|
||||
|
||||
```bash
|
||||
LITELLM_URL=http://localhost:4000
|
||||
LITELLM_API_KEY=
|
||||
LITELLM_MODEL=gpt-4o
|
||||
LITELLM_EMBED_MODEL=text-embedding-3-small
|
||||
```
|
||||
|
||||
## Git Providers
|
||||
|
||||
```bash
|
||||
# GitHub
|
||||
GITHUB_TOKEN=
|
||||
GITHUB_WEBHOOK_SECRET=
|
||||
|
||||
# GitLab
|
||||
GITLAB_URL=https://gitlab.com
|
||||
GITLAB_TOKEN=
|
||||
GITLAB_WEBHOOK_SECRET=
|
||||
```
|
||||
|
||||
## Issue Trackers
|
||||
|
||||
```bash
|
||||
# Jira
|
||||
JIRA_URL=
|
||||
JIRA_EMAIL=
|
||||
JIRA_API_TOKEN=
|
||||
JIRA_PROJECT_KEY=
|
||||
```
|
||||
|
||||
## External Services
|
||||
|
||||
```bash
|
||||
SEARXNG_URL=http://localhost:8888
|
||||
NVD_API_KEY=
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```bash
|
||||
KEYCLOAK_URL=http://localhost:8080
|
||||
KEYCLOAK_REALM=compliance
|
||||
KEYCLOAK_CLIENT_ID=compliance-dashboard
|
||||
REDIRECT_URI=http://localhost:8080/auth/callback
|
||||
APP_URL=http://localhost:8080
|
||||
```
|
||||
|
||||
## Observability
|
||||
|
||||
```bash
|
||||
# Set to enable OpenTelemetry export (omit to disable)
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
OTEL_SERVICE_NAME=compliance-agent
|
||||
```
|
||||
104
docs/deployment/keycloak.md
Normal file
104
docs/deployment/keycloak.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# Keycloak Authentication
|
||||
|
||||
Compliance Scanner supports Keycloak for SSO authentication. When configured, all dashboard access requires signing in through Keycloak, and all API endpoints are protected.
|
||||
|
||||
## How It Works
|
||||
|
||||
### Dashboard (OAuth2/OIDC)
|
||||
|
||||
The dashboard implements a standard OAuth2 Authorization Code flow with PKCE:
|
||||
|
||||
1. User visits the dashboard
|
||||
2. If not authenticated, a login page shows with a "Sign in with Keycloak" button
|
||||
3. User is redirected to Keycloak's login page
|
||||
4. After authentication, Keycloak redirects back with an authorization code
|
||||
5. The dashboard exchanges the code for tokens and creates a session
|
||||
6. All subsequent `/api/` server function calls require a valid session
|
||||
|
||||
### Agent API (JWT)
|
||||
|
||||
The agent API validates JWT Bearer tokens from Keycloak:
|
||||
|
||||
1. Dashboard (or other clients) include the access token in requests: `Authorization: Bearer <token>`
|
||||
2. The agent fetches Keycloak's JWKS (JSON Web Key Set) to validate the token signature
|
||||
3. Token expiry and claims are verified
|
||||
4. The health endpoint (`/api/v1/health`) is always public
|
||||
|
||||
If `KEYCLOAK_URL` and `KEYCLOAK_REALM` are not set on the agent, JWT validation is disabled and all endpoints are open.
|
||||
|
||||
## Keycloak Setup
|
||||
|
||||
### 1. Create a Realm
|
||||
|
||||
In the Keycloak admin console:
|
||||
|
||||
1. Create a new realm (e.g. `compliance`)
|
||||
2. Note the realm name — you'll need it for `KEYCLOAK_REALM`
|
||||
|
||||
### 2. Create a Client
|
||||
|
||||
1. Go to **Clients** > **Create client**
|
||||
2. Set:
|
||||
- **Client ID**: `compliance-dashboard`
|
||||
- **Client type**: OpenID Connect
|
||||
- **Client authentication**: Off (public client)
|
||||
3. Under **Settings**:
|
||||
- **Valid redirect URIs**: `http://localhost:8080/auth/callback` (adjust for your domain)
|
||||
- **Valid post logout redirect URIs**: `http://localhost:8080`
|
||||
- **Web origins**: `http://localhost:8080`
|
||||
|
||||
### 3. Create Users
|
||||
|
||||
1. Go to **Users** > **Create user**
|
||||
2. Set username, email, first name, last name
|
||||
3. Under **Credentials**, set a password
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Keycloak server URL (no trailing slash)
|
||||
KEYCLOAK_URL=http://localhost:8080
|
||||
|
||||
# Realm name
|
||||
KEYCLOAK_REALM=compliance
|
||||
|
||||
# Client ID (must match the client created above)
|
||||
KEYCLOAK_CLIENT_ID=compliance-dashboard
|
||||
|
||||
# OAuth callback URL (must match valid redirect URI in Keycloak)
|
||||
REDIRECT_URI=http://localhost:8080/auth/callback
|
||||
|
||||
# Application root URL (used for post-logout redirect)
|
||||
APP_URL=http://localhost:8080
|
||||
```
|
||||
|
||||
## Dashboard Features
|
||||
|
||||
When authenticated, the dashboard shows:
|
||||
|
||||
- **User avatar** in the sidebar (from Keycloak profile picture, or initials)
|
||||
- **User name** from Keycloak profile
|
||||
- **Logout** link that clears the session and redirects through Keycloak's logout flow
|
||||
|
||||
## Session Configuration
|
||||
|
||||
Sessions use signed cookies with these defaults:
|
||||
|
||||
- **Expiry**: 24 hours of inactivity
|
||||
- **SameSite**: Lax (required for Keycloak redirect flow)
|
||||
- **Secure**: Disabled by default (enable behind HTTPS)
|
||||
- **Storage**: In-memory (resets on server restart)
|
||||
|
||||
::: tip
|
||||
For production, consider persisting sessions to Redis or a database so they survive server restarts.
|
||||
:::
|
||||
|
||||
## Running Without Keycloak
|
||||
|
||||
If no Keycloak variables are set:
|
||||
|
||||
- The **dashboard** serves without authentication (all pages accessible)
|
||||
- The **agent API** accepts all requests without token validation
|
||||
- A warning is logged: `Keycloak not configured - API endpoints are unprotected`
|
||||
|
||||
This is suitable for local development and testing.
|
||||
139
docs/deployment/opentelemetry.md
Normal file
139
docs/deployment/opentelemetry.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# OpenTelemetry Observability
|
||||
|
||||
Compliance Scanner exports traces and logs via OpenTelemetry Protocol (OTLP) for integration with observability platforms like SigNoz, Grafana (Tempo + Loki), Jaeger, and others.
|
||||
|
||||
## Enabling
|
||||
|
||||
Set the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable to enable OTLP export:
|
||||
|
||||
```bash
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
```
|
||||
|
||||
When this variable is not set, telemetry export is disabled and only console logging is active.
|
||||
|
||||
## What Is Exported
|
||||
|
||||
### Traces
|
||||
|
||||
Distributed traces for:
|
||||
|
||||
- HTTP request handling (via `tower-http` `TraceLayer`)
|
||||
- Database operations
|
||||
- Scan pipeline phases
|
||||
- External API calls (LiteLLM, Keycloak, Git providers)
|
||||
|
||||
### Logs
|
||||
|
||||
All `tracing::info!`, `tracing::warn!`, `tracing::error!` log events are exported as OTel log records, including structured fields.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Collector gRPC endpoint | *(disabled)* |
|
||||
| `OTEL_SERVICE_NAME` | Service name in traces | `compliance-agent` or `compliance-dashboard` |
|
||||
| `RUST_LOG` | Log level filter | `info` |
|
||||
|
||||
## Docker Compose Setup
|
||||
|
||||
The included `docker-compose.yml` provides an OTel Collector service:
|
||||
|
||||
```yaml
|
||||
otel-collector:
|
||||
image: otel/opentelemetry-collector-contrib:latest
|
||||
ports:
|
||||
- "4317:4317" # gRPC
|
||||
- "4318:4318" # HTTP
|
||||
volumes:
|
||||
- ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
|
||||
```
|
||||
|
||||
The agent and dashboard are pre-configured to send telemetry to the collector:
|
||||
|
||||
```yaml
|
||||
agent:
|
||||
environment:
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4317
|
||||
OTEL_SERVICE_NAME: compliance-agent
|
||||
|
||||
dashboard:
|
||||
environment:
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT: http://otel-collector:4317
|
||||
OTEL_SERVICE_NAME: compliance-dashboard
|
||||
```
|
||||
|
||||
## Collector Configuration
|
||||
|
||||
Edit `otel-collector-config.yaml` to configure your backend. The default exports to debug (stdout) only.
|
||||
|
||||
### SigNoz
|
||||
|
||||
```yaml
|
||||
exporters:
|
||||
otlp/signoz:
|
||||
endpoint: "signoz-otel-collector:4317"
|
||||
tls:
|
||||
insecure: true
|
||||
|
||||
service:
|
||||
pipelines:
|
||||
traces:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [otlp/signoz]
|
||||
logs:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [otlp/signoz]
|
||||
```
|
||||
|
||||
### Grafana Tempo (Traces) + Loki (Logs)
|
||||
|
||||
```yaml
|
||||
exporters:
|
||||
otlp/tempo:
|
||||
endpoint: "tempo:4317"
|
||||
tls:
|
||||
insecure: true
|
||||
loki:
|
||||
endpoint: "http://loki:3100/loki/api/v1/push"
|
||||
|
||||
service:
|
||||
pipelines:
|
||||
traces:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [otlp/tempo]
|
||||
logs:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [loki]
|
||||
```
|
||||
|
||||
### Jaeger
|
||||
|
||||
```yaml
|
||||
exporters:
|
||||
otlp/jaeger:
|
||||
endpoint: "jaeger:4317"
|
||||
tls:
|
||||
insecure: true
|
||||
|
||||
service:
|
||||
pipelines:
|
||||
traces:
|
||||
receivers: [otlp]
|
||||
processors: [batch]
|
||||
exporters: [otlp/jaeger]
|
||||
```
|
||||
|
||||
## Verifying
|
||||
|
||||
After starting with telemetry enabled, look for this log on startup:
|
||||
|
||||
```
|
||||
OpenTelemetry OTLP export enabled endpoint=http://otel-collector:4317 service=compliance-agent
|
||||
```
|
||||
|
||||
If the endpoint is unreachable, the application still starts normally — telemetry export fails silently without affecting functionality.
|
||||
Reference in New Issue
Block a user