refactor(iac): cluster split — 1 cluster per plane, breakpilot-* naming
ci / shared (pull_request) Successful in 23s
ci / validate (pull_request) Successful in 4s

Restructures the draft to reflect the 2026-06-30 cluster decision:

Three Orca clusters, each becoming its own Gitea repo at migration time:
- breakpilot-edge   → vm-edge       (Identity + Infra: KC, Gitea, Infisical, PowerDNS, Orca-Proxy)
- breakpilot-control → vm-control   (Portal, tenant-registry, ERPNext, MariaDB, Stalwart)
- breakpilot-app    → vm-app-prod + vm-app-stage (CERTifAI, compliance-*, Mongo, MinIO, Qdrant, LiteLLM)

Key model points encoded:
- Identity (Keycloak) co-tenant with Infra on vm-edge (1 VM core), per
  INFRASTRUCTURE.md §6 — heap pinned so it cannot starve PowerDNS/Infisical
- Stage and prod live in the same breakpilot-app cluster on different
  VMs. Stage authenticates via prod Keycloak under tenant.kind = "stage";
  no duplicated identity, no duplicated control plane (per §5).
- The existing CERTifAI Keycloak will be repurposed for breakpilot-edge
  rather than standing up a fresh one — same realm, same users.
- Multi-VM rollout gated on legal entity being established so we can sign
  SysEleven / Hetzner business contracts. Until then, single-VM ops
  continues via ~/workspace/orca-infra; this repo is design-only.

Mechanical changes:
- manifests/{vm-edge,vm-control,vm-data,stage}/ → clusters/{breakpilot-edge,breakpilot-control,breakpilot-app/services/{prod,stage}}/services/
- vm-data → vm-app-prod, stage → vm-app-stage in node references and headers
- overlays/{stage,prod}/overlay.toml point at the new cluster paths
- scripts/validate.sh now enforces a per-cluster node whitelist
  (breakpilot-edge → vm-edge, breakpilot-control → vm-control,
  breakpilot-app → {vm-app-prod, vm-app-stage}) instead of dir-name equality
- New READMEs at clusters/, clusters/breakpilot-edge/,
  clusters/breakpilot-control/, clusters/breakpilot-app/ documenting
  scope, SLA targets, co-tenant notes, and the future-repo split
- Top README rewritten to lead with the cluster-split decision and the
  legal-entity gate; per-milestone fill-in table re-pathed

Validation:
- make validate → 38 files OK (35 manifests + 3 overlays)
- make plan ENV=stage → 11 resolved manifests in .orca-out/stage/
- make plan ENV=prod  → 24 resolved manifests in .orca-out/prod/
This commit is contained in:
Sharang Parnerkar
2026-06-30 22:14:51 +02:00
parent f1c3fd14b9
commit 8971152da0
44 changed files with 428 additions and 125 deletions
+96 -45
View File
@@ -1,6 +1,6 @@
# orca-platform
IaC for the Breakpilot Platform: per-VM Orca service manifests, per-env overlays, DNS zones, backup/restore tooling, and the `make plan`/`make apply` wrappers.
IaC staging for the Breakpilot Platform: per-cluster Orca service manifests, per-env overlays, DNS zones, backup/restore tooling, and the `make plan` / `make apply` wrappers.
> Part of the **Breakpilot Platform**. For the big picture see [`platform/docs`](https://gitea.meghsakha.com/platform/docs):
> [Architecture](https://gitea.meghsakha.com/platform/docs/src/branch/main/PLATFORM_ARCHITECTURE.md) ·
@@ -9,43 +9,94 @@ IaC for the Breakpilot Platform: per-VM Orca service manifests, per-env overlays
## What this is
The single source of truth for which container runs on which VM in which environment. Every change to prod infrastructure should flow through this repo — never through `orca deploy` from a laptop.
The staging area for the future per-cluster Gitea repos. Every change to prod infrastructure will eventually flow through one of `platform/breakpilot-{edge,control,app}` — until those repos exist, every change to the *design* flows through this one.
**Plane:** Infra
**Plane:** Infra (design only — no live cluster yet)
**Owner:** @sharang
**Status:** pre-alpha (M1.1 — layout only; real values land per the per-milestone schedule below)
**Linked milestone:** [M1.1](https://gitea.meghsakha.com/platform/docs/src/branch/main/IMPLEMENTATION_PLAN.md)
**Status:** pre-alpha — manifests reflect the 2026-06-30 cluster-split decision; real values land per the milestone schedule below
## The cluster split (decided 2026-06-30)
Three Orca clusters, each becoming its own Gitea repo at migration time:
| Cluster | Future repo | VMs (steady state) | Plane |
|---|---|---|---|
| `breakpilot-edge` | `platform/breakpilot-edge` | `vm-edge` (1) | Identity + Infra |
| `breakpilot-control` | `platform/breakpilot-control` | `vm-control` (1) | Control |
| `breakpilot-app` | `platform/breakpilot-app` | `vm-app-prod` + `vm-app-stage` (2) | App |
Identity (Keycloak) is part of `breakpilot-edge` on the **same VM** as the
rest of infra — single-VM core, JVM heap pinned so it cannot starve
PowerDNS/Infisical. The Keycloak instance already running for CERTifAI
will be repurposed for `breakpilot-edge` rather than standing up a new
one.
Stage and prod share the **same** `breakpilot-app` cluster but live on
**different VMs** (`vm-app-stage` is a small Hetzner CX22-class box).
Same orca config, no config drift; physically isolated, no "oops touched
prod" accidents. Stage authenticates via prod Keycloak and reads tenant
config from prod `tenant-registry` under `tenant.kind = "stage"`
no duplicated identity, no duplicated control plane.
### Gate: legal entity
Multi-VM rollout is gated on the legal entity being established so we
can sign SysEleven / Hetzner business contracts. Until then, everything
runs on the existing single VM via the `~/workspace/orca-infra` repo and
the manifests in this repo are design-only. `make validate` keeps the
design honest; `make apply` is a no-op for now.
When the entity lands the migration is mechanical: split each
`clusters/<name>/` subdir into its own Gitea repo, point its
`cluster.toml` at a real VM, run `orca deploy`. See
[`clusters/README.md`](./clusters/README.md) for the migration order.
## Directory layout
```
.
├── manifests/ # Base service.toml per VM × service (35 stubs)
│ ├── vm-edge/ Identity + Infra plane services
│ ├── vm-control/ Control plane services
│ ├── vm-data/ Data plane services
│ └── stage/ Stage (app plane only)
├── overlays/ # Per-env sparse deltas applied on top of manifests/
│ ├── dev/overlay.toml no-op; dev runs docker-compose per-service
├── stage/overlay.toml include manifests/stage/, image_tag=env-stage
│ └── prod/overlay.toml include vm-{edge,control,data}, image_tag=env-prod
├── clusters/ # one subdir per future cluster repo
│ ├── README.md the cluster split + migration plan
│ ├── breakpilot-edge/ Identity + Infra (vm-edge)
│ ├── README.md
│ └── services/ (8 services)
│ ├── breakpilot-control/ Control plane (vm-control)
│ ├── README.md
│ └── services/ (7 services)
│ └── breakpilot-app/ App plane (vm-app-prod + vm-app-stage)
│ ├── README.md
│ └── services/
│ ├── prod/ (9 services on vm-app-prod)
│ └── stage/ (11 services on vm-app-stage)
├── overlays/ # per-env sparse deltas
│ ├── dev/overlay.toml no-op; dev runs docker-compose per-service
│ ├── stage/overlay.toml → clusters/breakpilot-app/services/stage
│ └── prod/overlay.toml → all three clusters' prod paths
├── dns/
│ └── breakpilot.com.zone.template PowerDNS zone — body lands in M0.3
├── cluster.toml.tmpl # Cluster-level config (acme_email, backup, ai); rendered per env
├── cluster.toml.tmpl # cluster-level template (acme_email, backup, ai)
├── scripts/
│ ├── validate.sh # `make validate`
│ ├── plan.sh # `make plan ENV=<env>` → .orca-out/<env>/
│ ├── apply.sh # `make apply ENV=<env>` (no-op until M1.2)
│ └── restore-drill.sh.template M1.3 placeholder
── Makefile # validate / plan / apply / diff / clean
│ ├── validate.sh # `make validate` — TOML + per-cluster node check
│ ├── plan.sh # `make plan ENV=<env>` → .orca-out/<env>/
│ ├── apply.sh # `make apply ENV=<env>` (no-op until M1.2)
│ └── restore-drill.sh.template # M1.3 placeholder
── dev/ # local docker-compose for product devs
└── Makefile # validate / plan / apply / diff / clean
```
`scripts/validate.sh` enforces that every service's `placement.node`
matches one of the cluster's declared VMs:
- `breakpilot-edge``vm-edge`
- `breakpilot-control``vm-control`
- `breakpilot-app``vm-app-prod` or `vm-app-stage`
## Run locally
### IaC validation
```bash
make validate # check all manifests parse + have required fields
make validate # check all manifests parse + have required fields + cluster-correct nodes
make plan ENV=stage # resolve manifests for stage → .orca-out/stage/
make plan ENV=prod # same for prod
make apply ENV=stage # no-op until M1.2 stands up the Orca controller
@@ -67,42 +118,42 @@ Seed user: `test@breakpilot.dev` / `test` (tenant `acme`, products `certifai` +
## Per-milestone fill-in schedule
Each stub manifest in `manifests/` carries a header comment naming the milestone that finalises its real values. Summary:
Each stub manifest carries a header comment naming the milestone that finalises its real values. Summary:
| Milestone | What it fills in |
| Milestone | What it fills in |
|---|---|
| **M0.3** | `vm-edge/powerdns-auth.toml`, DNS zone body, orca-proxy routes |
| **M0.3** | `breakpilot-edge/services/powerdns-auth.toml`, DNS zone body, orca-proxy routes |
| **M1.2** | VM provisioning (Terraform/OpenStack in a separate repo); brings `make apply` online |
| **M1.3** | Backup cron services + `scripts/restore-drill.sh` |
| **M2.1** | `vm-edge/keycloak.toml` + `pg-keycloak.toml` |
| **M3.1** | `vm-edge/infisical.toml` + `pg-infisical.toml` + `redis-infisical.toml` |
| **M3.2** | `vm-control/stalwart.toml` |
| **M4.1** | `vm-control/tenant-registry.toml` + `vm-data/pg-app.toml` |
| **M5.1** | `vm-control/customer-portal.toml` + stage equivalents |
| **M6.x** | `vm-data/certifai-dashboard.toml`, `mongodb.toml`, `litellm.toml` |
| **M7.x** | compliance services on vm-data + stage |
| **M8.1** | `vm-control/erpnext.toml`, `mariadb.toml`, `redis-erpnext.toml` |
| **M9.1** | `vm-control/frappe-hd.toml` |
| **M1.3** | Backup cron services + `scripts/restore-drill.sh` |
| **M2.1** | `breakpilot-edge/services/{keycloak,pg-keycloak}.toml` (repurposed from CERTifAI Keycloak) |
| **M3.1** | `breakpilot-edge/services/{infisical,pg-infisical,redis-infisical}.toml` |
| **M3.2** | `breakpilot-control/services/stalwart.toml` |
| **M4.1** | `breakpilot-control/services/tenant-registry.toml` + `breakpilot-app/services/prod/pg-app.toml` |
| **M5.1** | `breakpilot-control/services/customer-portal.toml` + stage equivalents |
| **M6.x** | `breakpilot-app/services/prod/{certifai-dashboard,mongodb,litellm}.toml` |
| **M7.x** | compliance services on `breakpilot-app/services/{prod,stage}/` |
| **M8.1** | `breakpilot-control/services/{erpnext,mariadb,redis-erpnext}.toml` |
| **M9.1** | `breakpilot-control/services/frappe-hd.toml` |
Until the milestone PR lands, the stub still parses and `make validate` stays green — but `apply` will refuse a stub that hasn't replaced its `placeholder` image tag (gate to be added with the first real image).
## Endpoints / surface
| Target | What it does |
| Target | What it does |
|---|---|
| `make validate` | Parse + structural check (no cluster contact) |
| `make plan ENV=<env>` | Resolve manifests + overlay → `.orca-out/<env>/` |
| `make apply ENV=<env>` | Push to Orca controller at `$ORCA_API_URL` (M1.2 brings this online) |
| `make diff ENV=<env>` | Alias for `plan` |
| `make clean` | Remove `.orca-out/` |
| `make validate` | Parse + structural check + per-cluster node check (no cluster contact) |
| `make plan ENV=<env>` | Resolve manifests + overlay → `.orca-out/<env>/` |
| `make apply ENV=<env>` | Push to Orca controller at `$ORCA_API_URL` (M1.2 brings this online) |
| `make diff ENV=<env>` | Alias for `plan` |
| `make clean` | Remove `.orca-out/` |
## Deployment
| Env | Apply path | Trigger |
| Env | Apply path | Trigger |
|---|---|---|
| dev | `docker-compose` in each product repo | dev's machine |
| stage | `make apply ENV=stage` against the stage Orca controller | CI on merge to main + image build |
| prod | `make apply ENV=prod` against the prod Orca controller | release tag `vX.Y.Z` + sign-off |
| dev | `docker-compose` in each product repo | dev's machine |
| stage | `make apply ENV=stage` against the `breakpilot-app` Orca controller | CI on merge to main + image build |
| prod | `make apply ENV=prod` against the `breakpilot-app` Orca controller | release tag `vX.Y.Z` + sign-off |
`apply` for prod will be gated by the production-promotion gate (24h stage soak + manual sign-off) per `IMPLEMENTATION_PLAN.md §1.6`. Wiring lands in M1.2.
@@ -113,7 +164,7 @@ Until the milestone PR lands, the stub still parses and `make validate` stays gr
## Contributing
See [`CONTRIBUTING.md`](./CONTRIBUTING.md). Every PR touching `manifests/` MUST keep `make validate` green; CI enforces it.
See [`CONTRIBUTING.md`](./CONTRIBUTING.md). Every PR touching `clusters/` MUST keep `make validate` green; CI enforces it.
## License
+85
View File
@@ -0,0 +1,85 @@
# Clusters
Each subdirectory under `clusters/` is the manifest set for one Orca cluster.
At migration time each will become its own Gitea repository under
`platform/` so the cluster boundary doubles as a repository boundary.
| Subdir | Future repo | VMs | Plane |
|---|---|---|---|
| [`breakpilot-edge/`](./breakpilot-edge/) | `platform/breakpilot-edge` | `vm-edge` (1) | Identity + Infra |
| [`breakpilot-control/`](./breakpilot-control/) | `platform/breakpilot-control` | `vm-control` (1) | Control |
| [`breakpilot-app/`](./breakpilot-app/) | `platform/breakpilot-app` | `vm-app-prod` + `vm-app-stage` (2) | App (was "Data") |
## Why one cluster per plane
- **Independent failure domain.** A runaway reconcile loop in `breakpilot-app`
cannot restart Keycloak. An ERPNext migration on `breakpilot-control` does
not stall portal logins.
- **Independent backup policy.** Each cluster has its own `[backup]` block
and its own S3 bucket so RPO targets in `INFRASTRUCTURE.md §6` can be set
per plane without cross-talk.
- **Independent IaC.** Each future repo has its own CI, CODEOWNERS, and
reconcile cadence. Touching identity does not gate a portal hotfix.
- **Same Orca everywhere.** Per-node ingress (every Orca node binds 80/443
with its own ACME) means each cluster's services get A records pointing
at their own VM. No central proxy SPOF.
## Why this is still a single repo today
We hold the three-cluster design in this one staging repo until the legal
entity is established and we can sign SysEleven / Hetzner business
contracts to provision actual VMs. Until then, everything runs on the
existing single VM under `orca-infra` and these manifests are
design-only — `make validate` keeps them honest, `make apply` is a no-op
for now.
When the entity lands the migration is mechanical: split each subdir into
its own Gitea repo, point its `cluster.toml` at a real VM, run
`orca deploy`.
## VM topology after migration
```
breakpilot-edge cluster → vm-edge (Keycloak + Gitea + Infisical + PowerDNS + Orca-Proxy)
breakpilot-control cluster → vm-control (Portal + Tenant-Registry + ERPNext + MariaDB + Stalwart)
breakpilot-app cluster → vm-app-prod (CERTifAI + compliance-* + Mongo + MinIO + Qdrant + LiteLLM)
\→ vm-app-stage (slim parallel set for staging)
```
`vm-app-stage` lives in the same Orca cluster as `vm-app-prod` and points
at PROD `breakpilot-edge` Keycloak + PROD `breakpilot-control`
tenant-registry, isolated by `tenant.kind = "stage"` per
`INFRASTRUCTURE.md §5`. No duplicated identity, no duplicated control plane.
## Identity is part of the core (single VM)
The May 18 `INFRASTRUCTURE.md` had Identity (Keycloak) co-tenant with Infra
on `vm-edge`, with JVM heap pinned at 1.5 GB so it cannot starve PowerDNS
/ Infisical. That decision stands — Identity lives in `breakpilot-edge` on
the same VM as the rest of infra. We will revisit splitting Keycloak onto
its own VM only if the JVM vs. infra fight actually shows up in alerts.
## Repurposing the existing Keycloak
`benjamin_boenisch/certifai` already runs a Keycloak instance. Plan is to
**repurpose that instance into `breakpilot-edge/services/keycloak.toml`**
rather than stand up a fresh one — same realm exports, same client
configurations, same user database. Migration is a hostname swap and a
realm export/import, not a re-bootstrap.
## Adding a new service
Pick the plane, then drop the manifest into the right `services/` dir:
```bash
clusters/breakpilot-<plane>/services/<service>.toml
# breakpilot-app extras:
clusters/breakpilot-app/services/{prod,stage}/<service>.toml
```
`scripts/validate.sh` enforces:
- `placement.node` must match one of the cluster's declared VMs
(`vm-edge` / `vm-control` / `vm-app-prod` / `vm-app-stage`)
- `resources.memory` is mandatory (§8 rule 5)
- no nested `depends_on` / `extra_ports` / `cmd` / `mounts` under
`[service.placement|resources|env|volume]`
+83
View File
@@ -0,0 +1,83 @@
# breakpilot-app
App plane (formerly "Data plane" in the May 18 doc). Two VMs in one cluster:
- **`vm-app-prod`** — production workloads, customer data, real keys
- **`vm-app-stage`** — staging workloads, demo data, sandbox tenant
Becomes its own Gitea repo `platform/breakpilot-app` at migration time.
## Services
### Prod ([`services/prod/`](./services/prod/), 9 services)
| Service | Purpose |
|---|---|
| `admin-compliance.toml` | Compliance back-office admin |
| `ai-compliance-sdk.toml` | LLM-mediated compliance SDK |
| `backend-compliance.toml` | Compliance scanner backend |
| `certifai-dashboard.toml` | CERTifAI dashboard (first product) |
| `litellm.toml` | LLM gateway |
| `minio.toml` | S3-compatible object store |
| `mongodb.toml` | Compliance + CERTifAI data |
| `pg-app.toml` | Per-tenant application Postgres (SPOF — RISK-1 in `§7`) |
| `qdrant.toml` | Vector store (rebuildable) |
### Stage ([`services/stage/`](./services/stage/), 11 services)
| Service | Purpose |
|---|---|
| `admin-compliance.toml` | Stage admin |
| `ai-compliance-sdk.toml` | Stage SDK |
| `backend-compliance.toml` | Stage backend |
| `certifai-dashboard.toml` | Stage CERTifAI |
| `customer-portal.toml` | Stage portal (calls prod KC + prod tenant-registry with `tenant.kind = "stage"`) |
| `litellm.toml` | Stage LLM gateway (may reuse prod) |
| `mongodb-stage.toml` | Stage Mongo |
| `orca-proxy.toml` | Per-node ingress on `vm-app-stage` |
| `pg-app-stage.toml` | Stage Postgres |
| `qdrant-stage.toml` | Stage vector store |
| `tenant-registry.toml` | (placeholder; stage calls prod registry — manifest exists for parity) |
## SLA targets (per `INFRASTRUCTURE.md §6`)
```
Owns: DATA_RESIDENCY — all customer data (MongoDB, pg-app, MinIO) must stay EU
RPO (product) — compliance records ≤ 6h; chat history ≤ 24h
DATA_ISOLATION — every query scoped by org_id / tenant_id
AUDIT_TRAIL — product-level actions
AVAILABILITY — CERTifAI ≥ 99.5%; compliance ≥ 99.5%
```
## Why one cluster, two VMs
- **Same orca-platform config, different physical workloads.** Stage and
prod don't drift on infra config because they're in the same cluster.
- **No "oops touched prod" accidents.** Per-service `placement.node`
pins which VM each container lands on; prod manifests live in `prod/`,
stage in `stage/`.
- **Blast radius is physical.** A stage misconfiguration cannot exhaust
`vm-app-prod`'s memory because they share nothing but the Orca control
socket.
## Stage isolation contract
Stage services never:
- email real customers (Stalwart accept-rule on `breakpilot-control` drops
recipients not matching `*+stage@*`)
- trigger real Polar charges (`POLAR_API_URL` points at sandbox)
- carry real customer data (sandbox tenant resets nightly)
Stage services always:
- authenticate via prod Keycloak with `tenant.kind = "stage"`
- read tenant config from prod `tenant-registry` (read-only for stage tenants)
- expose `*.stage.breakpilot.com` (or whatever the staging subdomain is)
See `INFRASTRUCTURE.md §5` for the full stage-prod sharing contract.
## Sandbox tenant
The `sandbox` tenant in `stage/` is the only place that accepts writes
from public demo visitors. A nightly cron (lands as `sandbox-reset.toml`
when seed-data fixtures are wired in M13.1) resets its Mongo collections
and Keycloak user attributes from a versioned seed.
@@ -1,5 +1,5 @@
# admin-compliance stub — full config lands in M7.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "admin-compliance"
@@ -8,7 +8,7 @@ port = 3002
depends_on = ["backend-compliance", "ai-compliance-sdk"]
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "512Mi"
@@ -1,5 +1,5 @@
# ai-compliance-sdk stub — full config lands in M7.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "ai-compliance-sdk"
@@ -8,7 +8,7 @@ port = 3001
depends_on = ["pg-app", "qdrant", "litellm"]
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# backend-compliance stub — full config lands in M7.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "backend-compliance"
@@ -8,7 +8,7 @@ port = 3000
depends_on = ["pg-app", "minio"]
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# certifai-dashboard stub — full config lands in M6.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "certifai-dashboard"
@@ -8,7 +8,7 @@ port = 3000
depends_on = ["mongodb", "litellm"]
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# litellm stub — full config lands in M6.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "litellm"
@@ -7,7 +7,7 @@ image = "ghcr.io/berriai/litellm:main-stable"
port = 4000
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# minio stub — full config lands in M7.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "minio"
@@ -9,7 +9,7 @@ extra_ports = ["9001:9001"]
cmd = ["server", "/data", "--console-address", ":9001"]
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# mongodb stub — full config lands in M6.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "mongodb"
@@ -7,7 +7,7 @@ image = "mongo:7"
port = 27017
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "2Gi"
@@ -1,5 +1,5 @@
# pg-app stub — full config lands in M4.1.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# RISK-1 (§12): single instance owns tenant_registry + compliance schemas. Split into pg-registry + pg-compliance at Tier B.
[[service]]
@@ -8,7 +8,7 @@ image = "postgres:16-alpine"
port = 5432
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "3Gi"
@@ -1,5 +1,5 @@
# qdrant stub — full config lands in M7.x.
# Host: vm-data. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-prod. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "qdrant"
@@ -7,7 +7,7 @@ image = "qdrant/qdrant:v1.10.0"
port = 6333
[service.placement]
node = "vm-data"
node = "vm-app-prod"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# admin-compliance stub — full config lands in M7.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "admin-compliance"
@@ -7,7 +7,7 @@ image = "registry.breakpilot.com/admin-compliance:env-stage"
port = 3002
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "256Mi"
@@ -1,5 +1,5 @@
# ai-compliance-sdk stub — full config lands in M7.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "ai-compliance-sdk"
@@ -7,7 +7,7 @@ image = "registry.breakpilot.com/ai-compliance-sdk:env-stage"
port = 3001
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "512Mi"
@@ -1,5 +1,5 @@
# backend-compliance stub — full config lands in M7.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "backend-compliance"
@@ -7,7 +7,7 @@ image = "registry.breakpilot.com/backend-compliance:env-stage"
port = 3000
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "512Mi"
@@ -1,5 +1,5 @@
# certifai-dashboard stub — full config lands in M6.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "certifai-dashboard"
@@ -7,7 +7,7 @@ image = "registry.breakpilot.com/certifai:env-stage"
port = 3000
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# customer-portal stub — full config lands in M5.1.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "customer-portal"
@@ -8,7 +8,7 @@ port = 3000
domain = "*.stage.breakpilot.com"
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# litellm stub — full config lands in M6.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
[[service]]
name = "litellm"
@@ -7,7 +7,7 @@ image = "ghcr.io/berriai/litellm:main-stable"
port = 4000
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "512Mi"
@@ -1,5 +1,5 @@
# mongodb-stage stub — full config lands in M6.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Ephemeral.
[[service]]
@@ -8,7 +8,7 @@ image = "mongo:7"
port = 27017
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "512Mi"
@@ -1,5 +1,5 @@
# orca-proxy stub — full config lands in M1.2.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Stage proxy only routes to stage app containers.
[[service]]
@@ -8,7 +8,7 @@ image = "orca-managed/orca-proxy:placeholder"
port = 443
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "256Mi"
@@ -1,5 +1,5 @@
# pg-app-stage stub — full config lands in M4.1.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Ephemeral; no backup, no volume; reset on each release.
[[service]]
@@ -8,7 +8,7 @@ image = "postgres:16-alpine"
port = 5432
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "1Gi"
@@ -1,5 +1,5 @@
# qdrant-stage stub — full config lands in M7.x.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Ephemeral, tiny corpus.
[[service]]
@@ -8,7 +8,7 @@ image = "qdrant/qdrant:v1.10.0"
port = 6333
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "512Mi"
@@ -1,5 +1,5 @@
# tenant-registry stub — full config lands in M4.1.
# Host: stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Host: vm-app-stage. Resource budget per INFRASTRUCTURE.md §6 co-tenant notes.
# Calls PROD Keycloak per §2 "Calls OUT to prod"; audience is stage_client_id.
[[service]]
@@ -9,7 +9,7 @@ port = 8090
depends_on = ["pg-app-stage"]
[service.placement]
node = "stage"
node = "vm-app-stage"
[service.resources]
memory = "512Mi"
+47
View File
@@ -0,0 +1,47 @@
# breakpilot-control
Control plane. Single VM (`vm-control`).
Becomes its own Gitea repo `platform/breakpilot-control` at migration time.
## Services (7)
| Service | Purpose |
|---|---|
| [`customer-portal.toml`](./services/customer-portal.toml) | Next.js 15 customer-facing portal |
| [`erpnext.toml`](./services/erpnext.toml) | Internal-only ERP (sales, invoices, fulfillment) |
| [`frappe-hd.toml`](./services/frappe-hd.toml) | Helpdesk on the Frappe stack |
| [`mariadb.toml`](./services/mariadb.toml) | DB for ERPNext + Frappe HD |
| [`redis-erpnext.toml`](./services/redis-erpnext.toml) | ERPNext cache + queue Redis |
| [`stalwart.toml`](./services/stalwart.toml) | Outbound mail relay; per-stage accept-rule blocks real customers from receiving stage mail |
| [`tenant-registry.toml`](./services/tenant-registry.toml) | Tenant glue: orgs, entitlements, API keys, audit. M7.3's `TENANT_REGISTRY_URL` resolves here. |
## SLA targets (per `INFRASTRUCTURE.md §6`)
```
Owns: RPO (tenant) — tenant registry & compliance schemas RPO ≤ 6h
RPO (ERPNext) — sales orders, invoices RPO ≤ 6h
AUDIT_TRAIL — all portal actions (invites, IdP changes, impersonations)
AVAILABILITY — portal ≥ 99.5%; ERPNext ≥ 99% (internal)
RTO (portal) — ≤ 10 min
RTO (ERPNext) — ≤ 60 min
```
## Co-tenant note
ERPNext + Portal + Stalwart on one VM. Orca resource limits per `§6`:
| Service | Memory cap |
|---|---|
| customer-portal | 1 GB |
| erpnext | 6 GB |
| mariadb | 3 GB |
| stalwart | 1 GB |
| tenant-registry | 500 MB |
## Stage callers
Stage workloads in `breakpilot-app/services/stage/` call the **prod**
tenant-registry here under `tenant.kind = "stage"` rather than running a
duplicate. Isolation is by tenant kind, not by replicating the control
plane.
+42
View File
@@ -0,0 +1,42 @@
# breakpilot-edge
Identity + Infra plane. Single VM (`vm-edge`).
Becomes its own Gitea repo `platform/breakpilot-edge` at migration time.
## Services (8)
| Service | Purpose |
|---|---|
| [`gitea.toml`](./services/gitea.toml) | Source forge — every config change to every cluster lands here |
| [`infisical.toml`](./services/infisical.toml) | Secret store. Every other plane's `${secrets.*}` resolves here |
| [`keycloak.toml`](./services/keycloak.toml) | Single tenant of truth for identity. **Repurposed from `certifai` Keycloak — same realm, same users**. |
| [`orca-proxy.toml`](./services/orca-proxy.toml) | Ingress for vm-edge: TLS termination + Let's Encrypt ACME for `*.breakpilot.com` |
| [`pg-infisical.toml`](./services/pg-infisical.toml) | Infisical's Postgres |
| [`pg-keycloak.toml`](./services/pg-keycloak.toml) | Keycloak's Postgres |
| [`powerdns-auth.toml`](./services/powerdns-auth.toml) | Authoritative DNS for `breakpilot.com`; bound to port 53 on vm-edge |
| [`redis-infisical.toml`](./services/redis-infisical.toml) | Cache + session store for Infisical |
## SLA targets (per `INFRASTRUCTURE.md §6`)
```
Owns: AVAILABILITY ≥ 99.5% (root dep for every other plane)
RTO ≤ 15 min (Keycloak)
AUDIT_TRAIL — realm-level audit (logins, token issuance, IdP events)
DATA_RESIDENCY — Keycloak realm data must stay EU
STAGE_ISOLATION — rate-limits stage_client_id; rejects stage JWTs in prod audiences
```
## Co-tenant note
Keycloak's JVM heap is pinned to 1.5 GB in `keycloak.toml`
(`JAVA_OPTS_APPEND = "-Xms1g -Xmx1500m"`) so it cannot starve PowerDNS or
Infisical. If the heap fight ever shows up in alerts, the escape hatch is
to peel Keycloak off onto its own VM inside the same cluster — no schema
migration, just a `placement.node` flip.
## Bootstrap exception
Per `INFRASTRUCTURE.md §8 rule 3`, `KC_DB_URL` lives in Orca env (not in
Infisical), because Infisical runs on the same VM and we'd otherwise have
a circular bootstrap dependency.
-28
View File
@@ -1,28 +0,0 @@
# Manifests
One `service.toml` per service, grouped by host VM, per `INFRASTRUCTURE.md §2`.
| Directory | VM | Plane(s) | Owner milestone of "real" config |
|---|---|---|---|
| `vm-edge/` | vm-edge | Identity + Infra | M2.1 (Keycloak), M3.1 (Infisical), M0.3 (PowerDNS), M2.x (Gitea), M1.2 (proxy) |
| `vm-control/` | vm-control | Control | M5.1 (portal), M4.1 (tenant-registry), M8.1 (ERPNext), M3.2 (Stalwart) |
| `vm-data/` | vm-data | Data | M6.x (CERTifAI), M7.x (compliance), M4.1 (pg-app) |
| `stage/` | stage | App plane only | promotion target of stage builds |
Each file in this directory is currently a **shape-only stub** — fields are set but image references and env wiring will be finalised by the milestone listed in the file header.
## Adding a new service
1. Pick the owning VM per `INFRASTRUCTURE.md §2`.
2. Create `<vm-name>/<service-name>.toml` following the shape of an existing stub.
3. Set `placement.node = "<vm-name>"`, `resources.memory`/`cpu` per the co-tenant budget in `INFRASTRUCTURE.md §6`.
4. Reference secrets as `${secrets.NAME}` — Infisical resolves these. No plaintext values except the Keycloak bootstrap DB URI exception (`INFRASTRUCTURE.md §8 rule 3`).
5. Run `make validate` before pushing.
## Validation
`make validate` parses every TOML and checks required fields (`name`, image OR build OR module, `placement.node`, `resources.memory`). It does NOT contact a running cluster.
`make plan ENV=<env>` merges the base manifest with the matching overlay in `overlays/<env>/` and prints the resulting service definitions. It is a no-op until matching overlays exist for the env.
`make apply ENV=<env>` is gated on a real Orca controller URL — refuses to run until `ORCA_API_URL` is set (lands in M1.2).
+11 -3
View File
@@ -1,14 +1,22 @@
# Prod overlay.
#
# Selects manifests under vm-edge / vm-control / vm-data. Stage manifests
# (manifests/stage/) are excluded from prod apply.
# Selects services from all three clusters' prod paths:
# - breakpilot-edge: identity + infra (Keycloak, Gitea, Infisical, PowerDNS, Orca-Proxy)
# - breakpilot-control: portal, tenant-registry, ERPNext, Frappe HD, Stalwart
# - breakpilot-app/services/prod: certifai, compliance-*, Mongo, MinIO, Qdrant, LiteLLM
#
# Stage services (breakpilot-app/services/stage/) are excluded from prod apply.
[env]
name = "prod"
api_url = "${ORCA_PROD_API_URL}"
[deploy]
include_dirs = ["manifests/vm-edge", "manifests/vm-control", "manifests/vm-data"]
include_dirs = [
"clusters/breakpilot-edge/services",
"clusters/breakpilot-control/services",
"clusters/breakpilot-app/services/prod",
]
[image]
# Default tag for prod; release.yaml retags `env-stage` → `v$VERSION` + `env-prod`.
+5 -4
View File
@@ -1,15 +1,16 @@
# Stage overlay.
#
# Stage maps to the single 'stage' VM, app plane only. Selects only the
# services under manifests/stage/.
# Stage maps to vm-app-stage inside the breakpilot-app cluster (sibling of
# vm-app-prod). Stage workloads point at PROD Keycloak + PROD tenant-registry
# with tenant.kind = "stage" — identity is not duplicated, isolation is by
# tenant kind, per INFRASTRUCTURE.md §5.
[env]
name = "stage"
api_url = "${ORCA_STAGE_API_URL}"
# Service filter: only deploy manifests under this directory.
[deploy]
include_dirs = ["manifests/stage"]
include_dirs = ["clusters/breakpilot-app/services/stage"]
[image]
# Default image tag for stage builds. Per-service overrides may land later.
+19 -5
View File
@@ -11,7 +11,16 @@ import sys, tomllib, pathlib
root = pathlib.Path(sys.argv[1])
errs = []
count = 0
for p in sorted(root.glob('manifests/**/*.toml')):
# Cluster → expected node names. Each cluster ships its own VM(s); a service's
# placement.node must be one of the cluster's declared VMs.
CLUSTER_NODES = {
'breakpilot-edge': {'vm-edge'},
'breakpilot-control': {'vm-control'},
'breakpilot-app': {'vm-app-prod', 'vm-app-stage'},
}
for p in sorted(root.glob('clusters/*/services/**/*.toml')):
count += 1
try:
data = tomllib.load(open(p, 'rb'))
@@ -22,6 +31,13 @@ for p in sorted(root.glob('manifests/**/*.toml')):
if not svcs:
errs.append(f'{p}: no [[service]] block')
continue
# cluster name = first path component under clusters/
rel = p.relative_to(root)
cluster = rel.parts[1] # clusters/<cluster>/services/...
expected_nodes = CLUSTER_NODES.get(cluster)
if expected_nodes is None:
errs.append(f'{p}: unknown cluster "{cluster}" (expected one of {sorted(CLUSTER_NODES)})')
continue
for svc in svcs:
for required in ('name', 'image'):
if required not in svc:
@@ -32,13 +48,11 @@ for p in sorted(root.glob('manifests/**/*.toml')):
for fb in ('depends_on', 'extra_ports', 'cmd', 'mounts'):
if fb in svc[sub]:
errs.append(f'{p}: "{fb}" nested under [service.{sub}] — must be at [[service]] level')
# placement.node must match parent vm directory
node = (svc.get('placement') or {}).get('node')
vm_dir = p.parent.name
if node and node != vm_dir:
errs.append(f'{p}: placement.node "{node}" mismatches dir "{vm_dir}"')
if not node:
errs.append(f'{p}: missing placement.node')
elif node not in expected_nodes:
errs.append(f'{p}: placement.node "{node}" not valid for cluster "{cluster}" (expected one of {sorted(expected_nodes)})')
mem = (svc.get('resources') or {}).get('memory')
if not mem:
errs.append(f'{p}: missing resources.memory (mandatory per §8 rule 5)')