ERP-Unlocked Threat Model

Document version: 3.0 Rewritten: 2026-08-28 (v2.1 modelled pdf-api, pdf-worker, Trigger.dev, Upstash and Coolify-hosted apps — all retired; it has been replaced, not patched) Methodology: Microsoft STRIDE with DREAD risk scoring Scope: The deployed Ordermatic platform — Cloudflare edge, GCP Cloud Run and GCE, Cloudflare Workers, the data warehouse, outbound ERP integrations, and the CI/CD path Related: SOC 2 readiness · API security model · Security policies · Platform architecture

Executive summary

Ordermatic ingests customer purchase orders (emailed PDFs, email bodies, spreadsheets), extracts structured orders with LLMs, validates them against warehouse data synced from the customer’s ERP, and writes orders back into that ERP. It is multi-tenant, and it holds customer pricing, customer lists, and order history — commercially sensitive data whose disclosure across tenants is the single worst outcome the system can produce.

Three properties dominate the risk picture:

1. Tenant isolation is enforced in application code, by a clerk_organization_id (or connection_id) predicate on every query — not by separate databases or schemas. Every isolation failure is one missing where clause away.
2. The system writes to customer systems of record. A bad write is not a data-integrity bug in our database; it is a wrong order in the customer’s ERP. This is why every ERP write is Flagsmith-gated.
3. Customer documents leave our infrastructure to Google Vertex (Gemini, Mistral OCR) and Anthropic for extraction. That is a deliberate, contractually-disclosed processing boundary, not an incident — but it is a boundary, and it must stay documented.

System overview

Components as deployed

Layer	Components
Edge	Cloudflare DNS + proxy, WAF, Cloudflare Access, Cloudflare Email Routing
Edge compute	email-worker, content-mcp, graph-mcp, partner-data-api, tidycal-hubspot-sync (Cloudflare Workers)
Cloud Run	erp-webapp-{staging,prod}, erp-internal-{staging,prod}, chatwoot-web-* + chatwoot-worker-*
GCE VMs (VPC)	erp-prod-temporal (Temporal server, temporal-worker, ts-temporal-worker), erp-prod-dagster (Dagster + marimo workbench), erp-prod-internal-edge (cloudflared + ID-token auth proxy), Postgres VM
Static	Cloudflare Pages ordermatic-docs → docs.ordermatic.co
Data	Cloud SQL PostgreSQL (prod), Neon (staging), Memorystore Redis, Cloudflare R2 (PDFs + Iceberg bronze/silver/gold), Typesense
Identity	Clerk (multi-tenant orgs), Cloudflare Access (@ordermatic.co), Google ID tokens (Cloud Run service-to-service), MCP OAuth with PKCE (packages/mcp-oauth)
Secrets	Infisical (self-hosted, Hetzner erp-data), GCP Secret Manager (bootstrap only), 1Password (humans + terraform)
Feature flags	Flagsmith (self-hosted, Hetzner erp-data, /opt/flagsmith compose) — gates every ERP write
Orchestration	Temporal Server (self-hosted on GCE VM erp-{staging,prod}-temporal)
LLM	Google Vertex AI (Gemini, mistral-ocr-2505), Anthropic; Langfuse for tracing
Customer ERPs	Prophet21, WhereFour, Eclipse E4, Acumatica, NetSuite, Dynamics, GP, Odoo
Other SaaS	Stripe, PostHog, BetterStack, HubSpot, Plain, Linear
CI/CD	GitHub Actions → Cloud Build → Cloud Run + GCE (IAP SSH)

Coolify no longer deploys any application. It is retained solely as the control plane for the two self-hosted Hetzner services above (Infisical, Flagsmith).

Trust boundaries

#	Boundary	Control at the boundary
TB1	Internet → Cloudflare edge	WAF, DDoS, bot protection, TLS termination
TB2	Cloudflare → app.ordermatic.co	Public origin; Clerk session required by app middleware
TB3	Cloudflare Access → internal hostnames	team, docs, dagster, temporal, workbench, data* — @ordermatic.co identity, 24h session; workbench is named-email deny-by-default
TB4	Cloudflare Tunnel → GCE VM	cloudflared on erp-prod-internal-edge; local auth proxy mints a Google ID token so Cloud Run keeps allow_unauthenticated = false
TB5	Compute → VPC data plane	Private IP to Cloud SQL and Memorystore; no public database ingress
TB6	Tenant → tenant	clerk_organization_id / connection_id predicates in application queries. The critical boundary.
TB7	Platform → customer ERP	Outbound writes, every one Flagsmith-gated
TB8	Platform → LLM providers	Customer PO content crosses to Vertex/Anthropic
TB9	CI/CD → production	Cloud Build service account, Infisical machine identities, IAP SSH to VMs
TB10	AI dev tools → repository	Claude Code, CodeRabbit, Devin hold repo and PR-creation rights

Risk assessment methodology

DREAD, scored 1–10 per factor, reported as the mean:

• Damage — impact if exploited
• Reproducibility — how reliably it can be repeated
• Exploitability — skill and access required
• Affected users — blast radius
• Discoverability — how easily an attacker finds it

Mean	Level	Expectation
8.0–10.0	Critical	Fix immediately; page if live
6.0–7.9	High	Fix this quarter; compensating control meanwhile
4.0–5.9	Medium	Backlog with an owner
< 4.0	Low	Accept and document

---

STRIDE analysis

1. Spoofing

1.1 Forged inbound order email — DREAD 7.2 (High)

Component: Cloudflare Email Routing → email-worker → /api/webhook/inbound-email

An attacker who knows a customer’s intake alias can email a fabricated purchase order. Extraction and matching will treat it as genuine, and if that org is at high automation it can reach the ERP.

Existing controls: the webhook is HMAC-verified between email-worker and the webapp; intake aliases use subaddressing that must match the local-part; inbox routing rules suppress known non-PO traffic; ERP writes are flag-gated and, for most orgs, human-reviewed.

Gap: sender authenticity is trusted at the application layer. The trusted-sender authentication gap is tracked in PR #2849 CodeRabbit follow-up (CWE-290). Enforce SPF/DKIM/DMARC alignment on inbound intake and treat unaligned senders as untrusted, never auto-submitting.

1.2 Stolen Clerk session — DREAD 6.4 (High)

Component: webapp, packages/auth

Session cookies grant full org-scoped access. Clerk sessions are short-lived and refreshed via handshake; a stale cookie manifests as a sign-in bounce rather than silent access.

Controls: Clerk-managed rotation, HTTPS-only cookies, security headers middleware. Gap: MFA is not uniformly enforced for customer org members — see Access control policy.

1.3 Cloudflare Access bypass via the origin — DREAD 5.6 (Medium)

Access protects the hostname, not the origin. ordermatic-docs.pages.dev remains publicly reachable even though docs.ordermatic.co is behind Access; the Dagster and Temporal VMs are only reachable through the tunnel, which is the stronger case.

Control: treat anything on Pages as internal-but-not-secret. Do not publish credentials, customer data, or unredacted incident detail to the docs site. For real secrecy, put the content behind the webapp, not behind Access on a Pages origin.

1.4 MCP token replay — DREAD 5.2 (Medium)

Component: content-mcp, graph-mcp, packages/mcp-oauth

graph-mcp is gated by Cloudflare Access, content-mcp uses a static bearer token (MCP_AUTH_TOKEN), and the webapp’s OAuth endpoints (packages/mcp-oauth, retained after ops-mcp retirement) implement PKCE and JWT signing with redirect validation.

Control: short token lifetimes, PKCE on every OAuth flow, per-org scoping.

Gap — verified 2026-08: there is no live global kill-switch on outbound customer email. A ORDERMATIC_OPS_MCP_EMAIL_SEND_ENABLED check exists in apps/ts-temporal-worker/src/activities/order-confirmation-email-activity.ts, but that activity is orphaned — sendOrderConfirmationEmail() is never called, and the live workflow sends through order-emails.ts, which has no gate. Do not count this switch as a control until the live path reads it. See the CK complaint-coverage investigation (2026-08-13, repo history).

---

2. Tampering

2.1 Cross-tenant write via a missing org predicate — DREAD 8.6 (Critical)

Component: every webapp API route and service query

Tenant isolation is a where clerk_organization_id = … (or connection_id) predicate. A route that forgets it reads or writes another customer’s orders, pricing, or customer list. There is no database-level backstop.

Controls: apiAuthEnforcementMiddleware default-deny for /api/ routes with a runtime-public allowlist; org context resolved in middleware; review checklist.

Live gap — the highest-value item in this document: the default-deny middleware runs in log mode, not enforce. AUTH_ENFORCEMENT_MODE defaults to 'log', which records what would have been blocked and passes the request through. Until it is flipped to enforce in production, a route that omits its auth guard is not blocked — it is merely logged. Flipping it is the single largest available reduction in this threat.

Recommended: flip to enforce after reviewing the recorded decisions; add a schema-level or query-builder-level org predicate so isolation does not depend on per-route discipline.

2.2 Bronze Iceberg replace wiping other tenants — DREAD 8.2 (Critical)

Component: apps/dagster DLT writes to R2-backed Iceberg

Bronze tables are shared multi-tenant, partitioned by a connection_id column. A write with replace disposition does not replace one tenant’s partition — it replaces the whole table, destroying every other tenant’s bronze data. This has happened before; see Dagster ETL Architecture and apps/dagster/docs/patterns.md.

Controls: resolve_iceberg_write_disposition centralises the decision and permits only append/merge; the dbt silver layer owns deduplication so replace is never needed for correctness.

Recommended: keep the guard un-bypassable and covered by tests; treat any PR introducing a raw disposition string as a blocking review finding.

2.3 Wrong or duplicate ERP write — DREAD 7.4 (High)

Component: ERP service implementations, ts-temporal-worker ERP-write activities

An incorrect submit puts a wrong order into the customer’s system of record. Duplicate writes have previously occurred where a second save was unguarded.

Controls: every ERP write behind a Flagsmith flag (see ERP write flag policy); Temporal gives idempotent, retryable activities with visibility; most orgs require human review before submit.

Recommended: idempotency keys on every write path, not only on the paths where a duplicate has already been observed.

2.4 Supply-chain tampering via CI — DREAD 6.6 (High)

Component: GitHub Actions → Cloud Build → Cloud Run/GCE

A malicious dependency or workflow change reaches production images.

Controls: branch protection with a required Promotion Guard (only staging, or an explicit hotfix label, may target main); pinned action SHAs on sensitive workflows; pnpm audit gate on criticals; CodeQL; Python SCA; validate-secrets-map.yml; images tagged by commit SHA in Artifact Registry.

Gap: Cloud Build authenticates with a long-lived service-account JSON key (GCP_SA_KEY) because Workload Identity Federation was reverted in 2026-04. A leaked key is a direct path to production. Re-evaluating WIF is the fix.

---

3. Repudiation

3.1 Unattributed ERP submission — DREAD 5.4 (Medium)

Actions that change a customer’s ERP must be attributable to a person. Audit logging covers MCP activity and order state transitions, but per-CSR attribution is uneven — some paths log only the actor on the work item, not on the ERP write itself.

Recommended: stamp the acting Clerk user id on every ERP write record, and retain it for the contractual audit window in Data classification and retention.

3.2 Shared machine identities — DREAD 4.8 (Medium)

Infisical machine identities and the Cloud Build SA are shared across deployments, so machine actions attribute to a role rather than a run.

Control: per-app Infisical paths (/runtime/<app>, /build/<app>) narrow what a compromised identity can read. Keep them narrow.

---

4. Information disclosure

4.1 Cross-tenant read — DREAD 8.4 (Critical)

The read side of 2.1, and the more likely of the two: a GET that omits its org predicate leaks another customer’s pricing and order history. Same controls, same log-mode gap, same recommendation.

4.2 Customer documents sent to LLM providers — DREAD 6.0 (High, accepted)

Component: packages/pdf-shared, temporal-worker, Vertex AI, Anthropic, Langfuse

Purchase orders contain customer names, addresses, part numbers and negotiated pricing, and they are sent to third-party model providers for extraction.

Controls: Vertex AI runs under our GCP project with no customer-data training rights; PAN/CVV scrubbing keeps cardholder data out of scope; Langfuse traces carry document ids rather than raw content wherever practical.

This is an accepted, disclosed processing boundary, not a defect — it is a subprocessor relationship. Keep it listed in MSA security commitments and Vendor management; re-review whenever a model provider changes.

4.3 Secrets in logs or the repo — DREAD 6.8 (High)

Controls: secrets are read at runtime from Infisical and never written to files; secrets-map.json plus validate-secrets-map.yml enforce the path policy; the logger redacts known secret keys; pre-commit hooks and CodeQL scan for committed credentials.

Standing risk: a new integration logging a raw response body. Treat “never log a secret value” as a review gate, not a guideline.

4.4 Gold-layer SQL exposure to partners — DREAD 5.8 (Medium)

Component: partner-data-api Worker

Partners run read-only SQL against Gold-layer Parquet on R2.

Controls: Cloudflare Access in front of data.ordermatic.co and data-staging.ordermatic.co gates transport only. Data access requires an app-issued partner API token (Authorization: Bearer), a mcp_tokens row scoped to exactly one connection_id and applied to every table read, so a token cannot cross tenants; partner_token_scope_chk guards every mint path. Read-only by construction. A CF service token without an API token gets 403, and token validation fails closed (503) when the app DB is unreachable. Interactive SSO users on the Access allowlist browse only the playground connection. Partner tokens carry no console scopes and cannot authenticate to the internal admin console.

Resolved (2026-09): the earlier recommendation here, scope each partner to their own tenant’s rows instead of relying on the who-can-connect allowlist, shipped as the connection-scoped token above. Full auth model: apps/partner-data-api/README.md.

4.5 Docs site exposure — DREAD 4.4 (Medium)

docs.ordermatic.co is behind Access, but incident write-ups, runbooks and POC analyses now published there contain customer names and architecture detail. The pages.dev origin stays public (see 1.3).

Recommended: no credentials, no unredacted customer data, no live hostnames plus access paths in published docs.

---

5. Denial of service

5.1 Extraction pipeline flooding — DREAD 5.6 (Medium)

A large volume of inbound mail, or one very large PDF, consumes LLM quota and worker capacity.

Controls: Redis-backed rate limiter with in-memory fallback and a circuit breaker; Cloudflare WAF and bot protection at the edge; Temporal timeouts and retry policies bound any single workflow; page-count and size limits fall back from Mistral OCR to the Gemini path.

5.2 ERP-side rate limits — DREAD 4.6 (Medium)

Customer ERPs and vendor APIs rate-limit us (WhereFour 429s, Dynamics, Acumatica). Sync failure degrades matching quality rather than breaking the app.

Controls: shared Retry-After parsing and single-retry behaviour on 429; Dagster retries; sync freshness is monitored.

5.3 Single-VM dependencies — DREAD 5.0 (Medium)

Dagster, both Temporal workers, and the internal-edge proxy each run on a single GCE VM. Infisical and Flagsmith each run on a single Hetzner box. Loss of the Infisical box blocks deploys and worker restarts; loss of Flagsmith fails flag lookups.

Controls: flags fail closed for ERP writes, so a Flagsmith outage stops writes rather than permitting unsafe ones. See BCP/DR policy.

Recommended: document and rehearse the restore path for both Hetzner boxes.

---

6. Elevation of privilege

6.1 Support-admin role reach — DREAD 6.2 (High)

Support admins can read across organizations by design. That role is the intentional exception to TB6, which makes its membership and audit trail security-relevant rather than administrative.

Recommended: enumerate holders quarterly against Access control policy; log every cross-org support read.

6.2 Data workbench code execution — DREAD 6.6 (High)

marimo at workbench.ordermatic.co executes arbitrary Python with warehouse credentials on the production Dagster VM.

Control: deliberately never falls back to the domain-wide Access rule — the policy coalesces to a sentinel address matching nobody when the named-email allow-list is unset, so it is deny-by-default. Keep it that way.

6.3 AI development tools — DREAD 5.8 (Medium)

Claude Code, CodeRabbit and Devin hold repository access and can open PRs.

Controls: they cannot merge — branch protection requires review and the Promotion Guard blocks direct main targets; secrets are not in the repo; CI runs on PRs from these tools like any other.

Recommended: review their access scope on the same cadence as human access.

6.4 Cloud Run service-to-service — DREAD 4.2 (Low)

Cloud Run services run with allow_unauthenticated = false; callers present a Google ID token and need run.invoker. The erp-prod-internal-edge VM’s service account holds run.invoker on erp-internal-prod and mints a token per request. This is the pattern to copy for any new internal service.

---

Priority summary

#	Threat	DREAD	Status
2.1 / 4.1	Cross-tenant write / read via missing org predicate	8.6 / 8.4	Open — AUTH_ENFORCEMENT_MODE still log
2.2	Bronze Iceberg replace wiping tenants	8.2	Controlled by resolve_iceberg_write_disposition
2.3	Wrong or duplicate ERP write	7.4	Flag-gated; idempotency keys incomplete
1.1	Forged inbound order email	7.2	Open (CWE-290)
4.3	Secrets in logs or repo	6.8	Controlled; standing review gate
2.4	CI supply chain / long-lived GCP_SA_KEY	6.6	Open — WIF reverted
6.2	Workbench code execution	6.6	Controlled (deny-by-default)
1.2	Stolen Clerk session	6.4	MFA not uniformly enforced
6.1	Support-admin cross-org reach	6.2	By design; needs audit cadence
1.4	No live kill-switch on outbound customer email	5.2	Open — existing switch is orphaned code
4.2	Customer docs to LLM providers	6.0	Accepted and disclosed

Incident response

Detection, triage, severity, communication and post-incident review are owned by Incident response policy. Recent incidents are in the Incidents section — for example the fleet-wide email-pull dispatcher wedge and the GenFit P21 partial submission.

Threat-specific first moves:

• Suspected cross-tenant exposure: flip AUTH_ENFORCEMENT_MODE=enforce, identify affected orgs from the auth-enforcement decision records, and treat as a confidentiality incident under the MSA.
• Bad ERP write: disable the relevant Flagsmith write flag first — that stops the bleeding without a deploy — then reconcile with the customer.
• Suspected credential leak: rotate via the documented rotation path; assume the blast radius of the Infisical path the credential sat in.
• Runaway outbound email: there is no kill-switch (1.4) — stop the ts-temporal-worker on erp-prod-temporal, or pause the schedule in Temporal Cloud. Verify this before you need it.
• Bronze data loss: stop the offending Dagster job, then restore from Iceberg snapshots before they expire.

Maintenance

This document must be revised whenever a service is added or retired, a trust boundary moves, or a new subprocessor receives customer data. Version 2.1 was allowed to drift for over a year and ended up describing four services that no longer existed — which made it useless as SOC 2 evidence. Re-review at each SOC 2 cycle and after any architecture change that touches TB6, TB7, or TB8.