ERP-Unlocked Deployment Guide

How a commit reaches staging and production.

Coolify no longer deploys any application — that ended in 2026-06 (see Coolify decommission). Only one Hetzner server (erp-data) remains, hosting two self-hosted services: Infisical and Flagsmith (migrated onto erp-data in /opt/flagsmith when erp-failover was decommissioned in 2026-07, ENG-260). Those are managed by infrastructure/ terraform + ansible and are outside the app deploy path described here. If a doc mentions Coolify deploying webapp, pdf-api, or a worker, it is describing history.

Branch model

feature branch → PR → staging → (auto-deploy-main.yml fast-forward) → main

• Never commit directly to staging or main. Open a PR against staging.
• Production promotion is done by manually dispatching gh workflow run auto-deploy-main.yml --ref staging, which fast-forwards main to staging’s SHA. Only staging may be promoted into main, enforced by the Promotion Guard (main-promotion-guard.yml) branch-protection guard (the single exception is a PR explicitly labelled hotfix).
• Merging to staging deploys staging. Fast-forwarding main deploys production.

What deploys where

Surface	Deployed by	Target
webapp, internal	gcp-deploy.yml → cloudbuild.yaml	Cloud Run erp-{webapp,internal}-{staging,prod}
temporal-worker, ts-temporal-worker	cloudbuild.yaml step deploy-temporal-workers	systemd units on GCE VM erp-{staging,prod}-temporal, over IAP SSH
dagster	cloudbuild.yaml step deploy-dagster, plus deploy-dagster.yml	GCE VM erp-{staging,prod}-dagster
Chatwoot	cloudbuild.yaml step deploy-chatwoot	Cloud Run chatwoot-web-* + worker pool chatwoot-worker-*
Cloudflare Workers	one workflow per app (deploy-email-worker.yml, deploy-content-mcp.yml, deploy-partner-data-api.yml, deploy-tidycal-hubspot-sync.yml)	Cloudflare Workers
docs	deploy-docs.yml	Cloudflare Pages project ordermatic-docs → docs.ordermatic.co
DB migrations	cloudbuild.yaml step migrate-db	Neon PostgreSQL (DATABASE_URL, Phase 2 Cloud SQL cutover pending); Cloud SQL (erp-prod-cloudsql) backs prod Dagster & Temporal metadata

gcp-deploy.yml is a thin ~30s trigger: it authenticates to GCP with GCP_SA_KEY and hands off. All build, push, Cloud Run deploy, and VM deploy logic lives in cloudbuild.yaml. Debug deploys there, not in the workflow.

Cloud Build skips services whose paths didn’t change. To force everything, run gcp-deploy.yml via workflow_dispatch with force_all: true.

Pre-deployment checklist

• Tests pass locally (pnpm test) and in PR CI
• A changeset exists for any app/package change (pnpm changeset)
• Migrations were generated, never hand-written — edit the Drizzle schema in packages/db/src/schema/*.ts and run pnpm --filter @repo/db db:generate
• Any ERP-write behaviour is behind a Flagsmith flag
• No secrets in the diff

Staging deployment

1. Merge the PR into staging.
2. Watch GCP Deploy in Actions, then the Cloud Build run it triggers. Cloud Build is where per-service success or failure is visible.
3. Verify at https://staging.ordermatic.co:
   • Sign-in works (Clerk)
   • PDF upload and extraction complete
   • ERP integrations respond
   • No new errors in Cloud Logging

Note

A staging deploy showing CANCELLED is usually not a failure — concurrency cancels a run that a newer merge superseded. The real deploy is the run for the following chore: version packages merge.

Production deployment

Production ships by manually dispatching auto-deploy-main.yml, which fast-forwards main to staging’s SHA. Do not open a staging → main merge PR — the Promotion Guard permits one, but that is a guard rule, not the release path, and a merge commit breaks the fast-forward the workflow requires.

gh workflow run auto-deploy-main.yml --ref staging

A human triggering this workflow IS the prod-release approval. The workflow also runs on a 2 AM UTC cron, but the cron only reports staging↔main divergence — it never promotes.

Before it fast-forwards, the workflow gates on:

• main being an ancestor of staging (otherwise the branches diverged — merge origin/main back into staging as a merge commit to reconcile, then retry)
• the staging GCP Deploy run for that commit having finished successfully — it refuses to promote a commit whose staging deploy failed or is still running
• staging smoke passing

Then it pushes staging’s SHA to main, which triggers gcp-deploy.yml → Cloud Build → the -prod services, and watches that deploy.

Afterwards:

1. Confirm Cloud Build actually deployed — a green workflow only means the trigger fired. Check the Cloud Build step log for the services you expect; an unchanged-path skip reports success without deploying.
2. Verify at https://app.ordermatic.co and watch Cloud Logging error rates.

Versioning

Versions and changelogs are managed by changesets. The release.yml workflow runs hourly (not per-merge), collapsing several staging merges into one chore: version packages PR that it auto-merges. There is no VERSION file to bump by hand.

Rollback

Cloud Run (webapp, internal, chatwoot)

List revisions and pick the last good one:

gcloud run revisions list --service=erp-webapp-prod --region=us-east1 --project=ordermatic-prod

Then shift all traffic to it — no rebuild needed. Substitute the real revision name; do not paste a placeholder in angle brackets, the shell reads < as input redirection:

gcloud run services update-traffic erp-webapp-prod --region=us-east1 --project=ordermatic-prod --to-revisions=erp-webapp-prod-00042-abc=100

update-traffic applies to Cloud Run services (erp-webapp-prod, erp-internal-prod, chatwoot-web-prod). The Chatwoot worker is a Cloud Run worker pool, which has no traffic split — roll it back by redeploying the previous image:

gcloud beta run worker-pools update chatwoot-worker-prod --region=us-east1 --project=ordermatic-prod --image=us-east1-docker.pkg.dev/ordermatic-prod/erp-unlocked/chatwoot:<GOOD_SHA>

GCE VMs (dagster, temporal workers)

The systemd units pull a tagged image from Artifact Registry by commit SHA. Re-point the unit at the previous ${COMMIT_SHA} tag and restart it over IAP SSH — that is the precise, service-scoped rollback.

Re-running gcp-deploy.yml is not a reliable rollback: workflow_dispatch takes a branch or tag ref, not an arbitrary commit, so it will rebuild whatever that ref currently points at; and its unchanged-path detection can skip the very service you are trying to roll back. If you do use it, you must pass force_all: true and be certain the ref resolves to the known-good commit.

Cloudflare Workers and Pages

Use the dashboard’s Deployments tab for the Worker or Pages project and roll back to (or promote) the previous deployment.

Database

There is no automated migration rollback. Drizzle Kit does not generate down migrations and packages/db has no rollback script. Recovery is normally forward-only: write and generate a new corrective migration.

Do not treat “restore the backup” as a rollback. Restoring in place discards every write committed after the backup was taken — for an order-processing system that means losing real customer orders. If a restore is genuinely required: point-in-time restore to a separate Cloud SQL instance, validate it, reconcile the writes made since the restore point, and only then cut over. That is an incident with a data-loss decision in it, not a deploy step.

Post-deployment verification

• Target hostname loads and sign-in works
• Cloud Logging error rate is flat
• Temporal Cloud shows workflows completing, not piling up
• Dagster runs are still scheduled and succeeding
• A test order processes end to end

Troubleshooting

Workflow green but nothing deployed. Cloud Build skips unchanged services. Read the Cloud Build step log; re-run with force_all: true if you need the rebuild.

Cloud Build auth failures. GCP_SA_KEY is a service-account JSON key (WIF was reverted 2026-04-25). Build-time Infisical credentials come from Secret Manager as erp-{staging,prod}-infisical-build-client-{id,secret}.

Docs deploy fails with code: 10000. CLOUDFLARE_PAGES_API_TOKEN needs Account → Cloudflare Pages → Edit. The shared Worker token is not sufficient.

Migrations fail mid-deploy. migrate-db runs as its own Cloud Build step; check that step’s log. Fix forward with a new generated migration.