Skip to content
Ordermatic Docs

Architecture Decisions

Why Temporal for some things and Dagster for others

Section titled “Why Temporal for some things and Dagster for others”

Temporal handles anything that needs to be durable and user-facing:

  • PDF extraction workflow (the main pipeline a customer triggers)
  • ERP order submission (must not lose or double-submit)
  • Billing workflows
  • Email routing rules

Dagster handles scheduled batch orchestration:

  • ERP catalog syncs (pull products from P21/Acumatica/etc.)
  • dbt runs (Bronze → Silver → Gold medallion pipeline)
  • Image syncs
  • Typesense re-indexing

Rule of thumb: if a customer is waiting on it or it touches money, use Temporal. If it’s a background sync that runs on a schedule, use Dagster.

The codebase previously used Trigger.dev — that has been retired. Do not add new @trigger.dev/* dependencies. See apps/webapp/CLAUDE.md for more.

Why R2 for document storage

Section titled “Why R2 for document storage”

All uploaded PDFs and documents go to Cloudflare R2. We chose R2 because:

  • Zero egress fees (significant at our document volume)
  • Native Cloudflare integration (presigned URLs, email worker can upload directly)
  • Same vendor as our Workers stack

The R2 bucket is accessed via presigned PUT URLs — the client uploads directly, the server never proxies the bytes. 60-second upload timeout.

Iceberg catalog and the feature flag

Section titled “Iceberg catalog and the feature flag”

The Iceberg/Lakehouse pipeline (iceberg_catalog_enabled Flagsmith flag) is production-capable but not on for all customers. Before touching anything in apps/dagster/erp_pipeline/ that touches Iceberg, check whether the flag is on for the relevant connection.

Schema column order is enforced by schema_registry.py — FIELD_ORDER is the single source of truth. If you add a new field to any transformer, add it there first.

Medallion architecture

Section titled “Medallion architecture”

Data flows: Bronze (raw from ERP via DLT → Iceberg/R2) → Silver (dbt dedup) → Gold (dbt materialized views) → Typesense (search index).

Three event-driven Dagster sensors wire the stages together:

  1. bronze → dbt (Silver/Gold)
  2. products → image sync
  3. dbt → Typesense

ERP adapter pattern

Section titled “ERP adapter pattern”

All ERP integrations go through ERPSourceFactory. Adding a new ERP means implementing the service interface and registering in the factory — don’t add ERP-specific code anywhere else in the codebase.

Current adapters: P21, Acumatica, NetSuite, WhereFour, SAP B1, Eclipse E4, Demo.