Standardized ERP Revision Productionization Plan: Multi-ERP Architecture via Interfaces and Base Classes
Status: Proposed
Date: 2026-09-11
Scope: Buy-side vendor acknowledgments (Purchase Orders) and Sell-side customer revisions (Sales Orders) across all supported ERP systems (Prophet21, WhereFour, NetSuite, Acumatica, Dynamics, Odoo, Eclipse E4)
Target Safety Posture: Zero unverified mutations, fail-closed concurrency guards, and non-retrying durable execution.
1. Executive Summary
Section titled β1. Executive SummaryβThe initial revision implementation proved that Ordermatic can ingest revision documents, diff them against existing ERP records using a pure diff engine, and display proposed changes to human clerks. However, the existing write paths are fragmented, hardcoded, and tightly coupled to specific ERP quirks (e.g. ad-hoc WhereFour PATCH calls directly inside Astro API routes, and single-tenant Prophet21 transaction scripts tested only on Genfit).
This plan refactors the revision productionization strategy from an ad-hoc Prophet21 script into a standardized, extensible revision platform for all ERPs supported by Ordermatic.
The Core Paradigm Shift
Section titled βThe Core Paradigm ShiftβRevisions span two business directions that share identical operational and architectural requirements:
- Sell-Side Revisions (Customer Order Changes): A buyer issues an amended PO or confirming PO for an existing sales order (e.g. Cleveland Kitchen on WhereFour, P21, or NetSuite). The target ERP entity is a Sales Order or Sales Quote.
- Buy-Side Revisions (Vendor Order Acknowledgments): A supplier confirms or adjusts an order placed by the distributor (e.g. All State Fasteners or P.F. Markey on P21 or Acumatica). The target ERP entity is a Purchase Order.
Both directions follow the universal lifecycle: $$\text{Document Intake} \longrightarrow \text{Candidate Target Matching} \longrightarrow \text{Canonical Diff} \longrightarrow \text{Human Decision} \longrightarrow \text{Guarded Durable Apply} \longrightarrow \text{Read-Back Verification}$$
Initial Production Boundary
Section titled βInitial Production BoundaryβTo achieve immediate, safe customer productionization without boiling the ocean, Ordermatic enforces an immutable narrow contract for the first release across all connectors:
- First Named Capabilities:
purchase_order.existing_line.set_quantity.v1(Buy-side)sales_order.existing_line.set_quantity.v1(Sell-side)
- Contract Boundary:
- Exactly one active, existing line changes.
- The line has a proven, immutable ERP identity and matching item SKU.
- New quantity is finite, non-zero, and positive.
- The operation is set-to-value (absolute), never relative increment/decrement.
- No price, date, note, header, status, add, remove, or secondary line mutations are permitted in the automated write. Any acknowledgment containing mixed/unsupported changes is recorded for manual clerk entry.
- Explicit human clerk approval is mandatory; automated background apply is disallowed in v1.
- Durable write execution runs in Temporal with strict single-attempt (non-retrying) write activities and exact post-write read-back verification.
2. Current State & Technical Debt
Section titled β2. Current State & Technical DebtβWhat Exists Today
Section titled βWhat Exists Todayβ- Pure Diff Engine (
apps/webapp/src/lib/revision/engine.ts): Source-agnostic matching (scoreCandidates,matchLines,diffLines) operating on canonicalRevisionOrderandRevisionLinerecords. - WhereFour Sales Order Prototype: Basic diff persistence in
order_revisionsand direct HTTP PATCH inapps/webapp/src/pages/api/erp/orders/revision-apply.ts. - Prophet21 Buy-Side Prototype: Initial header and line read/write experiments validated on the Genfit sandbox.
Structural Shortcomings
Section titled βStructural Shortcomingsβ- Direct HTTP Apply in Web Handler:
revision-apply.tsexecutes live ERP mutations within an Astro API route. If a network blip occurs during the HTTP request or if the client disconnects, the system cannot distinguish between a dropped request, a timed-out commit, or a completed mutation. - ERP Coupling &
instanceofBranching: Web handlers and UI components contain hardcoded checks (if (!(service instanceof WherefourService)),wherefour_updated_atcolumn in PostgreSQL). - Unmitigated Concurrency Divergence: Different ERPs possess wildly different concurrency semantics (WhereFour has
updated_at, NetSuite has native optimistic locking tokens, Dynamics has OData ETags, while P21βs Transaction API lacks native Compare-And-Swap tokens). There is no unified concurrency abstraction. - No Write Attempt Ledger:
order_revisionssimply overwrites state (patch_started$\rightarrow$patch_succeeded/patch_failed). Prior attempts, payload hashes, transport responses, and verification images are lost on retry. - No Tenant Certification Framework: Connectors and customer tenants are enabled ad-hoc without formal evidence of identity stability, race condition handling, or collateral field safety.
3. Standardized Architecture: Interfaces & Base Classes
Section titled β3. Standardized Architecture: Interfaces & Base ClassesβThe refactored architecture establishes a strict separation of concerns across four layers:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ Unified UI & Human Review Layer ββ (Revision Tab, Candidate Cards, Diff Tables) ββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ β βΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ Pure Diff Engine (ERP-Agnostic) ββ scoreCandidates() β matchLines() β diffLines() ββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ β βΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ Target Adapter Abstraction Layer ββ IERPRevisionTargetAdapter<TOrder, TLine, TSnapshot> ββ β² ββ βββ BaseERPRevisionAdapter (Common verification, hashing, logging) ββ βββ WherefourSalesOrderRevisionAdapter ββ βββ Prophet21PurchaseOrderRevisionAdapter ββ βββ NetSuiteSalesOrderRevisionAdapter ββ βββ AcumaticaPurchaseOrderRevisionAdapter ββ βββ DynamicsSalesOrderRevisionAdapter ββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ β βΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ Durable Temporal Control Plane & Concurrency Guard ββ ERPRevisionApplyWorkflow (Stable ID, Non-Retrying Activities, ββ Pre-Read Snapshot, Concurrency Strategy, Read-Back Verification) βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ3.1 Target Adapter Interface (IERPRevisionTargetAdapter)
Section titled β3.1 Target Adapter Interface (IERPRevisionTargetAdapter)βAll ERP-specific target logic implements IERPRevisionTargetAdapter, located in packages/integrations/src/services/revision/types.ts:
export type RevisionTargetKind = 'sales_order' | 'purchase_order';
export interface ERPProtectedLineSnapshot { lineId: string; lineNo?: number; erpItemId: string; sku: string; quantity: number; unitPrice?: number; uom?: string; status: string; promisedDate?: string; customFields?: Record<string, unknown>;}
export interface ERPProtectedSnapshot { erpType: string; targetKind: RevisionTargetKind; targetId: string; targetNumber: string; companyId?: string; status: string; isOpen: boolean; isApproved: boolean; versionToken?: string; // ETag, updated_at timestamp, or revision number totalAmount?: number; activeLineCount: number; lines: Record<string, ERPProtectedLineSnapshot>; // Keyed by immutable lineId rawFingerprint: string; // SHA-256 hash of canonicalized ERP payload capturedAt: string;}
export interface VerificationResult { outcome: 'verified_success' | 'verified_noop' | 'verified_failure' | 'verification_mismatch'; message: string; targetLineMatches: boolean; collateralLinesUnchanged: boolean; versionTokenAdvanced: boolean; preSnapshot: ERPProtectedSnapshot; postSnapshot: ERPProtectedSnapshot; divergentFields?: Array<{ path: string; expected: unknown; actual: unknown }>;}
export interface IERPRevisionTargetAdapter<TOrder = unknown, TLine = unknown, TPayload = unknown> { readonly erpType: string; readonly targetKind: RevisionTargetKind; readonly concurrencyStrategy: IERPConcurrencyStrategy;
/** Resolves candidate ERP orders/POs for the incoming revision document */ findCandidates( connection: ERPConnection, query: { poNumber?: string; customerId?: string; vendorId?: string } ): Promise<RevisionOrder[]>;
/** Captures the live, immutable protected snapshot from the ERP */ fetchLiveSnapshot( connection: ERPConnection, targetId: string, companyId?: string ): Promise<ERPProtectedSnapshot>;
/** Maps the canonical ChangeSet into ERP-specific mutation payload */ mapChangeSet( changeSet: ChangeSet, preSnapshot: ERPProtectedSnapshot ): { payload: TPayload; supportedLines: DiffLine[]; unsupportedLines: DiffLine[]; };
/** Executes the single-shot mutation against the ERP (called only by Temporal activity) */ executeWrite( connection: ERPConnection, targetId: string, payload: TPayload, concurrencyToken?: string ): Promise<{ transportSuccess: boolean; rawResponse: unknown; reportedMutated: boolean; }>;
/** Compares pre-image, expected changes, and live post-write read */ verifyMutation( preSnapshot: ERPProtectedSnapshot, postSnapshot: ERPProtectedSnapshot, expectedChange: ChangeSet ): VerificationResult;}3.2 Pluggable Concurrency Strategy (IERPConcurrencyStrategy)
Section titled β3.2 Pluggable Concurrency Strategy (IERPConcurrencyStrategy)βDifferent ERPs enforce concurrency differently. We define an interface and three concrete strategies:
export interface ConcurrencyGuardResult { canProceed: boolean; reason?: string; concurrencyToken?: string;}
export interface IERPConcurrencyStrategy { readonly mode: 'native_cas' | 'pre_read_timestamp' | 'lease_and_verify';
evaluatePreconditions( connection: ERPConnection, targetId: string, recordedSnapshot: ERPProtectedSnapshot, liveSnapshot: ERPProtectedSnapshot ): Promise<ConcurrencyGuardResult>;}NativeOptimisticLockStrategy(e.g. NetSuite, Dynamics 365, Odoo):- Passes the
versionToken(ETag,write_date, or revision number) directly in the mutation header or payload. - The ERP rejects stale writes natively with HTTP 412 Precondition Failed or API concurrency faults.
- Passes the
PreReadTimestampStrategy(e.g. WhereFour):- Evaluates
liveSnapshot.versionToken === recordedSnapshot.versionToken(updated_atstring). - If drifted between clerk decision and write execution, aborts prior to firing the write.
- Evaluates
LeaseAndVerifyStrategy(e.g. Prophet21 Transaction API):- For ERPs lacking server-side CAS tokens.
- Acquires an application-level Redis distributed lease on
erp:${erpType}:${connectionId}:target:${targetId}. - Performs a fresh pre-read milliseconds before write, validates state, issues write, and immediately reads back.
- Requires explicit, recorded risk sign-off for the residual race window.
3.3 Abstract Base Class (BaseERPRevisionAdapter)
Section titled β3.3 Abstract Base Class (BaseERPRevisionAdapter)βProvides reusable verification, snapshot hashing, and change-set enforcement across all ERP connectors:
export abstract class BaseERPRevisionAdapter<TOrder, TLine, TPayload> implements IERPRevisionTargetAdapter<TOrder, TLine, TPayload>{ abstract readonly erpType: string; abstract readonly targetKind: RevisionTargetKind; abstract readonly concurrencyStrategy: IERPConcurrencyStrategy;
abstract findCandidates( connection: ERPConnection, query: { poNumber?: string; customerId?: string; vendorId?: string } ): Promise<RevisionOrder[]>;
abstract fetchLiveSnapshot( connection: ERPConnection, targetId: string, companyId?: string ): Promise<ERPProtectedSnapshot>;
abstract mapChangeSet( changeSet: ChangeSet, preSnapshot: ERPProtectedSnapshot ): { payload: TPayload; supportedLines: DiffLine[]; unsupportedLines: DiffLine[]; };
abstract executeWrite( connection: ERPConnection, targetId: string, payload: TPayload, concurrencyToken?: string ): Promise<{ transportSuccess: boolean; rawResponse: unknown; reportedMutated: boolean; }>;
/** Standardized SHA-256 fingerprinting for canonical snapshots */ computeSnapshotFingerprint(data: Record<string, unknown>): string { const canonicalJson = JSON.stringify(data, Object.keys(data).sort()); return createHash('sha256').update(canonicalJson).digest('hex'); }
/** Universal post-write verification across all ERPs */ verifyMutation( preSnapshot: ERPProtectedSnapshot, postSnapshot: ERPProtectedSnapshot, expectedChange: ChangeSet ): VerificationResult { const divergentFields: Array<{ path: string; expected: unknown; actual: unknown }> = [];
// 1. Verify active line count hasn't drifted unexpectedly if (preSnapshot.activeLineCount !== postSnapshot.activeLineCount) { divergentFields.push({ path: 'activeLineCount', expected: preSnapshot.activeLineCount, actual: postSnapshot.activeLineCount, }); }
// 2. Verify target line changes match expected values exactly let targetLineMatches = true; for (const changedLine of expectedChange.changed) { const lineId = changedLine.existing?.lineId; if (!lineId) continue;
const postLine = postSnapshot.lines[lineId]; if (!postLine) { divergentFields.push({ path: `lines[${lineId}]`, expected: 'present', actual: 'missing' }); targetLineMatches = false; continue; }
if (changedLine.conf && postLine.quantity !== changedLine.conf.quantity) { divergentFields.push({ path: `lines[${lineId}].quantity`, expected: changedLine.conf.quantity, actual: postLine.quantity, }); targetLineMatches = false; } }
// 3. Verify ALL collateral lines remain 100% immutable let collateralLinesUnchanged = true; const changedLineIds = new Set( expectedChange.changed.map(c => c.existing?.lineId).filter(Boolean) );
for (const [lineId, preLine] of Object.entries(preSnapshot.lines)) { if (changedLineIds.has(lineId)) continue; // Skip target line
const postLine = postSnapshot.lines[lineId]; if (!postLine) { divergentFields.push({ path: `collateral[${lineId}]`, expected: 'present', actual: 'deleted', }); collateralLinesUnchanged = false; continue; }
if ( preLine.quantity !== postLine.quantity || preLine.unitPrice !== postLine.unitPrice || preLine.status !== postLine.status ) { divergentFields.push({ path: `collateral[${lineId}]`, expected: preLine, actual: postLine, }); collateralLinesUnchanged = false; } }
const versionTokenAdvanced = Boolean(preSnapshot.versionToken && postSnapshot.versionToken) && preSnapshot.versionToken !== postSnapshot.versionToken;
if (divergentFields.length === 0 && targetLineMatches && collateralLinesUnchanged) { return { outcome: 'verified_success', message: 'Mutation successfully verified against live ERP state.', targetLineMatches: true, collateralLinesUnchanged: true, versionTokenAdvanced, preSnapshot, postSnapshot, }; }
return { outcome: 'verification_mismatch', message: `State drift detected during post-write verification: ${divergentFields.length} discrepancies.`, targetLineMatches, collateralLinesUnchanged, versionTokenAdvanced, preSnapshot, postSnapshot, divergentFields, }; }}4. Multi-ERP Concurrency & Adapter Matrix
Section titled β4. Multi-ERP Concurrency & Adapter Matrixβ| ERP Connector | Target Entities Supported | Concurrency Strategy | Native Version Token | Write Protocol | Residual Concurrency Risk Policy |
|---|---|---|---|---|---|
| WhereFour | Sales Orders (Sell-side) | PreReadTimestamp |
updated_at (ISO timestamp) |
REST PATCH /orders/:id |
Aborts if updated_at drifts before write. Negligible race. |
| Prophet21 | Purchase Orders (Buy-side), Sales Orders (Sell-side) | LeaseAndVerify |
None (Transaction API) | XML Transaction API /uiserver0/api/v2/transaction |
Distributed lease + fast pre-read. Residual race between read and write explicitly documented and signed off. |
| NetSuite | Sales Orders, Purchase Orders | NativeOptimisticLock |
etag / lastModifiedDate |
SuiteTalk REST API PATCH | Full CAS support. Native HTTP 412 on concurrent edit. Zero residual race. |
| Acumatica | Sales Orders, Purchase Orders | NativeOptimisticLock |
Line revision & LastModifiedDateTime |
Contract-based REST API PUT | Full CAS support. Rejects on stale timestamp. |
| Dynamics 365 / BC | Sales Orders, Purchase Orders | NativeOptimisticLock |
@odata.etag |
OData v4 PATCH with If-Match: <etag> |
Strict optimistic locking. 412 Precondition Failed on mismatch. Zero residual race. |
| Odoo | Sales Orders, Purchase Orders | NativeOptimisticLock |
write_date |
JSON-RPC /web/dataset/call_kw |
Atomic record version check within transaction block. |
| Eclipse E4 | Purchase Orders, Sales Orders | LeaseAndVerify |
Session revision number | Proprietary Web Services | Application lease + session lock verification. |
| Mock / Demo ERP | All (testing & CI) | Pluggable (all modes) | Simulated tokens | In-memory with fault injection | Proves all failure modes and fault-injection scenarios in automated tests. |
5. Durable Write Orchestration: The Temporal Control Plane
Section titled β5. Durable Write Orchestration: The Temporal Control PlaneβExecuting ERP mutations directly inside an HTTP request handler violates production safety invariants. All revision apply operations are executed via a dedicated, durable Temporal workflow: ERPRevisionApplyWorkflow.
Client / CSR Click β βΌPOST /api/erp/orders/revision-apply β βββΊ 1. Verify Org Authentication & IDOR gate βββΊ 2. Validate Dual-Gate: Capability Certified/Enabled + Feature Flag βββΊ 3. Claim DB Row: status = 'claimed' βββΊ 4. Start ERPRevisionApplyWorkflow (Stable WorkflowID: revision-apply-${revisionId}) β βΌTemporal Workflow: ERPRevisionApplyWorkflow β βββΊ Activity 1: AcquireDistributedLease (if LeaseAndVerifyStrategy) β βββΊ Activity 2: FetchLivePreSnapshot (calls TargetAdapter.fetchLiveSnapshot) β βββΊ Activity 3: EvaluateConcurrencyPreconditions (abort if stale or already applied) β βββΊ Activity 4: ExecuteRevisionWriteActivity [CRITICAL: Non-Retried! max_attempts = 1] β βββΊ Activity 5: VerifyPostWriteSnapshot (calls TargetAdapter.fetchLiveSnapshot + verifyMutation) β βββΊ Activity 6: RecordAttemptLedger (writes immutable row to revision_write_attempts) β βββΊ Activity 7: ReleaseDistributedLease & Update Work Item ProjectionInvariant Rules for Workflow Execution:
Section titled βInvariant Rules for Workflow Execution:β- Stable Workflow Identity: Workflow ID is strictly
erp-revision-apply-${revisionId}. Any duplicate click or duplicate HTTP request instantly attaches to the running workflow or receives its cached terminal outcome. - Strict Non-Retrying Write Activity: The activity executing the write (
ExecuteRevisionWriteActivity) has its retry policy set to:Under no circumstances will Temporal automatically retry a write activity.RetryPolicy(maximum_attempts=1,non_retryable_error_types=["ERPWriteError", "ERPAmbiguousTransportError"]) - Ambiguous Outcome Escalation: If transport times out during the write, the workflow enters
ReconcileAmbiguousAttemptActivity. If subsequent read-back cannot prove whether the mutation committed, the workflow transitions the attempt tooutcome_unknown, updates the work-item projection toneeds_reconciliation, pages the on-call engineer, and locks the order against further write attempts.
6. Database Schema: Standardized Attempt Ledger & Capability Registry
Section titled β6. Database Schema: Standardized Attempt Ledger & Capability RegistryβWe replace WhereFour-specific columns with generic target abstractions and introduce an append-only attempt ledger.
6.1 Generalizing order_revisions Table
Section titled β6.1 Generalizing order_revisions TableβIn packages/db/src/schema/order-revisions.ts:
export const revisionTargetKindEnum = pgEnum('order_revision_target_kind', [ 'sales_order', 'purchase_order',]);
export const revisionPatchStatusEnum = pgEnum('order_revision_patch_status', [ 'recorded', 'approved', 'claimed', 'patch_started', 'patch_succeeded', 'patch_failed', 'verification_failed', 'outcome_unknown', 'operator_resolved',]);
export const orderRevisions = pgTable('order_revisions', { id: uuid('id').defaultRandom().primaryKey(), clerkOrganizationId: text('clerk_organization_id').notNull(), extractedOrderId: uuid('extracted_order_id') .notNull() .references(() => extractedOrders.id, { onDelete: 'cascade' }), connectionId: uuid('connection_id') .notNull() .references(() => erpConnections.id),
// Standardized ERP Target Columns (Replaces WhereFour-specific fields) targetKind: revisionTargetKindEnum('target_kind').notNull().default('sales_order'), selectedErpOrderId: text('selected_erp_order_id').notNull(), revisesErpOrderNumber: text('revises_erp_order_number').notNull(), targetVersionToken: text('target_version_token'), // Generic version token / ETag / updated_at
// Reporting & Decisions candidateCount: integer('candidate_count').notNull().default(0), confidenceTier: revisionTierEnum('confidence_tier'), clerkDecision: revisionDecisionEnum('clerk_decision'), dollarsChanged: numeric('dollars_changed', { precision: 19, scale: 2 }), clerkUserId: text('clerk_user_id'), patchStatus: revisionPatchStatusEnum('patch_status').notNull().default('recorded'),
// Canonical Snapshots & Diff Evidence matchedSignals: jsonb('matched_signals'), diffSnapshot: jsonb('diff_snapshot').$type<ChangeSet>(), beforeImage: jsonb('before_image').$type<ERPProtectedSnapshot>(), afterImage: jsonb('after_image').$type<ERPProtectedSnapshot>(),
decidedAt: timestamp('decided_at', { withTimezone: true }), createdAt: timestamp('created_at', { withTimezone: true }).defaultNow().notNull(), updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull(),});6.2 New Append-Only Attempt Ledger: revision_write_attempts
Section titled β6.2 New Append-Only Attempt Ledger: revision_write_attemptsβNever overwrite write attempts. Every mutation execution creates an immutable ledger row:
export const attemptOutcomeEnum = pgEnum('revision_attempt_outcome', [ 'precheck_rejected', 'write_started', 'verified_success', 'verified_noop', 'verified_unchanged', 'verified_failure', 'outcome_unknown', 'verification_mismatch', 'operator_resolved',]);
export const revisionWriteAttempts = pgTable('revision_write_attempts', { id: uuid('id').defaultRandom().primaryKey(), revisionId: uuid('revision_id') .notNull() .references(() => orderRevisions.id, { onDelete: 'cascade' }), clerkOrganizationId: text('clerk_organization_id').notNull(), connectionId: uuid('connection_id') .notNull() .references(() => erpConnections.id), workflowId: text('workflow_id').notNull(), attemptNumber: integer('attempt_number').notNull().default(1),
capability: text('capability').notNull(), // e.g. purchase_order.existing_line.set_quantity.v1 concurrencyStrategy: text('concurrency_strategy').notNull(), actorUserId: text('actor_user_id').notNull(),
// Hashes & Snapshots operationHash: text('operation_hash').notNull(), preSnapshotHash: text('pre_snapshot_hash').notNull(), postSnapshotHash: text('post_snapshot_hash'), sanitizedPayload: jsonb('sanitized_payload').notNull(), preSnapshot: jsonb('pre_snapshot').$type<ERPProtectedSnapshot>().notNull(), postSnapshot: jsonb('post_snapshot').$type<ERPProtectedSnapshot>(),
// Outcome transportStatus: text('transport_status'), rawErpResponse: jsonb('raw_erp_response'), terminalClassification: attemptOutcomeEnum('terminal_classification').notNull(), divergentDetails: jsonb('divergent_details'),
startedAt: timestamp('started_at', { withTimezone: true }).defaultNow().notNull(), completedAt: timestamp('completed_at', { withTimezone: true }),});6.3 Connection Capability Table (erp_connection_capabilities)
Section titled β6.3 Connection Capability Table (erp_connection_capabilities)βexport const capabilityStateEnum = pgEnum('erp_capability_state', [ 'unsupported', 'shadow', 'certified', 'enabled', 'suspended',]);
export const erpConnectionCapabilities = pgTable( 'erp_connection_capabilities', { id: uuid('id').defaultRandom().primaryKey(), connectionId: uuid('connection_id') .notNull() .references(() => erpConnections.id, { onDelete: 'cascade' }), capability: text('capability').notNull(), // purchase_order.existing_line.set_quantity.v1 state: capabilityStateEnum('state').notNull().default('unsupported'), concurrencyStrategy: text('concurrency_strategy').notNull(), maxChangedLines: integer('max_changed_lines').notNull().default(1), evidenceReference: text('evidence_reference'), certifiedAt: timestamp('certified_at', { withTimezone: true }), certifiedBy: text('certified_by'), updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow().notNull(), }, table => ({ connCapIdx: uniqueIndex('conn_capability_unique_idx').on(table.connectionId, table.capability), }));The Dual-Gate Rule
Section titled βThe Dual-Gate RuleβA revision apply write is permitted IF AND ONLY IF both of the following evaluate to true:
1. erpConnectionCapabilities.state === 'enabled'2. Flagsmith.getFeatureFlag(connection, REVISION_APPLY_FLAG) === trueThis guarantees that an emergency Flagsmith toggle can immediately halt traffic globally or per-connection, but a broad flag toggle can NEVER inadvertently enable an uncertified ERP connection.
7. Universal Adapter & Tenant Certification Harness
Section titled β7. Universal Adapter & Tenant Certification HarnessβNo ERP adapter or customer connection may transition to certified or enabled without passing the standardized 6-step automated certification suite: ERPRevisionCertificationHarness.
Certification Suiteββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ 1. Stability Test Prove stable line identity (line_id) and open header status ββ 2. Concurrency Test Inject artificial race; verify precheck rejects stale token ββ 3. Mutation Test Execute set-quantity mutation on controlled test order/PO ββ 4. Idempotency Test Re-submit exact desired state; verify classified as no-op ββ 5. Fault Injection Simulate transport drop; verify terminal outcome & reconciliationββ 6. Collateral Proof Read back all untouched lines; prove ZERO collateral drift ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββStandardized Test Steps
Section titled βStandardized Test Stepsβ- Test 1: Immutable Identity & Read Stability:
- Pre-reads order header and lines multiple times.
- Asserts line keys remain stable across repeated queries and do not alter index positions.
- Test 2: Concurrency & Stale Write Rejection:
- Captures snapshot $S_0$.
- Directly mutates the test entity in the ERP to create $S_1$ (e.g. adjusts a line quantity or timestamp).
- Attempts to apply a revision based on $S_0$.
- Asserts that the adapter/concurrency strategy cleanly aborts with
precheck_rejectedand zero mutations occur.
- Test 3: Isolated Single-Line Mutation:
- Mutates quantity from $Q_0 \rightarrow Q_1$ on target line $L_1$.
- Asserts post-write read-back matches $Q_1$ exactly.
- Test 4: Idempotent Replay:
- Re-submits an apply payload where desired quantity is already $Q_1$.
- Asserts the adapter recognizes
verified_noop/verified_unchangedwithout generating invalid duplicate lines.
- Test 5: Transport Timeout & Fault Recovery:
- Injects network timeout immediately following payload dispatch.
- Executes the reconciliation activity.
- Verifies the system correctly detects whether the commit completed or failed without wedging the order.
- Test 6: Collateral Immutability:
- Validates that non-target lines ($L_2, L_3 \dots L_n$), unit prices, customer IDs, shipping addresses, and header metadata experienced zero alterations.
8. Operator Tooling, Observability & Support Runbooks
Section titled β8. Operator Tooling, Observability & Support Runbooksβ8.1 Internal Operations Portal (apps/internal)
Section titled β8.1 Internal Operations Portal (apps/internal)βThe internal support portal provides a dedicated Revision Control Center:
- Connection Capability Manager: View certification status, evidence links, and toggle capabilities.
- Attempt Ledger Inspector: Drill into any write attempt: view exact pre-snapshot JSON, payload JSON, post-snapshot JSON, and divergent field diffs.
- Manual Reconciliation Workbench: For any order in
outcome_unknownorverification_mismatch:- βFetch Live ERP State Nowβ: triggers a fresh read.
- βMark Manually Resolvedβ: requires clerk note, actor ID, and assigns a terminal classification.
- βSuspend Connectionβ: immediately drops connection capability to
suspended.
8.2 Observability & Metrics
Section titled β8.2 Observability & MetricsβEmit structured OpenTelemetry metrics tagged by erp_type, connection_id, and capability:
erp.revision.decisions.count(by decision:revisevsstart_clean)erp.revision.apply.attempts.counterp.revision.apply.outcome(tagged:verified_success,verified_noop,precheck_rejected,verification_mismatch,outcome_unknown)erp.revision.apply.latency_ms
8.3 P1 Alerting Thresholds
Section titled β8.3 P1 Alerting ThresholdsβImmediate on-call page fired via PagerDuty/Slack for:
- Any occurrence of
outcome_unknown. - Any occurrence of
verification_mismatch(indicates unexpected ERP side-effects or collateral damage). - More than 2 consecutive
precheck_rejectedevents on the same connection within 10 minutes.
9. Phased Implementation Roadmap
Section titled β9. Phased Implementation RoadmapβPhase 0: Core Contracts & Interfaces (TypeScript & Python Base Classes) β βΌPhase 1: DB Schema Migrations & Attempt Ledger (Drizzle Kit: order_revisions, attempts, capabilities) β βΌPhase 2: Durable Temporal Workflow & Verification Activities (ERPRevisionApplyWorkflow) β βΌPhase 3: Reference Adapters Implementation (WhereFour Sales Order + Prophet21 Purchase Order) β βΌPhase 4: Universal Certification Harness & Fault Injection Suite β βΌPhase 5: Secondary Adapters Rollout (NetSuite, Acumatica, Dynamics, Odoo) β βΌPhase 6: Operator UI, Metrics, Runbooks & Production CanaryPhase 0: Core Architecture & Base Abstractions (1β2 sessions)
Section titled βPhase 0: Core Architecture & Base Abstractions (1β2 sessions)β- Define
IERPRevisionTargetAdapter,BaseERPRevisionAdapter,IERPConcurrencyStrategy, andERPProtectedSnapshotinpackages/integrationsandapps/webapp/src/lib/revision/targets. - Implement
resolveRevisionAdapter(connection, targetKind)factory. - Remove hardcoded
WherefourServicechecks fromengine.tsandtypes.ts.
Phase 1: Database Migration & Attempt Ledger (1 session)
Section titled βPhase 1: Database Migration & Attempt Ledger (1 session)β- Author schema updates in
packages/db/src/schema/order-revisions.tsandpackages/db/src/schema/erp-connections.ts. - Generate migration using
pnpm --filter @repo/db db:generate. - Apply migration with
pnpm --filter @repo/db db:migrate. - Write unit tests for schema validations and constraints.
Phase 2: Durable Temporal Workflow & Verification (2 sessions)
Section titled βPhase 2: Durable Temporal Workflow & Verification (2 sessions)β- Implement
ERPRevisionApplyWorkflowinapps/temporal-worker/apps/ts-temporal-worker. - Create activities:
acquire_distributed_leasefetch_live_protected_snapshotevaluate_concurrency_preconditionsexecute_non_retried_write(max_attempts = 1)verify_post_write_snapshotrecord_attempt_ledgerreconcile_ambiguous_attempt
- Refactor
apps/webapp/src/pages/api/erp/orders/revision-apply.tsto claim the row and dispatch the Temporal workflow.
Phase 3: Reference Adapters Implementation (2 sessions)
Section titled βPhase 3: Reference Adapters Implementation (2 sessions)β- WhereFour Sales Order Adapter: Port existing WhereFour PATCH logic into
WherefourSalesOrderRevisionAdapterusingPreReadTimestampStrategy. - Prophet21 Purchase Order Adapter: Implement
Prophet21PurchaseOrderRevisionAdapterusingLeaseAndVerifyStrategyand live XML Transaction API client.
Phase 4: Certification Harness & Fault Injection (1β2 sessions)
Section titled βPhase 4: Certification Harness & Fault Injection (1β2 sessions)β- Build
ERPRevisionCertificationHarnessinpackages/evalorpackages/integrations/test. - Run harness against Mock ERP with synthetic network latency, timeouts, and state divergences.
- Run harness against WhereFour staging and P21 Genfit sandbox. Generate certification evidence artifacts.
Phase 5: Secondary Adapters Implementation (2β3 sessions)
Section titled βPhase 5: Secondary Adapters Implementation (2β3 sessions)β- Implement
NetSuiteRevisionAdapter(NativeOptimisticLockStrategyvia SuiteTalk REST API). - Implement
AcumaticaRevisionAdapter(NativeOptimisticLockStrategy). - Implement
DynamicsRevisionAdapter(NativeOptimisticLockStrategywith ODataIf-Match). - Implement
OdooRevisionAdapter.
Phase 6: Operator Surface, Canary & GA (1β2 sessions)
Section titled βPhase 6: Operator Surface, Canary & GA (1β2 sessions)β- Implement Attempt Ledger Inspector and Manual Reconciliation Workbench in
apps/internal. - Wire OpenTelemetry metrics and PagerDuty alert rules.
- Run low-volume canary on internal test org (
org_internal_test), then single customer canary perfeature-flag-policy-erp-writes.md.
10. Acceptance Criteria
Section titled β10. Acceptance CriteriaβA revision write capability is ready for production GA only when all criteria pass:
- Standardized Interfaces & Base Classes: Every revision target inherits from
BaseERPRevisionAdapterand implementsIERPRevisionTargetAdapter. Zeroinstanceofchecks remain in UI or API routes. - Dual-Gate Enforcement: No write occurs unless
capability.state === 'enabled'AND the connection Flagsmith flag is true. - Durable Execution: All ERP writes execute inside Temporal with a stable, deduplicated workflow ID. Zero direct ERP write requests occur in HTTP handlers.
- Strict Non-Retrying Write: The write activity retry policy is explicitly configured with
maximum_attempts = 1. - Immutable Attempt Ledger: Every apply execution generates a new row in
revision_write_attemptswith complete before/after snapshots, hashes, and classification. - Universal Post-Write Verification: Target line changes and collateral line immutability are independently verified against live ERP state prior to marking success.
- Ambiguous Outcome Isolation: Any transport error or verification mismatch immediately halts the order, records
outcome_unknown, and pages on-call. - Harness Certification: Target ERP connection has executed the 6-step
ERPRevisionCertificationHarnesswith verified evidence committed. - Operator Reconciliation: Support engineers can review diffs, re-check state, and mark resolutions directly in
apps/internalwithout SQL scripts or database manipulation. - Kill-Switch Tested: Toggling the Flagsmith flag or suspending the capability immediately disables apply in both UI and server without deployment.