Skip to content
Ordermatic Docs

14 β€” xref-first activation runbook

Companion to 13-xref-first-matching.md (see Xref Bootstrap from History) and the measured baseline 13a. The xref-first build is complete and merged to staging β€” every phase ships gated default-OFF, so production behaviour is unchanged today. This runbook is how you turn it on safely, per connection, with a verify + rollback at each step.

Golden rule: flip ONE lever, verify, then the next. Every lever is independently reversible (flip off β†’ off-path). Hold the live auto-submit flag until the identity-gate Ξ± is confirmed in dry-run.

0. What’s merged (all gated OFF)

Section titled β€œ0. What’s merged (all gated OFF)”

P-0 provenance Β· P-A desc-key uniqueness (migr 0085) Β· P-B configurable xref gate Β· P-C fail-closed ship-to Β· P-D Dagster pipeline + desc-norm projection Β· P-E CSR-confirm flywheel write Β· P-F xref-or-review gate Β· keystone provenance wire (#1762, migr 0087) Β· NBSP unify (0086) Β· dedupe guard (#1755) Β· Jaccard desc exemption (#1764) Β· step-4 refreshed-binds fix (this PR).

1. Code prerequisites (must land BEFORE flags do anything)

Section titled β€œ1. Code prerequisites (must land BEFORE flags do anything)”

These are unwired gaps β€” flipping flags without them is a no-op or inconsistent.

  • πŸ”΄ BLOCKER β€” wire the desc gate into the production match callers. MatchGateOptions.xref exists, but no production caller passes it, so the desc sub-search never fires in prod. Wire erp_connections.extraConfig.matchingConfig.xref (β†’ { queryBy:['id','name'], acceptFields:[...,'their_item_desc'] }) into all three callers, gated by the same config (default-off β†’ byte-identical for non-CK):

    • apps/ts-temporal-worker/src/activities/persist-auto-priced-matches.ts:322 (the auto-pricing path that writes the binds the gates read β€” the critical one; passes no options today)
    • apps/webapp/src/pages/api/erp/parts/auto-validate.ts:303 (gateOptions is {nameOverlap, onLowRelevance} β€” add xref)
    • apps/webapp/src/pages/api/erp/parts/auto-validate-with-price-check.ts:130

    Without this, P-D’s seeded desc xrefs are never queried β†’ the flywheel can’t bind in prod regardless of catalog or flags. (This was mis-attributed to β€œP-F” in the original plan; P-F shipped as the agentic gate, so the caller wiring was never done.)

  • βœ… step-4 ERP-validity consistency β€” landed in this PR (filter_invalid_items_activity now judges the same refreshed binds the gates use).

2. Data / infra prerequisites

Section titled β€œ2. Data / infra prerequisites”
  • Promote staging β†’ main. Carries the code + migrations 0085/0086/0087 and the #1755 dedupe guard to prod (still gated off). The guard cleans prod’s duplicate manual_cross_references rows before 0085’s unique index applies β€” they must travel together.
  • Catalog coverage (gates real binds). The matcher resolves an xref’s item_id (SKU) against the products collection; a SKU not in the catalog can’t bind. The staging CK sandbox catalog holds only 186 products, so the bulk --xref-real measure reads ~0 there. Before measuring or activating, ensure the connection’s product catalog covers the order corpus (full CK product sync / Dagster materialize).
  • Materialize P-D. Run the Dagster typesense_cross_references asset so manual_cross_references projects into Typesense cross_references carrying their_item_desc_norm (additive; verified safe).
  • Seed history (per connection): bun run packages/db/scripts/backfill-xrefs-from-golden.ts --source order_history --connection-id <uuid> --commit --push-typesense (quarantines (customer,desc)β†’β‰₯2 SKU).

3. Activation sequence (per connection, in order)

Section titled β€œ3. Activation sequence (per connection, in order)”
# Lever (where) Enables Verify Rollback
1 matchingConfig.xref (extraConfig) Matcher fires the desc gate (needs Β§1 wiring) --xref-real bulk binds materialize; spot-check an idiosyncratic desc binds xref unset key
2 matchingConfig.xrefWriteback.enabled (extraConfig) CSR confirm writes the next xref (P-E) confirm a line β†’ a manual_cross_references row appears (source operator_selection) unset β†’ no writes
3 pricingConfig.autoRepriceOnExtraction (extraConfig) Writeback persists erp_item_id + match_provenance extracted_order_items.match_provenance populated on new binds unset β†’ null stub
4 matching_shipto_resolver_v1 (Flagsmith) + matchingConfig.customerResolution.enabled Ship-to as primary customer key, fail-closed (P-C) abstain β†’ review (not a name-guess) unset both
5 agentic_require_identity_grade (Flagsmith) xref-or-review: only identity-grade auto-submits (P-F) a fuzzy-only line routes to review; identity lines stay eligible unset β†’ Phase-0a only
6 agentic_auto_submit_enabled + erp_write_enabled (Flagsmith) Actually submit (else dry-run) orders auto-submit; watch live Ξ± unset β†’ dry-run

Pre-step gate: re-confirm pricingConfig.autoRepriceOnExtraction is OFF on every connection you are not activating (staging verified 0/11 on 2026-06-18; re-confirm prod). If a connection already had it on, step 3’s effect is already live there.

3b. Alias memory and bind verifier (plan bison-alias-memory-and-verifier)

Section titled β€œ3b. Alias memory and bind verifier (plan bison-alias-memory-and-verifier)”

Per connection, in this order. Every step is a DB gate in erp_connections.extra_config.matchingConfig AND a fail-closed Flagsmith flag; unset, nothing changes.

  1. Bootstrap memory: EVAL_DB=staging bun run apps/webapp/scripts/bootstrap-alias-memory.ts --truth <orders.json> (dry run), then --commit. The input is the PO-to-ERP join the Bison corpus eval produces (orders with customer_id and lines with customer_text and item_id). Rows land in manual_cross_references as source = 'order_history' with support_count = distinct orders.
  2. matchingConfig.aliasMemory = { "enabled": true, "matcherPriceGate": true, "learnFromSubmit": true } plus matching_alias_memory_v1 on for the connection. Optional: "near": false (exact only), "nearThreshold" (0.6), "nearMargin" (0.1). The review route and the intake worker then run the memory tier before the catalog buckets, and keep a catalog bind only when the customer’s history price corroborates the PO price. Needs matchingConfig.customerHistory for the price gate’s evidence.
  3. matchingConfig.bindVerifier = { "enabled": true } plus matching_bind_verifier_v1: unverified binds get validationError.type = 'bind_unverified'; Jev reports BIND_UNVERIFIED.
  4. After a week with a hold rate on CSR-confirmed lines under 5%: matching_bind_verifier_submit_guard_v1.

Rollback: turn the flag off. Bootstrap rows can be deactivated by source_detail.

4. Measurement & monitoring

Section titled β€œ4. Measurement & monitoring”
  • Before activating: once the catalog covers the corpus, re-run the real end-to-end validation (--xref-real, the uncommitted eval mode β€” seeds tagged docs, real desc gate, auto-deletes) for the true bulk ceiling. The leakage-controlled simulation (--simulate-xref-lever) gives ~46–58% identity coverage / xref bucket 0% wrong as the upper bound.
  • After step 6: monitor the live identity-gate Ξ± (wrong-among-auto-submitted). Target ≀2%. The desc-xref bucket is 0% wrong by construction; any Ξ± is the UPC bucket β€” if it drifts, flip step 6 back to dry-run.

5. Notes

Section titled β€œ5. Notes”
  • The --xref-real eval mode and the per-confirm seed write to the shared Typesense cross_references collection β€” staging only; never run against prod casually.
  • Flag mechanics: per-connection extraConfig via the connection settings; Flagsmith flags via the per-env toggle tooling (see project memory reference_flagsmith_*).