CK Matching — Current State

This is the entry point for the CK matching work. Read this first; the numbered docs (01–13) are the detailed investigation trail.

Measured baseline (real matcher, history ground truth)

66 CK orders / 328 lines, WhereFour ERP, 2026:

Metric	Value
Recall	60.4%
Wrong-match rate	26.1%
Root cause	Candidate recall (not ranking) — 58% of wrong binds never had the right product in the candidate set

The catalog splits cleanly in two: ~56% of lines carry a UPC (matching is strong); ~43% (Walmart DCs, Harris Teeter, Stop & Shop) carry no UPC and match poorly (~38% recall / ~50% wrong). All improvement work targets the no-UPC half.

What’s live today

Change	Flag	Effect (measured A/B)
broadenUnion — wider candidate recall	matching_pack_variant_v1	Recall ↑ 56.5% → 71.1%
pricePackSelect — unit-price pack disambiguation	matching_price_pack_select_v1	Wrong-match ↓ 38.7% → 26.3%
Ship-to resolver	matching_shipto_resolver_v1	Customer ID accuracy 64.3% → 93.5% (built, gated OFF)
Xref-first pipeline	(P-A through P-F PRs)	Infra shipped; CSR flywheel wiring in progress

All flags are Flagsmith-gated and connection-scoped. No flag touches auto-submit without explicit sign-off.

Where it’s going: xref-first

Doc 13 defines the current direction. Four facts drive it:

1. Within a customer, the pack is deterministic — 97.8% of (customer, product-family) groups map to exactly one SKU.
2. Volume is repeat business — 96.3% of lines are in repeat (customer, description) groups.
3. Identity-grade binds are safe — xref/UPC/exact-id: 2.6% wrong. Fuzzy: 32% wrong (and confidence is a useless flat 0.92).
4. Ship-to identifies the customer — 93.5% top-1 with the resolver.

Conclusion: Learn a customer-scoped description→SKU cross-reference from confirmed order history. Auto-submit only identity-grade binds. Route everything else to human review, whose confirmation writes the next xref.

extracted PO line
    ↓ ship-to resolver → customer_id
    ↓
P1 xref (history-learned, customer-scoped)
P2 UPC exact / exact-id
──── identity-grade line: AUTO-SUBMIT ────
P3 fuzzy + pricePackSelect   ← review-assist only, never auto-submits
    ↓
HUMAN REVIEW → confirms product → writes xref → next identical order auto-submits

This hits ≤2% wrong-match by construction (only identity-grade binds go through), and grows coverage through the flywheel.

Key artifacts

Artifact	Path
Eval harness	apps/webapp/scripts/run-ck-match-eval.ts
Candidate experiments	apps/webapp/scripts/run-ck-candidate-experiments.ts
Scorer	packages/eval/src/scorers/ck-match-scorer.ts
Ship-to resolver	packages/pdf-shared/pdf_shared/matching/shipto_resolver.py
Typesense asset (UPC derivation)	apps/dagster/erp_pipeline/assets/typesense.py
Eval reports	packages/eval/reports/ck-*.json

Investigation docs (chronological)

Doc	What it established
01 — Measuring match quality	Harness design, history ground truth, metrics
02 — Baseline & root cause	60.4%/26.1% baseline; candidate recall is the root cause
04 — Fidelity roadmap	Ranked levers; broadenUnion first
09 — Ship-to resolution	Resolver design; 93.5% top-1 measured
10 — Pack variant matcher	Recall/ranking split; broadenUnion spec
11 — Price-pack selector	pricePackSelect spec; A/B numbers
13 — Xref-first matching	Current direction; supersedes doc 12
Decisions log	Why each major choice was made