04 — Fidelity Roadmap (ranked levers)

Architecture the investigation converged on: broaden recall first, then rerank for precision, plus a deterministic moat for repeat orders, plus semantic recall for cold- start no-UPC lines. Every change is org/connection-scoped and default-off.

1. Broaden candidate generation (union of buckets) — ship first

Union of upc/identifier + lexical-name (more fields) + pack-stripped family query, deduped. Directly attacks the root cause (58% of wrong binds never had the right product retrieved). The UPC bucket alone already reached 77.1% coverage; broadened lexical reached 74% coverage / 50.7% recall (precision weaker — needs a picker, see #2). Test: run-ck-candidate-experiments.ts --strategy broaden-union.

2. Gated LLM “select-or-0” rerank over the broadened top-K — the precision lever

With recall fixed, the residual failure is pack/variant disambiguation (2 gallon vs 9/36 oz) — what an LLM with name+pack+customer context does well and cosine can’t. Mandatory 0 = abstain + verbalized confidence so it never forces a wrong bind. This is not the earlier-cut LLM idea (that reranked a thin name-only set where the right answer was often absent — garbage-in); now the right candidate is present first. Test: --strategy llm-rerank (swap the picker for a structured select-or-0 call; add --repeat N to measure determinism); label abstention ground truth (does a correct CK product exist in the catalog at all?).

3. Deterministic customer-part-# cross-reference from confirmed history — the no-UPC moat

The only path that makes a no-UPC line bind deterministically: once a (customer, part#) is confirmed once, every repeat binds at ~100% via the existing exact-xref bucket — a compounding data moat. The threads found the real fix + bugs: the training pairs are the non-UPC tokens in extracted_order_items.item_ids[] (NOT the empty customer_part_number column — 0/6,126; NOT the golden UPCs — only 15 alias pairs), keyed per (connection, erp_customer_id) → numeric erp_item_id; the item_id-namespace self-gate bug must be fixed so mined xrefs survive. Caveat — supply-limited: CK’s validated history is ~171 lines / 17 customers on 4f234677; the no-UPC accounts (Walmart, etc.) barely appear yet, so this compounds over time rather than fixing the cold start. Test: counterfactual --strategy xref-union — write mined aliases as order_history cross_references, run xref-exact-first then broaden-union fallback.

4. Vector / semantic recall (Typesense name_embedding) — no-UPC cold-start complement

Live-verified: name_embedding (built-in MiniLM) auto-embeds and is populated for CK on the cluster; whole foods original kimchi → WFM 365 Original Kimchi at distance 0.279 — exactly the zero-keyword-overlap retailer phrasing the current Jaccard≥0.3 name gate rejects. Add a connection-gated vector name sub-search to findProductMatchesBatch, recall-only, ranked below the deterministic buckets, with a precision gate. Critical caveats: (a) the worktree branch’s typesense.py lacks the name_embedding schema even though the live cluster has it — verify/port before building; (b) pure vector, not naive hybrid — fusion demoted a correct match on a live test (the “Whole” false-friend); (c) a flat distance gate makes wrong-retailer worse (GV chips → generic at 0.22) — vector must never be the decider. Test: --strategy hybrid (pure-vector bucket) vs baseline on the no-UPC subset.

Cross-cutting priority: wrong retailer/customer (see doc 05)

Before/above the above, the email from: → customer-narrowing lever fixes the dominant wrong-customer failure that cascades into wrong matches. Highest priority.

Next: parallel bake-off (existing harness)

broaden-union vs llm-rerank vs xref-union (counterfactual) vs hybrid-vector, head-to-head over the 66-order set, with abstention ground truth labeled. All org-scoped; measured against a held-out split (doc 06) to bound overfit.

Caveats (apply to all)

• Every lift number is on 66/699 POs — overfit risk; require a held-out split + per-org metrics (doc 06). Overfitting to CK is acceptable only because config is org-scoped.
• Cross-encoder rerankers / stronger embedding swaps (Cohere/Voyage/bge) = research bake-off data points, not the default.