09 — Ship-To Resolution (turning the 93.7% ceiling into a real number)

Thesis: doc 05 proved domain + ship-to pins the WhereFour customer 100% on ground-truth ship-to and 93.7% under a production-realistic leave-one-out regime — but that was a ceiling, gated on two unmeasured quantities: ship-to extraction quality and the resolver that consumes it. This doc closes both. Measured end to end: the resolver hits 93.5% top-1 under EXTRACTED ship-to — within 0.6pp of the truth ceiling. Extraction is not the bottleneck; the residual gap is data coverage, not algorithm. The lone domain needing special handling is AWG (no street address on its eBilling docs), solved by a 9-row warehouse→DC table that extracts at 100%.

This pairs with doc 08 (extraction) and doc 05 (the resolver’s narrowing logic): doc 08 makes extraction emit clean structured ship_to; this doc measures that emission and specifies the resolver that turns domain + ship_to into a WhereFour customerId.

Measured ship-to extraction quality (real, EXTRACTED ship-to, n=154)

Ran the offline broker extraction lab over all 154 eligible broker docs (no sampling, --limit 0, 154/154 extracted OK) with the v1 prompt + structured ship_to schema:

cd /Volumes/DevStorage/Developer/Dev/erp-unlocked && set -a && source .env && set +a
./.venv/bin/python packages/eval/scripts/ck-broker-prompt-lab.py \
  --prompt packages/eval/prompts/ck-broker-v1.txt --structured \
  --label shipto-all --limit 0 --concurrency 8

shipto_hit (lab ck-broker-prompt-lab.py:48) = extracted city OR 5-digit postal OR line1 (street-number + distinctive street word) matches the golden truth ship-to.

metric	result
ship-to captured (non-empty)	139/154 = 90.3%
ship-to DC-correct	115/154 = 74.7%
supplier-grabs (our brand as customer)	0/154 = 0.0% ✓
retailer-brand correct	135/154 = 87.7%
exact customer	99/154 = 64.3%

Per domain — a clean split. Address extraction solves four of the five domains; the 74.7% aggregate is dragged down almost entirely by AWG, which carries no street address (eBilling portal — label only):

domain	docs	captured	DC-correct	verdict
kehe.com	25	100%	25/25 = 100%	SOLVED (address)
unfi.com	7	100%	7/7 = 100%	SOLVED (address) — note brand only 1/7, yet DC 7/7
johare.com	78	100%	77/78 = 98.7%	SOLVED (address)
empirefoods.com	7	100%	6/7 = 85.7%	SOLVED (address)
awginc.com	37	59.5% (label only)	0/37 = 0.0%	NEEDS warehouse path

Excluding AWG: 115/117 = 98.3% DC-correct on extracted ship-to. The 2 non-AWG misses are genuine multi-ship-to PDF ambiguity, not prompt defects: empirefoods PO 80157 (extracted KeHE Ellettsville IN; truth Kroger Blue Ash 45242) and johare PO 3087841 (extracted DeMoulas Wilmington MA; truth North Haven 06473). Defer — 2/117, addressable later by preferring the ship-to block nearest the PO line items.

unfi is the proof: brand-wrong 6/7 (extracts “UNFI”, not the division) yet DC-correct 7/7. This confirms doc 05’s thesis mechanically — ship-to, not customer_name, is the DC-granular signal, which is exactly what the resolver exploits.

Lab output (resolver input): /tmp/ck-prompt-lab/shipto-all.json — results[] each carry {po, domain, truthCustomer, shipTo(truth), extracted(customer), addr(extracted ship-to), whse(warehouse_code)} plus a byDomain rollup.

The resolver — REAL top-1 vs the truth ceiling

packages/eval/scripts/ck-shipto-resolver.ts (bun) implements and measures the production resolver logic. Per broker order O (sender domain D, extracted ship-to from shipto-all.json): candidates = truth WhereFour customers seen for D across golden threads except O’s own thread(s) (leave-one-out, no leakage); each candidate is scored by ship-to token match (city / 5-digit postal / line1 number + distinctive street-word) against its held-out historical ship-tos, with the AWG warehouse-code table as a fallback.

regime	top-1	top-3
production name-only (today)	64.3% (99/154)	—
resolver, EXTRACTED ship-to	93.5% (144/154)	94.2% (145/154)
resolver, TRUTH ship-to (ceiling)	94.2% (145/154)	—

Extraction tax = 0.6pp (truth 94.2% − extracted 93.5%). Reproduces doc 05’s ~93.7% ceiling and confirms extraction quality is not the bottleneck.
Resolver lift = +29.2pp over production name-only (64.3% → 93.5%).
By domain (extracted top-1): awginc 35/37, johare 75/76, kehe 25/25, empirefoods 7/7, unfi 2/7.
15/154 (9.7%) AWG orders have NO extracted street ship-to (warehouse-code only) — all 15 resolved correctly via the warehouse-code table.

The 10 remaining misses are coverage, not algorithm: 7 are leave-one-out-starved (the DC appears in exactly one golden thread, so holding it out leaves zero history — all 5 unfi DCs + 2 AWG; these resolve in production where prior history exists), and 2 are genuine (SpartanNash shipping via a “Caito Foods” alias; one true extraction error). These self-heal as production history accumulates.

Two design rules the build surfaced (both load-bearing):

Ship-to must dominate the warehouse code (resolver weight 1000 for ship-to; whse is a fallback used only when ship-to gives zero signal). A flat warehouse-code boost let johare’s generic line numbers (whse “1” → Four Seasons and Weis) override a correct ship-to match.
Match line1 on street-number + a distinctive street word (alpha, len≥4, non-stopword), never a bare house number. A bare “400”+1-char token gave Bozzutos 25 false hits against a Four Seasons “400 Wabash Road” address.

AWG warehouse→DC table + coverage

AWG eBilling docs carry no street address (0/37 DC-correct via text) but a numeric Whse: code that extracts at 37/37 = 100% and maps 1:1 to a DC (perfectly consistent in the data — each code → exactly one DC). This is the seed table:

Whse code	DC (city)	postal
0002	Kansas City	66106
0102	Springfield	65802
0302	Oklahoma City	73179
0502	Goodlettsville (Nashville)	37072
0602	Pearl River (Gulf Coast)	70452
1102	Norfolk	68701
1202	Kenosha	53144
1402	Hernando	38632
1602	St. Cloud	56301

Coverage: the 9 codes cover 100% of the AWG corpus (37/37). AWG is the only domain routed through this path; the other four resolve on the extracted street ship-to.

Production implementation plan

The resolver is a pre-match customer-resolution step. It plugs into the three places where the customer is resolved today (doc 05 §1 enumerated them); all are CK/org-scoped via the doc-06 matchingConfig.customerResolution block — a config-less org is unchanged.

Where it plugs in (all three resolution sites)

pdf_extraction match_customer (workflow/pdf_extraction.py:490-497) — the primary automated path. Before calling match_customer(customer_name, connection_id), run the resolver: pass (sender_domain, extracted ship_to); the resolver returns a WhereFour customerId; fall back to name-match when it abstains (cold-start ship-to, non-broker doc). Domain comes from email_events.from_email via pdf_documents.email_event_id (doc 05).
review-v2 customer picker — pre-rank the operator’s candidate list by the resolver’s score so the correct DC is the default selection (mirrors doc 05’s pre-rank).
LLM validator search_customers — add a domain+ship-to narrowing step (a search_customers_by_shipto tool + a SYSTEM_PROMPT step-1 change) so the validator’s candidate set is already DC-resolved, not name-only.

The structured `ship_to` schema field

The resolver’s input is the structured ship_to {descriptive_name, line1, city, state, postal_code} that doc 08 §2a adds to GEMINI_RESPONSE_SCHEMA (packages/pdf-shared/pdf_shared/llm/schemas.py:19-29) and CustomerInfo (models.py:84-105). This doc is the consumer of that field; doc 08 is the prerequisite that emits it. The resolver reads ship_to.city / ship_to.postal_code / ship_to.line1 (the lab’s shipto_hit keys) — never the free-text customer_shipping_address.

The AWG warehouse path

For awginc.com docs (no street ship-to), the resolver routes on warehouse_code instead of address. Two pieces:

Extraction: capture the numeric Whse: code (doc 08 notes this; the lab’s structured schema already extracts it — ck-broker-prompt-lab.py:136). Surface it as an optional ship_to-adjacent field (e.g. warehouse_code) so the resolver can read it.
Lookup: seed the 9-row AWG warehouse→DC table above into the org config (matchingConfig.customerResolution.disambiguation — a per-domain warehouseDcMap alongside the existing poPrefixRules / useShipToAddress). Hardcode/seed it; it is stable and 1:1.

Org-scoping & rules

All of the above lives under CK’s org (org_3DisstYyUfNutmFA68xw5hxXfq1) / connection (4f234677-7bc5-43c1-9059-bd616b33ef75), defaulted OFF for every other org, per doc 06.
Scoring rules (port verbatim from the resolver): ship-to dominates (weight 1000); warehouse code is a fallback used only when ship-to gives zero signal; line1 matches on street-number + distinctive street word, never a bare house number.
Add an alias/ship-to-name bridge for the rare Caito-style cases (SpartanNash via “Caito Foods”) — a small per-org alias map; low priority (1 case).

Expected production effect

Routing awginc on the warehouse path and the other four on extracted street ship-to lifts broker DC-granular customer resolution from 64.3% (name-only, production today) to a measured 93.5% top-1 under extracted ship-to — within 0.6pp of the ceiling. The honest residual (single-thread DCs, rare alias ship-tos) shrinks as production history accumulates and is not an algorithm limit.

Caveats (recorded)

The 93.5% is under leave-one-out-by-thread hold-out; 7 of the 10 misses are history-starved DCs (one golden thread each) that resolve in production. The number is therefore a floor for accounts with prior history and a true cold-start cost for first-occurrence DCs (13/207 corpus-wide in doc 05).
Two non-AWG DC misses are genuine multi-ship-to PDF ambiguity (empirefoods PO 80157, johare PO 3087841) — a known, deferrable extraction edge, not a resolver defect.
The AWG table is learned from this corpus (9 codes, all 1:1). A new AWG division not in the corpus is a cold-start miss until its code is added — the table is data, not logic.

Artifacts (code)

Lab (ship-to extraction): packages/eval/scripts/ck-broker-prompt-lab.py (shipto_hit:48, structured warehouse_code:136)
Prompt: packages/eval/prompts/ck-broker-v1.txt
Resolver (built + measured): packages/eval/scripts/ck-shipto-resolver.ts (run: cd <repo> && bun packages/eval/scripts/ck-shipto-resolver.ts)
Lab output / resolver input: /tmp/ck-prompt-lab/shipto-all.json (154 docs, extracted ship-to + warehouse_code, byDomain rollup)
Golden (ship-to + threadId history): packages/eval/test-data/golden-dataset/cleveland-kitchen-golden-dataset.json