CK Discount Intelligence Layer

Status: Research complete — architecture proposed, awaiting decisions (see §7) Owner: David Boone Trigger: Goal to make CK custom-line discount matching “wow” customers — reliably find/match discount definitions whose names have no consistent format ("Walmart Sauce Allowance" vs "Giant Eagle - 1616CC" vs universal), and proactively suggest from customer history during order review. Repo paths in this doc are relative to repo root. Sibling doc: docs/designs/typesense-matching-architecture.md (product/xref matching) — this doc reuses its primitives (built-in MiniLM auto-embedding, multi_search, alias swap, confirmation-count learning loop).

1. TL;DR — recommendation

Layer a retrieve-then-LLM-rerank intelligence stage on the existing Typesense store and existing deterministic resolver — do not rebuild. Three stacked layers, cheapest-first:

Deterministic fast path (keep as-is, high precision): the current token-boundary SKU + customer-prefix matcher in discount-resolver.ts. Resolves clean customer + SKU cases in-process with zero added cost. This stays the precision floor.
Hybrid recall (new, for the no-SKU / free-text gap): add a server-side auto-embedding field to the custom_line_definitions collection so a hybrid (keyword + vector) query can retrieve candidate defs from order context (customer, retailer, product name/brand/category) even when no SKU is present — which is exactly the case the deterministic path can never match today.
LLM “select-or-0” rerank (new, gated): when the deterministic path is empty/ambiguous, an LLM picks the best candidate from the retrieved list or abstains (0 = none), emitting a confidence score. Below threshold → fall back to the manual search path that already exists in DiscountMatchPanel.tsx.

Plus a per-customer history prior (operator pins are already persisted — see §4.3) powering “you usually apply X here,” and a golden set built from those pins to calibrate the abstention threshold and measure recall.

Scope correction (important): the premise “there’s no real intelligence, just basic Typesense lookup” is not accurate to the code. The resolver already does token-boundary SKU matching, customer-token prefix classification, a numeric-id→SKU bridge, and ranked customer_sku/sku_only suggestions. The real gap is free-text / no-SKU / universal lines (e.g. "Walmart Sauce Allowance"), which the SKU-keyed path structurally cannot match. The intelligence layer should augment the resolver for those, not replace it.

Decisions locked (2026-06-14, David): embedding is connection-scoped (WhereFour only — discounts don’t exist on other connections); the engine runs synchronously but batched, with two UX modes — bulk apply after auto-validate (Ctrl+Shift+V) and a per-line “relevant discounts” panel (§4.7); we update the Typesense SDK/client to match the prod cluster (~v29+); auto-apply stays very strict (deterministic-only; hybrid/LLM/history are always operator-confirmed); history starts as a tie-breaker. Details in §7.

2. The problem & root cause (grounded in code)

Every current matching path keys on one signal: does the validated SKU token appear inside the definition name?

resolveOrgDiscounts (auto-apply at submit/refresh): for each line’s SKU, skuBoundaryIndex(name, sku) must hit; then prefix is classified customer / universal / descriptive; customer wins, else universal; descriptive never auto-applies.
listCustomLineItemsForCustomerSku (ranked suggestions): same SKU-token gate, returns all hits ranked customer_sku > sku_only.
searchCustomLineDefinitions (manual): plain keyword q over name,barcode.
SKU identifier bridge fetchProductSkus: numeric erp_product_id → SKU code via the products collection (so the resolver gets 1616CC, not 775178).

Consequence — the gap:

Def name example	Matches today?	Why
`"Giant Eagle - 1616CC"`	✅ customer_sku	SKU `1616CC` token in name, customer token `eagle` in prefix
`"1616CC UNFI Promotional Allowance"`	✅ sku_only (universal)	SKU token in name, empty prefix
`"Generic Off Invoice Discount 1616CC"`	⚠️ suggestion only	SKU token in name, descriptive prefix → never auto-applied
`"Walmart Sauce Allowance"`	❌ invisible	No SKU token in the name at all. Auto-apply and per-SKU suggestions both return nothing; only findable if the CSR manually types “walmart”/“sauce”.

The discriminating signal for the hard cases is retailer/customer + product category/brand + allowance semantics, not a SKU code. As CK adds more free-text lines, the SKU-keyed matcher’s recall on them stays at zero. That is the structural problem to solve.

3. What the research says (cited, adversarially verified)

Six findings survived 3-vote adversarial verification (28 sources fetched, 25 claims verified, 4 killed). Full report: workflow wf_e7cbaf57-5c2.

F1 — Typesense natively does hybrid + server-side auto-embedding (high confidence). query_by can list keyword fields and an auto-embedding field for hybrid search in one query; an embed property auto-vectorizes documents at index time and the query at search time, using a built-in model (ts/all-MiniLM-L12-v2, 384-dim, no API key) or remote (OpenAI/Vertex). Results merge via rank fusion (0.7·keyword + 0.3·vector, tunable via alpha). Caveat: vendor docs themselves log real hybrid-quality issues (GitHub #2060 keyword hits dropped, #1964 fusion over-weights doc-id, #2702 curation breaks with rerank_hybrid_matches); the model name must be set explicitly. → Validate hybrid quality on CK strings; don’t assume. Sources: vector-search, semantic-search, ai-agents.

F2 — “Selecting” from a candidate list beats per-pair / direct LLM matching (high). An LLM that picks the best candidate from a retrieved list (output {0,1,…,n}, 0 = none) improves entity-matching F1 by +16.02% avg across 8 datasets / 10 LLMs (COLING 2025, Match, Compare, or Select?), and beats LLM-alone on all 9 datasets, recall-driven (CE-RAG4EM, arXiv 2602.05708). This is exactly “Typesense supplies candidates, LLM decides.” Caveat: the two most-central claims split 2-1 — the headline best system is the compound (ComEM); adding retrieval can cost precision on noisy records, so thresholds/abstention are mandatory. Sources: COLING 2025, CE-RAG4EM, Typesense conversational RAG.

F3 — Confidence + abstention: use “none of the above” + verbalized confidence, calibrate on a small golden set (high). Best mechanisms in evidence order: (a) the 0=none option gives explicit abstention; (b) verbalized confidence (LLM states a score) is better-calibrated than raw token probabilities (~50% relative ECE reduction for strong RLHF models); (c) a tiny linear-regression calibrator over candidate scores beats raw-score heuristics by ~10 mAP and needs <40–50 labels; (d) preserve the retriever’s order on high-confidence queries — only intervene when uncertain. Caveat: verbalized confidence is model/prompt-sensitive and can be overconfident — calibrate, don’t assume. Sources: COLING 2025, Just Ask for Calibration (arXiv 2305.14975), Towards Trustworthy Reranking (arXiv 2402.12997), LCR (Expert Systems w/ Apps 2026).

F4 — Embeddings drive RECALL, never the final decision, for short noisy strings (high). Every tested sentence encoder (S-BERT, USE, LASER, InferSent, Doc2vec) failed antonym-replacement and word-jumbling; modern models stay negation/antonym-insensitive. CK discount strings hinge on a single discriminating token — Walmart vs Whole Foods, allowance vs chargeback, 105GDC vs 105GDCDC. Pure cosine could rank the wrong-retailer def top-1. → This is the decisive argument for keeping the deterministic token-boundary matcher as the precision layer and using embeddings only to widen recall. Sources: Daunting Dilemma w/ Sentence Encoders (arXiv 2309.03747), NevIR (arXiv 2502.13506).

F5 — LLM reranker is training-free, black-box, plug-and-play (high). No fine-tuning or logit access — a single text-in/text-out call (we already wrap the Anthropic SDK in packages/ai). Caveat: real per-query latency (~+0.5–1s) and cost → gate behind the cheap deterministic matcher; invoke only on ambiguous/no-SKU lines, never every line. Source: LCR + RankGPT lineage.

F6 — Per-customer history as prior + HITL labels (medium; codebase-grounded). Operator pins are already written to extractedOrderItems.customLineDefinitionId via apply-discount.ts and preserved across re-resolution — a ready-made label sink of (extracted line text, customer, chosen def) tuples. Aggregated per (customer, definitionId) they form the history prior; they also seed the first discount golden set (none exists yet — packages/eval/test-data/synthetic has only product/email/image truth). Caveat: no cited source studies per-customer discount-history priors specifically; this is an evidence-adjacent design inference. Manage feedback-loop bias (operators reinforcing past picks) with a hold-out eval and an exploration allowance. Sources: design synthesis + calibration from arXiv 2402.12997; codebase (apply-discount.ts, pdf-documents.ts).

Do NOT build (refuted): special confidence-token fine-tuning (<CN>/<UN>, refuted 0-3); “abstention improves quality at near-zero cost” as a blanket claim (1-2); intrinsic-logit confidence as a standalone rerank signal (1-2); “Typesense eliminates ALL sync logic” (1-2 — you still own the collection sync; only the embedding call is absorbed).

4. Recommended architecture

4.1 Three-layer matcher

Extracted custom line (text, amount) + order context (customer, retailer, matched product name/brand/category, qty)
        │
        ▼
┌──────────────────────────────────────────────────────────────┐
│ Layer 1 — DETERMINISTIC (existing, in-process, ~ms)            │
│ resolveOrgDiscounts: SKU token-boundary + customer prefix      │
│  → confident customer_sku/universal hit?  ── YES ─► auto-apply │  (unchanged behavior)
└───────────────────────────────┬──────────────────────────────┘
                                 │ NO / ambiguous / no SKU
                                 ▼
┌──────────────────────────────────────────────────────────────┐
│ Layer 2 — HYBRID RECALL (new, Typesense multi_search)          │
│ keyword query_by: name,match_text                              │
│ + vector_query: name_embedding:([], alpha:0.35, k:20)          │
│ query string = customer + retailer + product name/brand/cat    │
│  → top-K candidate defs (with facets, amount, scope, history)  │
└───────────────────────────────┬──────────────────────────────┘
                                 ▼
┌──────────────────────────────────────────────────────────────┐
│ Layer 3 — LLM SELECT-OR-0 (new, gated, 1 Anthropic call)       │
│ input: line text + context + top-K candidates (+ history)      │
│ output: index 1..K | 0 (none)  + verbalized confidence         │
└───────────────────────────────┬──────────────────────────────┘
                                 ▼
                   Confidence gate (§4.2)
        high ─► confident suggestion / auto-apply (per policy)
        low / 0 ─► ABSTAIN ─► manual "Search all definitions" (existing)

Layer 1 keeps clean cases fast and precise (F4, F5). Layer 2 fixes the recall gap for no-SKU/free-text lines (F1, F4). Layer 3 makes the precision decision over a small candidate set and abstains when unsure (F2, F3). Both invocation modes — batch (auto-validate) and per-line (manual) — call this same engine; see §4.7.

4.2 Confidence & abstention

Compute a single confidence from: deterministic-match strength (exact customer+SKU = 1.0) · hybrid rank_fusion_score · LLM verbalized confidence · history prior (§4.3). Policy:

Start simple: verbalized confidence + a hand-set threshold; 0=none always abstains. (F3a/b)
Then calibrate: once ~40–50 pins accumulate, fit the tiny linear calibrator over candidate scores and set the threshold on the golden set. (F3c)
Abstain → manual. Below threshold or 0, surface nothing as “confident” and route to the existing manual search / “closest def” hint in DiscountMatchPanel.tsx (already has DegradedView / EmptyView / closestDef). Abstention is a feature, not a failure — never silently auto-apply a low-confidence free-text match.

4.3 Per-customer history prior + proactive suggestions (“the wow”)

Source (already persisted): extracted_order_items.customLineDefinitionId + customLineDefinitionName (pdf-documents.ts:516). Submit re-resolution merges pins over auto-winners (“must NOT overwrite this”).
Aggregate: per (connection_id, customer, definition_id) → confirmation_count, last_seen_at. Mirror the xref-learner pattern in the sibling doc: confirmation_count >= 2 → trusted; first occurrence = suggested.
Use three ways: (1) ranking boost in the candidate list; (2) a feature in the confidence score; (3) proactive “you usually apply Walmart Sauce Allowance for this customer + this product” surfaced in the panel before the CSR searches — this is the wow moment.
Bias guard: hold-out eval + small exploration allowance so a better, newer def can still surface.

4.4 Schema / field changes — `custom_line_definitions`

In apps/dagster/erp_pipeline/assets/typesense.py (CUSTOM_LINE_DEFS_SCHEMA); the resource already auto-PATCHes missing fields (ensure_collection):

match_text: string — concat of name + parsed facets + aliases (built at sync time).
name_embedding: float[] with embed.from: [name, match_text], model ts/all-MiniLM-L12-v2 (same model the product doc chose).
Offline canonicalization facets (the research’s “blocking/canonicalization” for fuzzy matching) — parse each def name once at sync time (regex + optional LLM parse), store: retailer/customer_token: string, sku_tokens: string[], category: string, allowance_type: string, is_universal: bool. These make retailer/category filtering and the LLM prompt precise.
Re-index via a collection alias (custom_line_definitions → _v2) for zero-downtime rebuild (house pattern).
History: a small Dagster aggregation asset (pins → (connection_id, customer, definition_id, confirmation_count, last_seen_at)) materialized to Postgres and/or a tiny discount_history Typesense collection for query-time joins.

4.5 Where it plugs in (grounded file map)

Concern	File	Change
Candidate assembly + suggestions	`discount-suggestions.ts`	Add hybrid `multi_search`; merge with deterministic candidates; attach history + confidence; optional LLM rerank
Auto-apply (submit/refresh)	`discount-resolver.ts` + `submit.ts`	New gated LLM path for confident matches; abstain otherwise (current behavior is the floor)
Batch suggestions (auto-validate / Ctrl+Shift+V)	`auto-validate.ts`	Batched hybrid + LLM across all matched lines; reuse existing `oiCache`; feed the bulk-apply UX (§4.7)
Pin / label capture	`apply-discount.ts`	No change to capture; ensure extracted line text is associable for the golden set
UI	`DiscountMatchPanel.tsx`	Render confidence, “from history” badge, abstain→manual; extend existing degraded/empty/closest states
LLM call	`packages/ai`	Single Anthropic select-or-0 + verbalized-confidence call
Python parity	`org_discount_resolver.py`	Keep parity for the deterministic layer; LLM/hybrid live in the webapp path

4.6 Alignment with the product-matching doc

Reuse, don’t reinvent: built-in ts/all-MiniLM-L12-v2 auto-embedding, multi_search precedence buckets, vector_query with alpha≈0.35, alias-swap re-index, and the confirmation_count >= 2 operator-confirmation learning loop are all already chosen in typesense-matching-architecture.md. Dependency: embeddings are Phase C there and not yet live on any collection — so the discount hybrid layer should either ride that Phase C or be the canary that proves embed.from works on our deployed Typesense (see §7).

4.7 Invocation modes & UX — batch (auto-validate) vs per-line (manual)

The intelligence is one engine — resolveDiscountIntelligence(lines[], orderContext) — that runs the three layers and returns a per-line result { line, candidates[], best, confidence, source, fromHistory }. Two call sites differ only in batch size and presentation. Both are synchronous.

Mode A — Batch (primary): auto-validate / auto-match (Ctrl+Shift+V). This is the flow operators prefer, so optimize for it. After auto-validate matches lines to ERP products, run the engine across all matched lines in one batched pass:

One batched hybrid multi_search (Typesense takes many sub-queries per request) covering every ambiguous/no-SKU line.
One (or a few chunked) batched LLM “select-or-0” calls — each line carries its own candidate set, structured output per line — instead of N sequential calls. Cheaper and lower-latency.
Reuse the existing per-SKU oiCache dedup in auto-validate.ts.
UX: a consolidated “Discounts” review surface (an interactive evolution of AppliedDiscountsCard): “We found N applicable discounts across M lines.” Each row = line → proposed def + confidence chip + source badge (deterministic / suggested / from history). High-confidence deterministic matches are pre-checked for one-click [Apply all]; hybrid/LLM matches render as unchecked “review” rows. The operator clicks Apply (bulk) — one action, but explicit. Abstained lines show “no confident match — search.”

Mode B — Per-line (secondary): manual, one line at a time. The existing per-line DiscountMatchPanel.tsx becomes a true “Relevant discounts” panel powered by the same engine: ranked relevant defs for THIS line with confidence + “from history” badges; pick to apply. Same gating, same abstention → manual fallback.

Strict-but-easy reconciliation. The very-strict auto-apply policy means nothing is applied without an explicit operator action. Batching makes that action a single click over a pre-vetted set; it never lowers the bar to silent application. High confidence changes the default checkbox state and ordering — never whether a human confirms.

4.7.1 Surface A — locked spec (reviewed 2026-06-14 via /plan-design-review)

Three layout variants were generated and compared (mockup: ~/.gstack/projects/ERP-Unlocked-ordermatic/designs/discount-batch-surface-20260614/). Locked decisions:

Layout — line-ordered (Variant B). Rows follow PO line order (01..n), matching the operator’s existing top-down scan and the per-line panel’s mental model. Each row: line number · product name + SKU (mono) · matched definition · source pill · amount (mono, right-aligned).
Strengthen the ready-vs-review signal (Variant B’s known weak spot). Confidence isn’t grouped, so carry it per-row beyond the pill: a left accent stripe + pre-checked checkbox on ready (deterministic customer/universal) rows; unchecked + neutral “Review” treatment on hybrid/LLM/history rows; abstained rows inline at their line position with no checkbox and a “Search” link. Footer “Apply N selected” with a running total (grafted from Variant C) is the bulk action.
Context preservation (D3 constraint). The surface must never strip the operator’s order context. Default to a side-by-side view that keeps the PO/line list visible (right-rail or split), AND make each row self-sufficient: it carries the extracted line text, SKU, matched product, amount, and a “why this match” affordance (definition name + which signal fired: customer-token / SKU / semantic / history + the doc line reference). A full-screen overlay is acceptable ONLY if it embeds all of that per row — never an overlay that hides the PO without the context coming with it.
Precompute + staleness guard (D4 choice). Compute the intelligence set off the operator’s critical path — at extraction and on the existing refresh-discounts triggers — and persist it so the surface opens instantly, fully populated. Guard freshness: if the customer or matched-SKU set changed since precompute, re-run via the existing refresh-discounts path and show a brief “updating” state on affected rows. Fits the resolver’s existing extract→refresh→submit invocation model.
Interaction states: populated (precomputed, instant); updating (stale → re-resolving, per-row shimmer); empty (“No off-invoice discounts for this order” + manual add); degraded (lookup unavailable → reuse DiscountMatchPanel’s DegradedView + Retry; never silently drop the persisted set); abstain (inline “No confident match · Search”).
Confidence / source display: qualitative-first pills — “Customer match” (green), “Universal” (gray), “Suggested” (neutral), “From history ·N×” (blue) — with the numeric % as secondary text, not the headline (avoids false precision); full evidence on expand.
Keyboard + a11y: consistent with the per-line panel — ↑/↓ move rows, Space toggles a row checkbox, ⏎ applies the focused row, ⌘K search, ⌫ remove; ⇧⏎ = Apply all selected. Confidence/source never by color alone (pill text carries it); pill text uses the darker shade of each family (≥4.5:1 contrast); 44px touch targets; checkbox + Apply-all are real focusable, labelled controls.
Copy: utility voice — “Apply N selected”, “No confident match”, “From history · 4× for Walmart”, “Updating…”. No marketing/mood language.

4.7.2 Surface B — per-line panel deltas (extend, don’t redesign)

DiscountMatchPanel.tsx is already polished (Pinned/Loading/Degraded/Empty/Suggestion states, ⏎/⌘K/⌫ model, amber/green vocabulary). The intelligence layer adds only: a source/confidence line on each suggestion (Customer match / Universal / Suggested / From history ·N× + %); a proactive “From history” suggestion surfaced even when the SKU path returns nothing (today that hits EmptyView); and routing the LLM-abstain case into the existing EmptyView (+ closestDef + Search). Keep the keyboard model and color vocabulary unchanged.

5. Evaluation & golden set

No discount golden set exists yet. Build it from accumulated pins (F6), then:

Recall@k of Layer-2 hybrid candidates — does the correct def appear in top-K? (sizes the embedding investment)
End-to-end top-1 precision of the full pipeline vs operator pins.
Abstention precision/recall — when it abstained, was it right to?
Shadow mode first: run Layers 2–3 against historical pins, log decisions, compare to ground truth, before changing any UI or auto-apply.
Fixtures under packages/eval/test-data/synthetic/discount-matching/; results to eval-results/discount-matching-<date>.json (house pattern).

6. Phased rollout

Phase 0 (days) — audit + golden set. Pull the real ~119 CK custom_line_definitions + a sample of extracted discount lines/pins; quantify how many fall outside the SKU path (the actual miss rate). Materialize pins → first discount golden set. Answers the biggest open question before spending on embeddings.
Phase A (this week) — hybrid recall, no behavior change. First, align the Typesense SDK/client + local docker to the prod cluster (~v29+) and confirm the embed / vector_query surface. Then add match_text + name_embedding + facets to custom_line_definitions (alias swap, connection-scoped); extend discount-suggestions.ts to merge hybrid candidates into the suggestion list. Measure recall@k. Auto-apply untouched.
Phase B (week+) — LLM rerank + confidence, shadow → suggest. Add Layer 3 in shadow mode; calibrate the threshold on the golden set; then surface confident matches as operator-confirmed suggestions in the panel (not auto-applied).
Phase C (later) — history prior + proactive + selective auto-apply. Wire the pin aggregation, confirmation_count >= 2, proactive “you usually apply X” suggestions; consider auto-applying only high-confidence customer+history-confirmed matches. Hold-out eval + exploration allowance.
Gating: per-connection flag mirroring discountConfig.enabled.

7. Decisions (locked 2026-06-14, David) & remaining questions

Embedding scope — connection-specific. ✅ Discounts exist only on the WhereFour connection today, so embed/enable custom_line_definitions per-connection behind a flag; do not re-embed other tenants. Re-materialize only enabled connections.
Rerank timing — synchronous + batched, two UX modes. ✅ Run the engine synchronously, but batch it. Primary flow is auto-validate (Ctrl+Shift+V): batch suggestions across all lines for one-click bulk apply. Secondary flow is one-by-one: a per-line “relevant discounts” panel. Both designed in §4.7.
Typesense version — update the client. ✅ The prod cluster is reportedly ~v29+ (hybrid/auto-embedding supported natively); the lag is our SDK/client (dagster typesense Python pkg, webapp raw-HTTP surface) and the local docker-compose pin (0.25.2). Work = bump the SDK + local dev to match the cluster and confirm the embed/vector_query API surface — not a server upgrade. (Typesense Natural-Language-Search exists in newer versions but we don’t need it — our own LLM rerank is stronger.)
Auto-apply policy — very strict. ✅ Only deterministic high-precision matches (exact customer+SKU, and even there conservatively) may be pre-checked for bulk apply. Hybrid/LLM/history matches are always operator-confirmed — never silently applied. Batching makes confirmation one click; it never removes the click.
History weighting — tie-breaker first. ✅ Use the per-customer prior only as a ranking tie-breaker initially; revisit weight after a hold-out eval, with an exploration allowance to avoid reinforcement bias.
Surface A layout — line-ordered (Variant B). ✅ Rows in PO order; ready-vs-review signal strengthened per-row (accent stripe + checkbox) since it isn’t grouped. See §4.7.1.
Surface A placement — context-preserving. ✅ Side-by-side keeping the PO/line list visible AND self-sufficient rows (extracted text, SKU, matched product, amount, “why this match”). No context-stripping modal.
Surface A timing — precompute + persist + staleness guard. ✅ Compute at extraction/refresh off the critical path, persist, open instant; re-run when customer or SKU set changed.

Still open — best answered by Phase 0 data:

Exact prod Typesense version, and whether to use the built-in ts/all-MiniLM-L12-v2 or point embed at a remote model.
Real no-SKU miss rate + def-format distribution → sizes the embedding/LLM investment.
LLM batch chunk size (lines per call) vs latency/accuracy trade-off.

8. Risks & non-goals

Embedding overconfidence (F4): never let cosine similarity be the decider; it is recall-only.
Precision regression from retrieval (F2 caveat): widening recall can drop precision on noisy lines — the confidence gate + abstention exist to contain this; shadow-eval before enabling.
LLM cost/latency (F5 caveat): gate Layer 3; never run it per-line unconditionally.
Don’t build (F refuted): confidence-token fine-tuning; intrinsic-logit confidence as the signal; any assumption that hybrid is high-quality or that Typesense removes sync work.
Non-goal: replacing the deterministic resolver or the Python parity activity. They stay as the precision floor.

9. Critical files

apps/webapp/src/services/erp/wherefour/discount-resolver.ts
apps/webapp/src/pages/api/erp/orders/[id]/items/[itemId]/discount-suggestions.ts
apps/webapp/src/pages/api/erp/orders/[id]/items/[itemId]/apply-discount.ts
apps/webapp/src/components/orders/review-v2/DiscountMatchPanel.tsx
apps/webapp/src/pages/api/erp/orders/[id]/submit.ts
apps/temporal-worker/activities/org_discount_resolver.py
apps/dagster/erp_pipeline/assets/typesense.py (CUSTOM_LINE_DEFS_SCHEMA)
packages/db/src/schema/pdf-documents.ts (extracted_orders.customLineItems, extracted_order_items.customLineDefinitionId/Name)
packages/eval/test-data/synthetic/discount-matching/ (new)

10. Sources

Verified primary/peer-reviewed:

Typesense: vector-search · semantic-search · conversational RAG · natural-language-search · AI agents
Entity matching / selecting: Match, Compare, or Select? COLING 2025 · CE-RAG4EM arXiv 2602.05708
Confidence / abstention / reranking: Just Ask for Calibration arXiv 2305.14975 · Towards Trustworthy Reranking arXiv 2402.12997 · LCR, Expert Systems w/ Apps 2026
Embedding limits: Daunting Dilemma with Sentence Encoders arXiv 2309.03747 · NevIR arXiv 2502.13506 · Robust Negation arXiv 2507.12782
Record linkage / fuzzy entity matching: Ditto arXiv 2004.00584 · fuzzylink

Full machine-readable findings + vote tallies: deep-research run wf_e7cbaf57-5c2.

11. Design review (2026-06-14)

Ran /plan-design-review + an inline design-shotgun on the two UI surfaces. Initial design-completeness of the doc: 4/10 (surfaces described in prose, no IA / states / confidence-viz / copy / a11y). After this pass: ~8/10 (Surface A fully spec’d in §4.7.1, Surface B deltas in §4.7.2; placement, timing, states, keyboard, copy locked).

Key reframes from the review:

Surface maturity is asymmetric. Surface B (per-line) already exists and is polished — it needs spec deltas, not a redesign. Surface A (batched) is greenfield — that’s where the variants and the real decisions were.
Three variants compared (grouped-by-confidence / line-ordered / ledger-table). Chose line-ordered (operators think line-first; low switch-cost from the per-line panel), with B’s weak “ready-vs-review” signal strengthened per-row.
Context preservation is a hard rule (D3): the surface keeps the PO visible AND each row is self-sufficient.
Precompute + persist + staleness guard (D4): instant open, compute off the critical path, re-run on customer/SKU change.

Approved mockups

Screen/Section	Mockup Path	Direction	Notes
Surface A — batched discounts review	`~/.gstack/projects/ERP-Unlocked-ordermatic/designs/discount-batch-surface-20260614/`	Line-ordered (Variant B)	Strengthen ready/review per-row; footer Apply-N + running total; context-preserving; precomputed. Mockups are hand-built HTML in the real `--om-*` tokens (design binary needs an OpenAI key, not configured).

Implementation Tasks

Synthesized from this review. P1 blocks ship; P2 same branch; P3 follow-up.

T1 (P1) — review-v2 — Build Surface A: line-ordered batched discounts panel (Variant B) with per-row ready/review signal (accent stripe + checkbox), footer “Apply N selected” + running total, context-preserving placement.
- Surfaced by: §4.7.1; D2/D3 decisions
- Files: new apps/webapp/src/components/orders/review-v2/DiscountBatchPanel.tsx; wire from auto-validate flow + ReadinessFooter.tsx
- Verify: renders all states; bulk-apply only applies checked rows
T2 (P1) — resolver — Precompute + persist the intelligence set at extraction/refresh; staleness guard reusing refresh-discounts on customer/SKU change.
- Surfaced by: §4.7.1; D4
- Files: discount-resolver.ts, .../refresh-discounts.ts, .../submit.ts, .../parts/auto-validate.ts, packages/db/.../pdf-documents.ts
- Verify: surface opens instant from persisted set; re-runs when customer/SKU changed
T3 (P2) — review-v2 — Surface B deltas: source/confidence line on each suggestion + proactive “From history” when SKU path is empty + route LLM-abstain to EmptyView.
- Files: DiscountMatchPanel.tsx, .../discount-suggestions.ts
T4 (P2) — review-v2 — “Why this match” evidence affordance per row (signal fired + doc line ref).
T5 (P2) — a11y — Space toggles row, ⇧⏎ Apply-all, focus management, pill contrast ≥4.5:1, 44px targets.
T6 (P3) — review-v2 — Confidence display: qualitative pill primary, numeric % secondary, evidence on expand.

12. Engineering review (2026-06-14)

Ran /plan-eng-review. Scope locked to Phase A only (D1): hybrid recall into the existing per-line panel — no LLM, no batch surface, no precompute/persist, no history, no behavior change. B / C / Surface A explicitly deferred.

Architecture decisions:

Typesense version/SDK bump = separate infra PR first (D2). The bump (dagster typesense SDK + local docker 0.25.2 + webapp raw-HTTP vector_query) touches the shared Typesense powering product/customer/xref search. Land it alone with regression coverage on those searches, then build Phase A on the known-good base.
Embedding-model availability [P1 verify]: confirm the prod cluster has built-in ts/all-MiniLM-L12-v2 enabled, else configure remote embeddings + key. Verify in the infra PR.
Connection-scoped embedding: the field exists collection-wide (can’t be per-doc in the shared custom_line_definitions); hybrid query gated per-connection at query time. CK is the only connection with defs today, so the embedding cost is moot.
Ranking [P1]: hybrid-only candidates rank BELOW deterministic (customer_sku > sku_only), carry a distinct “Related” label, dedupe by definition_id. Embeddings are recall-only (research F4) — a semantic hit must never outrank a real SKU/customer match.

Code quality: one shared fetchHybridCandidates(connectionId, queryContext) helper in discount-resolver.ts alongside fetchAllDefs/fetchProductSkus (DRY, composable for Phase B). Define the query-context assembly explicitly: extracted line text + customer + matched product name/brand/category.

Failure modes:

Hybrid query fails (model missing / vector error) → would blank the panel. Must degrade gracefully to deterministic-only suggestions (reuse the DiscountResolverDegraded pattern). Captured as required task T2 + test T4.
SDK bump regresses existing search → covered by the separate infra PR’s regression suite (T0).
Wrong-retailer semantic hit ranked high → mitigated by rank-below-deterministic + “Related” label + operator-confirm (no auto-apply).

Test coverage (Phase A):

CODE PATHS
[+] discount-resolver.ts
  ├── fetchHybridCandidates()             [GAP] query-build · empty · vector-error→degrade
  └── listCustomLineItemsForCustomerSku() [GAP] dedupe · rank-below-deterministic · Related group
[+] discount-suggestions.ts               [GAP] hybrid candidates in response
[+] DiscountMatchPanel.tsx                [GAP] renders "Related allowances" group
[+] typesense.py                          [GAP] v2 collection + name_embedding + match_text + alias swap
INVARIANT
  └── strict-no-auto-apply   [GAP-CRITICAL] hybrid candidate NEVER enters resolveOrgDiscounts auto-apply set
REGRESSION (IRON RULE)
  ├── discount-resolver.test.ts           deterministic behavior unchanged
  └── typesense-search-service            product/customer/xref pass after SDK bump (in infra PR)
EVAL [→EVAL]
  └── recall@k on discount golden set (needs Phase 0 data; Phase A = synthetic no-SKU fixture)

NOT in scope (deferred): LLM select-or-0 rerank + confidence/abstention (Phase B); batch Surface A + precompute/persist/staleness guard (Phase B / Surface A); per-customer history prior + aggregation (Phase C); Python parity for the intelligence layer (stays deterministic-only — intelligence is webapp-only); learned confidence calibrator (Phase B).

Parallelization: the infra PR (T0) is a sequential gate. Within Phase A: Lane A = typesense.py schema/sync (embedding field + match_text + alias); Lane B = webapp retrieval/merge/UI. They share the field-name contract — coordinate on that, otherwise parallelizable.

Implementation tasks (Phase A — supersedes the design-review T1–T6 above):

T0 (P1, separate PR) — Typesense version/SDK/docker alignment to prod (~v29+); verify embed.from/vector_query + embedding-model availability; regression on product/customer/xref search. Files: apps/dagster pyproject + typesense_resource.py, docker-compose.yml, typesense-search-service.ts.
T1 (P1) — typesense.py: custom_line_definitions_v2 with name_embedding (embed.from [name, match_text]) + match_text build; alias swap; per-connection query gating.
T2 (P1) — discount-resolver.ts: fetchHybridCandidates + merge into listCustomLineItemsForCustomerSku (dedupe, rank below deterministic, “Related” group); graceful degrade on hybrid failure.
T3 (P2) — discount-suggestions.ts + DiscountMatchPanel.tsx: surface a “Related allowances” group below the SKU suggestions.
T4 (P1) — tests: hybrid helper + merge/rank/dedupe + strict-no-auto-apply invariant + deterministic-resolver regression + synthetic no-SKU recall fixture.
T5 (P2) — eval: discount golden-set scaffold (recall@k); depends on Phase 0 data.

GSTACK REVIEW REPORT

Review	Trigger	Why	Runs	Status	Findings
CEO Review	`/plan-ceo-review`	Scope & strategy	0	—	—
Codex Review	`/codex review`	Independent 2nd opinion	0	—	—
Eng Review	`/plan-eng-review`	Architecture & tests (required)	1	clean	scope→Phase A; TS bump = separate PR; 4 arch findings; degrade-handling required (T2/T4)
Design Review	`/plan-design-review`	UI/UX gaps	1	issues_open	score 4/10 → 8/10, 8 decisions locked, 3 minor deferred
DX Review	`/plan-devex-review`	Developer experience gaps	0	—	—

UNRESOLVED: none blocking (3 minor design details + Phase-0 verifies deferred to build).
VERDICT: DESIGN + ENG CLEARED for Phase A — ready to implement (Typesense infra PR first, then Phase A). Phases B/C deferred.