Product Search Recall Improvement (the "kimchi" class)

Date: 2026-06-14 Author: Claude Status: Spec for review — no code yet. Siblings: typesense-matching-architecture.md (this refines its Phase C), ck-discount-intelligence-layer.md (reuses its retrieve→rerank→abstain pattern). Depends on: the Typesense v29 infra (PR #1648 — vector_query plumbing + cluster v29.1, built-in MiniLM verified enabled on prod).

1. Problem

Operators searching for products in review-v2 (ERPSearchModal → /api/erp/parts → TypesenseSearchService.searchProducts) frequently get no useful match and have to reword the query repeatedly. Staging logs caught the signature directly: ~12 product searches in 44 seconds, several within the same second — a human reformulating a query that won’t match. (Note: the exact terms weren’t recoverable from logs due to a separate logging bug, fixed in PR #1655.)

searchProducts is keyword-only: query_by: erp_product_id,ocr_variants,format_variants,name,upc,upc_case,search_text, num_typos: 2, prefix: true. No semantic layer.

2. Evidence (measured on live CK data, 1,375 products)

Clean tokens work (typo-tolerant): kimchi / kimchee / kim chi → all 128 kimchi products. Semantic phrasings fail — they keyword-match scattered tokens on the wrong products:

Query	keyword (today)	naïve hybrid (α=0.35)	naïve hybrid (α=0.8)	vector-only
”korean fermented cabbage”	1/5 kimchi (top: Korean chili flakes)	0/5 (Korean coleslaw)	0/5	3/5 ✅ (Classic/Spicy Kimchi)
“spicy fermented cabbage”	3/5 (kraut is also fermented cabbage — fair)	1/5	1/5	1/5
”fermented veg spicy”	0/5	1/5	1/5	1/5

(kimchi products in top-5; tested locally on typesense:29.1 with name_embedding via the built-in ts/all-MiniLM-L12-v2.)

3. The key finding — do NOT ship naïve hybrid

Adding name_embedding to product search as a fused hybrid query would regress recall, not fix it. For “korean fermented cabbage,” hybrid fusion (even at 80% vector weight) ranked Korean coleslaw over the kimchi products, because the keyword component token-matches distractors (“Korean”, “cabbage”) and rank-fusion lets those dominate. This matches the research caveats in the discount design (F1: Typesense hybrid fusion over-weights keyword/doc-id; F4: short noisy strings).

But the model is capable — pure vector-only surfaced the kimchi products top-3 for the same query. The semantic knowledge (“kimchi ≈ korean fermented cabbage”) is there; naïve fusion buries it under keyword noise. So the fix is to use the vector signal deliberately as a fallback / rerank input, not blended into the default keyword ranking.

4. Proposed approach (layered, cheapest-first, additive)

Keep keyword as the precision floor; add semantic recall only where keyword is weak.

1. Layer 0 — keyword (unchanged). Current searchProducts. High precision on literal/code/UPC matches. Stays the default and the precision floor.
2. Layer 1 — vector FALLBACK (not fused). When keyword returns zero or low-relevance results (reuse the existing relevance gate / result-count + Jaccard signals), fire a pure-vector query on products.name_embedding and surface those as a clearly-labeled “Did you mean / related products” group below the keyword results. Never blended into the keyword ranking (that’s what regressed). Operator picks — same confirm model as today. This alone recovers the “korean fermented cabbage” case.
3. Layer 2 — LLM query-understanding / rerank (gated). For ambiguous or still-empty queries, one LLM call either (a) rewrites the free-text query into catalog vocabulary, or (b) reranks the vector candidates and abstains if none fit (the discount design’s “select-or-0” Layer 3, reused). Gated behind Layer 0/1 being weak — never per-keystroke.
4. Layer 3 — curated synonyms (ongoing). For recurring, known mappings (e.g. kimchi ⇐ korean fermented cabbage / napa cabbage spicy), add Typesense Synonyms or ocr_variants entries. Deterministic, instant, no model — best for the handful of repeat offenders once we see them in logs (now possible post-#1655).

5. Schema / infra

• products gains name_embedding (embed.from: [name, description, manufacturer], built-in ts/all-MiniLM-L12-v2) — added via the existing additive-PATCH path (ensure_collection); verified on v29.1 that PATCH-adding an embed field re-embeds existing docs (no alias swap needed; same finding as the discount T1). Gated per-connection (CK first).
• Webapp vector_query support already shipped in #1648 (MultiSearchRequest.vector_query). The fallback uses it; or a query_by: name_embedding-only sub-search for pure vector.
• Build/demo around the retailer/semantic cases; measure before widening.

6. Evaluation

Unlike discounts (no operator pins yet — see the discount Phase 0 audit), product matching already has labels: extracted_order_items.erp_item_id + is_validated / match_confidence are operator-confirmed product picks. Build a recall@k golden set from validated CK items, focused on the semantic-miss class. Shadow-mode first: run Layers 1–2 against historical validated items, compare to keyword-only, before changing the operator UI. Fixtures under packages/eval/test-data/synthetic/product-search/.

7. Risks & non-goals

• Embedding noise on short strings (research F4): vector is recall-only and confirmed by the operator; never auto-selects.
• Fusion regression (measured): explicitly avoided — fallback/rerank, not blended fusion.
• Latency/cost: Layer 2 LLM gated behind weak keyword/vector; never per-keystroke.
• Non-goal: replacing keyword search or the deterministic match pipeline. This augments recall on the queries keyword can’t serve.

8. Phasing

• Phase 1 (small): products.name_embedding sync + vector-fallback in searchProducts (CK-gated) + shadow eval on validated items. No change to keyword precision.
• Phase 2: LLM query-understanding/rerank for the still-weak/ambiguous tail.
• Phase 3: synonyms/learning loop fed by the now-queryable search logs (#1655) — promote recurring reformulations into curated mappings.

9. Open questions

• Fallback trigger threshold: zero results only, or also low-relevance (and at what gate)?
• Embedding source: name only vs name+description+manufacturer (descriptions are sparse on CK; test both).
• Where the LLM query-understanding lives (reuse the discount engine’s packages/ai call) and its abstention threshold.
• Per-connection rollout: CK first; which connections benefit (those with descriptive catalogs vs pure part-number catalogs).