VMS POC — Extraction Analysis & Implementation Plan

May contain stale references

This document was audited on 2026-06-22 and contains references that need updating: vms-prompt-config.json is vastly more sophisticated than described. Cross-check critical details against the current codebase before relying on them.

Visual Marking Systems — Proof of Concept Date: 2026-02-19

---

1. Executive Summary

Visual Marking Systems (VMS) is a label/decal manufacturer in Twinsburg, Ohio. They receive purchase orders from industrial customers and generate quotes from their “Enterprise” MIS system.

Workflow: Customer sends PO → VMS user validates in Ordermatic review page → Ordermatic submits to Enterprise MIS → Enterprise MIS creates one quote per line item (not per PO). This is the target pattern for the quote_splitting_enabled feature flag.

Eval scope: 7 inbound customer POs are the extraction eval targets. 12 outbound VMS quotes are stored as reference material only (they show what the ERP output should look like). Key finding: several extraction patterns require custom prompt_config to handle correctly — no code changes needed for extraction. The quote splitting UX is gated behind quote_splitting_enabled in Flagsmith.

---

2. Document Taxonomy

Inbound Purchase Orders (7 documents)

Document	Customer	Items	Complexity	Key Challenges
PO-2512929	Gradall Industries	3 (same part)	Complex	Scheduled/blanket PO — same part 82173070 with 3 different due dates (02/19, 03/25, 04/22). Current “consolidate” prompt would merge these.
purchaseorder_101572_3013_109729	Facil NA de Mexico	1	Complex	International PO. European number format (22.500,00 = 22,500). Price per 1000 units. Dual part numbers (customer LL00736ADD = VMS 3518910C2).
PO 11372 VMS	Activities Press	1	Simple	Vendor Stock No (162474:01) and Job number (M28168). Ship-to is VMS itself.
S16661	Urschel Laboratories	1	Simple	Standard single-item PO. Part# 098141.
PO-0000024812	J. Sterling Industries	5	Medium	Canadian company (HST tax). Revision levels (AA.0, A.0, B.0). No vendor item numbers filled in.
PO1101011	Nelsen Corporation	2	Medium	Dual part numbers: customer Item ID (MYLAR LABEL-BT-LEHIGH-23175) + VMS Part No (23175BT-VLV). Different due dates per line (02/20, 03/12).
po_pdf_0926554087	Supply Technologies	3	Medium	Customer part numbers (4XH478351 etc) with VMS numbers embedded in descriptions (“Hyster Yale - 1627004”). Pack size info (PACK 25).

Outbound Quotes (12 documents) — Reference Only

All from VMS Enterprise system. These are NOT extraction input documents — they are VMS’s output quotes generated after a PO is approved through Ordermatic. They are stored as reference material to document expected ERP output fields and format.

PO → Quote mapping (quote splitting pattern):

Source PO	Output Quotes	Split Pattern
PO-2512929 (Gradall)	Gradall quote 1/2/3.pdf	1 quote per line item
purchaseorder_101572 (Facil)	Facil Quote.pdf	1 quote for whole order
PO-0000024812 (J. Sterling)	J. Sterling Quote.pdf	1 quote for all 5 items
po_pdf_0926554087 (Supply Tech)	Supply Technologies Quote 1/2/3.pdf	1 quote per line item
PO1101011 (Nelsen)	Nelsen Quote 1/2.pdf	1 quote per line item
S16661 (Urschel)	Urschel Laboratories Quote.pdf	1 quote for 1 item
PO 11372 VMS (Activities Press)	Activities Press Quote.pdf	1 quote for 1 item

---

3. Extraction Challenges

P0 — Critical for POC

1. Company Context Flip

VMS is the SELLER receiving POs, not a buyer/distributor. The prompt’s {{company_name}} context needs to identify VMS as the vendor so the SENDING company is correctly extracted as the customer. Straightforward via prompt_config.company_info.

2. Dual Part Numbers

4 of 7 POs have both customer AND VMS part numbers. Current item_ids array captures all IDs but doesn’t distinguish which is the customer’s vs vendor’s. For the POC, we handle this via prompt instructions: “Place the customer’s part number FIRST in item_ids, then the VMS/vendor part number SECOND.”

Where VMS part numbers appear varies by customer:

• Gradall: VENDOR ITEM # column
• Facil: Your part no.: field
• Nelsen: Part No.: below Item ID
• Supply Technologies: embedded in description after company name (“Hyster Yale - 1627004”)
• Activities Press: Vendor Stock No. column
• J. Sterling: Vendor Item Number column (empty on their PO)

3. Scheduled/Blanket POs

Gradall PO has the same part (82173070) on 3 lines with different due dates. The current prompt says “Single order: Consolidate all items into one order” which would INCORRECTLY merge these. Fix: Replace with “Each line item is a separate order_items entry, even if the same part appears multiple times.”

P1 — Important

4. Per-Line Due Dates

Gradall (3 different dates), Nelsen (2 different dates). Current schema has order_date at order level only. For POC: instruct extractor to put due dates in item_description (“Due: YYYY-MM-DD”). Long-term: add due_date field to OrderItem schema.

5. European Number Formatting

Facil uses period as thousands separator, comma as decimal (22.500,00 = 22,500). Price is “90,00 USD per 1000 ea” = $0.09/unit. Prompt must instruct conversion to standard US format.

6. Price-Per-Thousand

Facil prices at “90.00 USD per 1000 ea”. Actual unit_price should be 0.09. Prompt instruction: “Normalize unit_price to per-single-unit. If price is quoted per 100/1000/etc, divide accordingly.”

P2 — Nice to Have

7. Revision Levels — J. Sterling POs have part revisions (AA.0, B.0). Capture in item_description.

8. Job/Reference Numbers — Activities Press has a “Job” column (M28168). Capture in item_description.

9. Quote Document Handling — VMS quotes are outbound. Extractor should recognize customer_name = the entity being quoted (not VMS), and quotation number = po_number.

---

4. Implementation Approach

Phase 1: prompt_config (POC — Do Now)

• Set company_info for VMS org (name, location, aliases)
• Add detailed requirements and notes to handle VMS-specific patterns
• Override the “consolidate” behavior via prompt instructions
• No code changes required

Phase 2: Schema Extensions (Post-POC)

If the POC validates well, add to Gemini response schema (schemas.py):

Field	Type	Location	Purpose
due_date	string, optional	OrderItem	Per-line delivery date
customer_part_number	string, optional	OrderItem	Explicit customer reference
vendor_part_number	string, optional	OrderItem	Explicit vendor reference
unit_of_measure	string, optional	OrderItem	EA, PACK, etc.
revision	string, optional	OrderItem	Part revision level

These would also need corresponding columns in extracted_order_items table and frontend display changes.

---

5. prompt_config Design

Full JSON stored in erp_connections.prompt_config for VMS:

File: scripts/poc-setup/vms-prompt-config.json

{
  "company_info": {
    "name": "Visual Marking Systems",
    "location": "Twinsburg, Ohio",
    "aliases": ["VMS", "Visual Marking Systems, Inc.", "vmsinc"]
  },
  "requirements": [
    "CRITICAL: Each line item in a PO is a SEPARATE entry in order_items, even if the same part number appears multiple times with different due dates or delivery schedules. Do NOT consolidate duplicate part numbers into a single line item.",
    "For dual part numbers: When a document shows both a customer part number AND a vendor/VMS part number, place the CUSTOMER's part number FIRST in item_ids array, then the VMS/vendor part number SECOND.",
    "Always normalize unit_price to per-single-unit cost. If price is shown as 'per 100', 'per 1000', 'per M', divide accordingly.",
    "Handle European number formatting: periods as thousands separators and commas as decimal points (e.g., '22.500,00' = 22500.00).",
    "Capture per-line due dates in item_description when available (format: 'Due: YYYY-MM-DD'). order_date should be the PO issue date.",
    "For VMS Quote documents: customer_name is the entity BEING QUOTED. Quotation Number goes in po_number.",
    "Extract revision levels into item_description when present (format: 'Rev: [level]').",
    "Extract Job numbers, reference numbers, and drawing numbers into item_description."
  ],
  "notes": [
    "VMS is the SELLER/VENDOR on inbound POs. The sending company is the CUSTOMER.",
    "VMS part numbers appear in various fields: 'Vendor Item #', 'Vendor Stock No.', 'Part No.', 'Your part no.', or embedded in descriptions after company names.",
    "Pack quantity notations like '(PACK 25)' are packaging info, NOT order quantity.",
    "Some POs show Order/Recv/Due breakdowns — use 'Order' quantity.",
    "Common VMS customers: Gradall, Supply Technologies, Facil NA, J. Sterling, Nelsen, Urschel, Activities Press."
  ]
}

---

6. Eval Pipeline

Golden dataset created at packages/eval/test-data/golden-dataset/vms-golden-dataset.json.

• 19 entries total (7 POs + 12 quotes)
• 13 entries have full ground truth annotation
• 6 entries are placeholders pending annotation (remaining quote documents)
• Test PDFs stored at packages/eval/test-data/vms-documents/

Test scenarios covered: scheduled POs, dual part numbers, international formatting, European numbers, multi-item orders, price-per-thousand, pack quantities, multi-page documents.

Scoring priorities: customer_name accuracy, item_ids completeness (both customer + VMS parts captured), quantity accuracy, unit_price accuracy (especially price-per-thousand normalization).

Note: The 12 quote documents have testCategory: "reference" and shouldExtractOrders: false — they are skipped by the eval runner.

---

7. Org Setup Checklist

VMS POC

• Create VMS org in Clerk (slug: vms-poc)
• Add david+vms@ordermatic.co as admin
• Create ERP connection with vms-prompt-config.json
• Upload 19 test PDFs to R2 under vms-poc/ prefix
• Run eval pipeline against golden dataset
• Review extraction accuracy, iterate on prompt_config

Cleveland Kitchen POC

• Create Cleveland Kitchen org in Clerk (slug: cleveland-kitchen-poc)
• Add david+cle@ordermatic.co as admin
• Configure with cleveland-kitchen-prompt-config.json
• Await example documents from Cleveland Kitchen for eval dataset

Setup Script

Run: npx tsx scripts/poc-setup/setup-poc-orgs.ts

Requires CLERK_SECRET_KEY and DATABASE_URL env vars. Creates both orgs, sends invitations, and inserts ERP connections with prompt_configs.