Inbound email: shared triage + intake contract

Status: Implemented (in packages/contracts/src/inbound-email.ts, apps/webapp/src/pages/api/email/triage.ts, and apps/webapp/src/pages/api/webhook/inbound-email.ts)

Two pipelines bring email into Ordermatic:

Pipeline	Trigger	Code
Push	Cloudflare Email Routing	apps/email-worker (order-filter.ts, webapp-client.ts)
Pull	Temporal schedule polling Gmail / Microsoft Graph	apps/temporal-worker/activities/gemini.py (triage_email, _triage_decide)

Push-vs-pull is inherent. Everything downstream of “we have a normalized email” is meant to be shared, and the sharing boundary is two webapp HTTP contracts, plus a third that only the push path uses to hand over attachment bytes.

Contract 1 — POST /api/email/triage (classification)

apps/webapp/src/pages/api/email/triage.ts is the only LLM triage implementation.

• Model: Anthropic Claude, fixed classify_email tool. Prompt resolved from Langfuse (label chain) with a compiled default fallback.
• Deterministic pre-checks run before the LLM: per-connection sender denylist (email_triage_exclusion_rules_v1), trusted-sender allowlist, and hold/unverified-PO agent overrides.
• Taxonomy (canonical): process_attachments | process_body | process_both | skip, plus confidence, reasoning, engineStatus.
• Auth: HMAC-SHA256 over the raw body with INBOUND_EMAIL_SECRET (X-Inbound-Email-Signature), or X-Eval-Token for the eval harness. There is no Bearer path.
• Failure contract: 503 when the LLM is not configured, 500 when the LLM call failed. Callers must treat both as “fall back to heuristics”, never as “drop the email”.
• Image previews (push path): an email with no real document (PDF/Word/Excel/CSV/text) sends its 10-750 KB images (at most 4, 1.5 MB in all) as base64 preview with previewType: 'image' and inline, because file size cannot tell an order table pasted into the body from a signature logo. With email_triage_image_preview_v1 on for the connection’s org, the model is shown them and the reply carries imagePreviewUsed: true plus one attachmentAnalysis verdict per image; the worker then uploads only the inline images judged order content (dropRejectedPreviewedImages). Flag off: previews are ignored and the reply is unchanged, so uploads behave as before. The pull path sends no previews.

Callers

• Push: apps/email-worker/src/webapp-client.ts callTriage(...).
• Pull: apps/temporal-worker/activities/webapp_triage.py call_webapp_triage(...), used by _triage_decide when email_triage_canonical_endpoint_v1 is on for the connection. It signs with the same INBOUND_EMAIL_SECRET (secrets map: /runtime/temporal-worker).
• Eval: packages/eval/scripts/verify-email-archive-flow.ts (HMAC) and the email-triage eval (eval token).

Mapping into the Temporal decision model

Temporal’s downstream activities consume process | rfq | skip. _map_canonical_decision in gemini.py is the single translation point:

Canonical decision	Temporal decision	Notes
process_attachments / process_body / process_both	process	Canonical decision kept in TriageResult.category.
skip + _looks_like_inbound_rfq(email)	rfq	RFQ is a routing concern, not a classification the LLM owns.
skip + confidence < _TRIAGE_MIN_SKIP_CONFIDENCE	process	Low-confidence skips go to human review.
skip otherwise	skip

engine_status is passed through from the endpoint (llm_ok, denylist_*, trusted_sender, …). Endpoint failures of any kind fall back to _heuristic_triage and tag llm_error_fallback — triage never blocks.

Flags (defaults unchanged)

Flag	Default	Effect
email_triage_canonical_endpoint_v1	off	Pull path calls /api/email/triage instead of its local Gemini/mirascope prompt.
email_triage_image_preview_v1	off	Triage shows the model the ambiguous images of a document-less email and the worker uploads only those judged order content.
EMAIL_TRIAGE_LLM_ENABLED (env)	off	Legacy pull-path local LLM prompt. Only consulted when the flag above is off.

Rollout intent: enable email_triage_canonical_endpoint_v1 per connection, then delete _llm_triage / _build_llm_triage_prompt and EMAIL_TRIAGE_LLM_ENABLED once every connection is on the endpoint.

Contract 2 — POST /api/webhook/inbound-email (intake)

apps/webapp/src/pages/api/webhook/inbound-email.ts owns alias resolution, email_events insertion, the work_items dual-write, attachment registration, and Temporal workflow kick-off. The push pipeline calls it directly.

The pull pipeline does not call this endpoint today; it persists email_events from Python (activities/database.py) and mirrors the work-item rule in _ensure_email_work_item (“one work_item per email_event, assign only for user-type aliases, idempotent on email_event_id”). Any change to the webhook’s persistence semantics must be mirrored there until the pull path is moved onto the webhook.

Body extraction (email_body_extraction_v1, default off)

Before this lane, an email with no uploadable attachment produced an email_events row and a new work item and nothing else: only uploadedDocumentIds ever started a workflow. When triage says process_body or process_both, the body has content, Temporal is configured and the org has email_body_extraction_v1 on, the webhook starts EmailBodyExtractionWorkflow on the email-processing queue, id email-body-extraction-{emailEventId}. A start failure is logged, never a 500. When the webhook started no attachment workflow for the email (decided from what it actually started, not from the triage label, so a process_both email whose attachments were dropped or failed to start counts), the workflow input carries owns_completion: true and the workflow writes the terminal state on every path: the webhook does not mark the row completed, because with no pdf_documents row the inbox would show needs_review for an order still being extracted. save_email_order marks it completed (with the no-items marker when nothing was found), a failure (including the first read) marks it failed, and set_email_event_order_stage writes order_stage to match. When an attachment workflow exists it owns the email: a body result with no lines is dropped and a failure is not stamped on the event. Flag off or a failed start: the webhook completes the row as before, so deploy the worker before enabling the flag.

The workflow reuses what the pull lane already has instead of adding an extractor: extract_email_body_order calls _extract_from_email_impl (the text path) and save_email_order links the result to the existing email_events row and work item through a shell pdf_documents row (file_type='email'), so the review screen opens it like any document order. Customer and item matching mirror EmailProcessingWorkflow._process_order. process_both runs it next to the PDF workflows: the body order is a second order on the same email; if the body yields no lines the event is left untouched (the attachment lane owns it).

• HTML and revisions. When the HTML body carries colour or strike styling it, not text_body, is what the model reads: email_body_markup.html_to_marked_text keeps line structure and tags runs as [red]…[/red], [green]…[/green], [strike]…[/strike], and the extraction prompt gets BODY_MARKUP_PROMPT_SUFFIX (the colour refers to the number, struck text is superseded, extract the CURRENT value). Plain emails are sent as before.
• Intent. The order-gate classifier (classify_order_document, fail-open) labels the body; the mapping to order | order_change | quote_request | supplier_document | other is stored in email_events.provider_metadata.bodyExtraction. email_events.intent is deliberately untouched: it is a stable three-value vocabulary that routes the inbox (RFQ pill, ux-state SQL). A non-purchase_order class flags the order for review through the existing ORDER_GATE_NOT_PURCHASE_ORDER downgrade; it never drops it. Follow-up: promote the intent to a first-class column and detect direction (a customer’s quote request vs a supplier’s own quote), which the classifier cannot tell apart.
• Where in the body a line came from. A body has no page coordinates, so each line item carries bounding_box = {kind: "text_span", start, end}: [start, end) offsets into email_events.text_body, in UTF-16 code units (what the review pane’s String.slice uses), widened to the whole source line. They are recomputed against the stored text_body after extraction, because the model may have read the marked HTML rendition. HTML-only emails store no spans (the pane would slice a different string); the reviewer sees the rendered email without a highlight. Follow-up if wanted: persist the rendition’s offsets and have the pane use it.

Contract 3 — POST /api/pdf-documents (attachment upload, push only)

apps/webapp/src/pages/api/pdf-documents/index.ts is where the push pipeline puts an attachment’s bytes, before it calls Contract 2 to link them. The route is runtime-public (apps/webapp/src/lib/api/auth-allowlist.ts) because the worker has no Clerk session, so the signature is the whole authentication. It keys off the same INBOUND_EMAIL_SECRET as the other two contracts.

• Auth: HMAC-SHA256 over JSON.stringify({ filename, connectionId, organizationId }) in that key order (X-Inbound-Email-Signature). Only the metadata is signed, not the file bytes. The signed organizationId must be the connection’s organization, so a payload signed for one tenant cannot write into another.
• Not behind a feature flag on purpose: isFeatureEnabled reads false when Flagsmith is unreachable, which would turn the check off silently. pdf-documents-require-external-upload-token was retired once the signature became the real check: it ran before the signature and nothing ever sent that bearer token, so enabling it would have returned 401 for every legitimate attachment.
• emailEventId and userId are never read from the request. A public caller could otherwise attach its document to another organization’s email and have the review page show that email, or stamp any Clerk id on the row and change who can see it. Contract 2 sets the real link and owns assignment.
• The body is refused with 413 at the upload size ceiling before it is parsed, and the stream is counted rather than trusted to declare a Content-Length. Auth cannot come first here, because the signature covers the filename and that only exists once the body is parsed.
• A filename longer than pdf_documents.filename (varchar 255) is truncated by code point, not rejected: an attachment with a 300-character name should still land, and a 400 would only tell the worker to drop it. The R2 key is truncated separately by truncateForR2Key.
• Failure contract: 401 missing or invalid signature, 403 the organizationId does not own the connection, 503 when INBOUND_EMAIL_SECRET is unset, 500 with a generic body for external callers (a failed query’s message carries its SQL parameters, so the cause goes to the log instead). A failed insert deletes the R2 object it already wrote, because the orphan-cleanup sweep skips external-orders keys and would otherwise strand it for good.

Callers

• Push: apps/email-worker/src/webapp-client.ts uploadAttachment(...), once per attachment. It already signed these three fields, so the check needed no worker change.
• Pull: does not use this route. The Temporal activity uploads to R2 itself (see “Still duplicated” item 4 below).

Still duplicated (follow-up, not forced here)

These are deliberately left in place because a cross-language extraction would be larger than the benefit right now:

1. Deterministic pre-filter. apps/email-worker/src/order-filter.ts (shouldFilterEmail, classifyEmailType, extractPoNumber) vs. _heuristic_triage / _looks_like_inbound_rfq / _apply_rfq_po_guard in gemini.py. The Python side is also the fallback for the canonical endpoint, so it cannot simply be removed. Candidate: move the pre-filter into /api/email/triage as a pre-LLM stage so both pipelines get it from one place.
2. Intake persistence. Pull path writes email_events / work_items in Python instead of calling /api/webhook/inbound-email (see Contract 2).
3. Legacy local LLM triage. _llm_triage in gemini.py stays until email_triage_canonical_endpoint_v1 is on everywhere.
4. Attachment upload. The email worker posts to Contract 3 before calling the webhook (which only links the resulting pdfDocuments); the pull path uploads from the Temporal activity. Same bucket layout, two writers, and only one of them goes through a signed HTTP contract.