Extraction Latency – PR Review Findings (Feb 2026)

This doc summarizes review of recent PRs into main to identify causes of the ~60% increase in extraction latency (early vs late half of Jan 22 – Feb 4, 2026).

Primary cause: blocking Langfuse flush (fixed)

Location: apps/pdf-worker/tasks.py – process_document_task finally block.

What was happening: Every document task ended with:

if is_langfuse_enabled() and langfuse_client:
    langfuse_client.flush()

So after every extraction (and after span.end()), the worker blocked until Langfuse had sent all queued events. That meant:

• A synchronous network round-trip to Langfuse’s API on every task.
• Variable and sometimes large delay depending on Langfuse/network (and there is a known issue where flush() can block indefinitely in some environments).

Why it hurt more over time: The flush has been in the codebase for a while; the increase in latency likely came from:

• More data per trace (e.g. after Mirascope/Langfuse modernization: more spans/generations per request), so each flush sends more and takes longer.
• Langfuse/network variability or load making each flush slower.

Fix applied: Removed the per-task flush(). The worker is long-lived; Langfuse is configured with flush_at=20 and flush_interval=10 and sends in the background. Relying on that removes the blocking call and should remove most of the added latency. Traces may appear in Langfuse within a few seconds instead of immediately.

---

Other changes reviewed (unlikely main causes)

Change	PR/commit	Verdict
DB: fresh connection per use	celery_app.get_db_connection()	No per-request pool; each use opens/closes. Neon pooler handles pooling. Unlikely to explain a large, steady latency increase.
Atomic order save (ON CONFLICT + extra SELECT)	42f9cbe8	Only adds an extra SELECT when a conflict occurs (duplicate document/position). Normal path unchanged. Minor possible impact only in high-conflict cases.
Celery concurrency 2 → 4	f2e1dea0	More workers; no change to per-task DB or Langfuse behavior. Could increase contention on shared resources but not the main latency driver.
Gemini client timeout	f2e1dea0	180s timeout added; prevents indefinite hang, does not add delay.
task_acks_late + visibility_timeout	fa5376cb	Affects reliability/redelivery, not LLM or observability path latency.
Langfuse/Mirascope modernization	88d96d96	More/better tracing; could increase payload size and thus flush cost. The per-task blocking flush is what turned that into user-visible latency.

---

Recommendations

1. Deploy the change that removes the per-task langfuse_client.flush() and monitor extraction latency and trace completeness in Langfuse.
2. Optional: If you need traces to be visible before task ack in rare cases, consider a flush only on worker shutdown (e.g. Celery worker signals), not after every task.
3. Optional: Add a time-bounded flush (e.g. flush with a short timeout in a thread) only if you later see trace loss and can afford a small, capped delay.

---

Frontend / webapp latency

Why the UI can feel slow even when extraction is fast

1. Polling – The UI polls GET /api/pdf-documents/[id] every 3 seconds while a document is processing. Perceived latency is “time until next poll sees status = completed”, so users can wait 0–3s after the backend finishes.
2. API route latency – Each poll hits the webapp API, which does DB lookups (and for completed docs with includeOrders, extra queries for orders). Slow DB or cold connections add to each request.
3. Extraction time – Still the dominant delay: until the worker marks the document completed, the UI keeps polling and shows a loading state.

Tracking webapp API latency

The webapp records request duration and status for key API routes via OpenTelemetry metrics (same pipeline as extraction observability):

• Metrics: http_request_duration_seconds, http_requests_total with labels method, route, status_code.
• Route instrumented: GET /api/pdf-documents/[id] (route label: GET /api/pdf-documents/[id] to avoid high cardinality).

You can:

• In SigNoz / Better Stack / your OTLP backend: build dashboards for http_request_duration_seconds filtered by route="GET /api/pdf-documents/[id]" to see p50/p95/p99 and compare with extraction latency.
• Add the same timing + recordHttpRequest() pattern to other heavy API routes (e.g. orders list, submit) for consistent visibility.

---

How to re-check latency

After deploying:

• Re-export observations from Langfuse (same window or a new one).
• Run:
  uv run python scripts/analyze_langfuse_observations.py /path/to/lf-observations-export.csv
• Compare mean latency and “first half vs second half” trend to the previous run.