Prophet21 Pricing Refresh Runbook

This job no longer runs on Trigger.dev

The Trigger.dev job this runbook describes (p21-price-refresh / p21-price-refresh-v2 — audited 2026-06-22, task ID was already inconsistent then) is retired along with the rest of Trigger.dev. It’s replaced by the Dagster bronze_product_pricing asset (see apps/dagster/erp_pipeline/assets/bronze_pricing.py), which runs automatically as part of the scheduled bronze_sync job/schedule (every 2 hours, per-connection partition), Prophet21-only. The Prerequisites, Running the Job, Monitoring and Troubleshooting, and Support sections below have been updated for that; the Configuration, Best Practices, Data Validation, Rollback, and Maintenance sections still describe the old Trigger.dev job’s env vars and Postgres erp_products storage target and have NOT been re-verified — cross-check against the current codebase before relying on them.

This document provides instructions for running the Prophet21 pricing refresh job to update product pricing using the Transaction API instead of the potentially stale inv_mast view data.

Overview

The P21 pricing refresh job fetches accurate pricing data from Prophet21’s Transaction API and updates the erpProducts.pricing field for all products across all active Prophet21 connections. This addresses issues where the standard product sync may use outdated pricing information from the inv_mast view.

Note: The job uses the correct Prophet21 API endpoints:

• Locations: Multiple fallback endpoints are tried, with default location ID 1 if all fail
• Pricing: /api/inventory/v2/parts/{itemId} (Inventory API)

Prerequisites

• Active Prophet21 connections configured in the system
• Access to the Dagster UI (or IAP-tunneled access on staging/prod — see phase2-prod-dagster-restore)
• Environment variables configured (optional)

Configuration

Environment Variables

The following environment variables can be used to configure the job behavior:

# Optional: Maximum concurrent API requests per connection (default: 5)
P21_PRICE_REFRESH_CONCURRENCY=5

# Optional: Number of products to process in each batch (default: 200)
P21_PRICE_REFRESH_BATCH_SIZE=200

# Optional: Enable dry-run mode (default: false)
P21_PRICE_REFRESH_DRY_RUN=false

# Optional: Prevent normal product sync from overwriting refreshed pricing (default: false)
P21_SYNC_PRODUCTS_UPDATE_PRICING=false

Per-Connection Location Configuration

You can configure a specific location ID for pricing per connection by updating the connection’s metadata:

UPDATE erp_connections
SET metadata = jsonb_set(
  COALESCE(metadata, '{}'::jsonb),
  '{pricingLocationId}',
  '123'::jsonb
)
WHERE id = 'your-connection-id';

If no location is configured, the job will automatically select:

1. Location with DefaultLine = 'Y' (if available)
2. Location with ReplenishmentLocation = 'Y' (if available)
3. Location with the lowest LocationId (fallback)

Running the Job

No manual trigger needed under normal operation — bronze_product_pricing runs automatically every 2 hours as part of the bronze_sync schedule, one partition per connection.

To force an out-of-cycle run:

• Dagster UI (targeted, preferred): navigate to the bronze_product_pricing asset, select the connection’s partition, and materialize it directly.
• Webapp /api/erp/force-sync (requires admin access — handleForceERPSyncRequest calls requireAdmin): this endpoint has no pricing-specific entity type — entityType: 'products' only triggers the narrower products_sync job, not bronze_product_pricing. The only way to reach pricing through it is entityType: 'all', forceSync: true, which triggers the heavy full_sync_pipeline (all bronze assets + dbt + Typesense) for the connection, not a pricing-only refresh. Prefer the Dagster UI route above unless you specifically want a full resync.

Monitoring and Troubleshooting

Job Progress

Check the bronze_product_pricing asset’s materialization history in the Dagster UI, filtered to the connection’s partition — this replaces both the old Trigger.dev dashboard and the dropped erp_sync_logs table (see the caution banner above).

Common Issues

Rate Limiting

If you encounter rate limiting errors:

• Reduce the concurrency parameter
• The job includes automatic retry with exponential backoff

Missing Pricing Data

If some products don’t get pricing updates:

• Check that the location ID is valid for the connection
• Verify the item exists in the Transaction API
• Review job logs for specific error messages

Connection Failures

If a connection fails:

• Verify the connection is active and credentials are valid
• Check that the Transaction API endpoints are accessible
• Review connection-specific error logs

Log Analysis

Key log messages to monitor:

# Successful pricing fetch
"Successfully fetched item pricing" - Individual product pricing retrieved

# Bulk processing progress
"Completed bulk pricing fetch" - Batch of products processed

# Connection completion
"Completed pricing refresh for connection" - Full connection processed

# Job completion
"P21 price refresh job completed" - Entire job finished

Best Practices

Staging Testing

Always test on staging first:

1. Set dryRun: true to see what would be updated
2. Run on a single connection with connectionIds: ["staging-connection-id"]
3. Verify pricing data accuracy by comparing before/after values
4. Check that the correct location is being used

Production Deployment

1. Start with a small batch size (100-200) and low concurrency (3-5)
2. Monitor the first few connections for errors
3. Gradually increase batch size and concurrency if stable
4. Consider running during off-peak hours to minimize API impact

Data Validation

After running the job, validate the results:

-- Check pricing update statistics
SELECT
  connection_id,
  COUNT(*) as total_products,
  COUNT(CASE WHEN pricing IS NOT NULL THEN 1 END) as products_with_pricing,
  COUNT(CASE WHEN pricing->>'price1' IS NOT NULL THEN 1 END) as products_with_price1
FROM erp_products
WHERE connection_id IN (SELECT id FROM erp_connections WHERE type = 'prophet21')
GROUP BY connection_id;

-- Sample pricing data
SELECT
  erp_product_id,
  name,
  pricing,
  last_synced_at
FROM erp_products
WHERE connection_id = 'your-connection-id'
  AND pricing IS NOT NULL
LIMIT 10;

Rollback

If pricing data needs to be reverted:

1. The job is idempotent - you can re-run it
2. To restore previous pricing, you would need to restore from a database backup
3. Consider setting P21_SYNC_PRODUCTS_UPDATE_PRICING=true temporarily to allow normal sync to overwrite

Maintenance

Regular Refresh Schedule

Consider setting up a regular schedule for pricing refresh:

• Weekly or bi-weekly refresh for active connections
• Monthly refresh for all connections
• Monitor pricing accuracy and adjust frequency as needed

Performance Optimization

• Monitor job duration and adjust batch size/concurrency
• Consider running during off-peak hours
• Use connection-specific location configuration to avoid API calls for location detection

Support

For issues or questions:

1. Review the bronze_product_pricing asset’s Dagster run logs for the affected connection’s partition
2. Contact the development team with specific error messages and connection details