Prophet21 Pricing Refresh Runbook

This runbook covers running and monitoring the Prophet21 pricing refresh pipeline (bronze_product_pricing in apps/dagster/erp_pipeline/assets/bronze_pricing.py), which fetches customer/location pricing from Prophet21 and writes it to the R2 Iceberg bronze layer (bronze.product_pricing).

Overview

The bronze_product_pricing Dagster asset runs automatically every 2 hours as part of the bronze_sync schedule (partitioned per connection_id for Prophet21 connections). It extracts pricing records into the shared multi-tenant bronze.product_pricing Iceberg table using resolve_iceberg_write_disposition (merge disposition) and feeds downstream dbt Silver/Gold pricing models.

Prerequisites

• Active Prophet21 connection in erp_connections
• Access to the Dagster UI (dagster.ordermatic.co / dagster-staging.ordermatic.co, or via IAP tunnel)

Per-Connection Location Configuration

You can configure a specific location ID for pricing per connection by updating the connection’s metadata:

UPDATE erp_connections
SET metadata = jsonb_set(
  COALESCE(metadata, '{}'::jsonb),
  '{pricingLocationId}',
  '123'::jsonb
)
WHERE id = 'your-connection-id';

If no location is configured, the pipeline automatically selects:

1. Location with DefaultLine = 'Y' (if available)
2. Location with ReplenishmentLocation = 'Y' (if available)
3. Location with the lowest LocationId (fallback)

Running the Job

No manual trigger is needed under normal operation — bronze_product_pricing runs automatically every 2 hours as part of the bronze_sync schedule, one partition per connection.

To force an out-of-cycle run:

• Dagster UI (preferred, targeted): Navigate to the bronze_product_pricing asset, select the connection’s partition, and materialize it directly.
• Webapp /api/erp/force-sync (requires admin access): Calling /api/erp/force-sync with entityType: 'all', forceSync: true triggers full_sync_pipeline (all bronze assets + dbt + Typesense) for the connection. Prefer materializing bronze_product_pricing directly in the Dagster UI unless you want a full connection resync.

Monitoring and Troubleshooting

Job Progress

Check the bronze_product_pricing asset’s materialization history and structured run logs in the Dagster UI, filtered to the connection’s partition (connection_id).

Common Issues

• Rate Limiting: The Prophet21 client includes automatic retry with exponential backoff. If a connection consistently hits rate limits, check concurrent runs for the same partition in Dagster.
• Missing Pricing Data: Verify that pricingLocationId in erp_connections.metadata is valid for the tenant and that the item exists in the Prophet21 API.
• Connection Failures: Verify the connection is active in erp_connections and credentials decrypt cleanly with ENCRYPTION_KEY.