Bronze-to-DBT Sensor Partition Error - Root Cause & Fix

Date: December 7, 2025
Status: 🔴 Root Cause Identified

Root Cause

The error CheckError: Failure condition: Job has no PartitionsDefinition occurs because:

1. Job finds 0 assets - dbt_transform_job uses AssetSelection.tag("dbt_project", "erp_pipeline") but this tag doesn’t match the dbt assets
2. No partitions to infer - Since the job finds 0 assets, Dagster can’t infer partitions from them
3. RunRequest rejected - When sensor creates RunRequest with partition_key, Dagster checks if job has partitions, but it doesn’t (because no assets were found)

The Problem

Current Job Definition:

dbt_transform_job = define_asset_job(
    name="dbt_transform",
    selection=AssetSelection.tag("dbt_project", "erp_pipeline"),  # ← This finds 0 assets!
    description="Run dbt transformations (bronze → silver → gold)",
    tags={"kind": "transform"},
)

Evidence:

• Test shows: Selected assets: 0
• dbt assets don’t have tag "dbt_project": "erp_pipeline"
• dbt assets have asset keys like ["gold", "dim_products"], ["silver", "stg_products"], etc.

Modern Dagster Approach (No partitions_def on Jobs)

According to latest Dagster docs:

• ✅ Assets should have partitions_def (dbt assets already do: partitions_def=erp_connection_partitions)
• ✅ Jobs should infer partitions from selected assets (no explicit partitions_def needed)
• ✅ RunRequests can still use partition_key when job has inferred partitions

The issue: Job can’t infer partitions because it’s not finding any assets to infer from!

Solution

Change the job selection to match the actual dbt assets. The dbt assets have keys like:

• ["gold", "dim_products"]
• ["gold", "dim_customers"]
• ["silver", "stg_products"]
• etc.

Option 1: Use Asset Groups (if Dagster auto-creates groups from asset keys)

dbt_transform_job = define_asset_job(
    name="dbt_transform",
    selection=AssetSelection.groups("silver", "gold"),  # Select by group
    description="Run dbt transformations (bronze → silver → gold)",
    tags={"kind": "transform"},
)

Option 2: Use Specific Asset Keys

dbt_transform_job = define_asset_job(
    name="dbt_transform",
    selection=(
        AssetSelection.assets(AssetKey(["gold", "dim_products"]))
        | AssetSelection.assets(AssetKey(["gold", "dim_customers"]))
        | AssetSelection.assets(AssetKey(["gold", "dim_shipping_addresses"]))
        | AssetSelection.assets(AssetKey(["gold", "xref_products"]))
        | AssetSelection.assets(AssetKey(["silver", "stg_products"]))
        | AssetSelection.assets(AssetKey(["silver", "stg_customers"]))
        | AssetSelection.assets(AssetKey(["silver", "stg_cross_references"]))
        | AssetSelection.assets(AssetKey(["silver", "stg_shipping_addresses"]))
        | AssetSelection.assets(AssetKey(["silver", "stg_ship_to_links"]))
    ),
    description="Run dbt transformations (bronze → silver → gold)",
    tags={"kind": "transform"},
)

Option 3: Use Asset Key Pattern (if supported)

# Select all assets that start with "gold/" or "silver/"
dbt_transform_job = define_asset_job(
    name="dbt_transform",
    selection=AssetSelection.keys("gold", "silver").downstream(),  # If this API exists
    ...
)

Recommended Fix

Use Option 1 (groups) if Dagster automatically creates groups from asset key prefixes. Otherwise, use Option 2 with explicit asset keys.

File to modify: apps/dagster/erp_pipeline/jobs.py

Change:

# OLD (doesn't work - finds 0 assets)
dbt_transform_job = define_asset_job(
    name="dbt_transform",
    selection=AssetSelection.tag("dbt_project", "erp_pipeline"),
    ...
)

# NEW (selects actual dbt assets)
dbt_transform_job = define_asset_job(
    name="dbt_transform",
    selection=AssetSelection.groups("silver", "gold"),  # Or use explicit asset keys
    ...
)

Why This Works

1. Job finds assets - Selection matches actual dbt asset keys
2. Partitions inferred - Dagster sees all selected assets have partitions_def=erp_connection_partitions
3. RunRequest accepted - Job now has inferred partitions, so partition_key in RunRequest works
4. Sensor succeeds - Can trigger partitioned dbt runs

Additional Fix Needed

The typesense_sync_job will likely have the same issue when dbt_to_typesense_sensor tries to trigger it. Check if it needs similar fix.

Testing

After fix:

1. Verify job shows assets: Selected assets: > 0
2. Verify job has partitions: Has partitions: True (inferred)
3. Sensor should succeed: No more “Job has no PartitionsDefinition” errors
4. dbt runs should trigger with correct partition keys

---

Report Generated: 2025-12-07 12:45 UTC
Investigator: AI Assistant
Status: Ready for Fix Implementation