Project 2: Connector Abstraction Audit
What I Found
Section titled “What I Found”The connector architecture has two abstraction layers that don’t align: DltBaseConnector (the ABC) and the actual production orchestrators. Only NetSuiteDltConnector extends the base class, and the real NetSuiteSyncOrchestrator doesn’t even use it. The four production orchestrators (P21V2, Acumatica, WhereFour, Demo) are standalone classes with no shared base, each re-implementing pipeline creation, record counting, and error handling independently.
Where the Abstraction Works
Section titled “Where the Abstraction Works”The factory pattern is solid. ERPSourceFactory.create_orchestrator() cleanly routes by erp_type, handles credential decryption per-ERP, and supports lazy imports to avoid circular dependencies. Adding SAP B1 required ~15 lines in the factory — the pattern scales.
The ERPSourceProtocol defines the right interface shape. The problem isn’t the contract — it’s that nobody implements it.
Where It Breaks
Section titled “Where It Breaks”_create_pipeline() is duplicated 5 times with slight variations (env var names, local fallback behavior, validation). WhereFour already extracted this to create_r2_iceberg_pipeline() — the others should follow.
Auth model mismatch: get_auth_token() -> str assumes Bearer tokens. Acumatica uses session cookies (must logout to release license slots), SAP B1 uses session cookies with B1SESSION. The base class can’t represent these semantics.
No lifecycle hooks: run_sync() doesn’t call close() on completion or failure. Acumatica and SAP B1 both require explicit session cleanup to avoid server-side resource exhaustion.
run_sync() is too simple for production: Real orchestrators need per-entity column hints, transformers, image sync, pricing sync, and ERP-specific error handling. That’s why every connector rolled its own sync_products(), sync_customers(), etc.
Recommendation
Section titled “Recommendation”Rather than forcing all connectors into DltBaseConnector, extract the genuinely shared logic into composable utilities: a pipeline factory (WhereFour’s create_r2_iceberg_pipeline is the model), a shared get_records_processed(), and a base result dict builder. Keep orchestrators as standalone classes with a lightweight Protocol for type safety. The factory already handles routing — it doesn’t need inheritance to work.