Cloudflare-Native Data Backend Alternative

May contain stale references

This document was audited on 2026-06-22 and contains references that need updating: Coolify/Kubernetes references predate GCP cutover. Cross-check critical details against the current codebase before relying on them.

Question: Can we run the entire data backend on Cloudflare?
Answer: Yes, but with a different architecture (serverless lakehouse vs traditional containers)

---

Architecture Comparison

Option A: Traditional (Kubernetes-based) ✅ Original Proposal

graph TB
    subgraph "Traditional Infrastructure"
        A[Cube API<br/>Node.js Container]
        B[Trino Coordinator<br/>JVM Container]
        C[Trino Workers<br/>JVM Containers]
        D[Hudi Lakehouse<br/>Parquet on S3/R2]
        E[Spark<br/>ETL Processing]
        F[PostgreSQL<br/>Metadata]
    end

    A -->|SQL| B
    B --> C
    C -->|read| D
    E -->|write| D

    style A fill:#FF9800,color:#fff
    style B fill:#FF9800,color:#fff
    style E fill:#FF9800,color:#fff

Deployment: Kubernetes (Coolify, AWS EKS, GKE)
Cost: $725/month
Complexity: Medium-High (containers, orchestration)

---

Option B: Cloudflare-Native (Serverless) 🆕 Alternative

graph TB
    subgraph "Cloudflare Edge Network"
        A[Workers API<br/>GraphQL/REST]
        B[DuckDB WASM<br/>Query Engine]
        C[D1 Database<br/>Metadata SQLite]
        D[R2 Storage<br/>Parquet Files]
        E[Workers + Queues<br/>ETL Pipeline]
        F[Analytics Engine<br/>Time-series]
    end

    A -->|query| B
    B -->|read| D
    E -->|write| D
    E -->|metadata| C

    style A fill:#F68D2E,color:#fff
    style B fill:#F68D2E,color:#fff
    style E fill:#F68D2E,color:#fff

Deployment: Cloudflare Edge (330+ locations worldwide)
Cost: ~$200-400/month (estimated)
Complexity: Medium (serverless, different paradigms)

---

Detailed Component Mapping

Component	Traditional (K8s)	Cloudflare-Native	Notes
Storage	Hudi on R2	Parquet on R2	✅ Same
Query Engine	Trino (JVM)	DuckDB WASM in Workers	⚠️ Different tech
Semantic Layer	Cube Core (Node.js)	Custom Workers API	⚠️ Build from scratch
Metadata DB	PostgreSQL	D1 (SQLite)	⚠️ Different limits
ETL	Trigger.dev + Python	Workers + Queues	⚠️ Different model
Orchestration	Trigger.dev	Cloudflare Cron Triggers	✅ Similar
Authentication	Clerk JWT	Clerk JWT	✅ Same
RBAC	Cube RLS + Middleware	Workers Middleware	⚠️ Manual implementation

---

Cloudflare Services Required

1. Cloudflare Workers (Compute)

Use Cases:

• GraphQL/REST API (replaces Cube)
• Query execution with DuckDB WASM (replaces Trino)
• ETL pipeline logic (replaces Spark)

Pricing:

Free Tier:
  - 100,000 requests/day
  - 10ms CPU time/request

Paid Plan ($5/month base):
  - $0.30 per million requests
  - $0.02 per million GB-seconds (memory)
  - 10GB/month bandwidth included
  - Additional bandwidth: $0.36/GB

Estimated Cost (100K requests/day):
  - Base: $5/month
  - Requests (3M/month): ~$1/month
  - Compute: ~$10-20/month
  Total: ~$20-30/month

Limits:

CPU Time:
  - Free: 10ms per invocation
  - Paid: 30 seconds per invocation (enough for most queries)

Memory: 128 MB per instance
Script Size: 10 MB (after compression)
Subrequests: 50 per request (Free), 1000 (Paid)
Duration: 30 seconds max

2. Cloudflare D1 (SQLite Database)

Use Cases:

• Metadata storage (ERP connections, sync logs)
• User preferences
• Query cache

Pricing:

Free Tier:
  - 5 GB storage
  - 5 million reads/day
  - 100,000 writes/day

Paid Plan (included in Workers Paid):
  - $0.75 per million reads
  - $4.50 per million writes
  - $0.75 per GB storage/month

Estimated Cost (moderate usage):
  - Storage (5 GB): $3.75/month
  - Reads (10M/month): $7.50/month
  - Writes (500K/month): $2.25/month
  Total: ~$13-15/month

Limits:

Database Size: 10 GB per database (500 MB Free)
Databases: 50,000 per account
Query Time: 30 seconds max
Transactions: ACID compliant (SQLite)
Consistency: Strong consistency within single location

3. Cloudflare R2 (Object Storage)

Use Cases:

• Parquet file storage (ERP data)
• Hudi metadata files (if using Hudi format)

Pricing:

Storage: $0.015 per GB/month
Egress: $0 (FREE - major advantage!)
Class A Operations: $4.50 per million (write, list)
Class B Operations: $0.36 per million (read)

Estimated Cost (2.7 TB data):
  - Storage: ~$40/month
  - Reads (100K/month): ~$0.04/month
  - Writes (10K/month): ~$0.05/month
  Total: ~$40/month

Limits:

Storage: Unlimited
Bandwidth: Unlimited (no egress fees!)
Object Size: 5 TB per object
Requests: Unlimited

4. Cloudflare Queues (Message Queue)

Use Cases:

• ETL job orchestration
• Async processing
• Rate limiting ERP API calls

Pricing:

Free Tier:
  - 1 million operations/month

Paid Plan:
  - $0.40 per million operations

Estimated Cost:
  - 5M operations/month: ~$2/month

5. Cloudflare Analytics Engine (Time-Series)

Use Cases:

• Pricing history tracking
• Inventory snapshots over time
• Usage metrics

Pricing:

Included in Workers Paid plan:
  - 10 million events/month included
  - $0.25 per million additional events

Estimated Cost:
  - Included (within 10M events)

---

Implementation: DuckDB WASM in Workers

Why DuckDB WASM?

DuckDB is a columnar analytical database that:

• ✅ Runs in WebAssembly (works in Cloudflare Workers)
• ✅ Queries Parquet files directly from R2
• ✅ Supports SQL (similar to Trino)
• ✅ Very fast for OLAP workloads
• ✅ Small footprint (~8 MB WASM binary)

Example Worker: Query Products

apps/cloudflare-data-backend/src/workers/query-api.ts

import { Database } from '@duckdb/duckdb-wasm';

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    // 1. Verify JWT (Clerk)
    const token = request.headers.get('Authorization')?.replace('Bearer ', '');
    const session = await verifyClerkToken(token);
    if (!session) {
      return new Response('Unauthorized', { status: 401 });
    }

    // 2. Parse GraphQL query
    const body = await request.json();
    const { query, variables } = body;
    // 3. Initialize DuckDB WASM
    const db = await Database.create({
      // Use R2 as data source
      remote: {
        fetch: url => env.R2_BUCKET.get(url),
      },
    });
    // 4. Register R2 Parquet files as tables
    await db.run(`
      CREATE VIEW erp_products AS
      SELECT * FROM read_parquet('r2://erp-data/erp_products/*.parquet')
      WHERE clerk_organization_id = '${session.org_id}'
    `);

    // 5. Execute query (auto-filtered by org_id)
    const result = await db.query(`
      SELECT product_id, name, description
      FROM erp_products
      WHERE name LIKE '%${variables.searchTerm}%'
      LIMIT 100
    `);

    // 6. Return results
    return new Response(JSON.stringify(result.toArray()), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

ETL Pipeline with Workers + Queues

apps/cloudflare-data-backend/src/workers/etl-prophet21-products.ts

import { Queue } from '@cloudflare/workers-types';

export default {
  async scheduled(event: ScheduledEvent, env: Env): Promise<void> {
    // Triggered daily via Cron Trigger
    // 1. Get all active ERP connections from D1
    const connections = await env.D1.prepare(
      `
      SELECT * FROM erp_connections WHERE active = 1
    `
    ).all();
    // 2. Queue ETL job for each connection
    for (const conn of connections.results) {
      await env.ETL_QUEUE.send({
        connectionId: conn.id,
        orgId: conn.clerk_organization_id,
        erpType: conn.type,
      });
    }
  },
};

// Consumer worker
export const etlConsumer = {
  async queue(batch: MessageBatch, env: Env): Promise<void> {
    for (const message of batch.messages) {
      const { connectionId, orgId, erpType } = message.body;
      // 1. Fetch products from Prophet21 OData API
      const products = await fetchProphet21Products(connectionId);

      // 2. Convert to Parquet
      const parquetBuffer = await convertToParquet(products, orgId);

      // 3. Write to R2
      const filename = `erp_products/org_id=${orgId}/sync_${Date.now()}.parquet`;
      await env.R2_BUCKET.put(filename, parquetBuffer);

      // 4. Update metadata in D1
      await env.D1.prepare(
        `
        INSERT INTO erp_sync_logs (connection_id, status, records_processed)
        VALUES (?, 'completed', ?)
      `
      )
        .bind(connectionId, products.length)
        .run();
      message.ack();
    }
  },
};

---

Cost Comparison

Traditional Infrastructure (Original Proposal)

Monthly Costs:
  PostgreSQL (downsized): $450
  Trino (t3.large): $150
  Cube API: $35
  R2 Storage (2.7 TB): $40
  Trigger.dev: $50

Total: $725/month
Annual: $8,700

Cloudflare-Native Alternative

Monthly Costs:
  Workers (compute): $30
  D1 (metadata): $15
  R2 Storage (2.7 TB): $40
  Queues (ETL orchestration): $2
  Analytics Engine: $0 (included)

Total: ~$87/month
Annual: ~$1,044

Savings: 88% cheaper than traditional!
94% cheaper than current PostgreSQL!

BUT: This doesn’t account for:

• Development cost to build custom query layer
• Lost features from Cube (pre-aggregations, caching)
• Learning curve for DuckDB WASM

---

Feature Comparison

Feature	Traditional	Cloudflare-Native	Winner
Cost	$725/month	$87/month	✅ Cloudflare
Latency	50-200ms (regional)	10-50ms (edge)	✅ Cloudflare
Scalability	Manual (K8s scaling)	Auto (infinite)	✅ Cloudflare
Global Distribution	Single region	330+ locations	✅ Cloudflare
Setup Complexity	Medium-High	Medium	🟰 Tie
Vendor Lock-in	Portable (K8s)	High (Cloudflare)	✅ Traditional
RBAC	Built-in (Cube)	Manual	✅ Traditional
Time-Travel	Hudi native	Custom	✅ Traditional
SQL Compatibility	Full (Trino)	Good (DuckDB)	✅ Traditional
Pre-Aggregations	Cube native	Manual	✅ Traditional
Cold Starts	None (always on)	Minimal (~5ms)	✅ Traditional
Max Query Time	Unlimited	30 seconds	✅ Traditional
Development Time	Lower (use Cube)	Higher (build API)	✅ Traditional

---

Trade-offs Analysis

Cloudflare Advantages ✅

Cost: ✓ 88% cheaper ($725 → $87/month)
  ✓ No infrastructure management
  ✓ Pay-per-use (scales to zero)

Performance: ✓ Edge compute (10-50ms latency globally)
  ✓ No cold starts for Workers
  ✓ R2 has zero egress fees

Scalability: ✓ Auto-scales to millions of requests
  ✓ Global distribution (330+ cities)
  ✓ No manual scaling needed

Simplicity: ✓ No Kubernetes management
  ✓ No container orchestration
  ✓ Built-in CDN

Cloudflare Disadvantages ❌

Development: ✗ Must build custom query API (no Cube)
  ✗ Manual RBAC implementation
  ✗ Custom pre-aggregation logic
  ✗ Learning curve (DuckDB WASM, Workers)

Limits: ✗ 30-second max query time (vs unlimited)
  ✗ 128 MB memory per Worker
  ✗ D1 max 10 GB per database
  ✗ No JVM (can't run Trino natively)

Features: ✗ No Cube pre-aggregations
  ✗ No Hudi time-travel (need custom solution)
  ✗ D1 eventual consistency (multi-region)
  ✗ Limited SQL compatibility vs Trino

Vendor Lock-in: ✗ Heavy Cloudflare dependency
  ✗ Hard to migrate away
  ✗ Proprietary APIs

---

Hybrid Approach: Best of Both Worlds

Recommendation: Start Traditional, Add Cloudflare Caching

graph TB
    subgraph "Edge (Cloudflare)"
        A[Workers API<br/>GraphQL Endpoint]
        B[Workers Cache<br/>Hot Data]
    end

    subgraph "Core (Kubernetes)"
        C[Cube API<br/>Semantic Layer]
        D[Trino<br/>Query Engine]
        E[Hudi Lakehouse<br/>Parquet on R2]
    end

    A -->|cache miss| C
    A -->|cache hit| B
    C --> D
    D --> E

    style A fill:#F68D2E,color:#fff
    style C fill:#FF9800,color:#fff

How It Works:

1. Deploy Cube + Trino on Kubernetes (traditional)
2. Add Cloudflare Workers as edge cache layer
3. Hot queries cached at edge (10ms response)
4. Cold queries hit Cube backend (200ms response)

Benefits:

• ✅ Best performance (edge caching)
• ✅ Keep Cube features (pre-aggregations, RBAC)
• ✅ Gradual migration path
• ✅ Lower Cloudflare costs (cache-only)

Cost:

Traditional Backend: $725/month
Cloudflare Workers (cache): $5-10/month
Total: $735/month (minimal increase)

Performance Improvement:
  - 90% of queries: 10-50ms (edge cache)
  - 10% of queries: 200ms (Cube backend)
  - Average: ~35ms (vs 200ms without cache)

---

Decision Matrix

Choose Traditional (K8s) If:

✅ You need full SQL compatibility (Trino)
✅ Complex queries > 30 seconds
✅ Want Cube’s built-in features (pre-agg, RBAC)
✅ Team knows Kubernetes/containers
✅ Avoid vendor lock-in
✅ Need Hudi time-travel natively

Estimated Timeline: 12-18 weeks
Development Effort: Medium (configure existing tools)

Choose Cloudflare-Native If:

✅ Cost is primary concern (88% cheaper)
✅ Global edge performance critical
✅ Queries < 30 seconds
✅ Simple aggregations (not complex joins)
✅ Team comfortable with serverless
✅ Already heavy Cloudflare users

Estimated Timeline: 16-24 weeks
Development Effort: High (build custom API)

Choose Hybrid If:

✅ Want best performance + features
✅ Willing to pay small premium
✅ Need gradual migration path
✅ Want to evaluate Cloudflare first

Estimated Timeline: 14-20 weeks
Development Effort: Medium-High

---

Recommendation

Phase 1: Start with Traditional (K8s) ✅

Rationale:

1. Faster time-to-value - Use Cube/Trino/Hudi (proven tools)
2. Lower risk - Well-documented, large community
3. Full features - RBAC, pre-aggregations, time-travel
4. Portability - Can migrate away if needed

Phase 2: Add Cloudflare Edge Cache (Optional)

After traditional backend is stable:

1. Add Workers edge layer for caching
2. Improve latency for global users
3. Reduce load on Cube backend
4. Small cost increase ($5-10/month)

Phase 3: Evaluate Full Migration (Future)

After 6-12 months, evaluate:

• Is cost still a concern? ($725/month too high?)
• Are queries simple enough? (< 30 seconds?)
• Is team comfortable with Workers?
• Are Cube features critical?

If answers favor Cloudflare, migrate incrementally.

---

POC: Cloudflare-Native Quick Test

Want to test Cloudflare approach without full commitment?

2-Week POC Plan

Week 1: DuckDB WASM Query POC
  - Deploy Worker with DuckDB WASM
  - Query sample Parquet files on R2
  - Measure performance

Week 2: ETL Pipeline POC
  - Build Prophet21 → Parquet Worker
  - Use Queues for orchestration
  - Compare with Trigger.dev approach

Decision Point: ✓ If POC successful → Consider Cloudflare-native
  ✗ If POC struggles → Stick with traditional

---

Summary Table

Aspect	Traditional	Cloudflare-Native	Hybrid
Monthly Cost	$725	$87	$735
Setup Time	12-18 weeks	16-24 weeks	14-20 weeks
Performance	Good (200ms)	Excellent (50ms)	Excellent (35ms)
Scalability	Manual	Auto	Auto
Features	Full	Limited	Full
Complexity	Medium	Medium-High	High
Vendor Lock-in	Low	High	Medium
Risk	Low	Medium	Medium

---

Final Recommendation

Start with Traditional (Kubernetes) approach for these reasons:

1. ✅ Proven architecture - Hudi/Trino/Cube are battle-tested
2. ✅ Full features - RBAC, time-travel, pre-aggregations
3. ✅ Faster implementation - Configure vs build from scratch
4. ✅ Lower risk - Well-documented, large community
5. ✅ Portability - Can migrate to different cloud later

Consider Cloudflare in future if:

• Cost becomes critical (need 88% savings)
• Global edge performance is essential
• Queries remain simple (< 30 seconds)
• Team gains serverless expertise

Hybrid approach is best of both worlds:

• Traditional backend for features
• Cloudflare edge cache for performance
• Minimal cost increase
• Gradual migration path

---

Next Steps:

1. Proceed with traditional architecture (as proposed)
2. Monitor costs after 6 months
3. Evaluate Cloudflare if costs exceed budget
4. Consider hybrid for performance optimization

---

Questions? Reach out to discuss Cloudflare alternatives in detail.