CI/CD Pipeline (Coolify era)

This document outlines our CI/CD pipeline for the ERP-Unlocked monorepo, which uses Docker Bake for efficient builds and GitHub Container Registry (GHCR) for image storage, with dual deployment targets: Coolify for Docker services and Cloudflare Pages for the marketing site.

Overview

Our CI/CD pipeline follows these principles:

1. Selective Builds: Only build and push Docker images for services that have changed
2. Validation vs Deployment: PRs build for validation (no push), merges build and deploy
3. Docker Bake: Consolidate all Docker build configurations in a single docker-bake.hcl file
4. GHCR Integration: Store all images in GitHub Container Registry with proper tagging
5. Dual Deployment Strategy:
   • Docker Services (webapp, pdf-api, pdf-worker) → Coolify via GHCR images
   • Marketing Site (static Astro) → Cloudflare Pages via separate deployment workflows
6. GitFlow-inspired Branching: Use main, staging, develop, feature/*, hotfix/*, and release/* branches

Deployment Architecture

Main CI/CD Pipeline (ci-cd-pipeline.yml)

• Purpose: Validation and Docker image builds
• Docker Services: Builds images and pushes to GHCR for Coolify deployment
• Marketing Site: Only builds for validation (no deployment)

Separate Deployment Workflows

• deploy-production.yml: Deploys Docker services to Coolify + marketing site to Cloudflare Pages
• deploy-staging.yml: Deploys Docker services to Coolify + marketing site to Cloudflare Pages

Branching Strategy

• main: Production-ready code, protected branch
• staging: Pre-production testing environment, protected branch
• develop: Integration branch for features and fixes
• feature/*: Feature branches for ongoing development
• hotfix/*: Emergency fixes for production issues
• release/*: Preparation for releases with version bumps

Build Process

1. Two-Stage Build Process

Our CI/CD pipeline separates validation from deployment:

On Pull Requests (Validation)

1. Build Only: Validates that Docker images can be built successfully
2. No Push: Images are not pushed to prevent unnecessary deployments
3. Fast Feedback: Developers get quick validation without triggering deployments

On Merges to Main/Staging (Deployment)

1. Build and Push: Builds Docker images and pushes to GHCR
2. Marketing Site Build: Builds static Astro marketing site
3. Registry Tagging: Images are tagged with branch name and commit SHA for traceability
4. Version Consistency: All deployed services use images from the same commit SHA

2. Selective Builds with Docker Bake

The pipeline uses Docker Bake to build only the services that have changed:

1. Change Detection: Detects changes in webapp, pdf-api, pdf-worker, and marketing directories
2. Conditional Builds: Only builds services with detected changes
3. Path Filters: Monitors specific directories and configuration files for each service

3. Docker Bake Configuration

The docker-bake.hcl file defines all build targets and their configurations. Note: The default values (BRANCH_NAME = "main", COMMIT_SHA = "latest") are fallbacks that get dynamically overridden by the workflow:

variable "REGISTRY" {
  default = "ghcr.io"
}

variable "IMAGE_PREFIX" {
  default = "ghcr.io/your-org/erp-unlocked"
}

variable "VERSION" {
  default = "latest"
}

variable "BRANCH_NAME" {
  default = "main"
}

variable "COMMIT_SHA" {
  default = "latest"
}

group "default" {
  targets = ["webapp", "pdf-api", "pdf-worker"]
}

target "webapp" {
  context = "."
  dockerfile = "apps/webapp/Dockerfile"
  tags = [
    "${IMAGE_PREFIX}/webapp:${replace(BRANCH_NAME, "/", "-")}-${COMMIT_SHA}",
    "${IMAGE_PREFIX}/webapp:latest"
  ]
  cache-from = ["type=gha"]
  cache-to = ["type=gha,mode=max"]
  platforms = ["linux/amd64"]
}

# Other targets defined similarly...

4. GitHub Actions Workflow

The .github/workflows/ci-cd-pipeline.yml file orchestrates the build process:

1. Detect Changes: Determines which services have changed using path filters
2. Validate Configuration: Checks that the Docker Bake file is valid
3. Dynamic Variable Override: Passes current branch and commit to Docker Bake:

   --set *.args.BRANCH_NAME=${GITHUB_REF_NAME}
   --set *.args.COMMIT_SHA=${GITHUB_SHA}

4. Conditional Execution:
   • PRs: build-validation job runs (builds without push)
   • Merges: build-and-push job runs (builds and pushes to registry)
5. Marketing Site: Builds Astro site only on merges to main/staging (no deployment)
6. Notify: Provides detailed summary showing validation vs deployment mode

Service Change Detection

The pipeline monitors these paths for changes:

webapp:
  - 'apps/webapp/**'
  - 'packages/**'
  - 'pnpm-lock.yaml'
  - 'docker-bake.hcl'

pdf-api:
  - 'apps/pdf-api/**'
  - 'packages/pdf-shared/**'
  - 'pyproject.toml'
  - 'uv.lock'
  - 'docker-bake.hcl'

pdf-worker:
  - 'apps/pdf-worker/**'
  - 'packages/pdf-shared/**'
  - 'pyproject.toml'
  - 'uv.lock'
  - 'docker-bake.hcl'

marketing:
  - 'pnpm-lock.yaml'

Local Testing with Act

You can test the GitHub Actions workflows locally using Act before pushing changes to the repository. This helps identify issues early and reduces the need for multiple commits to fix workflow problems.

1. Setup for Local Testing

1. Install Act: Follow the installation instructions for your platform
2. Create Configuration Files:
   • .actrc: Configuration for Act
   • .env.act: Environment variables and secrets

Example .actrc:

-P ubuntu-latest=ghcr.io/catthehacker/ubuntu:act-latest
-P ubuntu-22.04=ghcr.io/catthehacker/ubuntu:act-22.04
--secret-file .env.act
--env-file .env.act

Example .env.act:

GITHUB_TOKEN=your_github_token

2. Using the Test Script

We provide a script to make testing workflows easier:

# Test a specific job in a workflow
./scripts/test-workflows.sh ci-cd-pipeline.yml detect-changes

# Run the job (not just dry run)
./scripts/test-workflows.sh ci-cd-pipeline.yml detect-changes run

# Test validation workflow
./scripts/test-workflows.sh validate.yml validate

3. Common Testing Issues

1. Authentication Errors: Ensure you have a valid GitHub token in .env.act
2. Missing Dependencies: Act might not install all dependencies correctly
3. Docker Access: Make sure your user has permissions to run Docker commands
4. Resource Limitations: Some jobs might require more resources than available locally

4. What to Test Before Pushing

Before pushing changes to the repository, test at least:

1. detect-changes: Ensure path filters work correctly
2. validate: Check that validation steps pass
3. build-and-push: Verify Docker Bake configuration (with --dryrun)

Deployment Process

1. Coolify Setup for Automatic Image Pulls

A. Add GHCR Registry to Coolify

1. Go to Coolify dashboard → Settings → Docker Registries
2. Add:
   • Registry URL: ghcr.io
   • Username: Your GitHub username
   • Password: GitHub Personal Access Token with read:packages scope

B. Configure Each Service in Coolify

For each app (webapp, pdf-api, pdf-worker):

1. Type: Docker Image
2. Image:
   • ghcr.io/<your-org>/erp-unlocked/webapp:<branch>-<sha>
   • ghcr.io/<your-org>/erp-unlocked/pdf-api:<branch>-<sha>
   • ghcr.io/<your-org>/erp-unlocked/pdf-worker:<branch>-<sha>
3. Restart Policy: unless-stopped
4. Network: coolify (default)
5. Environment Variables:
   • Set as needed (e.g., NODE_ENV, REDIS_URL, etc.)
6. Public Access: Only for webapp

C. Service Configuration Details

Web Service

Name: erp-unlocked-web
Image: ghcr.io/<your-org>/erp-unlocked/webapp:main-latest
Public Access: Yes (port 4321)
Environment Variables:
  NODE_ENV=production
  PDF_PROCESSOR_URL=http://erp-unlocked-pdf-api:8000
  REDIS_URL=redis://redis:6379/0
  PORT=4321

PDF API Service

Name: erp-unlocked-pdf-api
Image: ghcr.io/<your-org>/erp-unlocked/pdf-api:main-latest
Public Access: No
Environment Variables:
  REDIS_URL=redis://redis:6379/0
  DATABASE_URL=<your-database-url>

PDF Worker Service

Name: erp-unlocked-pdf-worker
Image: ghcr.io/<your-org>/erp-unlocked/pdf-worker:main-latest
Public Access: No
Environment Variables:
  REDIS_URL=redis://redis:6379/0
  DATABASE_URL=<your-database-url>

2. Automatic Deployment Options

Option 1: Tag-Based Deployments

Configure each service in Coolify to track a specific tag (e.g., main-latest for production, staging-latest for staging). Coolify will automatically pull new images when they are pushed with these tags.

Option 2: Webhook Deployments

1. Create a webhook in Coolify for each service
2. Add the webhook URL to GitHub Secrets as COOLIFY_PRODUCTION_WEBHOOK and COOLIFY_STAGING_WEBHOOK
3. Create a GitHub Actions workflow that triggers these webhooks after successful builds

Image Tagging Strategy

Images are tagged using the following convention:

• <branch>-<commit-sha>: Specific version (e.g., main-abc1234)
• <branch>-latest: Latest build for a branch (e.g., main-latest)
• latest: Latest production build (from main branch)

Build Triggers and Behavior

Normal Operation

• PRs to main/staging/develop: Validation builds only (no deployment)
• Merges to main/staging: Full build and deployment
• Marketing Site: Only builds and deploys on merges, not PRs

Special Build Triggers

You can force a full build (all services) using:

1. Manual Trigger: Run the workflow manually with full_build=true
2. Commit Message: Include [full-build] in your commit message
3. Release Branch: Pushes to release/* branches always build all services

Marketing Site Deployment

The marketing site (Astro static site) has special handling across two workflow types:

Main CI/CD Pipeline (ci-cd-pipeline.yml)

• PRs: Validates marketing site builds but does not deploy
• Merges to main/staging: Builds for validation only (no deployment)
• Purpose: Ensures marketing site builds successfully before merge

Deployment Workflows (deploy-production.yml, deploy-staging.yml)

• Cloudflare Pages Deployment: Separate workflows handle actual deployment to Cloudflare Pages
• Triggered: Manual or automated deployment triggers (independent of main CI/CD pipeline)
• Environment: Production workflow deploys to production Cloudflare Pages, staging to staging environment

Troubleshooting

Common Issues

1. Image Not Found in Coolify:

   • Verify the image was successfully pushed to GHCR
   • Check GHCR authentication in Coolify
   • Ensure the image name and tag match exactly

2. Service Cannot Connect to Another Service:

   • Verify all services are on the same Coolify network
   • Check that service names in environment variables match Coolify service names
   • Ensure internal ports are correctly exposed

3. Build Failing:

   • Check the GitHub Actions logs for specific errors
   • Verify that docker-bake.hcl is correctly formatted
   • Ensure Dockerfiles exist at the specified paths

Maintenance Tasks

1. Rotate GitHub PAT: Every 90 days, generate a new GitHub Personal Access Token and update it in Coolify
2. Clean Up Old Images: Periodically remove unused images from GHCR to save space
3. Update Base Images: Regularly update base images in Dockerfiles for security patches