CI/CD Pipeline

Before You Change Anything

These steps can affect shared dev workloads or the GitOps control layer. Confirm the target repo, environment, and intended owner before mutating anything.

Builds and deployments happen locally via crawbl app deploy. CI is a validation gate only — it does not build or push images.

Pipeline Overview

Click diagram to zoom

What Triggers What

Trigger	Action	Workflow
Push to main in crawbl-backend	E2E tests against live cluster + release tagging	deploy-dev.yml
crawbl app deploy zeroclaw (local)	ZeroClaw image build + deploy + release	local CLI

CI Jobs (deploy-dev.yml)

CI runs two parallel jobs on every push to main:

e2e-test

Builds the crawbl CLI binary, configures kubectl against the dev cluster, and runs the full E2E suite against https://dev.api.crawbl.com. This validates that the code already deployed to the cluster is working correctly.

release

Calculates the next semver tag from conventional commit messages (feat: → minor, breaking !: → major, everything else → patch). If the tag already exists on remote, patch is bumped until a free tag is found. Creates a Git tag and publishes a GitHub release with notes enriched by the local claude CLI (sonnet model) from commit messages, plus a full changelog link (compare/prev...new).

CI Secrets

Secret	Purpose
DIGITALOCEAN_ACCESS_TOKEN	DOCR login, kubectl cluster access
CRAWBL_E2E_TOKEN	E2E auth bypass (must match orchestrator Helm values)

Local Deploy Workflow

Deployments are driven locally using crawbl app deploy. Each call does these steps atomically:

1

Step 1

Verify working tree

Checks that the working tree is clean and all commits are pushed. Skipped for docs, website, and zeroclaw.

2

Step 2

Build the Docker image

The component image is built locally (Go binary compiled on your machine for platform, Docker build for others).

3

Step 3

Push to DOCR

The image is pushed to registry.digitalocean.com/crawbl/.

4

Step 4

Update crawbl-argocd-apps

The image tag is patched in the crawbl-argocd-apps repo and committed.

5

Step 5

Create Git tag and GitHub release

Creates a Git tag (auto-calculated; bumps patch if tag exists on remote). Publishes a GitHub release with notes enriched by the local claude CLI (sonnet model) and a full changelog link.

6

Step 6

Push and let ArgoCD sync

The apps repo commit is pushed. ArgoCD detects the Git change, renders Helm charts, and rolls the new image into the cluster.

Deploy commands

The tag is auto-calculated from conventional commits — --tag is optional. The working tree must be clean and fully pushed before deploying (skipped for docs, website, zeroclaw).

# Deploy a single component (tag auto-calculated via semver)
crawbl app deploy platform
crawbl app deploy auth-filter
crawbl app deploy docs
crawbl app deploy website
crawbl app deploy zeroclaw

# Deploy platform + auth-filter only
crawbl app deploy all

# Override with an explicit tag
crawbl app deploy platform --tag v1.2.3

Semver logic: scans commits since the last v* tag — feat: → minor bump, !: (breaking) → major bump, everything else → patch bump.

Makefile shortcuts

The Makefile uses auto-semver — no manual tag needed:

make deploy-dev        # deploy platform + auth-filter
make deploy-platform
make deploy-docs
make deploy-website
make deploy-zeroclaw

Prerequisites

Log in to the DigitalOcean Container Registry before deploying:

doctl registry login

The token expires periodically — re-run if you get authentication errors.

ZeroClaw Image Builds

ZeroClaw is deployed locally via crawbl app deploy zeroclaw from crawbl-backend. CI in crawbl-zeroclaw is disabled.

Tags follow the fork convention v<upstream>-crawbl.<N> (e.g. v0.6.5-crawbl.3). The N suffix auto-increments — no manual tag needed.

cd crawbl-backend
crawbl app deploy zeroclaw

Each call: builds the image → pushes to DOCR → updates crawbl-argocd-apps → creates the Git tag → publishes a GitHub release with Claude-enriched notes. ArgoCD detects the apps repo change and rolls the new runtime into the cluster.

Reusable Workflows

The following workflow files are kept for reference and for use with deploy-prod.yml, but are no longer called by deploy-dev.yml:

• reusable-build.yml
• reusable-deploy.yml
• reusable-update-argocd.yml
• reusable-rollback-argocd.yml
• reusable-infra-drift-check.yml
• reusable-e2e-test.yml

---

What's next: Local Deploy