Skip to main content

CI/CD Issues

Use this page when a deploy pipeline is blocked by infrastructure state problems rather than by application code.

These fixes are powerful. Read the cause and the plain-language explanation before you run the commands.

warning

Several commands on this page directly edit or replace infrastructure state. They are not "safe defaults."

14. Pulumi State Conflict After ArgoCD Migration​

Problem: pulumi up fails with pending operations errors for resources migrated to ArgoCD.

Cause: Resources were moved to ArgoCD management, but Pulumi still thinks an earlier infrastructure operation is in progress.

What this means in plain language: Pulumi's saved picture of the world still contains stale "I am working on this" markers, so new updates refuse to start.

Risk level: Disruptive. This edits Pulumi state, but it does not delete live infrastructure by itself.

Fix: Remove the stale pending-operation markers from Pulumi state:

pulumi stack export -s dev | python3 -c "
import json, sys
data = json.load(sys.stdin)
data['deployment']['pending_operations'] = []
json.dump(data, sys.stdout)
" | pulumi stack import -s dev

15. Pulumi Wants to Replace the Entire Cluster​

Problem: pulumi up plans to replace the DOKS cluster after out-of-band changes.

Cause: Manual changes through doctl or the DigitalOcean console caused Pulumi state to drift from reality.

What this means in plain language: the real cluster changed outside Pulumi, so Pulumi's saved picture is now too different from what actually exists.

Risk level: Starts as safe if you only run pulumi refresh. Becomes destructive if you delete and recreate the cluster.

Fix: Start by asking Pulumi to re-read the live infrastructure:

pulumi refresh

If the state is too far gone to reconcile cleanly, the fallback is destructive: delete the cluster, clear Pulumi state, and recreate it.

doctl kubernetes cluster delete crawbl-dev --force --dangerous
pulumi stack export -s dev | python3 -c "
  import json,sys; d=json.load(sys.stdin); d['deployment']['resources']=[r for r in d['deployment']['resources'] if r.get('type')=='pulumi:pulumi:Stack']; d['deployment']['pending_operations']=[]; json.dump(d,sys.stdout)
" | pulumi stack import -s dev
crawbl infra update --auto-approve

🔗 Terms On This Page

If a term below is unfamiliar, open its glossary entry. For the full list, go to Internal Glossary.

  • Pulumi: The infrastructure-as-code tool that creates the cluster and foundational platform resources.
  • ArgoCD: The GitOps deployment system that keeps the cluster aligned with what is committed in Git.
  • GitOps: An operations model where Git is the source of truth for live cluster state.
  • DOKS: DigitalOcean Kubernetes, the managed Kubernetes service used for the Crawbl cluster.