Dev Services & Access

Before You Change Anything

These pages often point at shared systems. Confirm the cluster, namespace, and ownership boundary before running mutating commands.

This page lists the shared development services, what each one is for, and how you reach them.

In plain language, this is the answer key for "what shared dev services exist, how do I access them, and where do their credentials come from?"

How To Read This Page

Not every entry here is the same kind of thing:

• some URLs are product-facing services you call from code
• some are browser tools used for inspection or operations
• some sections explain where credentials and secrets come from behind the scenes

If you are new, start with the service table, then read only the section for the tool you need.

Services

URL	Service	Auth	Purpose
dev.api.crawbl.com	Orchestrator API	Firebase token or E2E token	Main backend API for the mobile app, agents, and tests
dev.postgres.crawbl.com	pgweb	Basic Auth	Browser UI for checking database tables and running queries
dev.docs.crawbl.com	Documentation	Basic Auth	This internal docs site
dev.argocd.crawbl.com	ArgoCD	Username/password	Deployment dashboard that shows whether the cluster matches Git
dev.metrics.crawbl.com	VictoriaMetrics	None (internal)	Metrics storage and Prometheus-compatible query UI
dev.logs.crawbl.com	VictoriaLogs	None (internal)	Log storage and query UI — all container logs land here

How Traffic Reaches These Services

All shared dev services ultimately run inside the crawbl-dev Kubernetes cluster.

Public URLs first hit the gateway, then TLS is handled, and finally traffic is routed to the right internal service.

Implementation details:

• DOKS is the DigitalOcean Kubernetes cluster that hosts the dev environment
• Envoy Gateway is the public entry point
• cert-manager keeps TLS certificates valid
• external-dns updates Cloudflare DNS records automatically

Orchestrator API

URL: https://dev.api.crawbl.com

This is the main backend API.

It is used by the mobile app, by agents through MCP-backed platform calls, and by E2E tests.

If you are unsure where to start, this is usually the first service to check because most platform work eventually flows through it.

Authentication

The API accepts three auth paths, depending on who is calling it.

If the phrase Firebase JWT is unfamiliar, read it as "the signed login token produced after a user authenticates with Firebase."

Path	Header	Used by
Firebase JWT	X-Token: <firebase-jwt>	Mobile app
Firebase JWT	Authorization: Bearer <firebase-jwt>	Dev tooling (Postman, curl)
E2E bypass	X-E2E-Token: <token>	CI pipeline only

Public endpoints (no auth required):

• GET /v1/health — liveness probe
• GET /v1/legal — terms and privacy policy

Quick test

# Health check (no auth)
curl https://dev.api.crawbl.com/v1/health

# With E2E token (from .env)
source crawbl-backend/.env
curl -H "X-E2E-Token: $CRAWBL_E2E_TOKEN" https://dev.api.crawbl.com/v1/health

Where The API Credentials Come From

Credential	Location
Firebase JWT	Generated by Firebase SDK in the mobile app
E2E token	crawbl-backend/.env → CRAWBL_E2E_TOKEN
E2E token (cluster)	AWS SM → crawbl/dev/backend/orchestrator → CRAWBL_E2E_TOKEN

---

pgweb (Database UI)

URL: https://dev.postgres.crawbl.com

pgweb is the quickest way to inspect the dev PostgreSQL database from a browser.

It supports table browsing, schema inspection, and ad hoc SQL queries.

Use this when you need to inspect data quickly without opening psql.

Credentials

Field	Value
Username	admin
Password	crawbl2026

Your browser will show a Basic Auth prompt when you visit the URL.

Connected database

Parameter	Value
Host	backend-postgresql (cluster-internal)
Port	5432
User	crawbl
Database	crawbl
Schemas	orchestrator, public

Updating the password

The Basic Auth file is stored in AWS Secrets Manager.

The gateway expects the password in {SHA} format:

cd crawbl-backend && source .env

# Generate new htpasswd
SHA_HASH=$(printf "new-password" | openssl dgst -sha1 -binary | openssl base64)
HTPASSWD="admin:{SHA}${SHA_HASH}"

# Update AWS SM
aws secretsmanager update-secret \
  --secret-id crawbl/dev/pgweb/auth \
  --secret-string "{\".htpasswd\":\"${HTPASSWD}\"}" \
  --region eu-central-1

# Force ExternalSecret re-sync (or wait ~1 min)
kubectl annotate externalsecret pgweb-secrets -n backend \
  force-sync=$(date +%s) --overwrite

---

Documentation Site

URL: https://dev.docs.crawbl.com

This is the docs site you are reading now. It is hosted on Cloudflare Pages and deployed locally via crawbl app deploy docs.

Access Control

Internal docs at /internal are protected by Cloudflare Access (email OTP). Allowed identities:

• Any @crawbl.com email
• Specific team member emails configured in the Cloudflare Access policy

Public docs at / are accessible to everyone without authentication.

See the Cloudflare Pages guide for full details.

---

ArgoCD

URL: https://dev.argocd.crawbl.com

ArgoCD is the deployment dashboard for the dev cluster.

It shows whether the live cluster matches what is committed in the deployment repo.

If you are new to ArgoCD, the useful mental model is: it is the system that makes the cluster match what is committed in the deployment repo.

Credentials

Field	Value
Username	admin
Password	Retrieve with the command below

# Get the admin password
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath='{.data.password}' | base64 -d && echo

Alternative: kubectl port-forward

If you want local-only access instead of going through the public gateway:

kubectl port-forward -n argocd svc/argocd-server 8080:80
# Open http://localhost:8080 — same username/password

---

VictoriaMetrics

URL: https://dev.metrics.crawbl.com

VictoriaMetrics is the metrics storage backend for the dev cluster. It exposes a Prometheus-compatible query API and a built-in UI for ad hoc metric queries.

Use this when you need to inspect cluster or application metrics — resource usage, request rates, error rates.

Credentials

No authentication is required for the dev environment. The URL is accessible directly.

Quick queries

# Check VictoriaMetrics is up
curl -s https://dev.metrics.crawbl.com/health

# Query a metric (Prometheus-compatible)
curl -s 'https://dev.metrics.crawbl.com/api/v1/query?query=up'

---

VictoriaLogs

URL: https://dev.logs.crawbl.com

VictoriaLogs is the log storage and query backend for the dev cluster. All container logs from every namespace are collected by Fluent Bit and forwarded here.

Use this to search logs across all services without needing kubectl logs.

Credentials

No authentication is required for the dev environment. The URL is accessible directly.

Quick queries

# Check VictoriaLogs is up
curl -s https://dev.logs.crawbl.com/health

# Query recent logs from the orchestrator
curl -s 'https://dev.logs.crawbl.com/select/logsql/query?query=kubernetes.namespace:backend&limit=20'

Log collection

Fluent Bit runs as a DaemonSet and collects logs from all pods on every node. Logs are shipped to VictoriaLogs with Kubernetes metadata attached (namespace, pod name, container name).

---

Where Shared Secrets Live

AWS Secrets Manager is the source of truth for shared cluster secrets.

Kubernetes receives copies of those secrets through the External Secrets Operator.

If that name is unfamiliar, it is the controller that reads secrets from AWS and writes matching Kubernetes Secret objects for workloads to use.

AWS SM Path	K8s Secret	Namespace	Contents
crawbl/dev/backend/orchestrator	orchestrator-vault-secrets	backend	DB password, Redis password, JWT secret, E2E token, MCP signing key
crawbl/dev/backend/postgresql	backend-postgresql-auth	backend	PostgreSQL postgres and crawbl user passwords
crawbl/dev/backend/firebase	firebase-service-account	backend	Firebase Admin SDK service account JSON (FCM push)
crawbl/dev/infra/cloudflare	cloudflare-api-token	cert-manager, external-dns	Cloudflare API token for DNS-01 and DNS records
crawbl/dev/runtime/openai	runtime-openai-secrets	userswarms	OpenAI API key for ZeroClaw agent runtime
crawbl/dev/edge/hmac	hmac-wasm-config	envoy-gateway-system	HMAC signing key for edge auth filter
crawbl/dev/backup/aws	backup-aws-credentials	backend, userswarms	AWS credentials for PVC backup jobs
crawbl/dev/pgweb/auth	pgweb-htpasswd	backend	pgweb Basic Auth htpasswd

Viewing a secret value

cd crawbl-backend && source .env

# List all secrets
aws secretsmanager list-secrets --region eu-central-1 \
  --query 'SecretList[].Name' --output table

# View a specific secret
aws secretsmanager get-secret-value \
  --secret-id crawbl/dev/backend/orchestrator \
  --region eu-central-1 \
  --query SecretString --output text | jq .

---

How TLS Is Handled

Dev subdomains that route through the K8s gateway share one certificate managed by cert-manager. Domains hosted on Cloudflare Pages (dev.docs.crawbl.com, crawbl.com) manage their own TLS automatically.

• Issuer: Let's Encrypt (letsencrypt-cloudflare ClusterIssuer)
• Challenge: DNS-01 via Cloudflare
• Secret: public-edge-tls in envoy-gateway-system
• Domains: dev.api.crawbl.com, dev.argocd.crawbl.com, dev.postgres.crawbl.com, dev.metrics.crawbl.com, dev.logs.crawbl.com

To add a new subdomain, update the dnsNames list in crawbl-argocd-apps/components/cert-manager/resources/certificate.yaml.

🔗 Terms On This Page

If a term below is unfamiliar, open its glossary entry. For the full list, go to Internal Glossary.

• DOKS: DigitalOcean Kubernetes, the managed Kubernetes service used for the Crawbl cluster.
• Envoy Gateway: The public gateway that receives incoming traffic and routes it to internal services.
• cert-manager: The controller that requests, renews, and stores TLS certificates inside Kubernetes.
• external-dns: The controller that creates and updates DNS records from Kubernetes or gateway resources.
• Firebase JWT: The signed Firebase identity token used to authenticate a user request.
• ArgoCD: The GitOps deployment system that keeps the cluster aligned with what is committed in Git.
• AWS Secrets Manager: The AWS service that stores shared secret values before they are copied into Kubernetes.
• External Secrets Operator: The controller that copies secrets from AWS Secrets Manager into Kubernetes Secrets.