Technology Stack
Crawbl is built on proven, enterprise-grade technologies chosen for reliability, security, and operational simplicity.
Orchestration Layer (Control Plane)
The central coordination layer that manages all platform operations.
| Component | Technology | Purpose |
|---|---|---|
| Language | Go 1.25+ | High performance, strong typing, excellent concurrency |
| Framework | Custom orchestration engine | Plugin architecture for extensibility |
| API | REST + WebSocket | Comprehensive authentication, real-time events |
| Infrastructure | Kubernetes + Infrastructure as Code | Cloud-agnostic provisioning and orchestration |
| GitOps | GitOps continuous deployment | Declarative management, automated rollouts |
Key Capabilities
- Authentication & Authorization: Identity provider JWT tokens and API key-based programmatic access
- User & Workspace Management: Provisioning and lifecycle management
- Request Routing: Intelligent dispatch to appropriate agent runtimes
- LLM Mediation: Provider selection, cost controls, usage tracking
- Integration Management: OAuth connections to external services
- Audit & Compliance: Comprehensive logging and access controls
Runtime Layer (Data Plane)
Isolated execution environments for AI workloads.
| Component | Technology | Purpose |
|---|---|---|
| Agent Runtime | ZeroClaw (forked) | Multi-agent execution with custom extensions |
| Isolation | Kubernetes namespaces | Pod-level security boundaries |
| Scaling | HPA | Horizontal pod autoscaling based on workload |
| Storage | Persistent volumes | Runtime state and workspace persistence |
Key Capabilities
- Per-User Runtimes: Dedicated containers for each user/swarm
- Resource Isolation: CPU, memory, and network limits per runtime
- Secure Communication: Internal-only communication with orchestration
- Horizontal Scaling: Independent scaling based on demand patterns
Supporting Infrastructure
| Service | Technology | Purpose |
|---|---|---|
| Database | PostgreSQL | Platform metadata and user data |
| Caching | Redis | Real-time event processing, session management |
| Messaging | Socket.IO | Real-time communication between components |
| Secrets | Managed secrets service | Credential management with automated secrets synchronization |
| Observability | Logging + Monitoring | Comprehensive tracing and alerting |
Technology Decisions
Why Go for the Orchestrator?
- Performance: Compiled, efficient, low latency
- Concurrency: Goroutines handle thousands of simultaneous connections
- Type Safety: Strong typing catches errors at compile time
- Ecosystem: Rich libraries for HTTP, gRPC, database drivers
- Operations: Single binary deployment, fast startup
Why Kubernetes?
- Isolation: Namespaces provide security boundaries per tenant
- Scalability: Built-in autoscaling and load balancing
- Portability: Runs on any cloud or on-premises
- Ecosystem: Mature tooling for GitOps, secrets management, and observability
Why ZeroClaw?
- Proven Runtime: Battle-tested agent execution engine
- Extensibility: Custom webhook extensions for multi-agent support
- Integration: Native MCP server support for tool calling
- Community: Active development and documentation
Next Steps
- Architecture Overview - How the layers interact
- Security Model - Security principles and implementation
- Deployment Options - Infrastructure requirements