Architecture Overview
Crawbl's three-layer architecture separates concerns and enables independent scaling of each layer.
Layer Diagram
Click diagram to zoom
Layer 1: Client Interface
The user-facing layer provides multiple access points:
| Interface | Purpose |
|---|---|
| Mobile App | Primary user interface (Flutter) |
| REST API | Synchronous operations, CRUD actions |
| WebSocket | Real-time events, streaming responses |
All client communication goes through the orchestration layer—runtimes are never directly accessible from outside.
Layer 2: Orchestration Layer
The control plane handles all coordination:
Authentication & Authorization
- JWT-based identity verification for mobile users
- API key-based programmatic access
- Workspace-scoped permissions
Request Routing
- Routes requests to appropriate agent runtime
- Load balances across runtimes
- Handles failover and recovery
LLM Mediation
- Provider selection (Anthropic, OpenAI, Google, etc.)
- Cost controls and usage tracking
- Rate limiting and quota management
Integration Hub
- OAuth token management for external services
- Credentials stored in a managed secrets service
- Webhook handling for external events
Audit & Compliance
- Comprehensive request logging
- Access control verification
- Compliance reporting capabilities
Layer 3: Runtime Layer
The data plane provides isolated execution:
Agent Runtime Architecture
Each user gets an isolated runtime environment:
Click diagram to zoom
Key Properties
| Property | Implementation |
|---|---|
| Isolation | Dedicated Kubernetes namespace per user |
| Security | No direct internet access, mediated by orchestrator |
| State | Persistent volumes for workspace data |
| Scaling | Horizontal pod autoscaling based on demand |
| Recovery | Automatic restart on failure, state preserved |
Communication Patterns
Request Flow
Click diagram to zoom
Tool Calling
When an agent needs to use an external tool:
- Agent runtime sends tool request to orchestrator via MCP
- Orchestrator validates and executes the tool
- Result returned to agent runtime for context
This mediation ensures:
- All external calls are logged and auditable
- User permissions are enforced
- Secrets never leave the control plane
Deep Dive
For detailed technical documentation:
- System Overview - Component interactions
- Request Lifecycle - How requests flow through the system