title: ClawQL Celld Integration — Spec v0.1

ClawQL Celld Integration — Spec v0.1

Hands-on walkthrough: Streams getting started (Learn) — §3 celld reading order; Lab 5 for celld v0.4.0 local smoke; Labs 1–3 for schedule + NATS today.

Status: Draft · September 2026 · v0.1
celld baseline: v0.4.0 (2026-08-28) — pin with CELLD_VERSION=v0.4.0 on install; do not mix v0.3.x and v0.4.x in one fleet
Package surface: celld (self-hosted Durable Objects) for ClawQL Streams
Depends on: clawql-streams v0.2 · clawql-durable-objects.md · clawql-inference · clawql-core · mcp-api-adapter
Related: clawql-cellrt.md (ClawQL-owned Rust runtime) · aws-celld-burst.md (AWS burst / Karpenter / Istio ambient — draft) · celld docs · limitations · security · Cloudflare compat · denoland/celld (Apache 2.0)

0. Version baseline — celld v0.4.0

ClawQL Streams docs and labs assume celld v0.4.0 unless a page pins an older tag for a migration note.

Upgrade from v0.3.0: stop every v0.3.0 node before starting v0.4.0 — peer tunnel protocol and large KV epoch references are incompatible; no rolling update. See celld docs — Shut down and roll out a node.

1. What this is

This document specifies how ClawQL Streams runs on celld — Deno's self-hosted, S3-backed Durable Objects runtime. celld exposes the same Workers / Durable Object JavaScript API as Cloudflare, with SQLite per cell and LTX replication to an operator-owned bucket (RPO=0).

ClawQL's decision (Streams v0.2): do not build a custom DO runtime on Node worker_threads. Adopt celld for Workers/DO API–compatible self-hosted Durable Objects; keep Cloudflare for hosted; keep Kubernetes HPA for regulated until a DO runtime is production-stable. The ClawQL-owned production runtime is clawql-cellrt (Rust + Wasmtime) — complementary to celld, not a Node rewrite.

Why celld vs build-own

Facts of record: Apache 2.0 · ~58 MB binary · hibernation model (resident / idle / hibernated / inactive) · vendor density hint ~1000 resident / 8 GB (re-measure before GTM) · RPO=0 LTX · one application per fleet (alpha). No ClawQL-owned $/cell-month figure — see Streams §9.1.

2. Runtime constraints

Source of truth: Cloudflare compatibility and limitations. Unknown keys/APIs fail loudly at deploy or first use.

Available (use these)

Unavailable or unsafe (avoid + workaround)

3. Patterns: alarms, fetch, WebSocket reconnect

3.1 setAlarm instead of intervals

// api_poll / session TTL / reconnect backoff
await this.ctx.storage.put("alarm_intent", { kind: "api_poll", url, intervalMs });
await this.ctx.storage.setAlarm(Date.now() + intervalMs);

async alarm() {
  const intent = await this.ctx.storage.get<AlarmIntent>("alarm_intent");
  // do work…
  await this.ctx.storage.setAlarm(Date.now() + intent.intervalMs);
}

3.2 fetch() to clawql-inference — not subprocess

Model calls never spawn a process. The DO holds a virtual key and calls the inference gateway:

const res = await fetch(env.INFERENCE_URL + '/v1/messages', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${virtualKey}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(payload),
})

PAL, provider SDKs, and credential vaults stay in clawql-inference — outside the 64 MiB cell bundle.

3.3 WebSocket reconnect via SQLite intent

Outbound sockets keep a cell resident and do not continue when the cell moves nodes (limitations). Persist intent and reconnect after activation:

await this.ctx.storage.put('ws_intent', {
  reconnect: true,
  sourceUrl,
  authRef,
  lastEventId,
  backoffMs: 1000,
})

// on close / after constructor wake:
await this.ctx.storage.setAlarm(Date.now() + backoffMs)

Prefer pinning ingress for a cell to its owner node when latency matters; cross-node WS close/reconnect coverage is thinner than single-node.

4. DO architecture

Logical types match clawql-durable-objects.md; celld is the self-hosted runtime.

Ingress (TLS terminator)
    │
    ▼
Gateway Worker / GatewayDO
    │
    ├─ idFromName("sub:" + subscriptionId) → SubscriptionDO
    │
    └─ on significance pass:
         idFromName("sess:" + subscriptionId + ":" + eventId) → AgentSessionDO
              ├─ AuditSidecar  → storage.put (LTX WORM)
              ├─ InferenceSidecar → fetch(clawql-inference)
              └─ TrainingDataSidecar → RTP/OBT in SQLite → export

4.1 Naming for idempotency

Stable DO names make replay safe:

Gateway still allocates doInstanceId / virtualKeyId before first spawn and writes DO_CREATED once (guard with a spawned flag in session SQLite).

4.2 SQLite schemas (illustrative)

SubscriptionDO

AgentSessionDO — same contract as DO companion:

5. Bundle architecture

celld deploy (esbuild)
  └─ Worker + DO classes
        ├─ clawql-streams (router, filter, stream_* MCP) — planned package
        ├─ clawql-core/streams-slim (audit, cache, hash-chain) — Workers-safe entry
        ├─ fetch(CLAWQL_MCP_URL) → clawql-mcp search/execute/memory_* (Streamable HTTP)
        └─ fetch(CLAWQL_MCP_ADAPTER_URL) → mcp-api-adapter REST POST /{tool}
  env / vars ≤ 1 MiB
  code ≤ 64 MiB

In-process today: AgentSessionDO imports clawql-core/streams-slim for hash-chained audit + session cache. Example: docs/examples/streams-celld.

Out-of-process today: search / execute / memory_* via Streamable HTTP MCP (CLAWQL_MCP_URL). Optional protocol-fabric REST via CLAWQL_MCP_ADAPTER_URL (POST /\{tool\} on mcp-api-adapter). Inference via fetch(INFERENCE_URL). Do not embed full clawql-api, clawql-memory, mcp-api-adapter, or the full clawql-core barrel (webmcp-draft / node:fs).

Why not “full core” inside the cell?

Demo the entire product by running the sidecars for real — not by stuffing Express into the Worker:

# Process smoke (CI + local): celld + clawql-mcp-http + mcp-api-adapter + inference stub
STREAMS_CELLD_SMOKE_REQUIRED=1 bash docs/examples/streams-celld/scripts/full-stack-smoke.sh

# Optional containers for MCP + adapter (celld still on the host):
docker compose -f docs/examples/streams-celld/docker-compose.full.yml up --build

Durable audit (two layers):

1. DO bookkeeping — after each spawn, the isolate hash-chain is flushed to DO storage (audit:ring snapshot + audit:seq:\{n\}) so LTX can retain a snapshot across isolate restarts. The in-process ring does not loadTip from LTX — a new isolate starts at genesis. Treat this as session-resumption state, not compliance.
2. Host compliance — consequential events (SESSION_START, …) fetch CLAWQL_AUDIT_WORM_URL → host clawql-audit WORMAuditTrail (tip-load / dual-ack). See streams-celld-evidence.md.

Still deferred: offline Workers-safe clawql-api slim.

Provider specs: ship a slim default set in the bundle; load additional OpenAPI/GraphQL specs via fetch into SQLite on first use or at subscription create — do not embed full enterprise catalogs in env.

5.1 esbuild 64 MiB CI check

# clawql streams celld bundle-check
clawql streams celld bundle-check --project docs/examples/streams-celld
# Fail the job if Worker/DO artifact size > 67108864 bytes

CI must fail closed on oversize bundles. Prefer:

• Externalize clawql-inference (always)
• Import clawql-core/streams-slim, not clawql-core (full barrel)
• Tree-shake unused providers
• Avoid Node polyfills that pull fs / http (keep nodejs_compat for crypto / Buffer only)
• Optional further Streams-slim cuts if Effect + api ever approach budget (open question in Streams §15)

6. Bucket layout

celld uses one fleet bucket as administrative authority (deployments, SQLite/LTX, ownership leases, peer secret). ClawQL still separates concerns:

| Team vault sync bucket (existing ClawQL R2/S3) | Obsidian vault / memory_sync — not the celld fleet bucket | | Training export (optional) | RTP/OBT datasets (HF / dedicated prefix) — distinct from fleet authority |

Do not reuse fleet-bucket credentials for vault sync or public dataset upload. Scope each credential to one role (security).

7. Deployment

7.1 Install (pin v0.4.0)

CELLD_VERSION=v0.4.0 curl -fsSL https://celld.dev/install.sh | sh
celld --version   # expect v0.4.0
gh attestation verify --repo denoland/celld   # build attestation

Binary ~58 MB; replication is in-process (no external Litestream sidecar).

7.1b Local development (celld dev, v0.4.0+)

Before provisioning a fleet bucket, iterate with the upstream counter example:

git clone --depth 1 --branch v0.4.0 https://github.com/denoland/celld
cd celld/examples/counter
npm install   # if the example ships a package.json
celld dev --port 9876
# Worker default: http://127.0.0.1:9876

celld dev uses a persistent local object store under .celld/dev (add .celld/ to .gitignore). It rebuilds on source changes without Docker or cloud credentials. The internal operator API stays on loopback only.

Readiness during fleet rollout: probe GET /.well-known/celld/health (v0.4.0+), not /__celld/health.

7.2 Configure object storage

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_REGION=auto
export S3_ENDPOINT=https://ACCOUNT_ID.r2.cloudflarestorage.com
export CELLD_BUCKET=s3://clawql-streams-state

celld uses the AWS credential chain (not ~/.aws profiles/SSO).

7.3 Deploy application

# esbuild on PATH; wrangler.json or wrangler.jsonc (not .toml)
celld deploy . \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION"

Accepted config keys only: name, main, compatibility_date, compatibility_flags, durable_objects, migrations, assets, services, vars. Unknown keys abort deploy.

7.4 Start fleet

celld \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION" \
  --listen 0.0.0.0:8080 \
  --advertise node-a.internal:8080

Add nodes with the same bucket and distinct --advertise addresses. Discovery is via bucket leases — no join command.

7.5 Diagnose and inspect (v0.4.0+)

celld diagnose \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION"

# List Durable Object instances (after traffic creates ownership records)
celld cell list --bucket "$CELLD_BUCKET" --json | head

# KV / Queue ops when Wrangler bindings declare namespaces
celld kv list MY_NAMESPACE --bucket "$CELLD_BUCKET"
celld queue info MY_QUEUE --bucket "$CELLD_BUCKET"

Reports expired leases, bad advertise addresses, unreachable peers, auth failures, protocol skew. During rolling node updates, wait for restoring=0 on every node before stopping the next peer (celld docs).

7.6 Helm

streams:
  scalingBackend: celld
  celld:
    enabled: true
    bucket: s3://clawql-streams-state
    endpoint: https://….r2.cloudflarestorage.com
    region: auto

CLI wrappers: clawql streams celld install|deploy|start|diagnose|bundle-check (Streams §11).

8. Security hardening

Regulated tenants that need hostile multi-tenant isolation or certified controls should use scalingBackend: kubernetes until celld exits alpha.

9. Cloudflare vs celld

10. Testing

Evidence matrix (commands + honesty about gaps): streams-celld-evidence.md.

Do not treat Miniflare alone as production parity for LTX, peer HMAC, or cross-node WebSocket behavior.

11. Known gaps

Track against upstream celld alpha:

1. TCP stub — cloudflare:sockets connect() is a silent inert stub; Streams must not depend on raw TCP.
2. WebSocket cross-node — thinner test coverage for close codes/reconnect across nodes; prefer owner-node ingress; always persist reconnect intent.
3. Pressure shedding — off until safe defaults; tune CELLD_MAX_RESIDENT_CELLS / RSS manually.
4. Manual updates — installer immutable releases + current pointer; no auto-update agent; pin CELLD_VERSION=v0.4.0 until the next qualified release.
5. v0.3 → v0.4 fleet cutover — full stop required; no mixed-version fleet (§0).
6. One application per fleet — no multi-tenant scheduler; isolate ClawQL orgs with separate fleets/buckets.
7. Crypto gaps — no deriveKey; design around digest/HMAC/AES-GCM/sign.
8. Silent Node stubs — importing unimplemented node:* may not fail; lint/ban child_process, http, net in the DO package.
9. Streaming fetch retry — v0.4.0 peer tunnel does not retry after body transmission starts; use stable operation IDs.

Further reading

• docs/streams/clawql-streams.md — Streams Specification v0.2
• docs/streams/clawql-cellrt.md — ClawQL-owned Rust + Wasmtime cell runtime
• docs/streams/clawql-tee.md — hardware TEE path on cellrt
• docs/streams/clawql-tee-airgap-audit.md — QR air-gap audit transport
• docs/streams/clawql-durable-objects.md — session / sidecar / virtual key contract
• docs/inference/clawql-inference.md — virtual keys, PAL
• docs/mcp/mcp-api-adapter.md — MCP → APIs (out-of-process from cells today)
• docs/streams/streams-celld-evidence.md — evidence matrix + CI commands
• celld.dev · docs · limitations · security · compat · GitHub