Streams getting started

ClawQL Streams is the event-driven layer: external events (webhooks, NATS, cron, WebSocket) trigger agent sessions without a human starting each chat. This guide gives you a reading order through the draft spec chain — Streams → runtime → hardware trust — and what you can wire today vs what is still specification.

Canonical specs: ClawQL Streams · Protocol Fabric · NATS JetStream (K8s backbone)

Mental model

Event source → Streams router → Agent cell (DO / celld / cellrt) → slim core + fetch() → clawql-mcp / inference
                                      ↓                                    ↓
                         LTX cell state (bookkeeping)         host clawql-audit WORM (compliance)

Streams extends the schedule pattern from time-based triggers to arbitrary event sources. Together with search/execute and mcp-api-adapter, it completes the Protocol Fabric: MCP as the intermediate representation plus an autonomous event loop.

Before you start

1. Know Core vs Streams — Core MCP (search, execute, audit, cache) runs in interactive sessions today. Streams is draft (clawql-streams package planned) — read specs before assuming production APIs.
2. Pick a runtime path — three scaling backends, one conceptual cell:

3. Do not build Node worker_threads DO clones — adopt celld or wait for cellrt; the specs call this out explicitly.

Recommended reading order

Read top-to-bottom once, then dive into the runtime that matches your deployment:

1. ClawQL Streams (start here)

• Problem statement (Stripe Minions / managed agents pattern as platform)
• v0.2 embedded stack: clawql-streams + clawql-core + mcp-api-adapter in one cell
• Event loop, significance filters, stream_* MCP tools (planned)
• Cell-state durability: LTX → S3 on celld; compliance WORM stays host clawql-audit (Lab 5b dual-writes)

2. Durable Objects runtime

• Session sidecars: Audit / Inference / Training
• Virtual key lifecycle inside a cell
• Cloudflare vs self-hosted DO mental model

3. celld integration

• Workers API parity path (Apache 2.0, Deno celld)
• Baseline: celld v0.4.0 — celld dev, KV/Queues/R2 bindings, versioned peer tunnel
• Bundle constraints (64 MiB, no child_process, setAlarm not setInterval)
• LTX bucket = durable DO SQLite on the celld path (platform RPO=0) — not a substitute for host clawql-audit WORMAuditTrail; Lab 5b dual-writes compliance events via CLAWQL_AUDIT_WORM_URL

3b. Bursty Streams on AWS (draft)

• Hard-zero burst pattern (1M → quiet → 2M): Mechanism B (provisioning-latency) vs Karpenter
• Filler PriorityClasses + Istio ambient as independently-failing mesh layer
• clawql-k8s-operator (drift detection, not mesh-policy generation) — §12 unverified

4. clawql-cellrt

• ClawQL-owned Rust runtime (Wasmtime, embedded Vault/inference/OTel)
• Bootstrap: HTTP → local clawql-mcp before clawql-core.wasm in-process
• Fleet coordinator, cell leases, peer HMAC

5. clawql-tee → TEE air-gap audit

• Hardware layer on top of cellrt: SEV-SNP / TDX attestation
• Attestation-gated Vault secrets
• QR stream transport for unidirectional audit to regulators (QR transport spec)

6. Adjacent platform specs

• clawql-government — outcome accountability, FOIA vault
• clawql-network — peer mesh between cells (when published)

What you can try today (while specs are draft)

Use these to rehearse event → agent action → audit before stream_* tools ship. The labs below walk through each row with copy-paste steps.

Hands-on labs

These labs use shipped tools today. They mirror the Streams v0.2 loop — event source → ClawQL processes → durable trail — while stream_subscribe remains draft (Streams spec §4).

Deep dives: Schedule & notify · NATS JetStream · Audit & observability

Lab 1 — Time-based events with schedule (local, ~15 min)

Goal: Run an autonomous HTTP probe on an interval — the same cron fiber Streams will unify as sourceType: "cron".

1 — Enable schedule and start MCP

export CLAWQL_ENABLE_SCHEDULE=1
export CLAWQL_SCHEDULE_URL_ALLOWLIST_PREFIXES=https://httpbin.org,https://status.example.com
PORT=8080 npx -p clawql-mcp clawql-mcp-http

2 — Create a synthetic job (schedule tool, operation: "create"):

{
  "operation": "create",
  "name": "streams-lab-status",
  "schedule": {
    "frequency": {
      "type": "interval",
      "seconds": 120
    }
  },
  "action": {
    "kind": "synthetic",
    "synthetic_test": {
      "request": {
        "method": "GET",
        "url": "https://httpbin.org/status/200"
      },
      "assertions": [{ "type": "status_code", "equals": 200 }]
    }
  }
}

3 — Validate before leaving enabled

{ "operation": "trigger", "job_id": "<id-from-create>", "dry_run": true }

4 — Inspect run history

{ "operation": "get", "job_id": "<id>", "include_runs": true }

Success: Worker poll (default CLAWQL_SCHEDULE_POLL_MS=5000) records pass/fail, latency, and http_status in SQLite. Optional Slack bridge: Schedule § built-in notify.

Lab 2 — NATS JetStream backbone on Kubernetes (~30 min)

Goal: Stand up the in-cluster event bus the K8s Streams path uses for buffering, HPA lag signals, and cross-component pub/sub.

1 — Enable NATS in Helm

helm upgrade --install clawql ./manifests/charts/clawql-mcp -n clawql --create-namespace \
  --set nats.enabled=true \
  --set nats.persistence.enabled=true \
  --set nats.jetStream.enabled=true \
  --set nats.persistence.size=20Gi

2 — Verify rollout

kubectl -n clawql get deploy,svc | rg nats
kubectl -n clawql get deploy clawql-mcp-http -o yaml | rg "CLAWQL_NATS_URL|CLAWQL_NATS_JETSTREAM"
kubectl -n clawql port-forward svc/clawql-mcp-http-nats 8222:8222 &
curl -s http://127.0.0.1:8222/healthz
curl -s http://127.0.0.1:8222/jsz | head -c 400

3 — Publish a test event (requires NATS CLI):

kubectl -n clawql port-forward svc/clawql-mcp-http-nats 4222:4222 &
nats pub clawql.workflow.lab.hello '{"source":"streams-lab","ts":"'"$(date -Iseconds)"'"}' \
  --server nats://127.0.0.1:4222

Use stable subject prefixes from the chart (NATS JetStream § taxonomy): clawql.workflow.*, clawql.agent.*, clawql.document.*.

Success: NATS monitor /jsz shows JetStream enabled; clawql-mcp-http receives CLAWQL_NATS_URL. Streams cells on this path will subscribe to subjects the way Lab 3's bridge consumes pipeline.completed.

Lab 2b — IDP document pipeline overlay (~45 min)

Goal: Run the production-shaped document JetStream path — inbox webhook → NATS worker → run_idp_pipeline. Full walkthrough (KEDA, secrets, agent bridge, troubleshooting): NATS IDP pipeline (Learn).

1 — Apply the NATS IDP values overlay

helm upgrade --install clawql ./manifests/charts/clawql-mcp \
  -f ./manifests/charts/clawql-mcp/values-nats-idp.example.yaml \
  --set envFromSecret=clawql-provider-env \
  -n clawql --create-namespace

The overlay turns on:

Full file: values-nats-idp.example.yaml.

2 — Secret keys (minimum for webhook smoke)

Create clawql-provider-env (or extend your existing provider secret) with at least:

Production also needs NEXTCLOUD_* / CONESHARE_* vendor URLs and credentials — see NATS IDP E2E runbook.

3 — Offline Helm smoke (no live cluster HTTP)

SMOKE_HELM_ONLY=1 bash scripts/dev/smoke-nats-idp-webhooks.sh

Asserts rendered env for IDP + Coneshare consumers and (when KEDA is enabled) scaled-object triggers.

4 — Live webhook smoke

kubectl -n clawql port-forward svc/clawql-mcp-http 8080:8080 &

export CLAWQL_HTTP_BASE=http://127.0.0.1:8080
export CLAWQL_NEXTCLOUD_WEBHOOK_TOKEN=…   # from Secret
export CLAWQL_CONESHARE_WEBHOOK_TOKEN=…

bash scripts/dev/smoke-nats-idp-webhooks.sh

Check the worker consumed clawql.document.inbox.arrived:

kubectl -n clawql logs deploy/clawql-mcp-http-nats-worker --tail=80

Inbox payloads in the smoke script use dry_run: true — the worker plans hops without calling vendors. Expect a terminal pipeline.completed (or hop publish) when NATS publish is on.

5 — Architecture reference

POST /idp/nextcloud/webhook  →  clawql.document.inbox.arrived
                                        ↓
                         nats-worker (idpPipeline=true)
                                        ↓
                         run_idp_pipeline (dry_run or live)
                                        ↓
                         Stirling redact → processed upload → …

Optional Coneshare viewer follow-up publishes clawql.document.coneshare.viewer and can resume Argo + Slack notify. Sample pack: lending W-2.

6 — KEDA autoscaling (optional)

When inbox volume bursts, enable lag-based worker scale (#257):

helm upgrade --install clawql ./manifests/charts/clawql-mcp \
  -f ./manifests/charts/clawql-mcp/values-nats-idp.example.yaml \
  --set envFromSecret=clawql-provider-env \
  --set nats.keda.enabled=true \
  --set nats.keda.bootstrapConsumer=true \
  -n clawql

Requires KEDA and in-cluster NATS monitoring (8222). Verify: kubectl -n clawql get scaledobject | rg nats. Full tables and limits: NATS IDP pipeline § KEDA.

Success: Webhook POST → JetStream subject → worker log line → (dry-run) pipeline.completed. Continue to Lab 3 for Hermes/Pi-style MCP follow-up on completion events.

Lab 3 — NATS → MCP agent bridge (async follow-up, ~20 min)

Goal: Close the loop JetStream event → agent MCP tools without waiting for stream_* — the pattern Hermes/Pi use for IDP follow-up (#128).

Prerequisites: Lab 2 NATS reachable; MCP HTTP running (Lab 1 or in-cluster).

export CLAWQL_NATS_URL=nats://127.0.0.1:4222   # or in-cluster DNS
export CLAWQL_NATS_JETSTREAM=1
export CLAWQL_NATS_ENABLE_CONSUMER=1
export CLAWQL_NATS_AGENT_BRIDGE=1
export CLAWQL_MCP_HTTP_URL=http://127.0.0.1:8080/mcp
# optional on pipeline.failed:
# export CLAWQL_NATS_AGENT_BRIDGE_NOTIFY_CHANNEL=C01234567

npm run nats:agent-bridge

The bridge durable consumer (clawql-idp-agent-bridge) calls memory_ingest, notify, and audit on pipeline completion events — the same tool trio Streams agents will use for significance filtering and human escalation.

Sample pack: docs/examples/idp/idp-nats-agent (Hermes skill + Pi extension).

Lab 4 (optional) — Probes + audit breadcrumbs

Pair Lab 1 with audit so scheduled runs leave MCP-visible breadcrumbs before Loki/Grafana:

export CLAWQL_ENABLE_AUDIT=1
# optional: CLAWQL_AUDIT_LOKI_URL=…

After each synthetic run, agents (or the bridge in Lab 3) append audit entries with job_id, correlation ids, and runbook links. See Audit tool & observability.

Lab 5 — celld v0.4.0 local smoke (~25 min)

Goal: Run the upstream Durable Object counter on celld v0.4.0 with celld dev — the same Workers/DO surface ClawQL Streams targets for GatewayDO / SubscriptionDO / AgentSessionDO cells (celld integration §0).

Prerequisites: git, curl, esbuild on PATH (celld deploy requirements).

1 — Pin celld v0.4.0

CELLD_VERSION=v0.4.0 curl -fsSL https://celld.dev/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
celld --version
# optional supply-chain check:
gh attestation verify --repo denoland/celld

Important: v0.3.x and v0.4.x cannot share a fleet. If you already run v0.3 nodes, stop every node before upgrading (release notes).

2 — Start the counter example with celld dev

git clone --depth 1 --branch v0.4.0 https://github.com/denoland/celld /tmp/celld-v040
cd /tmp/celld-v040/examples/counter
celld dev --port 9876

celld dev (new in v0.4.0) uses a local object store under .celld/dev — no Docker or cloud bucket. Add .celld/ to .gitignore in your own Worker projects.

3 — Exercise the Durable Object

In another terminal:

curl -s http://127.0.0.1:9876/
curl -s http://127.0.0.1:9876/increment
curl -s http://127.0.0.1:9876/increment
curl -s http://127.0.0.1:9876/

Each increment wakes a named cell, persists SQLite state, and survives celld dev restarts (state lives in .celld/dev until you delete it).

4 — Map to ClawQL Streams

Model calls in a real Streams bundle still use fetch() to clawql-inference — never subprocess — and stay under the 64 MiB bundle cap.

5 — Optional: fleet deploy smoke

When you have S3-compatible credentials (R2, GCS, or S3):

export CELLD_BUCKET=s3://clawql-streams-state
export S3_ENDPOINT=https://ACCOUNT_ID.r2.cloudflarestorage.com
export AWS_REGION=auto
export AWS_ACCESS_KEY_ID=…
export AWS_SECRET_ACCESS_KEY=…

cd /tmp/celld-v040/examples/counter
celld deploy . --bucket "$CELLD_BUCKET" --endpoint "$S3_ENDPOINT" --region "$AWS_REGION"
celld --bucket "$CELLD_BUCKET" --endpoint "$S3_ENDPOINT" --region "$AWS_REGION" \
  --listen 127.0.0.1:8080 --advertise 127.0.0.1:8081

Verify readiness on the v0.4.0 health path (not /__celld/health):

curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8080/.well-known/celld/health
celld diagnose --bucket "$CELLD_BUCKET" --endpoint "$S3_ENDPOINT" --region "$AWS_REGION"
celld cell list --bucket "$CELLD_BUCKET" --json | head

Success: Counter increments locally; you understand celld dev vs fleet deploy, v0.4.0 health probes, and how DO naming maps to Streams sidecars. Continue to Lab 5b for the ClawQL Wrangler skeleton and bundle gate.

Lab 5b — ClawQL Streams Wrangler skeleton + bundle-check (~30 min)

Goal: Run the in-repo GatewayDO / SubscriptionDO / AgentSessionDO skeleton (docs/examples/streams-celld) under celld v0.4.0 and enforce the 64 MiB Workers bundle limit.

Prerequisites: Lab 5 celld v0.4.0 + esbuild on PATH.

1 — Local dev with the ClawQL skeleton

cd docs/examples/streams-celld
clawql streams celld dev --port 9876
# or: celld dev --port 9876

2 — Webhook → significance → session spawn

curl -s http://127.0.0.1:9876/health | jq .
curl -s -X POST http://127.0.0.1:9876/webhook/demo-lab \
  -H 'content-type: application/json' \
  -H 'x-clawql-event-id: lab-5b-1' \
  -d '{"hello":"streams"}' | jq .
curl -s http://127.0.0.1:9876/admin/status | jq .

Repeat the POST with the same x-clawql-event-id — AgentSessionDO should report idempotent: true.

3 — Bundle size gate + clawql-core

The skeleton embeds clawql-core/streams-slim (audit + cache + hash-chain). Build workspace packages first:

npm run build -w clawql-merkle -w clawql-core
clawql streams celld bundle-check
# equivalent: node docs/examples/streams-celld/scripts/bundle-check.mjs

Expect ≈ 0.4 MiB (~0.6% of the 64 MiB Workers limit). The full clawql-core barrel is not used — it would pull webmcp-draft (node:fs).

Webhook responses include session.audit.clawqlCore (hash-chained append), session.audit.ltx.snapshotKey: "audit:ring" (bookkeeping only), and — when CLAWQL_AUDIT_WORM_URL is set — session.audit.compliance pointing at host clawql-audit. With CLAWQL_MCP_URL set, MCP tools call Streamable HTTP. With CLAWQL_MCP_ADAPTER_URL set, adapter_search probes mcp-api-adapter REST.

Smoke with a mock MCP + mock adapter (requires celld):

STREAMS_CELLD_SMOKE_REQUIRED=1 bash docs/examples/streams-celld/scripts/smoke.sh

Verified by (CI): streams-slim unit tests, mcp-fetch / adapter-fetch units, bundle-check, Helm celld template tests, and (when celld installs) the smoke script above. Full matrix: Streams + celld evidence.

CI and local release gates fail closed when the bundled Worker exceeds 67108864 bytes (celld integration §5).

4 — Optional: fleet deploy + Helm celld backend

Deploy the skeleton to your fleet bucket, then enable the chart overlay:

export CELLD_BUCKET=s3://clawql-streams-state
export S3_ENDPOINT=https://ACCOUNT_ID.r2.cloudflarestorage.com

clawql streams celld deploy --project docs/examples/streams-celld \
  --bucket "$CELLD_BUCKET" --endpoint "$S3_ENDPOINT"

helm upgrade --install clawql ./manifests/charts/clawql-mcp \
  -f ./manifests/charts/clawql-mcp/values-streams-celld.example.yaml \
  --set envFromSecret=clawql-provider-env \
  --set streams.celld.bucket="$CELLD_BUCKET" \
  --set streams.celld.endpoint="$S3_ENDPOINT" \
  -n clawql --create-namespace

The celld StatefulSet probes /.well-known/celld/health (v0.4.0) and injects CLAWQL_MCP_URL + INFERENCE_URL. Full checklist: docs/examples/idp/streams-celld. Note: live cluster webhook→LTX is manual; CI asserts Helm templates only.

Success: Webhook spawns a session with in-process streams-slim audit/cache + LTX keys; optional MCP/adapter fetch; bundle-check under 64 MiB; optional Helm deploy with streams.scalingBackend: celld.

K8s operator sketch

# Event backbone (Lab 2) or celld fleet (Lab 5b)
helm upgrade --install clawql ./manifests/charts/clawql-mcp -n clawql --create-namespace \
  --set nats.enabled=true --set nats.jetStream.enabled=true

# celld Streams path (Lab 5b overlay)
# helm upgrade --install clawql ./manifests/charts/clawql-mcp -n clawql \
#   -f ./manifests/charts/clawql-mcp/values-streams-celld.example.yaml \
#   --set streams.celld.bucket=s3://clawql-streams-state

# MCP + inference (cells call inference over HTTP)
export CLAWQL_ENABLE_SCHEDULE=1
clawql inference serve --port 8080

When mcpProxy is enabled, north–south MCP hits the proxy before clawql-mcp-http — pair with Panguard enforcement for JWT ATR at the chokepoint.

Full deploy paths: Kubernetes & Helm · Operations guide.

cellrt vs celld decision

Streams v0.2 adopts celld for the JS/Workers path; cellrt is the long-run ClawQL-owned production runtime.

Troubleshooting orientation

What to read next

• Protocol Fabric — proven multi-protocol loop
• Agentic Gateway — HTTP target for cells
• Defense in depth — Streams sits in the audit / WORM layer
• Benchmarks — OpenBenchTrace / RTP training emission (Streams § training)