Get started with clawql-inference

Spin up ClawQL’s Agentic Gateway inference control plane in minutes: OpenAI-compatible /v1, bring-your-OpenRouter or direct BYOK providers, and a clear path to pair MCP + vault memory.

Website: docs.clawql.com/getting-started/inference

Deep reference: clawql-inference · Providers: Inference providers · Fabric: Zero-Trust Agentic Fabric

Already on OpenRouter?

If you already pay OpenRouter and do not have Anthropic/OpenAI/DeepSeek keys in ClawQL yet, start with the key you have:

1. Set OPENROUTER_API_KEY (local .env, GitHub Actions secret, or Vault).
2. Use catalog models such as openrouter/deepseek/deepseek-chat or openrouter/qwen/qwen3.6-plus.
3. Point clients at clawql-inference (OPENAI_BASE_URL / gateway /v1) — same OpenAI-compatible API, with ClawQL memory and policy in front.

That path is intentional: bring your aggregator key first, then add direct BYOK (deepseek/*, anthropic/*, openai/*, …) when you want to disintermediate OpenRouter for cost, latency, or residency.

OpenBench CI defaults to this OpenRouter-first model so live benchmarks work with a single secret. See OpenBench GitHub Actions.

What you get

On a laptop this is your Edge Agentic Gateway. Later you can grow into a Managed Gateway or Dedicated Virtual Gateway without changing the client contract (OPENAI_BASE_URL + key).

How it works

  OpenAI SDK / OpenCode / Cursor / curl
              │  OPENAI_BASE_URL=…/v1
              ▼
     clawql inference serve
              │
     ┌────────┴────────┐
     │  catalog + tiers │  virtual keys / entitlements
     │  cache + fallback│  call store / WORM lineage
     └────────┬────────┘
              │
     ┌────────┼────────────────────────────┐
     ▼        ▼                            ▼
  openrouter/*                 DeepSeek / Groq / OpenAI / …
  (existing aggregator key)    (direct BYOK when ready)

Day-one posture for OpenRouter customers: bring the aggregator key you already have. Add direct vendor keys later when you want to skip OpenRouter for production traffic. ClawQL routes provider/model the same either way.

Pair with MCP when you want agents to search / execute / remember against your APIs and vault:

  Agent ──► /v1  (models)     clawql-inference
       └──► /mcp (tools)      clawql-mcp-http  +  Memory/ vault

Today those are often two local processes; the product is one Agentic Gateway. Managed offerings co-host them behind one hostname.

Pick your path

Five-minute start

1. Install CLI

# From a ClawQL checkout
npm ci && npm run build
# or: npm i -g clawql-mcp

2. Policy + home

export CLAWQL_HOME="${CLAWQL_HOME:-$HOME/.clawql}"
mkdir -p "$CLAWQL_HOME/Inference"
cp examples/inference/policy.yaml "$CLAWQL_HOME/Inference/policy.yaml"
# From outside the repo, download the example or set:
# export CLAWQL_INFERENCE_POLICY_MANIFEST=/path/to/policy.yaml

3. Bring a provider key

Already on OpenRouter? Use that key only:

export OPENROUTER_API_KEY=sk-or-…     # recommended if you already have OpenRouter

Direct BYOK (when you have per-provider keys and want to skip the aggregator):

# export DEEPSEEK_API_KEY=sk-…        # direct DeepSeek
# export GROQ_API_KEY=…               # or another BYOK provider
# export OPENAI_API_KEY=…             # OpenAI / embeddings
# export ANTHROPIC_API_KEY=…          # Anthropic

Never commit keys. Local secrets belong in your shell / secret manager — not in git or MCP JSON.

4. Serve

clawql inference policy show     # policy_source: manifest+env
clawql inference serve --port 8080
# or: npx clawql-inference

5. Smoke test

curl -s http://127.0.0.1:8080/healthz

curl -s http://127.0.0.1:8080/v1/models | head

# OpenRouter-first (when OPENROUTER_API_KEY is set)
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"openrouter/deepseek/deepseek-chat","messages":[{"role":"user","content":"Say hi in one sentence."}]}'

# Direct BYOK example (when DEEPSEEK_API_KEY is set)
# curl -s http://127.0.0.1:8080/v1/chat/completions \
#   -H 'Content-Type: application/json' \
#   -d '{"model":"deepseek/deepseek-chat","messages":[{"role":"user","content":"Say hi in one sentence."}]}'

One-shot without HTTP:

clawql inference complete --model openrouter/deepseek/deepseek-chat --message "hello"

Point a client

OpenAI SDK / any OpenAI-compatible tool

export OPENAI_BASE_URL=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=sk-…   # vendor key, or a ClawQL virtual key when keys are enabled

Model ids use provider/model (e.g. deepseek/deepseek-chat, groq/llama-3.3-70b-versatile) or aliases (clawql/cheap-chat). OpenRouter ids keep the prefix: openrouter/qwen/….

OpenCode (headless / OpenBench)

# terminal 1
OPENROUTER_API_KEY=sk-or-… clawql inference serve --port 8080

# terminal 2
CLAWQL_OPENBENCH=1 clawql opencode --non-interactive \
  --model clawql/openrouter/deepseek/deepseek-chat \
  --inference-url http://127.0.0.1:8080/v1 \
  --message "Say hello" \
  --workdir /tmp/demo

A/B harness notes: Benchmarks · OpenBench Actions guide

Add MCP and memory

Inference answers models. MCP answers tools + memory.

# Vault-first MCP (desktop / stdio)
npx -p clawql-mcp clawql onboard --interactive

# Or HTTP MCP for local agents
PORT=8090 npx -p clawql-mcp clawql-mcp-http
# MCP endpoint: http://127.0.0.1:8090/mcp

Full agent wiring: Agent setup · Teams sync: For teams

Security defaults

Start local; harden before sharing a URL:

Defense-in-depth narrative: MCP proxy JWT ATR · Defense in depth

What “good” looks like next

1. Local Edge — clawql inference serve alone (models only).
2. Managed Edge Gateway (go-live) — one hostname for /v1 + /mcp + memory:

export OPENROUTER_API_KEY=sk-or-…   # or DEEPSEEK_API_KEY / other BYOK
clawql gateway create --profile process --team demo
# → MCP URL, Inference URL, virtual key (shown once)

Docker variant: clawql gateway create --profile local-docker --team demo
Example compose: examples/managed-gateway/.

Kubernetes (chart defaults off): set inference.enabled=true and managedGateway.enabled=true on charts/clawql-mcp for an in-cluster /mcp + /v1 edge. MCP accepts the same inference virtual key (tenantId = key.team); optional shared home via inference.home.

3. Dedicated Virtual Gateway — customer VPC, WORM, Vault, team sync (For teams, Packer / Pulumi).
4. Regional Hub — ClawQL-managed metering / routing pipe only — never holds your sovereign vault.

GTM narrative: Inference-first GTM · Architecture: Agentic Fabric

Troubleshooting

Next reading

• clawql-inference reference — policy schema, cache, flywheel, entitlements
• Inference providers — OpenRouter + BYOK builtins
• Agent setup — MCP + memory onboarding
• Token efficiency — twelve compounding layers
• Getting started overview