Skip to main content
ClawQL

Docker Desktop: Istio and the observability stack

When you install ClawQL on Docker Desktop Kubernetes with CLAWQL_LOCAL_K8S_ISTIO=ambient or sidecar, the repo also brings up a local observability lab in istio-system: Prometheus, Grafana, Grafana Loki (optional), Grafana Tempo, Kiali, and a small OpenTelemetry Collector that forwards app OTLP to Tempo.

Canonical doc (same content, PR-friendly, deeper detail): docs/deployment/docker-desktop-istio-observability.md. Install commands and env toggles: docker/README.md. Kubernetes overview: Kubernetes.

One command (recap)

CLAWQL_LOCAL_K8S_ISTIO=ambient make local-k8s-up
  • Skip all Istio sample addons: CLAWQL_ISTIO_INSTALL_KIALI=0
  • Keep Prometheus + Kiali but skip Grafana, Tempo, Loki, and the OTel collector (lighter RAM): CLAWQL_ISTIO_INSTALL_HEAVY_OBSERVABILITY_ADDONS=0
  • Skip Grafana Loki only (keep Grafana + Tempo + collector): CLAWQL_ISTIO_INSTALL_LOKI_TEMPO=0

Where everything lives

ToolNamespaceReach it
Prometheusistio-systemkubectl port-forward -n istio-system svc/prometheus 9090:9090 → http://localhost:9090
Grafanaistio-systemkubectl port-forward -n istio-system svc/grafana 3000:3000 → http://localhost:3000
Lokiistio-systemkubectl port-forward -n istio-system svc/clawql-loki 3100:3100 → API 3100 (Grafana datasource http://clawql-loki:3100)
Tempoistio-systemkubectl port-forward -n istio-system svc/clawql-tempo 3200:3200 → Grafana Tempo datasource http://clawql-tempo:3200
Kialiistio-systemkubectl port-forward -n istio-system svc/kiali 20001:20001 → http://localhost:20001/kiali
OTel Collector (ClawQL)istio-systemIn-cluster DNS clawql-otel-collector.istio-system.svc ports 4317 / 4318 (no browser UI)
ClawQL MCP metricsclawqlGET /metrics on the MCP HTTP port (OpenMetrics), separate from mesh Prometheus

Use kubectl --context docker-desktop if your default context is not Docker Desktop.

Prometheus — getting started (new users)

Idea: a time-series database plus a query UI. You write PromQL to ask questions like “how many requests per second?”

  1. Run the port-forward from the table above.
  2. Open http://localhost:9090.
  3. Run the query up — confirms scrape targets are reachable.
  4. Try istio_requests_total after you generate traffic to the mesh (for example MCP or the docs UI).

Grafana — getting started (new users)

Idea: dashboards on top of metrics. Grafana reads Prometheus (and other sources) and draws charts.

  1. Port-forward 3000 (see table).
  2. Open http://localhost:3000.
  3. Log in with the Istio sample defaults: admin / admin (change this outside a throwaway lab).
  4. Open Dashboards and pick an Istio dashboard folder, then generate traffic and watch panels update.
  5. Add data sources (Connections → Data sources): Loki at http://clawql-loki:3100, Tempo at http://clawql-tempo:3200 when those Services exist (default heavy install includes Tempo; Loki when CLAWQL_ISTIO_INSTALL_LOKI_TEMPO=1).

Loki — getting started (new users)

Idea: log aggregation — label selectors + log lines. ClawQL’s MCP audit tool can POST JSON lines here (CLAWQL_LOKI_PUSH_URL) — see Audit tool & observability.

  1. Port-forward 3100 to svc/clawql-loki (see table).
  2. From Grafana → Explore, pick the Loki data source and run LogQL (for example {job="clawql-audit"} if you use the default push labels).

Tempo — getting started (new users)

Idea: distributed tracing backend paired with Grafana — search and visualize traces in Explore.

  1. Port-forward 3200 to svc/clawql-tempo (see table).
  2. In Grafana, add Tempo → URL http://clawql-tempo:3200, then Explore → Tempo.

ClawQL MCP spans: set CLAWQL_ENABLE_OTEL_TRACING=1 and OTEL_EXPORTER_OTLP_ENDPOINT=http://clawql-otel-collector.istio-system.svc:4318/v1/traces on clawql-mcp-http (Helm extraEnv or equivalent). See commented block in charts/clawql-mcp/values-docker-desktop.yaml.

Kiali — getting started (new users)

Idea: Istio console — who talks to whom, traffic rates, and config health.

  1. Port-forward 20001 (see table).
  2. Open http://localhost:20001/kiali.
  3. Go to Graph, namespace clawql, and explore the service graph.
  4. Click a service for details and links to related signals.

OpenTelemetry Collector — getting started (new users)

Idea: a small relay: apps send OTLP; the collector forwards to Tempo over OTLP gRPC. You usually do not open a browser for it.

  1. Confirm it exists: kubectl -n istio-system get deploy,svc clawql-otel-collector.
  2. Point the MCP pod at http://clawql-otel-collector.istio-system.svc:4318/v1/traces (HTTP OTLP) as above.
  3. Confirm traces in Grafana → Explore → Tempo after calling MCP tools.

ClawQL metrics vs mesh metrics

  • GET /metrics on clawql-mcp-http = application OpenMetrics (native protocol gauges/counters, etc.) — see Deployment.
  • Prometheus in istio-system = mesh and platform scrapes (Istio sample config). Different scrape targets, same Prometheus query language.

See also