Gateway overhead, measured
See exactly what the gateway costs you in latency — 2 ms at the median, 5 ms at the 99th — measured on LiteLLM's open benchmark harness rather than one we wrote.
changelog
Everything shipped,
since day one.
Five months of continuous shipping — aggregated, sourced and labelled with every production proof on record.
66
changes shipped
every entry below, May 2026 to today
5 months
of continuous shipping
May to Sep 2026
2 / 4 / 5 ms
gateway overhead, p50 / p95 / p99
AIGatewayBench, 2026-09-07, two runs on a 4-vCPU host
14 MB
peak memory in that run
same run; the next-lightest gateway measured 101 MB
150+
providers, one endpoint
the compiled provider catalogue, held equal to this claim by a repo check
| Month | Entries |
|---|---|
| May 2026 | 14 |
| Jun 2026 | 10 |
| Jul 2026 | 14 |
| Aug 2026 | 12 |
| Sep 2026 | 16 |
entries per month · Sep to 7th
No entries match this filter — show all.
16
Nothing under this heading yet — try All .
OpenAI tool calling, end to end
Run an agent that calls tools through the gateway with the OpenAI SDK unchanged — real tool calls come back, the finish reason is the true one, and you can replay the model's own tool call in the next turn.
Dashboards you build yourself
Assemble your own dashboard from a palette of metric tiles, then resize, move and remove them — the layout persists across a reload.
Truncated answers are never cached
Ask again after a reply was cut short by a token limit and you get a fresh, complete answer instead of the truncated one served back from cache.
Self-hosting runs unmetered
Run the gateway on your own hardware with no control plane and it no longer applies the hosted free tier's request limit to your traffic.
Docs →Anthropic-native endpoint
Point Claude Code, or any Anthropic-native client, straight at the gateway and get full tracing, token counts and cost with no OpenAI-shape translation in between.
Ask your traces a question
Type a question about your own runs in plain English and get an answer, instead of writing the query yourself.
Multi-agent runs, as swim lanes
Read a run with sub-agents as one labelled lane per agent, with a failures-only filter, rather than as a single flat timeline.
Shareable trace links
Send someone a link to one trace, carrying its ledger badge, that they can open without an account — and revoke it when you are done.
Search the trace list
Find a trace by span name or content from a search box on the traces page, instead of paging until you spot it.
A playground inside the app
Try a prompt against a provider you have connected and land straight on the trace it produced.
Claude Code sessions record fully
See real token counts, computed cost, tool names and session grouping for a Claude Code session — all four were previously blank.
MCP for hosted workspaces
Query your own traces from any MCP client using your bearer token — this previously needed a self-hosted deployment with a direct database connection.
Docs →Your trace ids survive the gateway
Call the gateway from your own instrumented app and the two halves join into one trace, instead of appearing as two disconnected ones.
Line and area charts
Read latency percentiles, availability and error rate as a continuous line rather than as bars, which is the right shape for a level instead of a count.
One time range for the dashboard
Move one control and every chart on the dashboard answers to it, drawing a real interactive chart — several displays used to disagree with each other.
12
Nothing under this heading yet — try All .
Guides for the agent frameworks
Get traces out of LangChain, LangGraph, CrewAI or LlamaIndex by pointing their existing OpenTelemetry exporter at us — no Tracelane-specific code, and each guide states exactly what you get.
Docs →An eval gate for your CI
Run a dataset through your prompt on every pull request and fail the build when the score falls below a floor you set, with a separate exit code for "it got worse" and "we could not measure it".
Docs →Review queues for failures
Route a trace an automated judge scored as failing to a review queue, write the correct answer by hand, and keep it as a reusable test case.
Score live traffic automatically
Have an automated judge score a share of your production traffic against a rubric you choose, at a sample rate you set, under a spend limit that survives a restart.
Nine ways to assert an eval
Assert on contains, regex, JSON schema, an LLM judge, and cost and latency ceilings through the API — and a judge answer that does not match its schema errors instead of quietly scoring.
Docs →Datasets that persist
Save a collection of test cases and have it still be there afterwards — every dataset write had been failing silently.
Repeat requests served from cache
Send the same request twice and the second is answered from the gateway's own cache instead of the provider, with a response header naming the tier that served it.
Failover honours your config
Set a fallback provider in `tracelane.yaml` and it is actually used — the setting had been read and then ignored.
Send OpenTelemetry traces directly
Export a nested agent trace straight from your own instrumentation with your API key, and see the whole tree — planner step, each tool call, the retry — without proxying the model call.
Docs →Compare two traces
Open two runs side by side and see what differs between them.
Three permission gaps closed
A viewer can no longer promote or delete a production prompt, SSO and directory-sync setup is limited to the plan that includes it, and a billing-portal return address is checked against an allowlist.
Releases you can verify
Check any release binary yourself — 0.2.3 is the first to publish a Cosign keyless signature and a CycloneDX SBOM per artifact, with build provenance attested by GitHub.
Docs →14
Nothing under this heading yet — try All .
Gemini through Vertex AI
Route Gemini models via Google's Vertex AI endpoint with your own project credentials.
Docs →Self-host without a certificate authority
Run a single-tenant deployment without standing up SPIRE first, which removes the heaviest prerequisite from the self-host path.
Docs →Verify your chain on any plan
Check your workspace's hash chain from inside the product without paying for the audit add-on; the exportable, offline-verifiable bundle stays part of that add-on.
Docs →Slice the trace list
Filter, group by model, operation or status, sort by any column, set one date range for the page, and export what you are looking at to CSV or JSON.
Anchored to a public log
Prove to a third party that a batch of your records existed when you say it did — the first Merkle root reached the public Sigstore transparency log at index 19398597, and anyone can look it up without asking us.
Docs →Failure signatures from your traces
See the failure shapes actually detected in your own runs, each with a first-seen and last-seen, rather than a catalogue of failures in the abstract.
Offline verification, three languages
Hand an exported bundle to someone with no network access and let them verify it in Rust, Python or TypeScript against your own public key, served from a public endpoint so the trust root is not us.
Docs →Promotions are signed records
Promote or roll back a prompt and the decision is appended to the tamper-evident chain as a signed verdict you can hand to someone else.
Docs →Alerts you configure and test
Write your own alert rules, send them to Slack, Discord or an HTTP endpoint, and fire a test to confirm the route works before you rely on it.
Teams, roles and seats
Invite members, assign roles, stay inside the seat count your plan includes, and delete your own account or organisation without asking us.
Real cost on every call
See the USD cost of a call on its span, in the trace list, and rolled into a spend card — computed from a model price catalogue rather than estimated.
Sessions group a multi-turn run
Follow a conversation across its turns as one session, with its own list and detail view, instead of as unrelated calls.
Prompts with durable versions
Author a prompt, keep its versions, and promote one to production from the app — rather than keeping the text in your application code.
Docs →A gateway operations page
Watch per-provider health, failover state, circuit-breaker state and quota status from one page, with nothing on it that is not a measured signal.
10
Nothing under this heading yet — try All .
Sampling and quotas per workspace
Set a sampling policy, an ingest quota and a per-trace ceiling for each workspace — and an over-quota batch is refused with the reason, never accepted and quietly dropped.
Spans acknowledged only after storage
A restart mid-flight replays your span instead of losing it, because the queue is only acknowledged once the write has landed.
Pre-flight policy at the gateway
Apply your own policy to a request before it leaves — cost and step caps, secret and PII patterns, tool pinning, response format and topic scope — and have it allowed, redacted or refused.
Docs →A transcript-spine trace viewer
Read a trace top to bottom as the conversation it was, with the span tree and an inspector beside it, and filter the list down to the run you are looking for.
Verify the ledger in your browser
Watch your own browser recompute the chain and report the chain and signature verdicts separately — a passing badge is something your machine worked out, not something we asserted.
Docs →Onboarding ends in your own trace
Finish setup by looking at the first trace your key actually produced, so the confirmation is your data rather than a checkmark.
Docs →A dropped span is loud
Know when capture is broken: a span the gateway cannot record is counted, warned once and reported on the health endpoint, and an unset queue address refuses to start at all.
Tracelane Cloud is live
Point a client at the hosted gateway and start capturing, instead of running the stack yourself first.
Docs →Cross-provider failover
Name a fallback provider and a failing upstream is retried there, rather than the request ending at the first error.
Tool-definition drift detection
Find out when a tool's description or input schema changes underneath a running agent — the silent rug-pull an agent has no other way to notice.
Docs →14
Nothing under this heading yet — try All .
Breakers, canaries and a kill switch
Give each upstream its own circuit breaker, split traffic to a candidate prompt version, and turn a predictor off by name without a deploy.
Ingest limits answered on the spot
Send an oversized batch, span or attribute and get a synchronous refusal with the reason in a response header — nothing is accepted and then discarded behind your back.
A verifier a regulator can run
Hand an auditor a single signed binary that verifies an exported ledger offline against a public key they pin themselves, and exits non-zero when the chain does not hold.
Docs →Pricing that states its limits
Read each plan's included volume, its per-unit overage with a hard cap, its seat ladder and its retention window on the page, instead of inferring them.
Docs →Keys hashed, ledgers signed per workspace
Your API keys are stored as Argon2id hashes behind a peppered lookup, and your ledger is signed with a key belonging to your workspace rather than a shared one.
Auto-instrumentation for Python and TypeScript
Install the SDK and it attaches to the agent frameworks and provider clients you already use, so spans arrive without hand-written instrumentation.
Docs →Traces, trace detail and SLO
Browse your traces, open one to a span tree with an inspector, and read latency percentiles, error rate and token usage on an SLO page.
Docs →Prompt promotion with rollback
Promote a prompt version to production atomically and have a drift detector roll it back — suggested, automatic or human-confirmed, your choice.
Docs →Three independent verifiers
Check an exported ledger in Rust, Python or TypeScript against shared conformance vectors, so whoever has to verify it can do so in a language they already run.
Docs →The gateway, in one binary
Point any OpenAI-compatible client at one endpoint and your call is routed to your provider with your own key, traced and recorded — a model matching no provider is refused rather than sent somewhere by default.
Docs →A tamper-evident audit ledger
Prove after the fact that your record was not altered: every gateway-proxied call and policy verdict is appended to a per-workspace hash chain, so a later deletion, insertion, reorder or edit is detectable by anyone holding an export.
Docs →OpenTelemetry-native capture
Take your traces elsewhere whenever you want — spans are written in the OpenTelemetry GenAI conventions rather than a private schema.
Docs →EU AI Act Article 12 export
Export the record-keeping pack Article 12 asks for, mapped obligation by obligation, and hand it to whoever has to check it.
Apache 2.0, no relicensing
Self-host it, fork it, or run the hosted version — the whole project is Apache 2.0 with a written no-relicensing pledge and no separate enterprise tree.