Recorded before it is dispatched
Every call the hosted gateway admits is accepted by the audit pipeline before it goes to your provider. If the audit pipeline cannot accept it, the call does not go out.
How the ledger worksOPEN SOURCE INFRASTRUCTURE / AI AGENTS / BUILT EU-FIRST
The tamper-evident proxy
for AI agents.
Run agents in production without losing the audit trail. Every gateway call is recorded before dispatch and verifies offline.
No credit card · Apache-2.0 · Self-host · 0% markup · Hosted in Germany
SHA-256 Hash chainEd25519 Signed Merkle root205 ProvidersTwo Rust services — gateway and ingest — with ClickHouse and NATS.
205 providers, one canonical wire format
OpenAI
Anthropic
Google
Mistral
Llama
Bedrock
Groq
205 providers
What the runtime proxy does
Verifiable without trusting us
SHA-256 hash-chained, Ed25519-signed and anchored to Sigstore's public log. Three open verifiers, in Rust, Python and TypeScript, check it offline.
Security postureEvery attempt, and the model that answered
Each retry and failover hop is on the trace with its status and timing, and every call records the model you asked for next to the one the provider says served it.
What a trace shows2 ms added at p50, on your own keys
Measured with LiteLLM's open AIGatewayBench, full table published. 205 providers through one endpoint, your keys, 0% markup.
The benchmark, method and caveatsTHE REQUEST PATH
From admitted call
to verifiable record.
Hover or tap a step to see it recorded.
THE AUDIT LEDGERHash-chained
Model callsModel · tokens · cost
Tool callsRecorded metadata
Signed Merkle rootEd25519
A record you can inspect. A chain you can verify offline.
The problem
Agents fail without crashing.
State drift compounds
A 200 doesn't mean it worked
Every tool call is code running on your behalf
How it works
Record. Replay. Prove.
Two Rust services. Open core, Apache 2.0.
Record
Metadata on every call; prompt and response capture when enabled for your workspace. Your own provider keys.
Replay
Inspect the recorded calls to find what broke. Replay reads stored spans; it does not re-execute models or tools.
Prove
Hash-chained, signed, verifiable offline.
Built on the record
Failures, loops and cost spikes, read from the record.
Read from what was recorded. The analysis runs when you read, not on a model call's request path.
failed run → test case
From a failed trace to a regression fixture.
Each failed trace shows the failing signal and what changed since that agent's last good run: model, tools, environment, release. Export it as a dataset or Promptfoo config.
Record outcomes yourself with POST /v1/outcomes; a 200 is never read as success.
Limits: replays the recorded call (no tool runs, no live re-run); you write the assertion. Needs prompt capture and a plan with datasets.
loops and rescues
Repeated tool calls, and the requests failover or retry saved.
Flags the same tool called with the same arguments 3+ times in 5 minutes, per session or trace, with what the repeats cost. Works with prompt capture off.
Also counts requests that failover or retry saved.
Limits: detection only, never blocks. Calls need a session id or trace to be grouped. Self-host needs TRACELANE_APIKEY_PEPPER. Rescue counts start 2026-09-19.
spend spikes
When spend jumps, see who carried it.
Flags an hour costing 3× its weekly median, or a day 2.5× its two-week median, above a minimum spend. Then shows the top contributors by model, key, provider, user or tag.
Limits: top contributors, not a proven cause. Flagged on the chart only; no alerts are sent.
Measured, not claimed
2 ms added at p50, 4 ms at p95, 5 ms at p99.
Added latency with span capture on, measured with LiteLLM’s AIGatewayBench and our patches. Method
| Gateway | Version | p50 | p95 | p99 | Peak RSS |
|---|---|---|---|---|---|
| Tracelane | 1f3af9ad | 2 ms | 4 ms | 5 ms | 14 MB |
| Bifrost | v2.0.0 | 2 ms | 2 ms | 3 ms | 101 MB |
| Portkey (OSS) | 1.15.2 | 3 ms (run 1: 4 ms) | 5 ms (run 1: 7 ms) | 8 ms (run 1: 9 ms) | 177 MB (run 1; run 2 not sampled) |
| LiteLLM Python | 1.100.0 | 14 ms (run 1: 15 ms) | 34 ms (run 1: 36 ms) | 47 ms (run 1: 52 ms) | 323 MB |
| LiteLLM Rust | not measured | not measured | not measured | not measured | not measured |
Peak memory (RSS), gateway process only
Tracelane 14 MBBifrost 101 MBPortkey (OSS) 177 MB (run 1; run 2 not sampled)LiteLLM Python 323 MB
Method and caveats
Direct-to-mock baseline, already subtracted: 1 / 2 / 3 ms p50/p95/p99.
† Bifrost version caveat; ‡ LiteLLM Rust not measured.
Method
bash bench/gateway/aigatewaybench/run.sh Full method, caveats and the raw CSVs How we compare
Thirteen claims, sourced.
Against each vendor's own docs and pricing pages, fetched 2026-09-07. A dash means we didn't find it documented, not that the vendor lacks it.
| Capability | Tracelane | LiteLLM | Portkey | Langfuse | LangSmith |
|---|---|---|---|---|---|
| Measured gateway overhead (p50) | ✓ 2 ms † | ◐ Python proxy 14–15 ms, same bench; Rust build not public | ◐ 3–4 ms, same bench run | — no gateway | ◐ gateway not benchmarkable; no self-hostable runtime documented |
| Rust hot path | ✓ | — Python proxy measured; Rust gateway announced, no public build | — TypeScript/Node.js | — no gateway | — no gateway |
| BYOK, 0% token markup | ✓ | ✓ | ✓ | — | ◐ lists an LLM Gateway; BYOK not documented |
| OTel-native ingest | ✓ | ◐ partial | ◐ partial | ✓ | ◐ partial |
| Multi-agent swimlanes | ✓ | — | — | ✓ | ✓ |
| Sessions | ✓ per-user identity (x-tracelane-user-id) | — | — | ✓ | ✓ |
| Eval loop — datasets, experiments, online evals, annotation queues, CI gate | ✓ | — | — | ✓ | ✓ |
| Prompt management + playground | ✓ | — | ✓ | ✓ | ✓ |
| MCP server for traces | ✓ on npm, read-only, tenant-scoped | — | — | — | — |
| Claude Code flight recorder | ✓ | — | — | — | — |
| Tamper-evident ledger, offline-verifiable | ✓ | — | — | — | — |
| Self-host, single Compose file | ✓ | ✓ | ✓ | ✓ + Kubernetes | ◐ Enterprise BYOC |
| License | ✓ Apache 2.0 | ◐ MIT (core) | ✓ MIT | ✓ MIT | — none for OSS; Enterprise self-hosted SKU is proprietary |
shipped and documented · supported with a real limit · not documented on the vendor's own page as of 2026-09-07
† Benchmark: 2 ms p50 / 4 ms p95 / 5 ms p99 added latency on LiteLLM's own AIGatewayBench harness (4 vCPU, two runs, capture on) — full method and raw CSVs.
Not yet: SOC 2 report, configurable data residency, HIPAA/BAA, documented SAML SSO, contractual SLA, documented Slack/PagerDuty alert integrations.
The audit ledger
Gateway calls, on the record. Verifiable offline.
SHA-256 hash-chained, Ed25519-signed — verify your ledger offline with the open verifier.
Ledger: gateway-admitted calls and recorded guardrail verdicts. SDK/OTLP-only spans are captured, not chained. Details
# verify any trace offline — no trust in us required
$ tlane verify ./audit.ndjson --tenant-pubkey <b64>
✓ tlane verify: PASS
✓ hash_chain_valid: true · signatures_valid: true
✓ anchors_included: 12 (Sigstore Rekor v2)
# anchoring is per-batch and best-effort; coverage is partial; an unanchored root still verifies EU AI Act, India DPDP and HIPAA — what the ledger does and does not cover
eu ai act
Article 12 export: in preview, not yet sold; no per-record inclusion proofs or signed completeness attestation yet. Details
india dpdp
Primary storage: EU (Germany); no per-tenant region enforcement. Ledger kept for its lifetime. Details
hipaa
No signed BAA — do not route PHI through the hosted gateway. Redaction: structured IDs (e.g. E.164), not clinical text. Details
Migration
Moving off a tool that changed hands or changed its terms?
One command rewrites your config. Nothing leaves your infrastructure.
# rewrites HELICONE_* env vars, base URLs and custom
# headers across your source tree — printed as a diff,
# dry-run by default
$ tlane migrate helicone# reads litellm_config.yaml and prints your providers,
# routing and callbacks translated into Tracelane terms for review
$ tlane import-litellmPricing
Pricing.
Six meters, priced the same way on every paid tier. Self-host free forever.
OSS self-host
$0
- Full Rust gateway · 199 catalogued OpenAI-compatible providers, plus native adapters
- ClickHouse + NATS JetStream storage on your own node
- Hash-chained audit log with Ed25519 batch signatures — Rekor anchoring and the durable, restart-safe ledger need the Postgres control plane
- Community support on GitHub Discussions
Free
$0
- 1 GB ingest included / mo
- 60 RPM
- No credit card · never cut off by billing state
Builder
$29/mo
- 20 GB ingest included / mo
- 600 RPM
recommended
Team
$229/mo
- 100 GB ingest included / mo
- 6,000 RPM
- WorkOS SSO
Business
$799/mo
- 200 GB ingest included / mo
- 60,000 RPM
- WorkOS SSO
Enterprise
from $2,499/mo
- custom ingest included / mo
- Uncapped RPM
- WorkOS SSO
- 7-year ledger retention (included)
Every workspace starts on Free — no card. Upgrade to a paid plan inside the app when you need it.
Security posture
Built for procurement reviews.
release integrity
Sigstore Cosign keyless signing with attested build provenance. Trusted Publishing OIDC — no long-lived registry tokens.
sbom
CycloneDX SBOM published with every release, verifiable from GitHub release artifacts.
vuln scanning
Secret scans on non-draft pull requests and public pushes; dependency vulnerability scans nightly; a weekly public-repository SBOM. Details
pre-flight policy
Anthropic Messages: schema-violation and tool-definition-drift detection, fail-open, not enforcement. Details
transport
TLS 1.3 edge to origin; public edge accepts TLS 1.2. Ingest receiver uses SPIFFE/SPIRE mTLS. Details
standards
OTel GenAI semconv — tracking the pre-execution judgment / negative-proof proposal (#3244).
Tracelane is a member of NVIDIA Inception.
© 2026 NVIDIA, the NVIDIA logo, and NVIDIA Inception are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries.
FAQ
Frequently asked.
What is Tracelane?
Apache 2.0 Rust gateway, OTel observability and audit ledger. Events are SHA-256 hash-chained; every 100 events per tenant, the Merkle root is Ed25519-signed. OTLP/SDK traces are not chained. Details
How is it different from Helicone, LangSmith or Langfuse?
Gateway, observability, pre-flight policy and a tamper-evident ledger in one stack. Apache 2.0 core with a self-host path. Details
What about LangSmith Engine?
LangSmith Engine (May 2026) clusters failures and drafts fixes. Tracelane flags failure modes at the gateway, with neutral OTel ingest and 0% BYOK markup across 205 providers. Details
Is Tracelane really open source?
The core is Apache 2.0. The AFT-1 spec at /spec/aft-1 is CC0. Paid features are gated at the application layer. Details
Which providers are supported?
205 providers: 6 native adapters plus 199 catalogued OpenAI-compatible providers. Self-hosters can set their own base URL. Details
Does Tracelane support EU AI Act Article 12?
Ledger on every tier; the Article 12 export is in preview, not yet sold, and lacks per-record inclusion proofs and a signed completeness attestation. No per-tenant region enforcement, signed BAA or regulatory-conformance claim. Details
What's the latency overhead?
Measured added latency, excluding provider time: 2ms p50, 4ms p95, 5ms p99, capture on. Breakdown Method
Can I move over from Helicone or LiteLLM?
Yes: tlane migrate helicone rewrites your Helicone config as a dry-run diff, and tlane import-litellm translates a litellm_config.yaml for review.
Can I self-host?
Docker Compose runs gateway, ingest, NATS and ClickHouse on local disk; S3/R2 cold storage is a by-hand step. Helm is on the roadmap. Self-hosting guide
Is Tracelane available now?
Yes — V1 is live. Start free at app.tracelane.dev, or self-host the Apache 2.0 core.
Stay close
Get product updates.
One email when something ships — no drip, no list sharing. Reply to any message and you are removed.