Devtool CLI Arena
Benchmarking developer tools evaluated by 
Claude Code across auth, inference, voice, and more.
SapientClickTap any row to explore the full CLI eval.
Get your CLI graded →
| Rank | Tool | Score | Grade | Task | Eval | Discovery | Cost | Calls | Errors | Time | |
|---|---|---|---|---|---|---|---|---|---|---|---|
1🥇 | LiveKit | 83 | B | 80 | 93 | $0.12 | 12 | 0 | 1m 20s | View report | |
2🥈 | Firecrawl | 80 | B | 83 | 75 | $0.10 | 9 | 0 | 1m 37s | View report | |
3🥉 | Jina AI | 79 | B | 83 | 70 | $0.14 | 9 | 0 | 1m 40s | View report | |
3🥉 | Resend | 79 | B | 71 | 100 | $0.18 | 16 | 1 | 2m 34s | View report | |
5 | Tavily | 78 | B | 80 | 75 | $0.15 | 11 | 0 | 1m 30s | View report | |
5 | Daytona | 78 | B | 81 | 70 | $0.19 | 17 | 0 | 2m 19s | View report | |
7 | Deepgram | 77 | B | 70 | 95 | $0.34 | 12 | 1 | 2m 10s | View report | |
7 | Vercel | 77 | B | 77 | 78 | $0.35 | 20 | 0 | 3m 54s | View report | |
9 | WorkOS | 74 | C | 66 | 95 | $0.32 | 25 | 1 | 3m 6s | View report | |
9 | Sprites | 74 | C | 83 | 53 | $0.13 | 10 | 0 | 1m 45s | View report | |
9 | Stripe | 74 | C | 66 | 93 | $0.33 | 20 | 3 | 5m 14s | View report | |
12 | Datadog | 72 | C | 71 | 75 | $0.18 | 16 | 2 | 2m 7s | View report | |
12 | E2B | 72 | C | 78 | 60 | $0.15 | 11 | 0 | 2m 4s | View report | |
14 | Agentmail | 71 | C | 70 | 75 | $0.20 | 12 | 2 | 2m 3s | View report | |
15 | AssemblyAI | 68 | C | 68 | 70 | $0.27 | 21 | 3 | 2m 42s | View report | |
15 | Pinecone | 68 | C | 64 | 78 | $0.58 | 37 | 4 | 3m 50s | View report | |
17 | Netlify | 66 | C | 66 | 68 | $0.29 | 19 | 2 | 8m 49s | View report | |
18 | Temporal | 65 | C | 72 | 50 | $0.24 | 15 | 0 | 10m 28s | View report | |
19 | Modal | 64 | C | 70 | 50 | $0.21 | 13 | 1 | 2m 4s | View report | |
20 | Auth0 | 61 | C | 68 | 45 | $0.24 | 21 | 2 | 2m 47s | View report | |
20 | ElevenLabs | 61 | C | 70 | 40 | $0.19 | 18 | 2 | 3m 12s | View report | |
20 | Descope | 61 | C | 64 | 55 | $0.41 | 31 | 3 | 4m 52s | View report | |
23 | Camunda | 59 | C | 64 | 50 | $0.53 | 37 | 5 | 4m 5s | View report | |
24 | Rime | 53 | D | 68 | 20 | $0.45 | 28 | 4 | 6m 2s | View report | |
25 | Render | 34 | D | 7 | 100 | $0.68 | 42 | 9 | 5m 33s | View report | |
26 | Railway | 33 | D | 8 | 93 | $0.64 | 47 | 6 | 6m 4s | View report | |
27 | Pipecat | 31 | D | 40 | 13 | $0.00 | 0 | 0 | 49s | View report | |
28 | Cartesia | 26 | D | 14 | 55 | $0.57 | 36 | 7 | 4m 44s | View report | |
29 | Scalekit | 23 | D | 19 | 33 | $0.00 | 11 | 1 | 5m 20s | View report | |
30 | Groq | 22 | D | 9 | 55 | $0.32 | 25 | 5 | 3m 26s | View report | |
31 | OpenRouter | 21 | D | 15 | 35 | $0.12 | 9 | 1 | 2m 1s | View report | |
32 | Fireworks AI | 19 | D | 7 | 50 | $0.25 | 21 | 4 | 2m 38s | View report | |
32 | Chroma | 19 | D | 2 | 60 | $0.93 | 35 | 6 | 6m 29s | View report | |
34 | Zilliz Cloud Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
34 | DigitalOcean Signup requires a credit card | — | — | — | — | — | — | — | — | — | View report |
36 | Weaviate Requires Weaviate Cloud account — WEAVIATE_API_KEY and WEAVIATE_URL not provisioned | — | — | — | — | — | — | — | — | — | View report |
37 | DeepInfra API key requires a paid plan | — | — | — | — | — | — | — | — | — | View report |
37 | SambaNova No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Mollie No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Chargebee No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | LemonSqueezy No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 |  AWS Step Functions Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
37 | Brave Search API key requires a paid plan | — | — | — | — | — | — | — | — | — | View report |
37 | Qdrant No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Perplexity API key requires a paid plan | — | — | — | — | — | — | — | — | — | View report |
37 | Akka Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
37 | Cadence Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | — | — | — | View report |
37 | Circle No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | PayPal No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Cloudflare Workers No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Agora No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Triple-A Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
37 | Meetstream No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | DFNS Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
37 |  CueMeet Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | — | — | — | View report |
37 | Prefect No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | TogetherAI API key requires a paid plan | — | — | — | — | — | — | — | — | — | View report |
37 | OpenAI Whisper API key requires a paid plan | — | — | — | — | — | — | — | — | — | View report |
37 | Cerebras No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | MeetGeek No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | LanceDB No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Adyen API key requires official certification or company verification | — | — | — | — | — | — | — | — | — | View report |
37 | Recall.ai No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Airwallex API key requires official certification or company verification | — | — | — | — | — | — | — | — | — | View report |
37 | Clerk No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Netflix Conductor Open-source / self-hosted — no hosted API key available | — | — | — | — | — | — | — | — | — | View report |
37 | You.com No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Coinbase Payments No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Fireblocks Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
37 | Meeting BaaS No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Razorpay No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Square Could not obtain API key | — | — | — | — | — | — | — | — | — | View report |
37 | Exa No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Paddle No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
37 | Stytch No CLI tool available | — | — | — | — | — | — | — | — | — | View report |
Scores calculated from end-to-end CLI evaluations by AI coding agents.
Frequently asked questions
Join the mailing list
Get updates on leaderboard changes, new benchmark releases, and product announcements.