# Devtool Arena - API Leaderboard

74 APIs evaluated across 15 categories.

**Grades:** A (90-100) Agent-Ready | B (75-89) Friendly | C (60-74) Difficult | D (<60) Hostile

## Inference
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Groq](/leaderboard/groq) | 22 | D | $0.32 | 5 |
| [OpenRouter](/leaderboard/openrouter) | 21 | D | $0.12 | 1 |
| [Fireworks AI](/leaderboard/fireworks-ai) | 19 | D | $0.25 | 4 |
| [TogetherAI](/leaderboard/togetherai) | 0 | D | $— | 0 |
| [Perplexity](/leaderboard/perplexity) | 0 | D | $— | 0 |
| [Cerebras](/leaderboard/cerebras) | 0 | D | $— | 0 |
| [DeepInfra](/leaderboard/deepinfra) | 0 | D | $— | 0 |
| [SambaNova](/leaderboard/sambanova) | 0 | D | $— | 0 |

## Auth
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [WorkOS](/leaderboard/workos) | 74 | C | $0.32 | 1 |
| [Auth0](/leaderboard/auth0) | 61 | C | $0.24 | 2 |
| [Descope](/leaderboard/descope) | 61 | C | $0.41 | 3 |
| [Scalekit](/leaderboard/scalekit) | 23 | D | $0.00 | 1 |
| [Clerk](/leaderboard/clerk) | 0 | D | $— | 0 |
| [Stytch](/leaderboard/stytch) | 0 | D | $— | 0 |

## Payment
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Stripe](/leaderboard/stripe) | 74 | C | $0.33 | 3 |
| [Square](/leaderboard/square) | 0 | D | $— | 0 |
| [Adyen](/leaderboard/adyen) | 0 | D | $— | 0 |
| [Razorpay](/leaderboard/razorpay) | 0 | D | $— | 0 |
| [Mollie](/leaderboard/mollie) | 0 | D | $— | 0 |
| [Paddle](/leaderboard/paddle) | 0 | D | $— | 0 |
| [Chargebee](/leaderboard/chargebee) | 0 | D | $— | 0 |
| [LemonSqueezy](/leaderboard/lemonsqueezy) | 0 | D | $— | 0 |
| [Airwallex](/leaderboard/airwallex) | 0 | D | $— | 0 |
| [PayPal](/leaderboard/paypal) | 0 | D | $— | 0 |

## Search
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Firecrawl](/leaderboard/firecrawl) | 80 | B | $0.10 | 0 |
| [Jina AI](/leaderboard/jina-ai) | 79 | B | $0.14 | 0 |
| [Tavily](/leaderboard/tavily) | 78 | B | $0.15 | 0 |
| [Exa](/leaderboard/exa) | 0 | D | $— | 0 |
| [You.com](/leaderboard/you-com) | 0 | D | $— | 0 |
| [Brave Search](/leaderboard/brave-search) | 0 | D | $— | 0 |

## Voice
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [LiveKit](/leaderboard/livekit) | 83 | B | $0.12 | 0 |
| [AssemblyAI](/leaderboard/assemblyai) | 68 | C | $0.27 | 3 |
| [ElevenLabs](/leaderboard/elevenlabs) | 61 | C | $0.19 | 2 |
| [Rime](/leaderboard/rime) | 53 | D | $0.45 | 4 |
| [Cartesia](/leaderboard/cartesia) | 26 | D | $0.57 | 7 |
| [Deepgram](/leaderboard/deepgram) | 0 | D | $— | 0 |
| [OpenAI Whisper](/leaderboard/openai-whisper) | 0 | D | $— | 0 |

## Sandboxes
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Vercel](/leaderboard/vercel) | 77 | B | $0.35 | 0 |
| [Sprites](/leaderboard/sprites) | 74 | C | $0.13 | 0 |
| [E2B](/leaderboard/e2b) | 72 | C | $0.15 | 0 |
| [Modal](/leaderboard/modal) | 64 | C | $0.21 | 1 |
| [Cloudflare Workers](/leaderboard/cloudflare-workers) | 63 | C | $0.00 | 0 |
| [Daytona](/leaderboard/daytona) | 28 | D | $0.26 | 4 |

## Vector Databases
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Pinecone](/leaderboard/pinecone) | 68 | C | $0.58 | 4 |
| [Zilliz Cloud](/leaderboard/zilliz-cloud) | 26 | D | $— | 0 |
| [Chroma](/leaderboard/chroma) | 19 | D | $0.93 | 6 |
| [Weaviate](/leaderboard/weaviate) | 15 | D | $— | 0 |
| [Qdrant](/leaderboard/qdrant) | 0 | D | $— | 0 |
| [LanceDB](/leaderboard/lancedb) | 0 | D | $— | 0 |

## Stablecoin
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Coinbase Payments](/leaderboard/coinbase-payments) | 0 | D | $— | 0 |
| [Circle](/leaderboard/circle) | 0 | D | $— | 0 |
| [Fireblocks](/leaderboard/fireblocks) | 0 | D | $— | 0 |
| [Triple-A](/leaderboard/triple-a) | 0 | D | $— | 0 |
| [DFNS](/leaderboard/dfns) | 0 | D | $— | 0 |

## Meeting Bot
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Recall.ai](/leaderboard/recall-ai) | 0 | D | $— | 0 |
| [Meeting BaaS](/leaderboard/meeting-baas) | 0 | D | $— | 0 |
| [MeetGeek](/leaderboard/meetgeek) | 0 | D | $— | 0 |
| [Meetstream](/leaderboard/meetstream) | 0 | D | $— | 0 |
| [CueMeet](/leaderboard/cuemeet) | 0 | D | $— | 0 |

## Durable Workflow
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Temporal](/leaderboard/temporal) | 65 | C | $0.24 | 0 |
| [Camunda](/leaderboard/camunda) | 59 | C | $0.53 | 5 |
| [Prefect](/leaderboard/prefect) | 0 | D | $— | 0 |
| [Cadence](/leaderboard/cadence) | 0 | D | $— | 0 |
| [Netflix Conductor](/leaderboard/netflix-conductor) | 0 | D | $— | 0 |
| [AWS Step Functions](/leaderboard/aws-step-functions) | 0 | D | $— | 0 |
| [Akka](/leaderboard/akka) | 0 | D | $— | 0 |

## Cloud Hosting
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Netlify](/leaderboard/netlify) | 66 | C | $0.29 | 2 |
| [Render](/leaderboard/render) | 34 | D | $0.68 | 9 |
| [Railway](/leaderboard/railway) | 33 | D | $0.64 | 6 |
| [DigitalOcean](/leaderboard/digitalocean) | 26 | D | $— | 0 |

## Observability
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Datadog](/leaderboard/datadog) | 72 | C | $0.18 | 2 |

## Email
| Provider | Score | Grade | Cost | Errors |
|----------|-------|-------|------|--------|
| [Resend](/leaderboard/resend) | 79 | B | $0.18 | 1 |
| [Agentmail](/leaderboard/agentmail) | 71 | C | $0.20 | 2 |