Every AI agent team eventually has the same conversation: "Why is the bill so much higher than we projected?"
The answer is almost always the same. You budgeted for the per-token price. You didn't budget for everything else: retries, context window waste, rate limit backoff delays, and the compounding effect of running agents autonomously — where there's no human in the loop to catch runaway costs until the invoice arrives.
This guide breaks down the actual numbers so you can stop estimating and start planning. All prices are per million tokens, USD, as of April 2026.
Get weekly AI cost optimization tips
Join engineers cutting their AI bills. No spam — just strategies that actually move the number.
Per-Model Pricing Comparison
Here's what OpenAI and Anthropic charge per token for their current flagship models:
| Model | Input Price | Output Price | Best For | Cost Efficiency |
|---|---|---|---|---|
| GPT-4o OpenAI | $2.50 / 1M tokens | $10.00 / 1M tokens | Complex reasoning, multi-step agents | ▲ Baseline |
| GPT-4o-mini OpenAI | $0.15 / 1M tokens | $0.60 / 1M tokens | Classification, extraction, formatting | ↓ 94% vs GPT-4o |
| GPT-3.5-turbo OpenAI | $0.50 / 1M tokens | $1.50 / 1M tokens | Simple pipelines, high-volume tasks | ↓ 85% vs GPT-4o |
| Claude 3.5 Sonnet Anthropic | $3.00 / 1M tokens | $15.00 / 1M tokens | Long-context tasks, coding, analysis | ▲ 20% more than GPT-4o |
| Claude 3.5 Haiku Anthropic | $0.80 / 1M tokens | $4.00 / 1M tokens | Fast classification, simple extraction | ↓ 60% vs Sonnet |
| Claude 3 Opus Anthropic | $15.00 / 1M tokens | $75.00 / 1M tokens | Highest capability, complex research | ▲ 6x vs GPT-4o |
What Most Teams Actually Pay: Hidden Cost Multipliers
The per-token price is where the planning starts. Here are the multipliers that turn a clean projection into a shocking invoice:
Retry Loops
Every API has transient failures. Without an exponential backoff strategy, your agents retry immediately and pile up charges on failed requests. A 1% error rate with no backoff on 100,000 daily calls means ~1,000 duplicate charges. Implement retry logic with jitter and cap max retries at 3. Without this, expect 3–8% of your total spend to be retries.
⚠ Adds 3–8% to your billContext Window Waste
If your agents pass the full conversation history on every API call, you're re-sending tokens you've already paid for. A 50-message conversation with 2,000 tokens of history sent 50 times wastes 98,000 tokens per session. Cache system prompts via prompt caching (OpenAI: 50% discount; Anthropic: 90% discount on cache reads). Audit your per-call token counts — anything over 30% redundant context is a leak.
⚠ Adds 15–40% to input costsRate Limit Backoff Delays
When you hit rate limits, your agent waits and retries. But depending on your SDK configuration, the retry call might re-send the full payload, burning another round of tokens before the limit clears. Check whether your client library retries on 429s with the same payload or with a lighter probe. The difference can be a 2x multiplier on rate-limited calls.
⚠ Doubles cost on rate-limited requestsVerbose Output Padding
Models default to comprehensive responses. Your agent may only use 40% of what the model returns. A 2,000-token response where you only use 800 tokens means 1,200 wasted output tokens at $10/1M — $0.012 per call. Scale that to 50,000 calls per day and you've paid $600/day for content your agent discarded. Set explicit max_tokens limits. Use output schema instructions to suppress prose wrapping.
⚠ 10–30% of output tokens often discardedStaging Environments Running Against Production Keys
One of the most common overspend incidents we see: a staging or test environment with the wrong API key, running 500 calls per minute against production APIs. Without per-environment spend tracking, this goes unnoticed until the monthly bill. Set up separate API keys per environment and configure spend alerts per key — not just per account.
⚠ Can add thousands per week undetectedMonthly Cost Projections: 10, 50, and 100 Agents
Here's where the math gets concrete. These projections use a realistic workload: 500 API calls per agent per day, 50% input / 50% output tokens, 800 input tokens + 300 output tokens per call.
| Fleet Size | GPT-4o | GPT-4o-mini | Claude 3.5 Sonnet | Claude 3.5 Haiku |
|---|---|---|---|---|
| 10 agents | $1,012/mo~5,000 calls/day | $61/mo~5,000 calls/day | $1,215/mo~5,000 calls/day | $324/mo~5,000 calls/day |
| 50 agents | $5,062/mo~25,000 calls/day | $303/mo~25,000 calls/day | $6,075/mo~25,000 calls/day | $1,620/mo~25,000 calls/day |
| 100 agents | $10,125/mo~50,000 calls/day | $607/mo~50,000 calls/day | $12,150/mo~50,000 calls/day | $3,240/mo~50,000 calls/day |
500 calls/agent/day × 800 input tokens × $2.50/1M + 300 output tokens × $10/1M × 30 days. Output tokens are roughly 4x the cost of input tokens on a per-token basis, so workloads heavy on generation cost more per call than workloads dominated by analysis. The 16x gap between GPT-4o-mini and GPT-4o represents the most impactful single decision in fleet cost management — routing the right tasks to the right model.
The key number here: 50 agents running GPT-4o costs ~$5,000/month. Swap GPT-4o for GPT-4o-mini on the 70% of calls where it's appropriate, and that drops to ~$1,500/month. That's $3,500/month in savings for a routing layer configuration change.
How Costs Compound at Autonomous Scale
Everything above is for teams where a human reviews agent outputs and can catch cost anomalies before they spiral. Autonomous agents are different. When there's no human in the loop:
- No cost ceiling. A human can stop an agent mid-run if costs look high. An autonomous loop runs until it hits an explicit limit.
- Context grows unbounded. Agents that maintain conversation state accumulate history tokens on every call. Without explicit truncation, a 30-day autonomous agent could be sending 100,000 tokens of context per call — all billed.
- Failure cascades amplify. If an agent has a bug that causes it to re-call itself (a recursive loop), the cost compounds linearly with no human to interrupt it.
- Volume scales unpredictably. An agent that handles 100 user requests on a good day might handle 10,000 on a viral day — with no corresponding budget update.
An autonomous 50-agent fleet running without spend monitoring is one bad deployment away from a five-figure monthly bill. We've seen it happen. The teams that don't get surprises have two things: per-key spend alerts and hard token budgets per agent.
This is why spend monitoring isn't a nice-to-have for autonomous agents — it's the thing that makes autonomous operation survivable. You need to know the moment a fleet exceeds a threshold, not at the end of the month.
Making the Provider Choice
For most agent workloads, the decision isn't which provider — it's which models across both providers. The cost difference between GPT-4o-mini and Claude 3.5 Haiku is modest enough that you should pick based on quality for your specific task type:
- Long document analysis, coding, structured extraction: Claude 3.5 Sonnet wins on context window and instruction following.
- Fast, high-volume classification and formatting: GPT-4o-mini or Claude 3.5 Haiku, pick based on your output quality requirements.
- Complex multi-step reasoning: GPT-4o and Claude 3.5 Sonnet are competitive; benchmark against your specific use case.
The 50-agent fleet cost table above shows why provider choice matters far less than model routing: the difference between GPT-4o and GPT-4o-mini is 16x, while the difference between OpenAI and Anthropic for comparable models is roughly 20–30%. Route correctly and you save more than you would by switching providers.
Want to calculate your specific fleet's monthly cost? Try the free AI cost calculator with your actual call volumes and token counts across all six major models.
For a deeper look at how these costs compound and what the optimization levers are, see 7 strategies that actually cut your AI agent bill — model routing, prompt caching, and batch processing are the big three.
The real cost of running AI agents goes beyond per-token pricing. The Real Cost of Running AI Agents in 2026 covers the hidden multipliers, monitoring gaps, and why most teams don't know their real number until the invoice lands.