Must Have
- Intelligent routing
- Broader provider support: Oracle model configuration via environment variables, plus Cohere, Command A, Operational, and DeepSeek V3
- Budget management with limits per
user_pathand/or API key - Editable model pricing for accurate cost tracking and budgeting
- Full support for the OpenAI
/responsesand/conversationslifecycle - Prompt cache visibility showing how much of each prompt was cached by the provider
- Guardrails hardening: better UI, simpler architecture, easier custom guardrails, and response-side guardrails before output reaches the client
- Passthrough for all providers, beyond the current OpenAI and Anthropic beta
Should Have
- Cluster mode