Everything the gateway does today, grouped by what you need it for. All of it ships in the single open-source binary - only the Pro cards at the end are paid.
Routing & Providers
OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Ollama, vLLM, and more behind one endpoint - with round-robin rotation across multiple keys per provider.
Publish stable names like smart-chat and remap the real provider and model behind them - a config change, not an application change.
Spread a virtual model across targets with weighted round-robin, or let cost-based routing pick the cheapest capable model for each request.
Availability errors fail over to the next model or provider; retries with backoff and a circuit breaker absorb flaky upstreams.
Call any provider's native API through /p/:provider/* and keep GoModel's auth, usage tracking, and audit on the way through.
Control & Safety
Toggle cache, audit, usage, budgets, guardrails, and failover per provider, model, or user path - versioned, most specific scope wins.
Inject system prompts or rewrite messages with an LLM before dispatch, in ordered steps that run as parallel groups.
Hand teams managed keys bound to a user path and labels instead of raw provider credentials - revoke and rotate from the admin UI.
Cap request rate and concurrency per user path, provider, or model - saturated routes are routed around when alternatives exist, 429 + Retry-After when not.
Cost
Hard spend limits per user path or label, evaluated from tracked usage cost and enforced before a request is dispatched - the run stops at the cap, not at the invoice.
Exact-match caching out of the box, plus semantic caching backed by Qdrant, pgvector, Pinecone, or Weaviate.
Token and dollar accounting per request, user path, and label - with per-model pricing overrides when list prices don't match your contract.
Observability
Every request with its resolved route, workflow, cache result, and provider attempts - bodies and headers logged only when explicitly enabled.
Embedded UI for live request logs, usage breakdowns, keys, budgets, workflows, and provider status - no separate deployment to run.
Labels from headers or key metadata flow into usage and audit, so spend and incidents map to teams, tenants, and features.
/metrics with request, provider, and circuit-breaker gauges, plus health endpoints and optional pprof profiling.
OTLP traces and metrics for every inbound request and provider call, on the GenAI semantic conventions - Jaeger, Tempo, Honeycomb, or Datadog read them as they are.
Beyond Chat
Chat, embeddings, the Responses API with gateway-managed conversations, files, and batches - not just /v1/chat/completions.
Native /v1/messages and token counting - point the Anthropic SDK at GoModel and route to any provider behind it.
Text-to-speech, transcription, and realtime speech over WebSocket and WebRTC, through the same gateway pipeline.
Aggregate MCP servers behind one endpoint with namespaced tools - every tool call gets usage tracking and audit like any other request.
Send a real request from the dashboard against any model or alias, streaming or not, and read the exact JSON both ways - routed, logged, and metered like any client call.
Run It Anywhere
A single Go binary with Docker, Compose, and Helm recipes - the admin UI is embedded, so there is nothing else to deploy.
Start on SQLite with zero setup, move to PostgreSQL or MongoDB when traffic and retention demand it - same binary, different config.
Requests from one conversation or agent task stick to the target and key that served the first - warm provider prompt caches, threaded audit logs, per-session cost. Zero config.
SSE streaming with usage and audit recorded from the stream itself - no buffering, no lost accounting on long generations.
GoModel Pro
Deterministic rewrites remove 2-20% of input tokens on agentic traffic - nothing summarized, nothing dropped, and prompt caches keep hitting.
Pro · BetaScores each request into an easy or hard model tier, then picks the healthiest, cheapest provider inside it - every verdict explainable.
ProOkta, Entra ID, Google, Auth0, or Cognito in front of the dashboard - identities map onto your user-path policies, every login audited.
ProOne budget or rate-limit rule on a parent path gives every child its own counters - per-tenant isolation without pre-provisioning.
The same gateway, six different jobs.
A multi-tenant SaaS issues an API key per customer, tracks usage by user path, and enforces per-tenant budgets - billing numbers come from the dashboard, not guesswork.
A platform team publishes aliases like smart-chat with scoped workflows behind them - product teams ship features without ever holding provider keys.
Production traffic rides failover chains with retries and circuit breakers - a provider incident becomes a routing event, not a customer-facing one.
Caching absorbs duplicate prompts, cost-based routing picks the cheapest capable model, and budgets stop end-of-month surprises.
Compliance reviews replay any request: resolved route, guardrail versions, provider attempts, and full bodies where logging is explicitly enabled.
Developers run Ollama or vLLM locally behind the same endpoint the cloud providers serve in production - switching is config, not code.