GoModel Features

Everything the gateway does today, grouped by what you need it for. All of it ships in the single open-source binary - only the Pro cards at the end are paid.

Routing & Providers

38 Providers, One API

OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Ollama, vLLM, and more behind one endpoint - with round-robin rotation across multiple keys per provider.

Aliases & Virtual Models

Publish stable names like smart-chat and remap the real provider and model behind them - a config change, not an application change.

Load Balancing

Spread a virtual model across targets with weighted round-robin, or let cost-based routing pick the cheapest capable model for each request.

Automatic Failover

Availability errors fail over to the next model or provider; retries with backoff and a circuit breaker absorb flaky upstreams.

Provider Passthrough

Call any provider's native API through /p/:provider/* and keep GoModel's auth, usage tracking, and audit on the way through.

Control & Safety

Scoped Workflows

Toggle cache, audit, usage, budgets, guardrails, and failover per provider, model, or user path - versioned, most specific scope wins.

Guardrails

Inject system prompts or rewrite messages with an LLM before dispatch, in ordered steps that run as parallel groups.

Virtual API Keys

Hand teams managed keys bound to a user path and labels instead of raw provider credentials - revoke and rotate from the admin UI.

Rate Limits

Cap request rate and concurrency per user path, provider, or model - saturated routes are routed around when alternatives exist, 429 + Retry-After when not.

Cost

Budgets

Hard spend limits per user path or label, evaluated from tracked usage cost and enforced before a request is dispatched - the run stops at the cap, not at the invoice.

Response Caching

Exact-match caching out of the box, plus semantic caching backed by Qdrant, pgvector, Pinecone, or Weaviate.

Usage & Cost Tracking

Token and dollar accounting per request, user path, and label - with per-model pricing overrides when list prices don't match your contract.

Observability

Audit Logs

Every request with its resolved route, workflow, cache result, and provider attempts - bodies and headers logged only when explicitly enabled.

Admin Dashboard

Embedded UI for live request logs, usage breakdowns, keys, budgets, workflows, and provider status - no separate deployment to run.

Request Tagging

Labels from headers or key metadata flow into usage and audit, so spend and incidents map to teams, tenants, and features.

Prometheus Metrics

/metrics with request, provider, and circuit-breaker gauges, plus health endpoints and optional pprof profiling.

OpenTelemetry Traces

OTLP traces and metrics for every inbound request and provider call, on the GenAI semantic conventions - Jaeger, Tempo, Honeycomb, or Datadog read them as they are.

Beyond Chat

Full OpenAI Surface

Chat, embeddings, the Responses API with gateway-managed conversations, files, and batches - not just /v1/chat/completions.

Anthropic Messages API

Native /v1/messages and token counting - point the Anthropic SDK at GoModel and route to any provider behind it.

Audio & Realtime

Text-to-speech, transcription, and realtime speech over WebSocket and WebRTC, through the same gateway pipeline.

MCP Gateway

Aggregate MCP servers behind one endpoint with namespaced tools - every tool call gets usage tracking and audit like any other request.

Built-in Playground

Send a real request from the dashboard against any model or alias, streaming or not, and read the exact JSON both ways - routed, logged, and metered like any client call.

Run It Anywhere

One Binary

A single Go binary with Docker, Compose, and Helm recipes - the admin UI is embedded, so there is nothing else to deploy.

Storage That Grows

Start on SQLite with zero setup, move to PostgreSQL or MongoDB when traffic and retention demand it - same binary, different config.

Session Keeping

Requests from one conversation or agent task stick to the target and key that served the first - warm provider prompt caches, threaded audit logs, per-session cost. Zero config.

First-Class Streaming

SSE streaming with usage and audit recorded from the stream itself - no buffering, no lost accounting on long generations.

What teams use it for

The same gateway, six different jobs.

Bill AI back to tenants

A multi-tenant SaaS issues an API key per customer, tracks usage by user path, and enforces per-tenant budgets - billing numbers come from the dashboard, not guesswork.

One internal AI endpoint

A platform team publishes aliases like smart-chat with scoped workflows behind them - product teams ship features without ever holding provider keys.

Stay up through provider outages

Production traffic rides failover chains with retries and circuit breakers - a provider incident becomes a routing event, not a customer-facing one.

Cut spend without code changes

Caching absorbs duplicate prompts, cost-based routing picks the cheapest capable model, and budgets stop end-of-month surprises.

Pass the audit

Compliance reviews replay any request: resolved route, guardrail versions, provider attempts, and full bodies where logging is explicitly enabled.

Same API from laptop to prod

Developers run Ollama or vLLM locally behind the same endpoint the cloud providers serve in production - switching is config, not code.