Skip to main content

Good defaults

GoModel uses a good defaults philosophy. This means that the default settings should be enough to use it.

How to override the default settings?

We use a three-layer configuration pipeline. Every setting has a sensible default, so you can start the server with zero configuration.
As GoModel works out of the box with no configuration files, you can try it in a minute.Start here: Quick Start
GoModel automatically discovers providers from well-known environment variables.

Configuration Methods

1. Environment Variables

The most common way to configure GoModel. Set any of the variables below to override defaults.

Server

Request bodies may be compressed with Content-Encoding: gzip (or x-gzip) or zstd; the TypeSafe SDKs zstd-compress bodies of 1 KiB or more by default. The gateway decodes them before routing, so guardrails, audit, and providers see the plain body, and BODY_SIZE_LIMIT applies to both the compressed and the decoded size (413 when either is over). Other encodings return 415. A body is read and decoded only after the request is authenticated, so none of the body of a request without valid credentials is read, and its audit entry has no request body. Such a request gets 401 even when its body would be over the limit once decoded; only a declared size over the limit is still 413. STREAM_STALL_TIMEOUT (server.stream_stall_timeout in YAML) protects the gateway from clients that open a streaming response and stop reading it. Each write to the client gets a fresh deadline, so it fires only when the client’s socket stays full for that long, never while a slow model has nothing to send. When it fires the connection is closed, the upstream provider request is cancelled, and the audit entry records client_stalled, also when the stream was served from the response cache. Raise it for clients that legitimately pause reading a stream for over a minute; set 0 to disable.

Client addresses behind a proxy

Every client address GoModel reports — audit entries, rate limit keys, logs — is the address of the connection that reached the gateway, so behind a reverse proxy they all show the proxy. Set SERVER_TRUSTED_PROXIES to the networks your own proxies sit on and GoModel reads the forwarded client instead. Nothing is trusted until you list it, not even loopback, and changing these settings needs a restart.
Entries are CIDR networks, bare addresses (a single host), or a preset: Prefer the one subnet your proxy sits on. private trusts every private client to set its own forwarded address, which is the wrong answer when your clients are on private addresses too — a home lab whose clients live on 192.168.1.0/24 and whose cluster runs on 10.42.0.0/16 should list 10.42.0.0/16 alone. By default GoModel reads X-Forwarded-For, where each hop appends the address it saw, and records the nearest hop that is not one of your trusted networks: the last address your own infrastructure wrote, and the furthest one a client cannot overwrite. A request from outside the listed networks, a chain it cannot parse, and a chain made entirely of trusted networks all keep the connection address — in that last case every entry is a claim an internal client could have invented. Two settings change how the header is read:
  • SERVER_CLIENT_IP_HEADER names a different header. Anything other than X-Forwarded-For carries a single address and is taken verbatim, which is the correct reading behind an edge that resolves the client itself: SERVER_CLIENT_IP_HEADER=CF-Connecting-IP behind Cloudflare, or X-Real-IP behind an nginx that sets it. RFC 7239 Forwarded is not supported.
  • SERVER_TRUSTED_HOPS takes the X-Forwarded-For entry at a fixed depth instead of scanning the chain — 1 is the rightmost entry, 2 the one before it. Use it when your edge sits at a known depth but rotates through addresses you cannot enumerate. The connection must still come from a trusted network, and a chain shorter than the configured depth falls back to the connection address.
Both require SERVER_TRUSTED_PROXIES; on their own they are a startup error rather than a setting that silently does nothing. Set GOMODEL_DEMO_MODE=true for a public or shared demonstration instance. GoModel logs a warning at startup and every five minutes, renders a persistent warning at the top of the dashboard, and exposes DEMO_MODE=on through the allowlisted /admin/runtime/config response. Demo mode does not reset storage; schedule that separately in the deployment. Demo mode also does not bypass authentication. A startup log mode of managed_keys means the configured storage still contains at least one managed gateway API key, so requests require a bearer token even when GOMODEL_MASTER_KEY is empty. Disabled or expired managed keys keep authentication enabled as a fail-closed safety measure. Delete every managed key to return to unauthenticated mode, or configure a master key for recovery.

Disabling the master key

The master key is a single global credential: whoever holds it is every user of the gateway at once. Once managed gateway API keys are issued, set MASTER_KEY_DISABLED=true (server.master_key_disabled in YAML) to retire it and leave those keys — together with any authentication extension you run, such as SSO — as the only way in. Per-key user paths, budgets, rate limits, and audit trails then apply to every request, including the admin API.
GoModel forgets any configured master key at startup, so nothing accepts or derives from it: it fails against the gateway, against GET /v1/auth/verify, and against /admin. A key left in the environment or config.yaml is inert rather than an error, which is what makes the switch safe to flip ahead of removing it. Disabling is not the same as leaving GOMODEL_MASTER_KEY empty, which runs the gateway unauthenticated and opens /admin/* so the dashboard can bootstrap its first managed key. Unless an authentication extension is configured, /admin/* stays open even after managed keys exist, and GoModel logs a security warning at startup while it does, so set a master key or disable it as soon as the first key with dashboard access is issued. A disabled master key fails closed instead: every request needs a credential, and the admin API stays behind one carrying dashboard access. Issue that credential before flipping the switch — with no master key, no managed key, and no authentication extension, the gateway rejects every request and logs a warning at startup saying so.

Serving HTTPS

Leave the SERVER_TLS_* settings unset to serve plain HTTP behind a proxy that terminates TLS. Set the certificate and key (server.tls_cert_file and server.tls_key_file in YAML) when clients connect to the gateway directly:
The listener then serves only HTTPS, with TLS 1.2 or later and HTTP/2. The files are checked every 5 seconds and a rotated certificate is picked up without a restart; a rotation that fails to load keeps the previous one. A rotated client CA applies from the next TLS handshake, so open connections keep the access they were granted. Setting only one of the pair, or a file that is missing or invalid at startup, is an error. The settings take file paths, not PEM contents, and a value holding PEM is rejected. They accept ${env:...} references; ${file:...} resolves to a file’s contents, so use it only for a file that holds a path. These settings cover inbound connections only; outbound TLS trust is configured separately. See the production guide for a cert-manager example.

MCP Gateway

See MCP Gateway for the full feature guide.

Logging

Runtime logger output. For the persisted API audit trail (request/response bodies, headers), see Audit Logging below.

Cache

See Cache for exact-cache behavior, response headers, analytics endpoints, and how partitioning by user_path works and where its isolation ends.
CACHE_REFRESH_INTERVAL also sets how often the dashboard re-checks provider health (“Last checked”), and PROVIDER_RECHECK_INTERVAL how quickly a failed provider is re-probed for recovery. See Resilience.

Storage

Storage is shared by audit logging, usage tracking, and future features like IAM.
Upgrading a PostgreSQL deployment past v0.1.59. The workflow_versions table stored created_at as TIMESTAMPTZ on PostgreSQL and as unix seconds on SQLite. The two are now unified on unix seconds, and the first startup converts the column in place. No manual migration is needed, but the change is one-way: an older binary started against a converted database fails to boot. Take a backup before upgrading if you may need to roll back, and roll back by restoring it rather than by swapping the binary. Workflow created_at values are floored to whole seconds and rendered in UTC. SQLite and MongoDB deployments are unaffected.

Media Storage

Media files — the audio and images captured for the audit log when LOGGING_LOG_AUDIO_BODIES or LOGGING_LOG_IMAGE_BODIES is on — are kept outside the database. Their metadata lives in the storage backend above; only the bytes go here, so they are never capped in size and the database stays small. See ADR-0013. The default path follows the SQLite rule: ./data/media when a ./data directory exists next to the process, otherwise the OS per-user data directory. In containers mount a volume there; with several replicas use a shared mount (EFS, an S3 FUSE mount) so every instance can serve every file. Files are retained as long as the audit entries that reference them (LOGGING_RETENTION_DAYS) and removed by an hourly sweep.

Audit Logging

Audit entries record the client address the gateway resolved. Behind a reverse proxy that is the proxy until you list its network in SERVER_TRUSTED_PROXIES — see client addresses behind a proxy. Realtime dashboard previews and persisted audit logs are separate features. With DASHBOARD_LIVE_LOGS_ENABLED=true and LOGGING_ENABLED=false, requests can appear live but are not written to storage and disappear after a page refresh. When audit logging is enabled, writes are asynchronous and can take up to LOGGING_FLUSH_INTERVAL seconds to appear through the stored-log API.
On PostgreSQL, the audit log’s free-text search uses a trigram index from the bundled pg_trgm extension. GoModel installs the extension and builds the index on startup when its database role is allowed to; otherwise the search still works but scans every entry in the selected date range. To enable it on a restricted role, run CREATE EXTENSION pg_trgm; as a superuser once and restart GoModel.
On large audit tables, pick a field next to the dashboard’s audit search box instead of All fields. User path and Model match values that start with what you type (case-sensitive, so voxina finds /voxinate/events/…; Model checks the requested and the resolved model, for example openai/gpt-5). Request ID and Session ID match exactly. These filters use indexes on SQLite, PostgreSQL, and MongoDB. The API takes them as user_path_prefix, model_prefix, request_id, and session_id.
When LOGGING_LOG_BODIES is enabled, request and response bodies are stored in full, except where the per-endpoint settings below drop them or replace them with placeholders. These may contain sensitive data such as PII or API keys embedded in prompts.
LOGGING_LOG_REVISION_BODIES refines LOGGING_LOG_BODIES for the ingress request-rewrite chain — it has no effect unless body logging is enabled. With both on, every rewriter that changed a request (for example GoModel Pro token compression) and every prompt guardrail that edited the prompt store the request as they left it alongside the original client body in the audit entry, so a chain of edits reads step by step. LOGGING_LOG_GUARDRAIL_STEPS controls that per-step record (the request bodies are encoded off the request path): turn it off to record a chain’s edits as one revision carrying the request as forwarded, with no per-step snapshots. Disabling LOGGING_LOG_REVISION_BODIES keeps the revision metadata (rewriter name, byte sizes, tokens saved, change detail) but drops the extra body copies, roughly halving audit storage per rewritten request.
LOGGING_LOG_AUDIO_BODIES refines LOGGING_LOG_BODIES for audio endpoints — it has no effect unless body logging is enabled. With both on, /v1/audio/speech stores its text input and the generated audio (base64, capped at 8 MB) so the dashboard can play it back, and /v1/audio/transcriptions stores the uploaded audio (base64, capped at 8 MB, also playable) plus its upload metadata. It defaults to false because audio payloads are large and grow the audit store quickly. When body logging is on but this is off, audio bodies (the speech response and the transcription upload, which keeps its metadata) are recorded as a lightweight {__audio__, content_type, bytes, stored: false} placeholder; when body logging is off, no audio body is stored at all.
LOGGING_LOG_EMBEDDING_BODIES refines LOGGING_LOG_BODIES for embeddings — it has no effect unless body logging is enabled. It covers /v1/embeddings and provider passthrough calls to an embeddings endpoint (/p/{provider}/.../embeddings, llama.cpp /embedding, Cohere and Ollama .../embed, Gemini :embedContent and :batchEmbedContents). Embedding inputs can be whole documents or base64 images and the responses are float vectors, so bulk indexing jobs grow the audit store quickly. When body logging is on but this is off, embedding entries keep their metadata (model, provider, status, timing, user path, usage) and drop the request and response bodies. It defaults to true, which keeps the earlier behaviour.
LOGGING_LOG_IMAGE_BODIES refines LOGGING_LOG_BODIES for the Images API — it has no effect unless body logging is enabled. With both on, generated images (/v1/images/generations, /v1/images/edits) and uploaded edit sources and masks are stored as base64 (8 MB per entry, images beyond the budget become placeholders) so the dashboard can display them. LOGGING_LOG_IMAGE_BODIES_SCOPE narrows this to input (uploads only) or output (results only); the default all stores both. It defaults to false because image payloads are large and grow the audit store quickly. When body logging is on but this is off, image entries keep everything except the pixels: the prompt and parameters, the response envelope (usage, size, quality), hosted URLs, and a sized {role, content_type, bytes, stored: false} placeholder per image.

Token Usage Tracking

Budgets

Budgets use tracked usage cost records. If usage tracking is disabled, GoModel starts with budget management disabled and logs a warning. SET_BUDGET_<PATH> supports the standard periods hourly, daily, weekly, and monthly. The <PATH> suffix is lowercased; use double underscores (__) between path segments, while single underscores stay inside a segment. For example, SET_BUDGET_TEAM__ALPHA__SERVICE="daily=10" configures /team/alpha/service, and SET_BUDGET_TEAM_ALPHA="daily=10" configures /team_alpha. This differs from provider <SUFFIX> variables below, which convert underscores to hyphens in provider names; SET_PROVIDER_BUDGET_<NAME> follows the provider convention, so SET_PROVIDER_BUDGET_OPENAI_PROD limits openai-prod. SET_BUDGET_="monthly=500" means a literal environment variable named SET_BUDGET_, which configures the root path /. POSIX permits that name, but some shells and orchestrators, including some Kubernetes validators, may reject it. Use YAML or the dashboard when your environment cannot set it. Migration note: budget management depends on usage tracking. If USAGE_ENABLED=false, GoModel starts with budgets disabled and logs a warning, even when BUDGETS_ENABLED=true. Set both USAGE_ENABLED=true and BUDGETS_ENABLED=true to enforce budgets. See Budgets for YAML examples, periods, matching, and workflow enforcement.

Metrics

Prometheus support is experimental. See the Prometheus Metrics guide for details.

Admin

Live dashboard logs are compact previews for the Audit Logs and Usage pages. They update rows before the async database flush finishes. The stored audit and usage tables remain the source of truth; if a browser reconnects with a cursor outside the replay window, the dashboard reloads from the normal REST APIs.

HTTP Client

These control timeouts for upstream API requests to LLM providers. A stream that goes quiet for longer than HTTP_STREAM_IDLE_TIMEOUT is closed with a 504 provider error instead of hanging until HTTP_TIMEOUT. The wait for the first token does not count, so slow reasoning models are unaffected. Raise it only for a provider that legitimately pauses for minutes mid-stream. The standard HTTP_PROXY, HTTPS_PROXY, and NO_PROXY variables apply to every upstream connection, unless a provider sets its own outbound proxy.

Outbound TLS trust

These apply to the gateway’s outbound HTTPS clients: providers, the model catalog, the update check, MCP upstreams, vector stores, and embedding endpoints. The default trusts the operating system certificate store, which is right unless an internal CA or a TLS-intercepting proxy sits between the gateway and upstreams. They do not cover connections that other clients open, which keep their own settings: OpenTelemetry export reads OTEL_EXPORTER_OTLP_CERTIFICATE, PostgreSQL takes sslmode and sslrootcert in its URL (prefer sslmode=verify-full), MongoDB takes tls=true and tlsCAFile in its URL, and Redis over rediss:// trusts the system store. A missing or unparseable file is a startup error, so a typo in the path cannot silently fall back to trusting nothing extra. The files are re-read on a configuration reload but not watched, so reload or restart after rotating them. See the production guide for a worked example.

Provider API Keys

Set these to automatically register providers. No YAML configuration required. Most providers can use a custom base URL via <PROVIDER>_BASE_URL (for example OPENAI_BASE_URL). Chutes AI defaults to https://llm.chutes.ai/v1 and can be overridden with CHUTES_BASE_URL. DeepSeek defaults to https://api.deepseek.com; set DEEPSEEK_BASE_URL only for a compatible proxy or alternate DeepSeek endpoint. OpenRouter defaults to https://openrouter.ai/api/v1 and can be overridden with OPENROUTER_BASE_URL. Kilo AI defaults to https://api.kilo.ai/api/gateway and can be overridden with KILO_BASE_URL. Eden AI defaults to https://api.edenai.run/v3 and can be overridden with EDENAI_BASE_URL. Z.ai defaults to https://api.z.ai/api/paas/v4; set ZAI_BASE_URL=https://api.z.ai/api/coding/paas/v4 for the GLM Coding Plan endpoint. SGLang defaults to http://localhost:30000/v1 when SGLANG_API_KEY is set, but keyless deployments should set SGLANG_BASE_URL explicitly to register the provider. vLLM follows the same pattern at http://localhost:8000/v1. llama.cpp’s LLAMACPP_BASE_URL is always required (llama-server’s default port collides with GoModel’s own 8080, so there is no default); LLAMACPP_API_KEY is optional. llm-d has no universal endpoint, so LLMD_BASE_URL is always required; LLMD_API_KEY is optional. Azure uses AZURE_BASE_URL for its deployment base URL and accepts an optional AZURE_API_VERSION override; otherwise it defaults to 2024-10-21 on deployment URLs and sends none on /openai/v1 (Azure AI Foundry) URLs. Oracle requires ORACLE_BASE_URL because its OpenAI-compatible endpoint is region-specific. Every provider type also accepts a comma-separated configured model list via <PROVIDER>_MODELS, for example OPENROUTER_MODELS, ORACLE_MODELS, AZURE_MODELS, SGLANG_MODELS, VLLM_MODELS, or LLMD_MODELS. YAML providers.<name>.models provides the same model-list input for named provider blocks. How a list combines with the provider’s /models is set by CONFIGURED_PROVIDER_MODELS_MODE (default allowlist), or for one provider by <PROVIDER>[_SUFFIX]_MODELS_MODE (YAML: providers.<name>.models_mode), for example OPENROUTER_MODELS_MODE=fallback; see Models modes. An OpenAI-compatible provider without a /models endpoint (404 or 405) is served from its configured list and reported healthy, which suits single-API servers such as speech-to-text.
Breaking change: the default used to be fallback. Providers with a configured list now serve only those models, and other models return 404. See Models modes to restore the old behavior.
GET /v1/models lists provider-qualified IDs (openai/gpt-5) by default. Set UNQUALIFIED_MODEL_IDS_AT_MODELS_ENDPOINT=true (YAML: models.unqualified_model_ids_at_models_endpoint) to list bare model IDs (gpt-5) instead, for clients that validate the requested model against the list and only know plain names. Caveat: when two providers expose the same model ID, only the provider an unqualified request routes to (the first registered one) is listed. Pin a name to a specific provider with a virtual model, for example source: gpt-5 with target: azure/gpt-5. Every provider model is available by default. Set MODELS_ENABLED_BY_DEFAULT=false (YAML: models.enabled_by_default) to require an explicit allow entry in virtual models instead: requests for other models then fail with 400 model_access_denied, and the models stay out of GET /v1/models. To hide provider models from GET /v1/models while keeping them routable, set KEEP_ONLY_ALIASES_AT_MODELS_ENDPOINT=true (YAML: models.keep_only_aliases_at_models_endpoint); the endpoint then lists only enabled virtual model redirects. To narrow a provider’s inventory instead of listing it model by model, use <PROVIDER>_MODEL_FILTER_INCLUDE, <PROVIDER>_MODEL_FILTER_EXCLUDE, and <PROVIDER>_MODEL_FILTER_MAX_PRICE_PER_MTOK (YAML: providers.<name>.model_filter). See Filtering a provider’s models. Set MODELS_VLLM_DISCOVERY_COMPAT=true (YAML: models.vllm_discovery_compat) when a client discovers models only from a vLLM runtime — OpenCode’s vllm provider is one: it probes /health and /v1/models and drops every entry whose owned_by is not vllm or that lacks max_model_len. With the setting on, GET /v1/models reports owned_by as vllm and fills max_model_len on every entry, so the client picks the gateway’s models up with no hand-written list. It is a lie about the owner, so it stays off by default and is meant for gateways whose consumers need it. The limit has two sources, in this order:
  1. The provider’s own report. A runtime that serves the model itself — vLLM, SGLang, llm-d, and any OpenAI-compatible server that returns max_model_len in its /models listing — knows the real limit of the process it is running, so its value wins.
  2. metadata.context_window. The catalog value, used only for a model whose provider reported no limit. It is what makes a hosted model discoverable.
A model with neither — one the catalog does not know, served by a runtime that does not report its limit — is listed with owned_by: "vllm" and max_model_len: 0. Zero means the limit is unknown, not that the entry is incomplete: the field is never omitted, because these clients decode the whole listing against a schema where it is required, so one entry without it fails the response and every other model disappears with it. A client that reads the value only when it is positive keeps its own default. To advertise the real window, declare it on the model:
See Model metadata for the other fields an entry takes. For OpenRouter, GoModel also sends default attribution headers unless the request already sets them. Override those defaults with OPENROUTER_SITE_URL and OPENROUTER_APP_NAME.

2. .env File

GoModel automatically loads a .env file from the working directory at startup. This is convenient for local development.
Copy .env.template to .env and uncomment the values you need:
Real environment variables always override values from the .env file. The .env file is only loaded if it exists — missing it is not an error.

3. Configuration File (YAML)

For more complex setups, you can use an optional YAML configuration file. GoModel looks for it in two locations (in order):
  1. config/config.yaml
  2. config.yaml
If you are deciding whether you need YAML at all, see config.yaml. To get started, copy the example:
Then uncomment and edit the settings you want to change:
The YAML file supports environment variable expansion using ${VAR} and ${VAR:-default} syntax:
To read a value from a mounted file, or to fail startup when a variable is unset, use a secret reference: ${file:/run/secrets/openai} or ${env:OPENAI_API_KEY}.
The YAML file is entirely optional. Any setting you can put in YAML can also be set via environment variables. Use YAML when you need per-provider resilience overrides, generated provider names are not enough, or you prefer a structured config file.

Provider Configuration

Auto-Discovery from Environment Variables

The simplest way to add providers. GoModel checks for well-known API key environment variables and automatically registers providers:
Bare <PROVIDER>_* variables win over the config.yaml provider named after the type (openai). A provider you renamed (alpha: {type: openai}) only picks up fields it left empty; its explicit api_key or base_url is kept and the ignored variables are logged as a warning at startup. When several renamed providers share a type, the bare variables are ignored with the same warning. Use suffixed variables to register more than one instance of the same provider type without YAML. GoModel normalizes the suffix to lowercase and converts underscores to hyphens in the configured provider name:
The same pattern works for every registered provider type: <PROVIDER>_<SUFFIX>_API_KEY, <PROVIDER>_<SUFFIX>_BASE_URL, and <PROVIDER>_<SUFFIX>_MODELS. Azure also supports <PROVIDER>_<SUFFIX>_API_VERSION. Azure and Oracle still require their suffixed BASE_URL values because their endpoints are deployment- or region-specific. Google Vertex AI uses the VERTEX_* prefix and follows the same suffix-to-name rule as other providers:

YAML Provider Blocks

For more control (custom names, per-provider resilience, or larger structured settings), use the YAML file:
models: works for every provider block; how it combines with upstream /models depends on the models mode. For Oracle GenAI, see the Oracle GenAI guide for the required OCI policy and a tested configuration.

Ollama (Local Models)

Ollama does not require an API key. Set the base URL to enable it:
Or in YAML:

vLLM

vLLM uses its OpenAI-compatible /v1 API. In Docker, set VLLM_BASE_URL to register a keyless vLLM server:
If the upstream server was started with vllm serve ... --api-key token-abc123, also set:
You can also register more than one vLLM instance without YAML:
This registers providers vllm and vllm-test. Use YAML only when the generated provider names are not enough or you need a larger structured block.

Provider Behavior Notes

Anthropic max_tokens default

Anthropic requires max_tokens on every /v1/messages request. If a client omits it, GoModel injects a fallback so the request still succeeds. OpenAI and Gemini treat the field as optional and GoModel does not inject a default for them. Raise this for newer models that routinely produce longer outputs (Sonnet 4.6, Opus 4.7). Setting max_tokens explicitly on a request always wins over the env-driven default. Invalid or non-positive values fall back to 4096.
Last modified on November 8, 2026