Production Deployment

This page collects the best practices for running GoModel in production. The Quick Start gets you serving traffic in a minute; this page covers what to decide before that install carries real load. Each section states the default, when the default is fine, and when you should change it. A condensed checklist is at the end.

Small enough to run anywhere

GoModel is a single static Go binary with no runtime dependencies. It is built with CGO_ENABLED=0 and ships on distroless/static, so the container has no shell, no package manager, and no libc to patch.

	GoModel
Container image (compressed)	`14 MB`
Peak RAM under load	`60 MB`
Cold start	`0.76 s`

Those numbers are from the benchmarks page, measured against other gateways on identical hardware. The practical consequence is that GoModel fits comfortably where size and cold starts are penalized: Cloud Run, Fargate, Container Apps, Fly.io, container Lambdas, or a small VM next to your app. It does not need a dedicated node pool.

Scale-to-zero and autoscaling environments change two things. Local disk is ephemeral, so SQLite is not a valid backend there (see storage), and several in-memory controls are multiplied by the number of live instances (see multiple replicas).

Choose a storage backend

GoModel persists audit logs, usage records, budgets, virtual models, managed API keys, and other admin state through STORAGE_TYPE.

Backend	`STORAGE_TYPE`	Use it when
SQLite (default)	`sqlite`	A single instance with a durable local disk or a mounted volume.
PostgreSQL	`postgresql`	More than one replica, or you already operate Postgres. The usual production choice.
MongoDB	`mongodb`	You already operate MongoDB and prefer it over Postgres.

SQLite is the default so GoModel works with zero configuration, and it is a real production option for a single instance with a persistent volume. It removes a network hop and a component to operate. It stops being viable in two cases: more than one replica, and any runtime whose filesystem does not survive a restart. Two SQLite replicas do not share a database, they each get their own, so usage, budgets, and managed API keys diverge silently. A key created on one pod does not exist on the other. SQLite also runs with a single open connection by design, to keep one writer and avoid lock contention. Every audit flush, usage flush, and budget check serializes through it, which is a throughput ceiling under sustained load. For Postgres, POSTGRES_MAX_CONNS defaults to 10. Raise it first if you run high concurrency with budgets or rate limits enabled.

The Docker image does not declare a volume. The default database path resolves to /app/data/gomodel.db inside the image, which is an image-layer directory. Without an explicit volume mount, every audit log, usage record, budget, managed API key, and stored credential is lost when the container is replaced. Always mount a volume at /app/data, or move to Postgres/MongoDB. The gateway logs a warning at startup when it detects this situation, and the repository’s docker-compose.yaml mounts the gomodel_data volume there.

The default SQLite path is ./data/gomodel.db when a ./data directory already exists next to the binary, and the OS per-user data directory otherwise. That makes the default path dependent on the working directory, so set SQLITE_PATH explicitly in production. The resolved path is printed at startup as storage configured. See Where does GoModel store its data?.

Redis

Redis is optional. It backs the shared model catalog snapshot and the exact response cache. Without it, GoModel uses an in-process cache plus a local file cache, which is correct but not shared between replicas. Add Redis when several replicas should share discovery results and cache hits.

If a Redis model cache is configured and Redis is unreachable at startup, the gateway fails to start. Losing Redis later is only a degraded condition. Take that into account when ordering service startup.

Redis does not coordinate rate limits or budgets: replicas share both through the database. Redis is a cache, not a distributed lock.

Provider credentials

GoModel takes provider credentials from environment variables, from config.yaml, or from the dashboard’s Providers page. All three work. They do not carry the same risk. For production, configure providers through environment variables and supply the values from your secret manager. Inject them with AWS Secrets Manager, Google Secret Manager, Azure Key Vault, HashiCorp Vault, or a Kubernetes Secret populated by the External Secrets Operator or the Secrets Store CSI driver. There are two reasons, and the second is the one people miss. Secrets stay out of the database. Provider credentials saved from the dashboard are persisted to the provider_credentials store, and MCP server headers and guardrail secrets to theirs. Unless GOMODEL_ENCRYPTION_KEY is set, they are stored in plaintext: the *** masking is applied on the read path only, so the stored value is readable to anyone with the database file, a backup, a snapshot, or a read replica. With the key set they are encrypted at rest (see Encryption at Rest), but the key then has to be protected and kept. Credentials declared via environment variables or config.yaml are never written to the store at all. Config-declared providers cannot be edited at runtime. A provider declared in env or config.yaml is read-only in the dashboard. A provider stored in the database can be edited by anyone holding a dashboard-capable key, including its base_url. Repointing base_url at an attacker-controlled host silently redirects every prompt and the injected upstream key to it. Declaring providers as code means that change can only arrive through your deploy pipeline.

An environment variable is only as good as its source. A key baked into a Dockerfile, a committed .env, or a compose file is worse than the dashboard, not better. The point is that the value is delivered at runtime by a secret manager that owns its rotation, access control, and audit trail.Environment variables are also visible through /proc/<pid>/environ, docker inspect, and crash dumps, so restrict who can exec into the container or describe the pod.

If your secret tooling mounts files, point a setting at the file with a secret reference instead of copying it into an environment variable:

OPENAI_API_KEY='${file:/run/secrets/openai}'
POSTGRES_URL='${file:/run/secrets/postgres-url}'

A reference that cannot be resolved stops startup, and a failed reload keeps the running configuration. Reading secrets straight from HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, or Azure Key Vault comes with GoModel Pro. The dashboard path stays the right choice for development, evaluations, and adding a provider without a redeploy. If you use it in production, set GOMODEL_ENCRYPTION_KEY and still treat the database as sensitive: encrypt the volume, restrict access, and classify backups accordingly.

Other credential handling

Managed API keys are stored as a SHA-256 hash of a high-entropy random secret, never recoverably. The plaintext is shown once at creation. This is deliberately different from provider credentials.
Client credential headers are redacted before they are persisted. With LOGGING_LOG_HEADERS=true, Authorization, x-api-key, Cookie, and the other credential headers are replaced with [REDACTED] on the write path, so the audit store never receives them.
Set GOMODEL_MASTER_KEY. If it is empty and no managed keys exist, every request is allowed through unauthenticated, and /admin/* is added to the auth skip list as a lockout-recovery path. An internet-reachable gateway in that state hands anyone the full admin API.

TLS

By default GoModel serves plain HTTP and expects a reverse proxy or load balancer in front of it to terminate TLS. For any deployment that is not loopback-only, use one or turn on native TLS: without either, client API keys, managed gateway keys, and the master key all cross the network in cleartext. There is also no CORS middleware, so browser clients on another origin cannot call the gateway directly. Add the headers at your proxy if you need that.

Native TLS

Set a certificate and key and the listener serves HTTPS on the same port, with TLS 1.2 or later, Go’s default cipher suites, and HTTP/2. Use it when clients reach the gateway directly, for example pod to pod through a Kubernetes Service: no TLS sidecar to maintain, and the gateway sees the real client address.

SERVER_TLS_CERT_FILE=/etc/gomodel/tls/tls.crt
SERVER_TLS_KEY_FILE=/etc/gomodel/tls/tls.key
# Optional: require a client certificate signed by this CA (mTLS)
SERVER_TLS_CLIENT_CA_FILE=/etc/gomodel/tls/ca.crt

The files are checked for changes at most every 5 seconds. A rotated certificate, or client CA, is served from the next handshake on, with no restart or reload.
A rotation that fails to load, such as a pair caught half written, keeps the previous certificate and logs a warning. A missing or invalid file at startup is an error.
Changing the paths takes effect on a configuration reload.
Once TLS is on, the port no longer answers plain HTTP. gomodel --health and gomodel --ready, which the Docker HEALTHCHECK runs, switch to HTTPS automatically.

Kubernetes with cert-manager. Issue the certificate into a Secret and mount it as a volume. Do not use subPath: the kubelet does not update subPath mounts, so a renewed certificate would never reach the pod.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: gomodel-tls
spec:
  secretName: gomodel-tls
  dnsNames: [gomodel.ai.svc, gomodel.ai.svc.cluster.local]
  issuerRef: { kind: ClusterIssuer, name: internal-ca }

With the Helm chart in helm/:

env:
  SERVER_TLS_CERT_FILE: /etc/gomodel/tls/tls.crt
  SERVER_TLS_KEY_FILE: /etc/gomodel/tls/tls.key
extraVolumes:
  - name: tls
    secret: { secretName: gomodel-tls }
extraVolumeMounts:
  - { name: tls, mountPath: /etc/gomodel/tls, readOnly: true }
livenessProbe:
  httpGet: { path: /health, port: http, scheme: HTTPS }
readinessProbe:
  httpGet: { path: /health/ready, port: http, scheme: HTTPS }

The kubelet does not verify the certificate of an HTTPS probe. With mTLS on, /health and /health/ready still answer without a client certificate, so these probes work unchanged; every other path returns 401 without a verified one. The chart’s ServiceMonitor scrapes plain HTTP, so with native TLS define your own with scheme: https. Under mTLS /metrics needs a client certificate too, so give that ServiceMonitor a tlsConfig with ca set to the CA that issued the gateway certificate, plus cert and keySecret from a certificate the client CA signed. helm test and the port-forward install notes switch to HTTPS on their own when env.SERVER_TLS_CERT_FILE is set.

Upstream certificates behind a private CA

Outbound HTTPS — providers and MCP servers — is verified against the system CA bundle. The container images ship it (/etc/ssl/certs/ca-certificates.crt), so public endpoints need no setup. An upstream whose certificate is issued by an internal or corporate CA fails the connection with x509: certificate signed by unknown authority. Mount the CA certificate into the trusted directory, which GoModel reads in addition to the public roots:

docker run --rm -p 8080:8080 \
  -v "$PWD/corp-ca.crt:/etc/ssl/certs/corp-ca.crt:ro" \
  enterpilot/gomodel:latest

SSL_CERT_FILE works too, but it replaces the bundle file rather than adding to it, so point it at a bundle that also contains the public roots you need. Outside a container, or to also send a client certificate, set HTTP_TLS_CA_FILE instead; see Private CAs and proxies.

Air-gapped and offline deployments

GoModel runs fully air-gapped. There is no telemetry, no analytics, and no license check. The admin dashboard is embedded in the binary and its fonts are vendored, so the UI loads no CDN assets. Paired with local model servers such as Ollama, SGLang, or vLLM, the gateway needs no route to the public internet. By default the gateway makes exactly two outbound calls that are not to a configured provider, and one switch turns both off:

GOMODEL_OFFLINE=true

Call	What it does	Default
Update check at `GOMODEL_VERSION_CHECK_URL`	Daily fetch of the latest release number, with an anonymous install id	on
Model catalog at `MODEL_LIST_URL`	Pricing, context windows, and capabilities, fetched at startup and on each cache refresh	on

GOMODEL_OFFLINE=true (or offline: true in config.yaml) disables the update check and drops any HTTP catalog URL. It is applied after every other setting, so nothing in config.yaml or the environment can re-enable an outbound call underneath it. Calls to configured providers and to anything you declared yourself, such as an OTLP collector, MCP upstreams, or a vector store, are untouched. The startup log confirms the mode. You can also control the two calls individually with GOMODEL_VERSION_CHECK_ENABLED=false and MODEL_LIST_URL=off. The catalog fetch is best-effort. When it is off or fails, the gateway boots normally, /health and /health/ready return 200, provider discovery still works because model lists come from each provider’s own /models endpoint, and requests route normally. What you lose is metadata enrichment.

Without metadata enrichment, models have no pricing. Cost tracking then reports no cost, and since budgets read spend from usage cost records, budgets have nothing to charge against and will not enforce spend limits.

Keep pricing in an air gap in one of three ways:

Ship the catalog as a file. Download models.min.json outside the air gap, copy it in, and point the gateway at it. A local file is read, never fetched, so it stays active under GOMODEL_OFFLINE=true:
MODEL_LIST_URL=/etc/gomodel/models.json # or file:///etc/gomodel/models.json
The file is re-read on every cache refresh, so replacing it on disk updates pricing without a restart.
Mirror it. Serve the file from an internal HTTP host and set MODEL_LIST_URL to that URL. Leave GOMODEL_OFFLINE unset and disable the update check separately, or point GOMODEL_VERSION_CHECK_URL at an internal mirror too.
Declare pricing per model under providers.<name>.models[].metadata.pricing in config.yaml.

Setting MODEL_LIST_URL to an empty string does not disable the download: empty env values are skipped, so the default URL survives. Use off or GOMODEL_OFFLINE=true.

If a Bedrock provider is configured, the AWS SDK may also probe the link-local instance metadata endpoint (169.254.169.254) for credentials. An air-gapped install would not configure Bedrock, but it is worth knowing.

Private CAs and proxies

Every outbound connection the gateway opens shares one HTTP transport, so proxy and trust settings are configured once and apply to providers, the model catalog, the update check, MCP upstreams, vector stores, and embedding endpoints alike. Proxy. Set the standard HTTPS_PROXY and HTTP_PROXY (each covers upstreams of its own scheme; most deployments only talk to https:// upstreams, but a plain-HTTP model server would bypass a proxy set only through HTTPS_PROXY), plus NO_PROXY for local model servers. Nothing else is needed. If the proxy needs credentials, give it an https:// URL: the scheme in the proxy variable describes the hop to the proxy, and an http:// proxy URL sends Proxy-Authorization in cleartext. To send one provider through a different proxy, see outbound proxy. Private CA. Mount the CA bundle and point the gateway at it:

HTTP_TLS_CA_FILE=/etc/gomodel/ca.pem

The bundle is appended to the system store, so public providers keep working next to internal ones. A missing or empty file fails startup rather than silently trusting less than you asked for. Mutual TLS. When an internal model server or egress proxy requires a client certificate:

HTTP_TLS_CLIENT_CERT_FILE=/etc/gomodel/tls.crt
HTTP_TLS_CLIENT_KEY_FILE=/etc/gomodel/tls.key

A single client certificate is sent to every upstream that requests one. Rotation. GoModel reads these files at startup and on every configuration reload, but does not watch them. After a rotation, such as cert-manager renewing a mounted secret, run gomodel --reload (or send SIGHUP) or restart. If a rotated file fails to load, the reload is rejected and the running configuration keeps serving. The same settings live under http.tls in config.yaml. Avoid HTTP_TLS_INSECURE_SKIP_VERIFY=true outside a lab: it disables verification for every upstream, and the gateway warns about it at startup.

Running more than one replica

Request handling is stateless, so replicas need no affinity for ordinary traffic. Several features, however, keep state in process memory and do not coordinate across instances.

State	Scope	Consequence at N replicas
Budgets enforcement	Durable (database)	Correct across replicas. Spend is read from the shared usage table.
Usage and cost records	Durable (database)	Aggregated correctly.
Audit logs	Durable (database)	Complete across replicas.
SSO sign-out (Pro)	Durable (database)	A signed-out session is rejected on every replica within about 2 seconds.
Rate limits	In memory, shared through the database	Replicas enforce one limit together, approximately: each sees the others’ counts within about two flush intervals. A restart restores the last saved request and token counts once; a crash can lose those since the last flush. A reset clears those windows on every replica; active concurrency slots stay held until their requests finish.
Circuit breakers	In-memory, per process	Each replica learns an outage separately, so upstream sees up to `N x failure_threshold` failures before any breaker opens.
Session affinity pins	In-memory, per instance	Sessions stick per replica, not globally, unless the load balancer is sticky.
Live logs	Local events in memory, other replicas’ from the database	A dashboard sees its own replica’s requests instantly and every other replica’s typically within 7 seconds, once they are persisted. See live logs across replicas.
MCP downstream sessions	Durable (database)	A session opened on one replica continues on any other, so MCP clients need no sticky sessions. After a `DELETE`, every replica refuses its requests within about 2 seconds; an open `GET` stream is not closed early. A response stream in flight stays on the replica serving it.
Provider request health	In-memory, per instance	Dashboard provider status reflects the replica that answered.

The one that surprises people is rate limits: admission is in memory, so a replica learns the others’ traffic only when it reads their saved counts, about once a second by default. Three replicas with a 100 rpm rule admit about 100 rpm in aggregate, but can overshoot by about two seconds of the other replicas’ traffic, and a crashed replica’s concurrency slots stay taken for a few seconds. Do not rely on rate limits for a cap that must never be exceeded, such as one concurrent request across replicas, and use budgets as the durable cross-instance control for spend. See several replicas share one limit. RATE_LIMITS_FLUSH_INTERVAL=0 stops replicas sharing counts after start, so together they can again admit up to about N x the limit.

Live logs across replicas

A dashboard streams live logs from the replica it is connected to. That replica shows its own requests the moment they start, with in-flight and streaming previews. While at least one dashboard is connected, it also reads the audit and usage rows the other replicas persisted, every 2 seconds, and streams them into the same view. It adds no database writes: each replica stamps the batches it already writes with its own id, and with no dashboard open no replica reads anything. Requests served by another replica:

appear only once that replica has flushed them, so after up to LOGGING_FLUSH_INTERVAL / USAGE_FLUSH_INTERVAL (default 5 seconds) plus the 2-second poll, typically within 7 seconds; a write the database is slow to finish shows up once it completes, so later than that;
appear as completed entries, with status, duration, model, provider, usage and cost, but no in-flight or per-chunk streaming preview; bodies load from the database when you expand the entry;
appear only if they were persisted. Audit entries and usage entries come from separate features: a replica with audit logging off contributes no audit entries, and one with usage tracking off no usage entries. The replica your dashboard is connected to shows another replica’s audit entries only with audit logging on itself, and its usage entries only with usage tracking on;
are slotted into the list by request time, below anything newer.

Under heavy traffic, at most 200 audit and 200 usage entries from other replicas are streamed per poll, the newest first. The rest are still in the list after a refresh. A dashboard that reconnects, or a replica whose reads failed for a while, catches up on what the other replicas persisted in the meantime, up to 5 minutes back and within the same 200-entry limit per poll. Replica clocks should agree within a few seconds, which NTP gives you.

Config changes reach every replica within seconds

Admin-managed configuration is durable in the database and served from an in-memory snapshot on each replica. A change saved in the dashboard or admin API applies immediately on the replica that handled it, which also bumps a revision for that entity in the shared config_revisions table. Every replica polls the table every 2 seconds and reloads whatever moved, so the change reaches the others within a few seconds. Only the shared database is involved: no Redis, and it works the same on PostgreSQL and MongoDB. This covers budgets (definitions, settings, and resets), rate limit rules and counter resets, provider credentials (add, edit, disable, delete), managed API keys (including revocation), user policies, virtual models, pricing overrides, tagging rules, guardrails, workflows, and MCP servers. A provider added on another replica starts routing once its model list has been fetched. A provider whose credentials change, with its type unchanged, keeps routing the models it already served, through the new credentials, while its model list is fetched again. The edit’s configured models and model filter apply at once. A model removed from the configured list stops routing; if the upstream still lists it, it can be missing until the new list arrives. Once the new list arrives, models it no longer includes stop routing, except configured models in merge mode. If the fetch fails or returns no models while another provider is healthy, the previous models are kept as stale: requests that name them still route to the provider, but model listings and virtual-model load balancing leave them out. A provider whose type changes starts routing once its new model list has been fetched. A replica that fails to apply a change logs a warning once and retries on every poll until it applies. One exception: a stored provider that fails to build on a replica keeps whatever was already running under its name and is retried on the next change to any stored provider, since each change reloads all of them, or on restart. A replica that cannot record a change in config_revisions also retries on every poll. Until it succeeds, the other replicas pick the change up from a later change to the same feature, from the slower periodic refresh some features already had, from a manual runtime refresh, or on restart. POST /admin/runtime/refresh reloads managed API keys, virtual models, guardrails, workflows, MCP servers, budgets, and rate limits on the replica that receives it. It does not reload stored provider credentials.

Health probes and shutdown

GoModel exposes liveness and readiness separately, and the binary can probe itself, which works in a distroless image with no shell or curl.

Purpose	HTTP	CLI	Checks
Liveness	`GET /health`	`gomodel --health`	Nothing. Confirms the listener answers.
Readiness	`GET /health/ready`	`gomodel --ready`	Storage (503 if down), Redis (degraded, still 200), extension components (degraded: 200, down: 503).

Point Kubernetes readiness at /health/ready, not /health. The Helm chart in the repository’s helm/ directory does this already, runs the gateway as the image’s non-root user on a read-only filesystem, keeps SQLite on a persistent volume, and refuses more than one replica until you switch storage.type to postgresql or mongodb. See helm/README.md for the values.

Readiness does not wait for the model catalog, which loads asynchronously. A fresh pod can report ready while GET /v1/models is still empty and the first requests fail. Allow for that in rollout settings, or warm pods before shifting traffic.

On SIGTERM the gateway drains in-flight HTTP work for 10 seconds, then finishes teardown within 30 seconds, flushing buffered usage and audit entries on the way out. Neither timeout is configurable. In-flight streamed responses are cut at the 10-second mark, so a long completion in progress during a rolling deploy will be truncated. Set terminationGracePeriodSeconds above 30 (45 is a reasonable choice) so the teardown is not racing SIGKILL.

Data retention and disk growth

Audit logging and usage tracking write a row per request, and both sweep hourly.

Setting	Default	Notes
`LOGGING_RETENTION_DAYS`	`30`	Audit entries older than this are removed.
`USAGE_RETENTION_DAYS`	`90`	Usage records older than this are removed.
`LOGGING_LOG_BODIES`	`true`	Captures full request and response bodies, up to 1 MB each.

LOGGING_LOG_BODIES is on by default and stores complete prompt and response bodies with no content redaction. It is excellent for debugging and it is by far the largest contributor to database growth: up to about 2 MB per request, retained for 30 days. It also means anything sensitive a user types into a prompt is persisted for the full retention window.In regulated environments, or anywhere prompts carry personal data, set LOGGING_LOG_BODIES=false and keep the metadata-only audit trail.

Two further points at volume:

Usage and audit writes are buffered and dropped, not blocked, when the buffer fills. A burst or a slow database can silently lose usage entries, which means lost cost data and under-charged budgets. The only signal is a usage log buffer full warning in the logs, so alert on it.
Budget checks query the usage table on every request. Keep USAGE_RETENTION_DAYS tight when budgets are enabled, and prefer Postgres over SQLite, whose single connection makes this the dominant cost.

Retention deletes rows but does not shrink a SQLite file. Vacuum it separately if the file size matters.

Observability

Metrics are off by default. Set METRICS_ENABLED=true to expose Prometheus metrics at METRICS_ENDPOINT (/metrics). The endpoint is unauthenticated, so keep it inside your perimeter. See the Prometheus guide.
OpenTelemetry export is off by default. Set OTEL_ENABLED=true and OTEL_EXPORTER_OTLP_ENDPOINT to send traces and GenAI metrics to your collector. See the OpenTelemetry guide.
Logs are JSON automatically when stdout is not a TTY, so containers get structured logs with no configuration. Set LOG_FORMAT explicitly to override. LOG_LEVEL defaults to info; an invalid value fails startup.
gomodel_circuit_breaker_state is the gauge to alert on for provider health. It updates per request, so an idle provider keeps its last observed value.
Request duration for streaming responses measures time to stream establishment, not total stream duration. Read stream latency panels with that in mind.
There are no token or cost metrics in Prometheus. Spend and token accounting live in the usage database, the dashboard, and GET /v1/usage. Alert on spend from budgets, not from Prometheus.

Hardening notes

Put the admin surface behind your perimeter. The dashboard and /admin/* are served on the same port as the model API and cannot be moved to a separate listener. Restrict them at the proxy or network level, or disable them with ADMIN_ENDPOINTS_ENABLED=false. The dashboard shell and its static assets skip auth by design; only the data they load is gated.
Restrict the database file. The SQLite data directory is created 0755 and the database file with default permissions, so on a shared host it can be world-readable. Run GoModel as a dedicated user and tighten permissions, especially if any credentials are stored through the dashboard.
Keep BODY_SIZE_LIMIT sane. It defaults to 10M and is what stops an oversized body becoming a memory-exhaustion vector. An unparseable value falls back to the default with only a warning, so check the spelling.
Keep STREAM_STALL_TIMEOUT on. It defaults to 60 seconds and is what frees the goroutine and upstream provider connection a client holds when it opens a stream and stops reading. Disabling it (0) lets such a client pin both until the provider-side HTTP_TIMEOUT expires.
Leave pprof disabled. It is off by default and unauthenticated when on.
Scope managed API keys. Give each consumer its own key with its own user path, which is also what makes per-consumer rate limits and budgets possible. Grant dashboard_access and passthrough_access only where needed.

Production checklist

Set GOMODEL_MASTER_KEY to a strong value.
Terminate TLS in a proxy in front of the gateway, or turn on native TLS.
Restrict /admin/*, the dashboard, and /metrics at the network perimeter.
Use Postgres or MongoDB for more than one replica or any ephemeral filesystem; with SQLite, mount a volume and set SQLITE_PATH.
Declare providers in environment variables or config.yaml, with values from a secret manager.
Keep replica clocks synchronized (NTP); shared concurrency limits rely on it.
Decide on LOGGING_LOG_BODIES and set retention windows deliberately.
Point readiness at /health/ready and set terminationGracePeriodSeconds above 30.
Enable metrics and alert on gomodel_circuit_breaker_state and on usage log buffer full.
For air-gapped installs, set GOMODEL_OFFLINE=true, then ship the model catalog as a file or declare model pricing in config so cost tracking and budgets keep working.

​Small enough to run anywhere

​Choose a storage backend

​Redis

​Provider credentials

​Other credential handling

​TLS

​Native TLS

​Upstream certificates behind a private CA

​Air-gapped and offline deployments

​Private CAs and proxies

​Running more than one replica

​Live logs across replicas

​Config changes reach every replica within seconds

​Health probes and shutdown

​Data retention and disk growth

​Observability

​Hardening notes

​Production checklist

Small enough to run anywhere

Choose a storage backend

Redis

Provider credentials

Other credential handling

TLS

Native TLS

Upstream certificates behind a private CA

Air-gapped and offline deployments

Private CAs and proxies

Running more than one replica

Live logs across replicas

Config changes reach every replica within seconds

Health probes and shutdown

Data retention and disk growth

Observability

Hardening notes

Production checklist