Compare

GoModel vs vLLM

Different layers of the stack. vLLM serves open-weight models on your GPUs. GoModel sits in front of vLLM and hosted providers and adds keys, budgets, failover, and audit logs.

Get started with GoModelRead the docs

What each one does

Not rivals: vLLM makes tokens, GoModel decides who may ask for them and where the request goes.

GoModelvLLM
LayerGateway in front of model serversInference engine on your GPUs
What it runsRouting, auth, metering, logsThe model weights themselves
APIOpenAI and Anthropic compatibleOpenAI and Anthropic compatible
Access controlAPI keys per user, budgetsStatic shared API keys
Models per endpoint34 providers, hosted and self-hostedOne base model per process, plus LoRAs
CachingExact and semantic response cacheAutomatic prefix (KV) cache
FootprintAny CPU box, ~21 MB imageGPU server, ~9 GB image

Which one fits?

Most teams that self-host run both.

Use GoModel for

  • Per-team keys, budgets, and rate limits in front of a shared GPU server.
  • Failover from your vLLM box to a hosted provider.
  • One endpoint for self-hosted and frontier models, picked by model name.

Use vLLM for

  • Serving open-weight models from Hugging Face on your own GPUs.
  • High-throughput generation with PagedAttention and continuous batching.
  • LoRA adapters, multi-GPU parallelism, and non-NVIDIA accelerators.

Put GoModel in front of vLLM

One env var registers vLLM as a provider. GoModel adds a couple of milliseconds and gives every caller its own key, budget, and audit trail.

Connect vLLM
# On the GPU box
vllm serve meta-llama/Llama-3.1-8B-Instruct

# In GoModel's environment
VLLM_BASE_URL=http://your-vllm:8000/v1
# VLLM_API_KEY=...   only if vllm serve used --api-key

Call it as vllm/meta-llama/Llama-3.1-8B-Instruct. A second instance gets its own provider via VLLM_GPU2_BASE_URL. vLLM facts are from its docs and repository as of October 2026 (latest release v0.30.0).

Try it against your own workload

Benchmarks are an argument; your traffic is the proof. GoModel speaks the OpenAI and Anthropic APIs natively, so pointing an existing app at it is a base-URL change.

1 · Run GoModel
docker run --rm -p 8080:8080 \
  -e LOG_FORMAT=text \
  -e OPENAI_API_KEY="your-openai-key" \
  enterpilot/gomodel
2 · Point your SDK at it
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="your-gomodel-key",
)

Full setup, providers, and configuration live in the documentation. Questions? Ask on Discord or book a 30-minute call.