SGLang - GoModel

GoModel talks to SGLang through its OpenAI-compatible /v1 API and exposes SGLang-native endpoints through provider passthrough. Hugging Face model IDs with slashes work because GoModel splits provider-qualified selectors on the first slash only. Start SGLang first:

python -m sglang.launch_server \
  --model-path Qwen/Qwen2.5-0.5B-Instruct \
  --host 0.0.0.0 \
  --port 30000
# Add --api-key token-abc123 if the server should require bearer auth.

See SGLang’s OpenAI-compatible API documentation for current launch and accelerator-specific options.

Configure

SGLANG_BASE_URL=http://host.docker.internal:30000/v1   # include /v1
# SGLANG_API_KEY=token-abc123                          # only with --api-key
GOMODEL_MASTER_KEY=change-me

These examples assume GoModel runs in Docker and SGLang runs on the host. If both run in the same Docker or Kubernetes network, use the SGLang service name. If GoModel runs directly on the host, use http://localhost:30000/v1.

Run GoModel

docker run --rm -p 8080:8080 --env-file .env enterpilot/gomodel

docker run --rm -p 8080:8080 \
  -e GOMODEL_MASTER_KEY="change-me" \
  -e SGLANG_BASE_URL="http://host.docker.internal:30000/v1" \
  enterpilot/gomodel

make build
./bin/gomodel

Verify

curl -s http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer change-me" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sglang/Qwen/Qwen2.5-0.5B-Instruct",
    "messages": [{"role": "user", "content": "Reply with exactly ok."}],
    "max_tokens": 8
  }'

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="change-me")

completion = client.chat.completions.create(
    model="sglang/Qwen/Qwen2.5-0.5B-Instruct",
    messages=[{"role": "user", "content": "Reply with exactly ok."}],
    max_tokens=8,
)

print(completion.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080/v1",
  apiKey: "change-me",
});

const completion = await client.chat.completions.create({
  model: "sglang/Qwen/Qwen2.5-0.5B-Instruct",
  messages: [{ role: "user", content: "Reply with exactly ok." }],
  max_tokens: 8,
});

console.log(completion.choices[0].message.content);

GET /v1/models returns SGLang model IDs prefixed by provider name, for example sglang/Qwen/Qwen2.5-0.5B-Instruct.

Multiple SGLang instances

Use suffixed environment variables to register more than one instance without YAML:

SGLANG_BASE_URL=http://host.docker.internal:30000/v1
SGLANG_TEST_BASE_URL=http://host.docker.internal:30001/v1

This registers sglang and sglang-test. The suffix is lowercased and underscores become hyphens.

Native passthrough

Passthrough is enabled by default. Root-relative SGLang endpoints such as /generate are sent without the configured /v1 prefix:

curl -s http://localhost:8080/p/sglang/generate \
  -H "Authorization: Bearer change-me" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello",
    "sampling_params": {"max_new_tokens": 8}
  }'

import httpx

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080", api_key="change-me")
generation = client.post(
    "/p/sglang/generate",
    cast_to=httpx.Response,
    body={
        "text": "Hello",
        "sampling_params": {"max_new_tokens": 8},
    },
)

print(generation.json())

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080",
  apiKey: "change-me",
});

const generation = await client.post("/p/sglang/generate", {
  body: {
    text: "Hello",
    sampling_params: { max_new_tokens: 8 },
  },
});

console.log(generation);

Keep the explicit v1/ segment for SGLang endpoints that include it, such as /v1/rerank or /v1/tokenize:

/p/sglang/v1/rerank
/p/sglang/v1/tokenize

GoModel strips client authorization before forwarding and applies SGLANG_API_KEY when configured.

Responses API mode

Some SGLang builds return HTTP 500 from /v1/responses when the request includes function tools, which breaks clients such as Codex CLI. SGLang’s /v1/chat/completions handles those tools, so GoModel can translate the Responses request to chat completions and return a normal Responses reply. SGLANG_API_MODE (or api_mode in config.yaml) picks the path:

Value	Behavior
`auto` (default)	Chat completions when the request has function tools; `/v1/responses` otherwise.
`native`	Always `/v1/responses`. Use it when your SGLang build handles function tools there.
`chat_compatible`	Always chat completions, for one consistent upstream API.

The default fits most setups. Hosted tools such as web_search_preview have no chat equivalent: requests with only hosted tools stay on /v1/responses under auto, and translated requests drop them. Set native if you need hosted tools alongside function tools. previous_response_id works in every mode, because GoModel replays the stored conversation itself.

providers:
  sglang:
    type: sglang
    base_url: "http://localhost:30000/v1"
    api_mode: native   # auto (default), native, or chat_compatible

Capability notes

Chat completions, streaming, model listing, Responses, and embeddings use SGLang’s OpenAI-compatible API. Responses with function tools go through chat completions by default; see Responses API mode.
Embeddings and model-specific features depend on the model loaded by SGLang.
Native batch, file, and stored-response lifecycle interfaces are not yet exposed as typed GoModel provider capabilities; use passthrough where the installed SGLang version supports them.

​Configure

​Run GoModel

​Verify

​Multiple SGLang instances

​Native passthrough

​Responses API mode

​Capability notes

Configure

Run GoModel

Verify

Multiple SGLang instances

Native passthrough

Responses API mode

Capability notes