SGLang - GoModel

GoModel talks to SGLang through its OpenAI-compatible /v1 API and exposes SGLang-native endpoints through provider passthrough. Hugging Face model IDs with slashes work because GoModel splits provider-qualified selectors on the first slash only. Start SGLang first:

python -m sglang.launch_server \
  --model-path Qwen/Qwen2.5-0.5B-Instruct \
  --host 0.0.0.0 \
  --port 30000
# Add --api-key token-abc123 if the server should require bearer auth.

See SGLang’s OpenAI-compatible API documentation for current launch and accelerator-specific options.

Configure

SGLANG_BASE_URL=http://host.docker.internal:30000/v1   # include /v1
# SGLANG_API_KEY=token-abc123                          # only with --api-key
GOMODEL_MASTER_KEY=change-me

These examples assume GoModel runs in Docker and SGLang runs on the host. If both run in the same Docker or Kubernetes network, use the SGLang service name. If GoModel runs directly on the host, use http://localhost:30000/v1.

Run GoModel

docker run --rm -p 8080:8080 --env-file .env enterpilot/gomodel

docker run --rm -p 8080:8080 \
  -e GOMODEL_MASTER_KEY="change-me" \
  -e SGLANG_BASE_URL="http://host.docker.internal:30000/v1" \
  enterpilot/gomodel

make build
./bin/gomodel

Verify

curl -s http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer change-me" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sglang/Qwen/Qwen2.5-0.5B-Instruct",
    "messages": [{"role": "user", "content": "Reply with exactly ok."}],
    "max_tokens": 8
  }'

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="change-me")

completion = client.chat.completions.create(
    model="sglang/Qwen/Qwen2.5-0.5B-Instruct",
    messages=[{"role": "user", "content": "Reply with exactly ok."}],
    max_tokens=8,
)

print(completion.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080/v1",
  apiKey: "change-me",
});

const completion = await client.chat.completions.create({
  model: "sglang/Qwen/Qwen2.5-0.5B-Instruct",
  messages: [{ role: "user", content: "Reply with exactly ok." }],
  max_tokens: 8,
});

console.log(completion.choices[0].message.content);

GET /v1/models returns SGLang model IDs prefixed by provider name, for example sglang/Qwen/Qwen2.5-0.5B-Instruct.

Multiple SGLang instances

Use suffixed environment variables to register more than one instance without YAML:

SGLANG_BASE_URL=http://host.docker.internal:30000/v1
SGLANG_TEST_BASE_URL=http://host.docker.internal:30001/v1

This registers sglang and sglang-test. The suffix is lowercased and underscores become hyphens.

Native passthrough

Passthrough is enabled by default. Root-relative SGLang endpoints such as /generate are sent without the configured /v1 prefix:

curl -s http://localhost:8080/p/sglang/generate \
  -H "Authorization: Bearer change-me" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello",
    "sampling_params": {"max_new_tokens": 8}
  }'

import httpx

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080", api_key="change-me")
generation = client.post(
    "/p/sglang/generate",
    cast_to=httpx.Response,
    body={
        "text": "Hello",
        "sampling_params": {"max_new_tokens": 8},
    },
)

print(generation.json())

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8080",
  apiKey: "change-me",
});

const generation = await client.post("/p/sglang/generate", {
  body: {
    text: "Hello",
    sampling_params: { max_new_tokens: 8 },
  },
});

console.log(generation);

Keep the explicit v1/ segment for SGLang endpoints that include it, such as /v1/rerank or /v1/tokenize:

/p/sglang/v1/rerank
/p/sglang/v1/tokenize

GoModel strips client authorization before forwarding and applies SGLANG_API_KEY when configured.

Capability notes

Chat completions, streaming, model listing, Responses, and embeddings use SGLang’s OpenAI-compatible API.
Embeddings and model-specific features depend on the model loaded by SGLang.
Native batch, file, and stored-response lifecycle interfaces are not yet exposed as typed GoModel provider capabilities; use passthrough where the installed SGLang version supports them.

​Configure

​Run GoModel

​Verify

​Multiple SGLang instances

​Native passthrough

​Capability notes

Configure

Run GoModel

Verify

Multiple SGLang instances

Native passthrough

Capability notes