Serverless Inference

Serverless inference offers pay-per-request pricing with no idle compute cost, the right choice for low-volume or spiky traffic where reserving GPU replicas would be wasteful. The endpoints are OpenAI-compatible. Point any OpenAI client at the base URL below, swap in a Lyceum API key and a model name, and everything else, streaming, tool calling, usage accounting, works the same.

Try it

Enter a key and pick a model to send a real request from this page. It runs the same auth, credit and model checks as any client, so a green result means your key is valid, your org has credit, and the model ID resolves.

This sends one real chat completion and bills it to your org, the same as any other request. It is a few tokens.

Endpoint and authentication

The base URL for serverless inference is:

https://api.lyceum.technology/openai/v1

Authenticate with your Lyceum API key (lk_...) as a Bearer token. Most SDKs append the path for you, so you pass only the base URL above.

Method	Path	Purpose
`POST`	`/chat/completions`	Chat completions (streaming and non-streaming)
`POST`	`/embeddings`	Text embeddings
`GET`	`/models`	List the models currently available

Smart Routing is not part of the OpenAI-compatible surface and keeps its own absolute URL: POST https://api.lyceum.technology/api/v2/external/serverless/route — see Smart Routing.

There is no legacy text /completions endpoint. All text generation goes through /chat/completions using the messages format.

Quickstart

Python
curl

from openai import OpenAI

client = OpenAI(
    api_key="lk_...",
    base_url="https://api.lyceum.technology/openai/v1",
)

resp = client.chat.completions.create(
    model="z-ai/glm-5.2",
    messages=[{"role": "user", "content": "Explain prompt caching in one sentence."}],
)
print(resp.choices[0].message.content)

curl https://api.lyceum.technology/openai/v1/chat/completions \
  -H "Authorization: Bearer lk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "z-ai/glm-5.2",
    "messages": [{"role": "user", "content": "Explain prompt caching in one sentence."}]
  }'

Browse the available model IDs on the Active Models page, or let Smart Routing pick one for you.

Streaming

Set stream: true to receive tokens as they are generated. This is the recommended mode for anything user-facing or long-form, it lowers time-to-first-token and keeps the connection active for the full generation.

stream = client.chat.completions.create(
    model="z-ai/glm-5.2",
    messages=[{"role": "user", "content": "Write a short story."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Timeouts and long-running requests

The platform allows a single request up to 300 seconds of generation time. Large context windows or long outputs on a non-streaming call can genuinely take minutes, so the most common source of “cut off” errors is a client-side timeout that fires before the response finishes, not the server.

For long outputs, stream the response. Streaming keeps data flowing over the connection, which avoids idle/read timeouts and gives you tokens as they arrive instead of waiting for the whole completion.

If you must run non-streaming for long generations, raise your client’s read timeout:

client = OpenAI(
    api_key="lk_...",
    base_url="https://api.lyceum.technology/openai/v1",
    timeout=600.0,  # seconds; default in most SDKs is 600, but some frameworks set it lower
)

Some frameworks (for example litellm) apply a separate stream inactivity timeout that aborts a stream if no new token arrives within a fixed window. Under heavy load, time-to-first-token can spike, so if you stream through such a framework, raise its inactivity timeout as well as the overall request timeout.

Function calling and tools

All chat models support function / tool calling using the standard OpenAI tools and tool_choice parameters.

resp = client.chat.completions.create(
    model="z-ai/glm-5.2",
    messages=[{"role": "user", "content": "What is the weather in Berlin?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }],
    tool_choice="auto",
)
print(resp.choices[0].message.tool_calls)

A small number of models prefer to answer in text rather than call a tool when tool_choice is left on "auto". If you find a model returning prose instead of a tool call, set tool_choice="required" to force it.

Prompt caching

Prompt caching is automatic, there is nothing to enable. When consecutive requests share an identical leading prefix (for example a fixed system prompt or a large shared context), the cached portion is reused and billed at a reduced input rate. To benefit, keep the stable part of your prompt at the front and vary only the tail (the user’s latest message). Caching is best-effort, a request may or may not hit a warm cache depending on recent traffic. Cache hits are reported in the usage object:

resp = client.chat.completions.create(model="z-ai/glm-5.2", messages=[...])
print(resp.usage.prompt_tokens)                        # total input tokens
print(resp.usage.prompt_tokens_details.cached_tokens)  # portion served from cache

prompt_tokens_details is absent entirely on a cache miss rather than reporting zero, so read it defensively:

details = resp.usage.prompt_tokens_details
cached = details.cached_tokens if details else 0

Reasoning models

Several serverless models produce an explicit reasoning trace before the final answer. The trace is returned in reasoning_content and counts toward output tokens, so set max_tokens high enough to cover both the reasoning and the visible answer. Otherwise the whole budget can be spent on reasoning and content comes back empty or truncated.

Controlling reasoning

Reasoning is toggled through chat_template_kwargs, which Lyceum forwards to the model’s chat template unchanged. The default state and the exact key are model-specific:

Model	Reasoning by default	How to toggle
`deepseek/deepseek-v4-flash-0731`	Off	Enable: `chat_template_kwargs: {"thinking": true}`
`deepseek/deepseek-v4-pro`	Off	Enable: `chat_template_kwargs: {"thinking": true}`
`moonshotai/kimi-k3`	On	Disable: `chat_template_kwargs: {"thinking": false}`
`moonshotai/kimi-k2.6`	On	Disable: `chat_template_kwargs: {"thinking": false}`
`moonshotai/kimi-k2.7-code`	On	Disable: `chat_template_kwargs: {"thinking": false}`
`z-ai/glm-5.2`	On	Disable: `chat_template_kwargs: {"enable_thinking": false}`, `reasoning_effort: "none"`, or use `z-ai/glm-5.2-instant`
`minimax/minimax-m3`	On	Always on, cannot be disabled
`qwen/qwen3.5-9b`	On	Always on, cannot be disabled
`qwen/qwen3.8-2.4t-a95b`	On	Always on, cannot be disabled (`{"thinking": false}` is accepted but ignored)

Example, enabling reasoning on a model that defaults to off:

resp = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash-0731",
    messages=[{"role": "user", "content": "Solve step by step: ..."}],
    extra_body={"chat_template_kwargs": {"thinking": True}},
)

The kwarg key differs by model family: DeepSeek and Kimi use thinking; GLM-5.2 uses enable_thinking. Passing the other key has no effect.
Top-level enable_thinking and top-level reasoning_effort are not honored on these models (the one exception is GLM-5.2, which accepts reasoning_effort: "none" to disable). Use chat_template_kwargs as shown.
reasoning_effort levels (low / high / max) are forwarded to the model verbatim and are not reliably graded. Treat reasoning as on/off, not a depth dial.

GLM-5.2 without reasoning

z-ai/glm-5.2 reasons before answering and spends output tokens on it. Clients that cap max_tokens and don’t render reasoning_content (GitHub Copilot, Cursor and similar BYOK setups) can end up with an empty visible answer because the whole budget goes into reasoning. Two ways to turn it off:

Use the model id z-ai/glm-5.2-instant (also listed in GET /models). Same model, same pricing, answers directly without a reasoning phase. Pick this in clients where you can’t pass request parameters.
Or send reasoning_effort: "none" in the request body when calling z-ai/glm-5.2 through the API.

Embeddings

Text embeddings use the same base URL and key:

emb = client.embeddings.create(
    model="Qwen/Qwen3-Embedding-8B",
    input="text to embed",
)
print(len(emb.data[0].embedding))

Error handling

Errors come back in one of two shapes depending on where they originate. Gateway errors are raised by Lyceum before the request reaches a model: a bad key, an unknown model ID, or insufficient credit. These use a detail string.

{ "detail": "Model 'does/not-exist' is not available. Call GET /models for the list of supported models." }

Upstream errors are forwarded from the model backend unchanged, in the OpenAI-compatible shape:

{
  "error": {
    "message": "Rate limit reached. Please retry shortly.",
    "type": "rate_limit_error",
    "code": 429
  }
}

A client that reads only error.message will show nothing useful for authentication and model-name mistakes, which are the most common setup errors. Check for detail first, then fall back to error.message.

Upstream shapes hold for errors that occur mid-stream as well. Retry on 429 and on 5xx or timeout responses with backoff.

Serverless vs dedicated

Use case	Pick
Low-volume or spiky traffic, pay-per-request, no infra to manage	Serverless (this page)
Steady production traffic, predictable latency, your own model	Dedicated Inference
Let the platform choose a model per request by difficulty	Smart Routing

Active Models

Browse the model IDs and context windows currently available.

Smart Routing

Automatically route each request to the right model.

Getting Started

Serverless

Instances

Inference

Coding Agents

Workloads

Observability

Tools

Configuration

Account

Try it

Endpoint and authentication

Quickstart

Streaming

Timeouts and long-running requests

Function calling and tools

Prompt caching

Reasoning models

Controlling reasoning

GLM-5.2 without reasoning

Embeddings

Error handling

Serverless vs dedicated

Active Models

Smart Routing

​Try it

​Endpoint and authentication

​Quickstart

​Streaming

​Timeouts and long-running requests

​Function calling and tools

​Prompt caching

​Reasoning models

​Controlling reasoning

​GLM-5.2 without reasoning

​Embeddings

​Error handling

​Serverless vs dedicated

Active Models

Smart Routing

Try it

Endpoint and authentication

Quickstart

Streaming

Timeouts and long-running requests

Function calling and tools

Prompt caching

Reasoning models

Controlling reasoning

GLM-5.2 without reasoning

Embeddings

Error handling

Serverless vs dedicated