Streaming responses

Set "stream": true and BigModel.io returns the reply as server-sent events (SSE) while the model generates it. Streaming works the same way on every text endpoint your key's group serves.

Which endpoints a key can call depends on its group. GLM and DeepSeek groups serve Chat Completions only.

#Stream a chat completion

curl -N https://bigmodel.io/v1/chat/completions \
  -H "Authorization: Bearer $BIGMODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [{"role": "user", "content": "Write a haiku about rivers."}]
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://bigmodel.io/v1", api_key=os.environ["BIGMODEL_API_KEY"])

stream = client.chat.completions.create(
    model="glm-5.3",
    messages=[{"role": "user", "content": "Write a haiku about rivers."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://bigmodel.io/v1", apiKey: process.env.BIGMODEL_API_KEY });

const stream = await client.chat.completions.create({
  model: "glm-5.3",
  messages: [{ role: "user", content: "Write a haiku about rivers." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) console.log("\n", chunk.usage);
}

Each event is a data: line holding a chat.completion.chunk JSON object. Text arrives in choices[0].delta.content.

#The usage chunk

BigModel.io needs token counts to bill a streamed request, so on GLM and DeepSeek groups it always turns stream_options.include_usage on, even if you did not ask. The last chunk before [DONE] then has an empty choices array and a usage object:

{"object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":21,"total_tokens":35}}

Your code must not assume choices[0] exists in every chunk. The examples above check first. On other groups, send stream_options: {"include_usage": true} yourself if you want usage in the stream.

With LangChain's ChatOpenAI and a custom base URL, pass stream_usage=True to get usage metadata.

#Stream with the Messages API

On groups that serve /v1/messages, streaming uses Anthropic's event types: message_start, content_block_start, content_block_delta, content_block_stop, message_delta (with output usage) and message_stop.

curl -N https://bigmodel.io/v1/messages \
  -H "Authorization: Bearer $BIGMODEL_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"glm-5.3","max_tokens":256,"stream":true,"messages":[{"role":"user","content":"Write a haiku about rivers."}]}'

import os
from anthropic import Anthropic

client = Anthropic(base_url="https://bigmodel.io", api_key=os.environ["BIGMODEL_API_KEY"])

with client.messages.stream(
    model="glm-5.3",
    max_tokens=256,
    messages=[{"role": "user", "content": "Write a haiku about rivers."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    print("\n", stream.get_final_message().usage)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ baseURL: "https://bigmodel.io", apiKey: process.env.BIGMODEL_API_KEY });

const stream = client.messages.stream({
  model: "glm-5.3",
  max_tokens: 256,
  messages: [{ role: "user", content: "Write a haiku about rivers." }],
});
stream.on("text", (text) => process.stdout.write(text));
const final = await stream.finalMessage();
console.log("\n", final.usage);

The Anthropic SDK base URL is the root https://bigmodel.io, without /v1.

#Errors during a stream

Errors found before the first byte are returned as a normal HTTP error with a JSON body. Once the stream has started, the status is already 200, so a later failure arrives as an SSE error event and the stream ends. Treat a stream that ends without [DONE] or message_stop as incomplete.

If you send a models fallback list, fallback can only happen before any bytes are sent. Keep-alive pings count as bytes, so once a stream is open no other model is tried. See OpenRouter compatibility.

#Verify the setup

Run the cURL example. You should see many data: lines arrive one by one, a final chunk with "choices":[] and usage, then data: [DONE]. In Usage, the request shows the stream type and a first-token time.

#Troubleshooting

IndexError or Cannot read properties of undefined on choices[0]. The usage chunk has an empty choices array. Check chunk.choices before indexing, as in the examples.

The whole reply arrives at once. A proxy between you and BigModel.io is buffering. Use curl -N to rule out your client, then turn off response buffering in your own proxy (for nginx, proxy_buffering off). BigModel.io already sends X-Accel-Buffering: no.

400 invalid_request_error mentioning the stream type. stream must be a JSON boolean (true), not the string "true".

The stream stops with an error event. The upstream failed after the stream started. Retry the request. If it keeps happening for one model, check Status.

404 on /v1/messages or /v1/responses. The key's group serves Chat Completions only. Stream with /v1/chat/completions instead.

503 api_error "No available accounts". The model ID is misspelled or not served to your key's group. See Model IDs.

#Related

• Tool calling: streamed tool calls arrive in fragments.
• Reasoning & thinking: reasoning_content deltas in streams.
• Chat Completions API reference and Messages API reference.
• Errors for every error code.