# Streaming responses

> Stream tokens from BigModel.io over server-sent events on Chat Completions, Messages and Responses, read the final usage chunk, and handle mid-stream errors.

Web version: https://bigmodel.io/docs/streaming · Updated 2026-10-18

Set `"stream": true` and BigModel.io returns the reply as server-sent events (SSE) while the model generates it. Streaming works the same way on every text endpoint your key's group serves.

| Endpoint | Turn streaming on | Stream ends with |
|---|---|---|
| `POST /v1/chat/completions` | `"stream": true` | `data: [DONE]` |
| `POST /v1/messages` | `"stream": true` | `event: message_stop` |
| `POST /v1/responses` | `"stream": true` | a `response.completed` event |

Which endpoints a key can call depends on its group. GLM and DeepSeek groups serve Chat Completions only.

## Stream a chat completion

```bash [cURL]
curl -N https://bigmodel.io/v1/chat/completions \
  -H "Authorization: Bearer $BIGMODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3",
    "stream": true,
    "stream_options": {"include_usage": true},
    "messages": [{"role": "user", "content": "Write a haiku about rivers."}]
  }'
```

```python [Python]
import os
from openai import OpenAI

client = OpenAI(base_url="https://bigmodel.io/v1", api_key=os.environ["BIGMODEL_API_KEY"])

stream = client.chat.completions.create(
    model="glm-5.3",
    messages=[{"role": "user", "content": "Write a haiku about rivers."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)
```

```js [Node.js]
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://bigmodel.io/v1", apiKey: process.env.BIGMODEL_API_KEY });

const stream = await client.chat.completions.create({
  model: "glm-5.3",
  messages: [{ role: "user", content: "Write a haiku about rivers." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  if (chunk.usage) console.log("\n", chunk.usage);
}
```

Each event is a `data:` line holding a `chat.completion.chunk` JSON object. Text arrives in `choices[0].delta.content`.

## The usage chunk

BigModel.io needs token counts to bill a streamed request, so on GLM and DeepSeek groups it always turns `stream_options.include_usage` on, even if you did not ask. The last chunk before `[DONE]` then has an empty `choices` array and a `usage` object:

```json
{"object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":21,"total_tokens":35}}
```

Your code must not assume `choices[0]` exists in every chunk. The examples above check first. On other groups, send `stream_options: {"include_usage": true}` yourself if you want usage in the stream.

With LangChain's `ChatOpenAI` and a custom base URL, pass `stream_usage=True` to get usage metadata.

## Stream with the Messages API

On groups that serve `/v1/messages`, streaming uses Anthropic's event types: `message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta` (with output usage) and `message_stop`.

```bash [cURL]
curl -N https://bigmodel.io/v1/messages \
  -H "Authorization: Bearer $BIGMODEL_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"glm-5.3","max_tokens":256,"stream":true,"messages":[{"role":"user","content":"Write a haiku about rivers."}]}'
```

```python [Python]
import os
from anthropic import Anthropic

client = Anthropic(base_url="https://bigmodel.io", api_key=os.environ["BIGMODEL_API_KEY"])

with client.messages.stream(
    model="glm-5.3",
    max_tokens=256,
    messages=[{"role": "user", "content": "Write a haiku about rivers."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    print("\n", stream.get_final_message().usage)
```

```js [Node.js]
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ baseURL: "https://bigmodel.io", apiKey: process.env.BIGMODEL_API_KEY });

const stream = client.messages.stream({
  model: "glm-5.3",
  max_tokens: 256,
  messages: [{ role: "user", content: "Write a haiku about rivers." }],
});
stream.on("text", (text) => process.stdout.write(text));
const final = await stream.finalMessage();
console.log("\n", final.usage);
```

The Anthropic SDK base URL is the root `https://bigmodel.io`, without `/v1`.

## Errors during a stream

Errors found before the first byte are returned as a normal HTTP error with a JSON body. Once the stream has started, the status is already `200`, so a later failure arrives as an SSE `error` event and the stream ends. Treat a stream that ends without `[DONE]` or `message_stop` as incomplete.

If you send a `models` fallback list, fallback can only happen before any bytes are sent. Keep-alive pings count as bytes, so once a stream is open no other model is tried. See [OpenRouter compatibility](https://bigmodel.io/docs/openrouter-compat).

## Verify the setup

Run the cURL example. You should see many `data:` lines arrive one by one, a final chunk with `"choices":[]` and `usage`, then `data: [DONE]`. In [Usage](https://bigmodel.io/usage), the request shows the stream type and a first-token time.

## Troubleshooting

**`IndexError` or `Cannot read properties of undefined` on `choices[0]`.** The usage chunk has an empty `choices` array. Check `chunk.choices` before indexing, as in the examples.

**The whole reply arrives at once.** A proxy between you and BigModel.io is buffering. Use `curl -N` to rule out your client, then turn off response buffering in your own proxy (for nginx, `proxy_buffering off`). BigModel.io already sends `X-Accel-Buffering: no`.

**400 `invalid_request_error` mentioning the stream type.** `stream` must be a JSON boolean (`true`), not the string `"true"`.

**The stream stops with an `error` event.** The upstream failed after the stream started. Retry the request. If it keeps happening for one model, check [Status](https://bigmodel.io/status).

**404 on `/v1/messages` or `/v1/responses`.** The key's group serves Chat Completions only. Stream with `/v1/chat/completions` instead.

**503 `api_error` "No available accounts".** The model ID is misspelled or not served to your key's group. See [Model IDs](https://bigmodel.io/docs/model-ids).

## Related

- [Tool calling](https://bigmodel.io/docs/tool-calling): streamed tool calls arrive in fragments.
- [Reasoning & thinking](https://bigmodel.io/docs/reasoning): `reasoning_content` deltas in streams.
- [Chat Completions API reference](https://bigmodel.io/docs/api-chat-completions) and [Messages API reference](https://bigmodel.io/docs/api-messages).
- [Errors](https://bigmodel.io/docs/errors) for every error code.