Streaming responses with SSE

Stream output token by token with stream=true: SSE event format, the uttapen.meta cost event, client disconnects, and proxying from Next.js or Laravel.

With stream: true the model's answer reaches you piece by piece as it is generated, instead of arriving as one JSON blob at the end. For chat UIs, CLI tools, and anywhere time-to-first-token matters, this is the mode you want by default. The gateway puts no buffer between the provider and you — every event is flushed the moment it arrives.

Event format

The response has Content-Type: text/event-stream, and each event is a single data: line carrying a chat.completion.chunk JSON object. The last chunk before [DONE] also carries the usage block with the token counts. A : processing line is sent first; it is only a keep-alive comment and every SDK ignores it.

curl -N https://api.uttapen.ir/v1/chat/completions \
  -H "Authorization: Bearer $UTTAPEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-5-mini","messages":[{"role":"user","content":"Hello"}],"stream":true}'

: processing

data: {"id":"gen-2ebb2e50a57ff83a","object":"chat.completion.chunk","model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello"},"finish_reason":null}]}

data: {"id":"gen-2ebb2e50a57ff83a","object":"chat.completion.chunk","model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{"content":"! How can I help"},"finish_reason":null}]}

data: {"id":"gen-2ebb2e50a57ff83a","object":"chat.completion.chunk","model":"openai/gpt-5-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":24,"total_tokens":34}}

data: [DONE]

The -N flag turns off curl's own output buffering. Without it you will be convinced streaming is broken.

The uttapen.meta event

The format above is exactly OpenAI's, so that no strict SDK breaks on it. If you also want the toman cost of that particular response, send the X-Uttapen-Include-Meta: 1 header. One extra event then arrives immediately before [DONE]:

data: {"id":"gen-2ebb2e50a57ff83a","object":"uttapen.meta","cost_toman":"6.659688","balance_toman":"97557.754121","hold_toman":"100.000000"}

cost_toman — the amount deducted from your wallet, to six decimal places.
balance_toman — your balance after settlement.
hold_toman — the amount that was reserved before the request went out; useful for tuning max_tokens.

This event has no choices. The Python, Node, and Go SDKs accept it as a chunk without choices, and you can single it out with object == "uttapen.meta". The PHP SDK (openai-php/client) throws on it — in PHP, send the header only when you are driving the request with raw Guzzle (details).

Python

from openai import OpenAI

client = OpenAI(base_url="https://api.uttapen.ir/v1", api_key="sk-up-...")

stream = client.chat.completions.create(
    model="openai/gpt-5-mini",
    messages=[{"role": "user", "content": "Write a short poem about Postgres."}],
    stream=True,
    extra_headers={"X-Uttapen-Include-Meta": "1"},
)
for chunk in stream:
    if chunk.object == "uttapen.meta":
        print("\nCost:", chunk.model_extra["cost_toman"], "toman")
        continue
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\nOutput tokens:", chunk.usage.completion_tokens)

Node.js

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.uttapen.ir/v1", apiKey: process.env.UTTAPEN_API_KEY });

const stream = await client.chat.completions.create(
  {
    model: "openai/gpt-5-mini",
    messages: [{ role: "user", content: "Write a short poem about Postgres." }],
    stream: true,
  },
  { headers: { "X-Uttapen-Include-Meta": "1" } },
);

for await (const chunk of stream) {
  if (chunk.object === "uttapen.meta") {
    console.log("\nCost:", chunk.cost_toman, "toman");
    continue;
  }
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

To cancel a stream from the client side, pass an AbortController through the options (signal), or call close() on the stream object in Python.

Disconnects and billing

When a client hangs up mid-stream, the gateway cancels the upstream request immediately so no further tokens are produced. The tokens generated up to that point, however, really did cost money. Here is what happens:

If we already have the generation id (it arrives with the first chunk), the request moves to reconciling and a background job asks the provider for the actual cost and settles that exact amount. This usually takes less than a minute.
If not a single byte had arrived yet, the full reservation is released and you are charged nothing.

So hanging up is not a way to dodge the bill — but you are never charged more than you actually consumed either. Until settlement completes, the reserved amount stays in held_toman and is subtracted from your usable balance.

If the upstream sends no bytes at all for 120 seconds, the connection is closed with 504 upstream_timeout and the same reconcile path runs. With reasoning models that think for a long time, use stream: true so that the arriving reasoning chunks keep resetting this watchdog.

Concurrent streams

Each account may hold at most 10 open streams at once; the eleventh gets 429 too_many_concurrent_streams. If you run a multi-user chat app, remember that this limit applies to your backend, not to your individual end users — put a small queue in front of the gateway, or see rate limits for the other options.

Proxying a stream to the browser

Your API key must never reach the browser. Instead, your server consumes the stream from uttapen and relays it to the browser.

Next.js (route handler):

// app/api/chat/route.ts
export const runtime = "nodejs";

export async function POST(req: Request) {
  const { messages } = await req.json();
  const upstream = await fetch("https://api.uttapen.ir/v1/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.UTTAPEN_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ model: "openai/gpt-5-mini", messages, stream: true }),
  });
  return new Response(upstream.body, {
    status: upstream.status,
    headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "X-Accel-Buffering": "no" },
  });
}

If you are on the Vercel AI SDK, reach for createOpenAICompatible instead of relaying by hand (integrations).

Laravel (StreamedResponse with Guzzle):

Route::post('/chat', function (Request $request) {
    return response()->stream(function () use ($request) {
        $res = (new \GuzzleHttp\Client())->post('https://api.uttapen.ir/v1/chat/completions', [
            'headers' => ['Authorization' => 'Bearer ' . config('services.uttapen.key')],
            'json' => ['model' => 'openai/gpt-5-mini', 'messages' => $request->input('messages'), 'stream' => true],
            'stream' => true,
        ]);
        $body = $res->getBody();
        while (!$body->eof()) {
            echo $body->read(1024);
            if (ob_get_level() > 0) ob_flush();
            flush();
        }
    }, 200, ['Content-Type' => 'text/event-stream', 'Cache-Control' => 'no-cache', 'X-Accel-Buffering' => 'no']);
});

In both cases, set proxy_buffering off; on your own nginx, or the browser will receive everything in one lump at the end. The X-Accel-Buffering: no header achieves the same thing per route.