Batches

Run offline or long-running jobs using KushRouter’s batch APIs. Batches let you upload a JSONL file of requests and process them asynchronously while receiving status updates and final results.

OpenAI-compatible batches API

Endpoint family root: /api/openai/v1/batches

Create a batch

curl -X POST "https://api.kushrouter.com/api/openai/v1/batches" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_123",
    "metadata": { "label": "blog-drafts" },
    "settings": { "concurrency": 4 }
  }'

JSONL schema (input_file)

Each line in the JSONL file represents a single request, typically mirroring a Chat Completions request body.

{"model":"gpt-5-mini-2025-08-07","messages":[{"role":"user","content":"Summarize this: ..."}]}
{"model":"gpt-5-mini-2025-08-07","messages":[{"role":"user","content":"Generate a title"}]}

With tools (function calling):

{"model":"gpt-5-mini-2025-08-07","messages":[{"role":"user","content":"Weather in Boston"}],
 "tools":[{"type":"function","function":{"name":"get_weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]}

Key fields:

input_file_id – a file previously uploaded via the Files API (JSONL). Mutually exclusive with requests.
requests – inline array of request objects.
metadata – optional structured metadata stored with the batch.
settings.concurrency – optional concurrency hint.

Inline create example:

curl -X POST "https://api.kushrouter.com/api/openai/v1/batches" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      { "model": "gpt-5-mini-2025-08-07", "messages": [{"role":"user","content":"Say hi"}] }
    ]
  }'

List batches

curl -X GET "https://api.kushrouter.com/api/openai/v1/batches" \
  -H "Authorization: Bearer $API_KEY"

Retrieve status

curl -X GET "https://api.kushrouter.com/api/openai/v1/batches/batch_abc123" \
  -H "Authorization: Bearer $API_KEY"

Cancel

curl -X POST "https://api.kushrouter.com/api/openai/v1/batches/batch_abc123/cancel" \
  -H "Authorization: Bearer $API_KEY"

Results (JSON)

curl -X GET "https://api.kushrouter.com/api/openai/v1/batches/batch_abc123/results?offset=0&limit=100" \
  -H "Authorization: Bearer $API_KEY"

Returns { id, status, total, offset, limit, results[] } where each result has { index, status, result, usage?, error?, custom_id? }.

Statuses per item:

succeeded — result contains the endpoint response payload
failed — error contains { message, code, status }

Export (NDJSON or ZIP)

curl -X GET "https://api.kushrouter.com/api/openai/v1/batches/batch_abc123/export?format=ndjson&gzip=true&offset=0&limit=1000" \
  -H "Authorization: Bearer $API_KEY" \
  -o batch-results.ndjson.gz

format can be ndjson (default) or zip. Use gzip=true to gzip NDJSON.

NDJSON export emits one JSON document per line. Use streaming parsers for large files.

Limits

Max items per batch: 50,000 (typical; contact support for higher limits)
Max JSONL line size: 256 KB (after encoding)
Max total batch size: 500 MB (including metadata)

Large batches may be throttled internally; prefer moderate sizes and run multiple batches in parallel if needed.

OpenAI compatibility notes

The batches endpoints mirror OpenAI's object shapes (/batches, /batches/{id}, /batches/{id}/results).
Each item request should correspond to an OpenAI-compatible Chat request body.
Tool calling, response_format, and streaming flags are accepted per item but results are persisted as JSON (not SSE).

Monitoring progress

Batch status values: queued, processing, completed, cancelled, expired, failed.
Use the dashboard’s “Batches” tab for real-time progress, throughput, and retry counts.
NDJSON exports can be parsed using the examples in the developer docs.