Batch Requests

The Batch API allows you to submit large numbers of requests to be processed asynchronously. This is ideal for tasks that don't require immediate responses, such as data processing, evaluation, and embedding generation.

How It Works

Create a batch — Upload a JSONL file with your requests
Submit the batch — Start processing
Monitor progress — Check status until completion
Retrieve results — Download the output file

Creating a Batch Input File

Prepare a JSONL file where each line is a valid request object:

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "deepseek/deepseek-v4-pro", "messages": [{"role": "user", "content": "Hello!"}]}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "deepseek/deepseek-v4-pro", "messages": [{"role": "user", "content": "How are you?"}]}}
{"custom_id": "request-3", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "z-ai/glm-5.1", "messages": [{"role": "user", "content": "Translate to French: Hello"}]}}

Request Format

Field	Type	Description
`custom_id`	string	Your identifier for tracking this request
`method`	string	HTTP method (`POST`)
`url`	string	API endpoint path (e.g., `/v1/chat/completions`)
`body`	object	Request body (same as the synchronous API)

Uploading and Creating a Batch

# Step 1: Upload the input file
curl https://openapi.linkwo.ai/v1/files \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "purpose=batch" \
  -F "file=@batch_input.jsonl"

# Step 2: Create the batch
curl https://openapi.linkwo.ai/v1/batches \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Checking Batch Status

curl https://openapi.linkwo.ai/v1/batches/batch_abc123 \
  -H "Authorization: Bearer YOUR_API_KEY"

Status Values

Status	Description
`validating`	Input file is being validated
`in_progress`	Requests are being processed
`completed`	All requests finished
`failed`	Batch failed
`expired`	Batch exceeded the completion window

Retrieving Results

Once the batch is completed, download the output file:

curl https://openapi.linkwo.ai/v1/files/file_output_abc123/content \
  -H "Authorization: Bearer YOUR_API_KEY" > batch_output.jsonl

Each line in the output corresponds to an input request:

{"id": "batch_req_abc", "custom_id": "request-1", "response": {"status_code": 200, "body": {"id": "chatcmpl-abc", "choices": [{"message": {"content": "Hello! How can I help you?"}}]}}}

Limits

Limit	Value
Max requests per batch	50,000
Max input file size	200 MB
Completion window	Up to 24 hours
Max concurrent batches	10

Use Cases

Dataset evaluation — Run prompts against a benchmark dataset
Bulk embeddings — Generate embeddings for a large document corpus
Content generation — Process templates in bulk
Model comparison — Run the same prompts across different models

Best Practices

Use custom_id to match outputs back to your input records
Start with small batches to validate before scaling up
Monitor batch status and set up polling with exponential backoff
Split very large workloads into multiple batches

Chat Completions — Synchronous chat API
Embeddings — Embedding generation API

Structured Output

OpenAI SDK