A FAMILIAR WAY TO BUILD

Send it.
Come back to it.

Prism accepts asynchronous text batches through a subset of the OpenAI Batch API. The private pilot forwards inference to OpenRouter’s native batch service.

Start your private pilot

Read the pilot offer, billing policy and review workflow ↗

Open the hosted console at prism.autoperf.cloud/console with the Google account approved for your invitation. Ask your operator to assign a live pilot budget. Stripe test credits cannot pay for live inference. Start with one fictional request and check its maximum charge before submitting.

For SDKs, use https://prism.autoperf.cloud/v1 as PRISM_BASE_URL. The Sites address is the product page; it is not an API endpoint.

Batch is a delivery mode, not an answer format

The JSONL file is a transport envelope: one JSON request or response record per line. The model’s answer lives in response.body.choices[0].message.content and can be plain text, Markdown, code, or a JSON string. Omit response_format (or use type: text) for free text. Ask for Markdown or code in the prompt; Prism does not execute code or render model HTML.

{
  "custom_id": "text-001",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "ministral-3-8b",
    "messages": [
      {
        "role": "user",
        "content": "Translate into French. Return only the translation: Your order will arrive on Thursday."
      }
    ],
    "max_tokens": 256,
    "temperature": 0
  }
}

Use json_object only when you need a JSON object, and json_schema when you need supported fields and types checked. A batch may mix output formats while using the same model. Free text is the default.

1. Prepare your requests

Create a UTF-8 JSONL file, with one request per line and a unique custom_id. Use one model per batch.

{
  "custom_id": "product-001",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "ministral-3-8b",
    "messages": [
      {
        "role": "user",
        "content": "Extract a category and a color from: A blue cotton T-shirt. Return a JSON object."
      }
    ],
    "max_tokens": 256,
    "temperature": 0,
    "response_format": {
      "type": "json_object"
    }
  }
}

2. Create a key, then upload

Sign in to the console and create an application key. Store it in your server environment as PRISM_API_KEY. Set PRISM_BASE_URL to this workspace’s HTTPS address followed by /v1.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["PRISM_BASE_URL"],
    api_key=os.environ["PRISM_API_KEY"],
)

file = client.files.create(
    file=open("requests.jsonl", "rb"),
    purpose="batch",
)
batch = client.batches.create(
    input_file_id=file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    # Persist this key and reuse it for retries of this creation.
    extra_headers={"Idempotency-Key": "daily-extraction-2026-09-16"},
)

# Later, retrieve status and download results.
batch = client.batches.retrieve(batch.id)

3. Collect the results

if batch.status in ("completed", "failed", "expired", "cancelled"):
    if batch.output_file_id:
        result = client.files.content(batch.output_file_id)
        result.write_to_file("results.jsonl")
    if batch.error_file_id:
        errors = client.files.content(batch.error_file_id)
        errors.write_to_file("errors.jsonl")

Poll about every 30 seconds. Match responses to your input with custom_id. Errors are delivered separately and failed requests are not charged.

Structured extraction

For models that list json_schema in /v1/models/{id}/capabilities, supply a strict object schema. The console offers a structured extraction sample for verified models.

{
  "custom_id": "request-001",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "ministral-3-8b",
    "messages": [
      {
        "role": "user",
        "content": "Extract this fictional request: Lumen Labs needs 12 seats. Priority: high."
      }
    ],
    "max_tokens": 256,
    "temperature": 0,
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "support_request",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "company": {
              "type": "string"
            },
            "seats": {
              "type": "integer"
            },
            "priority": {
              "type": "string",
              "enum": [
                "low",
                "high"
              ]
            }
          },
          "required": [
            "company",
            "seats",
            "priority"
          ],
          "additionalProperties": false
        }
      }
    }
  }
}

Every object property must be required, with additionalProperties set to false. Nullable fields, nested objects, arrays, enums and simple length or numeric bounds are supported. Schemas are limited to 16 KiB; references, regular expressions and remote schemas are not supported.

Prism validates the returned JSON before charging. Invalid JSON, a schema mismatch, a refusal or a truncated structured answer produces an error record with no customer charge. This checks the structure, not whether extracted facts are correct: compare the first results with your source before scaling up.

Pilot limits

100 requests per batch; uploads up to 1 MiB; 10 unfinished batches per workspace.
48,000 UTF-8 bytes of message text per request. You choose max_tokens: up to 16,384 for Qwen 3.5 9B, or 2,048 for the Mistral models. Check each model’s capabilities for the current ceiling.
24-hour completion target, best effort. A delayed batch remains visible while the provider finishes.
Queued batches can be cancelled. Once submitted upstream, cancellation is unavailable.
Saved request and response content expires after 7 days. You can request earlier deletion from the console.

Choose your output limit

Set max_tokens in each JSONL request. If omitted, it defaults to 256. The console can apply one limit to all requests in your draft; imported per-request limits are preserved until you explicitly apply a change. Raising the limit increases the maximum quote, not the number of tokens the model must generate.

For Qwen, reasoning and the final answer share this budget. A small limit can end generation before any final answer appears. An output_token_limit error means the answer was incomplete; the failed request is not charged. Increase the limit, request a new quote and decide whether to resubmit. Prism never retries it automatically.

Test with business documents

In Create a batch, choose a model with verified JSON Schema support and load the six business test cases. They cover invoices, support tickets and returns, with corrections, mixed VAT and partial refunds. These are synthetic documents, not customer records. Download the expected fields and calculated totals to compare your results. Review the quote before submitting; running examples uses your pilot credits.

Predictable spending

The console shows a conservative maximum charge before submission. Submitting the batch reserves this amount; asking for a quote does not reserve credits. Once processing ends, the debit uses actual input and generated output tokens from accepted results, including reasoning tokens. The unused reserve is released. A larger max_tokens limit does not itself increase the final debit for identical usage. The quote is the maximum customer charge. Cached input uses the published input price. Payment and live inference balances are isolated during this pilot.

Retries and recovery

Use an Idempotency-Key header when creating batches. Repeating the same key and payload returns the existing job; a changed payload returns a conflict. Without an Idempotency-Key, each create request makes a new batch, even when it reuses the same file. Metadata is preserved (up to 16 string pairs). Ambiguous provider submissions are paused for operator review instead of being resent.

Capabilities and errors

Check /v1/capabilities and each model’s capability endpoint before submitting. Public batch states use the OpenAI names; prism.phase adds queue and operator-review details. expires_at is null when no hard deadline is enforced; target_completion_at is the best-effort target.

API errors contain error.message, error.type, error.code and error.param. New uploads preserve their original bytes. Deleting a file removes that Prism file; use DELETE /v1/batches/{id} to request deletion of batch content and upstream artifacts.

POST /v1/batch-quotes with input_file_id returns a signed quote valid for 15 minutes. Pass its id as quote_id when creating the batch. Quoting does not reserve or spend credits.

Supported API

GET    /v1/capabilities
GET    /v1/models
GET    /v1/models/{id}
GET    /v1/models/{id}/capabilities
POST   /v1/batch-quotes
GET    /v1/files
POST   /v1/files
GET    /v1/files/{id}
DELETE /v1/files/{id}
GET    /v1/files/{id}/content
POST   /v1/batches
GET    /v1/batches
GET    /v1/batches/{id}
POST   /v1/batches/{id}/cancel
DELETE /v1/batches/{id}

The API specification is served by your workspace server at /v1/openapi.json. Streaming chat, tool calls, media inputs and webhooks for batch completion are outside this initial product.

Open your workspace ↗

Send it.Come back to it.