A FAMILIAR WAY TO BUILD

Send it.
Come back to it.

Prism accepts asynchronous text batches through a subset of the OpenAI Batch API. Ministral and Mistral use OpenRouter’s batch service. Qwen 3.8 27B uses the Prism GPU queue on Modal.

Start your private pilot

Read the pilot offer, billing policy and review workflow ↗

Open the hosted console at prism.autoperf.cloud/console with the Google account approved for your invitation. Ask your operator to assign a live pilot budget. Stripe test credits cannot pay for live inference. Start with one fictional request and check its maximum charge before submitting.

For SDKs, use https://prism.autoperf.cloud/v1 as PRISM_BASE_URL. The Sites address is the product page; it is not an API endpoint.

Batch is a delivery mode, not an answer format

The JSONL file is a transport envelope: one JSON request or response record per line. The model’s answer lives in response.body.choices[0].message.content and can be plain text, Markdown, code, or a JSON string. Omit response_format (or use type: text) for free text. Ask for Markdown or code in the prompt; Prism does not execute code or render model HTML.

{
  "custom_id": "text-001",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "ministral-3-8b",
    "messages": [
      {
        "role": "user",
        "content": "Translate into French. Return only the translation: Your order will arrive on Thursday."
      }
    ],
    "temperature": 0
  }
}

Use json_object only when you need a JSON object, and json_schema when you need supported fields and types checked. A batch may mix output formats while using the same model. Free text is the default.

1. Prepare your requests

Create a UTF-8 JSONL file, with one request per line and a unique custom_id. Use one model per batch.

{
  "custom_id": "product-001",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "ministral-3-8b",
    "messages": [
      {
        "role": "user",
        "content": "Extract a category and a color from: A blue cotton T-shirt. Return a JSON object."
      }
    ],
    "temperature": 0,
    "response_format": {
      "type": "json_object"
    }
  }
}

2. Create a key, then upload

Sign in to the console and create an application key. Store it in your server environment as PRISM_API_KEY. Set PRISM_BASE_URL to this workspace’s HTTPS address followed by /v1.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["PRISM_BASE_URL"],
    api_key=os.environ["PRISM_API_KEY"],
)

file = client.files.create(
    file=open("requests.jsonl", "rb"),
    purpose="batch",
)
batch = client.batches.create(
    input_file_id=file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    # Persist this key and reuse it for retries of this creation.
    extra_headers={"Idempotency-Key": "daily-extraction-2026-09-16"},
)

# Later, retrieve status and download results.
batch = client.batches.retrieve(batch.id)

3. Collect the results

if batch.status in ("completed", "failed", "expired", "cancelled"):
    if batch.output_file_id:
        result = client.files.content(batch.output_file_id)
        result.write_to_file("results.jsonl")
    if batch.error_file_id:
        errors = client.files.content(batch.error_file_id)
        errors.write_to_file("errors.jsonl")

Poll about every 30 seconds. Match responses to your input with custom_id. Errors are delivered separately and failed requests are not charged.

Qwen 3.8 27B batch

Use model qwen-3.8-27b with up to 48,000 UTF-8 bytes of message text per request. Omit max_tokens to use the remaining model context capacity instead of an arbitrary default. Qwen has a 200,512-token context; the Mistral models permit up to 209,715 output tokens within their context. The quote reserves the maximum possible cost, so automatic output can require substantial available credit. Actual accepted tokens determine the final charge. You can optionally set max_tokens to impose your own ceiling. GPU execution also has a finite time budget: automatic output does not guarantee infinite generation. Batches are grouped for GPU execution, with a best-effort 24-hour window. Reasoning is off by default. To enable it, set chat_template_kwargs: {"enable_thinking": true} in the request body. Reasoning shares the output token limit. Infrastructure failures may be retried within the original deadline; each accepted request is charged at most once. Qwen 3.5 9B is retired from new submissions; historical results remain accessible.

Structured extraction

For models that list json_schema in /v1/models/{id}/capabilities, supply a strict object schema. The console offers a structured extraction sample for verified models.

{
  "custom_id": "request-001",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "ministral-3-8b",
    "messages": [
      {
        "role": "user",
        "content": "Extract this fictional request: Lumen Labs needs 12 seats. Priority: high."
      }
    ],
    "temperature": 0,
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "support_request",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "company": {
              "type": "string"
            },
            "seats": {
              "type": "integer"
            },
            "priority": {
              "type": "string",
              "enum": [
                "low",
                "high"
              ]
            }
          },
          "required": [
            "company",
            "seats",
            "priority"
          ],
          "additionalProperties": false
        }
      }
    }
  }
}

Every object property must be required, with additionalProperties set to false. Nullable fields, nested objects, arrays, enums and simple length or numeric bounds are supported. Schemas are limited to 16 KiB; references, regular expressions and remote schemas are not supported.

Prism validates the returned JSON before charging. Invalid JSON, a schema mismatch, a refusal or a truncated structured answer produces an error record with no customer charge. This checks the structure, not whether extracted facts are correct: compare the first results with your source before scaling up.

Pilot limits

100 requests per batch; uploads up to 1 MiB; 10 unfinished batches per workspace.
48,000 UTF-8 bytes of message text per request. Omit max_tokens for automatic output within the model context, or set your own ceiling. Check each model’s capabilities for current limits.
24-hour completion target, best effort. A delayed batch remains visible while the provider finishes.
Queued batches can be cancelled. Once submitted upstream, cancellation is unavailable.
Saved request and response content expires after 7 days. You can request earlier deletion from the console.

Choose your output limit

Omit max_tokens for automatic output within the remaining model context, or set an optional ceiling in each JSONL request. The console can apply one limit to all requests in your draft; an empty Automatic value removes explicit limits when applied. Imported per-request limits are preserved until you apply a change. A larger ceiling increases the maximum quote, not the number of tokens the model must generate.

For Qwen, reasoning and the final answer share this budget. A small limit can end generation before any final answer appears. An output_token_limit error means the answer was incomplete; the failed request is not charged. Increase the limit, request a new quote and decide whether to resubmit. Prism never retries it automatically.

Test with business documents

In Create a batch, choose a model with verified JSON Schema support and load the six business test cases. They cover invoices, support tickets and returns, with corrections, mixed VAT and partial refunds. These are synthetic documents, not customer records. Download the expected fields and calculated totals to compare your results. Review the quote before submitting; running examples uses your pilot credits.

Predictable spending

The console shows a conservative maximum charge before submission. Submitting the batch reserves this amount; asking for a quote does not reserve credits. Once processing ends, the debit uses actual input and generated output tokens from accepted results, including reasoning tokens. The unused reserve is released. A larger max_tokens limit does not itself increase the final debit for identical usage. The quote is the maximum customer charge. Cached input uses the published input price. Payment and live inference balances are isolated during this pilot.

Retries and recovery

Use an Idempotency-Key header when creating batches. Repeating the same key and payload returns the existing job; a changed payload returns a conflict. Without an Idempotency-Key, each create request makes a new batch, even when it reuses the same file. Metadata is preserved (up to 16 string pairs). Ambiguous provider submissions are paused for operator review instead of being resent.

Capabilities and errors

Check /v1/capabilities and each model’s capability endpoint before submitting. Public batch states use the OpenAI names; prism.phase adds queue and operator-review details. expires_at is null when no hard deadline is enforced; target_completion_at is the best-effort target.

API errors contain error.message, error.type, error.code and error.param. New uploads preserve their original bytes. Deleting a file removes that Prism file; use DELETE /v1/batches/{id} to request deletion of batch content and upstream artifacts.

POST /v1/batch-quotes with input_file_id returns a signed quote valid for 15 minutes. Pass its id as quote_id when creating the batch. Quoting does not reserve or spend credits.

Supported API

GET    /v1/capabilities
GET    /v1/models
GET    /v1/models/{id}
GET    /v1/models/{id}/capabilities
POST   /v1/batch-quotes
GET    /v1/files
POST   /v1/files
GET    /v1/files/{id}
DELETE /v1/files/{id}
GET    /v1/files/{id}/content
POST   /v1/batches
GET    /v1/batches
GET    /v1/batches/{id}
POST   /v1/batches/{id}/cancel
DELETE /v1/batches/{id}

The API specification is served by your workspace server at /v1/openapi.json. Streaming chat, tool calls, media inputs and webhooks for batch completion are outside this initial product.

Open your workspace ↗

Send it.Come back to it.