A model may have several provider channels. Token Market first builds an eligible candidate set from model capabilities, workspace permissions, and current channel status, then Smart Routing selects a channel automatically. This page covers API routing; see Section 3 for provider data, filtering, and table sorting on the model details page.
Token Market API examples use automatic routing by default. Without a fixed provider, the gateway weighs capability, pricing, latency, throughput, and reliability across currently available channels. If a request fails, it continues through the remaining eligible candidates. Available fields and filters vary dynamically by model type, billing method, and live data.
If your workload has explicit routing requirements, open “Unified API Endpoint and code examples” at the bottom of the model details page. The page shows only controls supported by the current model, commonly including:
After you select criteria, the page updates the copyable API example. OpenAI-compatible requests put provider routing conditions in extra_body.provider, for example:
{
"model": "<MODEL_ID>",
"extra_body": {
"provider": {
"only": ["<PROVIDER>"],
"ignore": [],
"sort": ["latency", "output_price"],
"latency_range": [0, 2],
"allow_fallbacks": false
}
}
}
only=[], ignore=[], and order=[] are valid and equivalent to leaving them unset. Each range must contain exactly two nonnegative values, and its lower bound cannot exceed its upper bound; range=[] is invalid. Price dimensions, length limits, and metrics vary by model, so use the request example generated by the page and the units currently shown.
Generate an example request on the model details page and copy the final code. Do not invent unsupported routing fields.
Pricing, latency, throughput, and reliability vary by model, provider, time window, and request conditions. In production, set timeouts, bounded retries, and an appropriate fallback strategy, then verify the actual route in request logs. See Performance metrics for metric definitions.
When setting routing through the API, use extra_body.provider; put only the actual model ID in the model field and do not append routing conditions to it. See Section 2 for fields and fallback behavior.
All public APIs use extra_body.provider for routing conditions. OpenAI-compatible JSON requests pass it as an object inside extra_body; the OpenAI Python SDK uses extra_body={"provider": {...}}, and multipart requests send extra_body as a JSON-string form field.
extra_body={"provider": {"sort": ["latency"]}}
Core fields and their behavior:
| Field | Example | Purpose |
|---|---|---|
sort | ["latency", "output_price"] | Sort candidates by latency, throughput, input/output pricing, or supported input length. |
| Metric range | latency_range: [0, 2] | Filter candidate channels by a metric; provide exactly two nonnegative values in ascending order. |
order | ["<PROVIDER>"] | Give the listed provider priority after sorting. |
Common sort strategies include:
latency: Prioritize lower latency for interactive requests;throughput: Prioritize throughput for concurrent or batch workloads;input_price / output_price: Prioritize input or output pricing for cost-sensitive workloads;input_length: Prioritize supported input length for long-context requests.Provider scope and fallback parameters:
| Parameter | Example | Description |
|---|---|---|
only | ["<PROVIDER_A>", "<PROVIDER_B>"] | Select only from the specified provider set. |
ignore | ["<PROVIDER>"] | Exclude the provider from the candidate set; fallback never relaxes this condition. |
| Metric range | "latency_range": [0, 2] | Provide exactly two nonnegative values; price uses USD per million tokens, latency uses seconds, throughput uses tokens/s, and input length uses tokens. |
order | ["<PROVIDER>"] | Raise the specified provider’s priority after the other routing conditions. |
allow_fallbacks | false | Determines whether the gateway may relax some preferred conditions when no channel matches. |
Complete request example:
curl "$BASE_URL/chat/completions" \
-H "Authorization: Bearer $TOKEN_MARKET_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "<MODEL_ID>",
"messages": [{"role": "user", "content": "Hello"}],
"extra_body": {
"provider": {
"only": ["<PROVIDER>"],
"ignore": [],
"sort": ["latency"],
"latency_range": [0, 2],
"allow_fallbacks": false
}
}
}'
The gateway evaluates user-controlled conditions in this order: ignore → ranges → only → sort → order; after those filters and sorting, order raises the listed providers’ priority. A non-empty only defaults to allow_fallbacks=false; without a provider constraint, the default is true. With false, routing stays within the candidate set formed by only and each range. With true, if the preferred set is empty, the gateway can relax only, ranges, sort, and order, but never ignore, model access permissions, endpoint capabilities, or channel availability. only and ignore cannot overlap. Validation failures return HTTP 400 with invalid_request; if no channel remains, the gateway returns HTTP 503 with no_available_provider.
The gateway supports Session ID only on Chat Completions, Responses, Claude Messages, Gemini Generate Content, and Stream Generate Content. It reads the ID in this order: top-level session_id, extra_body.session_id, then X-Session-Id, and caps it at 256 characters. The ID provides soft sticky routing and upstream cache reuse; it applies only while the current request allows the channel and that channel remains available. The gateway keeps an ID valid for about 1 hour by default and renews it after successful use. Use different Session IDs for different models or unrelated conversations.
No available channel: If no provider satisfies the hard filters (for example,
latency_range), the gateway returns HTTP 503 with the fixed error codeno_available_provider.
At the top of the model details page, filter providers to match your requirements and click a metric heading, such as “Throughput,” to sort by it. Output throughput sorts from highest to lowest by default. The provider table shows the final price range available to the current account; route-level price filters and sorting use base USD prices per million tokens. Prices, latency, throughput, and reliability vary by model, provider, and time.