Smart Routing

A model may have several provider channels. Token Market first builds an eligible candidate set from model capabilities, workspace permissions, and current channel status; Smart Routing then selects a channel automatically. This page covers API routing configuration. Section 2 below explains how to view, filter, and sort provider data on the model details page.

1. Configure Smart Routing for your workload

Token Market API examples use automatic routing by default. Without a fixed provider, the gateway weighs capability, pricing, latency, throughput, and reliability across currently available channels. If a request fails, it continues through the remaining eligible candidates. Available fields and filters vary dynamically by model type, billing method, and live data.

If your workload has explicit routing requirements, open “Unified API Endpoint and code examples” at the bottom of the model details page. The page shows only controls supported by the current model, commonly including:

• Input pricing, output pricing, or another model-specific price dimension;
• Maximum input and output lengths;
• Latency and throughput;
• Thinking and non-thinking options for models that support thinking mode.

After you select criteria, the page updates the copyable API example. OpenAI-compatible requests place these criteria in extra_body.provider, using the structure below. An empty array leaves that criterion unrestricted:

{
  "only": [],
  "order": [],
  "sort": null,
  "input_price_range": [],
  "output_price_range": [],
  "input_length_range": [],
  "output_length_range": [],
  "throughput_range": [],
  "latency_range": []
}

Page controls generate range values, and the displayed units apply. Price dimensions, length limits, and quality metrics vary by model type; do not copy one model’s complete field set into every request.

Generate an example request on the model details page and copy the final code. Do not invent unsupported routing fields.

Pricing, latency, throughput, and reliability vary by model, provider, time window, and request conditions. In production, set timeouts, bounded retries, and an appropriate fallback strategy, then verify the actual route in request logs. See Performance metrics for metric definitions.

Use the “model-string routing syntax” below when a request must specify a sorting preference quickly. It provides the same class of routing controls as extra_body.provider generated on the model details page. For production, prefer a page-generated request that you have validated.

2. Specify sorting in the model string

To keep the request body unchanged, add the sort policy and filters to the model string:

Model ID:sort strategy:filter parameters

For example, this request prefers lower-latency channels and restricts candidates to the listed providers:

DeepSeek-R1:latency:only=SiliconFlow,Alibaba Cloud Bailian,latency<500

The syntax has three parts:

Common sort policies include:

• latency: Prioritize lower latency for interactive requests;
• throughput: Prioritize throughput for concurrent or batch workloads;
• input_price / output_price: Prioritize input or output pricing for cost-sensitive workloads;
• input_length: Prioritize supported input length for long-context requests.

Common filters and fallback parameters include:

Complete request example:

curl ${base_url}/chat/completions \
  -H "Authorization: Bearer $TOKEN_MARKET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "DeepSeek-R1:latency:only=SiliconFlow,Alibaba Cloud Bailian,latency<500",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

When parsing, the final parameter section takes precedence. Colons separate the model, policy, and filter section; ASCII commas separate conditions within the filter section. Do not encode each filter as another colon-delimited section. Model names, policies, and provider names must match values supported by the current gateway. If routing options in the model string conflict with extra_body.provider, the gateway strictly gives the structured extra_body.provider object highest priority and ignores the corresponding options in the model string.

Production error behavior: If no candidate provider can satisfy every hard filter (for example, latency<200), the gateway returns HTTP 503 with the fixed error code no_available_provider.

3. Review provider data for your requirements

At the top of the model details page, filter providers to match your requirements and select a column heading, such as “Throughput,” to sort by that metric. The table sorts by output throughput from highest to lowest by default.