Documentation

Quickstart

Four steps in the dashboard, then one line in your code. Add a provider (an API key you hold with OpenAI, Anthropic, Gemini or an OpenAI-compatible server), add an endpoint and draw its route in the editor, then open the endpoint's page from the Endpoints page (its Keys button), mint a gateway key there and copy the endpoint's URL. Point your SDK at that URL, https://modelwaygate.com/USER_ID/ENDPOINT/api/v1, in place of the provider's own. The OpenAI SDKs need nothing else: the gateway speaks their dialect and translates it for whichever provider the endpoint's route reaches.

Request Headers

• Base URL: https://modelwaygate.com/USER_ID/ENDPOINT/api/v1, which the Endpoints page shows in full for each endpoint, with a Copy button beside it. The OpenAI SDKs take it as base_url (Python) or baseURL (TypeScript).
• USER_ID and ENDPOINT: USER_ID is your account's public id, 12 characters given to the account when it is made and never changed. ENDPOINT is the slug you gave the endpoint when you added it (lowercase letters, digits and dashes), which never changes either; renaming an endpoint changes only the name the dashboard shows.
• model: every dialect requires the field and the SDKs always send it, but the endpoint's route decides which provider and which model answer, so its value is not used for routing. Sending the endpoint's slug keeps logs readable. The model field of the response names the provider model that actually answered.
• Keys: start with mw_ and are sent as the bearer token. A key belongs to one endpoint and works for that endpoint alone: it is minted, capped, disabled and deleted on the endpoint's page, and deleting the endpoint deletes its keys. An endpoint may have several keys, one per app or machine, each with its own monthly cap if you want one. Deleting a key keeps what it was billed: the usage log keeps its requests and the month's spend still counts against the account's cap.

Code Examples

curl https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/chat/completions \
  -H "Authorization: Bearer mw_your_gateway_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ENDPOINT",
    "messages": [{"role": "user", "content": "Say hello in three words."}]
  }'

from openai import OpenAI

client = OpenAI(base_url="https://modelwaygate.com/USER_ID/ENDPOINT/api/v1", api_key="mw_your_gateway_key")

response = client.chat.completions.create(
    model="ENDPOINT",
    messages=[{"role": "user", "content": "Say hello in three words."}],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://modelwaygate.com/USER_ID/ENDPOINT/api/v1", apiKey: "mw_your_gateway_key" });

const response = await client.chat.completions.create({
  model: "ENDPOINT",
  messages: [{ role: "user", content: "Say hello in three words." }],
});
console.log(response.choices[0].message.content);

OkHttpClient client = new OkHttpClient();

String json = "{\"model\": \"ENDPOINT\", "
    + "\"messages\": [{\"role\": \"user\", \"content\": \"Say hello in three words.\"}]}";

Request request = new Request.Builder()
    .url("https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/chat/completions")
    .header("Authorization", "Bearer mw_your_gateway_key")
    .post(RequestBody.create(json, MediaType.get("application/json")))
    .build();

try (Response response = client.newCall(request).execute()) {
    System.out.println(response.body().string());
}

import Data.Aeson (encode, object, (.=))
import Network.HTTP.Client
import Network.HTTP.Client.TLS (tlsManagerSettings)

main :: IO ()
main = do
  manager <- newManager tlsManagerSettings
  initial <- parseRequest "POST https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/chat/completions"
  let body = object ["model" .= "ENDPOINT", "messages" .= [object ["role" .= "user", "content" .= "Say hello in three words."]]]
      request = initial{requestHeaders = [("Authorization", "Bearer mw_your_gateway_key"), ("Content-Type", "application/json")], requestBody = RequestBodyLBS (encode body)}
  response <- httpLbs request manager
  print (responseBody response)

Anthropic SDK

The gateway speaks the Messages API at https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/messages, so the Anthropic SDKs work against it with their base URL changed to https://modelwaygate.com/USER_ID/ENDPOINT/api: the SDKs add /v1/messages and /v1/models themselves, as they do against api.anthropic.com, so their base URL ends at /api where the OpenAI SDKs' ends at /api/v1. Pass the gateway key as the SDK's auth token (auth_token in Python, authToken in TypeScript and Java), which the SDKs send as a bearer token. The endpoint's route may reach any provider, so a Messages request can be answered by an OpenAI or Gemini model too.

Request Headers

• Base URL: https://modelwaygate.com/USER_ID/ENDPOINT/api for the SDKs, which add /v1/messages and /v1/models to it. A request made by hand goes to https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/messages, as the curl and Haskell examples do.
• max_tokens: required, as the Messages API requires it. Thinking budgets map to a reasoning effort for providers that have one.
• Streaming: stream: true answers Anthropic's event stream (message_start, content_block_delta, message_delta, message_stop) from whatever provider the route reaches.

Code Examples

curl https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/messages \
  -H "Authorization: Bearer mw_your_gateway_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ENDPOINT",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Say hello in three words."}]
  }'

from anthropic import Anthropic

client = Anthropic(base_url="https://modelwaygate.com/USER_ID/ENDPOINT/api", auth_token="mw_your_gateway_key")

message = client.messages.create(
    model="ENDPOINT",
    max_tokens=256,
    messages=[{"role": "user", "content": "Say hello in three words."}],
)
print(message.content[0].text)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ baseURL: "https://modelwaygate.com/USER_ID/ENDPOINT/api", authToken: "mw_your_gateway_key" });

const message = await client.messages.create({
  model: "ENDPOINT",
  max_tokens: 256,
  messages: [{ role: "user", content: "Say hello in three words." }],
});
console.log(message.content[0].type === "text" ? message.content[0].text : "");

AnthropicClient client = AnthropicOkHttpClient.builder()
    .baseUrl("https://modelwaygate.com/USER_ID/ENDPOINT/api")
    .authToken("mw_your_gateway_key")
    .build();

MessageCreateParams params = MessageCreateParams.builder()
    .model("ENDPOINT")
    .maxTokens(256)
    .addUserMessage("Say hello in three words.")
    .build();

Message message = client.messages().create(params);
System.out.println(message.content().get(0).text().map(TextBlock::text).orElse(""));

import Data.Aeson (encode, object, (.=))
import Network.HTTP.Client
import Network.HTTP.Client.TLS (tlsManagerSettings)

main :: IO ()
main = do
  manager <- newManager tlsManagerSettings
  initial <- parseRequest "POST https://modelwaygate.com/USER_ID/ENDPOINT/api/v1/messages"
  let body = object ["model" .= "ENDPOINT", "max_tokens" .= 256, "messages" .= [object ["role" .= "user", "content" .= "Say hello in three words."]]]
      request = initial{requestHeaders = [("Authorization", "Bearer mw_your_gateway_key"), ("Content-Type", "application/json")], requestBody = RequestBodyLBS (encode body)}
  response <- httpLbs request manager
  print (responseBody response)

Providers

A provider is one API key you hold with a model provider, added on the dashboard's Providers page under a name you choose. The name is for you and your routes only: it never appears in a URL, may contain spaces, and must differ from your other providers' names, so several providers of one kind sit side by side (a work and a personal Anthropic key, or two OpenAI-compatible servers). Every call goes out on your own key, so the provider bills you for its tokens directly.

• Kinds: Anthropic, OpenAI, Gemini (through Google's OpenAI-compatible API), and OpenAI-compatible, which takes the base URL of any server speaking OpenAI's API, up to and including /v1.
• Names: up to 60 characters, trimmed, and unique among your providers.
• Secrets: stored encrypted and used only to forward your requests; the dashboard shows the last four characters and never the key again.
• Models: the route editor asks a provider for its model list when you pick it for an output or a decider, and offers the list as suggestions (Refresh asks again). Any model name may be typed, so a provider whose list cannot be fetched still works.
• Deleting: a provider that any endpoint's route sends to, as an output or a decider, cannot be deleted; the page names the endpoints using it. Take it out of their routes first.

Routing

Each endpoint is a route you draw in the dashboard: Endpoints, then Edit route. A request to the endpoint starts at the route's input and follows it to one provider's answer. A route is checked when it is saved, so a saved route always holds together, and it is read afresh for every request, so a saved change applies to the next one.

• Input: one per route, fixed, with one edge to the node that takes every request. A new endpoint's input leads nowhere, and requests to it are refused with a 409 until a route is saved.
• Output: one provider and one model, both fixed in the node. The request goes to that provider in its own dialect with the node's model in place of the caller's. An output ends its branch: nothing comes after it.
• Fallback: children tried in their order, which the side panel moves up and down, until one answers. Anything short of an answer moves on to the next child: the provider cannot be reached, it has not started answering within 2 minutes or finished a buffered answer within 5 minutes, it answers any 4xx (429 included) or 5xx, or redirects (the gateway follows no redirect), it answers 2xx with a body that cannot be read, or it cannot do what the request asks, as an Anthropic output cannot embed. When every child fails, the caller gets the last child's error. A fallback may not list the same child twice.
• Streams and fallbacks: a streamed request falls back only until its stream begins, which is when a provider answers 2xx with an event stream. Once that has started reaching the caller nothing is retried: a provider that fails partway ends the stream with an error event in the caller's dialect.
• Decision: a decider (one provider and model, best a small fast one that does not reason first) picks one of the decision's options. Each option has a label, a description and an edge to the branch it leads to, and a default branch takes everything else. The decider is sent a system prompt listing the labels and descriptions and asking for exactly one label, then a transcript of the request: the caller's system prompt (its first 4,000 characters), then the latest turns that fit in about 12,000 characters, with earlier turns marked as left out, tool results cut to 300 characters and images shown as [image]. It may answer in up to 1,024 tokens, which a reasoning model spends on its thinking too.
• Matching the decider's answer: the answer picks the option whose label it equals, ignoring case and surrounding space; failing that, the one label left when quotes, asterisks and a trailing full stop are stripped from both. An empty answer, an answer that matches no label or matches two, and a decider call that fails all take the default, and the route taken says which: decision default (empty answer), (decider answered "…") or (decider failed). Labels must be unique, ignoring case, and not blank.
• Shared nodes and loops: a node may be reached along several paths, but no path may lead back to where it came from; a decision's default counts as a path.
• Size: a route may hold up to 200 nodes and lead to at most 256 provider calls with every branch counted, and one request may make at most 32 upstream calls, deciders included. A request that needs more is stopped with a 502 naming the last failure.
• Upstream budget: every output is sent the whole request, so one request may send at most 64 MiB of requests upstream in all, deciders included. The call that would take it past that is not sent, and the route stops there with a 502, unbilled, whatever outputs or defaults are left; a request of the largest size, 20 MiB, can try about 3 outputs.
• Route deadline: no call starts once 15 minutes have passed since the request arrived; the route stops there with a 504, unbilled. A call already under way runs to its own limits.
• Embeddings: POST /embeddings follows the same route. A decider sees the inputs as one user turn, one input per line, and an output whose provider cannot embed fails, so a fallback moves past it.
• The route taken: every request records the way it went, such as fallback#2 → Work Anthropic (claude-sonnet-4-5), or decision "code" via Router (gpt-4o-mini) → Coder (gpt-4o). The dashboard's recent requests and GET /generation show it.
• Editing: drag nodes to place them, drag from a node's port onto another node to connect them, add outputs, fallbacks and decisions from the palette, and select a node to set its provider, model, order or options in the side panel; the Delete key or button removes the selection. Save stores the route only if it holds together; otherwise the nodes and edges at fault are marked and every problem is listed. A save is refused when the endpoint was saved or renamed elsewhere since the editor loaded it, and the editor offers to reload. Leaving with unsaved changes asks first. The editor needs JavaScript and WebAssembly.

API routes

Eight routes under each endpoint's /USER_ID/ENDPOINT/api/v1, all behind the bearer token of a key minted for that endpoint. A missing or unknown key is a 401 whatever the URL; with a good key, an unknown user or endpoint, another endpoint of the same account and another account's endpoint are one and the same 404. The four inference routes take POST only and are answered along the endpoint's route, whatever dialect the route speaks; the four informational routes are GETs, are not rate limited, and carry no cost headers.

• POST /chat/completions: OpenAI Chat Completions, streamed with stream: true as data: frames ending in [DONE].
• POST /responses: OpenAI Responses API, with function tools, text formats and reasoning effort.
• POST /messages: Anthropic Messages API, with tools, structured output and thinking budgets.
• POST /embeddings: OpenAI embeddings, float encoding only, answered by outputs whose provider embeds.
• GET /models: the endpoint itself as the one model, its id the endpoint's slug and owned_by your account's public id, so SDKs that list models first find it. No provider is asked.
• GET /key: the calling key's name, effective monthly spend cap as limit and this month's spend as usage (dollars), its bytes as usage_bytes, the slug of the endpoint it belongs to as endpoint, and the rate limit.
• GET /credits: the account's monthly spend cap as total_credits, this month's spend as total_usage (dollars) and its bytes as total_bytes, across every endpoint and key (deleted ones included); then the calling key's endpoint as endpoint, and the key's own label, limit, usage and usage_bytes under key, as GET /key gives them.
• GET /generation?id=: the recorded request behind a response id (gen-...), for ids this endpoint served: its status, the provider model that answered as model, the caller's model as requested_model, the route taken, token counts, latency, and the bytes_in, bytes_out and cost it was billed.

Translation

Every request is parsed into one canonical shape and rendered in the provider's own dialect, and every response and stream comes back the same way. A field the canonical shape cannot carry is refused with a 400 that names it, never dropped on the quiet. The exceptions are hints that change the price or the latency of an answer rather than its meaning; those are accepted and ignored, and are listed here so nothing is silent.

• Carried: system and developer prompts, text and image parts, tool definitions (strict included), tool calls and their results, tool_choice and parallel tool calls, the text, json_object and json_schema response formats, temperature, top_p, stop sequences, the token limit, reasoning effort (Anthropic thinking budgets map to an effort level and back), and streaming with usage.
• Accepted and ignored on /chat/completions: stream_options, service_tier and image_url.detail.
• Accepted and ignored on /responses: metadata, prompt_cache_key, safety_identifier, user, service_tier, stream_options, reasoning.summary, input_image.detail, and the output preferences include, text.verbosity and truncation.
• Accepted and ignored on /messages: cache_control, metadata and service_tier.
• Refused with 400 on /chat/completions: n other than 1, logprobs, top_logprobs, logit_bias, audio, prediction, modalities, seed, frequency_penalty and presence_penalty.
• Refused with 400 on /responses: previous_response_id, conversation, store, background, prompt, top_logprobs, max_tool_calls, hosted tools (anything but function tools), input items other than messages, function calls and their outputs, and output_text annotations.
• Refused with 400 on /messages: top_k, mcp_servers, container, output_config, context_management, citations, an output_format other than json_schema, and the tool fields input_examples, defer_loading, allowed_callers and eager_input_streaming.
• Refused with 400 on /embeddings: an encoding_format other than float.
• Known limitations: thinking and redacted_thinking blocks in assistant history are discarded, as are echoed reasoning items on /responses, since the canonical shape has no reasoning part and no provider reads earlier-turn reasoning; Anthropic structured output carries the schema only (as a strict format named output); OpenAI credentials receive max_completion_tokens while compatible servers and Gemini receive max_tokens; usage.cost is always null and stop_sequence is never reported; native_finish_reason carries the provider's own finish reason beside the translated one.

Limits and billing

Each gateway key may make 60 requests a minute and each account 600, each account may spend $10.00 a month by default, and an answered request costs $0.01 per MB of request and response. A request that fails costs nothing, and every answer says what it cost.

• Rate limit: 60 requests a minute per key, and 600 a minute per account, all its keys together, so more keys do not buy more requests. Both are counted in one-minute windows; past either the answer is a 429 with Retry-After in seconds, and a request the key's limit refuses does not count against the account's. Every request to an inference route counts, one to an unknown endpoint included. The informational GET routes are outside the limits, so a client that has been told to wait can still ask why.
• Monthly caps: $10.00 of spend a month per account by default, counting only billed requests. Lower it any time; raising it above the default takes a subscription. A key may carry its own cap, and the lower of the key's and the account's applies. Once the month's spend reaches the cap the answer is a 402 until the month turns. The cap is checked as a request starts, so requests already under way when it is reached finish and are billed, and a month can end over it by what those requests cost.
• Fee: $0.01 per MB, where a MB is a million bytes, of the request body read and the response body written: (bytes in + bytes out) × the fee, rounded up to a whole micro-dollar. Headers never count. Bytes out is the body as the caller receives it; for a stream, every frame written, its closing frames included, but not the cost comment.
• What is billed: only an answer a provider gave with a success and the gateway passed on. A buffered answer is billed when it is sent. A stream is billed once it has begun, for what it sent, even if the provider fails partway or the caller hangs up.
• What is never billed: errors from the gateway (401, 402, 404, 409, 413, 429 and the rest), a provider's error passed on, a 2xx from a provider that could not be read, a route whose every output failed, a request stopped at the call limit, its upstream budget or its route deadline, the calls a decision makes to its decider, and the failed outputs a fallback moved past. A provider outage never costs you anything. An unbilled request is still listed in the usage log, with zero bytes and no cost.
• Stream cost comment: a billed stream writes the final figures as an SSE comment, : gateway-cost bytes_in=… bytes_out=… cost=…, just before its last event ([DONE] on Chat Completions, message_stop on Messages, response.completed on Responses), or as its last line when it ends without one. Comments are lines beginning with a colon, which every SDK skips, so read the raw stream to see it. A caller that has hung up gets none.
• Invoices: billed bytes are metered through Polar and invoiced monthly. Polar prices the exact byte total, while each answer rounds its own cost up to a micro-dollar, so an invoice can come to a little less than its answers' X-Gateway-Cost added up, never more.
• Credits: GET /credits reports the cap as total_credits and this month's spend as total_usage, in dollars, with its bytes as total_bytes, and GET /key does the same for one key. The provider bills you for tokens directly.
• Request size: a request body may be up to 20 MiB; a larger one is refused with a 413 in the caller's dialect, without being read to its end.

Response Headers

Errors

Errors come back in the dialect of the API route you called: OpenAI's error object on /chat/completions, /responses, /embeddings and the GET routes, Anthropic's on /messages. The status and code are the same in both. A provider's own 4xx passes through with its status; any other failure of the provider's is a 502 carrying what it sent. When a fallback has tried every child, the error is the last child's. An internal failure carries a request id to quote.

Error Responses

Known limits

What the gateway does not do, or does in a way you might not expect, stated here so none of it is a surprise.

• Junk in an event stream: a stream is committed as soon as a provider answers 2xx with an event stream, so a provider that then sends something other than its events is relayed as it goes: the caller's stream ends empty or with an error event, and no fallback is tried. A 2xx that is not an event stream at all is refused before anything is sent, and a fallback moves on.
• Large replies: a provider's buffered reply over 64 MiB is refused as unreadable (unbilled, and a fallback moves on), and a single stream event over 8 MiB ends the stream with an error event, billed for what was sent.
• Provider error bodies: a provider's error is passed on cut to its first 4 KiB, marked [cut], whether it came as a reply or partway through a stream.
• Timeouts: a provider has 2 minutes to start answering, and a buffered answer must be finished 5 minutes after it was asked for; past either it counts as unreachable and a fallback moves on, so an answer slower than that must be streamed. A stream may run as long as it keeps sending, but one that sends nothing for 2 minutes ends with an error event, billed for what it sent. No call starts 15 minutes after the request arrived.
• Deciders: a decider that explains itself rather than answering with a bare label ("I would pick code") matches nothing and takes the default. A very long conversation reaches the decider as its system prompt and latest turns only.
• Reasoning models as deciders: a model that thinks before it answers (o4-mini, gpt-5-mini, Gemini 2.5 Flash and the like) spends its thinking from the decider's 1,024 tokens, so it is slower and costs more on your key, and one that thinks past the budget answers nothing and takes the default: the route taken then reads decision default (empty answer). A small model that does not reason decides faster.
• Spend cap: checked as each request starts, so requests in flight when the cap is reached still finish and are billed.
• Unknown endpoints: a request with a good key to a URL that names any endpoint but the key's own counts against the key's and the account's rate limits and is listed in the usage log, with no endpoint.
• Route editor: no panning, zooming or undo yet; the view fits the nodes when the endpoint is loaded. While two of a decision's options carry the same label, the edge for that label shows on the first of them; a route with repeated labels cannot be saved, so this lasts only until one is renamed.