How to move an OpenAI SDK app to an OpenAI-compatible API in one change

What "OpenAI-compatible" actually guarantees

Compatibility means the HTTP contract is the same: POST /v1/chat/completions takes a model, a messages array and the optional knobs (temperature, max_tokens, stream, tools, response_format) and returns a choices[] array with message.content, a finish_reason and a usage block. The official Python and Node SDKs serialise exactly this, so the SDK does not know or care which host it is talking to.

What compatibility does not guarantee is identical model behaviour. You are moving from one vendor's models to a catalog of open-weight and wholesale models, each with its own context window, tokenizer and quirks. The protocol is stable; the model you choose is the variable. FastInfra currently exposes 785 model ids through the same endpoint, and the model catalog lists each one with its live price.

Step 1: change base_url and the API key

Create a key on the API Keys page of the console, then set two values. Python:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.fastinfra.ai/v1",
    api_key="fi-...",
)

response = client.chat.completions.create(
    model="qwen/qwen3.6-27b",
    messages=[{"role": "user", "content": "Summarise this ticket in one line."}],
)
print(response.choices[0].message.content)

Node:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.fastinfra.ai/v1",
  apiKey: process.env.FASTINFRA_API_KEY,
});

const completion = await client.chat.completions.create({
  model: "qwen/qwen3.6-27b",
  messages: [{ role: "user", content: "Summarise this ticket in one line." }],
});

Most teams keep both configurations behind an environment variable so the old and new providers can be A/B tested on the same traffic. Nothing else in the call site changes. The exact host for your deployment is printed at the top of the API documentation.

Step 2: map model ids

The one real code change is the model string. Instead of a vendor model name you pass a catalog id such as qwen/qwen3.6-27b or meta-llama/llama-3.3-70b-instruct. Three ways to find the right id:

• GET /v1/models returns every id you can call with your key and is not gated by credits.
• The pricing page lists input and output prices per million tokens for every id, sortable by cost.
• Each /models/{slug} page shows context length, capabilities and a copy-ready snippet for that id.

A practical mapping rule: pick by task size, not by name. Short classification and extraction tasks are well served by models in the 7B–30B range at a fraction of the price of frontier models; long-form reasoning and agentic tool use benefit from the 70B+ tier. The cheapest paid chat route on the platform today starts at $0.0179 per million input tokens.

Step 3: keep streaming, tools and vision as they are

Set stream: true and the endpoint returns server-sent events with data: lines carrying choices[0].delta.content, terminated by data: [DONE]. The SDK's streaming iterator consumes this unchanged. Tool calling uses the same tools and tool_choice fields and returns tool_calls on the assistant message; vision uses image_url content parts; response_format: {"type": "json_object"} works on models that support JSON mode. Embeddings move with the same trick: POST /v1/embeddings accepts the OpenAI model ids as aliases and is served from self-hosted models at $0.

The one shape difference you may notice is in usage: FastInfra reports prompt_tokens and completion_tokens for every model, including free ones, so cost attribution per key works regardless of the upstream provider.

Step 4: handle the two status codes that differ

Vendor APIs bill in arrears, so a 402 is new to most migrations. The cleanest pattern is a two-model config: a paid default and a free fallback that the client selects when it sees insufficient_quota. Rate limits are published in full on the rate limits section of the docs; they only ever go up and are never lowered under load.

What changes operationally

Routing is the biggest behavioural change. For each model id FastInfra tracks live prices and health across several upstreams and sends each request to the cheapest healthy one, failing over automatically if a route errors. You see one endpoint and one bill; the provider selection is invisible. The result is lower unit cost without any client-side logic, but it also means latency can vary slightly between routes for the same model. If a workload needs fixed latency, pin a smaller model or run it on a dedicated GPU Cloud pod and point the same SDK at that server.

Billing is prepaid: usage is deducted from credits after each successful response, and the console shows per-key usage so you can attribute cost to a service or customer. There is no monthly minimum.

A migration checklist

1. Create an API key and store it as FASTINFRA_API_KEY.
2. Add a base_url setting; default it to the FastInfra host in staging first.
3. Replace each hard-coded model name with a catalog id from GET /v1/models.
4. Run your existing evaluation set against two or three candidate ids and record quality and cost per task.
5. Add handling for 402 and honour Retry-After on 429.
6. Check that streaming consumers stop on [DONE] rather than on connection close.
7. Switch production traffic gradually by percentage, watching p95 latency and token usage in the console.

Teams that follow this list typically complete the move in an afternoon; the evaluation step is where the time goes, and it is the step that decides whether the cost saving comes with a quality trade-off you accept.