LiteLLM

LiteLLM calls this API as an OpenAI-compatible endpoint. Put openai/ in front of the model id so LiteLLM uses the OpenAI format; the part after it is our model id. It works the same in the Python SDK and in the LiteLLM proxy server.

Install

pip install litellm

uv add litellm

For the proxy server, install litellm[proxy] instead.

Configure API keys

.env

API_KEY=sk-...

Load it into the environment before you run the code (for example with python-dotenv). Keep the key out of your code.

Run

Text

import os
import litellm

r = litellm.completion(
    model="openai/gpt-6-luna",
    api_base="https://pinerouter.ai/v1",
    api_key=os.environ["API_KEY"],
    messages=[{"role": "user", "content": "Name three rivers in Europe."}],
)
print(r.choices[0].message.content)

Stream

stream = litellm.completion(
    model="openai/gpt-6-luna",
    api_base="https://pinerouter.ai/v1",
    api_key=os.environ["API_KEY"],
    messages=[{"role": "user", "content": "Write a haiku about rivers."}],
    stream=True,
)
for part in stream:
    print(part.choices[0].delta.content or "", end="", flush=True)

Tool calling

Pass tools in OpenAI's format. The answer has choices[0].message.tool_calls, as with the OpenAI SDK.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
    },
}]
r = litellm.completion(
    model="openai/gpt-6-luna",
    api_base="https://pinerouter.ai/v1",
    api_key=os.environ["API_KEY"],
    messages=[{"role": "user", "content": "What is the weather in Paris?"}],
    tools=tools,
)
print(r.choices[0].message.tool_calls)

Images

img = litellm.image_generation(
    model="openai/z-image",
    api_base="https://pinerouter.ai/v1",
    api_key=os.environ["API_KEY"],
    prompt="A red paper boat on a calm lake",
    size="1024x1024",
)
print(img.data[0].url)

Proxy server

Add our models to the proxy's config.yaml, then call the proxy as usual:

config.yaml

model_list:
  - model_name: gpt-6-luna
    litellm_params:
      model: openai/gpt-6-luna
      api_base: https://pinerouter.ai/v1
      api_key: os.environ/API_KEY

litellm --config config.yaml

Specifics

api_base ends in /v1. Do not add /chat/completions or any other path; LiteLLM adds it.
The openai/ prefix is for LiteLLM only. It is not sent to us.
For video, music, speech or a model's own settings, call Jobs with the HTTP helper.

LiteLLM OpenAI-compatible endpoints
CrewAI: agents with roles and tasks
Chat completions: every field