Skip to main content
POST
Create a chat completion

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
model
string
required

HuggingFace model id. See /v1/models.

Example:

"Qwen/Qwen2.5-7B-Instruct"

messages
object[]
required
max_tokens
integer

Maximum tokens to generate.

Example:

256

temperature
number
Required range: 0 <= x <= 2
Example:

0.7

top_p
number
Required range: 0 <= x <= 1
top_k
integer

vLLM-specific. Top-k sampling.

stop
seed
integer

Deterministic seed for sampling.

frequency_penalty
number
Required range: -2 <= x <= 2
presence_penalty
number
Required range: -2 <= x <= 2
repetition_penalty
number

vLLM-specific. Penalty for repeated tokens.

stream
boolean
default:false
tools
object[]
tool_choice
Available options:
none,
auto,
required
response_format
object

Optional response constraints — e.g. {"type": "json_object"} for JSON mode, or {"type": "json_schema", "json_schema": {...}} for structured outputs.

Response

A chat completion (or an SSE stream when stream: true).

id
string
object
string
Example:

"chat.completion"

created
integer
model
string
choices
object[]
usage
object