Skip to content

Commit e5bf8b7

Browse files
committed
docs: add The Grid custom LLM guide
The Grid is an OpenAI-compatible inference marketplace. Document it as a custom-llm provider alongside the existing OpenRouter/Together guidance, and note that Vapi's default maxTokens of 250 is shared with reasoning tokens on several instruments.
1 parent 88b7668 commit e5bf8b7

2 files changed

Lines changed: 68 additions & 0 deletions

File tree

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
---
2+
title: The Grid
3+
subtitle: Use The Grid instruments in Vapi through the custom-llm provider
4+
description: Wire The Grid, an OpenAI-compatible inference marketplace, into Vapi with the custom-llm provider and Custom Credentials.
5+
slug: customization/custom-llm/thegrid
6+
---
7+
8+
The Grid is an inference marketplace that serves models from several labs behind one OpenAI-compatible API. You address capabilities rather than a specific lab's model name: `text-standard`, `code-prime` and `agent-max` each route to a current model for that tier, so an assistant keeps working when the underlying model is replaced.
9+
10+
Because The Grid implements the OpenAI chat completions contract, it plugs into Vapi as a [custom LLM](/customization/custom-llm/fine-tuned-openai-models) — the same path used for OpenRouter or Together AI. Streaming and tool calling both work over that path.
11+
12+
<Note>A The Grid account and API key are required. Keys are created in the dashboard at [thegrid.ai](https://thegrid.ai).</Note>
13+
14+
## Add your credential
15+
16+
POST your key to the `/credential` endpoint with the `custom-llm` provider:
17+
18+
```json
19+
{
20+
"provider": "custom-llm",
21+
"apiKey": "<YOUR THE GRID API KEY>"
22+
}
23+
```
24+
25+
## Create the assistant
26+
27+
Point `url` at The Grid's base URL and set `model` to the instrument you want:
28+
29+
```json
30+
{
31+
"name": "My Assistant",
32+
"model": {
33+
"provider": "custom-llm",
34+
"url": "https://api.thegrid.ai/v1",
35+
"model": "text-standard",
36+
"maxTokens": 600,
37+
"messages": [
38+
{
39+
"role": "system",
40+
"content": "You are a helpful voice assistant. Keep replies short."
41+
}
42+
]
43+
}
44+
}
45+
```
46+
47+
## Choosing an instrument
48+
49+
`GET https://api.thegrid.ai/v1/models` returns the current list with context window, pricing and capability flags for each entry. The tiers most relevant to voice are:
50+
51+
| Instrument | Use it for |
52+
| --- | --- |
53+
| `text-standard` | General conversation. The usual starting point. |
54+
| `text-prime`, `text-max` | Harder reasoning, at higher latency and cost. |
55+
| `agent-standard`, `agent-max` | Multi-step tool use. |
56+
| `code-standard`, `code-prime` | Assistants that read or write code. |
57+
58+
## Set maxTokens above the default
59+
60+
Vapi's `maxTokens` defaults to **250**, and on The Grid that budget is shared with reasoning tokens, which are billed and counted but never spoken. Several instruments reason before answering, so a 250-token cap can truncate the spoken reply mid-sentence.
61+
62+
Measured on a single short prompt: `text-standard` spent 114 of 250 tokens on reasoning, and `agent-max` reached the cap and stopped with `finish_reason: "length"`. Setting `maxTokens` to 600 or higher leaves room for the spoken turn. Keep the system prompt explicit about brevity — that shortens replies more reliably than a low token cap, and without the risk of cutting one off.
63+
64+
## Tool calling
65+
66+
Tools work over the standard OpenAI function-calling fields, so the [tool calling integration guide](/customization/custom-llm/tool-calling-integration) applies unchanged. `agent-standard` and `agent-max` are tuned for multi-step tool use.

‎fern/docs.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,8 @@ navigation:
324324
path: customization/custom-llm/using-your-server.mdx
325325
- page: Tool calling integration
326326
path: customization/custom-llm/tool-calling-integration.mdx
327+
- page: The Grid
328+
path: customization/custom-llm/thegrid.mdx
327329
- section: Examples
328330
icon: fa-light fa-code
329331
contents:

0 commit comments

Comments
 (0)