|
| 1 | +--- |
| 2 | +title: The Grid |
| 3 | +subtitle: Use The Grid instruments in Vapi through the custom-llm provider |
| 4 | +description: Wire The Grid, an OpenAI-compatible inference marketplace, into Vapi with the custom-llm provider and Custom Credentials. |
| 5 | +slug: customization/custom-llm/thegrid |
| 6 | +--- |
| 7 | + |
| 8 | +The Grid is an inference marketplace that serves models from several labs behind one OpenAI-compatible API. You address capabilities rather than a specific lab's model name: `text-standard`, `code-prime` and `agent-max` each route to a current model for that tier, so an assistant keeps working when the underlying model is replaced. |
| 9 | + |
| 10 | +Because The Grid implements the OpenAI chat completions contract, it plugs into Vapi as a [custom LLM](/customization/custom-llm/fine-tuned-openai-models) — the same path used for OpenRouter or Together AI. Streaming and tool calling both work over that path. |
| 11 | + |
| 12 | +<Note>A The Grid account and API key are required. Keys are created in the dashboard at [thegrid.ai](https://thegrid.ai).</Note> |
| 13 | + |
| 14 | +## Add your credential |
| 15 | + |
| 16 | +POST your key to the `/credential` endpoint with the `custom-llm` provider: |
| 17 | + |
| 18 | +```json |
| 19 | +{ |
| 20 | + "provider": "custom-llm", |
| 21 | + "apiKey": "<YOUR THE GRID API KEY>" |
| 22 | +} |
| 23 | +``` |
| 24 | + |
| 25 | +## Create the assistant |
| 26 | + |
| 27 | +Point `url` at The Grid's base URL and set `model` to the instrument you want: |
| 28 | + |
| 29 | +```json |
| 30 | +{ |
| 31 | + "name": "My Assistant", |
| 32 | + "model": { |
| 33 | + "provider": "custom-llm", |
| 34 | + "url": "https://api.thegrid.ai/v1", |
| 35 | + "model": "text-standard", |
| 36 | + "maxTokens": 600, |
| 37 | + "messages": [ |
| 38 | + { |
| 39 | + "role": "system", |
| 40 | + "content": "You are a helpful voice assistant. Keep replies short." |
| 41 | + } |
| 42 | + ] |
| 43 | + } |
| 44 | +} |
| 45 | +``` |
| 46 | + |
| 47 | +## Choosing an instrument |
| 48 | + |
| 49 | +`GET https://api.thegrid.ai/v1/models` returns the current list with context window, pricing and capability flags for each entry. The tiers most relevant to voice are: |
| 50 | + |
| 51 | +| Instrument | Use it for | |
| 52 | +| --- | --- | |
| 53 | +| `text-standard` | General conversation. The usual starting point. | |
| 54 | +| `text-prime`, `text-max` | Harder reasoning, at higher latency and cost. | |
| 55 | +| `agent-standard`, `agent-max` | Multi-step tool use. | |
| 56 | +| `code-standard`, `code-prime` | Assistants that read or write code. | |
| 57 | + |
| 58 | +## Set maxTokens above the default |
| 59 | + |
| 60 | +Vapi's `maxTokens` defaults to **250**, and on The Grid that budget is shared with reasoning tokens, which are billed and counted but never spoken. Several instruments reason before answering, so a 250-token cap can truncate the spoken reply mid-sentence. |
| 61 | + |
| 62 | +Measured on a single short prompt: `text-standard` spent 114 of 250 tokens on reasoning, and `agent-max` reached the cap and stopped with `finish_reason: "length"`. Setting `maxTokens` to 600 or higher leaves room for the spoken turn. Keep the system prompt explicit about brevity — that shortens replies more reliably than a low token cap, and without the risk of cutting one off. |
| 63 | + |
| 64 | +## Tool calling |
| 65 | + |
| 66 | +Tools work over the standard OpenAI function-calling fields, so the [tool calling integration guide](/customization/custom-llm/tool-calling-integration) applies unchanged. `agent-standard` and `agent-max` are tuned for multi-step tool use. |
0 commit comments