Skip to content

Commit c970d63

Browse files
Document simulating a call from a specific phone number (#1292)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent d09c2c6 commit c970d63

1 file changed

Lines changed: 24 additions & 0 deletions

File tree

‎fern/observability/simulations-advanced.mdx‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -318,6 +318,29 @@ Variables provide values for the [**dynamic variables**](/assistants/dynamic-var
318318

319319
Use variables to test specific inputs, such as a customer name or account tier, without editing the assistant or squad.
320320

321+
### Simulate a call from a specific phone number
322+
323+
Assistants that identify callers by phone number, for example by passing `{{customer.number}}` to an account lookup tool, need a known caller number to test against. Set `customer.number` in the scenario's `targetOverrides.variableValues`. The **Variables** tab only accepts flat names, so set this value through the API:
324+
325+
```bash
326+
curl -X PATCH "https://api.vapi.ai/eval/simulation/scenario/<scenario-id>" \
327+
-H "Authorization: Bearer $VAPI_API_KEY" \
328+
-H "Content-Type: application/json" \
329+
-d '{
330+
"targetOverrides": {
331+
"variableValues": {
332+
"customer": { "number": "+14155550123" }
333+
}
334+
}
335+
}'
336+
```
337+
338+
For a transient scenario, set the same `targetOverrides` on the inline `scenario` in `POST /eval/simulation/run`.
339+
340+
- The value replaces the whole `customer` variable, so also set any other `customer` fields the assistant references, such as `customer.name`.
341+
- Only `{{customer.number}}` changes. The `call.customer` object in server messages does not, so a tool that reads the caller from the message payload won't see the override.
342+
- To test the conversation without a real lookup, [mock the tool](#mock-tool-responses) instead.
343+
321344
## Troubleshooting
322345

323346
| Issue | What to check |
@@ -328,6 +351,7 @@ Use variables to test specific inputs, such as a customer name or account tier,
328351
| There is no audio or recording | Check whether the run used chat mode. Use voice mode to test or record audio. |
329352
| Start or end webhooks do not trigger | Confirm the corresponding hook and its URL are configured on the scenario. |
330353
| A tool mock does not apply | Confirm that `toolName` matches the configured tool name exactly and that `enabled` is `true`. |
354+
| A tool looks up the wrong caller | Set `customer.number` in the scenario's variable values. See [**Simulate a call from a specific phone number**](#simulate-a-call-from-a-specific-phone-number). |
331355
| A latency limit fails without a measured value | No turn in the call measured that metric. Confirm the run used voice mode and that the conversation completed at least one exchange. |
332356
| Latency limits show as skipped | Check whether the run used chat mode or the assistant under test uses GPT-Live. Both skip latency limits. |
333357

0 commit comments

Comments
 (0)