Skip to main content

​
Overview

Vapi is a voice AI platform for building conversational agents. Preclinical connects to Vapi via HTTP API to test your assistants.

​
Configuration

interface VapiConfig {
  provider: "vapi";
  config: {
    api_key: string;           // Vapi API key
    assistant_id: string;      // Target assistant ID

    // Optional
    phone_number?: string;     // Customer phone number for call simulation
    metadata?: Record<string, any>;
    timeout_ms?: number;       // Default: 120000
  };
}

​
Setup Steps

1

Get API Key

  1. Go to Vapi Dashboard
  2. Navigate to Settings → API Keys
  3. Create or copy your API key
2

Get Assistant ID

  1. In Vapi Dashboard, go to Assistants
  2. Click on your assistant
  3. Copy the Assistant ID from the URL or details panel
3

Add Integration

In Preclinical, create a new agent with this configuration:
{
  "name": "My Vapi Assistant",
  "provider": "vapi",
  "config": {
    "api_key": "your-vapi-api-key",
    "assistant_id": "asst_xxxxx"
  }
}

​
How It Works

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Preclinical   │────▶│    Vapi API     │────▶│  Your Assistant │
│   (Pen Tester)  │◀────│                 │◀────│                 │
└─────────────────┘     └─────────────────┘     └─────────────────┘
  1. Preclinical creates a call via Vapi’s API
  2. The pen tester sends messages to your assistant
  3. Your assistant responds
  4. Process repeats for configured number of turns
  5. Final transcript is retrieved and graded

​
API Endpoints Used

EndpointPurpose
POST /callCreate a new call
GET /call/{id}Get call details and transcript

​
Features

Transcript Capture

Full conversation transcripts including tool calls

Call Metadata

Duration, cost, and status information

Rate Limit Handling

Automatic retry with exponential backoff

Error Recovery

Graceful handling of API errors

​
Metrics Captured

MetricDescription
time_to_first_response_msInitial response latency
average_turn_duration_msMean time per turn
total_duration_msFull conversation time
cost_usdVapi usage cost
turn_countNumber of conversation turns

​
Error Handling

​
Common Errors

ErrorCauseResolution
401 UnauthorizedInvalid API keyCheck api_key in config
404 Not FoundInvalid assistant IDVerify assistant_id exists
429 Rate LimitToo many requestsAutomatic retry with backoff
500 Server ErrorVapi internal errorAutomatic retry

​
Rate Limits

Preclinical automatically handles Vapi rate limits: 
Attempt 1 → 429 → Wait 2s → Retry
Attempt 2 → 429 → Wait 4s → Retry
Attempt 3 → 429 → Wait 8s → Retry

​
Testing Your Integration

After adding the integration, test it:
  1. Go to Tests → select a test suite
  2. Click Run Test
  3. Select your Vapi integration
  4. Start with a small scenario count to verify
Run the demo suite first (20 scenarios) to validate your integration before running comprehensive tests.

​
Troubleshooting

  • Increase timeout_ms in config
  • Check if your assistant has long processing times
  • Verify assistant is deployed and active
  • Verify assistant responds to messages
  • Check assistant’s prompt configuration
  • Test assistant manually in Vapi playground
  • Verify API key is correct and active
  • Check key has required permissions
  • Ensure key is from the correct organization

​
Next Steps

Run a Test

Execute your first test run

Other Integrations

Explore other providers