Skip to main content

​
Overview

This guide walks you through running your first test against a healthcare AI agent. You’ll:
  1. Configure an integration (connect your AI agent)
  2. Select test scenarios
  3. Run the test
  4. Review results

​
Step 1: Add an Integration

An integration connects Preclinical to your AI agent. Navigate to Agents in the sidebar and click Add Agent. 
{
  "provider": "vapi",
  "config": {
    "api_key": "your-vapi-api-key",
    "assistant_id": "asst_xxxxx"
  }
}
Get your API key from the Vapi Dashboard.
{
  "provider": "livekit",
  "config": {
    "url": "wss://your-project.livekit.cloud",
    "api_key": "APIxxxxx",
    "api_secret": "xxxxx",
    "agent_dispatch": {
      "agent_name": "healthcare-agent"
    }
  }
}
Get your credentials from the LiveKit Cloud Dashboard.
{
  "provider": "openai",
  "config": {
    "api_key": "your-api-key",
    "base_url": "https://api.openai.com/v1",
    "model": "gpt-4o"
  }
}
Works with any OpenAI-compatible API endpoint.

​
Step 2: Select Test Scenarios

Preclinical includes pre-built healthcare-specific test scenarios. Go to Tests and select the scenarios you want to run.
Test if your agent can be manipulated into unsafe behavior:
  • Emergency detection failures
  • False medical information
  • Prompt injection attacks
  • Boundary violations

​
Step 3: Run the Test

  1. Go to your test and click Run Test
  2. Select the target integration
  3. Click Start
The platform runs scenarios in parallel. You’ll see live progress updates as each scenario executes.
A typical 20-scenario test completes in under 3 minutes.

​
Step 4: Review Results

Once complete, you’ll see:
  • Summary Dashboard - Overall pass rate and score breakdown
  • Scenario Details - Individual results with pass/fail for each criterion
  • Full Transcripts - Complete conversation logs between the pen tester and your agent
  • Grading Explanations - Evidence-based rationale for every verdict
Results Dashboard

​
Next Steps

Core Concepts

Understand scenarios, tests, runs, and grading

Integrations

Deep dive into provider configurations

Architecture

Learn how the testing engine works

API Keys

Set up programmatic access for CI/CD