experiment_create
Pack: ai-model-experiments · Endpoint: https://gateway.pipeworx.io/ai-model-experiments/mcp
Create and start an experiment: run each prompt against each model (× reps), collecting output, tokens, latency, and billed cost per cell. PREPAID: requires Pipeworx credit balance ≥ max_spend_usd (top up via experiment_topup); bills actual provider cost × 1.5 with a $0.10 minimum per experiment. ASYNC: returns experiment_id immediately — execution starts within ~1 minute; poll experiment_status until complete, then call experiment_results. Do NOT wait synchronously. Set summary:false to skip the AI-written model-comparison summary. Example: experiment_create({ name: “tone test”, prompts: [“Rewrite formally: …”], models: [“anthropic/claude-haiku-4.5”, “openai/gpt-5-mini”], reps: 2, max_spend_usd: 2 })
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
name | string | no | Short experiment name |
prompts | array | yes | Prompts to test (max 20) |
items | string | no | |
models | array | yes | Model ids from experiment_models (max 12) |
items | string | no | |
reps | number | no | Repetitions per prompt×model for variance, 1-5 (default 1) |
params | object | no | Optional {system, temperature, max_tokens (default 512)} |
summary | boolean | no | AI-written comparison of the models’ outputs when the run completes (default true) |
max_spend_usd | number | yes | REQUIRED hard spend cap in USD for this experiment (max 100). Execution stops when reached. |
Example call
curl -X POST https://gateway.pipeworx.io/ai-model-experiments/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"experiment_create","arguments":{}}}'Connect
Add this to your MCP client config, or use one-click install buttons:
{
"mcpServers": {
"ai-model-experiments": {
"url": "https://gateway.pipeworx.io/ai-model-experiments/mcp"
}
}
}See Getting Started for client-specific install steps.