Running a Campaign Variant Test End-to-End with the Clay CLI

Clone a campaign's sequence into an even A/B split, preview it against a real sample lead, and end the test by keeping a winner, using Clay's real campaigns variants command group.

What you will build

A full variant-test cycle: start-test clones the sequence into two even variants, preview-sequence renders one against a sample lead, and end-test collapses back to a single winning variant.

clay campaigns variants start-test <campaign-id>
    ↓
clay campaigns variants preview-sequence <campaign-id> --variant-id <id>
    ↓
clay campaigns variants end-test <campaign-id> --keep-variant-id <id>

AI Prompt

Using the Clay CLI, run a campaign variant test from start to finish.

Requirements:
- `campaigns variants start-test <campaign-id>` clones the current sequence
  into a second variant and splits weight evenly (weightBps 5000/5000). The
  campaign must be draft or paused, with exactly one variant, and not
  already mid-test.
- `campaigns variants preview-sequence <campaign-id> --variant-id <id>
  [--lead-index N]` renders ONE persisted variant against a real sample lead
  from the campaign's audience -- resolving lead variables, fallbacks,
  links, and spintax, and generating AI snippets once if present. It fails
  not_found with "Campaign audience preview has no leads" if the campaign's
  audience segment has zero matching records.
- Preview generation does not use workspace credits, and this command makes
  exactly one generation attempt with no automatic retry.
- `campaigns variants send-test <campaign-id> --variant-id <id>
  --sequence-step-index N --email-account-id <id> --recipient <email>`
  actually sends one real test email -- requires an active sender account
  in the campaign's settings.senderAccountIds. Sending is non-idempotent;
  never retry automatically without checking the recipient inbox first.
- `campaigns variants end-test <campaign-id> --keep-variant-id <id>` keeps
  one variant and removes the other(s); status affects the exact behavior
  (draft removes immediately, active/paused reassign future leads, completed
  rejects the action).
- Run the verification step below before finishing.

Prerequisites

• The clay CLI on PATH, authenticated via clay login (an OAuth session, not a Public API key)
• jq
• A draft or paused campaign with exactly one variant containing at least one sequence step

Note: JSON samples below are trimmed to the fields relevant to each step. Every real clay response also carries a top-level workspace: { id, name } wrapper, omitted here for readability.

1. Start the test

clay campaigns variants start-test cam_0tl84os74M6XWkVnp2s | jq '.variants[] | {id, name, weightBps}'

Real output, the sequence was cloned into a second variant with an even 50/50 split:

{"id": "cvar_0tl84osg4SnpDFyhZZa", "name": "Variant A", "weightBps": 5000}
{"id": "cvar_0tl8649AE44QHVcTXg7", "name": "Variant B", "weightBps": 5000}

2. Preview a variant against a sample lead

clay campaigns variants preview-sequence cam_0tl84os74M6XWkVnp2s --variant-id cvar_0tl84osg4SnpDFyhZZa

On an audience with real leads, this returns the resolved sequence. Against an empty audience (zero matching records), the real response is:

{"error": {"code": "not_found", "message": "Campaign audience preview has no leads."}}

This is real, documented behavior: preview-sequence needs at least one real lead in the campaign's audience to render against.

3. End the test, keeping a winner

clay campaigns variants end-test cam_0tl84os74M6XWkVnp2s --keep-variant-id cvar_0tl8649AE44QHVcTXg7 | jq '.variants[] | {id, name, weightBps}'

Real output, Variant B was kept at full weight, Variant A removed:

{"id": "cvar_0tl8649AE44QHVcTXg7", "name": "Variant B", "weightBps": 10000}

Verify the result

Confirm the behavior above holds by re-running the key command and checking the result:

clay campaigns variants start-test cam_0tl84os74M6XWkVnp2s && clay campaigns variants end-test cam_0tl84os74M6XWkVnp2s --keep-variant-id <variant-b-id>

Expected: start-test clones the sequence into two variants at weightBps 5000/5000; end-test collapses back to the kept variant at weightBps 10000. preview-sequence against a zero-record audience returns a real not_found rather than empty content.

How it works

start-test/end-test bracket a variant test's lifecycle; preview-sequence and send-test are read-only/side-effecting checks you can run at any point in between. Preview is subsidized and safe to call repeatedly during development; send-test is the one command in this group that has a real, non-idempotent, external side effect (an actual email lands in an actual inbox), which is why it demands an explicit --recipient and an active sender rather than defaulting to anything.

Common issues

not_found "Campaign audience preview has no leads"

Cause: previewing against a campaign whose attached audience segment currently matches zero records.

Fix: attach or populate an audience with at least one real matching record before previewing. This is a real state, not a bug, confirmed live against an audience that legitimately had zero records.

send-test rejected with auth_forbidden

Cause: no sender account connected in this workspace, or the --email-account-id passed isn't one of the campaign's settings.senderAccountIds.

Fix: check clay campaigns options sender-accounts for a real, active sender id, and confirm it appears in the campaign's own settings before passing --email-account-id.

Next steps

• Spam-check both variants' copy before starting the test with clay campaigns sequence spam-check.
• Once ended, clay campaigns get <campaign-id> confirms only the kept variant remains.

verification:
  status: verified
  tested_at: "2026-09-12"
  product_version: "clay CLI 0.19.0"
  command: "clay campaigns variants start-test cam_0tl84os74M6XWkVnp2s && clay campaigns variants end-test cam_0tl84os74M6XWkVnp2s --keep-variant-id <variant-b-id>"
  expected_result: "start-test clones the sequence into two variants at weightBps 5000/5000; end-test collapses back to the kept variant at weightBps 10000. preview-sequence against a zero-record audience returns a real not_found rather than empty content."