A/B Testing

Split test subject lines, email content, and from names to find what resonates with your audience. A/B testing works across campaigns, automations, and transactional sends.

How It Works

Create 2-4 variants of a subject line, content, or from name. Pigeon Perch splits your audience and tracks opens, clicks, and revenue per variant. When one variant wins with statistical confidence, it becomes the default.

Test Types

Winner Metrics

• Open Rate: Best for subject line tests
• Click Rate: Best for content tests (recommended default)
• Revenue: Best for e-commerce (requires revenue attribution)

Campaign A/B Testing

Campaigns use a "test then send winner" model:

1. You set a test audience size (10-50% of recipients)
2. The test group is split evenly across variants
3. After a wait period (1-72 hours), the system picks a winner
4. The winning variant is sent to the remaining audience automatically (or manually)

Automation A/B Testing

Automations use an "ongoing split" model. Each contact entering the step is randomly assigned a variant. Once enough data accumulates (minimum sends per variant), the system evaluates the winner and locks it in for all future contacts.

Transactional Send A/B Testing

Create a test linked to a template. Each API call to /sends is randomly assigned a variant. Winner evaluation works the same as automations.

Statistical Significance

Pigeon Perch uses a two-proportion z-test to determine statistical confidence:

• High confidence (99%): Very clear winner
• Confident (95%): Reliable result (default threshold)
• Likely winning (90%): Probable winner, more data helps
• Too close to call: Not enough data or very similar performance

Plan Requirements

A/B testing is available on the Growth ($79/mo) and Scale ($199/mo) plans.

API Endpoints