Incrementality Testing: Proving Marketing Really Works

Incrementality testing is a measurement trend that asks a sharper question than "which ad got the click?" It asks: would this sale have happened anyway? The answer helps you stop paying for conversions you would have earned for free.

The core idea

Compare a group that sees your marketing with a similar group that does not. The difference between them is the incremental effect.

Simple tests for small teams

• Geographic holdout: pause ads in one comparable city or region and compare results.
• Time-based pause: switch a channel off for a short, planned period and watch total enquiries, noting seasonality.
• Audience holdout: exclude a random slice of an email list from a campaign and compare purchases.
• Brand search check: see whether turning off ads for your own name changes total leads.

Running a fair test

1. Write the question and the metric before you start.
2. Keep other variables stable, such as offers and budgets.
3. Run long enough to reduce chance effects.
4. Record everything.

Interpret carefully

Small samples can mislead. If results are unclear, repeat or extend the test, and treat conclusions as guidance rather than proof.

Why it matters

Incrementality can show that some retargeting or brand-term spend mostly claims credit for sales that were already coming. It can also reveal underrated channels.

Pair with other data

Use test results alongside attribution views and your KPI report.

Be practical

Do not pause something that risks your business. Start with low-risk experiments.

Common mistakes to avoid

• Pausing a major channel without a plan or end date.
• Comparing regions that are very different.
• Ending a test early because the first week looked exciting.
• Forgetting to account for seasons, holidays and price changes.

Start this week

Choose one low-risk test, such as excluding a random ten percent of an email list from a promotion. Write down the question, the metric and the end date, run it fairly and record the outcome in a simple test log.

An illustrative example

Picture a retailer that pauses retargeting ads for one comparable region for three weeks while keeping them elsewhere. Sales in both regions stay similar, suggesting the ads were claiming credit for purchases that would have happened anyway. This is a hypothetical example, and it shows why a careful holdout can reveal where money is not adding value.