What Is Incrementality Testing? Methods & How to Run It

Here’s a scenario that keeps performance marketers up at night: your ROAS dashboard looks great 4x, 5x, even 6x returns across channels. However, when you stop a campaign to redirect the dollars, sales hardly flinch. Did those ads ever generate any revenue? Or did they simply take the credit for the sales they would have made?

This is the fundamental issue for which incrementality testing was created. Traditional attribution models are based on “last click” and “first click,” and even multi-touch models rely on correlation rather than causation. They inform you who converted (not whether the ad led to the conversion). That’s the difference for ecommerce brands, DTC companies, and growth marketers who spend thousands (or millions) of dollars a month.

Incrementality testing cuts through the noise. It answers the one question marketers care about most: If I weren’t displaying ads, would this revenue have occurred? Done right, it helps you stop wasting budget on channels that only take credit and invest more in channels that truly drive revenue.

This guide will explore what incrementality testing entails, the best ways to conduct it, metrics to consider, such as incremental ROAS (iROAS), and the ways conversational AI analytics can help you make smarter decisions.

What is Incrementality Testing?

Incrementality testing is an experimental method that measures the true causal impact of a marketing activity (campaign, channel, creative) by comparing results between a test and a control group.

A simple way to understand incrementality testing is to compare it to a clinical drug trial. Treatment is given to one group of patients, and a placebo to another. The difference between the two groups’ results shows the drug’s actual impact. In marketing, the ad acts as the treatment. It’s never seen by the “placebo” group. The difference in conversions, revenue, or signups between the two groups is your incremental lift, and the revenue that you actually generated as a result of your marketing.

In contrast to attribution, which is backward-looking and allocative, incrementality testing is forward-looking and causal. Instead of asking where a customer came from, incrementality testing asks whether the ad actually influenced their behavior.

Why It Matters for Ecommerce & Performance Marketing?

In 2026, incrementality testing has become a crucial issue for several reasons:

• Cookie deprecation: Third-party signals that previously powered attribution models have disappeared or become limited, resulting in unreliable user-level tracking.
• Walled gardens: Facebook, Google, and Amazon each report their own attribution, often leading to conversions being counted twice. Self-reported ROAS metrics are often inflated.
• Budget pressure: CFOs want causal evidence, not correlation-based dashboards, of marketing’s ROI.
• Retargeting bias: Retargeting campaigns often attract customers who would have bought anyway, without actually increasing sales, resulting in high ROAS.

The stakes are especially high for ecommerce brands running incrementality tests. A grocery company found that they were unable to generate any sales lift by stopping all non-branded paid traffic in 12 test markets, proving they had wasted their budget. 

All spending was redirected to CTV, yielding actual incremental revenue. That’s the power of causal measurement, which can even be visualized on an ecommerce analytics dashboard for easy interpretation.

Key Concepts: Incremental Lift, iROAS & Holdout Groups

Incremental lift measures the true additional impact of a campaign by comparing exposed users to a control group that didn’t receive the ad. iROAS and holdout groups help isolate causality in marketing performance by separating incremental conversions from baseline or organic activity.

Incremental Lift

The incremental lift is the percentage of total conversions or revenue driven by the ad alone. The formula is simple:

iROAS – Incremental Return on Ad Spend

In simple terms, iROAS measures the revenue generated for every dollar spent on marketing. iROAS isolates causation, unlike standard ROAS.

A serious warning for marketers who rely on platform-reported ROAS: research consistently finds that iROAS across all DTC brands is 40-70% lower than what they see on their platforms.

Holdout Groups

A holdout group (also known as a control group) is the group of people you intentionally exclude from viewing your ad during the test. Their actions establish a standard and set the expectations of what would have occurred without your marketing. Your incrementality signal is the difference between the test group and the holdout group.

Top Incrementality Testing Methods

Each strategy fits specific kinds of data environments, budgets, and channels. Here are some facts about each.

1. Holdout Test Marketing (User-Level Holdout)

This is the most commonly used incrementality testing method. Your audience is divided into two groups: a test group that sees the campaign and a control group that does not, usually 10–20% of your audience. Once the test window closes, the conversion rates of the two groups are compared to determine the incremental lift.

• Best for: Standard display and mobile, video, and audio advertising
• Advantage: User-level insights, relatively quick to deploy
• Limitation: Requires platform support for audience exclusions and is susceptible to spillover in small audiences.

2. Geo Lift Test (Geographic Holdout)

A geo lift test segments the audience based on geographical regions like cities, DMAs, states, or postal codes and assigns half of the audience to the test group while the other half is assigned to the control group. Ads are shown in treatment regions, while control regions receive no campaign exposure. The incremental impact is measured in the post-test, where aggregate sales are compared between the two groups.

• Best for: TV/CTV, out-of-home, radio, or any channel where the user-level tracking is not available.
• Advantage: Privacy-safe; no reliance on third-party cookies; works across walled gardens
• Limitation: Depends on similar markets; may be affected by seasonal factors or market events.

Example: One Texas-based personal care brand performed geo experiments to measure channel incrementality, raised ad spend by 13%, and measured a 3.1X lift in marketing efficiency by removing all non-incremental channel tactics.

3. Ghost Ads Test

The control group in a ghost ad test is shown a Public Service Announcement (PSA) or a blank ad from the campaign instead of the actual ad creative. This method helps measure the true impact of ad exposure instead of simple ad delivery, and it shows the incremental effect of the campaign’s message.

Best For: Brand lift studies, measurement of the effectiveness of creatives

Advantage: Highly controlled; minimizes selection bias

Limitations: Higher cost to implement; PSA placements use actual media spend

4. PSA Test Advertising (Public Service Announcement Method)

The PSA test advertising method is very similar to ghost ads, except that these ads are given to PSA creatives in the control group to account for ad-serving behavior without presenting the brand’s own ads. The comparison identifies whether the campaign itself, and not just the act of the delivery, is the reason for the incremental conversions.

5. Time-Based Holdout

This method does not divide audiences into separate groups, and it suspends campaigns for a specific period and then compares conversion rates before and after the break. Easy to put in place but sensitive to seasonality, competition, and market forces.

Method	Best Use Case	Privacy-Safe	Complexity	Speed
User Holdout Test	Paid social, programmatic	Partial	Medium	Fast (1–2 weeks)
Geo Lift Test	CTV, OOH, cross-channel	Yes	High	Medium (4–6 weeks)
Ghost Ads / PSA	Brand lift, creative testing	Partial	High	Medium
Time-Based Holdout	Flighting schedules, paused campaigns	Yes	Low	Fast
Matched Market	National brand campaigns	Yes	Very High	Slow (6–12 weeks)

Combining incrementality insights with sales forecasting software allows you to forecast revenue from channels that truly impact sales.

Step-by-step guide to run an incrementality test

There is a methodological discipline needed to run a credible incrementality test. Skipping steps can produce false results that can result in expensive mistakes. Use this systematic procedure:

1. Define Your Hypothesis & KPI 

What will you be measuring? “Does our Meta retargeting campaign drive extra purchases in addition to organic demand?” Before a test, pick one of the primary KPIs that you want to look at, whether incremental revenue, incremental ROAS, or incremental conversions.

2. Size Your Test & Control Groups

For user-level holdouts, typically a 10-20% holdout is enough. When testing for geo lift, ensure that two markets have similar baseline sales, seasonality, and demographic characteristics. Larger sample sizes help produce more reliable results.

3. Establish a Pre-Test Baseline

Run a calibration period (1-2 weeks) to verify that the test and control groups perform in a similar manner prior to the campaign. This is to confirm the similarity of your groups.

4. Launch the Test & Maintain Isolation 

Serve ads to the test group, and do not serve ads to the control group. Try not to have groups overlapping in the audience. Avoid running multiple campaigns to the same audience during the test, as this can distort results.

5. Run for a Statistically Sufficient Duration

Traffic volume and the size of the anticipated lift determine the required duration of the incrementality test, which is determined by running for a Statistically Sufficient Duration, generally 2-6 weeks. Do not end the test early based on initial performance trends: statistical significance is not achieved through a snapshot.

6. Analyze Results & Calculate iROAS

Compare test vs. control conversions and revenue, analyze results & calculate iROAS. Understand them, and if the iROAS is not above the threshold, the channel is not generating real incremental revenue, and the budget is adjusted accordingly.

7. Iterate & Calibrate Other Models 

Use incrementality results to calibrate your Media Mix Model (MMM) or Multi-Touch Attribution (MTA). Other measurement models are dramatically more accurate because they have causal ground truth from incrementality tests.

These insights can be presented in a conversational format, allowing teams to easily examine data in a natural, interactive way without getting bogged down in spreadsheets.

Incrementality Testing vs. A/B Testing vs. MMM

While all three approaches measure marketing effectiveness, they answer different business questions and operate at different levels of granularity. Understanding when to use Incrementality Testing, A/B Testing, or Marketing Mix Modeling (MMM) helps teams choose the right framework for optimization, attribution, and budget allocation.

Dimension	Incrementality Testing	A/B Testing	Media Mix Modeling (MMM)
Question answered	Did this ad cause additional revenue?	Which variant performs better?	How does each channel contribute over time?
Causal?	Yes	Yes (within variants)	Correlation-based
Privacy-safe?	Geo methods, yes	Requires user tracking	Aggregate data
Speed	Weeks	Days–weeks	Months
Best for	Budget allocation, channel validation	Creative, landing page optimization	Long-term strategic planning

The most advanced measurement stacks combine the three: incrementality testing supplies causal calibration data, MMM offers long-term trend analysis, and A/B testing fine-tunes creative and messaging. ProactiveAI’s one-stop analytics engine brings all three together.

Best Practices to Get Accurate Results

Accurate results depend on clean data, consistent tracking, and correctly defined metrics across your funnel. Without these foundations, even the best optimization efforts can lead to misleading conclusions.

1. Avoid audience overlap: 

Make sure that the test and control groups are different. Results are null and void if there is contamination among groups.

2. Consider seasonal data: 

If it is not explicitly a period you’re measuring, do not conduct tests during the major sales days (Black Friday, Q4). The seasonality adds complicating factors, and it’s impossible to measure the impact of ads.

3. Use adequate sample sizes:

Small sample sizes often produce unreliable results. Run a power analysis before starting your study.

4. Pre-register your hypothesis:

Make it before conducting the test, not after. One type of p-hacking that can yield false positives is post hoc metric selection.

5. Don’t interpret neutral results as failure:

A 0% lift indicates that the campaign may not be generating additional revenue because it is NOT incrementally bringing you business, and you should allocate your budget elsewhere without delay.

6. Test one variable at a time: 

If you test an ad group with both creative and targeting changes and bidding changes, you can’t learn anything about which variable worked, as it could be any of the three.

How ProactiveAI Powers Incrementality Testing

ProactiveAI is an advanced business intelligence and marketing analytics platform for eCommerce and performance-based brands. We lift the incrementality measurement of the spreadsheet and put it into a single, continuous analytics layer.

We automatically match the regions that serve as controls and test regions via our Automated Geo Lift Experiments, eliminating one of the most subjective parts of the geo lift experiment.

Our platform combines dashboard visibility with detailed iROAS reporting and calculates incremental ROAS by channel, campaign, and audience segment.

In addition, the Incremental Revenue Testing Dashboard provides a visual representation of lift by channel, time period, and campaign objective, and the MMM Calibration Layer leverages the results to enhance long-term media mix planning.

Based on a Privacy-First Architecture, no third-party cookies are needed for all measurements, future-proofing your stack. Repeatable, automatic, and operational incrementality testing is achieved through our platform.

Conclusion

If you’re still only measuring your marketing based on platform attribution data, you’re making budget decisions based on what correlates with conversions, not what causes them. That’s a tactic that consistently overvalues retargeting and undervalues brand channels in a privacy-constrained, multi-channel, cookie-limited world.

Nothing can close that gap more than incrementality testing. You can run an uncomplicated user-level holdout test on Meta, a geo lift experiment to assess the performance of your CTV investment, or a ghost ads test to measure true brand lift. No attribution method can match the methodology.

The first question is: What would happen if I stopped this campaign tomorrow? Incrementality testing provides the evidence to answer that question with confidence and the intelligence to decide to act decisively.