The short answer on marketing mix modeling vs incrementality testing: they answer different questions, and a serious measurement program eventually runs both. Marketing mix modeling (MMM) is a statistical model that estimates how every channel, plus price, seasonality, and the economy, contributed to sales over two or three years. Incrementality testing is a controlled experiment that measures what one channel or tactic caused over a few weeks by withholding it from a comparison group. MMM is broad and correlational. A test is narrow and causal. If you can only afford one today, the right choice depends on your spend, your data history, and the decision in front of you.
This guide is written for marketing leaders at luxury and high-consideration brands, where customer lifetime value runs past $5,000 and sales cycles past 30 days. Those two facts change the math on both methods, and most of the generic advice online ignores them.
What Is the Difference Between Marketing Mix Modeling and Incrementality Testing?
Both methods try to answer the same underlying question, "what would sales have been without this advertising?", and both exist because platform-reported attribution cannot answer it. They get there in opposite ways.
How marketing mix modeling works
An MMM is a regression, today usually a Bayesian one, fitted to aggregated time-series data. Inputs are weekly (sometimes daily) spend or impressions by channel, alongside control variables: pricing, promotions, distribution changes, seasonality, competitor activity, and macro indicators. The model applies two transformations to each media channel:
- Adstock (carryover). Advertising keeps working after it runs. The model estimates how quickly each channel's effect decays.
- Saturation (diminishing returns). The tenth exposure is worth less than the second. The model fits a response curve that flattens as spend rises.
The output is a decomposition of sales into base and incremental contributions by channel, an ROI figure for each, and response curves you can use to simulate reallocations. Open-source frameworks such as Google's Meridian and Meta's Robyn have lowered the build cost considerably, and commercial vendors offer always-on versions that refresh weekly.
No user-level data is involved. That makes MMM privacy-durable and able to cover channels that never produce a click: linear TV, out-of-home, print, sponsorships, and events.
How incrementality testing works
An incrementality test splits a population into a group that sees the advertising and a group that does not, then compares outcomes. The split can be made at three levels:
- User or household level. The platform randomly withholds ads from a holdout, often serving a public service announcement or "ghost ad" in its place. Clean, but dependent on the platform's identity graph and limited to what that platform can see.
- Geographic level. Markets (DMAs, states, or ZIP clusters) are assigned to test and control. Sales are read from your own systems, which makes geo tests platform-independent and the standard for CTV, audio, and DOOH.
- Time level. Media is switched on and off in pulses and compared against a forecast. The weakest design, used when geography cannot be split.
The output is a lift estimate with a confidence interval: incremental conversions, incremental revenue, and from those, incremental ROAS and incremental cost per acquisition. Our guides to holdout testing and geo-experiment design cover the mechanics in detail.
Marketing Mix Modeling vs Incrementality Testing: Side-by-Side Comparison
| Dimension | Marketing mix modeling | Incrementality testing |
|---|---|---|
| Question it answers | How should the total budget be split across channels? | Did this specific channel or tactic cause sales? |
| Type of evidence | Correlational, model-based | Causal, experiment-based |
| Scope | All channels plus non-media factors | One channel, tactic, or audience at a time |
| Data required | 2 to 3 years of weekly spend and sales, plus controls | 8 to 12 weeks of pre-period sales by geography or user |
| Time to first read | 6 to 12 weeks to build; refresh monthly or quarterly | 4 to 8 weeks of flight, plus a post-period for long cycles |
| Typical cost | Roughly $50K to $250K+ a year with a vendor; lower in-house with open-source tools, but requires a data scientist | Opportunity cost of dark markets (often 5 to 20 percent of reach) plus analysis; vendor platforms from the low five figures |
| Minimum practical spend | About $2M+ a year across 4 or more channels | About $50K to $100K a month in the tested channel |
| Granularity | Channel level; struggles below that | Campaign, audience, creative, or publisher level |
| Main weakness | Correlated spend and small channels produce unstable estimates | A result is a snapshot of one tactic in one period |
| Best used for | Annual and quarterly budget allocation | Validating a channel, tuning a tactic, calibrating the model |
All ranges above are planning figures drawn from the engagements we see; your numbers will move with category, vendor, and data readiness.
When Should You Use Marketing Mix Modeling?
MMM earns its cost when the decision is about the whole portfolio and you have the history to support a model.
You have enough variation in the data. A model learns from changes. If your spend on every channel has risen and fallen together for three years, tracking the same seasonal curve, the model cannot separate one channel from another. This problem, multicollinearity, is the most common reason an MMM returns results nobody believes.
You run meaningful offline or non-clickable media. Brands with linear TV, print in titles such as the Robb Report or the Financial Times, event sponsorships, and direct mail have no user-level path to measure those channels. MMM is the only method that puts them on the same scale as digital.
You need to explain the business, not just the media. A wealth management firm's inflows respond to market volatility. A luxury real estate developer's inquiries respond to mortgage rates and inventory. MMM accounts for those forces explicitly, which prevents media from taking credit, or blame, for the macro environment.
The spend justifies it. Below roughly $2M a year in media, or with fewer than four channels, there is usually not enough signal for stable channel estimates. We go deeper on fit in our guide to marketing mix modeling for luxury brands.
Where MMM breaks down for high-consideration brands
Three properties of luxury categories work against the method:
- Low conversion volume. A private aviation operator may close 15 to 40 new jet card members a month. Weekly sales series that sparse are noisy, and the model's intervals widen accordingly. Modeling a higher-volume leading indicator, such as qualified inquiries, is usually necessary.
- Long, variable lag. When exposure-to-sale stretches 60 to 180 days, the adstock assumptions carry much of the result. Small changes in the assumed decay rate can move a channel's ROI by 30 percent or more.
- New channels have no history. A model cannot value CTV if you started it eight weeks ago. It will either ignore the channel or lean entirely on the prior you gave it.
When Should You Use Incrementality Testing?
Testing is the right first move when the decision is specific and the stakes are a single line of the plan.
You are adding a channel. A brand that has run paid search and paid social for years and is evaluating CTV needs a causal read on that one addition, and needs it this quarter. A matched-market geo test delivers it. We outline budget sizing in CTV test budgets for high-AOV brands.
You suspect a channel is claiming credit it did not earn. Branded search and retargeting are the usual suspects. Holdouts on these tactics commonly show that a large share of reported conversions would have occurred anyway, though the size of the gap varies widely by brand and should be measured, not assumed.
You need tactic-level answers. Does the private marketplace deal on premium streaming outperform the broader curated package? Does a 30-second spot beat a 15? Does the top wealth segment respond differently from the lookalike? MMM cannot resolve questions that fine. A test can.
Your budget is below MMM scale. A brand spending $75K to $200K a month can run two to four well-designed tests a year and make better decisions than it would from an underpowered model.
Where incrementality testing breaks down
- Statistical power. With few conversions and high variance, small lifts are undetectable. If the smallest lift your design can detect is 25 percent and the true effect is 10 percent, you will read a real effect as "no significant result." Calculate minimum detectable lift before spending anything.
- Sales cycles longer than the test. A six-week flight on a 120-day cycle captures a fraction of the effect. Extend the read period, measure a validated leading indicator, or both.
- Contamination. Affluent households travel, own second homes, and see national media. A control market is rarely as dark as the plan says, which biases lift toward zero.
- A result does not generalize automatically. Lift at $100K a month in spring does not tell you lift at $300K in the fourth quarter. Response curves bend, and creative and competitive conditions change.
Can You Use Marketing Mix Modeling and Incrementality Testing Together?
Yes, and this is where the "vs" framing stops being useful. The two methods cover each other's blind spots, and the practice of combining them is usually called calibration or triangulation.
The workflow looks like this:
- Build the model. Fit the MMM on available history and review which channel estimates have wide intervals or implausible values.
- Let the model choose the tests. The channels with the highest uncertainty and the largest budgets are where an experiment is worth the most. Test those first.
- Run the experiments. Geo-holdouts for CTV, audio, and DOOH; platform or geo holdouts for social and search.
- Feed results back as priors. In a Bayesian MMM, an experiment's lift and interval become an informative prior on that channel's coefficient. The model is then constrained to agree with causal evidence where it exists.
- Reallocate, then retest. Move budget along the calibrated response curves, and schedule the next test on whichever channel is now least certain.
A calibrated model answers the portfolio question with experimental grounding. Without calibration, MMM is a sophisticated opinion. Without a model, a set of tests is a collection of accurate but disconnected facts.
Platform attribution still has a role in this structure: it is the fast, daily signal for in-flight optimization, corrected by a multiplier derived from tests. Our comparison of incrementality vs. attribution explains how to set that multiplier.
What to do when the two methods disagree
They will. A model might credit CTV with an ROI of 4.0 while a geo test reads 2.2. Before choosing a winner, check four things:
- Time horizon. MMM with long adstock includes carryover that a six-week test read does not. Compare like for like.
- Spend level. The test measured marginal return at one spend level. The model's ROI may be an average across the curve.
- Test validity. Was the control contaminated? Was the test powered for the lift it found?
- Model stability. Does the channel's estimate hold when you refit on a different date range?
When the test is well designed and the disagreement persists, trust the experiment for that channel and recalibrate. Randomized evidence outranks modeled evidence.
Which Should You Start With? A Decision Framework by Spend
| Annual media spend | Recommended starting point | Cadence |
|---|---|---|
| Under $1M | Incrementality tests only; one channel at a time | 1 to 2 tests a year |
| $1M to $3M | Tests first; a lightweight open-source MMM once 2 years of clean data exist | 2 to 4 tests a year |
| $3M to $10M | Calibrated MMM plus a standing test calendar | Model refresh quarterly; 4 to 6 tests a year |
| Over $10M | Always-on MMM, continuous experimentation, tactic-level tests | Model refresh monthly or weekly; rolling tests |
Two qualifiers apply to every row. First, data readiness beats budget: a $5M advertiser with inconsistent channel taxonomies and no weekly sales series is not ready for a model. Second, organizational appetite matters. A test that will not change a budget decision, whatever the result, should not be run.
A Worked Example: Wealth Management Firm, $4M in Annual Media
Consider an illustrative registered investment adviser targeting households with $5M or more in investable assets. Media runs across paid search, LinkedIn, premium CTV, streaming audio, and print. New client relationships average 20 a month, with a 90-day cycle from first inquiry to funded account.
- Month 1 to 3. The firm builds an MMM on three years of weekly data, modeling qualified consultation requests (about 110 a month) as the dependent variable because funded accounts are too sparse. The model credits search heavily and returns a wide interval on CTV, which has only 14 months of history.
- Month 4 to 6. A matched-market geo test holds CTV dark in markets representing 15 percent of the target population for eight weeks, with a six-week post-period. It reads a 13 percent lift in consultation requests in exposed markets.
- Month 7. The lift is entered as a prior. CTV's modeled contribution rises and its interval narrows. A parallel holdout on branded search shows that a meaningful share of those conversions were not incremental, and search's contribution falls.
- Month 8 onward. The firm shifts roughly 12 percent of budget from branded search to CTV and audio, and schedules a LinkedIn test next.
Neither method alone would have produced that reallocation with confidence. The numbers here are hypothetical and chosen to show the sequence, not to predict your results.
Common Mistakes in Choosing Between MMM and Incrementality Testing
- Buying an MMM to settle one channel question. A model takes months and may still be uncertain on the channel you care about. Run the test.
- Treating one test as permanent truth. Retest material channels at least annually and after major changes in spend or creative.
- Modeling final sales when volume is too low. Use a leading indicator with a validated conversion rate to closed business.
- Skipping the power calculation. An underpowered test produces a false negative, and the channel is cut for the wrong reason.
- Letting the media seller grade its own work. Platform lift studies are useful, but the sales data in a decisive test should come from your systems. See true incremental lift measurement.
- Ignoring the model's uncertainty. A point estimate of 3.1 with an interval of 0.8 to 6.0 is not a 3.1.
Frequently Asked Questions
Is marketing mix modeling more accurate than incrementality testing?
For a single channel, no. A well-run experiment gives a causal estimate, while MMM infers contribution from historical correlation. For the full budget across many channels, MMM is the only practical tool, and it becomes far more reliable once calibrated with test results.
How much data do you need for marketing mix modeling?
Plan on two to three years of weekly data, roughly 100 to 150 observations, covering spend or impressions by channel, the sales or lead metric, and control variables. Less history can work with strong priors from experiments, but channel estimates will be less stable.
How long does an incrementality test take?
Most geo tests fly for four to eight weeks, preceded by a pre-period used to match markets. For purchases with long cycles, add a post-period at least as long as the typical lag between exposure and the outcome you measure.
Build a Measurement Program That Matches Your Spend
Stillwater Media designs incrementality tests into every engagement and helps clients decide when a marketing mix model is worth the investment. We work with a limited number of luxury and high-consideration brands each quarter across premium CTV, programmatic, DOOH, and streaming audio. If your customer lifetime value exceeds $5,000 and you want to know what your media is causing, apply for a strategy session.
About Stillwater Media. Stillwater Media is a selective performance media agency for luxury and high-consideration brands, based in Charlotte, NC. We plan and buy premium CTV, programmatic, DOOH, streaming audio, and affluent audience campaigns through private marketplace deals, and we measure them with incrementality testing. Signal. Strategy. Scale.
Frequently Asked Questions
What is the difference between marketing mix modeling and incrementality testing?
Marketing mix modeling is a statistical model that estimates each channel's contribution to sales from two to three years of aggregated historical data. Incrementality testing is a controlled experiment that measures the sales one channel or tactic caused by comparing an exposed group with a holdout group.
Should I use MMM or incrementality testing first?
Brands spending under about $2M to $3M a year, or evaluating a single new channel, should usually start with incrementality tests because they are faster, cheaper, and causal. Brands with larger multi-channel budgets and at least two years of clean weekly data benefit from building an MMM and calibrating it with tests.
Can incrementality tests be used to calibrate a marketing mix model?
Yes. In a Bayesian MMM, the lift and confidence interval from an experiment are entered as a prior on that channel's effect, which constrains the model to agree with causal evidence. Calibration narrows uncertainty and corrects channels the model over- or under-credits.
How much does marketing mix modeling cost compared with incrementality testing?
Vendor-run MMM commonly runs from about $50,000 to more than $250,000 a year, while open-source builds cost less in fees but require data science time. An incrementality test costs mainly the reach you withhold from control markets, often 5 to 20 percent, plus analysis, which makes it accessible at $50,000 to $100,000 a month in channel spend.
Does marketing mix modeling work for luxury brands with few conversions?
It can, but sparse weekly sales make estimates unstable, so most luxury and high-consideration brands model a higher-volume leading indicator such as qualified inquiries. Pairing the model with geo-holdout tests is the most reliable way to validate channel-level results when conversion volume is low.
