Marginal ROAS, Not Blended Average

The dollar that matters is the next one, not the average one you already spent.

Every paid channel hands you back one number, blended ROAS, the average return across every dollar you've spent since the campaign went live. That number is comfortable and it is close to useless, because it tells you nothing about the dollar you're about to spend next. The first dollars into a channel chase the easiest conversions sitting in the pool. Every dollar after that is fishing in a smaller pond, and the platform's own dashboard has every incentive to keep telling you the water is fine.

What to do: Run a holdout or geo-control test on any channel you are using platform-reported ROAS to justify spending more on. Turn off spend in a matched slice of geographies or accounts, leave the rest of the campaign running, and measure total conversions in the dark markets against the live ones, not just what the platform attributes. Grade the channel on that gap going forward, not on the number in the dashboard.

Why it works: Attribution platforms take credit for whatever converts near an ad, including demand that would have converted anyway, so the reported ROAS only meets reality once you physically remove the spend and watch what actually changes.

Example: eBay ran this exact test in 2013. Economists Tom Blake, Chris Nosko, and Steve Tadelis, working inside eBay's own research team, turned off the company's paid search ads across 68 of the largest U.S. media markets for 60 days while leaving them running everywhere else. Sales in the dark markets came back statistically indistinguishable from the markets still running ads, brand keywords included. The paper ran in Econometrica in 2015 under the title "Consumer Heterogeneity and Paid Search Effectiveness." eBay's own attribution had been crediting those brand-keyword ads with real conversions the entire time.

Walk it through

There's no live page to pull up for this one. The whole point is that the dashboard is the thing you don't trust. Here's the test, worked through with real numbers from the largest version of it anyone has run in the open.

1. Pull the number you're being asked to trust. Say your ads platform reports a 19x return on your branded-search campaign. That's a completely normal figure for that campaign type. Brand terms convert at absurd rates, because the person searching your company name already decided to buy before they typed anything.

2. Build a holdout, not a hunch. Pick a set of geographies or accounts that look like the rest of your footprint in size and seasonality. Turn off spend in that slice for a fixed window, four to eight weeks depending on your sales cycle, and leave everything else running exactly as it was.

3. Measure total conversions, not attributed ones. Compare total revenue in the dark markets against the live markets over the same window, adjusted for any size difference between the two groups. The gap you find, not the platform's post-click attribution, is your actual incremental lift.

4. Turn the gap into a factor and re-grade the channel. This is the exact test Common Thread Co ran at scale for its Q1 2026 channel benchmark, covering 299 DTC brands with a combined $1.01 billion in revenue and $231 million in paid spend, a finding Demand Curve's Frontier newsletter flagged in issue #335. Google brand search reported a 19.07x platform ROAS across that sample. Once geo-holdouts and media-mix modeling were layered in, the same channel's incremental ROAS came out at 5.72x, an incrementality factor of 0.30. Run that ratio backward and it means roughly 70 cents of every reported branded-search dollar was demand that would have shown up anyway, through the organic listing sitting right under the ad.

The read

• Blended always flatters marginal. Averaging a great first dollar with a mediocre three-hundredth dollar produces one dishonest number for the whole channel, and the dishonesty gets worse the longer the campaign has run.
• Brand search is the classic offender, but not the only one. Anything where the ad sits next to demand that already existed, retargeting a hot audience, look-alike lists deep into a mature funnel, inflates the same way for the same reason.
• The incrementality factor is a rate, not a permanent grade. Competitive pressure, seasonality, and brand awareness all move it. A number from six months ago is a guess by now, not a fact.

Steal it

Pick the campaign your team argues about most, usually brand search, and run the holdout on it first. You don't need a data-science team. If you're too small to split by geography, alternate whole weeks instead, spend on for two weeks, off for two, and compare total conversions across the two periods rather than just what the platform attributes. Hold the test for at least a full sales cycle, or a single slow week early on reads as a crash that never actually happened.

Defend the result before you present it. The first pushback comes from whoever owns the campaign you just tested, because the metric they're graded on is about to look worse. Pre-commit to the test window and the decision rule, kill spend below a set incrementality factor, keep it above, before you see a single number, or you'll find a reason to explain away a bad result after the fact. The platform's account rep will push back too, since a lower reported ROAS is bad for them as well. Neither objection is a reason to skip the test. It's the reason you needed one.

Gotchas

• A holdout needs real size to mean anything. A handful of small geographies on an account spending a few thousand dollars a month produces noise, not a number. If you can't get statistical power from a geo-split, run the whole account through an on-off time split instead, and let it run longer to compensate.
• Watch for contamination. Someone in a dark market who sees the ad through a friend's device, or whose location is tagged wrong, blurs the line between test and control. The cleaner and larger your geographic split, the cleaner the read.
• Honest caution: a low incrementality factor is not an argument for cutting a channel to zero. eBay did not stop selling through Google, and Common Thread Co's benchmark is not a case against brand search. It's a case against paying full attributed price for a dollar that was already walking through the door. Some of that spend is defensive, keeping a competitor's ad off your own brand's results page, and that alone can justify running a smaller version of it even at a lower marginal return.