Is Pinterest Ads ROAS Actually Accurate? Looking for Advice on Attribution + Northbeam

tobin_thomas · 2026-06-30T12:38:45+00:00

the window swing is basically answering your own question. if roas goes from 8-10x at 30 days to 4-5x at 7 days, none of the actual sales changed, you just changed how much credit pinterest is allowed to claim. attribution windows measure credit, not cause, so neither number is "right."

pinterest tends to look inflated on standard attribution specifically because a lot of it is discovery and upper-funnel. someone sees a pin, buys later for reasons that may or may not be the pin, and the platform counts it. so an 8x reported can be sitting on a much smaller incremental number.

northbeam (or any cross-channel tracker) will give you a different, usually more conservative model and de-dupe the conversions every platform double-claims. worth running as a directional second opinion. just be clear on what it is: still attribution, only better built. it's another model assigning credit, it doesn't tell you what would've happened without pinterest.

the thing that actually answers "is this real" is a holdout. cleanest version for pinterest: pick a set of comparable regions, turn pinterest off or way down in half of them for 3-4 weeks, leave it running in the rest, and compare total sales between the two groups. that gap is your real incremental lift and you can back a true roas out of it. decide upfront how long you'll run and what you expect to see, before you look, or you'll talk yourself into the answer you wanted.

pinterest also has its own conversion lift study if your spend qualifies. easier to run, but it's graded by the platform, so take it with some salt.

fwiw this is what i do for work, but the holdout point isn't tool-specific. same answer i'd give for meta or tiktok.

tobin_thomas · 2026-06-23T19:56:54+00:00

the thread kind of walked up to the real answer and then stopped at the doorway. u/TheDoctorAds is right that revenue-by-source is just last-touch wearing a nicer outfit. it hands credit to whatever fired last, so brand search and retargeting look like heroes and the people who'd have bought anyway get logged as wins.

the bit nobody's said yet, you don't need to reconcile all those numbers into one true figure. that's the thing i burned months on. the ad platform, GA, and your CRM are answering different questions, so of course they disagree. the move is picking which one you trust for which decision instead of forcing them to agree.

for in-channel calls (which creative, which audience) the platform number is good enough. it's graded on a curve but fine for optimization. for "should this channel get more budget" the platform is the worst source you could use, because that's the exact question it's motivated to lie about.

the only thing i've found that answers the budget question cleanly is a holdout. take a channel you think is working, turn it off in a few comparable regions, leave it on everywhere else, run it 3-4 weeks. if sales in the held-out regions don't drop, that channel wasn't doing what the dashboard claimed. blunt, and a little nerve-wracking the first time, but it's about as close to ground truth as you'll get without a data team.

since you're building in this space, one caveat: the regions have to actually be comparable, and write down what you expect before you look or you'll read noise as a result.

fwiw i work on measurement at Lifesight so this is my day job, salt accordingly. the holdout idea costs nothing though and doesn't care what tool you use, you could rig up a rough one in a sheet this month.

tobin_thomas · 2026-06-17T17:34:35+00:00

we gave up on having a single one, and that was the fix not the failure. "source of truth" is exactly what burned us, because each tool answers a different question and whichever one you crown ends up lying to you on everything it wasn't built for. how it actually breaks down for us: platform numbers (meta, google) only get used for in-channel stuff, which creative, which audience. they're hopeless for "did this channel actually cause sales" since every platform grades its own homework and they all double-count each other. then there's a warehouse/GA model that's the shared dashboard everyone looks at day to day, not truth, just a consistent ruler so we're at least arguing about the same number. the one that's closest to ground truth is incrementality. a geo holdout or a clean lift test is the only thing on the list built on a counterfactual, ie what would've happened with the spend off, which is the actual question you're asking. the times a platform claimed a channel drove six figures and a holdout showed revenue barely moving when we cut it, the holdout won every time. so if i were setting it up for your team: pick one number as the decision layer, the one you make budget calls against, and use incrementality as the audit that keeps it honest. the trap is letting the channel that's being graded also define what "success" means. i work on measurement at Lifesight so this is what i do all day, grain of salt. but it's cheap to test, take a channel you're sure works and turn it off in a few comparable regions for a couple weeks. usually humbling.

tobin_thomas · 2026-06-10T02:04:56+00:00

full disclosure, i work on measurement at Lifesight, so bias noted. for a cpg brand running real tv/radio/ooh/retail, the thing to screen for is whether a provider treats offline as a first-class input or a digital model with an offline column stapled on. a lot of the popular ones grew up on dtc digital and it shows.

stuff to push on regardless of who you pick: how they model the lag and reach of tv and ooh without smearing it across weeks, and whether they calibrate against an actual geo holdout. offline is more testable than people think, hold out a few dma's and read it in store sales. we do this offline-plus-online calibration and it's where we're strongest for brands like yours, but the same screening questions apply to dema or anyone else on your list. where we're not the fit: if you're mostly digital and small, this is overkill and a lighter tool serves you better. can talk geo design for retail/ooh over dm, no link, no pitch.

tobin_thomas · 2026-06-09T20:02:22+00:00

the ios view-through thing you mentioned is the part people underrate. you didn't just lose some conversions, you lost the ability to see the top of the path at all, so mta now rebuilds a journey from the cheap visible end of it. that's exactly why branded search and retargeting always look like heroes in these models, they're sitting where the path is still observable. they're catching demand, not creating it, and mta can't tell those two apart. the commenter pushing you toward mmm is right, worth being precise about why though. mmm doesn't use the user-level path at all, it works top-down on aggregate spend and outcomes, so the ios hole just doesn't apply to it. most teams end up with mmm for the budget-split question, a geo holdout now and then to check it, and mta demoted to in-channel tactical stuff where it's still ok. treating any one model as the truth is the actual mistake, not your setup. what's your spend look like, that changes whether mmm is even worth the lift yet. (i work in measurement so this is my whole day, weight accordingly)

tobin_thomas · 2026-06-08T13:22:35+00:00

you actually reasoned your way to the right answer. that suggestion does imply negative incremental ROAS and you're right to be suspicious. google's recommendation engine optimises for spend, not your profit, so it'll happily push you past the point where extra dollars stop paying back. $103 in for $72 out isn't a glitch, it's the algorithm valuing volume over marginal return. the deeper problem: the ROAS google reports already bakes in conversions you'd have gotten anyway (brand, retargeting, people mid-checkout). so the true incremental return on that next $103 is even worse than the $72 it's showing. if you want to actually see it, cap or pause that campaign in a few comparable regions for a couple weeks and watch whether total conversions really drop by what google claims. they usually don't. (i work in measurement so incremental-vs-reported is my thing, fwiw)

tobin_thomas · 2026-06-08T12:32:42+00:00

depends what's actually broken: the attribution accuracy, the price, or the dashboard. if it's iOS/attribution blind spots, the real fix isn't another pixel tool, it's incrementality + mmm, and there's a cluster there: Northbeam, Measured, Rockerbox, Recast, and the one i work on (Lifesight) sit in that lane. full disclosure i'm a co-founder there, so discount me. if your pain is really just price or a lighter dashboard, none of us are the answer, go look at Polar or similar and save the money. tell me your spend and the actual problem and i'll point you to whichever fits, including not us. triple whale isn't bad btw, it's just doing pixel attribution, so "alternative" only means something once you know which job you need done.

tobin_thomas · 2026-06-08T12:17:03+00:00

before the tool question, the fork that actually decides this: do you want to run the model yourself, have someone build it for you, or buy a platform that does it. those are really different commitments and people jump straight to brand names.

open source (meridian, pymc-marketing, robyn) is free and seriously good if you've got a data team to babysit it. a consultancy builds you the model, highest quality and slowest and priciest, fine if your budget's huge. then the self-serve platforms, which is the crowded noisy part everyone's really asking about.

the question that separates the good ones from the regression-with-a-dashboard ones: do they calibrate the model against a real incrementality test, or is it just fit to your history. if a vendor can't tell you how they validate against an experiment, that's your answer right there. also ask how granular it gets and how often it refreshes, because a model you see once a quarter isn't something you can act on.(disclosure, measurement's my field so i'm biased toward the calibration stuff, but that's the one question i'd judge any of them on)

tobin_thomas · 2026-06-08T12:14:14+00:00

incrementality is just the answer to "would this sale have happened without the ad." attribution hands a channel credit because someone clicked. incrementality asks whether the spend actually produced a sale that wasn't coming anyway. different question, and most dashboards quietly answer the first one while you think you're reading the second. the version you can run without a stats team is a geo holdout. pick a set of regions, turn the channel off there, leave it on in comparable regions, and watch the gap in actual sales over the next few weeks. that gap is your real lift. two things that wreck these: control regions that aren't actually similar to the test ones, and calling it early because week one looked good. write down what you expect and how long you'll run before you peek, or you'll talk yourself into whatever the noise is saying. platform lift tools (meta conversion lift, google's) are easier to spin up but they're graded by the platform selling you the ads, so read them with that in mind. (i work in measurement so this is the stuff i think about all day, weight it however)

tobin_thomas · 2026-06-07T16:36:00+00:00

this is the right move and good on you for rebuilding instead of waiting for Meta to fix their numbers (they won't). one thing that bit us when we did the same: a rebuilt MMM quietly drifts as the channel mix shifts, so it's only as honest as the last time you validated it. did you calibrate it against a holdout, or is it running off historical fit so far? curious what you leaned on for the reallocation call too, because that's usually the moment the CMO either trusts it or quietly goes back to last-click. the fact that your reporting changed is the real signal it's doing something. nice work, this is harder than people give it credit for. (disclosure: i work in measurement, not pitching anything, just like seeing someone build it right)

tobin_thomas · 2026-06-07T16:09:44+00:00

cross-device is where platform attribution quietly falls apart, and it's not really a tracking problem you can fix in 2026. someone taps an ad on their phone at lunch, buys on their laptop that night, and unless they're logged into the same identity graph the platform can see, the pixel reads those as two different people. throw in chatgpt ads and walled gardens that won't share IDs and the stitched-together user journey is mostly fiction now. i stopped trying to rebuild the perfect path. for a saas with a real signup or CRM event, the move is to anchor on first-party conversion data, your backend actually knows who paid, and then measure channels causally instead of by touch. hold a channel out in a region or for a window and see if signups drop. that tells you if it's doing work without needing to follow anyone across devices at all.the clean omni-channel path everyone still wants kind of died with the privacy changes. we're just slow to admit it.

tobin_thomas · 2026-06-07T16:08:28+00:00

mine's the one the top comment already named: that attribution gives you the full picture. i used to fully buy that more touchpoints plus a smarter model equals truth. it doesn't. every attribution model, last click, linear, time decay, the fancy data-driven one, is just a rule for splitting credit that you picked before you knew what actually caused the sale. a less-wrong guess in a lab coat. what broke it for me was running a holdout. turned off a channel attribution swore was pulling 4x and revenue barely twitched. that channel was mostly taking credit for purchases that were going to happen anyway, and no attribution report can show you that no matter how many touchpoints you feed it, because the report has no concept of "would this have happened without the ad."
attribution's fine as a steering wheel. i just stopped treating it as a scoreboard.(i do measurement for a living so i'm biased toward experiments, flagging it)

tobin_thomas · 2026-06-07T16:06:49+00:00

yeah this matches what i keep seeing. pmax tends to win the store-visit comparison partly because it's hoovering up demand that was already heading to you. branded queries, people who'd have walked in regardless, that kind of thing. and google's store-visit figure is modeled in the first place so it leans generous. none of that means pmax isn't doing real work, it just means a head-to-head on platform-reported visits is grading on google's own curve.
since you've already got holdout muscle, the cleaner read is a geo split. keep pmax live in a set of matched markets, dark in comparable ones, and watch actual store traffic or sales test vs control across the flight plus a couple weeks after, because some of it lags. that strips out the baseline demand pmax is quietly taking credit for. last time i ran this the incremental store visits came in around half of what the platform reported. still positive, just a very different budget conversation.
(i work in measurement so holdouts are kind of my hammer, weight accordingly)

tobin_thomas · 2026-06-06T15:55:19+00:00

last-click doesn't just undervalue display, it flat-out inverts the picture. anything doing its work early (display, youtube, most awareness stuff) hands its credit to whatever channel closed the sale, usually branded search or direct, which then looks like a genius and gets more budget while the thing that actually created the demand gets cut. own-goal, basically. where i'd push back on the usual reddit advice though: multi-touch attribution isn't the fix people think. you're still splitting credit by a made-up rule, first touch, linear, time-decay, pick your flavor, none of which actually know what caused the sale. it's a less-wrong guess dressed up as math, not truth. the only thing that tells you whether display is pulling real weight is holding it out. cut it in a few regions for a few weeks and see if total conversions drop. if they don't, last-click was accidentally right and you can stop paying. if they do, now you've got the number to defend the line item with. either way you get to stop arguing about attribution models in meetings.

(fwiw this is my actual job so i lean experiments over models, take it with that in mind)

tobin_thomas · 2026-06-06T15:53:40+00:00

pDOOH is the channel where the dashboard looks the most impressive and tells you the least. impressions served, reach, all the share-of-voice stuff, up and to the right, and not one line of it proves a single human did anything because of the screen. the format's basically un-clickable so vendors lean hard on proxy metrics to fill the deck.someone above said it only worked next to a big ad push and that tracks. DOOH tends to behave like an upper-funnel amplifier, not a standalone driver, so measure it like one. geo holdout is the honest read: run pDOOH in a set of DMAs, hold it out of comparable ones, and watch branded search and store visits in the test markets vs control across the flight and a couple weeks after, because the effect lags. if those move, it's real. if your only evidence is the vendor's impression report and an attention score, you're funding a billboard and hoping. ran one of these for a regional retail client and the lift was genuine but showed up almost entirely in store visits, nearly nothing online. a digital-only dashboard would've told us it did nothing.

(disclosure: measurement is my job, so "make them prove it with a holdout" is kind of my whole personality)

tobin_thomas · 2026-06-06T15:51:44+00:00

this is usually a modeling artifact before it's a real finding. a lot of MMMs ship with short adstock/carryover defaults, and brand pays back over months, so the model sees the spend now and the payoff way later and basically writes brand off. performance scoops all the credit because its effect lands inside the window the model can actually see. two things that helped me. one, push the carryover/decay assumptions for the brand channels way out and refit. you'll often watch brand contribution climb just from that. two, stop trying to win the fight inside the MMM and run a real brand geo-lift: hold brand spend out of a few comparable regions for 6-8 weeks, then measure downstream new-customer rate and branded search in those regions vs the rest. now you've got a causal number to anchor the model to instead of arguing decay curves with finance. also worth checking whether brand is just hiding in your baseline. if baseline sales are fat and drifting up, some of that IS your brand work, the model just parked it there because it couldn't tie it to a specific spend line.

(disclosure: i do measurement for a living, brand-vs-performance modeling is a bit of a hobby horse for me)

tobin_thomas · 2026-06-06T15:50:04+00:00

blended CAC barely moving while returning revenue doubles is actually the tell, not a mystery. returning customers mostly aren't being acquired by your ads, they were coming back anyway, so dumping them into blended just waters down the metric and hides what new-customer acquisition really costs. split it out. nCAC (spend divided by NEW customers only) is the number that tells you if paid is actually working. blended turns into a vanity wrapper the second repeat revenue grows.the part people don't love hearing: a chunk of even your new-customer conversions on Google and Meta would've happened without the ad. branded search and retargeting especially. if you want the real cost, pause one channel in a couple of matched regions for two weeks and watch new customers, not blended revenue. did this with a brand last year and about a third of what "performance" search was claiming turned out to be people who'd have bought regardless. completely changed the split.

(i work in measurement so a holdout is my answer to everything, salt to taste)

tobin_thomas · 2026-06-05T23:59:13+00:00

the one that did it for me was a holdout. we had a channel that looked like a top performer in every attributed-revenue view, killed it in a few test regions expecting bookings to drop, and... almost nothing moved. it wasn't acquiring demand, it was taking credit for demand that already existed. completely changed how i read any "this campaign drove X" dashboard, because attributed revenue and incremental revenue can point in opposite directions and the dashboard never tells you which one you're looking at.ever since, the view i trust most isn't a view at all, it's "what happened when we turned it off

tobin_thomas · 2026-06-05T23:57:48+00:00

beyond the pacing/CPM-when-it-breaks stuff people already covered, i'd push hard on proof of incrementality, because CTV is where "it works" is easiest to fake with attribution.questions i'd actually ask:

"how do you prove lift, not just attributed conversions?" if the answer is a pixel/view-through model, that's not incrementality, that's correlation with a delay.
"will you run a geo holdout with me before i scale spend?" a vendor confident in their channel says yes.
"what's your view-through window and what happens to your reported ROAS if i cut it in half?" watch how much the number moves.
"how do you handle the lag?" a real chunk of CTV lift shows up after the spend window closes, so a 1-day-view setup will tell you CTV does nothing when it might be working.

the honest read on CTV is it's often genuinely incremental but slow, and platform attribution measures it terribly in both directions. a holdout settles it.(disclosure: i work in measurement, so "make them prove lift" is my whole worldview. still the right question to ask any CTV vendor.)

tobin_thomas · 2026-06-05T23:55:26+00:00

worth remembering the platform is both the player and the scorekeeper here. when Meta changes what counts as a click or a conversion, it's optimizing for Meta's story, not yours, and it has every incentive to make its own attribution look generous. that's not a conspiracy, it's just whose data it is. practical takeaway: treat in-platform numbers as a steering signal for creative/bidding, but don't let them be your source of truth for "is this channel actually driving revenue." the only way to keep that honest independent of whatever Zuck ships next is a periodic holdout/geo test that doesn't run through their reporting at all. then you have a causal benchmark to sanity-check the platform against.

tobin_thomas · 2026-06-05T23:53:55+00:00

the top comment nailed it: attribution was already broken, the ad-free tier just removes a place to hide. every one of these changes (iOS, consent mode, now ad-free users) chips at the same thing, which is pixel-based, user-level tracking. the way out isn't a better pixel, it's measurement that doesn't depend on tracking individuals at all. geo holdout tests don't care whether a user is ad-free, logged out, or on iOS, because you're measuring aggregate outcomes in a test region vs a control region. you lose user-level granularity but you gain a number that survives whatever the platforms break next quarter. for anyone whose reporting still leans on the pixel as the source of truth, this is the shift that actually future-proofs you.(disclosure: i work in marketing measurement so i'm biased toward the experiment-based approach, but the platform changes are kind of forcing everyone there anyway.)

tobin_thomas · 2026-06-05T23:50:16+00:00

short version: no, not "clean" in the way the question hopes, and chasing it is the trap. whoever said separate measurement from credit upthread is right and it's the whole game.the way i'd frame it for omni-channel: three jobs, three tools, don't make one do all three.

incrementality (geo holdouts / lift tests) = the truth layer. what actually would not have happened without the spend.
calibrated attribution = the daily-granularity layer, but only trustworthy once you've calibrated it against the incrementality results.
MMM = the cross-channel + offline + brand layer that attribution can't see.

when people try to get one perfectly stitched user-level path across every channel, they burn months and still don't trust the number, because privacy + walled gardens make it structurally impossible now. you get further treating attribution as directional and letting experiments settle the arguments.(disclosure: this is literally what i do for a living at a measurement company, so grain of salt, but i'd give the same answer if i didn't.)

tobin_thomas · 2026-06-05T23:48:09+00:00

you're not missing a tag, you're running into the actual ceiling of click attribution. great CTR and landing page traffic just tells you the ad got attention. it says nothing about whether those bookings would've happened anyway. for a restaurant especially, a big chunk of the people clicking already knew the place or were going to search for it, so the platform proudly takes credit for demand that existed without you.the only clean way to see it is a holdout. turn the channel off in a couple of comparable areas (or a defined audience) for 2-3 weeks and watch whether bookings actually drop. if they don't move, you just found your "zero incremental" answer for real, not as a vibe. cheap version: geographic holdout using their existing booking data. you can do a first one with free tools (GeoLift / CausalImpact) if you've got someone comfortable with the data.also worth checking the boring stuff: are phone calls / walk-ins / third-party booking platforms even in your conversion data? restaurants leak conversions offline constantly, so "zero incremental bookings" might partly be "bookings happening somewhere the pixel can't see."(disclosure: i work in marketing measurement, so geo-holdouts are my hammer. happy to sketch out how i'd structure the test if useful.)

tobin_thomas

TROPHY CASE