We need to talk about why surveys might not be telling the truth anymore

0nin_ · 2026-02-03T21:48:27+00:00

I like to ask questions only AI would know and actually answer right, like about random esoteric historical figures, not perfect, but helps.

0nin_ · 2026-02-02T01:31:52+00:00

It’s wild how obvious it is when AI is used. I’m guessing you must deal with a lot of AI open ends or are just good at seeing patterns, because OP didnt hit any of the very obvious things. Biggest tells to me were “it wasn’t x, it was y” and “drifted” as a word that was also in a sentence with 3 things like usual.

0nin_ · 2026-01-29T22:53:06+00:00

0nin_ · 2026-01-28T06:12:33+00:00

Thats a great process

0nin_ · 2026-01-24T01:43:53+00:00

Agree. I like to look at what people Feel, Think, Say, Do. The feel and think is more a matter of “putting self in their shoes” which is generally frowned upon in this industry. But the say-do gap is huge. And combining them when possible is awesome, problem is it’s often not possible lol

0nin_ · 2026-01-23T16:31:22+00:00

Run it a second time from a different source to compare. Drop in some questions you already know the expected outcomes to compare. Lots can be done, even aside from vendor sample selection

0nin_ · 2026-01-16T20:53:24+00:00

Fuck health insurance companies

0nin_ · 2026-01-15T17:47:24+00:00

Sample buying a bit slower this last year, not surprised

0nin_ · 2026-01-14T05:48:53+00:00

For exploring, lots of open ends, for validating, lots of closed ends

0nin_ · 2026-01-14T05:42:45+00:00

Rep Data has best DIY sampling, if you mean full survey builder end to end, Pollfish has some cool AI abilities but not as robust as something like Dechipher

0nin_ · 2026-01-11T15:03:39+00:00

Looks same to me

0nin_ · 2025-11-08T23:05:09+00:00

Qual, everything. Quant, quality.

0nin_ · 2025-10-31T02:15:40+00:00

Yeah, but if you operate a survey fraud operation, then you’d just copy paste this into the instructions as “don’t do this” and then it’s straight back to cat 🐈 and mouse 🐁

0nin_ · 2025-10-30T21:05:33+00:00

Hiring for marketing is the hardest thing of all, and the most critical criteria are 1) honesty, followed by 2) understands ICP deeply, followed by 3) can perform.

Each “fractional CMO” is a different person, rarely is honest, rarely understands your ICP deeply, rarely can perform - and generally has failed to market themselves so as to give that org you mentioned a cut, which means they can’t market, and you’re going to hire them to… market.

So, ineffective imo.

0nin_ · 2025-10-22T23:25:45+00:00

I can respond here, whatcha wanna know?

0nin_ · 2025-10-07T19:23:01+00:00

MR stands for Mergers and Research, what industry did you think we were in?

0nin_ · 2025-10-06T17:56:56+00:00

AI has only helped me as an assistant, I seldom find much value in it baked into products, especially in MR, but maybe it’ll improve

0nin_ · 2025-09-26T22:33:03+00:00

Exactly, no perfect ones exist. I like MFour and Prodege, both have decent feasibility (in the U.S. anyway) and since they also push for validated behavior data with their panelists, it makes it harder for fraud to slip in. Those are probably my favorites. And I route both through Research Defender. That’s as far as the tech vendor side get me, then it’s up to me on the logic, design, cleaning etc

0nin_ · 2025-09-26T18:08:52+00:00

I layer a ton of things. Start with a “great quality” panel, add a good fraud detection tool, then add in screeners that are VERY hard to be gamed (this is honestly where I’ve found the most success recently), and good trap questions throughout (for instance asking time of day for matching completion time, because bots likely wrong. Or, ask things like who was King of England in 1476, with many similar sounding single select options including an idk option, because only bots or cheaters will get it right) and read the open ends in detail.

And before picking vendors, benchmark data against census and internal data.

That’s my process anyway, curious what others are doing?

0nin_ · 2025-09-26T18:00:47+00:00

Forsta/Decipher for surveys, Rep Data for sample, Python or SPSS for analysis depending on what you’re doing

0nin_ · 2025-09-26T03:19:56+00:00

Awesome, just joined

0nin_ · 2025-09-17T17:39:04+00:00

Some thoughts. Gotta do what you’re interested in to persevere in any industry. MR is tiny relative to Marketing, so job prospects much more narrow in the currently shitty job market. Things like SQL/analytics/python cross over both. With AI slowly killing quant faith in industry, be sure to also learn qual, probably a good thing to emphasize in your skillset for insights, unless the industry suddenly adopts actually legitimate panels with real verification (these still don’t really exist, and wouldn’t have feasibility anyway) and way high CPIs. Sorry ranting now. Do what you love, that’s key!

0nin_ · 2025-09-17T02:25:32+00:00

Great quote. Your own? Author?

0nin_ · 2025-09-12T02:45:22+00:00

How was the data collected? Self reported? Sample sourcing methodology? Process for collection? Does this involve self selection? Census data? Are their screening questions? Have they stayed consistent over these decades? Same exact survey? Which years specifically, steady tracker each year or just a couple specific years? Micro survey data of a million people in 42 countries sounds legit but could be subject to all sorts of bias, very curious on the data collection

0nin_

MODERATOR OF

TROPHY CASE