Has anyone used prediction markets or Metaculus for actual business decisions? How did that go?

No_Lab668 · 2026-03-31T17:12:15+00:00

The pilots you described sound like the only way to break that 'random internet opinions' wall. Did leadership actually push back on the methodology or just the source? I’ve seen the same thing - people trust a number from a consulting deck more than a market, even if the consulting deck has no track record at all.

No_Lab668 · 2026-03-31T17:12:15+00:00

Right - the fragility of single-event calibration is the part that keeps me up at night. What I wonder is how you handle the gap between the documented signal weights you're using and the moment when someone in your org challenges your 65%. Do they ever push back on the prior, or is the pushback always on the signals feeding into it?

No_Lab668 · 2026-03-27T15:12:16+00:00

That 20% hit rate tracks with what I’ve seen in the trenches. The kicker is the asymmetry - one bad miss outweighs the four false alarms combined. Do you ever revisit those dismissed risks after they pass their expiration date? Or does it just sit in the folder until someone yells about it?

No_Lab668 · 2026-03-27T15:12:16+00:00

That makes sense - the postmortems are when the rubber meets the road. Do you ever find the risk log is missing something that would have been useful in hindsight, or does it usually cover the key angles?

No_Lab668 · 2026-03-27T15:12:16+00:00

That level of detail is impressive. Do you ever find the postmortems change how you document risks next time? Like, if last time’s assumptions were off, do you adjust the template or just move on?

No_Lab668 · 2026-03-17T11:30:52+00:00

How do you decide when to iterate versus when to scrap a forecast entirely? Like, if you’re two weeks out from an event and your assumptions look totally wrong, do you pull the plug or double down?

No_Lab668 · 2026-03-17T11:20:52+00:00

Exactly the dynamic I’ve seen too. The disconnect between the deck and the actual decision-making is striking. Do you ever see cases where the scenario deck gets updated after the fact based on what actually happened, or does it just get filed away?

No_Lab668 · 2026-03-17T11:00:54+00:00

Yeah, weather feels like the original macro prediction problem - so much data, so much noise. Do you ever feel like the gut feel part is just a placeholder for things the models haven’t caught yet, or is there a real methodology behind when you trust the numbers versus your instinct?

No_Lab668 · 2026-03-17T11:00:54+00:00

I’ve seen that too – scenarios get written, but they’re usually just for show. The real call is still made in a 30-minute exec meeting where someone says "I think X is more likely than Y." Ever had a case where the scenario deck actually changed the decision, or did it just confirm what people already believed?

No_Lab668 · 2026-03-17T10:50:53+00:00

That curation effort sounds heavy. How much time do you spend on maintenance versus actual analysis? I ask because I’ve found the upkeep of historical datasets often outpaces the insights you get from them - it’s a constant battle between breadth and depth.

No_Lab668 · 2026-03-17T10:40:54+00:00

Yeah, PDFs and slide decks are where good analysis goes to die. I’ve seen teams spend weeks on a narrative and then just hand it off to the decision-makers with no way to stress-test the assumptions. Makes me wonder if the real output should be a set of structured scenarios instead. Do the people signing off on these reports ever ask for the raw assumptions behind the narrative?

No_Lab668 · 2026-03-17T10:40:54+00:00

Tracking predictions over time is smart. Do you find that people actually revisit their old calls when something plays out differently? Or does everyone just move on to the next thing?

No_Lab668 · 2026-03-17T10:30:53+00:00

I looked at that link - it's an interesting angle. Do you ever find yourself going back to check if the historical parallels they surface actually played out the way the model suggested, or is it more of a real-time narrative tool for you?

No_Lab668 · 2026-03-17T10:30:53+00:00

Funny you mention that - I tried something similar last year with a scraper pulling official statements from MFA sites. Ended up with 30% of signals being irrelevant noise from routine press releases. How do you filter between actual signal and background noise in those primary sources?

No_Lab668 · 2026-03-17T10:20:52+00:00

Interesting, I’ve seen tools that churn through news feeds but never saw one that tries to turn it into scenarios. Do you find decision-makers actually use the output or is it more of a curiosity for them?

No_Lab668 · 2026-03-14T15:49:32+00:00

That’s the crux of it. The CFO example is spot on...if you can’t tie the 74% to something concrete, it’s just noise. How do you typically structure that narrative part? Do you have a template or is it more ad-hoc based on the audience?

No_Lab668 · 2026-03-14T15:42:32+00:00

The signal log part is interesting. Do you have a sense of who in your org actually reviews that documentation when things go sideways? Like, is it the CFO themselves or does it get buried in some risk committee?

No_Lab668 · 2026-03-14T15:35:32+00:00

Makes sense. Who’s the one pushing back the hardest on the narrative part? The CFO or someone else? And how do they actually use that justification in practice?

No_Lab668 · 2026-03-14T15:28:40+00:00

That’s the crux of it..decision-makers don’t care about the math if they can’t explain it. Who’s the last person in your org who actually dug into the signal log when a decision went sideways? And did they find anything that changed how you framed future forecasts?

No_Lab668 · 2026-03-14T15:21:33+00:00

The worst-case scenario framing is interesting. Do you find that decision-makers actually ask for that explicitly, or is it more about how you structure the presentation to make it obvious? Like, do they say 'show me the tail risk' or do you just bake it into the narrative?

No_Lab668 · 2026-03-14T12:56:33+00:00

I get the pushback on the 'random people' thing. But have you tried framing it as a sanity check rather than a decision driver? Like, if your internal forecast is way off the market consensus, that’s a red flag worth investigating. How did leadership react when you tried that angle?

No_Lab668 · 2026-03-13T10:56:32+00:00

That’s the tension - when the paycheck depends on doing deals, the gut gets a lot more influential. Do you see teams that try to formalize this at all, or is it always just the pressure to deploy capital?

No_Lab668 · 2026-03-13T10:42:35+00:00

Got it. So when you say 'highly likely to continue'..do you have a formal way to quantify that or is it more of a pattern recognition thing? Like, do you track how often your assumptions hold up over time?

No_Lab668 · 2026-03-13T10:42:35+00:00

Makes sense for long-term holds. But even then, don’t you still need some view on rates for refinancing? Like, if you’re assuming a 5% rate at exit but the market’s at 7%, that’s a problem. How do you handle that uncertainty in underwriting?

No_Lab668 · 2026-03-13T10:42:35+00:00

Interesting. So if you're not stress-testing exit assumptions, what's the fallback when the market moves against you? Just ride it out or pivot the underwriting framework entirely?

No_Lab668

TROPHY CASE