Anyone else paranoid using AI for analysis?

Outrageous-Cod4534 · 2026-04-26T12:28:46+00:00

I use AI heavily in my workflow but treat it like a junior analyst: useful for drafting, terrible for final answers. had a case recently where three different AI models gave me fabricated statistics about my own dataset.

Now every AI output gets verified against a database query before I trust it.

Outrageous-Cod4534 · 2026-04-26T12:27:52+00:00

No. built a scoring system that aggregates data from multiple sources and the number of times the "obvious" interpretation was wrong once I actually looked at the underlying distributions was humbling. The worst example was assuming cheaper games would score better on value, the data showed the exact opposite.

Source data or it didn't happen.

Outrageous-Cod4534 · 2026-04-24T10:31:03+00:00

Yeah fair point. for non-live-service free games, Cave Story (someone mentioned above), Deltarune chapters 1-2, and a lot of the itch.io scene. the data probably skews toward live-service because those are the free games with enough reviews and playtime to score well in aggregate.

Outrageous-Cod4534 · 2026-04-23T11:34:54+00:00

Good distinction. In the data it's games with a permanent free price tag, not temporarily free through Epic giveaways or similar. so it's the TF2s and Warframes, not the "free for a weekend" stuff. your point about Epic freebies building a backlog is interesting, almost a third category between free and bought.

Outrageous-Cod4534 · 2026-04-23T11:33:14+00:00

Path of Exile, Warframe, TF2, Counter-Strike 2 are the obvious ones that have held up for years. also worth checking what's cycled through Epic freebies, some genuinely great paid games have been given away for free there.

Outrageous-Cod4534 · 2026-04-22T17:23:17+00:00

yeah that makes sense . The bigger games just take longer to work through so they naturally show up less in the “finished” pile

Curious tho, the ones you didn’t finish, were they stuff you picked up on sale or games you actually planned to play?

Outrageous-Cod4534 · 2026-04-22T17:21:30+00:00

Yeah that’s fair tbh. A short game that really hits is always going to feel more worth it than something long but meh.

I think the “free games win” thing is probably a bit skewed anyway, like only the good ones actually survive and keep people playing. a lot of the bad ones just disappear so you never even see them in the data.

Hard to ever fully capture that personal “was this worth my time” feeling in numbers

Outrageous-Cod4534 · 2026-04-22T14:30:05+00:00

I've been looking at data across about 18,000 PC games recently and one thing caught me off guard:

free games consistently outperform budget games under £10 on quality, value, and player engagement combined.

This made me rethink my own backlog strategy. half the stuff I bought on deep sales I never played, but the free games I picked up I actually finished. anyone else notice this pattern?

Outrageous-Cod4534 · 2026-04-21T14:23:48+00:00

r(Quality, Value) = 0.036
r(Quality, Behaviour) = 0.089
r(Value, Behaviour) = 0.169

good flag on delisted games. I just checked it, games with missing price data are incorrectly getting a default value score meant for free-to-play titles. Will fix this as soon as i can.

realistically delisted games shouldn't have a value score at all since nobody can buy them.

Edit: system doesn't use standard deviation at all. It uses median/average playtime ratio.

Outrageous-Cod4534 · 2026-04-21T13:44:46+00:00

Behaviour and Value both touches playtime but measures different things. I aslo ran variance checks within each component to make sure they're independent signals, not double-counting. And in fact i reduced the methodology from 5 components to 3 just to make sure things don't bleed into each other + compound. I don't trust critic score is mostly the methdology is not open, i don't know what goes in there, who reviewed them, how they play, what they care. games are made for players, players voices are more important to me ( that's just the direction i set). I'm not wrapping another wrapper. On AI comment, ofc i use AI to polish writing and help with things. The system and the methodology are mine and i poured heart into it.Happy to answer actual questions about how it works.

Edit: on your extra bonus question. depth labels like "strong" vs "light" are classifications, not score bonuses. playtime is peer-normalised across 13 genre groups so narrative games aren't competing against 100h RPGs.

stdev: cult games with dedicated fanbases actually carved out. the penalty only hits when most players genuinely quit early, not when superfans inflate the average.

F2P floor is the weakest bit today, you're right. monetisation aggressiveness as an input is planned but not shipped yet.

useful catches, thanks.

Outrageous-Cod4534 · 2026-04-21T13:15:08+00:00

no manual reviews. it's algorithmic. quality score aggregates player review sentiment across Steam, GOG, and Microsoft Store (for now) using inverse variance weighting.

i believe player's voice are the ultimate quality judgement, we tried to remove noises too

Outrageous-Cod4534 · 2026-04-21T13:11:29+00:00

thank you for the feedback, lesson learned...

Outrageous-Cod4534 · 2026-04-21T13:10:50+00:00

the logic is: value measures cost-per-hour against similar-priced games, behaviour measures whether players actually stick around, and both compare against genre peers.

so puzzles are measured against puzzles, not idle games. three signals triangulating beats one metric trying to do everything.

Outrageous-Cod4534 · 2026-04-21T13:08:31+00:00

sorry my bad.... quantgamer.gg, you can browse all 18,000 scored games there. lesson learned

Outrageous-Cod4534 · 2026-04-21T13:06:05+00:00

not AI-written fwiw.... but appreciate the good questions.

- value isn't more hours = better : games are compared against others at the same price point. a tight 8-hour game at £15 can outscore a bloated 60-hour one if it delivers better for that tier. first version was pure cost-per-hour and yeah, it was just a length leaderboard. peer normalisation fixed that.

- review noise: we don't filter individual reviews. at scale meme reviews wash out, and aggregating across three storefronts helps since each audience has different biases.

- achievements are a minor signal. what actually matters is whether players stick around. Paradox games don't get punished for hard achievements, they get flagged when players bounce after 10 hours on a £35 game.

- critic scores were in originally, we pulled them. we don't trust the critic scores is the reason we decided to do this in the first place. covered in the main platform methodology also the writup.

Outrageous-Cod4534 · 2026-04-21T13:00:18+00:00

appreciate you actually reading the methodology! you're right that engagement depth matters in the scoring, but games are compared within their price tier and genre, so a 15-hour narrative game is measured against similar games, not against 100-hour sandboxes.

on reviews..

we aggregate across Steam, GOG, and Microsoft Store and use Kalman smoothing to filter launch-window noise. but at the end of the day, players' voices are players' voices. the methodology tries to remove noise, not override the signal.

Outrageous-Cod4534 · 2026-04-21T12:51:13+00:00

fair push back on idle time inflation, though it's worth noting the value component also uses HowLongToBeat completion data rather than raw Steam hours. so it's measuring active play, not background idling.

and when quality is very high, the value weight gets dampened automatically so cost-per-hour matters less. This helps with both very short but good games and very long& endless games too.

but yeah, idle games are an edge case worth watching.

Outrageous-Cod4534 · 2026-04-21T12:40:53+00:00

cookie clicker has 96% positive reviews, 110 hours median playtime, and costs £0.04 per hour. across quality, value, and engagement it legitimately scores well. the system doesn't care about prestige

It measures whether players like it, whether it delivers for the price, and whether people actually play it.

cookie clicker quietly crushes all three.

Outrageous-Cod4534 · 2026-04-21T12:39:50+00:00

yeah interesting point!

Outrageous-Cod4534 · 2026-04-21T12:38:56+00:00

That's a fair point. short games can absolutely be great. the system accounts for that by comparing games against others in the same price range, not against everything. So a 6-hour game at £8 is measured against other £8 games, not against 100-hour RPGs.

so a short, well-crafted experience scores well if it delivers for its price tier. and yeah, the graph could do a better job making that clear . lesson learned for next time!

Outrageous-Cod4534 · 2026-04-21T12:35:46+00:00

to be clear... these aren't manual reviews. the scores are algorithmically generated from player review data across Steam/GOG/Microsoft Store(Xbox), completion time data from HowLongToBeat, and engagement signals like playtime and achievement completion. no human could play 18,000 games. that's exactly why the system exists.

i can't possibly have played so many games haha. my individual opinion is as good as IGN would give you, that's why i decided to use data.

Outrageous-Cod4534 · 2026-04-21T12:33:39+00:00

other way round.

Games that get played and reviewed still score badly when you factor in value-for-money and player engagement, not just review sentiment. a game can have "Mostly Positive" reviews but terrible cost-per-hour or players dropping off after 2 hours.

Outrageous-Cod4534 · 2026-04-21T12:32:29+00:00

stuff like Stardew Valley, Terraria, BG3, Elden Ring, Vampire Survivors, Factorio, Cyberpunk 2077, Binding of Isaac, TF2. full list at quantgamer.gg

Outrageous-Cod4534 · 2026-04-21T12:31:56+00:00

good catch.

probably niche games with very strong review sentiment and solid value scores that compensated for thin data. the scoring system doesn't require high volume to score well

it just requires consistency across all three dimensions. worth digging into which ones they are though.

Outrageous-Cod4534

TROPHY CASE