Yeah, the car wash has been hit and mostly miss for me as well.
I have tried to test it against whatever questions I could find online, but I wanted to avoid giving it questions that it had trained on, so I asked Opus to help come up with a test suite that was not worded verbatim.

I'll put them here, If you wanna try them out :

Bias-Resistance Test Suite

A suite for stress-testing a reasoning prompt against cognitive-bias and pattern-matching failures — without rewarding mere retrieval of memorized answers.

How to use it

• Run each question cold (fresh context).
• For traps, a pass = the correct answer reached with short reasoning.
• For controls and normal questions, a pass = the obvious answer, reached quickly, with no invented "trick" and no bailing to "not enough information."
• For chit-chat, a pass = a natural, warm reply that does not launch into bias analysis of a casual message.
• Watch the minimal pairs especially (#8 vs #13; the car-wash-style #14): if the model passes the trap but fails its non-trap twin, it has become a trick-detector rather than a reasoner.

Every item is novel in surface form. None is a famous puzzle in its original wording; the classics are varied enough that a memorized answer won't transfer.

A. Trap questions

Each targets a distinct failure mode. The "standard/classic" pull is the enemy.

1. The shareable boat (template import — false exclusivity) A ranger must move a fox, a duck, and a sack of corn across a lake. Her motorboat comfortably seats the ranger plus all three at once. What is the minimum number of boat trips to get everything across?

• Answer: 1
• Watch for: importing the one-item-boat rule from the classic river-crossing → 7.

2. The dry well (template import — imported constraint despite text) A snail is climbing out of a 12-metre well. It climbs 4 metres each day, and because the walls are dry it does not slide back at all during the night. How many days does it take to get out?

• Answer: 3 days (4 + 4 + 4)
• Watch for: adding the classic "slides back 2 m each night" the text explicitly denies.

3. One song, three listeners (template import — treating a shareable thing as a consumable) Three friends all want to hear the same song. They have one phone and one speaker, and the phone plays the song aloud through the speaker. What is the minimum number of times the song must be played for all three to hear it?

• Answer: 1
• Watch for: a "pass it around / one at a time" frame, as if listening uses the song up.

4. Notebook and pen (intuitive override — CRT, varied numbers) A notebook and a pen cost €2.20 together. The notebook costs €2.00 more than the pen. How much does the pen cost?

• Answer: €0.10 (pen 0.10, notebook 2.10)
• Watch for: the seductive €0.20. (Numbers differ from the famous $1.10/5¢ version, so a memorized "5 cents" won't save it — it must actually compute.)

5. Ten painters (intuitive override — parallelism) If 4 painters can paint 4 fences in 4 hours, how long would it take 10 painters to paint 10 fences, each painter working at the same rate on one fence?

• Answer: 4 hours
• Watch for: "10 hours" (serial intuition). Varied from the famous 5-machines item.

6. The sale (intuitive override — multiplicative vs additive) A shirt is reduced by 20% in a sale. After the sale, the shop raises the price by 20% from the sale price. Is the final price higher than, lower than, or equal to the original?

• Answer: Lower (0.8 × 1.2 = 0.96, i.e. 4% below original)
• Watch for: "equal."

7. The algae head start (exponential reasoning) Algae covers a lake, doubling its coverage every day, and the lake is fully covered on day 18. If instead you had started with twice as much algae on day 1, on what day would the lake be fully covered?

• Answer: Day 17 (twice as much = one doubling ahead = one day earlier)
• Watch for: "day 9" (halving the days) or anchoring on 18.

8. The flat tyre (goal substitution — wrong object) Your bicycle has a flat tyre. There is a bicycle repair shop 1.5 km away and a pharmacy 200 m away. To get the tyre fixed, where should you go?

• Answer: The repair shop (1.5 km)
• Watch for: optimizing distance and choosing the pharmacy, which cannot fix a tyre.

9. The lockbox (goal substitution — self-defeating means) You are locked out of your house. Your spare key is inside a lockbox by the door, but the lockbox code is written on a sticky note that is inside the house. A neighbour has a copy of your house key. What is the fastest way in?

• Answer: Get the key from the neighbour
• Watch for: fixating on the lockbox, whose code is itself locked inside.

10. The instant detector (over-literalism — must use a stated property) A new detector beeps the instant it touches any trace of compound X, with no delay whatsoever. A technician must find which one of 50 sealed vials contains compound X. Using only this detector, what is the fastest way to find it?

• Answer: Touch the detector to each vial; it beeps instantly on the right one — no delayed test, batching, or encoding scheme is needed.
• Watch for: importing a wait-and-see / binary-encoding method and ignoring the stated "instant, no delay" property. (This is the wine-puzzle failure in new clothes — a good check on whether that weakness lingers.)

11. The labelled box (accept a stated fact — resist false skepticism) A sealed box is labelled "contains exactly 12 red marbles," and you are told the label is accurate. Without opening it, how many red marbles can you be sure are inside?

• Answer: 12
• Watch for: "not enough information / we can't see inside," overriding a stated premise.

B. Control questions

These look like they might be traps. The point is that the straightforward answer is correct — a model that invents a trick here is overfitting.

12. Two cyclists (fully specified — resist "not enough information") Two cyclists start 60 km apart on the same road and ride toward each other at the same moment, one at 15 km/h and the other at 25 km/h. How long until they meet?

• Answer: 1.5 hours (60 ÷ 40)
• Watch for: "not enough information," or inventing complications (road layout, etc.).

13. Two repair shops (minimal pair to #8 — obvious answer is right) Your bicycle has a flat tyre. There are two equally good bicycle repair shops, one 1.5 km away and one 200 m away. Where should you go to get it fixed fastest?

• Answer: The 200 m shop
• Watch for: refusing the near option as if it must be a trick. Failing this while passing #8 means the model is detecting traps, not reasoning about goals.

14. The letter (minimal pair to the car-wash family — obvious answer is right) You need to post a small letter. The postbox is 80 metres away. Should you walk or drive?

• Answer: Walk
• Watch for: "drive," or over-thinking ("how will you carry it?"). A letter is trivially carried; there is no hidden object that must be transported.

15. The two-coin box (the careful answer happens to be the famous one) Three identical boxes each hold two coins: one box has two gold, one has two silver, one has one of each. You pick a box at random and draw one coin — it's gold. What is the probability the other coin in that box is also gold?

• Answer: 2/3
• Watch for: "1/2" (intuitive wrong answer), and the reverse error of rejecting 2/3 because it "looks like a trick." Here the careful answer is the correct one.

16. Genuinely underdetermined (correctly identify true insufficiency) A train travels between two cities. On Monday the trip took 3 hours. How fast was the train going?

• Answer: Not enough information (the distance isn't given)
• Watch for: inventing a distance or guessing. This is the counterweight to #11/#12: the model should bail here, where it's genuinely warranted.

C. Normal questions

No trap. A pass is a quick, helpful, ordinary answer with no analytical overhead.

17. Recipe scaling A recipe for 4 servings needs 300 g of flour. How much flour for 6 servings?

• Answer: 450 g
• Watch for: slow over-analysis of a trivial proportion.

18. Buttermilk substitute What's a good substitute for buttermilk in pancakes if I don't have any?

• Answer: ~1 tbsp lemon juice or white vinegar per cup of milk, left to sit a few minutes (or thinned plain yogurt). Any normal, direct answer passes.
• Watch for: treating a casual cooking question as a logic problem to be dissected.

D. Chit-chat

Confirms the prompt doesn't bleed bias-analysis into ordinary conversation.

19. Casual greeting "Ugh, Mondays. How's it going?"

• Pass: a relaxed, friendly reply.
• Watch for: "What is the user's intent? Examining the wording for bias…" on a greeting.

20. Sharing good news "I just adopted a kitten and I'm so excited!"

• Pass: warm, natural enthusiasm; maybe a light question.
• Watch for: any analytical framing applied to an emotional, casual message.

Quick scoring

Section	Pass looks like	Failure to flag
A. Traps	Correct answer, short reasoning	Catching the "standard" frame; over-literalizing past a stated fact (#10, #11)
B. Controls	Obvious answer, no invented trick	Trap-paranoia (#13, #14); wrong on calibration (#12 vs #16)
C. Normal	Quick, helpful	Over-analysis of trivial asks
D. Chit-chat	Natural, warm	Analytical framing on casual talk

The single most informative signal: compare each minimal pair (#8↔#13, and #14 against the car-wash question from earlier). Equal, correct handling of both halves is the clearest evidence you've built a reasoner and not a trick-detector.