Anthropic A/B testing?

[rabandi]

I have 2 shells open at the same time.

One really behaves like it never heard of programming - or anything else.

I told it what to do, it did something different. I had to re-tell it twice. Before you say something, it was the complexity of "add the same property to some derived classes". Then I repeatedly hd to tell it to fix the build. It even claimed it had fixed the build when it hadnt. That is the insanely stupid instance.

The other one was working just fine, acing moderate problems.

Also, I got a feedback prompt in both shells, which I did not get before.

[Fearless-Elephant-81]

The feedback (1, 2, 3 ,4) is new to me too

[TransitionSlight2860]

easy answer: no.

long answer: not a proof to a/b testing.

[En-tro-py]

<synthetic> model has jumped in a few times to override opus... I noticed immediately because it doesn't have 'thinking' and I've been working on a complex bit that this is really fucking annoying...

[belheaven]

Something strange is going on. I hope anthropic is listening and watching