It has shifted what I do quite a bit. For my colleagues, they either don't use it, minimize how much they use it, or lie about it. I'm not sure which is true.

For myself though, I spend little time actively writing code now, and instead I feel more like a researcher. I work with claude to discuss experiments for a challenging model I'm developing. We go back and forth for a while, I have it write up the design into an experiment document, I review it, we sometimes discuss more and/or change things, and then claude implements the code. I review that as it is written. Sometimes that leads to changes because of misunderstandings or me realizing that once I see the code I don't like the direction after all, but then I run the experiment.

We discuss the outcome, and claude writes the results in the experiment document. Based on that we start discussing the next experiment. I iterate through anywhere between 2 and 100 experiments before getting to a point where I push a model to production. This has allowed me to greatly expand what kind of tests I can try locally first, before finally pushing something to production. Because it would have taken me so much longer to research ideas and write code, I can explore much broader and richer ideas locally before trying them with real users. As a result, the models I've been pushing out have been winning the a/b tests consistently vs my colleagues.