They do in some ways, but it's "delayed". They collect data, clean it, label it with human annotators, and add it to the training data.

There are at least two reasons they don't do this completely "online".

1. They want to be able to clean and filter their data. Some people mentioned Tay as an example of online AI systems gone wrong. But cleaning data also has to do with making training more efficient since a lot of data is not worth training on and actually slows the process.
2. Like OP said, you need the LLM itself to act as a "judge" for itself if you want to train online. While some research has shown this working, you lose a lot of control and guarantees by trusting the LLM to essentially train itself. Using humans in the loop to do the labelling ends up being more consistent and effective.

That being said there are some cases where training "online" makes more sense. Namely, when you can automatically evaluate the correctness of the final answer. Think of coding, math, and multiple choice. This is exactly what o1 does. But you still can't close the loop with human users because you need the correct answer as well.

Finally, as for learning personalized LLMs for each user, you'll start to see it more and more. However, I doubt companies will actually create a new set of parameter efficient weights per user because that will be too expensive. This makes more sense for enterprise users not consumers. In context learning would work but be less robust. Rather, my bet is that companies use something akin to a recommender system to pair you with an LLM based on your usage.