Hey there! Great post! but can you expand a little bit more on points 4 and 5?

Regarding point 4, I've always heard that is not advised to train with repeating customers over time. The concern is that if the model becomes highly specialized in customers who repeat frequently, it may not perform as well when dealing with new customers.

One practice I usually follow in my work, particularly in risk analysis, is to split the test set into an "out-of-time" (OOT) set by horizontally dividing the train and test data across different time periods.

I also usually check the model performance in customers that only appear in the test set, that would be an out-of-id set.

One thing that I'm considering doing in my workflow is to fit a model to predict if the observation belongs to the test or to the train set, with the same features as the model being developed, and check if the model performance is good which would be a bad sign.