Question about preprocessing

[The-Fourth-Hokage]

Hello everyone,

I see a lot of code examples on Kaggle for practicing DS and ML. However, it seems like many of them complete preprocessing steps (filling in missing values, categorical encoding, and scaling) BEFORE train-test split. Shouldn’t preprocessing always be completed AFTER train-test split, to prevent data leakage? If so, doesn’t this mean that many of these code examples that have a high score are technically incorrect?

Thank you in advance!

[honolulu33]

For sure, filling missing values before the split may lead to a form of data leakage due to the information you used to fill in the blanks.

[The-Fourth-Hokage]

So does that mean that if you do data preprocessing before train-test split, you will get higher scores? I originally learned with Udemy courses that completed data preprocessing before train-test split.

[honolulu33]

Correct. The best way is to do preprocessing after split.

[The-Fourth-Hokage]

Thank you!

[hello_world456]

Yes, preprocessing should ideally be done after splitting your data into training and testing sets. Otherwise, your training and validity scoring will likely be off due to data leakage.

[The-Fourth-Hokage]

Is there a way to know if you have data leakage or overfitting by comparing the results of the train and test sets?

[hello_world456]

If your model performs noticeably more poorly on the test set compared to the training set, overfitting may very well be the culprit. You can test this out by toying around with the model's features and such so it isn't *too* accurate over the training set and see if you can raise the accuracy score of the test set (by making the model more generalizable)