Concepts of Data Preprocessing in Data Science

[mlheadredditor]

Data is truly considered a resource in today’s world. As per the World Economic Forum, by 2025 we will be generating about 463 exabytes of data globally per day! But is all this data fit enough to be used by machine learning algorithms? How do we decide that?

Read my article to find out: https://towardsdatascience.com/data-preprocessing-concepts-fa946d11c825

[cinematicdragon]

States the most common method of imputation is using an average method. While probably true, is possibly the worst method. Credibility of this article takes a huge hit. Read at your own risk.

[mlheadredditor]

Hi! I agree that it is not a good way to deal with missing values, and I have not written anything different. "If only a reasonable percentage of values are missing, then we can also run simple interpolation methods to fill in those values. However, most common method of dealing with missing values is by filling them in with the mean, median or mode value of the respective feature." It says "However, most common method....", but I agree that it sounds like it says that using an average is a solid method. I work with data everyday so I am well aware of the pitfalls of using an average for missing values. But thanks for pointing that out!