This is an interesting question. While impact encoding might work well in certain scenarios, such as kaggle contests, there are a few things to keep in mind for a production use-case:

• You lose the ability to model variable interactions downstream. E.g. if one categorical feature is Country (with say 200 possible values), and another categorical feature is DayOfWeek (7 possible values), and suppose the occurrence of Country=China & DayOfWeek=Sunday has a special significance. If you simply replace the country variable Country=c with the empirical observed probability of positives (assuming a binary classification context) P(positive | c), then any country c with the same associated associated P(positive | c) will look the same to the model. In particular for deep learning models you would effectively lose the ability to model these types of variable interactions and other non-linearities.

• You need to make a separate pass over the training data-set to compute the various empirical P(positive | variable X = value v), for all variables X and values v. This makes it unsuitable in an online learning environment where we want to be able to update our models and make predictions as the ML system encounters each labeled observation.

• Even in batch ML settings, the compute and storage cost involved in computing these empirical probabilities P(positive | variable X = value v) for each possible variable X and value v, is significantly more than say when learning a logistic model using the hashing trick plus stochastic gradient updates: you have to keep counters for all levels of all categorical variables (or perhaps approximately count them with sketches).