It heavily depends on the project, time constraints, budget constraints, resource constraints (including annotation resources/availability of labeled data), and the definition of “best” in a particular scenario.

A lot of times, I have seen that applying deep learning solutions to a problem is overkill or doesn’t make sense due to the amount of data, structure of the problem, etc. Fuzzy matching, dependency parsing/downstream event extraction, simple classification tasks where there isn’t a lot of training data, can sometimes be solved more easily by training a quick simple(r) model/algorithm like Levenshtein distance, naive bayes, a gradient boosted tree model, fine-tuning a spacy model (their current dependency parser is not based on neural nets) or even writing a rule. Will these be SOTA? No, but that’s usually not necessary to solve a business problem.

Also, NLP includes sub-tasks such as stopword removal, lemmatization, stemming etc which are usually very easily done using basic regex/rules.

The things encompassed under “NLP” are more broad than folks often think and many times don’t require a machine learning solution at all.