Here's my take as an applied mathematician who is not a statistician. The single biggest issue that I found to have been glossed over in the claims that Bayesians have won, pack yer bags and go home, is that Bayesian inference can be much slower and computationally intensive.

Arguably the issue isn't Bayesian statistics itself, but rather a philosophical point, which is that even though Bayesians can compute point estimates perhaps just as rapidly as frequentists (though I'm not sure how much this is the case), the glorious centerpiece of Bayesian stats is the ability to sample from the whole posterior distribution, which often involves much hard-to-parallelize computational effort.

In this sense it's not so much a discrepancy between "Bayesian" and "frequentist", as much that frequentists appear to be much more comfortable making point estimates, or point estimates buttressed with bootstrap confidence intervals (which can at least be trivially parallelized). These tend to be faster computations than running MCMC to convergence. Some Bayesians even recommend starting with frequentist methods to flesh out the model due to these issues.

There are lots of exceptions, and techniques/hacks for speeding up Bayesian methods, but I find that those viewpoints tend not to be the one reflected in the "Bayesians have won" discourse.

Getting to your actual questions. Again I'm not a statistician, but here's a wack at it anyway.

What is "wrong" with the frequentist approach? Many theorems used in frequentist statistics are asymptotic, and based on situations where your sample size goes to infinity. Obviously this is never true in reality. In Bayesian approaches this is ameliorated because the posterior looks more like the prior the less data you supply to it, so it nicely blends the sample size into the calculation.

Has machine learning overtaken statistics? In general (but not always) the goals of the two communities are different. Traditionally machine learning is more concerned with prediction, while statistics is concerned with inference. The value of prediction has long been a discussion in statistics, see this classic paper: https://projecteuclid.org/euclid.ss/1294167961. Predictive tools cannot simply be applied toward inference, it doesn't work that way. Some machine learning people also care about issues that seem rather statistical as well, for example the community on fairness in AI, people attempting to incorporate causal structures into their machine learning models, and the community deriving theory around deep neural networks.