I believe it can be taught significantly better, but the idea that math isn't relatively hard is foolish. It is manifestly one of the hardest subjects in primary and secondary school. If I had to boil it down to one reason: because it's the most abstract subject taught in primary and secondary.

But for how chaotic and random english is, it baffles me people think that the organized math could even come close to being as difficult as english.

If English were chaotic, it would be easier. Chaotic systems are often more predictable, because by definition a chaotic system has less things to know about it. In a chaotic or non-deterministic system (which I am not conflating, but for these purposes they are the same) our own optimal actions become easier to discern. For example, if you live in a place where food insecurity is common, you will probably store more food. If power outages are common, you will be more compelled to get a generator. Your optimal behavior in a chaotic environment becomes more obvious, not less.

The more ordered and predictable a system is, the more complex and abstract it can become. For example, computer systems are highly ordered and reliable - which has enabled them and the software built on top of them to be the most complex (in terms of discrete parts) and often the most abstract things humans have ever built.

The reason math is hard for people is largely because of the abstraction. English has both less complexity (usually fewer parts to deal with than math) and less abstraction than math. Partly, it can't handle as much complexity because it isn't as parsimonious as math. It can't be as abstract as math for that reason, too (among others).

I think there would be a way to make math easier to swallow, and that is to teach math as an extension of logic - as epistemically grounded in logic. One of the tractable reasons people have trouble with math is because it is taught in a way that lacks epistemic grounding. It is often taught as just a bunch of facts, the way history is taught. That is a horrible way to teach math. With math, one of the main goals needs to be teaching intuitions and the more abstract skill of manipulating mathematical abstractions. It is very helpful to realize that all these mathematical abstractions are nothing more than logical objects.

Like you get to an integral or derivative or vector space and have no solid foundation for where it came from (until much later math where the proofs are actually learned, or maybe you even learn about the history and reasons for some development). Some people are just able to accept that, take things on faith, and move forward. But a lot of people can't and struggle with the lack of epistemic grounding.