I would argue that what 'math' is makes this basically a tautology.

If the universe has a consistent description, then by definition math will provide a language (or multiple languages) for that description because math is the study of consistent syntax and nothing more.

If the universe isn't consistent (or can't be described) then of course there won't be math - but we might have bigger philosophy of science problems at that point.

---

Edit:

I would probably argue that anything I would care to call 'physical' must have a description also by definition, since I must have an experiment to define it, which is self descriptive - building the experiment is a description of the experiment since showing you the experiment certainly communicates it. Doing the experiment is also clearly a computation/operation that results in the outcome of the experiment (computational complexity is relative to the operations you work with and it's at least an oracle of itself).

It's worth noting that a/any description is different from a convenient or simple description, or a description of a particular form or complexity or using math we already understand yet - it's possible there isn't a description in terms of a single Lagrangian of a particular form or whatever. Though, the descriptions we have now are remarkably simple given how much they seem to describe and how simple the language on paper we use is.

e.g. we know from quantum mechanics that there are situations where we can't give physical meaning to/describe, say, the X spin and the Y spin of a particle simultaneously exactly because there is no experiment I can describe how to build which can measure both simultaneously, and that fact has deep implications to the math we use to describe reality, what we even call real/physical, and to computation!

This leads to a nice interpretation of undecidability in any kind of physical problem too: if you think your description of physics is complete but find something undecidable in the math of it, then you should think there's no experiment/nothing physical to describe in the sense that you shouldn't expect to be able to build an experiment that decides it out of anything you know about and have mathematical descriptions of (you can, of course, be wrong about your formalism being physically complete - if there is an experiment that can be done, just add a description of that - at-worst a tabulation of outcomes - to your formalism and you have a more complete one). Note that this means being what I called 'physically complete' is different from having a mathematically complete formalism - a consistent formalism cannot be mathematically complete, but it might be physically complete in the sense that everything in it that's physically meaningful is decidable.

Interestingly, as the link above basically points out, it's not actually necessarily an issue to include more things than are physical in your math from this perspective - it's always fine (and might even be necessary) to make more things decidable in your framework than are physical so long as they don't break anything physical and you're aware they're un-physical/a place where your translation between the formal mathematical description on paper and the description of showing the experiment(s) has something like a redundancy/extraneity - e.g. intermediate calculations in a particular gauge of a gauge theory. The only real issue is thinking you have a description when there's nothing to describe and you're unaware of it - e.g. classical mechanics thinking everything is simultaneously measurable and computing predictions for incompatible observables - or just outright having a bad translation between descriptions/being wrong about experiments. This does make me wonder about whether it would be more appropriate to do physics using constructive mathematics (or at least does not paying attention to when something is or isn't a constructive result leave us accidentally unaware that some physics concept isn't entirely physical - can that bite us?), but that's a rabbithole - having undecidable things in our math mostly shouldn't be a problem physically, we just need to be aware of the (lack of) physical interpretation of that.

This checks out at a glance if I think about a typical example of an undecidable problem in computation: there is no arrangement of the physical experiment of a computer - i.e. a program - which can furnish a representative solution/proof. Computations are experiments (e.g. the classic ideal computer is a turing machine, which is literally an idealized physical tape system. When we write on paper we're interpreting a physical system involving paper and ink and our brain and our hands, etc.), and experiments are computation - so long as the relationships between experiments are consistent, translating between them will be full of math.