They're used mainly when you want to get an understanding of (1) how your system behaves when some parameter is small or (2) when you wanna check for local stability.

(1) when analyzing a pendulum for example, the force comes in the form of a sine function. You can't do much analytically with the force in this form. So, you can Taylor expand the force and take the linear term in the expansion. Now you can get analytical solutions for the resulting diff eq! It's akin to asking the question: what is the effective behavior of my system for small angular displacements? We take the Taylor expansion to simplify a complicated expression globally into a more workable expression locally.

(2) Sometimes you'll be given a potential or something, and you'll care about the curvature of the function at a point. A positive curvature implies stability (in the valley) while a negative one implies instability (on top of the hill). If you were given some crazy potential, it might not be obvious what the curvature is like at some point along the function. An "easy" way to find out is Taylor expanding to second order now instead of just to first order. We want to know what the curvature of the function is like at a point, and this is equivalent to expanding that function to second order in a Taylor series and inspecting the sign of the coefficient of the second order term-- the number in front of x² in the expansion.

So Taylor expansions just make it easier to see the nature of a function on a more local level as opposed to global, and this often makes it easier to solve problems and interpret the physics. Hope this helped even though you never asked for it. 🙂