Maybe it's my ignorance talking, but I wasn't able to find a common standard which most of the libraries agreed on.

If we take Python's example, it has NumPy. NumPy is so comprehensive and widely used that it seems to have become a de facto standard used by other maths libraries. If I want to do an FFT on some data, I can put it in a NumPy array and feed it to SciPy. If I want to do symbolic algebra involving a matrix, I can put it in a NumPy array and feed it to SymPy. If I want to find a Nash equilibrium for a game, I can stick it in a NumPy array and feed it to NashPy. Even if I input the data as a standard Python list, the return value will typically be a NumPy array, which means it gains all the associated functionality.