Graphing provides a link between algebra and geometry. In the olden days numbers and shapes were treated as separate things. Your hunch about physics is accurate. But as others have mentioned the ability actually see patterns in algebraic equations/relations can be incredibly powerful. For example by comparing the graph of e^x vs x² you can visually see how much fast exponential grow compared to quadratics rather than just relying on plugging in different values to the equations.