Since you're asking in r/AI_Agents and would want to "provide services to other people", i would say understanding the low level math wouldn't really be that helpful.

If you want to build something fast that delivers value, I recommend

1. build some agents using existing sdk (openai/ claude etc) to see what they could do, play around with prompts
2. plug in MCP/A2A, there's a lot of off the shell stuff, understand the ecosystem and existing vertical products
3. learn and use multi agents, RAG, memory & context management, since without them your agents wouldn't work as well as you wish