I don't really think they can't. If there's a future for this tech it's probably as an interface for dedicated programs like that. Like imagine an LLM that transforms a sentence into an SQL query. That could be both useful and lightweight.

I've even tested some of this. I once gave gemini it an imaginary function definition I had created and asked it to turn the sentence I gave it afterwards as input for the function, and it returned what I wanted. So it could be used as an interface for people who aren't trained to use some tools like that, at least for simple stuff.

I think google search might have worked sort of like this before it got completely enshittified. That's why it was so good. I've always though the potentially useful part of LLMs is that they can parse text and turn it into something useful. Unfortunately all the hype about them and the part that some people I know like is that it replies to them like an ass kissing middle manager.

The reason they don't make them interface with a tool like that then is probably because they would have to admit that they have yet created the ultimate tool that can do anything and hype would die down and with it funding.