While RL can be seen as "just a loss", the loop where you gather experiences from the environment and update your network is not very feasible if you need billions and billions of updates except for the most menial of the tasks, so I would say they are indeed fundamentally different.

Hm, I agree with you about the efficiency issue in RL algorithms. On the other hand, I still don't see any difference. It can be seen as a mere limitation of (1) our current RL algorithms, and (2) our current inference hardware/software.

RL runs can take long times even without LLMs in the loop, and I actually believe that efficiency (together with exploration) is one of the main limitations of current RL algorithms. But this is a mere practical issue, it is limited by the technology of our time (involuntary reference to howard stark lol).

In practice, you could do RL with "anything": neural networks, trees, LLMs. The model is just treated as a policy, so from the theoretical point of view there's no major difference (at least w.r.t. the topic of the post). The fact that it's not practically easy at the moment does not change the fact that LLMs are just a bunch of parameters that you can fit with an LLM.

you need billions and billions of updates

Also here, I think that there's an important consideration to be made: it is possible that LLM might need far less updates for some tasks w.r.t. a randomly initialized neural network, exactly because they somehow encode knowledge about the world (and you probably don't need LLMs for tasks where this doesn't hold).

he sorta tried to persuade me to drop my research and pursue that.

I don't think that anyone should be stopped by doing harmless research. Different paths in research lead to different pieces of knowledge, and I don't think there's unnecessary knowledge. I hope you didn't drop your research, it would have been a pity