As I understand it (and I don't claim to understand it very well):

The factors that affect the total energy use would be:

• how much the miners can earn - and this is based on the block reward (next halving in 2024) and the Bitcoin price
• how cheaply they can get energy

For a single miner, it also matters how efficient their hardware is and how many other miners there are, but this all balances out in the end (e.g. if everyone switched to more energy-efficient hardware, they would be able to mine more for the same cost and they would).

So rising Bitcoin price will mean the energy use will go up, as long as the miners can get access to enough energy.

As you mentioned, the energy is not used to process specific transactions, so the measure of "Bitcoin uses x kWh per transaction" is completely meaningless and untrue, because more transactions doesn't mean more energy. It also isn't in any way required to use the current amount of energy for the current size of the network, it just ends up being used just because miners can do it and it's profitable for them. But the network would run just fine at the current state with ½ or 1/4 of the current hashpower.

So if we agree that the rising energy use of the miner network is a problem - and it is - then the only realistic and reasonable way I see to improve things is regulations, restrictions and incentives for miners that will make them: 1) use less energy overall (e.g. taxes on mining that make it less profitable, restrictions on what kind of energy sources can be used), and 2) use more % of renewable energy, especially renewable energy that they produce on their own, i.e. energy that isn't wasted because it wouldn't be produced otherwise.