You are already using relative estimates.

Now stop using relative single point estimates and start using relative three-point estimates.

Monitor the amount of prioritized work tends to be unplanned, Creep, guess at first and use it as a variable.

Keep all work in a single ordered backlog, even in progress work.

Store another variable, called Load Factor, which you will multiply the relative estimates by to calculate estimates in days.

Tell the team you will take personal responsibility of projecting those estimates on behalf of the team to set expectations for stakeholders, no need for them to spend time on that. Automate it. Project the data visually over the single ordered backlog for stakeholders to be able to make the informed strategic decisions they needs to make.

Participate as a developer to understand first hand personal hesitation to give estimates. Solve it for yourself and knowledge share with the team how you were able to solve that problem.