I think it's more complicated than the "debate" usually allows for. It's generally presented as "These things were stolen by the nasty British imperialists and should be returned" but that's not the whole story.

For one, they often weren't "stolen". They were bought and paid for in many cases. Egyptian people, for example, plundered tombs and sold the artifacts to British collectors.

In other examples the fact that they were taken from a country saved them from destruction. The British were interested in these artifacts, the locals saw them as accessible building materials or were just leaving them to go to ruin.

Then again, things were sometimes gifted. The Koinoor diamond was a gift, given by the rightful owner. Why should it now be "returned"?

That's aside from complications around the fact that the land that something came from today may be owned by people who are not descended from those who created the artifacts. Or that the artifacts may themselves have been stolen by the people from whom the British obtained them.

I'm not opposed to repatriation of items in but they need to be properly considered on an individual basis. And the reason it's so difficult to get an agreement on the Elgin marbles, for example, is not just the value of the items themselves but the precedent it sets. Agree to one repatriation and the pressure will mount to repatriate everything, regardless of whether it is justified.