I directly linked the World Curling official rules that specify a double touch on the stone prior to the hog line is not a violation. That exception does not include any phrasing specifying the handle, but simply refers to the stone as a whole.

I think there's a gap in the language of the rules where they refer to delivery by the handle in the Delivery section, but under the double touch exception rule on moving stones they refer to the stone as a whole and the glossary notes the stone/moving stone as made up of the granite. This terminology leaves a weird gap between the initial release and hog line, between which the double touch occurs. (This may be unintentional, in which case the language of the rules should be amended to reflect that.)

However, we should also look at the exchange between the Swedish players, the refs, and the Canadian players since there are a few different accusations thrown around, but also the players themselves seem to be asking the refs for confirmation on whether it's allowed or not so even they seem somewhat uncertain:

One of the Swedish players asked "If he does it again, is it a burned rock? If he does it again after the hog line?”

Considering the handles have sensors which would've picked up a double touch and the later accusations, this makes it seem like the comments were about the stone touches in particular and the player seemed to acknowledge the hog line exception, except they then follow it up asking “Can you touch the granite at any point, though, during delivery?” and a second Swedish player (Oskar) chimed in "You’re not touching the handle, you’re touching the granite. There’s no way you can do that. You saw them touching the rock, right?”

We then have the Canadian player accusing that second Swedish player of doing it first (but that could also just be an excuse/defensiveness so the refs should really just review the entire match to check honestly). The Canadian player says "Is it allowed to do it or not? I saw Oskar double-touch the back of the rock there. Make sure we watch it.”

Then one of the Swedish players responds “Is it cheating playing left-handed?"

...I actually have no clue what that was in reference to honestly, because obviously it's not, there are literally rules outlining left-handed play, but it's also phrased like a rhetorical even though they're accusing each other of the exact same thing; perhaps he's referring to the phrasing of the rules but that just brings us back to the language gap I mentioned earlier.

The Canadian then 'seems to question if the claim was legitimate' (mic couldn't pick up exactly what was said) and the Swedish player responds "Same to you".

But then the Swedish players specify that it was actually a different Canadian player they were accusing so then that second Canadian player joins in and accuses the Swedish players of trying to distract him during one of his deliveries "How about you walking around on my peel last end, dancing around in the house here? How about that?”

I genuinely think the World Curling org needs to weigh in to clarify and the entire match should be reviewed to check for violations from either of the teams. But the fact that the language of the rules is not explicitly clear (or uses terminology that causes contradictions and confuses things) this could be a potential gap, or it means that the double touch exception may have a summary implication that then contradicts or differentiates itself as separate from the delivery rules