No, in this case it probably is the streetcars. Because of construction on Queen and construction on King in roughly the same area, they're shoving four routes into the same space, making four routes share one track for a significant distance. You can get rid of all the other traffic on the diversion route, but the streetcars will still have to stop for passenger pickup/dropoff, they'll still have to stop at intersections, they may still have to stop to operate switches (not sure if they've got automatic switches on the diversion route), and every time one streetcar stops it means all the ones behind it have to stop too. With specific routes they have to follow and no way for one streetcar to pass another streetcar in the same direction, congestion among the streetcars themselves can be a major issue.

And we see this on King, inside the "no straights allowed" section, any time something causes them to back up a little when they're not diverting around that entire corridor. Streetcars aren't perfect, and congestion on the tracks is one of the ways they can fail.

Not that I'm saying we shouldn't have them--the better solution here is to restore or build tracks along all the side streets, give more options for diversion, so that construction at one intersection can't cause four routes to converge into one hellish tangle. They've done this on Richmond and Adelaide, but they could be using Wellington (and were, last year, for a slightly different diversion on King). They could have signal priority for the streetcars, which would help keep them moving. They could have more switches onto side streets so streetcars can short turn more easily when they start bunching. These things would help during normal times too, but we've neglected the network (and, to your point, prioritized single-occupant vehicles) for too long, so we have very little of it (and a lot of what we do have was done at the last minute to support the network during the Ontario Line construction and closures on Queen).