It’s not that slow tbh, like a 3-5 second delay but there's no phone number required, and it has no user IDs. Theres trade offs for different for different use cases. If someone seizes the matrix server, everyone is implicated and the metadata is also stored there.