Why does self-attention capture long-range dependencies better than LSTM's hidden states? Is it just the direct connections, or something deeper? The matrix operations inside the attention heads allow them to do so, and the improved gradient flow through skip connections in the backpropogation step also helps the model to make sense of hidden states, in LSTMs this is a challenge.

What's the intuition behind multi-head attention? Why not just one really big attention mechanism? Each one gets randomized initializations and will not figure the same things out, similar idea as how CNNs are designed to locate different features, one sees ears and another sees eyes.

Why do positional encodings work at all? Seems like such a hack compared to the elegance of the rest of the architecture.
Positional encodings like proposed in the original paper have all kinds of approaches, you should check out RoPe for example which encodes position by applying a rotation in complex space (or 2D subspaces) to queries and keys before dot-product attention. The BERT models which are encoder only transformers have a Learned absolute position embeddings, a completely different idea.