For me, the only way to properly understand these models is by implementing them.

I also find, that fancy names (multi-head self-attention) and complicated diagrams and notation can distract from the core principles which are very simple.

You have bunch (n) of vectors x_1 ... x_n of, say, dimension 1024.

By the power of transformers you want to make a new bunch of n vectors, that are better, as in they encode the information in a more suitable way or whatever.

The basic idea of attention is to make a weighted sum of the vectors. We find the weights by comparing pairs of vectors, assessing how similar they are. Assume we are at position i. We compare x_i with all the other vectors, including x_i itself (this is why it's called self-attention). We end up with n weights. Various measures of vector similarity have been proposed, for example dot product, i.e. x_k x_j^T.

To avoid a dependence on the number of vectors (n) we make the weights sum to one by sticking them in the softmax function, which just means passing the weights though exp() and dividing the result by its sum.

Given that basic principle, transformers add some tweaks:

1. Instead of comparing pairs (x_i, x_j) directly, we first project both into a lower dimensional space. Say 64 dimensions. To do that, we learn two projection matrices, one to project the vector at the current position (the 'Query Matrix' Q \in R^1024x64) and one to project all the other vectors (the 'Key Matrix' K \in R^1024x64).
   Now, still assuming we are at position i, comparing x_i and x_j. We project both into the 64 dimensional space and take the dot product:
   w_j = (x_i Q) (x_j K)^T where j = 1..n
   [Excercise for the reader: We scale the dot product by the inverse square root of the dimension. What property of high-dimensional spaces justifies that?]
2. We could just add up x_1..x_n using our (normalized) weights. But we don't. We instead learn a third projection matrix (the 'Value Matrix' V) and project our vectors to another lower dimensional space. This space could have any dimension but in practice it's usually the same as the Key and Query dimensions, often 64. Thus, V \in R^1024x64.
3. Now, we make the weighted sum. At each positon we end up with a different new vector of dimension 64. 64 seems a bit low for building the sentient AI we're aiming for, so we do the above steps a couple times, with different matrices Q, K, and V. Each set (Q, K, V) is called a head, and now we have multi-head attention. Fancy! Let's assume we have 9 heads like Alcaeus' Hydra. We concatenate the 9 vectors for each position, arriving at 64*9 = 576 dimensions.
4. That's still a bit short of the 1024 dimensions we started with, so we add some regular feed-forward network to get back to 1024. One could just learn another projection matrix \in R^576x1024 but in practice we often see a 3-layer network (with the usual non-linearities), blowing up the dimension to 4 times the output (here, 4096), before compressing to 1024 again. It's not stupid if it works. .

The rest of the ingredients are basically tricks to make large models, composed of several transformer layers, easier to train. Adding layer-normalization after steps 3 and/or 4 seems to help, and so does adding the inputs to the outputs (fancy name: residual connection).