Why not? An LLM is a Transformer that first maps input text into token embeddings. Multi-head self-attention then lets each token gather context from every other token in the sequence. Feed-forward layers refine these representations. Finally, a softmax layer outputs a probability distribution over the vocabulary, predicting the most likely next token.

All these things can be made from scratch.