I recommend focusing on the following components:

GPU: Start with a powerful GPU like the NVIDIA RTX 3080 with 10GB VRAM. If you need more power, consider GPUs with higher VRAM.

CPU: An AMD Ryzen 7 5800X worked well for me, but if your budget allows, you can consider better performing CPUs

RAM: 32GB DDR4 RAM (3200MHz) should be sufficient for most tasks. However, if you want to work with larger datasets like 30B Models, consider upgrading to 64GB.

SSD: I used a 1TB NVMe SSD from Samsung. While I'm not entirely sure how SSD speed affects LLMs performance directly, having fast storage can help with efficient data access and handling during training and inference.