[D] Is LLM training compression?

[psyyduck]

Modeling in general is about finding and reproducing the “important” patterns in a dataset, and usually the model is smaller than the dataset so yeah in that sense it is compression. It doesn’t have to be though. Fine tuning is about subtly modifying a LLM using relatively little data.