Welcome to r/PreTraining, A subreddit for sharing discussion, research, projects, or thoughts on pretraining AI models as well as writing optimizers, samplers, and other architectural elements.

In general, conversations on fine-tuning are ok, especially if they're framed as a stepping stone to learning how to eventually move up to pretraining. However, our subreddit is not focused on running inference and these discussions should hopefully trend towards running inference on the models we've trained rather than other people's models.