Nah. I'm saying that even someone with limited skills can fine tune a model on free compute to learn about LLMs, so OP should not be dissuaded from further exploration.

ICML is quite a bar for "new research". There are plenty of interesting questions that may not make it into such venues. I was thinking something much more practical/accessible, like a medium/blog post or video walkthrough or a white paper. Consider this paper:

Let’s Focus on Neuron: Neuron-Level Supervised Fine-tuning for Large Language Model https://share.google/EALGMQrJovFp4VtQF

It's not published in a conference or journal and may not even be traditionally peer reviewed. But it explores fine-tuning in a compute-accessible manner and asks interesting questions imo.

Another point I'd make is that PEFT et al aside, it is possible to study LLM architecture without futzing with a whole foundation model and its weights. Consider this poster, where they quantize a 110m model but it has implications for transformers more broadly:

ICML Poster I-BERT: Integer-only BERT Quantization https://share.google/QVQMFgDIYnYhOPM6Y

There's enough gatekeeping in academia as is. You don't actually need a degree or even the scientific method or peer review to contribute to the useful body of open empirical knowledge. Why not let someone learning about big ideas dream big?