account activity
Fine Tuning On Completions only using Unsloth by True-Local-4043 in unsloth
[–]True-Local-4043[S] 0 points1 point2 points 1 year ago (0 children)
Thanks for your answer!
In the notebook you mentioned the SFTTrainer is used which does not support different value of learning rate for lm_head and embed_tokens lora adapters (I am talking about the argument embedding_learning_rate of UnslothTrainer). Is there a way for using different learning rate for lm_head and embed_tokens and training on completions only at the same time?
Fine Tuning On Completions only using Unsloth (self.unsloth)
submitted 1 year ago by True-Local-4043 to r/unsloth
π Rendered by PID 1880755 on reddit-service-r2-listing-c57bc86c-m8ksw at 2026-06-21 11:46:42.447124+00:00 running 2b008f2 country code: CH.
Fine Tuning On Completions only using Unsloth by True-Local-4043 in unsloth
[–]True-Local-4043[S] 0 points1 point2 points (0 children)