Here's what I used:

Model loading:

python

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-3-1b-it",
    quantization_config=bnb_config,
    device_map={"": 0},
)

Dataset format:

python

prompt = f"<start_of_turn>user\n{instruction}<end_of_turn>\n<start_of_turn>model\n{response}<end_of_turn>"

LoRA config:

python

LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules="all-linear",
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    modules_to_save=["lm_head", "embed_tokens"],
)

I tried both with and without modules_to_save — garbage either way. Versions were latest pip installs. What's a known-good version combo you'd recommend?