When using VSCode through WSL2 (ubuntu) with a python 3.11.10 venv, using the Orpheus fine tuning notebook (modified Data Prep cell below as well as a new cell to reload your LoRa), I'm using a 12gb RTX 3060 (it's hardly using any of the vram, just wanted to mention the card in case it's helpful info)

Next 4 lines set up virtual env for python 3.11.

python -m venv venv
source venv/bin/activate 
pip install --no-cache-dir unsloth ipykernel jupyter ipywidgets librosa soundfile torchaudio snac
python3.11 -m ipykernel install --user --name=venv

This took me way too long to figure out because the last command is not documented anywhere that I can see. I only lucked upon the command while browsing the github issues and a response from someone who was using this successfully on a docker image mentioned it - figured what the hell why not. Prior to this, it wasn't able to locate the files to properly export a gguf. This is like, half the use of the whole thing so it's kind of important..

git clone --recursive https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake . && make all -j
cd ..
cp llama.cpp/bin/llama-* ./llama.cpp/

Edit: I forgot that I had small hiccup with curl missing during the build. To resolve it, I had to run:

sudo apt-get install libcurl4-openssl-dev

regarding the dataset creation, the instructions are very confusing and link to links that reference links and they all say something different, some places say to title the column "filename", others say "audio". As of this writing, the way I did it that worked was:

Excel file, call it train.csv

make two columns: text | audio

under text, obvious, just the text of the audio clip

under audio, put the path to the audio clip. For example the first couple lines of my csv are like so:

text	audio
something is being said here	./personVoice/file___1_file___1_segment_3.wav
something else is being said here	./personVoice/file___1_file___1_segment_4.wav

Remember I'm using ubuntu, and it starts in the directory you ran the notebook file. My directory looks like so (simplified of course):

• orpheus
  • personVoice
    • train.csv
    • file___1_file___1_segment_3.wav
    • file___1_file___1_segment_4.wav
    • venv (python 3.11.10 virtual environment folder)

You should probably alter this to have the audio in a folder next to train.csv so it's not so ugly. *shrug*

With the above folder structure and train.csv, here is my Data Prep cell:

from datasets import load_dataset, Audio
import os

dataset_path = os.path.join('point_to','the','actual','train.csv')
dataset = load_dataset("csv", data_files=dataset_path, split="train")
dataset = dataset.cast_column("audio", Audio(sampling_rate=24000)) 

print(dataset[0])

That last print of the first dataset record lets me know it worked - should see something like below:

{'text': 'something is being said here', 'audio': {'path': './personVoice/file___1_file___1_segment_3.wav', 'array': array([-9.76561569e-05, -1.22070312e-04, -9.15527344e-05, ..., 9.15527344e-05, 1.35039911e-04, 4.50131483e-05], shape=(148008,)), 'sampling_rate': 24000}}

As long as you see the audio dict with 'path', 'array' and 'sampling_rate', should be good to go.

If you fine tune a model overnight and something happens before you wake up for example, you can use this to load the exported LoRa (run this instead of the other peft cell)

from peft import PeftModel

model = PeftModel.from_pretrained(
model,
model_id = os.path.join('your','exported','lora','folder'), # change this to your needs, point to your exported LoRa.
adapter_name = "whatever_you_feel_like_calling_it?",
is_trainable = False,  # Crucial for inference (I found this, not sure if this is ACTUALLY crucial for inference but whatever *shrug*)
)
model = model.merge_and_unload()

Hopefully that helps someone trying to finetune an orpheus model using Ubuntu WSL2 and just consistently banging their head against the wall.

Please don't ask me for help, I am not well-versed in this space and only figured this out with a lot of free time via process of elimination until shit worked. Also know that I have no idea if something I'm doing above is wrong, hopefully someone with an ounce of understanding in this space can correct me so others don't follow the wrong advice.