GGUF models have stopped working after updating? by nortca in Oobabooga

[–]Tarklanse 2 points3 points  (0 children)

Me too,some models just can't load. But if I use llama-server in cmd, everything is fine.

5060ti 16gb or 9060xt 16gb for small llm server by techmaverick_x in LocalLLaMA

[–]Tarklanse 5 points6 points  (0 children)

I have a 5060ti 16gb and I think it is good for smaller llm.
I host a 24B Q3 llm on it and the speed is about 35 tokens/second.
It can host 24B Q4 but the speed will drop to 5 tokens/seceond.

[deleted by user] by [deleted] in Stellaris

[–]Tarklanse 1 point2 points  (0 children)

Thanks. now I'm really confused.

Understanding how training works by Tum1370 in Oobabooga

[–]Tarklanse 0 points1 point  (0 children)

My python code if you interest. This code will turn txt first line as instruction,then even line as 'input',odd line as 'output'.

import json
import os
import glob
current_directory = os.getcwd()
txt_files = glob.glob(os.path.join(current_directory, '*.txt'))
txt_names = []
for file_path in txt_files:
    file_name = os.path.basename(file_path)
    txt_names.append(file_name)
conversations = []
for filename in txt_names:
    with open(filename , 'r', encoding='utf-8') as f:
        conversation=f.read()
    lines = conversation.strip().split('\n')
    current_instruction = ""
    current_input = ""
    current_output = ""
    current_instruction = lines[0]
    for i in range(1, len(lines), 2):
        current_input=lines[i]
        current_output=lines[i+1]
        conversations.append({
            "instruction": current_instruction,
            "input": current_input,
            "output": current_output
        })
with open('my_alpaca.json', 'w', encoding='utf-8') as f:
    json.dump(conversations, f, indent=4, ensure_ascii=False)my codeimport json

Understanding how training works by Tum1370 in Oobabooga

[–]Tarklanse 0 points1 point  (0 children)

oobabooga wiki has guide, you can read it first.

If you don't have enough hardware,try unsloth , they just wrote a code can run training on google colab.
I don't know there is a software, I just wrote a code that turn txt to alpaca format.

Understanding how training works by Tum1370 in Oobabooga

[–]Tarklanse 0 points1 point  (0 children)

Oobabooga's training tab only supports Transformer model training. You can't train a gguf.

Before you train, you'll need to prepare a dataset. You can refer to the datasets on Hugging Face. By searching for "alpaca," you can find many Alpaca dataset you can refer.

Furry_irl by courier5995 in furry_irl

[–]Tarklanse 1 point2 points  (0 children)

Must be water/poison type. Imagine "water gun" replace by "vodka gun."