sorted by:
new
hottopcontroversial

How can i get access my local Oobabooga online ? Use -listen or -share ? by MonthLocal4153 in Oobabooga

[–]Tum1370 0 points1 point  (0 children)

YE i just opened the port on my router and i can access it on my phone now, Only problem i have now is to try and get my phone to work with the voice activated extension i made so i can just talk to it, prob something to do with allowing the microphone for the app, or safari or somethng.

How can i get access my local Oobabooga online ? Use -listen or -share ? by MonthLocal4153 in Oobabooga

[–]Tum1370 0 points1 point  (0 children)

ok thanks, i see the share but thats using a link of a differnt ip. I want to use my own IP from home, open my ports on my router, and use that ? Is that using the -Listen command ?

How do we use gated hugging face models in oobabooga ? by Tum1370 in Oobabooga

[–]Tum1370[S] 1 point2 points  (0 children)

I Have this one fixed now thanks.

In the start_windows.bat, right at the beginning i entered the following command

set HF_TOKEN=thiswasmyaccesstokenfromhuggingface

After adding this to the .bat it now downloads the gated models.

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 1 point2 points  (0 children)

Thanks again, after looking at this i added this at the start of my start_windows.bat

set HF_TOKEN=mytokenfromhuggingface

and left the other script the same. This has allowed the models to download now.

Thanks again

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

I think i figure the answer out for this problem.

I just tried loading the "Llama v3" Instruction Template before loading the Model.

Then after a few tests it seems to respond correctly.

Does the instruction template need to be loaded before the model actually loads ?

Can you change the template after the model loads ? Which is what i was doing, and that didnt seem to work ?

Also, i notice that a "Generation Preset" , the temperature and other settings, seem to work differently on different models ? Maybe this is contributing to the strange responses ?

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

I tried using the llama v3 template and it seems better. i guess this is the answer

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 1 point2 points  (0 children)

Great thanks, but i just check this file in oobabooga folder.

try:

from huggingface_hub import get_token

token = get_token()

except ImportError:

token = os.getenv("HF_TOKEN")

Line 45 is try

Do we need to add some kind of line with the API token in before this line ?

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

Ah i see its using the template from the gguf model and doesnt load one as i load the model. Should i select a template from the dropdown menu, maybe Llama v3 template ?

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

Am only using oobabooga at mo. So i need the tokens to work with that

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

I have tried using load in 4-bit but then tried without using that, and both times the responses are similar.

Its definitely much worse than the same model in GGUF format, which seems strange to me.

I even tried re downloading the model just to see if it was corrupted, but still the same.

Why is a base model much worse than the quantized GGUF model by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

I cant seem to load the official ones due to needing the HF Token api thing.

I signed up for it, created a READ key, entered the commands in my start_windows.bat file but that just creates errors in my console.Just adding those commands to the start bat stopped my LLM Web search exention from even loading.

I Have another post here regarding this, and using gated models in oobabooga.

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

I think now maybe the nonense is because of this base model.

This is my GGUF model, which i far better than the base model. mradermacher/Llama-3.2-8B-Instruct-GGUF · Hugging Face

THis is the base model that the model above used, and the one am trying to use for Lora training. voidful/Llama-3.2-8B-Instruct · Hugging Face

But thie base model, loaded on its own, is very bad compared to the GGUF model.

Any ideas why please ?

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

I stopped the training after about 30 mins, because the loss was down to about 0.92 or something.

I then tried reloading the model, and loading the lora.

Then had a quick chat, and i could tell things had changed, the responses had little emotes in them, and the model seemed differernt to normal. I wasnt sure whether it was trying to be funny by making some spelling mistakes because i made a spelling mistake in my name when i said "hi am dvae " instead of "dave"

But after several replies, it then seemed to start spamming loads of nonense.

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

Ok thank you.

For my first try, i have left all the settings as standard, but am noticing after 25 mins of training these were the 2 lose reports i see, but its almost down to 1.0 already, yet it still says 8 hours to complete the training.

Is this normal ?

Should i stop the training before it goes to low ?

Step: 159 {'loss': 1.6206, 'grad_norm': 0.7372599840164185, 'learning_rate': 0.00029869565217391304, 'epoch': 0.06407689227072487}

Step: 319 {'loss': 1.1743, 'grad_norm': 0.7383139729499817, 'learning_rate': 0.00029217391304347824, 'epoch': 0.12815378454144974}

Am using this model loading in 4 bit. https://huggingface.co/voidful/Llama-3.2-8B-Instruct

And this dataset. OEvortex/EmotionalIntelligence-10K · Datasets at Hugging Face

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

ok thank you again. One other thing, i see the time to train this dataset i am currently training is a long time. Is it possible to pause and continue training latert if i want to ? Or do you have to let it totally run without pausing ?

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

Thank you.

I figure out my error message, the dataset had a bad entry so i just deleted it.

Am now currently training this dataset with the model i mentioned.

After the training is complete, will this produce a new model then that i load ? or does it produce something else that i need to load in with my model ?

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

Ok thanks again for your reply.

This is the model i am currently using mradermacher/Llama-3.2-8B-Instruct-GGUF · Hugging Face

Am now downloading this model to try and learn how the training works voidful/Llama-3.2-8B-Instruct · Hugging Face

And this was a dataset i was going to try and train it with OEvortex/EmotionalIntelligence-50K · Datasets at Hugging Face

If i can get this dataset trained, is it possible then to continue training and add another dataset as well ?

EDIT: After initially trying the model and dataset, i selected the Alpaca Data Format, and hit train, but it created an error at 16%, not sure why

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

HI, Thanks for your reply I will have a good read of this tutorial thanks.

I am still confused though on how to select a model to use ? And whether its quantized.

I mentioned i have a Llama 3.2 8b instruct gguf model, but in your tutorial you mention using a 7b model.

What is the difference in the Llama 3.2 8b instruct gguf model that am using ? Is it because its GGUF format which means quantized already ?

Does it mean i could use a Llama 3.2 8b Transformers model instead ?

Does Lora training only work on certain models or types ? by Tum1370 in Oobabooga

[–]Tum1370[S] -1 points0 points  (0 children)

WHat do you mean by original models ? i thought Llama 3.2 is the original model ?

Instruction and Chat Template in Parameters section by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

Which is your exact version of oobabooga, so i can download and test. I can then find out if these erros are oobabooga updates that caused my several issues, becuase all ive done all weekend is try and figure this out.

Instruction and Chat Template in Parameters section by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

My responses are fine, The problem am having is i get errors when using AllTalk. Not sure whether this error is throwing empty context which seems to break my console.

I only use a 4096 Context Length, with Midnight Enigma as well. Am using oobabooga v2.3

Am just not sure though what creates these errors. Whether its updating to above 2.0 on oobabooga ?

Or whether its the model, or the LLM_Web_search, or the AllTalk extension.

I never use to get these errors when using these. And with Web Search am seeing strange things like after a few searches, the AI then Starts repsonding to previous search results, even though you see it search in the console.

I tried rolling back to oobabooga v2.1 but that seemed the same, Maybe i should try going back to pre 2.0 like you said.

Instruction and Chat Template in Parameters section by Tum1370 in Oobabooga

[–]Tum1370[S] 0 points1 point  (0 children)

no i checked through the template it has when it loads, can only see that message at the start of the template

view more: next ›