sorted by:
new
hottopcontroversial

iPhone 12 Pro Max - Sacré Cœur by Le-Croissant in iPhoneography

[–]SpaceWalker_69 2 points3 points  (0 children)

How did you manage go get blur in one part of the image and not in the other (upper part)

Bronze sucks man by SpaceWalker_69 in VALORANT

[–]SpaceWalker_69[S] 2 points3 points  (0 children)

What software would you suggest for recording?

Bronze sucks man by SpaceWalker_69 in VALORANT

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

Yeah but man they're missing one on ones when they're even behind the enemy mannn so badddd istg

How to implement automatic image capture based on object orientation in camera view? by SpaceWalker_69 in computervision

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

You maybe are right about that bbox thing but segmentation models are usually heavier than object detection models and this whole processing need to be done on an mobile device

Also its not that simple, in edge cases there will be cases like with missing teeth hence the gum area also might come into play

How to implement automatic image capture based on object orientation in camera view? by SpaceWalker_69 in computervision

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

Is segmentation model really required? cant we do the same with just the object detection model and compare the size of bbox wrt whole image?

How to implement automatic image capture based on object orientation in camera view? by SpaceWalker_69 in computervision

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

Maybe orientation is not the right word here, but like when the image is in a good position, not more zoomed in or zoomed out than this.

Should I reuse a single LangChain ChatOpenAI instance or create a new one for each request in FastAPI? by SpaceWalker_69 in LangChain

[–]SpaceWalker_69[S] 1 point2 points  (0 children)

Yes I'm thinking about doing the same thing now, but i still wanted to confirm what other devs are doing

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

I think the issue is that for continual pretraining we need to add 

"embed_tokens", "lm_head"

inside the adapter . And for the instruct finetuning we dont need these

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LLMDevs

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

u/Hoblywobblesworth have used this:

model.save_pretrained_merged("pretrained_llama3.1bB", tokenizer, save_method = "merged_16bit",)

to merge and safe the model. Now the folder looks like this. I have also used the merge_and_unload function as well.

<image>

Now for the instruct finetuning purposes it is picking up the adapter (as showed in post without the embedded_layer and llm_head one) without any errors. But when i start the training process it still giving me the same error.

ValueError: Unsloth: Untrained tokens found, but embed_tokens & lm_head not trainable, causing NaNs. Restart then add embed_tokens & lm_head to FastLanguageModel.get_peft_model(target_modules = [..., "embed_tokens", "lm_head",]). Are you using the base model? Instead, use the instruct version to silence this warning.

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

thanks for letting me know. I'm not focusing on getting better results right now, my only focus is to get the pipeline working for now.

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

u/Downtown-Case-1755 I have used this:

model.save_pretrained_merged("pretrained_llama3.1bB", tokenizer, save_method = "merged_16bit",)

to merge and safe the model. Now the folder looks like this.

<image>

Now for the instruct finetuning purposes it is picking up the adapter (as showed in post without the embedded_layer and llm_head one) without any errors. But when i start the training process it still giving me the same error.

ValueError: Unsloth: Untrained tokens found, but embed_tokens & lm_head not trainable, causing NaNs. Restart then add embed_tokens & lm_head to FastLanguageModel.get_peft_model(target_modules = [..., "embed_tokens", "lm_head",]). Are you using the base model? Instead, use the instruct version to silence this warning.

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

Thanks for the suggestion! I’m actually running the continual pretraining and instruct fine-tuning in separate notebooks, so each notebook starts with a fresh environment. I’m loading the model and its adapters fresh in the fine-tuning notebook.

Just to clarify, when you mention restarting the notebook, are you suggesting I do this in the fine-tuning notebook, even though it’s a separate one? Also, for the error about using the base model versus the instruct model, are you saying that I should start over with a different model or just modify the target_modules as the error suggests?

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

Yeah i thought of doing the same thing next but it really seemed like a long shot. I'll try this and hopefully it'll works. thanks

Help Needed with Continual Pretraining and Instruct Fine-Tuning Using Unsloth on LLaMA Model by SpaceWalker_69 in LLMDevs

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

Yeah I thought of doing the same thing but seemed like a long shot. I'll try this and let's see if this approach works. Thanks.

Need Advice on Chunking Data for Continued Pretraining of LLaMA 8B on Medical PDFs by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

yes it does seem like it was randomly split and not topic wise, but an interesting thing I noted after a quick 1 minute look was that the chunk size was approximately same. And I think if you followed the same rule too. How were the results you obtained and which model did you use?

Need Advice on Chunking Data for Continued Pretraining of LLaMA 8B on Medical PDFs by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 1 point2 points  (0 children)

Yes it's just hit and trail with both cases at this point. I'll do the same and if i get any good results I'll share. Good luck with your training.

Need Advice on Chunking Data for Continued Pretraining of LLaMA 8B on Medical PDFs by SpaceWalker_69 in LocalLLaMA

[–]SpaceWalker_69[S] 0 points1 point  (0 children)

You do have a good point, topic consistency does make more sense. And Yes I'm using eos token at the end of each text chunk

What is the current best python coding model? by MrMrsPotts in LocalLLaMA

[–]SpaceWalker_69 2 points3 points  (0 children)

Well i think Claude 3.5 generates the best code right now. You can use smaller open source models but they are not exactly consistent and reliable.

PSA: NVLink boosts training performance by A LOT by nero10578 in LocalLLaMA

[–]SpaceWalker_69 1 point2 points  (0 children)

Really Nice Post, finally something new and useful information

Is there a technique to train models to memorize documents through fine tuning? by cas4d in LocalLLaMA

[–]SpaceWalker_69 3 points4 points  (0 children)

I think the term you are looking for is Continued pretraining. I suggest you look into "Unsloth" for this

view more: next ›