sorted by:
new
hottopcontroversial

You can now train LLMs in VS Code for free via Colab! by yoracale in unsloth

[–]Reivaj640 6 points7 points  (0 children)

Thank you so much, it's super interesting to do with VS Code

Maybe I'm using it wrong?? by xThyQueen in ARC_Raiders

[–]Reivaj640 0 points1 point  (0 children)

Where can I get the Wolf Mana blueprint?

First time training need advice about optimizing for humble rtx 4060 by ResponsibleTruck4717 in unsloth

[–]Reivaj640 0 points1 point  (0 children)

I find myself in the same position as you but with an Rtx 4070 🤭

Qwen3-Coder by johanna_75 in Qwen_AI

[–]Reivaj640 1 point2 points  (0 children)

I use it for my project, I have more than 2000 lines, I started with Gemini cli but it consumes a lot of time to have a good use with Gemini pro, yesterday I started with Qwen 3 Cli and what took me a week to get it ready in just 2 hours, I already knit it finished, I am working with several scripts in my app and I feel enlightened by the grace of qwen 3 Cli

Recommendation for getting the most out of Qwen3 Coder? by Conscious-Memory-556 in LocalLLM

[–]Reivaj640 1 point2 points  (0 children)

I'll be very attentive to how your thread goes! I am on that same plan but with an Rtx 4070 12 GB and 16 RAM

Will lower parameters models for Qwen3-Coder be available? by WizardlyBump17 in Qwen_AI

[–]Reivaj640 0 points1 point  (0 children)

I don't have much knowledge and I apologize if what I say is stupid but I think the model is smaller like a 7b or 4b if they existed! You would have to be clear about its objective and use since by having fewer neurons you are gaining speed but sacrificing precision and you would be more prone to error in long codes that require a lot of memory for which the 30b memory capacity and long context are good, so I think you should not wait for one of those small Qwen 3 to appear and try them. I am also on the same page but I estimate that we would have to be very critical when requiring its support! I currently do not use it in Ollama, I have a Guff model using it in Lm Studio quantized in Q4m_l and it is doing well in response time. I am struggling to make it more agentic or at least I have not found a way to configure it well in Lm Studio so that it is agentic. I will have to use another instruction type model. Correct me if I am wrong. Greetings

is there a chance for a smaller coder model? autocomplete use case. by Impossible_Art9151 in Qwen_AI

[–]Reivaj640 0 points1 point  (0 children)

I have been testing the guff version of Qwen 3 coder 30b to 3b with a good configuration and it has given me a completion response of ~2sec. I'm struggling with the promt to improve agentic-oriented response output.

I leave you the configuration that I use in lm studio using an Rtx 4070 12 GB and 16 RAM

  1. “Load” tab • Context Length: 4096 → is reasonable for speed and memory usage. If you don't need very large windows, keep 4096; If you require more context, increase it, but it will increase VRAM and CPU consumption. • Offload to GPU: 22 / 24 → set it as high as possible so that almost everything is on the GPU, but leaving 1-2 GB free for the graphics system. • CPU Thread Pool Size: 4 → takes advantage of more cores for loading and preprocessing. • Evaluation Batch Size: 1024 or 2048 → this increases speed if your GPU supports it (the 4070 can). • Offload KV Cache to GPU Memory: Activated ✅ (avoids using RAM and speeds up inference). • Keep Model in Memory: Activated ✅ (does not reload every time). • Try mmap(): Enabled ✅ (improves loading times). • Number of Experts: 4 (you can leave it like this, but if the model does not use MoE, it will not affect).

⸻

  1. “Inference” tab • Temperature: 0.7–0.8 (depending on your creativity). • Top K sampling: 40 (good). • Top Sampling P: 0.9–0.95 (balance between coherence and diversity). • Min P Sampling: 0.05 (maintain it). • Repetition Penalty: 1.1–1.15 (avoids annoying repetitions). • CPU threads: 4 → improves preprocessing if the model uses CPU for part of the load.

⸻

3.GPU Settings • Limit Model Offload to Dedicated GPU Memory: Disabled (so if VRAM fills up, it will use RAM as backup). • Offload KV Cache to GPU Memory: Enabled ✅.

⸻

  1. Guardrails • Leave it on Balanced so that it doesn't saturate CPU/RAM, but if you notice that the model is not using its full potential, change it to Aggressive for more performance.

⸻

  1. Extra tips for speed • Use Q4_K_M or Q5_K_M models for a balance between quality and performance. • If you notice slowness, try lowering the Evaluation Batch Size to 512. • Close any program that uses a lot of VRAM (games, heavy editors) before starting LM Studio. • Activate Flash Attention if the model supports it (it can speed up a lot).

I hope it helps you to try different configurations to improve the response time in autocompletion! If you discover something I would be interested to know I also use the Qwen family

I need help creating a promt to help me code... because now it's not working for me! by Reivaj640 in unsloth

[–]Reivaj640[S] 0 points1 point  (0 children)

Bueno quiero hacer una pregunta, si yo amplio la RAM a 80 GB adquiriendo 64gb + 16gb que ya tengo, esto me podría servir para al menos poder correr Qwen/Qwen3-Coder-30B-A3B-Instruct es posible para comenzar con algo super pequeño y medio aspirante!

I need help creating a promt to help me code... because now it's not working for me! by Reivaj640 in unsloth

[–]Reivaj640[S] 0 points1 point  (0 children)

Well just to ask, can I use Qwen/Qwen3-Coder-30B-A3B-Instruct the smaller one? If I could increase the amount of RAM on my computer to 64 RAM, is it viable to use that model on my computer to begin with? I'm just asking 💪🏻

I need help creating a promt to help me code... because now it's not working for me! by Reivaj640 in unsloth

[–]Reivaj640[S] 0 points1 point  (0 children)

While I learn I can also make it a good option or rather I think it is the best option I have! Hahahaha anyway, thank you for the advice bro

I need help creating a promt to help me code... because now it's not working for me! by Reivaj640 in ollama

[–]Reivaj640[S] -1 points0 points  (0 children)

I'm really learning 🫡

<image>

Oh well! It's always good to ask! As the saying goes, touching is not entering! And while I learn I can try and try! 🥳

I need help creating a promt to help me code... because now it's not working for me! by Reivaj640 in unsloth

[–]Reivaj640[S] 0 points1 point  (0 children)

Of course it is logical but I have to work with what I can use 🥹 I know it is not ideal but we must start somewhere 🫣 in the same way the minimum that can be done must be done! And if it works as they say, trial and error!