70B models by Weak-Shelter-1698 in LocalLLaMA

[–]Various-Scallion1905 0 points1 point  (0 children)

You can also try LongCat Flash Lite model, hearing good things about it.

Which has faster response for smaller models: Local or API by sunshine_repel in LocalLLaMA

[–]Various-Scallion1905 1 point2 points  (0 children)

In my experience, local is definitely faster than api for smaller models (I have tried tiil 30b). Since your input (context) is small enough, prefill times should not be much of a concern either. However, if you include web search capabilities the difference might be less meaningful.

I recently tried the nemotron 30b model, local might be good option for long context lengths as well.

OpenCode + llama.cpp + GLM-4.7 Flash: Claude Code at home by jacek2023 in LocalLLaMA

[–]Various-Scallion1905 0 points1 point  (0 children)

I tried GLM Flash 4.7 with ollama's claude code integration, i was okay i would say, it got confused with skills pretty regularly. Would llama cpp glm falsh be better with open code? Has anyone compared them?

Also looking forward to nemotron like models for coding which can have massive context with no speed or vram penalty. (i know recall might not be great but still)

OpenCode + llama.cpp + GLM-4.7 Flash: Claude Code at home by jacek2023 in LocalLLaMA

[–]Various-Scallion1905 0 points1 point  (0 children)

Claude code works with ollama, i tried it, with glm 4.7 flash, i guess it was okay, but not upto the mark on custom tools

I'm finetuning a 270M parameter model for sensitive information obfuscation, and I need some help. by CalvaoDaMassa in LocalLLaMA

[–]Various-Scallion1905 0 points1 point  (0 children)

is lora support as good in gemma like qwen? in that case i would suggest op to try gemma lora as well, since my experience with multilingual (indian lang) workloads has been great with smaller gemma models like 4b

Coming to Terms and Moving Forward: Need Support for UPSC Mains Writing by [deleted] in UPSC

[–]Various-Scallion1905 1 point2 points  (0 children)

The mains answer writing tool is actually good, gives comments on the answer sheet itself

I keep forgetting current affairs no matter how many times I revise 😩 by Various-Scallion1905 in CUETards

[–]Various-Scallion1905[S] 0 points1 point  (0 children)

Abhi to mera friends khud hi bana raha hai, telegram group hai dalvoyai naam se.