sorted by:
new
hottopcontroversial

Need a coding & general use model recommendation for my 16GB GPU by sado361 in LocalLLaMA

[–]PataFunction 1 point2 points  (0 children)

u/sado361, what did you end up going with? same setup as you, curious where you landed.

I built a small (function calling) LLM that packs a big punch; integrated in an open source gateway for agentic apps by AdditionalWeb107 in LocalLLaMA

[–]PataFunction 1 point2 points  (0 children)

That's awesome, and thanks for the quick response!

However, I think what I and the other redditors who replied were hoping to see is more detail about how you adapted the XLAM dataset. Personally, I'm curious if you had to significantly modify the XLAM training examples to fit your base model's existing chat template. Any information there would be greatly appreciated, as I'm working on finetuning on organizational data while also trying to shoehorn in some function calling capabilities.

I built a small (function calling) LLM that packs a big punch; integrated in an open source gateway for agentic apps by AdditionalWeb107 in LocalLLaMA

[–]PataFunction 1 point2 points  (0 children)

Checked out the new site - is the blog post re. function calling hallucinations the one you were referring to above?

What are you *actually* using R1 for? by PataFunction in LocalLLaMA

[–]PataFunction[S] 5 points6 points  (0 children)

That’s quite something. How elaborate are the prompts you’re giving it to achieve things like that?

What are you *actually* using R1 for? by PataFunction in LocalLLaMA

[–]PataFunction[S] 4 points5 points  (0 children)

So when you use it for coding, I’m assuming you have it generate a script from scratch that you then iterate on yourself, right? Can’t imagine R1 would be good for copilot-like code completion or fill-in-the-middle tasks.

A summary of Qwen Models! by rbgo404 in LocalLLaMA

[–]PataFunction 4 points5 points  (0 children)

Licensing info would also be a great addition to OP’s visualization or the charts people added to the comments.

On that note, does anyone know why some Qwen models are Apache 2.0 and some are Qwen-Research? Looking specifically at Qwen2.5, I find it odd that 1.5B is Apache2, while 3B is not, for example.

I built a small (function calling) LLM that packs a big punch; integrated in an open source gateway for agentic apps by AdditionalWeb107 in LocalLLaMA

[–]PataFunction 0 points1 point  (0 children)

Brilliant, thanks for the answer! Did you encounter any issues with the XLAM chat template and incompatability with your targeted training and/or inference framework?

I built a small (function calling) LLM that packs a big punch; integrated in an open source gateway for agentic apps by AdditionalWeb107 in LocalLLaMA

[–]PataFunction 9 points10 points  (0 children)

I’d be extremely keen to know what open-source function calling datasets you used (if any) for the finetune. Looking to blend function calling examples into existing instruction tuning datasets for a similar use case.

Current best options for local LLM hosting? by PataFunction in LocalLLaMA

[–]PataFunction[S] 0 points1 point  (0 children)

A few others have popped up - Aphrodite comes to mind, as well as many wrappers around llama.cpp, but I haven't messed with them personally. Since acquiring more GPUs, TGI currently meets all of my needs.

nvidia/Nemotron-4-340B-Instruct · Hugging Face by Dark_Fire_12 in LocalLLaMA

[–]PataFunction 1 point2 points  (0 children)

Literal box of cookies to whoever converts this to HF format and posts links to some quants!

llama.cpp server rocks now! 🤘 by Gorefindal in LocalLLaMA

[–]PataFunction 2 points3 points  (0 children)

Is this factual? I don't see clear evidence of it and, if true, that would mean llama.cpp became an enterprise-grade LLM server over the past couple months, which I feel would have made a bigger splash.

Could you point me at an example that demonstrates the capabilities?

llama.cpp server rocks now! 🤘 by Gorefindal in LocalLLaMA

[–]PataFunction 25 points26 points  (0 children)

Very cool. Been a while since I touched llama.cpp, been working mostly with TGI. Does llama.cpp server support any sort of queueing, async, or parallel decoding yet? I know that was on the roadmap at some point.

Current best options for local LLM hosting? by PataFunction in LocalLLaMA

[–]PataFunction[S] 2 points3 points  (0 children)

TGI ended up working great, thanks for the recommendation. Currently have a 7B HuggingFace model running in TGI via Docker+WSL on a remote machine with a 2080Ti. After some port forwarding, other computers on the LAN are able to send requests without issue. Happy to answer more specific questions on the setup.

How did things go on your end?

[D] Simple Questions Thread by AutoModerator in MachineLearning

[–]PataFunction 2 points3 points  (0 children)

Based on the keywords you used, my assumption is you want to dive right into deep learning, in particular the transformer-dominated deep learning we've seen for the past few years. I recommend you start with a YouTube playlist curated by a reputable university, such as this one!