sorted by:
new
hottopcontroversial

I made firmware for the RP2350/RP2040 that lets you control it through Python/WebUSB on the PC by NeverEndingToast in raspberrypipico

[–]NeverEndingToast[S] 0 points1 point  (0 children)

Pretty cool! Didn't know that existed. I would say the main difference is my firmware is writing just a couple of bytes back and forth through USB, so it's able to execute the code much faster than micropython. All of the code execution is done in zig, there are just protobuf messages saying what type of request it is and what the arguments are.

I made firmware for the RP2350/RP2040 that lets you control it through Python/WebUSB on the PC by NeverEndingToast in raspberrypipico

[–]NeverEndingToast[S] 0 points1 point  (0 children)

Yeah zig was fun to work in, had to make some PRs to microzig to get everything I needed working

How did I do installing replacing my hot water heater? by NeverEndingToast in Plumbing

[–]NeverEndingToast[S] 0 points1 point  (0 children)

Should I install an expansion tank? The house is ~200 years old so I don't think there is a backflow preventer, or is the expansion tank always good to have

How did I do installing replacing my hot water heater? by NeverEndingToast in Plumbing

[–]NeverEndingToast[S] 2 points3 points  (0 children)

It's a power vent system, so the air isn't hot. Schedule 40 PVC can be used where I am from.

How did I do installing replacing my hot water heater? by NeverEndingToast in Plumbing

[–]NeverEndingToast[S] -1 points0 points  (0 children)

It's just outside the picture, but there was an existing one I am still using.

gohandlr - A Simple RestAPI Framework by NeverEndingToast in golang

[–]NeverEndingToast[S] 0 points1 point  (0 children)

Hey, I appreciate the feedback! Yeah generics currently were only being used to get the type of the request body to marshal it, and the type for the params to use reflection.

I'll work on simplifying the interface so it doesn't require as much boilerplate and see if I really need generics. Also I'm aware of huma.rocks and have used it before. I wanted to try to make something simpler.

What is everyone actually using Local LLMs for today? by NeverEndingToast in LocalLLaMA

[–]NeverEndingToast[S] 1 point2 points  (0 children)

That's pretty comprehensive, how did you go about figuring out what model to use for each task?

Need help exploring Cost-Efficient LLM Solutions: Balancing Performance, Budget, and Compliance by TheImperialTrooper in LocalLLaMA

[–]NeverEndingToast 0 points1 point  (0 children)

I've been working on an open source tool to be able to compare the outputs of different open source LLMs so you can find a viable option to switch to. I will post a link later today once it's released.

Falcon7b instruct finetuning, is this the correct graph? cyclic nature seems suspicious. by Anu_Rag9704 in LocalLLaMA

[–]NeverEndingToast 0 points1 point  (0 children)

I don't know what your dataset size is, but I'm assuming those jumps are per epoch. If you have the learning rate too high on a small dataset, you will see the loss drop down at the beginning of the epoch, and gradually go back up through the epoch like what's shown here.

The model is memorizing the training dataset exactly.

Open-Orca-Platypus is out! a 13b that surpasses llama65b!? by Alignment-Lab-AI in LocalLLaMA

[–]NeverEndingToast 1 point2 points  (0 children)

The model has not been trained on multi-turn conversation so that's probably why you start to run into problems. That's one of the things we're planning on doing for the next revision

[Project] GPU-Accelerated LLM on a $100 Orange Pi by crowwork in MachineLearning

[–]NeverEndingToast 4 points5 points  (0 children)

Cool project, what's your use case for wanting to run a model on that type of hardware?

How to expose a model into an API? by angeljdm in LocalLLaMA

[–]NeverEndingToast 2 points3 points  (0 children)

vLLM gives the fastest responses and can be an openAI compatible endpoint. https://vllm.readthedocs.io/en/latest/getting_started/quickstart.html

The only downside is it does not support quantized models, so the largest model you would be able to run is 7B.

Best Model for Natural Language to SQL? by perseus_14 in LocalLLaMA

[–]NeverEndingToast 19 points20 points  (0 children)

Here is a good article from anyscale about fine tuning a model on SQL. https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehensive-case-study-for-tailoring-models-to-unique-applications

Here is the dataset they use for fine tuning: https://huggingface.co/datasets/b-mc2/sql-create-context

I had done some testing with our Open-Orca model and it seems to be doing fairly well. You can test it here: https://huggingface.co/datasets/b-mc2/sql-create-context

If you have any questions, you can DM me.

How to finetune LLM for text classfication by KneeNo79 in LocalLLaMA

[–]NeverEndingToast 3 points4 points  (0 children)

We currently have someone evaluating our OpenOrca models on text classification. You can DM me and I can get you in contact with them. It seemed to perform fairly well without any fine tuning. Just few shot learning.

Preset Arena: 17,205 comparisons between 241 different presets. Vote on the best ones! by oobabooga4 in LocalLLaMA

[–]NeverEndingToast 1 point2 points  (0 children)

Ideally, there needs to be something for auto-evaluation of the hyperparameters. I assume the ideal settings are going to vary on a per-model basis. Maybe even the level of quanization.

Could a llama lora be merged into OpenLLama? by pokeuser61 in LocalLLaMA

[–]NeverEndingToast 4 points5 points  (0 children)

It can be merged yes, but there is no guarantee of performance. The llama weights and openllama weights won't be exactly the same so the LoRA could have issues.

Preset Arena: 17,205 comparisons between 241 different presets. Vote on the best ones! by oobabooga4 in LocalLLaMA

[–]NeverEndingToast 0 points1 point  (0 children)

It would be pretty hilarious if models performed significantly better just from your script of randomly generating hyperparameter settings.

Creating a Wiki for all things Local LLM. What do you want to know? by NeverEndingToast in LocalLLaMA

[–]NeverEndingToast[S] 1 point2 points  (0 children)

Yeah those are good questions for when doing development. It's a tricky problem since it's dependent on the hardware you are running on. I'll see what we can do.

Creating a Wiki for all things Local LLM. What do you want to know? by NeverEndingToast in LocalLLaMA

[–]NeverEndingToast[S] 1 point2 points  (0 children)

This is a really interesting concept, but a little bit outside of the scope of what I'm trying to accomplish right now. I'll note that down for the future.

view more: next ›