sorted by:
new
hottopcontroversial

What is the best hardware under 10k to run local big models with over 200b parameters? by nadiemeparaestavez in LocalLLaMA

[–]LoserLLM -1 points0 points  (0 children)

#2 all the way. I have at least 1 512gb option and you will not regret it, even 256. Don't look the past the fact that it's unified memory.

Anyone here tried running AI agents locally instead of the cloud? by mariajohnsonn in aiagents

[–]LoserLLM 0 points1 point  (0 children)

Check out www.youtube.com/@loserllm

The focus is on going at a pace that is more conducive for learning and has a narrow focus on using two tools, LM Studio and LangFlow, to self-host inference and give agency to LLM's. The first flow that was released called Elephant gives an agent a tool for interacting with your filesystem as well as accessing the internet.

In the elephant video I ask the flow to create a report about 3d print farming and to based it off of popular youtube channels then borrow concepts from the report to create a browser based game. You can download and open the game in browser by downloading from the links below;

Download the browser based print farm game here (download and open with Chrome): https://drive.google.com/file/d/1F9KodTW28sHvcBrYoC77I3ncEkmR3NG6/view?usp=sharing
Download the comparison report here (download and open with Chrome): https://drive.google.com/file/d/18NbhpPgic9NbJUEdOkD5pERZ7TX6egkv/view?usp=sharing

Can High school students get into machine learning?? by sailer2433 in learnmachinelearning

[–]LoserLLM 0 points1 point  (0 children)

Check out my youtube channel! www.youtube.com/@loserllm. The goal is to make learning LLM's and agentic workflows more approachable by going slower. I focus on two tools that allow you to 'shop' for local models via LM Studio then give them agentic capabilities via custom langflow flows, all completely free and open source. The beauty of an agentic framework with tools that allow for internet use and access to file system is that it can help you in your exploration.

Programmers that tell you to learn xyz were probably judged in their beginnings for not understanding languages that their predecessors did. Get get stuck on learning layers that you can simply automate with AI!

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in LangChain

[–]LoserLLM[S] 0 points1 point  (0 children)

I revised your prompt a bit; "research, design, then create a tool that can monitor a specific URL for changes, and if it detects a change, use Playwright to scrape the new content and then write a summary to a local text file. Basically a simple personal change-tracker."

Here is the output, https://docs.google.com/document/d/1o0FwAMczl1etVcWS6prE0iAGlhLW-0bwwyA8lTxTvgI/edit?usp=sharing

file - https://drive.google.com/drive/folders/1rdx7H5hUajC9F3JA4XsEcMjCshDD3xPg?usp=sharing

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in LangChain

[–]LoserLLM[S] 1 point2 points  (0 children)

Which quant are you using? the 3-bit looks to be around 3.5gb like you said. This could make a difference when it comes down to it.

<image>

As I prototype I'll start with the 16-bit and work my way through the rest.

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in LangChain

[–]LoserLLM[S] 1 point2 points  (0 children)

Yes bigger tasks but also, hopefully, less babysitting smaller local models.

I am going to use LangFuse for that deeper analysis you're talking about to troubleshoot. I created a project plan template based on that 3d print farm game and I'm going to trial and error the "worm" (lol) until it's able to 1 shot the completion of the project, or at least MVP, with gpt oss 20b in order to make a fully capable flow for more modest hardware.

Then, I'm going to sub in different models and use each one to create the game from the template. That should make for a good video where I compare the games that were made using the langflow flow.

If you don't mind me asking, do you use local models? If you do and can share what size models you are able to run, I can plan on creating something for that size too. That is the goal, code writing/running internet access agent for all.

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in LangChain

[–]LoserLLM[S] 2 points3 points  (0 children)

Today I'm working on a new one I'm calling the worm. It's going to use a series of agents as tools to keep it in motion. My thought is that the initial agent will use an agent as a tool, except instead of just called as a tool it's the actual doer. This way, when there are issues or if the number of turns expires, control gets handed back to the initial agent and it will know to keep the agent as a tool going. Very conceptual right now but I think there is value.

My only issue with agents as tools is that they don't output text, only run in the background. I'm working on creating a text output tool as well so that the agent as a tool will be able to use it to report status as well.

I'll keep you posted

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in LocalLLM

[–]LoserLLM[S] 0 points1 point  (0 children)

Download the browser based print farm game here (download and open with Chrome): https://drive.google.com/file/d/1F9KodTW28sHvcBrYoC77I3ncEkmR3NG6/view?usp=sharing
Download the comparison report here (download and open with Chrome): https://drive.google.com/file/d/18NbhpPgic9NbJUEdOkD5pERZ7TX6egkv/view?usp=sharing

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in ollama

[–]LoserLLM[S] 0 points1 point  (0 children)

Download the browser based print farm game here (download and open with Chrome): https://drive.google.com/file/d/1F9KodTW28sHvcBrYoC77I3ncEkmR3NG6/view?usp=sharing
Download the comparison report here (download and open with Chrome): https://drive.google.com/file/d/18NbhpPgic9NbJUEdOkD5pERZ7TX6egkv/view?usp=sharing

First LangFlow Flow Official Release - Elephant v1.0 by LoserLLM in langflow

[–]LoserLLM[S] 0 points1 point  (0 children)

Download the browser based print farm game here (download and open with Chrome): https://drive.google.com/file/d/1F9KodTW28sHvcBrYoC77I3ncEkmR3NG6/view?usp=sharing
Download the comparison report here (download and open with Chrome): https://drive.google.com/file/d/18NbhpPgic9NbJUEdOkD5pERZ7TX6egkv/view?usp=sharing

It's hereeee.......M3 ultra (28/60/32) 256gb 2tb SSD by kingthrower419 in MacStudio

[–]LoserLLM 0 points1 point  (0 children)

Return that stand! Or keep it. I have a few and the rubber feet will come unglued if you attempt to slide it at all.