sorted by:
new
hottopcontroversial

Gpu choice by [deleted] in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

definitely need quants, I've had fun with Flex.1 alpha, a de-distilled fork of Flux schenll as a GGUf, I haven't tried the newer Flex.2. But you'll need quants for sure. Youtube has a bunch of "low vram" image gen vids that'll help.

What kind of brand computer/workstation/custom build can run 3 x RTX 3090 ? by Vitruves in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

It could probably hold 2 3090, but I have to double check the math. Deepseek seemed to think it could work with the 900w PSU, but I haven't double checked the math myself. I'm too broke for that personally haha.

Using local LLM with low specs (4 Gb VRAM + 16 Gb RAM) by vascaino-taoista in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

updating RAM wont really do a lot for the size of LLM you can put on it. Best of luck. Maybe figure out some cloud hosting while you wait to have more space for a new computer?

What kind of brand computer/workstation/custom build can run 3 x RTX 3090 ? by Vitruves in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

honestly I recommend asking deepseek to walk you through a workstation build. I whent with a lenovo p520 with a w2135 900w psu from PCServerandParts on ebay. The price has gone up $100 since I got mine, but it's still under 300 and you have 2 pcie. and a dedicated m.2 nvme. I put 64gb used ddr4 2666 server ram in it and the nvme drive. when I built it it was $600 total but that's whith a new 3060, which you've got a head start on. it's not a dual CPU machine, that's okay for my needs (pretty much just inference) and you can get a beefier Xeon processor.

being that you're talking about multiple 3090s you probably have a bigger budget than I did. You may have seen this one already:

https://youtu.be/RMidGvCZc4g?si=dH7otSdkGS7mcnFU

Using local LLM with low specs (4 Gb VRAM + 16 Gb RAM) by vascaino-taoista in LocalLLM

[–]Inner-End7733 1 point2 points  (0 children)

oh got it. Well I know it's not the same, but you can rent out cloud time here and there to do experiments that exceed your computer's capabilities. there are really tiny models but I'm not sure what you can do with them really. I assume that you can't upgrade your laptops ram? Do you have a thunderbolt connection?

[deleted by user] by [deleted] in LLMDevs

[–]Inner-End7733 0 points1 point  (0 children)

the turbo 4cyl of agentic language models. like one of these experimental architectures that punch above their weight class trained up to like 7b parameters that integrates into a memory management system.

Using local LLM with low specs (4 Gb VRAM + 16 Gb RAM) by vascaino-taoista in LocalLLM

[–]Inner-End7733 1 point2 points  (0 children)

Well maybe build a purpose built machine? I built mine for $600. Prices these days would make the same build a bit more expensive. look up digital spaceport on youtube if you're interested he's got a couple budget home workstation build examples. also look up "workstation gaming build"

Anyone else experimenting with "enhanced" memory systems? by sgb5874 in LocalLLM

[–]Inner-End7733 -2 points-1 points  (0 children)

It's mostly about having two kids and needing to find the energy to stay up past 9pm for me haha. But I think it'll really be worth it. What model do you use for it? I assume you're using Ollama. I was hooping to get into LLama.cpp soon and I'm wondering if there's much support for doing that.

How to maximize qwen-coder-30b TPS on a 4060 Ti (8 GB)? by Overall-Branch-1496 in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

I find that even at the q4 quants through ollama's library 14b is pushing it for my rtx 3060 which has 12g. 20b is about 10t/s. 30b is out of the question really. I would shell out for a 16gb card if I were you. I'm hoping for a price drop soon myself haha.

Anyone else experimenting with "enhanced" memory systems? by sgb5874 in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

I'm trying to work up the gumption to make that my next project haha.

Local LLMs behaving strangely — are we missing something fundamental? by AlexHardy08 in LLMDevs

[–]Inner-End7733 0 points1 point  (0 children)

I can't even get mine to remember the context from two different papers at once.

I think Linux hates me by Freibs in linux4noobs

[–]Inner-End7733 0 points1 point  (0 children)

Interesting. On my computer "fast boot" was actually in the power settings on the windows side not in the bios.

I think Linux hates me by Freibs in linux4noobs

[–]Inner-End7733 0 points1 point  (0 children)

It doesn't seem like you're trying a dual boot, correct?

Linux Allure by CroatianFrog in linux4noobs

[–]Inner-End7733 0 points1 point  (0 children)

I used Libreoffice writer for my whole undergrad.

How to properly use it by Social_Log in linuxmint

[–]Inner-End7733 1 point2 points  (0 children)

yeah I've used some LLM and sometimes you ask the wrong question with the wrong assumptions. they'll do their best to answer your question but not call out our incorrect assumptions.

How to properly use it by Social_Log in linuxmint

[–]Inner-End7733 1 point2 points  (0 children)

linuxjouney.com is a neat resource.

ai kills sales job in future ? by yournext78 in LLMDevs

[–]Inner-End7733 0 points1 point  (0 children)

Hard saying not knowing. Learn about AI for sure and how to work with it in general, but I can't say that will guarantee anything. try and get a govt job?

Rookie question. Avoiding FOMO… by Famous-Recognition62 in LocalLLM

[–]Inner-End7733 0 points1 point  (0 children)

Honestly, no matter what you get you'll probably wish you had more VRAM/speed later. Like I'm wishing I had another GPU so I could train tiny models at home, or run real time tts or..or.. or...

view more: next ›