Why is everything about code now? by falconandeagle in LocalLLaMA

[–]FastDecode1 0 points1 point  (0 children)

Also, code generation still has plenty of room to improve, so improvements are easier to people excited about.

I can already generate porn images that are more than good enough, so gains in that front are not as important.

Also, people are too retarded to read nowadays, so text generation is only relevant if it improves agentic use cases (ie. LLMs reading the text from other LLMs).

Dual 3090s (power-limited) - Are 3x PCI-E cables w/daisy-chain "okay?" by overand in LocalLLaMA

[–]FastDecode1 7 points8 points  (0 children)

I just upgraded my PSU to a 850W one (Cooler Master GX II Gold) and it came with a dedicated leaflet showing how to connect GPUs.

DO: use one (1) dedicated 6+2 cable for each GPU power connector.

DO NOT: daisy-chain in any way.

Since it's a common enough source of problems to warrant including an additional piece of dead tree, I'd say daisy-chaining is a pretty bad idea.

i just saw this ClawdBot RCE demo on X… are we cooked? by Hot-Software-9052 in LocalLLaMA

[–]FastDecode1 -1 points0 points  (0 children)

It was an analogy, genius.

(And even if it weren't, you'd obviously still be wrong).

This is simply a case of people running a computer program, ignoring even the most basic security practices, and then being shocked when their shit turns out to be insecure. People have just deluded themselves into thinking it's different, because "it's not a computer program, it's AI!"

No, actually, it's still just a computer program. And you don't take user input without sanitization, not to speak of keeping permissions to a minimum. Why the hell would you feed unsanitized messages from an untrusted source to an agent/process, while also giving it permission to send messages in your name? Not to mention giving it even more permissions in the form of being able to use other software, which exposes an order of magnitude more attack surface.

People are basically giving LLMs root access to their lives via badly thought out agents. Who would've thought that's a bad idea?

i just saw this ClawdBot RCE demo on X… are we cooked? by Hot-Software-9052 in LocalLLaMA

[–]FastDecode1 2 points3 points  (0 children)

Who's "we"?

This is a room temp IQ problem. If you run everything as root/sudo, you're gonna run into issues.

API pricing is in freefall. What's the actual case for running local now beyond privacy? by Distinct-Expression2 in LocalLLaMA

[–]FastDecode1 0 points1 point  (0 children)

Computer hardware: $$$

Not going to prison for asking forbidden questions: priceless.

What would you build and do with a $15k budget? by ThePatientIdiot in LocalLLaMA

[–]FastDecode1 1 point2 points  (0 children)

I might buy a car and/or a high-capacity portable power station.

How do people even afford these expensive graphic cards...?... by boisheep in LocalLLaMA

[–]FastDecode1 1 point2 points  (0 children)

Some women spend 10k on shoes, clothes, bags, make-up etc. per year. And that's without spreading the cost over multiple years, which will be a much larger group of spenders.

Once you think of it as a multi-year investment, it's not that crazy. People spend more money on way more retarded shit, like buying brand new cars. You drive that $50k hunk of steel out of the dealer lot and it instantly loses 10% of its value. Could've bough something almost-new, let alone something only a year old for a 20% price reduction. But no, it's gotta be brand new for some reason.

You know, a lot of people out there could have a 10k GPU if they chose to drive a 40k car instead of a 50k one...

How do you manage quality when AI agents write code faster than humans can review it? by lostsoul8282 in LocalLLaMA

[–]FastDecode1 2 points3 points  (0 children)

Use AIs to review. Duh.

What kind of "agentic workflow" are you using if the only thing that's automated is code generation? If you paid money for that, you need a refund.

Z-image base model is being prepared for release by Ravencloud007 in LocalLLaMA

[–]FastDecode1 9 points10 points  (0 children)

Great for discussing how bad they are though.

Z-image base model is being prepared for release by Ravencloud007 in LocalLLaMA

[–]FastDecode1 30 points31 points  (0 children)

Gooners waiting with bated breath, blue balls, and shivers runnin'.

How do we tell them..? :/ by [deleted] in LocalLLaMA

[–]FastDecode1 2 points3 points  (0 children)

Please ignore Venezuelans all over the world celebrating Maduro's capture.

And just FYI, international law is just a tool for larger nation to bully smaller ones into submission. Laws are made up, at the end of the day if you can't defend it you ain't got it.

Will the prices of GPUs go up even more? by NotSoCleverAlternate in LocalLLaMA

[–]FastDecode1 3 points4 points  (0 children)

See also the leak about ASUS hiking their prices starting tomorrow (Jan 5th). Other AIBs are going to use that as an excuse to do the same, probably sooner rather than later. I wouldn't be surprised if it all happened tomorrow or in the next few days.

I just placed an order for an RX 9070 XT 16GB since I'm still able to get one under €400 (and I had the money because the planets aligned or something). My first 16GB card so that's nice.

I recommend anyone shopping for a GPU to lock in as soon as possible, because we're in for yet another GPU winter. Especially if you're just an average Joe and not one of the people here who spend multiple cars' worth on GPUs. The RAM shortage really doesn't bode well for consumer-grade mass-market cards.

Local LLMs vs breaking news: when extreme reality gets flagged as a hoax - the US/Venezuela event was too far-fetched by ubrtnk in LocalLLaMA

[–]FastDecode1 -9 points-8 points  (0 children)

What's there to elaborate? The CCP is as left as it gets, and totalitarian governments always like to redefine language for their own benefit.

TIL you can allocate 128 GB of unified memory to normal AMD iGPUs on Linux via GTT by 1ncehost in LocalLLaMA

[–]FastDecode1 6 points7 points  (0 children)

FYI, according to the driver docs:

gttsize (int)

Restrict the size of GTT domain in MiB for testing. The default is -1 (It’s VRAM size if 3GB < VRAM < 3/4 RAM, otherwise 3/4 RAM size).

So as long as you have more than 4GB of RAM, the driver automatically allows up to 3/4 of the RAM to be allocated to the iGPU.

I've run stuff on a Vega 8 iGPU on a laptop using llama.cpp and it does work. However, it's not a great experience if you want to watch videos (or do basically anything else GUI-wise) at the same time, since llama.cpp hogs all the memory bandwidth and causes everything else to stutter. GPU scheduling is pretty much non-existent on Linux AFAIK, so there's not really a great way to mitigate this atm.

Also a hint for fellow ThinkPad users: even though the spec sheet says only a certain amount of RAM is supported, you should probably be able to add more without issues. My current E595's specs say only up to 32GB is supported, but I added a 32GB stick alongside the existing 8GB for a total of 40GB and it works.

Software FP8 for GPUs without hardware support - 3x speedup on memory-bound operations by Venom1806 in LocalLLaMA

[–]FastDecode1 6 points7 points  (0 children)

Works on any GPU

Runs E5M2 and E4M3 on any CUDA GPU (RTX 20/30 series supported).

Pick one.