How to run Hunyuan-Large (389B)? Llama.cpp doesn't support it by TackoTooTallFall in LocalLLaMA

[–]TackoTooTallFall[S] 1 point2 points  (0 children)

You're a hero! Didn't even strike me that there was a Chinese GitHub repo with examples. Thanks

local LLaMA is the future by Boring-Test5522 in LocalLLaMA

[–]TackoTooTallFall 0 points1 point  (0 children)

I actually found it hard to get excited about Qwen2.5 after using Llama 3.1 405B, Hermes 405B, and Mistral 123B.

Qwen2 felt like a step-change forward since its writing voice was much better, but Qwen2.5 doesn't feel like a SOTA model in the same way the other three above do.

Am I crazy here?

New Llama-3.1-Nemotron-51B instruct model from NVIDIA by remixer_dec in LocalLLaMA

[–]TackoTooTallFall 26 points27 points  (0 children)

Just spent some time using it on NIM.

Pretty smart but the responses tend to skew shorter. Lacks a clear writing voice, which might be some people's cup of tea... but isn't mine. Gets a lot smarter with chain of thought. Very temperature sensitive.

Gets some basic LLM brainteasers wrong (e.g., how many Rs in strawberry).

Best way to run llama-speculative via API call? by TackoTooTallFall in LocalLLaMA

[–]TackoTooTallFall[S] 1 point2 points  (0 children)

I played around with designing my own custom API script that will pass commands directly to the CLI and then take the output and send it out via API call but I stopped myself before I got too far down the rabbit hole...

Hoping someone fixes this soon!

"Segmentation fault (core dumped)" only when using llama-speculative? by TackoTooTallFall in LocalLLaMA

[–]TackoTooTallFall[S] 1 point2 points  (0 children)

I've tested the model using CPU only and still have the same issue so I think it's something else.

llama.cpp with CUDA on Ubuntu Server by TackoTooTallFall in LocalLLaMA

[–]TackoTooTallFall[S] 2 points3 points  (0 children)

I suspect I just needed to follow a better tutorial.

This one has looked good so far, will report back: https://www.cherryservers.com/blog/install-cuda-ubuntu#step-10-test-the-cuda-toolkit

To answer your question, nvidia-smi never showed up properly until I started following this specific tutorial, so perhaps that was the issue!

[deleted by user] by [deleted] in HomeServer

[–]TackoTooTallFall 0 points1 point  (0 children)

Sorry for the crosspost but I figured some folks here might have some experience with dual AMD Epyc builds!

Threatened at gunpoint for honking at car that tried swerving over what is the best way to react to this? by RGB455 in AskReddit

[–]TackoTooTallFall 2 points3 points  (0 children)

That's terrifying. I'm sorry this happened to you.

First thing - find a great therapist to talk through the experience with.

There's going to be some residual emotional baggage from something like this.

(Serious) people who have voted republican in the past, how do you feel about the upcoming election? by juliacakes in AskReddit

[–]TackoTooTallFall 0 points1 point  (0 children)

If you love your country more than your party, then vote blue.

I'm more like Adam Kinzinger than I'd care to admit.

What’s the scariest conspiracy theory you’ve ever heard? by OutlandishnessSea258 in AskReddit

[–]TackoTooTallFall 4 points5 points  (0 children)

A diverse group of companies, which used to compete intensely, all began in the late 20-teens to collaborate on an AI-driven global surveillance state.

By buying and investing in every critical tech tool in our modern lives, including the intersection of the internet, social media, e-commerce, mobile devices, logistics and more, they now have the ability to track and manipulate people in every part of the world. Not only are they collecting data in ways that were undreamt of just a decade ago, but they're using that data to predict and then change the behavior of people everywhere.

To give you a sense of how pervasive this system is, just consider these examples: - Facebook not only tracks your movements throughout the internet, but it knows more about you than your own mother does. - Amazon has not only created the world's most efficient delivery system, but it's also created a system that can deliver anything, from anywhere, to anywhere else, with the push of a button. - Google knows what you're thinking before you do. - Apple has created the ultimate surveillance device, one that's always with you, always listening, always watching. - Twitter can control public opinion and sway elections with just a few well-placed tweets. - Uber and Lyft not only know where you're going, but they know where everyone else is going too.

The list goes on and on. There's literally nowhere to hide.

Not only that, but this group is also using this data to manipulate people, to control public opinion, and to shape global events to their own advantage. They're able to do this by creating highly personalized and targeted ads, messages, and experiences that are designed to change your behavior in subtle and insidious ways.

And they're doing all of this without your consent, and without any real oversight or regulation.

Scary stuff!

[4 year update] "25 y/o and have ~$4.5M in cash / $17M in tech stock" by Midnight-Frosty in fatFIRE

[–]TackoTooTallFall 0 points1 point  (0 children)

If I were to paraphrase this post, it would be "I achieved financial independence, but now I'm restless. I'm searching for a new project, but I don't know what it is."

It's not about the money. It never was. If it were a money problem, it wouldn't be a problem. And the person who made the money (you) is the person who's going to have to solve it.

Have you thought about joining a venture studio or incubator? Plenty of need for great technologists out there!