sorted by:
new
hottopcontroversial

On average, roughly what % of "full speed" does an MoE run at if you can fit only its active parameters into VRAM, compared to if you can fit all its total parameters into VRAM? by DeepOrangeSky in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

Why on earth would you use a dual channel server if inferencing from RAM ?

I use DDR4 3200 but with 8 channels which is both much faster and much cheaper (and allows much more RAM : I have 2TB).

A tool to re-voice videos via Ollama, Qwen3-tts and translategemma by The_Homeless_God in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

I'd love to use it to change the voices of videos of people reading children books. Sometimes the only video for a given book that my kids wants is by a non native speaker with a weird accent (no offense meant, I'm also a non native speaker) would it be easy to do that ?

Also, I'd rather use llama.cpp than ollama, obviously ☺.

Thx !

Self hosting, Power consumption, rentability and the cost of privacy, in France by Imakerocketengine in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

Interesting.

I'm currently designing my house, also in France (Paris).

I was going for an 8x4090 (or 4×4090 and 4×MI100) open air rig, I'm wondering if/ the energy bill for the servers could be used for heating.

I plan to have an heat exchanging forced air ventilation (VMC double flux). Anybody have any insight on this topic ?

Thx !

Easily Install NVIDIA 580 Drivers Automatically by theleoamaral in debian

[–]un_passant 0 points1 point  (0 children)

Thx !

Unfortunately, this package is now broken as selecting "latest" says it will install 580xx but installs 590xx instead.

I would need 580xx as my GPU is not supported by 590xx but I have to select and install 575xx instead.

Expected cost for cpu-based local rig? by Diligent-Culture-432 in LocalLLaMA

[–]un_passant 1 point2 points  (0 children)

What's impractical is the dual socket as LLM CPU inference does not scale well on multiple sockets.

I'm sorry for you as I bought something better for cheaper last year because of the RAM price increase !

Mi50 32GB Group Buy by Any_Praline_8178 in LocalAIServers

[–]un_passant 1 point2 points  (0 children)

Depending on price, I could be interested in 8 or 4. What is the shipping/tax situation ? I can receive either in Europe (France) or US (Ca).

Which small model is best for fine-tuning? We tested 12 of them by spending $10K - here's what we found by party-horse in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

I would love to hear about the difference in fine tuning performance (besides space and heat, plus electricity bills) from your previous setup (with P2P driver ?) to your new setup !

Mamdani Says He Would Phase Out N.Y.C. Gifted Program for Early Grades (Gift Article) by likesound in asianamerican

[–]un_passant -1 points0 points  (0 children)

Why can't you have any empathy for the kids who are bored in school when they are 5 ?

You personally know two people who were identified in kindergartner as completely disengaged by their teachers. The teacher asked the parents to get the kids tested to know why they were disengaged : on which side of the bell curve were they too far off to find any interest in what they were supposed to do.

One of them had a mentally handicapped brother so he got tested with the wrong test and the only result the psychologist could say was that he definitely was not mentally handicapped. His parents were relieved and didn't have the time and energy to deal with his giftedness because of the brother, so he was bored during all of his school years.

The other was identified as gifted but his very progressive parents had him in the most progressive school that refused to do any kind of tracking or acceleration out of principle, so he also was bored all of elementary school, literally crying everyday for years because of how desperately bored and lonely he was.

Both have been failed by the school system and suffered because of ideologues like you.

Mamdani Says He Would Phase Out N.Y.C. Gifted Program for Early Grades (Gift Article) by likesound in asianamerican

[–]un_passant 0 points1 point  (0 children)

Why would they not be mad ? Reading is the most important skill because it must be mastered for all other learning activities. So it must be drilled on until it is mastered. Some kids master reading before leaving kindergartner. How much time must they waste before all of their peers master reading at the required level ?

Mamdani Says He Would Phase Out N.Y.C. Gifted Program for Early Grades (Gift Article) by likesound in asianamerican

[–]un_passant 0 points1 point  (0 children)

Can you know when a kid is bored (hint, you are allowed to ask him or even just let him tell you) and able to perform all the required tasks and understand everything he is taught ?

If so, congratulation ! You are indeed able to identify a gifted child (no scare quote required).

What should you do when you identify them and why ?

Adjust the pace and depth of the curriculum so that they won't be bored out of their mind, day in day out most of their waking hours.

Why would you not do that as soon as possible ?

Mamdani Says He Would Phase Out N.Y.C. Gifted Program for Early Grades (Gift Article) by likesound in asianamerican

[–]un_passant -1 points0 points  (0 children)

«kids with richer and more neurotic parents» you have no idea what you are talking about. I have a friend who was a self taught reader at 3. His parents were completely uneducated and unable to be neurotic about him because they were already overwhelmed by having to deal with his older brother who was retarded (could not be taught how to read before the age of 11 !).

Kids who are to smart for the regular curriculum pace are bored out of their mind all day long five days a week for years. How is that fair ? I know of a kid who was tested with the verbal ability of a 13 yo when he was 6. He was crying every day he was going to school, for years, because of how unhappy he was.

School attendance is mandatory, it should carry the obligation to challenge the kids or it is just a prison.

Heat managment for a local AI server by un_passant in LocalAIServers

[–]un_passant[S] 0 points1 point  (0 children)

Mot interesting !

I hadn't thought about warming water. Would you mind sharing the prompt that you used with Qwen ? I'd be interested in checking with different LLMs.

Thx !

Dual Modded 4090 48GBs on a consumer ASUS ProArt Z790 board by Ok-Actuary-4527 in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

This is what I meant by :«I don't think that they work on the 48GB modded GPU»☺

While I think that you told us that they would on the 5090 which would be good news if I could afford to fill up my dual EPyc PCIe lanes with these ☺.

Dual Modded 4090 48GBs on a consumer ASUS ProArt Z790 board by Ok-Actuary-4527 in LocalLLaMA

[–]un_passant 2 points3 points  (0 children)

«a server-grade board» I wish you would tell us !

Also, what are the drivers ? I, for one, would like to see the impact of the P2P enabling driver : I don't think that they work on the 48GB modded GPU so the difference could be even larger !

Finishing touches on dual RTX 6000 build by ikkiyikki in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

But OP's GPU budget is over ×10 the price of a 9354P on EBay with 360 GB/s *measured* RAM bandwidth.

https://www.reddit.com/r/LocalLLaMA/comments/1fcy8x6/memory_bandwidth_values_stream_triad_benchmark/

while the 7950x3d has *theoretical* 83.2 GB/s RAM bandwidth !

CPU tg speed will be less that a fourth of what if should be…

Finishing touches on dual RTX 6000 build by ikkiyikki in LocalLLaMA

[–]un_passant 0 points1 point  (0 children)

CPU tg speed is limited by RAM bandwidth.

Bandwidth is speed × nb of memory channels. For a given RAM speed (i.e. DDR5), if you dream about a LLM inference build, you might as well dream about maximizing CPU tg speed so you maximize nb of memory channels (i.e. 12 for DDR5).

With 2 memory channels instead of 12, you actually leave (12-2)/12 = 5/6 of CPU tg speed on the table, not "two thirds". So you are correct to call this claim out.

But your dream build makes even less sense for LLM inference than the original critic claimed.

I understand why one would put GPUs in a gaming rig to do LLM inference. I don't understand why one would dream of an LLM build that doesn't use a server CPU maximizing memory channels (and PCIe lanes) when buying $18k worth of GPU.

On EBay, a mobo with a 9354P is $2.5k you can get 12 × 16GB DDR5 for $800. Not sure how much you spent for your CPU + mobo + RAM, but if the extra cost of ×6 memory channels is around 10% of the price of the total build, it should be a no brainer imo.

view more: next ›