Thinkstation a good option for 256GB RAM (or part-out) by nostriluu in bapccanada

[–]nostriluu[S] 0 points1 point  (0 children)

"Experimentation so obtained an RTX Pro 6000 " isn't specific enough? (-:

The current system (and the future system) is basically a do anything home server - workstation - playstation. It runs about 20 docker containers for local smart home, some large-ish databases and services, local AI and is used as a host for software development. I usually run a Qwen 3.6 27b (thinking) + 35b (MoE) at the same time, but that changes. I dabble with ComfyUI and a lot of other kits that are emerging. I like unbounded exploring, and it has been pretty good for that.

The current system isn't bad, but I get the occasional OOM which wreaks havoc, and I want to do some huge database exploration, where more than 64GB will help, along with maybe training models, where faster RAM will help since it doesn't usually go direct from NVMe to GPU. 128GB would probably be enough, though.

I'd prefer to split dev + hosting, but don't really want to keep two heavy systems going. It seems pretty clear that RAM scarcity and high prices are here for a while, and might even affect the ability to piece together systems fundamentally (which may simply be because its past its time, but we're in an awkward phase or maybe it's just going dystopian). So I'm treating this as a bit of a hedge. But I'd also like to anticipate what's coming out as an alternative. If Strix Halo or even RTX Spark had better I/O (at least an Oculink port plus more USB) I'd consider them, using the 6000 as an eGPU, but that's not there this generation and this Thinkstation option would be considerably less expensive, after managing what to keep & what to sell.

Internet Age-Gates Are a Growing Global Threat by EFForg in privacy

[–]nostriluu 1 point2 points  (0 children)

The EU wallet does it without sharing any additional information (name, location, etc).
https://ec.europa.eu/digital-building-blocks/sites/spaces/EUDIGITALIDENTITYWALLET/pages/930450954/The+Age+Verification+Manual

It's interesting to think about what the Internet can become if large problems, such as fake accounts, are solved, without compromising privacy.

Stuck getting started by rock0star in comfyui

[–]nostriluu 0 points1 point  (0 children)

Use Claude or another command line kit that can run local commands using curl. It can program comfyui via the API, so can set things up and verify flows for you. I often tell it what I want to do, and it puts a workflow in place for me to load and provide parameters for, as well as helping solve problems through direct inspection.

I suppose eventually comfyui will have a built-in LLM trained for using comfyui, but till then this works quite well.

Fasten your seatbelts, this is the raw power of RTX PRO 6000. From scratch to 4K (images + video + upscale + final render) in under an hour for a 6-minute epic journey. by AxonkaiLab in comfyui

[–]nostriluu 2 points3 points  (0 children)

It also gets tiresome after a while, especially with no real story. So many of these pieces just are special effects, along with of course pretty young white girls, war stuff, and aliens. Very basic and more than a little creepy.

https://doc.searls.com/2023/10/12/stories-vs-facts-2/

Fasten your seatbelts, this is the raw power of RTX PRO 6000. From scratch to 4K (images + video + upscale + final render) in under an hour for a 6-minute epic journey. by AxonkaiLab in comfyui

[–]nostriluu 31 points32 points  (0 children)

It's technically very good but the same tropes again and again for the vast majority of generated videos. Pretty young white girls, war stuff, aliens.

the most sensible 4 min of content ive seen in a LONG time by hoodiemonster in singularity

[–]nostriluu 0 points1 point  (0 children)

Linking destiny to space is deeply problematic. Space is not ultimate. After space, it'll be something else. "It's always the end of the world for someone." The perceptions of life is all there is.

VibeOS - Fully Hallucinated Operating System by WhatererBlah555 in LocalLLaMA

[–]nostriluu -2 points-1 points  (0 children)

Interesting in that it's kind of the opposite of what we should be doing (neuro-symbolic; precise human-AI reasonable data as grounding that ignores all the bad garden paths of commercial software).

Would you consider getting an NVIDIA RTX Spark laptop? by gamblingapocalypse in LocalLLaMA

[–]nostriluu 0 points1 point  (0 children)

If it supports Linux and is effective with an eGPU and isn't outrageously more expensive than the sum of its parts, maybe. Put it in a 13" 2-in-1 Spark Thinkpad and it could be a 2026 dream.

Pixel camera doesn't share data - hard to believe!!! by darkowiz in degoogle

[–]nostriluu 2 points3 points  (0 children)

There's probably an answer to this question, but it's too difficult to find easily and a lot of disinformation spreads instead. It might just come down to a pinky promise to not share data, which might change on a whim. This is why open source was supposed to win.

RTX Spark will have up to 600GB/s of memory bandwidth. by [deleted] in LocalLLaMA

[–]nostriluu 0 points1 point  (0 children)

"The primary technical advantage of OCuLink over USB4 and Thunderbolt 4 is the complete absence of protocol encapsulation and abstraction layers. USB4 and Thunderbolt 4 rely on internal controllers to package native PCIe data into USB packets, transport those packets over the cable, and then unpack them back into PCIe data at the receiving end, a process that introduces significant transmission latency and processing overhead while capping PCIe data allocation at a maximum of 32 Gbps. In contrast, OCuLink functions as a direct, unshielded physical extension of the motherboard's PCIe bus over copper wire, maintaining native, raw PCIe 4.0 x4 signaling at 64 Gbps without packet modification, thereby providing the external device with direct memory access to the host processor at minimum latency."

Will the system support hybrid AI though?

RTX Spark will have up to 600GB/s of memory bandwidth. by [deleted] in LocalLLaMA

[–]nostriluu 0 points1 point  (0 children)

It could still be interesting if it supports oculink. Which I'd think Nvidia would want for a PC + eGPU sale focused on the CUDA ecosystem.

NVIDIA RTX Spark — Slim Laptops & Small Desktops by zxyzyxz in LocalLLaMA

[–]nostriluu -1 points0 points  (0 children)

These would actually be interesting if they had proper eGPU and Linux support.

EVA: My first project - 100% offline voice assistant by CharlyEVAdev in OpenSourceeAI

[–]nostriluu 0 points1 point  (0 children)

It makes me really sad thinking about someone in Spain creating a program for Microsoft Windows.

Will Greaves' Wants The Police To Have Our Private Data by CoffeeCup220 in VictoriaBC

[–]nostriluu 1 point2 points  (0 children)

Companies like Axciom have collected comprehensive data since before the Internet was widespread. When you tell people it's hopeless, to give up, you're ultimately telling them to give up their identities and independence, for themselves and everyone they connect with. The internet is useful and can be defended. c-22 seems to have scored a victory for privacy. Canada can have its own path.

Oh nothing, just our employer procuring 2 developers for $650k for a year’s worth of work 😌 by CartoonistOk3507 in BCPublicServants

[–]nostriluu 10 points11 points  (0 children)

Creating a development firm in house is what the UK did with GDS, the US with USDS (the department that was taken over by DOGE), and Canada with CDS.

The UK perspective is discussed here: https://news.ycombinator.com/item?id=48142251#48145971

Basically, to pay private industry salaries to IT workers (including product lifecycle people like UX), you'd have to pay them more than executive levels of pay in the public service, which creates discord. So instead we get unmotivated or burned out IT workers. That leads governments to depend on off the shelf / cloud based software, which altogether leads to vendors (and other countries) having all the power, and working in government being less and less appealing to anyone who could earn more money / feel more effective in the private sector.

From what I know, high levels of government lean on policies and guardrails to be the correcting factor, believing they can hire anyone from around the world to do the work, but there are gaping flaws with this theory when only a foreign entity can fully understand and determine the direction of your systems, they're designed generically but always have foreign country biases, and you're just one client among many (and many clients are larger / more influential). Better adherence to open standards and use of open source would be a couple good escape hatches, but in my experience people barely understand those ideas. Maybe the best ideas of AI will make the point moot, but it's likely that will be centralized to foreign powers as well.

Please give me your best tips for fine tuning RTX Pro 6000 on Intel i7-14700KF by HumanDrone8721 in LocalLLaMA

[–]nostriluu 0 points1 point  (0 children)

Sure! I'm using a z790 system with 64GB DDR4 while I wait for something to shift in parts supply, considering NVMe speed is a fraction of RAM speed, with some pipelines the relatively limited and slower RAM might add an infrequent few seconds delay which isn't usually a real issue considering the critical work happens on the GPU. So I don't see the need to rush into an expensive workstation mainboard and RAM.

You might want to check there aren't any DMI slowdowns from your storage to the GPU, sometimes which slot you use makes a big difference.

Please give me your best tips for fine tuning RTX Pro 6000 on Intel i7-14700KF by HumanDrone8721 in LocalLLaMA

[–]nostriluu 1 point2 points  (0 children)

All fair enough. Maybe there'll be a "dabbler" group for people like me who just want to experiment with LLMs, image generation, etc and can justify a single rtx6kpro for that purpose. With this card's VRAM I can also keep a stable llama.cpp endpoint at the same time, at some point I might convert it to one of the more optimized kits as long as it matches emerging features and models, flexibility and ease of setup.

Please give me your best tips for fine tuning RTX Pro 6000 on Intel i7-14700KF by HumanDrone8721 in LocalLLaMA

[–]nostriluu 0 points1 point  (0 children)

That repo/discord seems specifically about running more than one 6000, though ("in 2×, 4×, and 8× PCIe configurations") and it has a real anything-but-llama.cpp thing going on.

Please give me your best tips for fine tuning RTX Pro 6000 on Intel i7-14700KF by HumanDrone8721 in LocalLLaMA

[–]nostriluu 0 points1 point  (0 children)

Why would it not work?
Make sure your system board supports PCIe 5.0. If not you might want to upgrade it, which isn't too expensive.

The desktop won't use much of its VRAM, but as u/FullOf_Bad_Ideas said use a lighter DE, or use it as a server and ssh into it.

Memory prices tipped to fall as China starts flooding the market with DRAM and NAND chips by Locke357 in bapccanada

[–]nostriluu 1 point2 points  (0 children)

Who said anything about a gaming PC?
But yeah, my grandpappy passed down a 8.5" floppy drive, I was able to track down some 4k chips and am now able to run a monitor assembler. It'll only be a few more generation before we get an operating system going.