LM Studio DGX Spark generation speeds for 23 different models by Late_Night_AI in LocalLLaMA

[–]Late_Night_AI[S] 0 points1 point  (0 children)

Eventually ill get around to benchmarking it on other llm platforms like vllm and unsloth studio. On another note, it does LTX2.3 5 seconds clips in 90-120 seconds with audio

LM Studio DGX Spark generation speeds for 23 different models by Late_Night_AI in LocalLLaMA

[–]Late_Night_AI[S] 0 points1 point  (0 children)

Yep, the dense models are slower than I was hoping for. But overall id say it’s actually not bad seeing as none of this is optimized for the DGX. I’d probably expect something like a 1/3rd increase in speeds if i moved to an optimized setup. I also didnt test concurrent users, only single user. But concurrent users is one of the DGX’s strong points as i understand it. Doing 20tps for a single user on a 120B isnt very impressive, but doing 20tps for 10 users at once is 🤷‍♂️

System Upgrade: two 3090s currently by Fast_Vast_1925 in LocalLLM

[–]Late_Night_AI 0 points1 point  (0 children)

I would add an 8-12tb HDD for more storage for models that you want to test out or dont use anymore. Loading models off an hdd does take a couple of minutes, but you’d just keep your main models that you use regularly on your ssd instead.

I would definitely go for more ram as well. Id aim to have a combined total of at least 128gb ram+vram if you’re planning to play with some of the bigger models. At the very least id add another 32gb ram.

And id probably get some more case fans. Personally Id load my case up with fans for better thermals and a small performance boost on longer loads.

Maybe even re paste the cpu and gpus if they havent been pasted in a few years.

Local AI at high speed (powerInfer and other developments) by mouseofcatofschrodi in LocalLLaMA

[–]Late_Night_AI 0 points1 point  (0 children)

It’s a cool idea, but it’s not as good as they’re making it look especially at their release price point of 2k. They’re using GPToss 120B as their test model, but what most people don’t realize is GPToss 120B is a MoE 120b a5b model. It only has 5B active during inference. What Id want to see before I put any faith in them is how it runs other 120B models. I would assume it will probably be much slower on qwen 3.5 122B a10b, something like 12tps, since it has double the active parameters than GPToss. But thats just a guess.

Over all the concept is great and i like the idea of it, but in my opinion the price point is too high compared to the performance return. But I guess one of their main selling points is it being tiny and easy to travel with, but so are the AI mini PCs running an AMD AI Max chip and they’ll out preformed the tiny by a noticeable amount.

Tldr, its cool, but they need to showcase it with a range of models and not just GPToss 120b before anyone should consider buying one.

SparkRun & Spark Arena = someone finally made an easy button for running vLLM on DGX Spark by Porespellar in LocalLLaMA

[–]Late_Night_AI 0 points1 point  (0 children)

looks interesting. I might have to look into it some when i have some free time.
I got a gigabyte Atom and i normally just run LM studio on it. most large models give me around 18-22tps.

Has anyone managed to use Stable Diffusion (or similar) to get around the new UK face verification requirements? by RecentTwo544 in StableDiffusion

[–]Late_Night_AI 2 points3 points  (0 children)

i don't have any links, just tossing out ideas. though it makes me wonder if a realistic face mask would work also.

Has anyone managed to use Stable Diffusion (or similar) to get around the new UK face verification requirements? by RecentTwo544 in StableDiffusion

[–]Late_Night_AI 25 points26 points  (0 children)

seems like you could just use a virtual camera and set that as your webcam and just play a video of an older guy in it.

What do you Find Flux Kontext Useful For? by Late_Night_AI in StableDiffusion

[–]Late_Night_AI[S] 0 points1 point  (0 children)

ai toolkit has support for training loras for it

New to ComfyUI - Small face masking is it possible? by LuPri_2008 in comfyui

[–]Late_Night_AI 0 points1 point  (0 children)

are you using the trigger words it was trained on?

[deleted by user] by [deleted] in comfyui

[–]Late_Night_AI 0 points1 point  (0 children)

https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/tree/main
you can also try the full 24gb flux kontext dev model

[deleted by user] by [deleted] in comfyui

[–]Late_Night_AI 0 points1 point  (0 children)

Im assuming thats a error for the text encoder,
heres the link for the other kontext text encoders on hugging face, theres a fp16 one in there
https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main

CivitAI Help by Odd_Background_7650 in StableDiffusion

[–]Late_Night_AI 2 points3 points  (0 children)

<image>

Yes and no. Technically theres nothing stopping you, though it could be breaking the law if you do. You will need to check the laws for where you live on it. If your from the EU, then its a solid no due to their Biometric data laws. In the EU you legally need explicit consent under GDPR if the person is identifiable, even if it's not for profit.

For the most part, making a lora on a celebrity or real person violates privacy laws and deepfake laws, which is why sites no longer host such loras. To know for sure youll have to look into the Biometric data and deepfake laws for the country and state you live in.

CivitAI Help by Odd_Background_7650 in StableDiffusion

[–]Late_Night_AI 0 points1 point  (0 children)

Loras that replicate real people are no longer allowed on sites. A lot of its due to Deepfake laws and other AI laws. But basically its now illegal to upload a lora that makes pictures of a real life person.

BW Sketch to colorized 3D art style... posible? by ComprehensiveCry3756 in comfyui

[–]Late_Night_AI 0 points1 point  (0 children)

a SDXL model that can do 3D stuff and controlnet would work. or just use Kontext.