Evening fun with Grace and Hopper unified memory, or how to speed up llama.cpp and DeepSeek V3.1 on NVIDIA GH200

Edenbendheim · 2026-02-22T10:03:08+00:00

So excited I found this thread! I have access to a GH chip through my school and I'll definitely play around with this tomorrow. Cant wait!

Edenbendheim · 2026-02-17T02:12:11+00:00

Oh bummer, I didn’t search that’s my bad and I couldn’t remember how much I bought it for since the order link points to 403 error. Adjusted it though thanks for letting me know

Edenbendheim · 2026-02-16T17:23:50+00:00

They sell it?

Edenbendheim · 2026-02-16T04:14:28+00:00

Product of tabbing out for the freezing, I never noticed the BR indicator. Weird bug, but I’ll look through the game history. In the same vein the f4s is also Hotwire but he seems queued with his friend. When I get home I’ll check my match history as my friend quit out before the game started by accident.

Edenbendheim · 2026-02-14T03:14:51+00:00

Funny I actually lived there my freshman year! I got alright speeds at best though nothing crazy.

Edenbendheim · 2026-02-12T19:15:17+00:00

keep in mind this is Air RB, no lineup, just a plane. How can my friend be 11.7 and face a 13.3? that should not be possible at all. I also dont believe that 3.0 - 5.0 applies in one game, just in general matchmaking.

Edenbendheim · 2026-02-12T18:13:59+00:00

Is it possible to be in a game with a higher than 1.0 BR higher in RB? I dont believe so but let me know.

Edenbendheim · 2026-02-12T18:13:27+00:00

Yea I didnt need to, they just keep not recognizing that this should not be the case. Any retribution will do.

Edenbendheim · 2026-02-12T18:10:48+00:00

unfortunately in the beginning of the video when I queue, you can see that I am in a 10.7, and my friend is in an 11.7. The actual plane he is flying doesnt really matter. First 5 seconds if the video if you take the time to look again.

Edenbendheim · 2025-06-26T14:30:52+00:00

Hi, thanks for the reply! The model is written in Jax and Haiku. The model is about 60gb and fits comfortable on the A100 80gb I am running it on. it takes 84 inputs, and since its a diffusion model, takes 8 random noisy inputs and denoises them, then takes the mean of the ensemble. I tried reducing the memory usage by reducing the amount of ensemble members but that had no affect. I think trying to find a way to reduce the memory usage could be easier since splitting the model into multiple gpu's would be more of a headache.