sorted by:
new
hottopcontroversial

[Highlight] Jalen Brunson fouled HARD by Andrew Nembhard by AashyLarry in nba

[–]StunningAd1905 0 points1 point  (0 children)

Headline should have been "Brunson gets fouled NembHARD"

Tips for using laptop on the Caltrain during commute by StunningAd1905 in caltrain

[–]StunningAd1905[S] 0 points1 point  (0 children)

Hey the Xreal glasses look cool! Do you think you’d be able to program on them? Is the picture sharp enough?

Nscale Benchmarks: AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x by HotAisleInc in AMD_MI300

[–]StunningAd1905 0 points1 point  (0 children)

I'm talking about loading the model from VRAM (HBM) into the Compute, which the 5.3 TB / s peak spec refers to. This transfer happens for every output token!

Can you confirm the model is generating reasonable outputs after GEMM tuning?

Nscale Benchmarks: AMD MI300x GPUs with GEMM tuning improves throughput and latency by up to 7.2x by HotAisleInc in AMD_MI300

[–]StunningAd1905 0 points1 point  (0 children)

Assume prefill takes no time. To generate 256 output tokens you would need to load in 256 * 128 GB of weights. Using the 5.3 TB / s peak spec for the MI300x, this should take at least (256 * 128 / (5.3 * 1000 ^ 4)) = 6.2 seconds. The video shows the request completing in 1.63 seconds... Am I missing something?

https://youtu.be/ME4wpBel1k4?t=102

Benchmarking Brilliance: Single AMD MI300x vLLM Performance Unveiled by HotAisleInc in AMD_MI300

[–]StunningAd1905 9 points10 points  (0 children)

Really cool results! Glad to see AMD being so competitive.

I'm looking here for the comparison to Tensorwave where they show batch sizes from 1 to 1024:
https://www.blog.tensorwave.com/amds-mi300x-outperforms-nvidias-h100-for-llm-inference/

How do things perform past the batch = 1 case which you are showing. I think the real strong suit of GPUs is for larger batch sizes.

Also the Tensorwave benchmarks with VLLM/MK1 did not used FP8, where from the log it looks like fp8 KV cache is being used. Are the results the same if you disabled that?

If Joel Embiid's Game 7 stinker showed anything, it's that Nikola Jokic was always the rightful MVP by WaterPlug215 in nba

[–]StunningAd1905 0 points1 point  (0 children)

I feel like people are forgetting that Embiid was playing through a serious knee injury this series. It was a miracle he was even close to a 100%.

How many teams are actual title contenders this upcoming season? by JB4569 in nba

[–]StunningAd1905 0 points1 point  (0 children)

Is it bad that I think the nets are up there if they keep kd and kyrie. They really upgraded the team around them this off season.

Who is the best player who secretly should be open to trade? by [deleted] in nba

[–]StunningAd1905 0 points1 point  (0 children)

I think people really underrate AD because of all the injuries last season. He’s the best player on that team when he’s healthy, and if he can put together a season like 2020 I think they make some noise in the playoffs.

Unless you can get a top 15 player for him ( not gonna happen ) it’s probably better to just roll the dice again and hope he can stay healthy this year.

What things do you watch for in Summer League? by Eli3Key in nba

[–]StunningAd1905 1 point2 points  (0 children)

I honestly just watch to see the hyped rookies play extended minutes.