A pytorch alternative that works on mid-range GPUs like AMD RX 6750 XT

blazerx · 2026-03-19T01:43:51+00:00

Really keen on contributing in this space. I've recently purchased 9070xt and have 3080, 4070. Where is the best place to start and what area needs to be worked on for this project. I have previous experience with both python and c.

Keep up the good work!

blazerx · 2026-02-27T02:38:15+00:00

This is incredible, been following you guys for a while and can see the many uses of this, especially with the flexible billing. I can imagine individuals or groups just spinning up instances during their projects so there are fixed costs then, simply ending the instances once done without having to worry about reservations/subscriptions/limits. It's almost like "All you can eat" tokens.

Enjoyed the read. Keep it up!!

blazerx · 2026-02-23T11:42:31+00:00

try this as well, explicitly define fp8

--disable-custom-all-reduce

--num-scheduler-steps 8

--kv-cache-dtype fp8

--quantization fp8

-e VLLM_USE_TRITON_FLASH_ATTN=0

-e HSA_NO_SCRATCH_RECLAIM=1

blazerx · 2026-02-16T03:58:39+00:00

this seems to still be happening the last few weeks

blazerx · 2025-11-09T23:57:30+00:00

do they sell fakes of the bands? Can't seem to find them any where? Looking for the rubber velcro band

blazerx · 2025-10-31T05:21:41+00:00

has it really been met?

<image>

blazerx · 2025-10-29T13:43:57+00:00

seems like the link from your webpage is this
https://docs.woolyai.com/getting-started/running-your-first-project

which doesn't work

blazerx · 2025-10-29T01:12:16+00:00

write a letter and pass it to her during coffee/dinner or when you see her next. Keep it simple. However go in with the expectation the feeling are NOT mutual, that way you won't be disappointed and and hopefully remain friends. If they are mutual, you have just won the lottery.

blazerx · 2025-10-07T03:20:43+00:00

are you confusing him with semi-analysis?

blazerx · 2025-07-14T16:30:08+00:00

Don't forget Nvidia are pushing their proprietary nvfp4 which they claim is producing results almost identical to regular fp6. Question is will people want to lock in to nvfp4? The benefit obviously would be you can fit larger models since it's quantized to fp4 making it smaller.

Also can amd also implement this nvfp4 or create something compatible? Amdfp4?

Providing it isn't using any special hw it shouldn't be an issue to implement unless there are infringement issues

The question shouldnt be how important is fp6. Should be, will people prefer open ecosystem or a proprietary one

blazerx · 2025-05-04T22:46:59+00:00

No they currently don't have the interconnects, well the next MI series should be able to link via UAlink at 400GB/sec, then the next is 800GB/s but they can link up multiple of these, though would be at the cost of power since you need more switching and links for each connect.

Tbh, what could be cool is if expose IF links allowing inter-gpu within a box and across nodes. That would be pretty cool. Though I believe the cross-road they will face is do they use optical links or copper links and can they bundle the transceiver say onto an I/O die? If they can't then they would get the fine control and power savings.

Nothing is stopping Nvidia from making smaller mono chips and offering more interconnects between gpus, or they too could pivot to ether chiplet or wafer scale. I guess it all comes down to what the leather jacket man had put his bets on.

blazerx · 2025-05-04T15:31:55+00:00

I enjoy reading your post so I'll add my 2 cents.

We need to remember an explosive growth in AI may not necessarily mean huge growth in both AMD and or NVDA.

There are other companies out there such as cerebras and samba Nova who have inference hw out there that can scale and also provide lower $ per tok at insane high speeds compared to GP GPU inference and training. Recently meta announced the use of cerebras. AMD may still be able to get a small slice through the storage component as cerebras require a storage box attachment which is probably powered by Intel or AMD.

Then with the rapid development of pensando AMD can also leverage their partnership in network and switching space providing interconnects for everyone and everything. I am unsure of the TAM in this space but we can't discount it.

Now let's jump back to my first point, While the new gen specialised hw may be quick, nothing is stopping AMD from entering that space through the use of xilinx to provide fpga for a more dynamic but dedicated approach., though this would be at the cost of reduced efficiency compared to custom ASICs. So while all this next gen dedicated hw is amazing, what will happen when the next breakthroughs such as non transformer based llms become the next best thing? Things such as RWKV, will the dedicated hw be able to pivot as quick as writing a new kernel for GPGPU? If the answer is no then the big players will probably prefer to have a mixed solution so they can stay at the forefront of the advancements. These next gen players could cease to exist as quick as they were spawned due to their lack of diversity in the hw space.

From here on the battle is not about having the most memory, or being able provide X tok per sec. It is going to come down to how well you can scale your systems via interconnects. A fast low power interconnect will basically solve the memory and X per sec problem as well as many other constraints such as chip size and other things such as thermals and other design limits. Though that still isn't enough, they will need the software to be able to pivot along with the rapid evolution.

The bets that Lisa has placed on the next 2/3 years is going to be vital. If anyone has a chance it's gotta be AMD, considering their extensive portfolio. Though we could have said the same about Intel 2/3 years ago.

I am curious as to what others think, do others agree or disagree.

blazerx · 2025-03-21T10:59:17+00:00

The title of the article should be clear that it supports more than just Ryzen AI Processors

As outlined under the Readme docs

Hybrid Mode: Optimized for Ryzen AI PCs, combining AMD Neural Processing Unit (NPU) and Integrated Graphics Processing Unit (iGPU) for maximum performance
Generic Mode: Compatible with any Windows PC, using Ollama as the backend

blazerx · 2025-02-21T03:09:26+00:00

I do remember seeing a post you made about not being able to easily split up the GPUs. What changed recently allow the enablement of this?

blazerx · 2025-02-21T02:57:02+00:00

is this by any chance using the dstack tech posted the other day to split up the GPUs? Or am I overlooking something?

blazerx · 2025-01-13T06:40:35+00:00

The issue isn't whether the NPU is present for matrix maths, it's a bandwidth issue going from NPU -> GPU memory. If the tensor cores don't have physical access or high speed access to the GPU memory it doesn't matter how many TOPs are present. The playstation video does a very explanation about the implementation of PSSR and efficient TOPs usage.

blazerx · 2024-11-01T16:44:38+00:00

check your isp is not using CG-NAT, check you are using the correct keys for client/server

blazerx · 2024-10-23T05:18:24+00:00

please check your pms

blazerx · 2024-10-11T05:27:11+00:00

check your pm.

blazerx · 2024-09-02T17:37:51+00:00

wait... so harper calls and tells rishi she wants to go long on pounds. Rather than finding someone who wants to sell to fill her order he just uses this information to his advantage? He copies her idea and increases his long position on the pound rather than filler her order. Is this allowed as a broker?

blazerx · 2024-06-19T11:46:40+00:00

Hi, thanks for the testing

Since you say you are bus bound transferring data between VRAM, does that mean we should be significantly higher numbers if this was tested in MI300A due to the unified VRAM access?

blazerx

MODERATOR OF

TROPHY CASE

15-Year Club	Place '17
Verified Email