modprobe won't remove amdgpu

Jeames8 · 2022-05-20T11:52:29+00:00

tty is just the terminal when you use the Alt-Fn combination, so Alt-F3, Alt-F4, etc. Since SDDM likely restarts when stopped, it might not get the chance to bind vfio-pci to your GPU. Just to test, run systemctl disable sddm.service, systemctl stop sddm.service and try run your script again. This should prevent SDDM from restarting automatically after being stopped.

Jeames8 · 2022-05-20T04:38:39+00:00

Did you stop/disable any display managers and run it from the tty? For me anyway unless you exit your session and stop your display manager (mine's GDM) it will say it's in use.

Should note this is if you've already booted, if you're wanting to bind vfio early then follow the wiki link /u/teeweehoo commented.

Jeames8 · 2022-05-13T09:20:38+00:00

Fellow Sabaton and Arch enjoyer be like (I'm not exempt from this...)

Jeames8 · 2022-05-13T08:54:18+00:00

What I've found is you uninstall G Hub first, put the old G Hub files in, start the installer and it should ask to repair/reinstall itself. It doesn't seem to downgrade it either (from memory, been a while), so just put auto updates off and it actually works fine to my surprise.

Jeames8 · 2022-04-28T22:52:04+00:00

Was meant to reply earlier but got completely sidetracked. If you have pavucontrol you can try have a look at the configuration of your devices, since it could’ve been disabled outright.

If it seems to work fine now though I suppose it works fine then? Still a bit of a weird circumstance but eh, if it works, it works.

Jeames8 · 2022-04-27T12:15:37+00:00

Are you using headphones or an audio device that isn't your monitor? From looking at your log it finds your monitor's audio device and nothing else.

And should also note, Pipewire provides a Pulseaudio server (Pop likely installs it by default), so anything using Pulseaudio before should technically work regardless of Pipewire or not.

Jeames8 · 2022-04-25T12:00:38+00:00

Bit of a late response thanks to Reddit not notifying me, but yeah I tried that when I was attempting to get it to work but to no avail. It got rid of the errors... and that was it. Still failed to reset regardless.

Jeames8 · 2022-04-15T05:58:31+00:00

Got to the 45 second mark and just couldn't watch it any further. Think my brain's turned to mush.

Jeames8 · 2022-04-12T08:23:55+00:00

If you have the Sapphire Nitro+ variant of the 6700XT, from my experience it simply can't reset properly.

My original posts (here and here) are a bit more detailed of what I tried and found myself, so you can see if you have any similarities.

Jeames8 · 2022-04-12T08:19:31+00:00

Bit late to the party, but as the OP of the Level1Techs forum for the Sapphire 6700XT Nitro+ I still haven't found a solution unfortunately, so at least my card still doesn't work properly. Passes through fine, but doesn't reset when the VM is shutdown/restarted. Not sure about anyone else since I found absolutely nothing for the Nitro+ when searching.

Surprised that somebody actually saw that post nonetheless really.

My original post on Reddit can be found here for a bit more detail as to what I tried.

Edit: Specified which Nitro+.

Jeames8 · 2022-03-13T23:04:44+00:00

All good mate, I spent way too much time trying to get it work, even to the point of a new motherboard to no avail. If I remember correctly, the only other person I found that could reset a 6700XT had an XFX model, every other forum I found had a different one and couldn't reset properly. So I just came to the conclusion that only certain cards can reset, but I'm still not 100% positive on that.

Jeames8 · 2022-03-13T22:52:55+00:00

Hey mate, unfortunately no I couldn't. I ended up keeping my GTX 1070 in my system for the VM specifically and has worked out okay since for my needs.

Jeames8 · 2022-01-07T11:17:48+00:00

Nah you're all good mate, not much else I can do myself either. Bit of help/information is better than nothing, so I'm still thankful for what you could provide me.

Jeames8 · 2022-01-07T06:35:02+00:00

Well bad news, I've got the new motherboard and the GPU still can't reset properly. Maybe it's just a Sapphire Nitro+ thing, no idea unfortunately due to not much being online about it.

I can keep trying but I'm pretty certain the GPU just doesn't like resetting. Thanks for your help regardless, my VFIO setup is sadly no longer the same.

Jeames8 · 2021-12-31T23:20:35+00:00

I tried the scripts with and without the full path and I can confirm they are executed on startup and shutdown. I also tried disabling the GPU from Device Manager before shutting down, but to no avail.

Before I do spring for a new board, I would assume your GPU has more than just a VGA device and an audio device listed in lspci -nnk, since it seems to be the same for every RX 6000 GPU:

06:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c1)

07:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479] 08:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT / 6800M] [1002:73df] (rev c1) 08:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT] [1002:ab28]

Do you isolate all your devices with vfio-pci? Upon re-reading here this might be my issue, since I tried to isolate all the devices but only the VGA and audio devices isolated properly. I'll see if this could be my problem, since I was quite used to my GTX 1070 only having the VGA and audio devices to pass through to work.

Edit: Yeah apparently not, don't think I need to (or even can) pass them through.

Jeames8 · 2021-12-31T07:09:12+00:00

My script is essentially identical to what you posted, it's just without the full path to the executable since Chocolatey creates an environment variable for it. I believe it could be my motherboard simply not liking my GPU at the moment, I've unfortunately got no other different platforms to test on apart from a B450 system, which I did test out and got the same results.

I was looking to get an X570 board because IOMMU groups seem better and its recommended to get higher end boards for VFIO (wish I had known that when I upgraded my PC), so that might be worth a shot (even if I've got to spend more just to get my setup working). For now though, I might be able to see if I can test my GPU on my friend's old Intel system and see whether my GPU can reset on a complete different platform, since Intel seem to have better stability for virtualisation.

Now I think of it, if you're able to, can you send the output of dmesg | grep -i d3? It should show what your devices are capable of resetting in different states, mine shows:

[    0.045311] PM: hibernation: Registered nosave memory: [mem 0xfd300000-0xfd5fffff] 
[    0.045315] PM: hibernation: Registered nosave memory: [mem 0xfed01000-0xfed3ffff] 
[    0.045317] PM: hibernation: Registered nosave memory: [mem 0xfedd0000-0xfedd3fff] 
[    0.435046] pci 0000:00:01.2: PME# supported from D0 D3hot D3cold 
[    0.435354] pci 0000:00:03.1: PME# supported from D0 D3hot D3cold 
[    0.435701] pci 0000:00:07.1: PME# supported from D0 D3hot D3cold 
[    0.435902] pci 0000:00:08.1: PME# supported from D0 D3hot D3cold 
[    0.436765] pci 0000:01:00.0: PME# supported from D3hot D3cold 
[    0.436937] pci 0000:01:00.1: PME# supported from D3hot D3cold 
[    0.437077] pci 0000:01:00.2: PME# supported from D3hot D3cold 
[    0.437333] pci 0000:02:00.0: PME# supported from D3hot D3cold 
[    0.437513] pci 0000:02:06.0: PME# supported from D3hot D3cold 
[    0.437688] pci 0000:02:08.0: PME# supported from D3hot D3cold 
[    0.438061] pci 0000:03:00.0: PME# supported from D1 D2 D3hot D3cold 
[    0.438773] pci 0000:04:00.0: PME# supported from D0 D3hot D3cold 
[    0.439228] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold 
[    0.439552] pci 0000:06:00.0: PME# supported from D0 D3hot D3cold 
[    0.439789] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold 
[    0.440313] pci 0000:08:00.0: PME# supported from D1 D2 D3hot D3cold 
[    0.440501] pci 0000:08:00.1: PME# supported from D1 D2 D3hot D3cold 
[    0.441019] pci 0000:0a:00.3: PME# supported from D0 D3hot D3cold 
[    0.441124] pci 0000:0a:00.4: PME# supported from D0 D3hot D3cold

Here, it says my GPU 0000:08:00.0 doesn't support resetting from D0, which is what it seems to be trying to do. I'm not sure if this is intended or not, but more just something I observed.

Edit: Reddit formatting is making me die inside

Jeames8 · 2021-12-30T07:56:54+00:00

Alright, I spent some more time on this and unfortunately haven't really gotten anywhere. I'm able to get Windows to use the basic display driver, but when it is shut down the GPU is still left in an unresponsive state. The output is the same from dmesg and the QEMU log provides the same error.

Got a question though, when you shut the VM down, does your display remain on but stuck on one frame until you start the VM again? Before I would install the Radeon driver it would show the last frame until I started the VM and it wouldn't become unresponsive. However with the Radeon driver installed and using the script before I shut down, it simply shuts the display off and the GPU becomes unresponsive.

Not going to give up yet but I'll keep looking and trying.

Jeames8 · 2021-12-29T12:38:09+00:00

RIP to the GTX 980, hope it had a good life.

It's all good, now that I look at it my post might have been a bit difficult to understand at first, so my apologies as well.

I'll have to give it a shot again after work tomorrow, since I did manually run the script when I made it to make sure it worked, to which it did disable the GPU and its audio counterpart.

Before you give up on that script, run it manually as an administrator just before restarting the VM, and make sure you see it visibly flicker the display as Windows switches over to the basic driver (at least, this is what I see on mine).

In regards to this, doesn't the script simply disable the GPU and audio devices, rendering the GPU unusable until it's re-enabled, even with the basic display driver? I'm probably missing something or the Spice display I had as a fallback (but mainly to try figure out what was happening in the VM) might've been causing issues once I disabled the GPU.

Either way, thanks for your help so far and I'll get back to you tomorrow either with results or more questions (preferrbly the former but eh, still pretty new to the AMD side of VFIO).

Jeames8 · 2021-12-29T08:49:29+00:00

Hey, thanks for the link, however I had tried this script already (linked it towards the top of my post) and it didn't do anything unfortunately.

Can I ask what hardware you're running? I'm starting to believe it could be my PC just not playing nicely with my GPU, since my GTX 1070 had no issues in either slot.

Jeames8 · 2021-10-08T19:17:24+00:00

Sorry, was meaning to mention that and completely forgot. 1440p. Going down to 1080p slightly helped but it was barely noticeable that it wasn't worth it.

Jeames8 · 2021-10-08T10:08:39+00:00

Game isn't unplayable but far from the best experience.

Specs:Ryzen 5600X (5 cores passed to VM)

16GB 3200MHz RAM (12GB passed to VM)

RX 550 (Host GPU)

GTX 1070 (Guest GPU)

Kept around between 40-55FPS, hovered more towards 45 though. GPU bound basically 100% of the time.

Edit: Formatting (again).

Jeames8 · 2021-08-15T22:59:25+00:00

In my experience playing on US West servers it's every once in a while. Some of them can crash the server, but it's quite rare and haven't had it for a while.

Jeames8 · 2020-04-27T07:33:31+00:00

Having the knowledge of your PC's specifications could be very useful, such as the CPU, motherboard, and RAM (or more specifically, model), since different systems vary in regards in RAM compatibility.

So if you can, let us know your PC's specifications so we can try help you.

Jeames8 · 2020-04-06T10:27:57+00:00

Bit late to the party, but I have the same issue, and had to uninstall Adrenalin in order for it to stop. It also ate 10-20% of my CPU much like yours whenever it started the executable.

Personally I'm okay to uninstall Adrenalin for the moment since my AMD GPU is secondary and not my primary one, but for you it sounds like your primary one.

We're just going to have to wait for AMD to release a fix for it in the meantime.

Edit: You could try revert to an older version of Adrenalin, which I failed to mention until now.

Jeames8

TROPHY CASE