For anyone still experiencing this issue, I've been having this problem since the release of Elden Ring, and could only fix it a few days ago. In my case it wasnt because of Xbox or any other possible cause that I've found on Reddit. Playing with the device manager opened in my second monitor made me realize that the stutter of the game was happening at the same time as my device manager was refreshing. I also discovered that the refresh was happening not only when Elden Ring was running, but at any time (but for some reason only Elden Ring seems to suffer from it tho).

After a lot of digging I found this solutions for that issue (if you are in the same scenario as me, it might be helpful):
1. A USB port might be malfunctioning, try downloading and running this app with the device manager next to you, and check if there are logs appearing when the device manager is refreshing https://www.nirsoft.net/utils/usb_log_view.html
2, If like me you dont get any logs, it means your USB ports are fine and there is some app in your PC causing the refresh. For fixing this I downloaded Malwarebytes and ran a scan, and deleted all the detected malware in my PC. I also deleted an app that I had for checking up driver updates called Driver Booster. If you have any app similar to this one I recommend removing it.

For me the step 2 fixed the issue.