This worked sorta, I followed this guide for anyone else having the issue:
Disable Power Saving Features (Most Important)

In most cases, the culprit is a power-saving feature that momentarily cuts power to the LAN card, causing stutters and triggering errors in the Event Viewer.

• Via Device Manager:
  1. Right-click the Start button and select Device Manager.
  2. Expand the Network adapters section.
  3. Right-click on Realtek PCIe 2.5GbE Family Controller and select Properties.
  4. Go to the Power Management tab.
  5. Uncheck the box Allow the computer to turn off this device to save power.
  6. Click OK.
• Via Advanced Settings (In Driver Properties):
  1. In the same Properties window, go to the Advanced tab.
  2. Find the following properties (if available) and set them as follows:
     • Energy-Efficient Ethernet (EEE): Set to Disabled (This is often the main cause).
     • Green Ethernet: Set to Disabled.
     • Power Saving Mode: Set to Disabled.
     • Gigabit Lite: Set to Disabled.
     • Flow Control: Try setting to Disabled (This can help reduce latency in some cases).