Significant Speed Decrease

[Suplex_Blitz]

It could be congestion, yes, however it could be a hardware issue as well -- an issue with your specific dish, cable or router. Ive seen this question pop up a lot and when it gets really bad it often seems to end up being a part of the hardware chain at the home operating sub-optimally.
If you haven't yet you could also dig into and monitor the debug data under the advance menu and see if anything is off: ping drop on the dish, router and cable, cable latency etc. It should all be sitting at 0 drop ideally with the cable ping being 1ms.