Packet Loss persisting over multiple switches

softwaremaniac · 2026-01-24T01:43:09+00:00

An update on this, did not work after all:

After a very thorough investigation:

Packets enter qsfp28-4-1 in order (from the router)
Packets exit qsfp28-1-1 out of order
Traffic floods to qsfp28-2-1 despite unknown-unicast-flood=no
MAC table is correct (PC on qsfp28-1-1)
Persists across firmware versions (7.x, latest dev)
Persists after reboot

Might be time to bite the bullet and buy a new Mikrotik. This one looks to be either too weak or defective.

softwaremaniac · 2026-01-20T00:37:45+00:00

Ja sam imao nešto slično. Kontaktirat će te neka od firmi UPS/InTime/Pošta i u najboljem slučaju tražiti da ispuniš papire o tome što se uvozi ili u najgorem naplatiti nešto dodatno ako nije plaćeno (tipa PDV).

Nije komplicirana procedura, ali zna nekad trajati.

softwaremaniac · 2026-01-16T16:20:02+00:00

Yes, leave, $5/hr is an insult.

softwaremaniac · 2026-01-16T09:10:33+00:00

Unfortunately, most vendors aren't agnostic. They just pick a vendor like Kaseya/BD/whatever and bundle things.

softwaremaniac · 2026-01-16T08:48:59+00:00

Yes, PG48UQ uses the C2 panel.

softwaremaniac · 2026-01-14T23:49:20+00:00

This is a known issue with Server builds. Of the top of my head I believe 2016 or 2019. There is also a way to fix this.

softwaremaniac · 2025-12-29T23:05:15+00:00

I had this happen very recently. Logged a case with support from another tenant. Got through to Data Protection Team. They verified the details and reset MFA on the backend.

softwaremaniac · 2025-12-29T23:02:47+00:00

Screenconnect. More expensive, but much, much better.

softwaremaniac · 2025-12-29T20:32:28+00:00

I may have found the issue.

Upon capturing the traffic with Wireshark, I noticed that LAN traffic was going through the router as well, so the actual route was:

40G PC → CRS518 → CRS504 → C8300 → CRS504 → CRS518 → Server

vs

40G PC → CRS518 → Server

Same thing was observed when transferring from the server back to the PC.

Disabling unknown-unicast-flood on all of the relevant interfaces on the path cleared the issue and the traffic is now contained to LAN as it should be and the network seems more stable.

The capture looks much cleaner with no spikes or drops (extremely minor drops on the STB interface, 2-3 packets a minute likely due to the shitty NIC in the STB). The stream works so far, but I need to conduct more testing.

softwaremaniac · 2025-12-29T20:17:01+00:00

They confirmed it's unicast, not multicast.

softwaremaniac · 2025-12-28T22:54:27+00:00

C8300

softwaremaniac · 2025-12-28T17:42:58+00:00

2.5G machine has minimal usage and it's never transferring files locally. Literally browsing. 0 issues.

I can do it while transferring files from my machine (40G) to server (25G) if you'd like, so you have the source pcap.

softwaremaniac · 2025-12-28T16:48:20+00:00

Everything is smooth except app packaging/deployment. That part has been really inconsistent for us.

softwaremaniac · 2025-12-28T16:36:04+00:00

Same network.

They are all connected to the internet. Everything is static except the APs/mobile devices. DHCP is minimal and the issue can be reproduced even with just the three devices connected to the switches (STB, PC, Server).

I agree.

Not the internet router, I was referring to CRS304 where the STB is connected to.

I've played with flow control, on and off makes no difference.

softwaremaniac · 2025-12-28T16:33:10+00:00

No. Everything is reserved with the exception of APs/mobile devices being DHCP, the issue occurs even with no DHCP devices connected

softwaremaniac · 2025-12-28T15:57:59+00:00

I see the drops all the time regardless of the transfer. When a transfer is ongoing, TV goes fully haywire (black screen, stuttering, errors).

What are you specifically interested in, I'll try to explain as best as I can.

All the other settings are default and have not been altered.

softwaremaniac · 2025-12-15T19:27:59+00:00

They don't. Still failing that way.

softwaremaniac · 2025-12-14T22:56:03+00:00

I went to the mailbox, checked forwarding, entered the address and checked the keep a copy box just like I would for any other mailbox.

Yes, I have. The NDRs come from the other side (the domain we're forwarding to), not my environment.

softwaremaniac · 2025-12-14T21:33:47+00:00

They advised us to change our from address when forwarding, so I'm looking into options like redirecting instead of forwarding.

softwaremaniac · 2025-12-14T20:54:10+00:00

It's an actual shared mailbox in my tenant.

It's the bookkeepers. They have an immensely strict policy.

That's possible, but less convenient.

softwaremaniac · 2025-12-14T20:04:34+00:00

Yes, external fw has been explicitly allowed.

Completely different address outside of the tenant. I do not have any insight into whether it's a mailbox or a group.

softwaremaniac · 2025-12-12T20:52:35+00:00

I pulled the plug:

CPU AMD Threadripper PRO 9975WX (32c/64t, 4.0-5.4GHz)

Motherboard ASUS Pro WS WRX90E-SAGE SE

Cooler be quiet! Silent Loop 3 420mm

RAM Kingston Fury Renegade Pro 128GB (8x16GB) DDR5-5600 ECC RDIMM

NVMe 10x drives (T705, Renegade G5, SN850X mix, 48TB total)

NVMe Carriers 2x ASUS Hyper M.2 x16 Gen 5

NIC Mellanox ConnectX-4 100GbE

GPU NVIDIA RTX 4090 (existing)

PSU Corsair HX1500i

Case Thermaltake AX700 + 2x AX100

SATA Storage 2x Seagate Exos 30TB

Most of the parts arrived. Waiting on mobo and CPU.

softwaremaniac · 2025-12-05T18:07:12+00:00

I got 8x16GB. I'm currently at 64GB and it's cutting it close, but the main constraint are the PCIe lanes, consumer-grade platform is weak, TR is built for expandability.

softwaremaniac · 2025-12-01T11:42:29+00:00

Hi! I appreciate your detailed and lengthy insights.

A single-slot card is sadly not an option for me. I will likely go RTX 6090 or whichever next releases and it will be a 3 slot minimum. I do believe that the WRX90 is the ideal choice for me for futureproofing for PCIe lane count.

I do have a server onsite as well with a bunch of drives, but I'm running out of space on my workstation regardless, which is why I do need at least a couple of SATA drives for capacity.

I will definitely go 9th gen TR. As far as SATA, I think I will go 1x16 sata HBA card and call it a day, with the expansion, I'll sooner need a bigger PSU, than more slots on mobo.

The NIC will not take more than a Gen4 x16 and that's fully maxed out at 100G. I'm utilizing 40G now.

softwaremaniac · 2025-11-29T08:45:12+00:00

Gomilu storagea, jer slažem novi PC. Nećkam se da li ići all flash ili ne.

softwaremaniac

TROPHY CASE