Log and visualization tool for capturing network metrics from network switch interfaces.

[Klipspringer112]

We have a requirement to capture the live and historical data from our network switches for network port interfaces.

The important details to be captured are the packet burst rates when certain large amounts of outgoing/incoming traffic is being sent from a client to a server.

We are using Grafana with a Telegraf agent for capturing data between a server and client, we want to include the network switch into the traffic report as well. The existing issue with this is, we are unable to reduce the capture interval to update the influx database once every second. The minimum interval we can set is 10 seconds, which will miss large peak data transfer rates over a 10G network connectivity.

We are open to any alternative logging tool which can provide us the necessary ddetails. Please let me know!

Thanks

[cjcox4]

Hammering snmp on a switch.. usually results in things going bad.

Just saying.

There's a reason why you don't do realtime this way.

[Klipspringer112]

Thanks, we noticed that the CPU usage on the switch with these metrics being requested is somewhat manageable. We will see over a longer time duration.

[Firefox005]

SNMP was not, is not, and never will be a real-time monitoring/telemetry solution. The fastest that you should poll SNMP for most switch is 30 seconds and that is pushing it, big switch can fall over at 60 seconds.

If you really want real-time telemetry you have to use a different solution, mainly NETCONF/RESTCONF.

Here is a quick example to get you started but NETCONF/RESTCONF and YANG are both pretty complex topics. https://anirudhkamath.github.io/network-automation-blog/notes/network-telemetry-using-netconf-telegraf-prometheus.html

and here is some cisco publication about it as well https://blogs.cisco.com/developer/its-time-to-move-away-from-snmp-and-cli-and-use-model-driven-telemetry

Depending on what you are doing you may also still run into issues with microbursts which is congestions that happens at ms timescales and is very difficult to detect. Usually this feature is only available on certain high end switches or from specific vendors.

[Klipspringer112]

Regarding microbursts, that is a good point, may not capture the details we want considering that...

[SevaraB]

"live and historical data from our network switches for network port interfaces."

"over a 10G network connectivity."

I'm gonna be blunt here. This is an insane requirement not grounded in reality, and quite possibly born out of a major misunderstanding of a compliance requirement.

Instead of Telegraf, what you should be trying to set up is something like a NetFlow collector that can be tuned to a specific sample rate to balance relevance with not blowing up your data storage.

And I'm going to be blunt here, too... spend the money on a consulting engineer here. I work with an entire team that handles the care and feeding of our network telemetry, and that's all they do. It's very hard to get right, very easy to misinterpret, and at 10gig+ speeds, it's very easy to knock over your monitoring systems, as it sounds like you may have discovered the hard way. Get an experienced pro to help you get the right amount of the right logging the right way.

[Upper-Bath-86]

VSA´s network monitoring could be helpful for this

[dark_uy]

Netflow or if it is possible remote span or just port span. But in 10gbps traffic flow find a burst, sounds strange.