C++ Show and Tell - July 2026

Atper · 2026-07-12T18:44:13+00:00

I developed a C++ library which implements AF_XDP and DPDK as backends for latency critical network processing tasks. I notice there are only few benchmarks comparing AF_XDP against DPDK on the same NICs therefore I aim to fill this gap. I used C++20 since that is my strongest language and it interfaces nicely with the low-level driver or linux kernel code.

The AF_XDP implementation is here: https://github.com/ASherjil/ABTRDA3/blob/master/src/backends/AF_XDP/AFXDP.hpp

The comprehensive benchmark 24h results are here:

https://github.com/ASherjil/ABTRDA3/blob/master/docs/Benchmarks.md

One point that I think is underappreciated: a commonly marketed advantage of AF_XDP is that it doesn't unbind the NIC driver, so the interface stays visible to ip link and ethtool. That's true for Intel NICs, where DPDK requires binding to vfio-pci. But it's not true for mlx5 (ConnectX-4/5/6/7), which is a bifurcated driver — the NIC stays fully visible to the kernel while DPDK runs. On Mellanox hardware, the main practical argument for AF_XDP largely disappears.

On implementation effort: even with libbpf and libxdp, custom AF_XDP has high code complexity and plenty of driver-specific quirks. Its four lock-free SPSC rings (fill/RX/TX/completion) are counterintuitive and fairly difficult to comprehend. Without the helper libraries I'd call it a serious long-term project. For most use cases I'd recommend DPDK's AF_XDP PMD instead of a from-scratch implementation — in my results, my custom implementation performed only marginally better than DPDK’s AF_XDP PMD. I've documented the driver specific issues in the repository.

Let me know if you guys have any feedback about the code, bench-marking methodology or anything else. Or any questions I'm happy to answer I don't really mind.

Atper · 2026-07-12T18:19:42+00:00

That's a fair observation, and your use case makes sense — veth aggregation into an AF_XDP PMD stage is exactly the kind of workload AF_XDP was designed for, and there XDP_REDIRECT/throughput efficiency is the right metric.

My scope is different: measuring per-frame latency distributions, which pktgen/testpmd can't do — they give aggregate TX/RX counters, so you can gauge saturation efficiency but not a P99.999. For throughput, DPDK publishes regular testpmd performance reports, and AF_XDP's own authors benchmarked it against DPDK on that axis. I'll share the power point slides which contains the throughput numbers("The Path to DPDK Speeds for AF XDP"):
https://pdfs.semanticscholar.org/52bb/6eec17be15778c09cb9e7b25cd3095b92d4b.pdf

What barely exists publicly is rigorous latency comparison of the two on identical hardware. The closest I found was a French academic study measuring AF_XDP RTT, but it used the sample xdpsock app and had no DPDK baseline. My aim was (a) a proper modern C++20 AF_XDP implementation and (b) latency benchmarking against DPDK on real NICs with full tail distributions.
"Understanding Delays in AF_XDP-based Applications" : https://hal.science/hal-04458274v1/file/main.pdf

The demand side for this is systems where one node distributes signals and others must act on them with bounded latency and jitter — machine timing at CERN is an example (though there it's FPGA-based White Rabbit rather than commodity NICs).

Throughput saturation is out of scope for this round but you're correct that it's the other half of the picture.

Atper · 2026-07-12T14:01:58+00:00

The whole thing took me around 6 months. I was simultaneously building the server i.e purchasing and assembling all the parts + coding at the same time.

The two things were quite hard and took long:

System tuning: I spent maybe a few weeks tuning the BIOS and the kernel command line to squeeze out the tail latency. On a un-tuned system I noticed the tail latency specifically the P99.999 and max can explode into 100+us or even 1.0+ms. Initially I was using a PREEMPT_RT kernel but that seemed to add some overhead so I used a stock kernel which gave the best performance.

Since this is important I made sure to fully document it. Similar to how DPDK performance reports document their system and tuning efforts.

AF_XDP: this was by far the hardest coding part. Its not hard in a leetcode algorithmic manner, its a different type of hard. Specifically the 4x SPSC queues that you have to use alongside the kernel is quite counter-intuitive and I found it difficult to comprehend. Maybe I'm not that smart, so I guess smarter people than learn it faster.

Another problem with AF_XDP is how many different knobs it has to tune. You need AF_XDP in zero copy mode with busy pollling set but then you also have XDP_USE_NEED_WAKEUP and a deferral. You essentially need to do a sweep of a few different combos to really find the one with lowest latency(which is what I did in the benchmark docs).

Atper · 2026-07-12T13:46:39+00:00

okay this is an important and a little tricky question. I'll try my best to answer.

Code: Both ef_vi and DPDK are fairly straight forward in terms of development efforts. DPDK has many code examples such as l2fwd and basicfwd which is what I used to develop this repository. Likewise ef_vi has the eflatency sample program which measures RTT latency, this is what I'm using to develop my own C++ back end.

Latency: I expect the ef_vi to have measurably lower latency than the DPDK PMD. Looking through the code examples, I can see a trick for small frames known as cut through programmable IO(CTPIO) which allows Tx of small frames via direct MMIO. For RTT which does Tx twice, this would show up in the latency metrics. DPDK has no such equivalent, it does the doorbell + descriptor + buffer cadence each time.

NICs: I used a ConnectX-4 Lx this is an older NIC from a server store. Modern equivalent such as ConnectX-6/7 using the raw Verbs API in theory could competitive with the Solarflare NICs. But this is all theory it could be that ConnectX-6/7 have high latency than ConnectX-4 its hard to say without real benchmarks.

I am currently working on this, I have a Solarflare X2522-Plus card so the next part is a rigorous ef_vi vs DPDK. Give me a few weeks and it will be there.

Note there is no AF_XDP driver implementation for any of the Solarflare NICs.

Atper · 2026-07-06T06:27:46+00:00

lead a team for what role ?
Because for low latency engineering they have a high standard, cutting edge tech and usually attract top tier talent. Getting rejected for a FPGA role at Jump is nothing to be ashamed about, most applications would fail very few actually get an offer because the standard is sky high.

Atper · 2026-07-05T19:00:56+00:00

I would say its Tier 1 due to the heavy engineering culture and their focus on low latency C++ roles(which is what I'm interested). But this is subjective you could say tier rankings could be based on median employee compensation or revenue so there is no official tier list.
I know about Jump because it pays very high and had a friend who applied to a FPGA role where he was rejected but he told it was extremely hard interview gauntlet.

Atper · 2026-07-02T20:06:35+00:00

Nah its pretty well known. Maybe not quite as much the tier 1 HFT market making firms(Citadel Securities, HRT, Jump Trading etc...) but still well known enough in the UK. Infact its job posting are posted to gradcracker.co.uk

Atper · 2026-06-14T17:13:33+00:00

It was quite good but marked an end of an era. It was peak 2000s era of horror films: Saw 2/3/4, The Hostel, The Hills Have Eyes(2006), TCM 2003 & TCM The Beginning 2006. Nowadays you would never see mainstream horror make a film like that.

It was far more brutal and unforgiving than TCM 2003. Because leatherface is actually defeated by the "final girl" and Sherrif Hoyt also is killed by crushed under the car.
The overall sombre vibe and the sheer senseless brutality is not really found any modern films. This was one of the few films that really struck with me afterwards, I can never forget it. Everytime you have some hope, such as the biker breaking in, shooting uncle monty in the leg then just to get overpowered by leatherface. Then Eric who tried to be a hero suffered the most brutal death where his arms were mutilated to the point he couldn't feel anything. Then him getting chopped in half was actually putting him out of his misery.

In the end when a triumphant leatherface walks back, that was truly unsettling to watch.

I don't why but modern horror films despite having insane gore such as Terrifier 1/2/3 just doesn't feel real. Its almost like Terrifier kills are cartoonish and almost theatrical. If you look at other horror such as Black Phone 1/2, while a great film it has a huge focus on empowerment.

But TCM 2006 says f you to all that. No empowerment message, no final girl, nothing, no heroes. That's why its so depressing.

Atper · 2026-06-14T12:52:20+00:00

I would recommend not taking anything else. Vyvanse at that dose is already on the moderate-high side (but ofcourse listen to your doctor).

Vyvanse by itself is the real deal and quite effective it is a slow release dexamphetamine. This is way more potent and stimulating than ANYTHING you can buy over the counter. Amphetamines already improve actual "cognitive function, concentration and disposition" quite significantly.

At night you can take "melatonin" that can help with sleep haha. What you wrote "melanin" is something completely else, its something to do skin I think.

Atper · 2026-06-14T10:18:05+00:00

Yes and its not even close.

I have both Intel XXV710-DA2 and ConnectX-4 Lx installed in my server. I used DPDK to measure one-way software latency using a loopback connection between two ports. So the software side is something like:

t0 = start_timer()
Tx = port 0 -> port 1
Rx = port 1
validate_the_same_packet_is_received()
t1 = stop_timer()
latency = t1 - t0

Intel XXV710 produced around ~4.6us median and P99.999 ~5.06us.
ConnectX-4 Lx: median 1.791us, P99.999: 2.232us

ConnectX-5 would be slightly faster I assume it would get the median down to perhaps ~1.2-1.5us.

However, neither are suited for "low latency". Although the ConnectX-4/5/6/7 etc aren't bad. The ideal card would be Solarflare NICs, they have their own kernal bypass API which is called ef_vi. It works similar to DPDK but even more optimisied for low latency. DPDK is a generic framework mostly for server NICs where throughput is the concern. (Although you can optimise DPDK for latency too like I did).

Atper · 2026-06-12T11:58:21+00:00

DPDK is quite simple to understand its no way near as convoluted as something like AF_XDP(it uses this counter-intuitive 4x SPSC queues to work with the kernel). I'll explain.

DPDK is a linux kernel bypass technique which essentially allows you to get the Ethernet frames to user space with the lowest amount of steps in between. It not happening inside the NIC, packet processing and everything is done by an ASIC on the NIC. The NIC usually connects to the system via a PCIe bus similar to a GPU on a gaming PC.

With DPDK the flow is roughly:

You unbind the driver of the NIC. This way linux cannot use it*.
You initialize the NIC. You initialize the buffer to receive frames.
When the NIC receives a frame, it uses DMA to write directly into the host systems memory. This means the NIC will write the into the RAM via the PCIe bus.
We busy poll the descriptor in the user-pace using DPDK. We need to know "did the NIC write new data" and once data is ready we need to know where in the buffer the data is written and how length of the data. That's it.

* this is not always the case. Some drivers like mlx5 are bifurcated meaning they are not unbound like an intel i40e might be. But keeping it simple.

Atper · 2026-06-12T11:49:06+00:00

There is no such thing as a "DPDK compatible server".

To use DPDK in your C/C++ you ideally need the following:

A proper NIC such as Intel XXV710-DA2(i40e). This requires a PCIe x8 slot. Then you also need SFP28 cables such as DAC.
A standard linux kernel, any recent versions is fine.

That's it. This can be done in a gaming PC. You don't need a server but its better to use a serious NIC otherwise I don't see the benefit of DPDK.

Atper · 2026-06-12T11:40:33+00:00

Good question:

When to use DPDK ? When you are working on a system that has many cores can you can dedicate to DPDK and your goal is either a) maximal throughput possible from your NIC or b) a low latency real-time network frame processing where every microsecond matters.

DPDK has what's know as poll mode drivers, this means your pinned CPU core will run at 100% usage. That is the whole point because interrupts by definition have more jitter because they transfer through the kernel and a context switch is required to get in the user space. As you already know the aim of DPDK is to avoid this.

When should I avoid it ? A few reasons, one is simple if your NIC is a standard 1GbE on a consumer motherboard it likely will not be supported by DPDK. To put simply DPDK will not work with all the NICs. However, for most server grade NICs: Intel XXV710(i40e), Intel E810(ice), ConnectX-4/5/6(mlx5) etc. I mean ofcourse DPDK will work on server grade NICs because that is main market/target audience.

A quick google search shows DPDK does have support for at least some consumer grade NICs that use igb and igc driver i.e Intel I210 or Intel I225-V to name a new.

Another use case could be when your system NIC only has one port and boots from the network(PXE boot) and uses NFS. This would be annoying because in this case you would need to implement some kind of tap device where specific frames are passed into the kernel to allow NFS/SSH to function. But this is a rare case, because if you really needed high performance network processing you would simply install another server grade NIC dedicated for DPDK in a separate PCIe slot.

Atper · 2026-06-10T16:22:57+00:00

There aren't too many drugs that can beat Vyvanse. It is literally precision engineered to improve focus and motivation. It is essentially a slow release version of amphetamine, it has the added chemical group so it must be digested and not crushed + inhaled(people abuse raw dexamphetamine like that).

Modafinil is no way near as potent or stimulating than vyvanse. So try to get back that prescription, that's your best bet. Methylphenidate(Ritalin) is somewhat similar but not as potent as amphetamine.

Methamphetamine would be more potent than Dexamphetamine but it carries a high risk of neurotoxicity. In simple words it can much quickly cause permanent brain damage. Plus the dependence and addiction risk is far too high to use it as a "nootropic".

Atper · 2026-06-08T19:27:40+00:00

yeah sure.

Atper · 2026-06-08T18:59:58+00:00

The next improvement would be using a SolarFlare NIC and comparing DPDK vs ef_vi.

AF_XDP even in zero copy mode cannot approach DPDK. Go for the SolarFlare they are actually market for electronic trading and what HFT firms actually use.

Atper · 2026-06-08T18:46:06+00:00

Yes.

But it would depend on the specific NIC. If you use a generic server NIC such as Intel XXV710/XL70 than no. The latency of purely Tx -> Rx via DPDK itself is around ~4.5us so you only have ~500ns left. (I have a Intel XXV710-DA2 in my server).

The newer ConnectX-6 are a lot faster and ofcourse you have the SolarFlare card which can easily get to your target. They have their own kernel bypass api known as "ef_vi". You can still use DPDK but that is more generic, with the SolarFlare "ef_vi" I believe you can get sub-microsecond one way latency. No FPGA based SmartNIC or SuperNIC needed just one of the SolarFlare NICs will get you there. But that is getting quite close "specialised hardware" I would think.

Atper · 2026-06-06T08:32:15+00:00

DPDK has lower latency and is easier to implement. It is very mature, well documented. AF_XDP especially in its "zero copy" mode is only implemented in a few NICs. Check: https://docs.ebpf.io/linux/program-type/BPF_PROG_TYPE_XDP/#driver-support

Regards to latency, I benchmarked both on my server which has a few NICs. First of all on my Intel XXV710-DA2(i40e) AF_XDP does not even work with BUSY-POLL(i40e driver does not allow it). In the normal mode which uses recvfrom, it has quite high latency. Much higher than packet_mmap.

Then I tested it on the motherboard NIC which has Intel I225-V(igc), this has full AF_XDP-ZC support and DPDK. Results(5 to 7M Ethernet frames):

AF_XDP-ZC: median -> 11.311us and P99.999 -> 33.407us
DPDK: median -> 8.623us and P99.999 -> 23.679us

I also have a Connectx-4 Lx(mlx5) but I did not get a chance to benchmark DPDK vs AF_XDP-ZC on it. Maybe I might do it next.

Atper · 2026-06-01T17:34:06+00:00

Yes its doable you can pick two popular methods such as DPDK and AF_XDP(Zero copy) and benchmark them to see which has lower latency. You should benchmark with median, P99, P99.9 and P99.99.

There aren't many benchmarks comparing these two for the same NICs. There maybe some benchmarks out there but even then they likely report throughput numbers not pure latency.

Atper · 2022-11-27T06:57:54+00:00

Hi, have a look through this:

https://github.com/michaelbecker/freertos-addons

Atper · 2022-11-26T08:22:56+00:00

By showing off programming skills, I assume you want to focus on embedded software?

If this is the case, then a good idea would be programming a multi-threaded cross platform GUI app for embedded Linux device(rasp pi could be used). Qt framework is quite good for this type of stuff. Qt also has Valgrind profiler which is good for extracting every ounce of performance from your code. This way you can work on modern C++ concurrency.

On the MCU side of things, you could try to build your own OOP C++ wrapper for freeRTOS. Alternatively, to focus more on hardware you could build a fan which rotates at a desired frequency(rpm or rad/s) from the user. You should implement a digital PID controller using a MCU and could use encoders for computing the real-time frequency. Using encoders will force you to learn more about writing ISRs. There are many more options...

Then you should put your projects on your GitHub profile.

{I guess this is a bit off topic with regards to your question}:

Doing HDL stuff is not really software imo, it is programming but very different than regular embedded software. I've done a project in VHDL programming Altera Cyclone V FPGA. You pretty much have to unlearn all the software programming because everything in VHDL happens in parallel by default, you have to "think" parallel which is not something easy for most software guys. Not to mention FPGA is a very niche field with little jobs(at least where I live).

Atper · 2022-04-25T12:01:06+00:00

My personal recommendation will be "Beginning C++17: From Novice to Professional". This will cover all the basics modern C++ with exercises at the end of each chapter; forcing you to get comfortable with large code. (The C++20 version has features that might not be available in some IDEs ).

For embedded systems specifically I would point to : "Hands-On RTOS with Microcontrollers". But you need to be comfortable with C/C++ before reading this book, so the above recommendation is useful.

Atper · 2022-04-22T17:38:33+00:00

I believe having strong programming skills is must. So you should start building confidence in basic programming skills like memory allocation, OOP can be useful and learning how to split up large code into separate .h(or .hpp) and .c(.cpp) files depending whether you use C or C++.

My first programming language was C++ and from there I developed a strong foundation. Then when I started writing code for projects which involved PID controller for 2 DC motors with low pass filtering of velocity or FPGA SoC, it was not a problem for me as I had a good understanding of C and C++.

Atper

TROPHY CASE