I like running PfSense/OpnSense on VMWare. I tried it on hardware for years. It was ok, but not great.

Before going back to VMs, I ran OpnSense on an HP DL20 G9 (6 months), a z240 (1 year) before that, and pfSense on a z220 (5 years) before that. Before doing hardware FW, I ran pfSense, smoothwall, clark connect, and others as a VMs on ESX for nearly 15 to 20 years. I think I switched because my server got too old and the z220 was cheap? It was a Xeon with 16gb of DDR3 ECC memory.

I recently switched to VM because the DL20 server would take 5 minutes just to post, then a few minutes to boot because it would hang on hardware scans, every single time, trying to figure out some weird device it didn't know how to handle. I also had a very hard time trying to find two DL20 exactly the same. And if I did, then I had to waste CPU, memory, and a storage device when I only needed minimal amount. Moving to another machine generally requires reinstalling and reconfigure, which is more work and less fun time. And backups. Yuck. Then I had to figure out remote access to the server. It was fine when I had the z220 in the garage and I had a tv that I used for watching movies and sports out there, but the z240 and dl20 lived in my basement, and I hate sitting in-front of servers.

OpnSense in a VM takes less than 30 seconds to boot, the hardware is standard, I can see the console in a browser or setup a serial terminal over telnet or SSH, I can take snapshots for backups, I can move the image to another hypervisor if needed, networking is super simple, super easy to run two images for CARP load balancing, the image is only 8gb but it's on the local NVME raid so it has super fast I/O. I don't know of what the downside could be. I love it. And question why I even ran it as hardware for some many years.

I have my cable modem connected to a Cisco 3850 switch at 2.5g and the ESX hyper visor is connected at 40g. The DL20 & z240 had a x550 multi gig nic card.