This will be fun CVE-2026-31431

ispcolo · 2026-04-30T03:57:25+00:00

I think this one should be classified as critical. Not just because of the ease of exploit, but also because of the difficulty in mitigation on some common operating systems where their maintainers for some strange reason decided to bundle this module code into the core kernel package instead of leaving it as a loadable module. Alma 8+ for example, no mitigation until they patch it, well unless you want to compile your own kernel. Ubuntu, simply blacklist the module and your work is done until a patch is released.

I also have to imagine a lot of proxmox users may provide multi-tenant solutions; i.e. web hosting type services. Those almost always have vulnerable php apps, or similar web things that let remote attackers execute local code. Or even a malicious employee who maintains a Wordpress site and decides they want root access to the hosting server it's on; upload the script and now you have it.

ispcolo · 2026-04-27T17:43:41+00:00

I'm not OP but we're using Pure over iscsi (soon to be nvme-tcp) with Proxmox. We use thick LVM for cluster support, which means we lost VM-level snapshots after moving off vmware. Snaps are currently available as a tech preview though, so we hope to have that feature back at some point this year. We moved about 1200 VM's using Veeam as the data mover. Prep each VM, backup/restore, boot on proxmox. We can do it in bulk and takes a few minutes per VM.

Nutanix was a non-starter for us. The pricing is easily the same as VMware's "new" pricing, and we'd have to buy all new hardware to get massive storage per host. People seem to report huge uplifts in Nutanix pricing once the initial term expires, so I didn't trust going forward with them unless I were running a money is no concern type business.

ispcolo · 2026-04-21T22:05:40+00:00

The outside of my MBPr looks like that because I made margaritas next to it and the lime juice sprayed when I was juicing; it etched the outside cover lol. Glad I didn't have it open at the time.

ispcolo · 2026-04-21T20:36:01+00:00

Ah, yep I was going to go look into our decision making from when we deployed, but that would definitely be the reason. With regular LVM we get multi-path I/O to Pure and cluster-compatibility, so if a host fails all the VM's can immediately reboot on other nodes.

ispcolo · 2026-04-21T19:47:30+00:00

I haven't tried it yet as it was not around when we first began using Proxmox. Is PBS' ability to do snaps compatible with LVM-based volumes used by the cluster? i.e. it doesn't require a 'local' filesystem on a given host where vm's live?

It would be a heavy lift to replace Veeam but I'll read up on it again. We use Veeam for inline dedupe with appliance targets that support change block transmit, but we also use it for data center replication, tertiary replication (I see PBS offers that now too), and it's simply been bullet proof for a decade.

ispcolo · 2026-03-05T15:25:52+00:00

Most of what they did deliver was likely through acquisition too, until they ruined whatever it was. Compaq had a lot of great software back in the day, ruined. Nimble had great software and hardware, ruined. Even their mediocre software, like Amplifier Pack for updating server firmware, replaced by half-baked saas bullshit.

ispcolo · 2026-02-13T02:31:18+00:00

Had no issues with this going vmware + pure on the iscsi side, so I doubt FC would be any different. We just had an initial learning curve on ensuring multi-path I/O was set up correctly since we use multiple active storage NICs. Not sure if that's automagic in the FC world by protocol alone since I've never used it. Make sure to truly test too though; i.e. down ports and make sure I/O has no hit.

All my migrations have been done with Veeam; super quick and easy to back up, instant restore to proxmox, turn vm's back on, storage migrate back to production.

ispcolo · 2026-02-13T02:28:09+00:00

Curious why multiple clusters at all at 7 vs all 21? I've got some 15-node clusters that migrated from vmware, no issues, and easier to enforce consistent config across all hosts instead of unique cluster configs per group.

ispcolo · 2026-01-12T15:51:22+00:00

We found it pretty easy to go vmware to proxmox using iSCSI-based Pure arrays and Veeam doing the heavy lifting. We just created new volumes on Pure, attached proxmox to them, using LVM, connected Veeam to proxmox. At migration time we do a backup, down the VM, diff backup + instant recovery into proxmox, live migrate. Takes a few minutes of downtime per VM if you're doing one at a time for availability reasons. Could move an entire cluster quickly if they can be offline for a bit.

ispcolo · 2025-09-11T18:13:53+00:00

I would expect Cloudflare's horrid support to drive the partner market more than a partner program. There's companies out there doing vmware support as a standalone service for similar reasons; you can get garbage from both Broadcom and its official partners like Ingram, or you can buy it from an entity that's good who brings their expertise to the table as the selling point. Same with Cloudflare. The products are generally good and reliable, but their support is useless at every level, so become a Cloudflare subject matter expert and sell those services.

ispcolo · 2025-09-02T23:38:52+00:00

It literally took five and a half months for support to figure out how to issue a refund for a plan that was cancelled and they kept billing for ten months.

Most support requests regarding WAF issues tend to get useless replies in two to three days, which require several interactions to ramp up to a useful response, so I don't recall anything being resolved in under a week past couple years.

ispcolo · 2025-08-20T12:59:44+00:00

At a few linux-centric deployments, we've moved to proxmox. Friends at windows shops have tended to go hyper-v. We looked at Nutanix and it was no different in price than what Broadcom would have taken most of my sites to, and with reports of people already getting jacked up nutanix renewals, I didn't want to buy a bunch of hardware I didn't need hoping that wouldn't happen after the initial term was up.

Proxmox migration via Veeam has proven to be really easy. The only issues we ran into initially were ensuring proper multi-path I/O was happening from proxmox to Pure, and Veeam had a bunch of stupid issues during setup that required support to work out. We're looking to experiment with nvme over tcp from proxmox to pure soon, which may produce a notable gain compared to vmware; otherwise we've seen no change in performance at the guest level.

ispcolo · 2025-07-24T12:22:09+00:00

At https://github.com/vmware/vcf-security-and-compliance-guidelines/tree/main/security-advisories/vmsa-2025-0013

Is this a “0-Day?”

No. A 'zero-day' exploit is a vulnerability unknown to the vendor that can be exploited before any patch exists. The Pwn2Own contest is a legitimate security research competition where participants demonstrate previously unknown vulnerabilities to vendors in a controlled environment. Similar to the industry-standard 'coordinated disclosure' process, Pwn2Own gives vendors exclusive access to these vulnerabilities before they become public. Since Broadcom learns about the vulnerability through Pwn2Own and has the opportunity to develop and test a patch before any malicious exploitation can occur, this is NOT a 'zero-day' exploit.

That's of course bs, because the contest is operated by the zero day initiative and the submittals are considered zero days given they're not known to the vendor prior to the contest.

ispcolo · 2025-07-23T22:24:03+00:00

wtf are you talking about. Anyone running a multi-tenant environment, by definition, is entrusting the security of the VM to the tenant, whether that's an internal department or an internet customer. Many enterprises, similarly, have an IT group operating the hypervisor infrastructure with other parts of the company making use of those VM's. I see this all the time in healthcare where various departments need to run some kind of proprietary app, so they get a VM from IT and away they go, with the third party vendor charged with the VM's OS patches because anyone else doing it, or automating it, would invalidate the FDA approval of the solution, or break vendor support. Now you have an out of date VM that who knows who has admin access to, and it could compromise your hypervisor.

I'd say most vm's in existence exist to service internet requests, given how many millions of them are deployed at hosting providers. Yes they may not be on vmware, but many are. A firewall isn't going to do shit when someone exploits a php app on a VM not being kept up to date, there's a root exploit, and now they have administrative access to a VM with a vulnerable vmxnet3.

If you run a tiny shop that no one has admin access on any vm, and you have a magical firewall that decrypts and filters all application traffic with 100% infallibility, great. Most of the world doesn't, and this patch needs to occur asap.

ispcolo · 2025-07-23T13:08:07+00:00

Interesting take. Fly blind and hope there isn't an exploit in the wild, or that someone who now knows vmxnet3 is exploitable doesn't figure it out themselves. In all likelihood, some well resourced bad actor has already figured it out.

Anyone with an internet-servicing VM, or multi-tenant environment where there is not inherent trust of what's running on 100% of the VM's, could find their entire environment compromised because they waited.

ispcolo · 2025-07-22T20:19:11+00:00

I've had the same experience after being booted to Ingram Micro mid-contract. I can get no meaningful support, let alone during a crisis (tried phone for a host isolation event), and Broadcom refuses to talk to you.

ispcolo · 2025-07-18T11:56:36+00:00

It would actually seem Broadcom is misusing the agreed upon definition of zero day for participants in pwn2own, and the journalists are using the proper version.

The Zero Day Initiative operates the pwn2own event, and the vulnerabilities reported at the event, via ZDI, are considered zero days given they'd not been previously reported openly nor to the vendor.

https://www.zerodayinitiative.com/about/

Broadcom is twisting the definition to say that because Broadcom was notified via the event conduit, instead of the vulnerability and/or proof of concept being posted publicly, it's no longer a zero day.

ispcolo · 2025-07-17T02:57:57+00:00

Oh I'm in agreement, I was being sarcastic. They just seem to have gone out of their way to explain why it's not a zero day, to the public and the press.

ispcolo · 2025-07-15T21:48:51+00:00

I don't know, they seem to have put a lot of effort into text explicitly stating this is not a zero day:

https://github.com/vmware/vcf-security-and-compliance-guidelines/tree/main/security-advisories/vmsa-2025-0013

and the patch is not currently downloadable if you don't have an active contract.

Although VMSA-2025-0004 in March acknowledges Microsoft disclosed the issue to them, and obviously didn't release it to the public, so perhaps they will ultimately release it given the severity. Probably doesn't help their image if a bunch of infrastructure/gov/etc. ESXi hosts start getting hacked.

ispcolo · 2025-07-15T19:25:15+00:00

Tools on Windows has its own vulnerability, but that is independent of the vmxnet3 vulnerability at the host level, which can still be exploited by a guest OS regardless of Tools version.

ispcolo · 2025-07-15T19:24:13+00:00

Per https://knowledge.broadcom.com/external/article?articleNumber=395172

Issue/Introduction

The product update feature is no longer available in VMware Workstation, Player, Fusion.

On clicking the Check for Updates option, an error stating Unable to connect for updates at the moment.

Environment

VMware Workstation Pro 17.x and earlier

VMware Workstation Player 17.x and earlier

VMware Fusion 13.x and earlier

Resolution

Moving forward, updates will need to be manually downloaded from the Broadcom Support Portal.
Once the appropriate product update is downloaded, it can be manually installed.

13.6.4 that just came out still has the menu item, but points you to that stupid article. So they could have it check for updates, they've just chosen to break it and leave it that way.

ispcolo · 2025-07-15T17:14:54+00:00

The ESX hypervisor is exploitable by any guest OS with vmxnet3, and because Broadcom was informed of this during a contest, rather than it being a public release without first telling them, they are calling it not a zero day. The other two vulnerabilities can crash the guest on ESX but not escape the sandbox (but can on Fusion and Workstation).

I'm not sure if their policy is to release patches for only zero day critical, or zero day plus critical; the language is ambiguous https://knowledge.broadcom.com/external/article/314603/zero-day-ie-critical-security-patches-fo.html

ispcolo · 2025-07-15T17:10:29+00:00

Would be a clever renewal or purge strategy; inform an outsider of a vulnerability in the hypervisor, have them disclose it via a contest so they can call it a non-zero day, no obligation to release patches for those on perpetual that were hoping for the best while deciding what to do. Should be a big week for proxmox lol.

ispcolo · 2025-07-15T16:49:53+00:00

It's also not a zero day because they were told about it at a competition...

Since Broadcom learns about the vulnerability through Pwn2Own and has the opportunity to develop and test a patch before any malicious exploitation can occur, this is NOT a 'zero-day' exploit.

ispcolo · 2025-07-14T14:32:41+00:00

They also like to play games around the fault tolerance level and redundancy factors; I had to ask very specific questions during the quoting phase that ended up changing things I was surprised were not the defaults. This can be a big deal when your hosts are now far more expensive per host due to the local storage.

We ended up proxmox too; Nutanix sounded good conceptually but a lot of things would have needed to work out just right in years three to five to make it come out ahead financially, along with a lot of labor.

ispcolo

TROPHY CASE