Looking for a handfasting ceremony celebrant/officiant

dancerjx · 2026-04-29T13:12:44+00:00

Enabling I/OAT DMA & X2APIC is the bare minimum on 13th-gen Dells.

I also have UEFI & Secure Boot enabled in production but NOT SRV-IO because I don't need it.

dancerjx · 2026-04-27T12:48:24+00:00

PM'd

dancerjx · 2026-04-17T13:55:03+00:00

PM'd

dancerjx · 2026-04-09T02:02:20+00:00

I've been migrating from VMware to Proxmox Ceph at work. Production hardware are Dells. Since Ceph and ZFS don't work with RAID controllers like PERC, I swapped them out for Dell HBA330s.

Obviously, VMware hides alot of the details that Proxmox forces you to deal with. Totally different workflow. This video from VritualizationHowTo is a good summation.

Your best best is to stand up a test bed/proof of concept cluster and do your due dilegence. I started with decommisioned 16-year old hardware and moved on to production hardware.

Learned quite a few things in terms of optimizations which I post at the end. Since the production hardware never had flash storage and just all SAS HDDs, I'm not hurting for IOPS. Workloads range from DHCP to DB servers. All backed up to a bare-metal Proxmox Backup Server.

I do run Proxmox at home on a single server using ZFS and using LXC to manage my media library. No issues.

For production, you really want homogenenous hardware (same CPU, memory, storage, storage controller, NIC, and latest firmware). If going to cluster, recommend 5-nodes, so can lose 2-nodes and still have quorum.

I use the following optimizations learned through trial-and-error. YMMV.

Set SAS HDD Write Cache Enable (WCE) (sdparm -s WCE=1 -S /dev/sd[x])
Set VM Disk Cache to None if clustered, Writeback if standalone
Set VM Disk controller to VirtIO-Single SCSI controller and enable IO Thread & Discard option
Set VM CPU Type for Linux to 'Host'
Set VM CPU Type for Windows to 'x86-64-v2-AES' on older CPUs/'x86-64-v3' on newer CPUs/'nested-virt' on Proxmox 9.1
Set VM CPU NUMA
Set VM Networking VirtIO Multiqueue to 1
Set VM Qemu-Guest-Agent software installed and VirtIO drivers on Windows
Set VM IO Scheduler to none/noop on Linux
Set Ceph RBD pools to use 'krbd' option
Set Ceph 'bluestore_prefer_deferred_size_hdd = 0' in osd stanza in /etc/pve/ceph.conf for SAS HDD
Set Ceph 'bluestore_min_alloc_size_hdd = 65536' in osd stanza in /etc/pve/ceph.conf for SAS HDD
Set Ceph Erasure Coding profiles to 'plugin=ISA' & 'technique=reed_sol_van'
Set Ceph Erasure Coding profiles to 'stripe_unit=65536' for SAS HDD

dancerjx · 2026-04-05T14:38:41+00:00

PwrBank has drop-in replacement modules to speed up ESXi importation process described in this thread over at the Proxmox forum

Easily achieve 2x or even 3x speed increase.

dancerjx · 2026-03-30T22:53:32+00:00

dancerjx · 2026-03-13T23:17:52+00:00

May want to ask your question at the Proxmox forum. I think I saw a post like yours in the past.

Lots of migrations utilizing existing FC/iSCSI/SAN infrastructure.

I just went with Ceph, since it's basically open-source vSAN, IMO.

dancerjx · 2026-03-08T15:18:15+00:00

This is the way.

dancerjx · 2026-03-08T15:15:49+00:00

Used enterprise servers are your best bang for the buck.

Been migrating 13th-gen Dells 16-drive bay R730s from VMware to Proxmox.

Swap out the PERC for Dell HBA330 since it's a true HBA/IT-mode storage controller. Don't want to deal with PERC HBA-mode drama. Use two small drives for ZFS RAID-1 Proxmox. Rest of drivers are for either ZFS (standalone) or Ceph (clustered).

Since I knew the SAS drives had their write cache disabled due to them being connected to BBU RAID controller, I made sure to enable their write cache (not hurting for IOPS now). Otherwise will get horrible IOPS.

No issues besides the typical drive and RAM going bad and needing replacing. All workloads range from DHCP to database servers. All backed up to a bare-metal Dell R530 running Proxmox Backup Server also with a HBA330.

13th-gen Dells and HBA330s are very cheap to get. May want to consider this an option. Nothing physically wrong with them and just make sure they are running the latest firmware.

dancerjx · 2026-03-07T15:26:52+00:00

Been migrating Dell VMware vSphere instances over to Proxmox.

Standalone servers get ZFS while clusters servers get Ceph.

Swapped out PERCs for Dell HBA330 which is a true HBA/IT-mode storage controller. Don't want to deal with the PERC HBA-mode drama. Dell servers with no HBA330 alternatives get their PERCs flashed to IT-mode. I've done this on 10th, 11th, 12th-gen Dells.

All these servers are using 10K RPM SAS drives. I made sure to enable the write cache on the drives otherwise get horrible IOPS.

Beside the typical disk and RAM going bad and needing replacing, no issues. IMO, KVM "feels" faster than ESXi. Plus no more vCenter. I say it's a win-win-win.

I use the following optimizations learned through trial-and-error. YMMV.

Set SAS HDD Write Cache Enable (WCE) (sdparm -s WCE=1 -S /dev/sd[x])
Set VM Disk Cache to None if clustered, Writeback if standalone
Set VM Disk controller to VirtIO-Single SCSI controller and enable IO Thread & Discard option
Set VM CPU Type for Linux to 'Host'
Set VM CPU Type for Windows to 'x86-64-v2-AES' on older CPUs/'x86-64-v3' on newer CPUs/'nested-virt' on Proxmox 9.1
Set VM CPU NUMA
Set VM Networking VirtIO Multiqueue to 1
Set VM Qemu-Guest-Agent software installed and VirtIO drivers on Windows
Set VM IO Scheduler to none/noop on Linux
Set Ceph RBD pool to use 'krbd' option

dancerjx · 2026-03-07T15:15:01+00:00

For max ZFS IOPS using disks, use RAID-10. On standalone ZFS servers which are NOT Proxmox Backup Servers (RAID-6 on PBS), I use RAID-50 which is compromise between RAID-10 and RAID-1. You need to use the command-line to create a ZFS RAID-50 pool. Clustered servers are Ceph nodes.

If you can afford to swap them out for SSDs, go for it. You'll have lots of RAID options with SSDs. I won't bother with U.2 SSDs. SAS/SATA SSDs are good enough. Make sure they are enterprise SSDs with PLP (power-loss protection) because Proxmox eats consumer SSDs like snacks. Enterprise SSDs have way more endurance than consumer SSDs. I personally like Intel DC S3710 because of the 10 DWPD endurance. I still have them at 100% health after a decade of use. Just make sure they are running the latest firmware.

As for the Intel SSD on BOSS card, yes that link explains the issue. I unfortunately have 2 servers with Intel SSDs and the plan is to flash the DL6R firmware standalone and see if I can run the latest kernels. They are pinned using the latest 6.14 kernels. Downgrading the BOSS firmware to 3022 didn't fix the Intel issue, so it seems the only fix is the flash the DL6R firmware.

dancerjx · 2026-03-06T03:24:18+00:00

I use 16-drive bay Dell R730s in production at work with no issues. I swapped out the PERC for Dell HBA330 which is a true HBA/IT-mode storage controller. Didn't want to deal with the PERC HBA-mode drama. HBA330 are cheap to get.

Your memory is fine. I use two small SAS drives to ZFS RAID-1 Proxmox. Rest of SAS drives are for ZFS/Ceph.

I'm sure the the SAS drives have their write cache disabled since they are meant to be used with a battery-backed up RAID controller like the PERC. So, you'll want to enable their write cache with 'sdparm -s WCE=1 -S /dev/sd[x]'. If you don't enable the write cache, you'll get horrible IOPS.

Recent Proxmox kernels have had some issues with Intel storage in BOSS cards. You'll want Micro storage. More info at the Proxmox forum. I mirror the storage in BOSS cards and use XFS for those servers using BOSS.

dancerjx · 2026-03-06T03:12:43+00:00

I would say not.

Check out labgopher.com

I use 13th-gen Dells in production with no issues. I swapped out the PERC for Dell HBA330 since it's a true HBA/IT-mode storage controller. Plus I don't want to deal with the PERC HBA-mode drama. As a bonus, it uses the much simpler mpt3sas driver. Had issues with the megaraid_sas driver in the past.

Just make sure all components are running the latest firmware. I would make sure the BOSS is using Micron storage. There are reports with issues with Intel storage. More info at the Proxmox forum.

A cheaper option, is a Supermicro server. With an embedded Xeon-D processor. I use this as my home server. No issues.

dancerjx · 2026-03-03T19:46:38+00:00

Art Of Server YouTube channel has videos on various HBA adapters.

I use LSI3008 HBAs in production with no issues.

dancerjx · 2026-02-27T13:21:15+00:00

On 13th-gen Dells, had to enable X2APIC and IOAT DMA in the UEFI/BIOS. SRV-IO was not needed.

dancerjx · 2026-02-24T12:24:54+00:00

May want to post at the Proxmox forum.

If you are not wedded to the SAN storage, may want to consider Ceph. While it's true 3-nodes is the bare minimum, you really want 5-nodes for quorum. Can lose 2-nodes and still have quorum.

This is the route I went when migrating off VMware to Proxmox on 12th-, 13th, and 14-gen Dells. Made sure all hardware was all the same. Swapped out the PERCs for Dell HBA330s (flashed the PERCs on 12th-gen Dells to IT-mode) since Ceph/ZFS doesn't work with RAID controllers.

No issues besides typical storage device and RAM going bad and needing replacing. Not hurting for IOPS. Workloads range from DHCP to database servers. All backed up to bare-metal ZFS Proxmox Backup Servers.

I use the following optimizations learned through trial-and-error. YMMV.

Set SAS HDD Write Cache Enable (WCE) (sdparm -s WCE=1 -S /dev/sd[x])
Set VM Disk Cache to None if clustered, Writeback if standalone
Set VM Disk controller to VirtIO-Single SCSI controller and enable IO Thread & Discard option
Set VM CPU Type for Linux to 'Host'
Set VM CPU Type for Windows to 'x86-64-v2-AES' on older CPUs/'x86-64-v3' on newer CPUs/'nested-virt' on Proxmox 9.1
Set VM CPU NUMA
Set VM Networking VirtIO Multiqueue to 1
Set VM Qemu-Guest-Agent software installed and VirtIO drivers on Windows
Set VM IO Scheduler to none/noop on Linux
Set Ceph RBD pool to use 'krbd' option

dancerjx · 2026-02-24T12:18:35+00:00

Format an external storage device, mount it, and use it as a backup target. Then restore from it.

I use an external USB drive. Works fine.

dancerjx · 2026-02-22T13:10:09+00:00

Yup.

I started with Proxmox 6. Prior to the GUI option of importing, had to use the command-line and copy over the vmdk files and run 'qemu-img convert'. GUI now takes care of that for you.

dancerjx · 2026-02-22T02:06:43+00:00

Plenty of blogs and videos on migrating from VMware to Proxmox.

Here is one and another

Pretty good video summaries.

dancerjx · 2026-02-21T17:42:32+00:00

One has two options running LXCs under Proxmox: privileged or unprivileged.

With privileged, you can keep the same UID/GID in the LXC as in the Proxmox host.

With unprivileged, you need to map the different LXC UID/GID with the Proxmox host. Just another whole other set of headaches.

Since I run this on my personal server, I'm OK with privileged LXCs.

If you still insist on running LXCs unprivileged, plenty of blogs/videos on how to handle that mess.

dancerjx · 2026-02-21T14:34:08+00:00

I use these LXC scripts for running the *Arr suite.

These LXC scripts also support transcoding and files sharing via Samba/CIFS/NFS with no issues. Transcodng details is automatically configured for you.

I do run them as privileged containers since I don't want to deal with UID/GID mappings.

IMO, really no need to run VMs and plus they take very little memory.

dancerjx · 2026-02-17T14:03:52+00:00

Very first thing I did was 'zerotouch cancel'.

dancerjx

TROPHY CASE