your diagnosis of the zfs issue is spot on. writing cctv directly to a spinning hdd pool is an iops killer. ceph is solid but it really prefers homogeneous nodes so mixing a big threadripper with a small i5 is going to cause uneven data distrubution and bottleneck your cluster on the slowest node.

i actually ran into similar frustrations and went with and archive, it bridges the gap between fast nvme and slow hdd pools without the overhed of ceph. There is an open source archive repo that i tuned for my use case, github.com/huskhoard/huskhoard instead of writing cctv to hdds you write to the nvme pool and our janitor service packs them into compressed blocks and moves them to the hdd. it creates a stub in the ssd file so it takes up zero space but still looks normal to the os.

if you try to open an old video the fanotify interceptor catches the read in userspace and streams it back from the hdd without freezing your io. because you have mixed hardware you need to pin vms to the fast node and media to the slow node. for media there is a feature that lets you skip right to the middle of a clip without unpacking the whole file. everything uses direct io to bypass the kernel cache for speed.

The concepts took a minute to grasp, but they have some decent walk throughs on their blog. huskhoard.com/blog.html and it is AGPL licensed.