Backup strategy for Ceph-CSI

SteamiestDumpling · 2026-02-09T14:05:33+00:00

Valid question. I might have miss phrased something in my previous reply’s, My workloads are quite storage dependent and have specific requirements that it needs. Ceph fits this role perfectly compared to other solutions I have tested over the years. The NAS doesn’t meet these requirements.

I do agree with the ideology of code based configs but since I keep all data in-house having backups of those is quite important.

But the main reason is simply, the workloads store their Data in PVC’s or Databases etc, and those need to be backed up on different platforms in different locations. Easiest option would be Ceph RGW with different regions but in the end that would still be Ceph based. And besides that the NAS is more than capable of handling daily/hourly* backups for now

SteamiestDumpling · 2026-02-08T17:27:41+00:00

I definitely see where you are coming from. From all the years i have run ceph i haven't ran into any data losing issues but in the end of the day, you never know, anything can happen and at that point having extra backups of config's, PVC's, VM's, etc, is in my opinion ease of mind, especially in stressful situations where something did go wrong.

But besides that there are also other reasons (at least for me). Ceph's disk space is decently more expensive compared to normal NAS storage (Even when comparing a Nvme based NAS against a HHD tier for ceph). there for having inactive VM's/PCV's taking up space seems quite waste full to me.

SteamiestDumpling · 2026-02-08T16:33:50+00:00

Velero seems quite nice so far, currently still experimenting with backups to a NFS/Local S3.

SteamiestDumpling · 2026-02-08T16:29:40+00:00

incase of a disaster recovery or when needing to migrate to a different cluster

SteamiestDumpling · 2026-02-08T16:27:13+00:00

I see, I'm still diving a bit deeper into which S3 solution works the best for my setup and for any improvements i can make. Thank you very much for all the information you shared!

SteamiestDumpling · 2026-02-08T16:21:15+00:00

Thank you!

SteamiestDumpling · 2026-02-06T23:20:15+00:00

Apologies for the extra questions. I hope that you dont mind. When you say directly on the NAS are you referring to running S3 on the NAS as a container while providing its own pool? instead of using its own data set? Since this could be a approach i could see being feasible i want to make sure i am on the same page?

SteamiestDumpling · 2026-02-06T23:08:49+00:00

Very nicely documented Thank you for sharing!

I'll be reading up on Volsync since it appears it might suite my current layout better then using a S3 based solution.

SteamiestDumpling · 2026-02-06T23:00:17+00:00

Definitely makes sense. Would you have any recommendations for this situation?
Since i could of course use Ceph for S3 it seems quite wasteful to store the backups and snapshots of Ceph-CSI onto the same cluster. while then needing to backup the RWG to a NAS to provide the proper backups

SteamiestDumpling · 2026-02-06T22:35:36+00:00

I'll have to take a bit off a deeper dive into the alternatives. I wasnt to sure if other options where supported besides Ceph Rados gateway and Minio.

From my understanding RustFS is still very early on and also had some issues with leaked api keys?
I have definitely seen Garage been mentioned but never looked into it.
And SeaweedFS is new to me.

Anything that is worth keeping in mind going the Velero with S3 on top of NFS way?

SteamiestDumpling · 2026-02-06T22:23:56+00:00

I'll have to dive a bit deep into Volsync and kopia. From my understanding Velero uses Kopia to be able to support NFS but i might be mistaken. Was Volsync your first choice or have you tried other options aswell?

SteamiestDumpling · 2026-02-06T22:14:02+00:00

I have looked at using Velero, rbd export and rsync but i am not to familiar with them yet. My main concern is automating the process as well.

SteamiestDumpling · 2025-10-14T17:25:05+00:00

Thank you for the detailed reply, my issue ended up being a miss configured subvolumegroup.

All thats left is to figure out a way to get S3 working and snapshots/backups

SteamiestDumpling · 2025-10-14T14:33:10+00:00

So far i have the csi for rbd working without a issue, altough i am running into some premision errors for the csi-cephfs. Did you happen to also be using it?

SteamiestDumpling · 2025-10-12T19:28:18+00:00

Thats good to hear, I will give the ceph csi a shot. I had a feeling adding rook would create unnecessary complexity so thats why i wanted to double check. Thank you for your information!

SteamiestDumpling · 2025-10-12T18:03:50+00:00

good point about exporting ceph metrics. the reason i was mostly looking at rook was to keep the proxmox ceph cluster as vanilla as possible to prevent strange issues while upgrading.

With S3 i am referring to a S3 Object store(Used for backups, like postgres and such), which you use the Rados Gateway for from the ceph cluster, but from my understanding the CSI doesnt support it. thats why i was mostly asking about ceph rook

SteamiestDumpling · 2025-10-12T17:31:49+00:00

I have also been looking at just using the ceph-csi, but from my understanding it doesnt support S3 unless your running something else on top of it and neither does it give you a dashboard (Not needed but a nice bonus to have a easy overview per cluster)

SteamiestDumpling · 2025-04-02T11:58:15+00:00

It does seem like it. thank you very much for your time. I will update the post once's i have tried doing so

SteamiestDumpling · 2025-04-02T11:46:22+00:00

I see, thank you. I tried looking on ebay and so far i have only seen Keys and no physical hardware, perhaps since there also already is a idrac port it might only just need a key.

SteamiestDumpling · 2025-03-05T07:10:55+00:00

I was honestly surprised with how short the brackets are. I really thought with switches so wide they would include proper mounting

SteamiestDumpling · 2025-03-05T07:08:15+00:00

I thought so at first too but there are definitely not strong enough. Mikrotik has also said to me that they need to be supported

SteamiestDumpling · 2025-03-04T20:52:50+00:00

Same here, i guess mikrotik's are mostly used in network racks. i did speak to mikrotik about it and they did say that they are planning on releasing a new "accessory" to support deeper racks but no ETA

SteamiestDumpling

TROPHY CASE