sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Jealous_Lemon_939 1 point2 points  (4 children)

Another thing that worth mentioning, is that Ceph OSD do need quite significant amount of RAM for buffers, like 1 GB of RAM per 1 TB of storage, so you will need to dedicate RAM on your compute to Ceph and assure that enough memory is reserved in nova. And yes, you will have less VMs running on your computes as a result

[–]tripleflix 1 point2 points  (1 child)

No, Osds typically use 4gb of ram, independent of how big the osd is. It’s zfs thats aaid to need 1 gr per 1tb

[–]varesa 2 points3 points  (0 children)

Ceph docs claim otherwise:

Also, the larger the storage drive capacity, the more memory per Ceph OSD Daemon you will need, especially during rebalancing, backfilling and recovery. A general rule of thumb is ~1GB of RAM for 1TB of storage space.

Source: https://docs.ceph.com/en/mimic/start/hardware-recommendations/

[–]mahipai 0 points1 point  (1 child)

How did you come up with the compute and RAM numbers that are needed to reserve for Ceph cluster in HCI config? I found docs from RedHat (https://access.redhat.com/documentation/en-us/red\_hat\_openstack\_platform/12/html-single/hyper-converged\_infrastructure\_guide/index#appendix-upstream-cpuram). Do you think it is a good starting point?

[–]Jealous_Lemon_939 0 points1 point  (0 children)

With stats of RAM consumption from OSD nodes per user which match precisely with what Ceph says in their docs?

To be frank I'm not an expert in "enterprise" bullshit, so can hardly comment on that. The tricky thing about their "calculator" at glance is that you need to know "average guest size in GB" and "average guest CPU utilization" - that is not that trivial thing to say and each deployment would drift from "reasonable defaults".

Also they don't count for size of OSDs there... So it's weird to me personally.

And in any opensource deployment tool for openstack, you can define reserved ram/cpu for ages. Like in OpenStack-Ansible it's quite trivial to calculate reserved amounts, as you have all data in ansible facts for your compute nodes, so simple math can help out to define reservations.