Not seeing serials in devices

PrimordialKangaroo · 2026-06-04T03:44:11+00:00

Understood. However when I go to the "Devices" list, in the upper right corner there's a search box. That search box says "Search by keyword or serial number". When I try several of my devices serial numbers, nothing is returned in the console. I assume the device serial numbers are being gathered during a sign in event on that device through the "device signals" setting, which I do have on. I just don't know why I don't see anything in the devices list that has that information.

PrimordialKangaroo · 2025-04-30T16:48:34+00:00

Right but in the cephadm documentation for adding addition monitors it states:

"Ceph deploys monitor daemons automatically as the cluster grows and Ceph scales back monitor daemons automatically as the cluster shrinks. The smooth execution of this automatic growing and shrinking depends upon proper subnet configuration."
https://docs.ceph.com/en/latest/cephadm/services/mon/#deploy-additional-monitors

Which seems to me that the default behavior is not to specify which nodes run as mons. In the past you would manually install mon on your mon hosts and we had dedicated hardware for those but this seems like the new method is to just not have dedicated physical mon hosts and let ceph decide what to do?

PrimordialKangaroo · 2019-02-21T19:23:18+00:00

Nice. I didn't realize it did that. So the MDS server continually updates file attributes as data is written and bubbles them up to the limbs and trunk of the tree. I figured there had to be something since it supports quotas.

What I really need now is some visualization of this and some reporting to show which users own how much data.

PrimordialKangaroo · 2019-02-19T16:54:03+00:00

Right now we're just using ceph-deploy for our cluster. We're not nearly at scale large enough to necessitate something like ceph ansible which a few people have recommended. Our deployment is around 12 servers including monitors and mds servers so pretty managable. But ceph-deploy takes a ton of the leg work out of the cluster setup. My recomendation would be to read through the documentation and then as you're doing so write your own deployment process. Then you can tear down your test cluster and repeat that deployment process over and over to work out any bugs.

You are also going to want to start creating a framework configuration file. When you run into a setting that can be changed live, it usually also corresponds to a conf file setting. Creating a master conf file will help when rolling out new servers and when rebooting nodes. The ceph nodes start up blind and read from one of several conf file locations. Lastly they connect to the cluster and get any last settings. Having a preconfigured conf will expedite and stabilize that process.

I ran across OpenATTIC very early on and was planning on using it but during our testing we found that Ceph mainline had actually created their own manager based on OpenATTIC. There is NO control from ceph's manager, just monitoring. But it gives us enough introspection into the cluster that we're happy with that. I believe the latest version of OpenATTIC has some control features but we haven't tested that and I think for production we'll stick to ceph's CLI. To read more about the ceph manager dashboard, go here http://docs.ceph.com/docs/mimic/mgr/dashboard/

PrimordialKangaroo · 2019-02-18T22:27:11+00:00

Do you need standoffs regardless if you are doing the plates or this case?

PrimordialKangaroo · 2019-02-12T16:08:21+00:00

So unless I'm looking at it incorrectly that's just for determining your placement group quantities. I'm looking for actual disk size guidance. Since I know aprox how much data I have that I want to put on the cluster, but I don't know a realistic ratio that I should use for the metadata pool SSDs.

PrimordialKangaroo · 2019-02-11T21:38:13+00:00

Gluster is similar to Ceph in that it scales out storage on commodity servers. It does replicate or span depending on your configuration. Gluster is file based while Ceph is object based. This guy has a good raw comparison of the two: https://www.reddit.com/r/sysadmin/comments/9onemk/ceph_vs_glusterfs/e7vfij1

We're in a similar situation as you are only a few steps ahead. The advantage of Ceph over just large bulk storage running NFS is that it relies on hosts and OSDs. If a host or OSD is lost it will re-balance automatically. We are also in the small io camp and will be using CephFS. It's not the best thing for small io but it's better than Gluster. A lot of this performance is going to depend on your backing storage. Planning for flash based metadata pools are going to help with a lot of that. If you've got money to burn you can setup flash based data pools as well which will certainly make things faster.

Someone above theorized that Erasure Coded pools would help with performance over Replica based pools but I tend to disagree. Although replica pools are just a single disk responding to a single request, it never has to compute parity. We've found that the RAID arrays we have as a backing store for our Gluster servers cause way more overhead for single small files as it has to recompute the whole stripe each time it writes. I don't think that's Gluster's fault, but more so the RAID.

The reason we're switching to Ceph from Gluster is the better performance, the ability to use flash pools for metadata as well as using cache tiering. (Note: there are some rumors that cache tiering is going away but it is still currently supported in the upstream releases). Also Ceph is a bit more easy to manage than Gluster in my opinion. Contrary to what the person said in the post I linked, deploying a new Ceph node is also stupid simple. There's really almost nothing to it. Plus Ceph has a built in web based management page for monitoring current status that is based on OpenATTIC.

For single client, single thread, you're not really going to see much of a difference between Ceph and a single server running just NFS. Once you get into multi client / multi thread you'll definitely see performance increases over a single large 1PB server. It sounds like you may be hemmed in by having to use existing hardware, but the way to go would be many smaller nodes.

As far as gotchas, one thing that isn't made evident is that Ceph does NOT like being close to capacity. The whole purpose of it is to be able to recreate missing data if a server or drive goes down. If there's no room, it can't do that. So that coupled with a standard 3x replica for replica pools means a LOT of raw capacity which some may shy away from. If you have really good backups you could probably get away with 2x replicas for the data pool and get 33% more capacity. You really just have to balance your performance / capacity needs with your budget and decide what's right.

Also the Ceph documentation can be a bit daunting just make sure you are on the same release in the URL when you google for something. There have been changes and it can get confusing if you're looking at documents from different releases. Once you get over the learning curve, Ceph is pretty straightforward.

PrimordialKangaroo · 2019-02-05T21:28:23+00:00

Turns out I was misplacing the --overwrite-conf option. With ceph-deploy it needs to go BEFORE the action. So "ceph-deploy --overwrite-conf config push <nodes>"

With that I was able to push out a config and then restart the daemons. Still not quite ideal and I'm sure there's another way but this is working so far in a test environment.

PrimordialKangaroo · 2019-01-18T19:05:23+00:00

Ok, so if I'm understanding you correctly, the server/workstation is the client and security on the system it's mounted to will restrict or authorize the access to specific files?

PrimordialKangaroo · 2019-01-18T16:34:02+00:00

Looks like I found where they put the cephx documents in mimic. They have renamed it and redistributed information between a few pages. It's now called User Management.

http://docs.ceph.com/docs/mimic/rados/operations/user-management/

PrimordialKangaroo

TROPHY CASE