I think that would only happen if pg_num for a 3R pool were less than like ~ 1/3 the number of OSDs. Assuming aligned device classes, proper CRUSH rules and topology etc. Mind you if pg_num is low, the balancer won’t be able to do a great job of distributing data uniformly. If you set pg_num to a non power of two that complicates things as well since you’d have PGs of very different sizes, but this is something rarely seen. Sharing `ceph osd tree` and `ceph osd dump | grep pool` and `ceph osd df` would help.
On Feb 13, 2025, at 1:11 PM, Work Ceph <work.ceph.user.mailing@gmail.com> wrote:
Exactly, that is what I am assuming. However, my question is. Can I assume that the PG number will affect the Max available space that a pool will be able to use?
On Thu, Feb 13, 2025 at 3:09 PM Anthony D'Atri <anthony.datri@gmail.com <mailto:anthony.datri@gmail.com>> wrote:
Assuming that the pool is replicated, 512 PGs is pretty low if this is the only substantial pool on the cluster. When you do `ceph osd df`, if this is the only substantial pool, the PGS column at right would average around 12 or 13 which is suuuuper low.
On Feb 13, 2025, at 11:40 AM, Work Ceph <work.ceph.user.mailing@gmail.com <mailto:work.ceph.user.mailing@gmail.com>> wrote:
Yes, the bucket that represents the new host is under the ROOT bucket as the others. Also, the OSDs are in the right/expected bucket.
I am guessing that the problem is the number of PGs. I have 120 OSDs across all hosts, and I guess that 512 PGS, which is what the pool is using, is not enough. I did not change it yet, because I wanted to understand the effect on PG number in Ceph pool usable volume.
On Thu, Feb 13, 2025 at 12:03 PM Anthony D'Atri <anthony.datri@gmail.com <mailto:anthony.datri@gmail.com>> wrote:
Does the new host show up under the proper CRUSH bucket? Do its OSDs? Send `ceph osd tree` please.
> Hello guys, > Let's say I have a cluster with 4 nodes with 24 SSDs each, and a single > pool that consumes all OSDs of all nodes. After adding another host, I > noticed that no extra space was added. Can this be a result of the > number > of PGs I am using? > > I mean, when adding more hosts/OSDs, should I always consider increasing > the number of PGs from a pool? >
ceph osd tree
shows all up and with correct weight?
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>