Hi Herve, Why you don't use the automatic adjustment of the number of PGs. This makes life much easier and works well. Cheers, Michel Le 03/10/2023 à 17:06, Hervé Ballans a écrit :
Hi all,
Sorry for the reminder, but does anyone have any advice on how to deal with this?
Many thanks! Hervé
Le 29/09/2023 à 11:34, Hervé Ballans a écrit :
Hi all,
I have a Ceph cluster on Quincy (17.2.6), with 3 pools (1 rbd and 1 CephFS volume), each configured with 3 replicas.
$ sudo ceph osd pool ls detail pool 7 'cephfs_data_home' replicated size 3 min_size 2 crush_rule 1 object_hash rjenkins pg_num 512 pgp_num 512 autoscale_mode on last_change 6287147 lfor 0/5364613/5364611 flags hashpspool stripe_width 0 application cephfs pool 8 'cephfs_metadata_home' replicated size 3 min_size 2 crush_rule 3 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 6333341 lfor 0/6333341/6333339 flags hashpspool stripe_width 0 application cephfs pool 9 'rbd_backup_vms' replicated size 3 min_size 2 crush_rule 2 object_hash rjenkins pg_num 1024 pgp_num 1024 autoscale_mode on last_change 6365131 lfor 0/211948/249421 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd pool 10 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode warn last_change 6365131 flags hashpspool stripe_width 0 pg_num_min 1 application mgr,mgr_devicehealth
$ sudo ceph df --- RAW STORAGE --- CLASS SIZE AVAIL USED RAW USED %RAW USED hdd 306 TiB 186 TiB 119 TiB 119 TiB 39.00 nvme 4.4 TiB 4.3 TiB 118 GiB 118 GiB 2.63 TOTAL 310 TiB 191 TiB 119 TiB 119 TiB 38.49
--- POOLS --- POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL cephfs_data_home 7 512 12 TiB 28.86M 12 TiB 12.85 27 TiB cephfs_metadata_home 8 32 33 GiB 3.63M 33 GiB 0.79 1.3 TiB rbd_backup_vms 9 1024 24 TiB 6.42M 24 TiB 58.65 5.6 TiB .mgr 10 1 35 MiB 9 35 MiB 0 12 TiB
I am going to extend the rbd pool (rbd_backup_vms), currently used at 60%. This pool contains 60 disks, i.e. 20 disks by rack in the crushmap. This pool is used for storing VM disk images (available to a separate ProxmoxVE cluster)
For this purpose, I am going to add 42 disks of the same size as those currently in the pool, i.e. 14 additional disks on each rack.
Currently, this pool is configured with 1024 pgs. Before this operation, I would like to extend the number of pgs, let's say 2048 (i.e. double).
I wonder about the overall impact of this change on the cluster. I guess that the heavy moves in the pgs will have a strong impact regarding the iops?
I have two questions:
1) Is it useful to make this modification before adding the new OSDs? (I'm afraid of warnings about full or nearfull pgs if not)
2) are there any configuration recommendations in order to minimize these anticipated impacts?
Thank you!
Cheers, Hervé _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io