Dear all I (try to) manage a ceph cluster which is used for different use cases: cephfs, block storage for OpenStack and for a proxmox cluster, object storage. I have a hdd-cephfs device class used only for the OSDs for the cephfs data pool (cephfs_data). There has been a massive writing to the cephfs file system and so the pool and one OSD of that device class became 'nearfull' To try to address the problem I then moved 3 osds to the hdd-cephfs device class. During the rebalance: - there was a short down of one node hosting one of the OSD belonging to the hdd-cephfs device class - many files in the cephfs file system got deleted Now I have an on going backfill that doesn't seem to converge: pgs: 8824966/525185097 objects misplaced (1.680%) 2580 active+clean 17 active+remapped+backfill_wait 13 active+clean+scrubbing+deep 13 active+clean+scrubbing 2 active+remapped+backfilling 1 active+remapped+backfill_wait+backfill_toofull and one pg is is in active+remapped+backfill_wait+backfill_toofull I am also told that the pool needs more pgs: [root@ceph-mon-01 ~]# ceph health detail HEALTH_WARN Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; 1 pools have too few placement group\ s [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull pg 60.4 is active+remapped+backfill_wait+backfill_toofull, acting [10,189,110] [WRN] POOL_TOO_FEW_PGS: 1 pools have too few placement groups Pool cephfs_data has 32 placement groups, should have 128 These are the details of the pool: [root@ceph-mon-01 ~]# ceph df | grep 'cephfs_data ' cephfs_data 60 32 16 TiB 9.48M 48 TiB 47.39 18 TiB [root@ceph-mon-01 ~]# pool 60 'cephfs_data' replicated size 3 min_size 2 crush_rule 11 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode warn last_change 4259332\ flags hashpspool stripe_width 0 application cephfs read_balance_score 1.88 and this [*] is the status of the OSDs belonging to that device class. As I said, the backfilling has been running for several days and doesn't seem to get finalized (the number of misplaced objects even increases from time to time, maybe because of the balancer ?) Should I simply be patient and wait for the end of the backfilling ? What about the pg in active+remapped+backfill_wait+backfill_toofull ? Should it "fix" by itself ? Or should I increase now the number of pgs of that pool ? Any suggestion will be highly appreciated ! Thanks, Massimo [*] [root@ceph-mon-01 ~]# ceph osd df | grep cephfs 70 hdd-cephfs 9.18259 1.00000 9.2 TiB 1.8 TiB 1.7 TiB 92 KiB 2.6 GiB 7.4 TiB 19.08 0.59 3 up 110 hdd-cephfs 11.00099 1.00000 11 TiB 6.5 TiB 6.4 TiB 114 KiB 9.8 GiB 4.5 TiB 59.36 1.85 13 up 130 hdd-cephfs 14.63959 1.00000 15 TiB 7.0 TiB 6.9 TiB 11 MiB 10 GiB 7.6 TiB 47.98 1.49 14 up 0 hdd-cephfs 18.26270 1.00000 18 TiB 5.9 TiB 5.8 TiB 10 MiB 9.8 GiB 12 TiB 32.42 1.01 12 up 189 hdd-cephfs 21.91499 1.00000 22 TiB 6.0 TiB 5.9 TiB 10 MiB 10 GiB 16 TiB 27.31 0.85 12 up 10 hdd-cephfs 14.72659 1.00000 15 TiB 6.0 TiB 5.9 TiB 10 MiB 9.9 GiB 8.7 TiB 40.93 1.27 12 up 25 hdd-cephfs 14.72659 1.00000 15 TiB 6.1 TiB 5.9 TiB 10 MiB 10 GiB 8.6 TiB 41.56 1.29 12 up 26 hdd-cephfs 14.72659 1.00000 15 TiB 1.3 TiB 1.1 TiB 10 MiB 2.5 GiB 13 TiB 8.60 0.27 2 up 170 hdd-cephfs 14.72659 1.00000 15 TiB 8.0 TiB 7.8 TiB 10 MiB 13 GiB 6.8 TiB 54.10 1.68 16 up 171 hdd-cephfs 14.72659 1.00000 15 TiB 368 GiB 190 GiB 11 MiB 1.3 GiB 14 TiB 2.44 0.08 0 up [root@ceph-mon-01 ~]#