[ceph-users] Size and capacity calculations questions

Hi! We have a ceph cluster with 42 OSD in production as a server providing mainly home-directories of users. Ceph is 14.2.4 nautilus. We have 3 pools. One images (for rbd images) a cephfs_metadata and a cephfs_data pool. Our raw data is about 5.6T. All pools have replica size 3 and there are only very little snapshots in the rbd images pool, the cephfspool doesnt use snapshots. How is it possible that the status tells us, that 21T/46T is used, because thats much more than 3 times the raw size. Also, to make that more confusing, there as at least half of the cluster free, and we get pg backfill_toofull after we added some OSDs lately. The ceph dashboard tells aus the pool ist 82 % full and has only 4.5 T free. The autoscale module seems to calculate the 20T times 3 for the space needed and thus has wrong numbers (see below). Status of the cluster is added below too. how can these size/capacity numbers be explained? and, would be there a recommendation to change something? Thank you in advance! best Jochen # ceph -s cluster: id: 2b16167f-3f33-4580-a0e9-7a71978f403d health: HEALTH_ERR Degraded data redundancy (low space): 1 pg backfill_toofull 1 subtrees have overcommitted pool target_size_bytes 1 subtrees have overcommitted pool target_size_ratio 2 pools have too many placement groups services: mon: 4 daemons, quorum jade,assam,matcha,jasmine (age 2d) mgr: earl(active, since 24h), standbys: assam mds: cephfs:1 {0=assam=up:active} 1 up:standby osd: 42 osds: 42 up (since 106m), 42 in (since 115m); 30 remapped pgs data: pools: 3 pools, 2048 pgs objects: 29.80M objects, 5.6 TiB usage: 21 TiB used, 25 TiB / 46 TiB avail pgs: 1164396/89411013 objects misplaced (1.302%) 2018 active+clean 22 active+remapped+backfill_wait 7 active+remapped+backfilling 1 active+remapped+backfill_wait+backfill_toofull io: client: 1.7 KiB/s rd, 516 KiB/s wr, 0 op/s rd, 28 op/s wr recovery: 9.2 MiB/s, 41 objects/s # ceph osd pool autoscale-status POOL SIZE TARGET SIZE RATE RAW CAPACITY RATIO TARGET RATIO BIAS PG_NUM NEW PG_NUM AUTOSCALE images 354.2G 3.0 46100G 0.0231 1.0 1024 32 warn cephfs_metadata 13260M 3.0 595.7G 0.0652 1.0 512 8 warn cephfs_data 20802G 3.0 46100G 1.3537 1.0 512 warn