Hi, we have a cluster with this ceph df --- RAW STORAGE --- CLASS SIZE AVAIL USED RAW USED %RAW USED hdd 240 GiB 205 GiB 29 GiB 35 GiB 14.43 hddvm 1.6 TiB 1.2 TiB 277 GiB 332 GiB 20.73 TOTAL 1.8 TiB 1.4 TiB 305 GiB 366 GiB 19.91 --- POOLS --- POOL ID PGS STORED (DATA) (OMAP) OBJECTS USED (DATA) (OMAP) %USED MAX AVAIL QUOTA OBJECTS QUOTA BYTES DIRTY USED COMPR UNDER COMPR device_health_metrics 1 1 0 B 0 B 0 B 0 0 B 0 B 0 B 0 308 GiB N/A N/A 0 0 B 0 B rbd-pool 2 32 539 B 19 B 520 B 9 539 B 19 B 520 B 0 462 GiB N/A N/A 9 0 B 0 B cephfs.sharedfs.meta 3 32 299 MiB 190 MiB 109 MiB 87.10k 299 MiB 190 MiB 109 MiB 0.03 308 GiB N/A N/A 87.10k 0 B 0 B cephfs.sharedfs.data 4 32 2.2 GiB 2.2 GiB 0 B 121.56k 2.2 GiB 2.2 GiB 0 B 0.23 308 GiB N/A N/A 121.56k 0 B 0 B rbd-pool-proddeb02 5 32 712 MiB 712 MiB 568 B 201 712 MiB 712 MiB 568 B 0.08 308 GiB N/A N/A 201 0 B 0 B So as you can see we have 332GB RAW but data really are 299+2.2G+712M POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL device_health_metrics 1 1 0 B 0 0 B 0 308 GiB rbd-pool 2 32 539 B 9 539 B 0 462 GiB cephfs.sharedfs.meta 3 32 299 MiB 87.10k 299 MiB 0.03 308 GiB cephfs.sharedfs.data 4 32 2.2 GiB 121.56k 2.2 GiB 0.23 308 GiB rbd-pool-proddeb02 5 32 712 MiB 201 712 MiB 0.08 308 GiB How to clean Dirty ? How is that possible ? any cache issue or not committed flush from client ? Best regards Alessandro