Den tors 25 jan. 2024 kl 17:47 skrev Robert Sander <r.sander@heinlein-support.de>:
forth), so this is why "ceph df" will tell you a pool has X free space, where X is "smallest free space on the OSDs on which this pool lies, times the number of OSDs". Given the pseudorandom placement of objects to PGs, there is nothing to prevent you from having the worst luck ever and all the objects you create end up on the OSD with least free space.
This is why you need a decent amount of PGs, to not run into statistical edge cases.
Yes, just take the experiment to someone with one PG only, then it can only fill one OSD. Someone with a pool with only 2 PGs could at the very best case only fill two and so on. If you have 100+ PGs per OSD, the chances for many files to end up only on a few PGs becomes very small. -- May the most significant bit of your life be positive.