Hey guys, I'm trying to understand what is happening in my cluster, I see the number of degraded objects increasing, while all OSD's are still up and running. Can someone explain what's happening? I would expect the number of misplaced objects to increase when ceph's balancing algorithm decides blocks should be on different OSDs, but I would only expect the degraded objects to increase when OSD's die?? 6/15/23 11:31:25 AM[WRN]Health check update: Degraded data redundancy: 809035/219783315 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:31:20 AM[WRN]Health check update: Degraded data redundancy: 809044/219788544 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:31:15 AM[WRN]Health check update: Degraded data redundancy: 809044/219788616 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:31:10 AM[WRN]Health check update: Degraded data redundancy: 808944/219777540 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:31:05 AM[WRN]Health check update: Degraded data redundancy: 808944/219776271 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:31:00 AM[WRN]Health check update: Degraded data redundancy: 808821/219683475 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:30:55 AM[WRN]Health check update: Degraded data redundancy: 808740/219672240 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) 6/15/23 11:30:50 AM[WRN]Health check update: Degraded data redundancy: 808667/219645417 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized (PG_DEGRADED) health: health: HEALTH_WARN Degraded data redundancy: 810779/220602543 objects degraded (0.368%), 6 pgs degraded, 6 pgs undersized services: mon: 3 daemons, quorum ceph-mon01,ceph-mon02,ceph-mon03 (age 12h) mgr: ceph-mon01.vzbglj(active, since 12h), standbys: ceph-mon02.qtuntk mds: 1/1 daemons up, 2 standby osd: 118 osds: 118 up (since 12h), 118 in (since 23h); 510 remapped pgs data: volumes: 1/1 healthy pools: 5 pools, 1249 pgs objects: 44.55M objects, 60 TiB usage: 118 TiB used, 2.0 PiB / 2.1 PiB avail pgs: 810779/220602543 objects degraded (0.368%) 39586388/220602543 objects misplaced (17.945%) 739 active+clean 406 active+remapped+backfill_wait 98 active+remapped+backfilling 6 active+undersized+degraded+remapped+backfilling io: client: 585 KiB/s rd, 22 MiB/s wr, 357 op/s rd, 1.41k op/s wr recovery: 150 MiB/s, 82 objects/s Kind regards, Angelo Hongens