Hi Boris, thank you for your answer. What i really did not expect is, that purging an OSD leads to yet another rebalancing, after already having it destroyed(!), after it was taken out and after the rebalancing seemed to have been completed. So with the next host - i have some more to remove - i'll do as you suggest. Thanks again, cheers, toBias From: "Boris" <bb@kervyn.de> To: "Tobias Tempel" <tobias.tempel@desy.de> Cc: "ceph-users" <ceph-users@ceph.io> Sent: Thursday, 4 April, 2024 11:02:09 Subject: Re: [ceph-users] purging already destroyed OSD leads to degraded and misplaced objects? Hi Tobias, what we usually do, when we want to remove an OSD is to reweight the crush map to 0. This stops the rebalancing after removing the OSD from the crush map. Setting an OSD to out, keeps it weighted in the crush map and when it gets removed, the cluster will rebalance the PGs to reflect the new crush map. Am Do., 4. Apr. 2024 um 10:54 Uhr schrieb tobias tempel <tobias.tempel@desy.de>:
Dear Cephers,
reorganizing one of our clusters i'm removing some hosts from it, taking "out" all OSDs on these hosts and waiting until all PGs are fine. After stopping and destroying all OSDs on one host i notice, that "purge" of such destroyed OSDs temporarily leads to degraded and misplaced objects (tried "reweight 0" as well, same picture) . Why that? I would - due to my limited experience - expect, that completely removing an already destroyed OSD could not have any effect on object placement at all.
This effect yet seems not to be a real problem, as recovery works fine ... but perhaps it indicates some other issue. All of that takes place at Pacific 16.2.14 AlmaLinux 8.9, yes i know, there's some work to do.
Perhaps you can give me a hint, where to look, for an explanation?
Thank you cheers, toBias
-- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groüen Saal. -- Tobias Tempel Deutsches Elektronen-Synchrotron - IT Notkestr. 85, 22607 Hamburg, Germany email: tobias.tempel@desy.de