Hi, I don't have any suggestions yet, but your attachment wasn't uploaded, you'll have to use a pastebin or add the query output in this thread. Zitat von Michael Thomas <ceph-users@ceph.io>:
The background:
We are preparing to migrate our Octopus cluster to Quincy (as a first step on the way to Tentacle). The primary use of the cluster is cephfs, with separate data pools on HDD and SSD.
During the migration we accidentally reformatted 6 OSDs in a SSD 6+2 EC pool. Fortunately, all PGs recovered, except for one. Three of the lost OSDs contributed to this PG, so we are accepting that it is gone and any files that were placed on this PG are lost. We identified the lost files with cephfs-data-scan and found that all can be recovered from our non-ceph backup.
The problem:
Now that we've determined that all lost data can be recovered from backup, we'd like to get the PG active again so we can start this recovery. Unfortunately, the PG is stuck in the incomplete state. We've tried a few things to get it going again, some of which may have made matters worse:
ceph osd reweight $osd 0.0 ceph osd lost $osd
This left the PG blocked from peering due to 'history_les_bound'. After about an hour we tried:
ceph tell $osd config set osd_find_best_info_ignore_history_les 1
This didn't get it peering again, so a couple of hours later we tried forcing a repeer:
ceph pg repeer 8.7b
That didn't get it going either. Over the next couple of days we then tried a combination of repeer and restarting OSDs. During this we tried doing a force-create on the pg:
ceph osd force-create-pg 8.7b
However none of this helped. Attached is the latest output from 'ceph pg 8.7b query'. I'm at a loss at this point on what we can do to get the pg active again so that we can start restoring the lost files. Any suggestions?
--Mike _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io