Hello all.
I have a cluster with ~80TB of spinning disk. Its primary role is
cephfs. Recently I had a multiple drive failure (it was not
simultaneous) but it's left me with 20 incomplete pg's
I know this data is toast, but I need to be able to get what isn't
toast out of the cephfs. Well out of that pool and into a new pool.
The issue is the PG's that are incomplete block IO and that hinders
browsing the filesystem.
I'm attempting to use the "new" ceph-objectstore-tool mark-complete
operation, but I'm struggling to work out what to mark complete. Being
that it's EC and each PG is made up of multiple shards (I think is the
right word) and they all have their own status.
I did manage to mark one of these shard pg's complete on what appeared
to be the primary OSD, however it had no effect on that shard when
checking it with ceph pg X query. By that I mean the shard was marked
incomplete before and after using ceph-ojbectstore-tool.
I'm running Ceph 18.2.2, I have all HDD osd's. I can get any logs that
will help, I'm just not 100% sure where to start and don't want to
just dump 20 PG's of ceph pg X query on the mailing list
Thanks so much
Mal
Show replies by date