Unable to start scrub on PG with unfound object
Hi. We have a single unfound object. All OSDs are up, so a simple “put down to up” of an OSD is unlikely to recover it. Before we notify the object owner and mark the object as lost, I would like to scrub the PG to confirm that only this one object is missing. However, we are unable to force either a normal scrub or a deep scrub on the affected PG (e.g., ceph pg scrub 133.186 or ceph pg deep‑scrub 133.186). We even increased osd_max_scrubs from 1 to 3 on all OSDs (which are participating on PG 133.186): for i in 328 233 768 1202 84 423; do ceph config set osd.$i osd_max_scrubs 3 done Despite this, the scrub still does not start. We have set osd_scrub_during_recovery to true. I am attaching the configuration of the primary OSD (osd.233); the other OSDs have identical settings. WHO MASK LEVEL OPTION VALUE RO global basic device_failure_prediction_mode local osd advanced osd_max_backfills 10 osd.233 advanced osd_max_scrubs 3 osd advanced osd_mclock_override_recovery_settings false osd advanced osd_op_queue wpq * global advanced osd_pool_default_pg_autoscale_mode warn osd advanced osd_repair_during_recovery true osd advanced osd_scrub_auto_repair true osd advanced osd_scrub_during_recovery true osd.233 advanced osd_scrub_load_threshold 10.000000 State of PG: pg 133.186 is stuck undersized for 6d, current state active+backfill_unfound+undersized+degraded+remapped+forced_backfill, last acting [NONE,233,768,1202,NONE,423] Our ceph version: 18.2.1 (Reef) Do you have any ideas on how to get the scrub to run, or on alternative steps we should take to verify that only this single object is lost? Best regards, Michal
participants (1)
-
Michal Strnad