Maybe this issue: https://tracker.ceph.com/issues/44286 ``` bash ceph osd pool set <pool> hit_set_count 0 # and mark_unfound_lost ``` After there is no unfound hit_set_archive object, reset hit_set_count value 在 2026/3/25 17:54, simon kirchhofer via dev 写道:
Hi,
I think we got a bug with the tiering because of a hit_set object.
We got a cluster with HDD servers and NVME servers. We got a tiering setup to use one filesystem over all servers. It worked for years but we recently experience a problem :
At first, we got a notification for a missing object on a pg. The object from the namespace ceph_internal is a hit_set object. We tried to fix it with `ceph pg [pgid] mark_unfound_lost revert` 4 hours later, all 3 osd of this pg went down, and for each replication, all new osd go down in a row. We had to add the flag "no out" to investigate. It looks like the trimming of old hit_set object crash the osd. The hit_set missing object was the oldest on the list. With ceph_objectstore_tool, we see that the hit_set object is not in the pg. We suspect that the timmer try to delete it, and doesn't found it. We found out that we can restart osds of this pg until the pg is available. Then, all osd go down again. Here is a manipulation to get the pg active :
1. All OSD of this pg are down, flags norecover, nobackfill, norebalance, notieragence. Pause read and pause write 2. We export and remove the pg in all osd but one (osd 48 in this case) 3. Restart all osd. They stay all up and the pg is active. 4. BUT When removing the pause, the only osd of the pg get down.
When we remove some flags (recovery, backfill, rebalance) after step 3, we got 3 osd involved in the pg. The pg is still active after recovery. And 3 goes down after removing pause. You can find the core dump in attachement. Do you know how to recover from this state ? Is there a procedure to fix errors on hit_set objects ? Best regard, Simon