BTW, guys, is there some sort of limit (via parater or in the code) to the maximum number of objects an OSD/PG might hold? I mean, something similar to the max PG number an OSD can implement, which defaults to 250. We are suspecting that this might be related to the large number of objects in a PG that has only a handful of PGs. On Tue, Jun 3, 2025 at 4:53 PM Work Ceph <work.ceph.user.mailing@gmail.com> wrote:
Ok, thanks for the clarification.
Is there some log that I can check to see the progress of the "pg repair"?
Also, do you have any clues on what can cause the "extra object situation"?
On Tue, Jun 3, 2025 at 4:51 PM Wesley Dillingham <wes@wesdillingham.com> wrote:
I have seen the issue.
In my experience, running "ceph pg repair <pg>" fixes the issue, but one must wait for the repair to commence and finish. So dont expect it to fix it immediately; the repair needs to be scheduled and processed.
Respectfully,
*Wes Dillingham* LinkedIn <http://www.linkedin.com/in/wesleydillingham> wes@wesdillingham.com
On Tue, Jun 3, 2025 at 3:46 PM Work Ceph < work.ceph.user.mailing@gmail.com> wrote:
Thanks for the prompt response!
I think that I did not understand. So, you have seen the issue I mentioned? Or, the issue of "ceph pg repair" not working properly?
If you have seen the issue I mentioned, how did you overcome it?
On Tue, Jun 3, 2025 at 4:43 PM Wesley Dillingham <wes@wesdillingham.com> wrote:
Yes, seen it, issuing a "ceph pg repair <pg>" has never not fixed it post repair.
Respectfully,
*Wes Dillingham* LinkedIn <http://www.linkedin.com/in/wesleydillingham> wes@wesdillingham.com
On Tue, Jun 3, 2025 at 3:40 PM Work Ceph < work.ceph.user.mailing@gmail.com> wrote:
Hello guys,
We are facing an unexpected error with a single PG after a node addition.
We are seeing the following error message: ``` scrub : stat mismatch, got 299586/299585 objects, 0/0 clones, 299586/299585 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 366757864470/366755158108 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes. log_channel(cluster) log [ERR] scrub 1 error ```
It looks as if there is an extra object in the PG. Have you guys ever seen such a situation?
The command "rados list-inconsistent-snapset" does not return anything. The command "rados list-inconsistent-pg" only returns the affected PG. The command "rados list-inconsistent-obj" does not return anything.
We are suspecting that this might be caused by the number of PGs in the pool, which is too small. However, before trying to increase it, we are trying to understand what is going on.
Is it safe to increase the number of PGs with a PG inconsistent with this situation with an extra object?
We are running Ceph on Ubuntu 22. We are using the standard Ceph binaries from the distro. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io