On 12/4/23 16:25, zxcs wrote:
Thanks a lot, Xiubo!
we already set ‘mds_bal_interval’ to 0. and the slow mds seems decrease.
But somehow we still see mds complain slow request. and from mds log , can see
“slow request *** seconds old, received at 2023-12-04T…: internal op exportdir:mds.* currently acquired locks”
so our question is, why it still see "internal op exportdir”, any other config also need to set 0? and could please shed light here which config we need set .
IMO, this should be enough. Venky, Did I miss something here ? Thanks - Xiubo
Thanks, xz
2023年11月27日 13:19，Xiubo Li <xiubli@redhat.com> 写道：
On 11/27/23 13:12, zxcs wrote:
current, we using `ceph config set mds mds_bal_interval 3600` to set a fixed time(1 hour).
we also have a question about how to set no balance for multi active mds.
means, we will enable multi active mds(to improve throughput) and no balance for these mds.
and if we set mds_bal_interval as big number seems can void this issue?
You can just set 'mds_bal_interval' to 0.
Thanks, xz
2023年11月27日 10:56，Ben <ruidong.gao@gmail.com> 写道：
with the same mds configuration, we see exactly the same(problem, log and solution) with 17.2.5, constantly happening again and again in couples days intervals. MDS servers are stuck somewhere, ceph status reports no issue however. We need to restart some of the mds (if not all of them) to restore them back. Hopefully this could be fixed soon or get docs updated with warning for the balancer's usage in production environment.
thanks and regards
Xiubo Li <xiubli@redhat.com> 于2023年11月23日周四 15:47写道：
Thanks a ton, Xiubo!
it not disappear.
even we umount the ceph directory on these two old os node.
after dump ops flight , we can see some request, and the earliest complain “failed to authpin, subtree is being exported" And how to avoid this, would you please help to shed some light here? Okay, as Frank mentioned you can try to disable the balancer by pining
On 11/23/23 11:25, zxcs wrote: the directories. As I remembered the balancer is buggy.
And also you can raise one ceph tracker and provide the debug logs if you have.
Thanks
- Xiubo
Thanks, xz
> 2023年11月22日 19:44，Xiubo Li <xiubli@redhat.com> 写道： > > > On 11/22/23 16:02, zxcs wrote: >> HI, Experts, >> >> we are using cephfs with 16.2.* with multi active mds, and recently, we have two nodes mount with ceph-fuse due to the old os system. >> and one nodes run a python script with `glob.glob(path)`, and another client doing `cp` operation on the same path. >> then we see some log about `mds slow request`, and logs complain “failed to authpin, subtree is being exported" >> then need to restart mds, >> >> >> our question is, does there any dead lock? how can we avoid this and how to fix it without restart mds(it will influence other users) ? > BTW, won't the slow requests disappear themself later ? > > It looks like the exporting is slow or there too many exports are going on. > Thanks > > - Xiubo > >> Thanks a ton! >> >> >> xz >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-leave@ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io