On Tue, Mar 26, 2024 at 7:30 PM Yongseok Oh <yongseok.oh(a)linecorp.com>
wrote:
Hi,
CephFS is provided as a shared file system service in a private cloud
environment of our company, LINE. The number of sessions is approximately
more than 5,000, and session evictions occur several times a day. When
session eviction occurs, the message 'Cannot send after transport endpoint
shutdown' or 'Permission denied' is displayed and file system access is not
possible. Our users are very uncomfortable with this issue. In particular,
there are no special problems such as network connection or CPU usage. When
I access the machine and take a close look, there are no special problems.
After this, users feel the inconvenience of having to perform umount/mount
tasks and run the application again. In a Kubernetes environment, recovery
is a bit more complicated, which causes a lot of frustration.
I tested that by setting the mds_session_blocklist_on_timeout and
mds_session_blocklist_on_evict options to false and setting
client_reconnect_stale to true on the client side, the file system can be
accessed even if eviction occurs. It seemed like there was no major problem
accessing the file system as the session was still attached.
What I'm curious about is if I turn on the above
option, will there be any
other side effects? For example, should I take some action if the integrity
of the file is broken or if there is an issue on the mds side? I am asking
a question because there are no details regarding this in the official
CephFS documentation.
The suggestion you are following is documented here
https://docs.ceph.com/en/latest/cephfs/eviction/#advanced-configuring-block…
As you mentioned rightly, it doesn't talk about the side effects if any.
But that said, it's kind of a workaround even if it works. Do we know the
exact reason for session eviction from the logs ?
It could be any of these things mentioned here
https://docs.ceph.com/en/latest/cephfs/eviction/#automatic-client-eviction
Thank you
Yongseok
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io