Hi all,
I am working on a ceph-ansible playbook[1] that removes an MDS from an
already deployed Ceph cluster. Going through documentation and
ceph-ansible codebase I found out 3 ways to stop an MDS -
* ceph fail mds fail <mds-name> && rm -rf /var/lib/cephmds/ceph-{id} [2]
* systemctl stop ceph-mds@$HOSTNAME
* ceph tell mds.x exit
How do these 3 ways compare to each other? I ran these commands on
ceph-ansible deployed cluster and all 3 had the very same effect. Is
any one of these better than the rest?
What about "ceph mds rm" and "ceph mds rmfailed"? The first time I
was
looking for various ways to stop an MDS, I tried "ceph mds fail
<mds-name> && ceph mds rm <global-id>" and it did not work since
"ceph
mds rm" requires an MDS to inactive[3]. Is there a way to render an
MDS inactive? I couldn't find one.
I also tried "ceph mds fail <mds-name> && ceph mds rmfailed
<mds-rank>" but this did not stop MDS. It only changed MDS's state to
'standby" -
(teuth-venv) $ ./bin/ceph fs dump | grep -A 1 standby_count_wanted 2> /dev/null
dumped fsmap epoch 4
standby_count_wanted 0
4232: [v2:192.168.0.217:6826/2113356090,v1:192.168.0.217:6827/2113356090]
'a' mds.0.3 up:active seq 4
(teuth-venv) $ ./bin/ceph mds fail a 2> /dev/null && ./bin/ceph mds
rmfailed --yes-i-really-mean-it 0 2> /dev/null && ./bin/ceph fs dump |
grep -A 3 Standby 2> /dev/null
dumped fsmap epoch 6
Standby daemons:
4286: [v2:192.168.0.217:6826/401505106,v1:192.168.0.217:6827/401505106]
'a' mds.-1.0 up:standby seq 1
(teuth-venv) $
Also, I find the usage of "remove" in this doc[2] ambiguous -- it can
mean removing MDS from cluster by changing MDS's state to standby or
it can mean killing/stopping it altogether. Reading [2] my impression
was that it meant killing/stopping it but "remove" is also used to
describe "ceph mds rm" and "ceph mds rmfailed" commands. Of these, at
least "ceph mds rmfailed" does not stop the MDS. If I am not the only
one to find this ambiguous, I'll go ahead and change the docs
accordingly.
- Rishabh
[1]
https://github.com/ceph/ceph-ansible/pull/4083
[2]
http://docs.ceph.com/docs/master/cephfs/add-remove-mds/
[3]
http://docs.ceph.com/docs/master/man/8/ceph/