15 Aug
2020
15 Aug
'20
10:47 p.m.
Hi all,
We just completed maintenance on an OSD node and we ran into an issue where all data
seemed to stop flowing while the node was down. We couldn't connect to any of our VMs
during that time. I was under the impression that by setting the 'noout' flag, you
would not get the rebalance of the data but you would update the pointers to use the 2nd
and 3rd copies of the data. Is that not correct, and what is the proper workflow for
taking down an OSD node for maintenance?
Thanks,
15 Aug
15 Aug
10:51 p.m.
Do you mean I/O stopped on your VMs?
Sent from mobile
Op 15 aug. 2020 om 17:48 heeft Matt Dunavant
<mdunavant(a)convokesystems.com> het volgende geschreven:
Hi all,
We just completed maintenance on an OSD node and we ran into an issue where all data
seemed to stop flowing while the node was down. We couldn't connect to any of our VMs
during that time. I was under the impression that by setting the 'noout' flag, you
would not get the rebalance of the data but you would update the pointers to use the 2nd
and 3rd copies of the data. Is that not correct, and what is the proper workflow for
taking down an OSD node for maintenance?
Thanks,
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:07 p.m.
Yeah, the VMs didn't die completely but they were all inaccessible during the
maintenance period. Once the maintenance node came back up, it started flowing again.
16 Aug
16 Aug
12:45 a.m.
What are size and min_size for that pool?
Zitat von Matt Dunavant <mdunavant(a)convokesystems.com>om>:
Yeah, the VMs didn't die completely but they were
all inaccessible
during the maintenance period. Once the maintenance node came back
up, it started flowing again.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:19 a.m.
Did you check the ceph status? ("ceph -s")
On 16/08/2020 1:47 am, Matt Dunavant wrote:
Hi all,
We just completed maintenance on an OSD node and we ran into an issue where all data
seemed to stop flowing while the node was down. We couldn't connect to any of our VMs
during that time. I was under the impression that by setting the 'noout' flag, you
would not get the rebalance of the data but you would update the pointers to use the 2nd
and 3rd copies of the data. Is that not correct, and what is the proper workflow for
taking down an OSD node for maintenance?
Thanks,
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Lindsay
17 Aug
17 Aug
8:07 p.m.
Hello all,
Thanks for the help. I believe we traced this down to be an issue with the crush rules. It
seems somehow osd_crush_chooseleaf_type = 0 got placed into our configuration. This
caused ceph osd crush rule dump to include this line ' "op":
"choose_firstn",' instead of 'chooseleaf_firstn'. Which I believe
would tell ceph to make its failure domain an OSD instead of a node.
1355
days inactive
1357
days old
5 comments
4 participants
participants (4)
-
Eugen Block -
Lindsay Mathieson -
Matt Dunavant -
William Edwards