Different ceph versions on nodes in cluster
Hello, I was wondering if it is ok to have different Ceph version numbers among the nodes in a Ceph cluster? I am asking because I started setting up a new Octopus cluster with cephadm bootstrap which installed my initial node with version 15.2.10 and added a new mon node today and that new node got installed with 15.2.11... What is the best practice here? Best regards, Mabi
Hi, you can't really avoid having different versions in a cluster, think about an update/upgrade. Usually it's not an issue, but I would recommend to have that state for as short as possible. Just a recently a customer complained about inconsistent objects on a regular basis although all disks were healthy. They run a multi-site RGW and after updating all nodes to the same version they haven't faced those inconsistencies anymore. Regards, Eugen Zitat von mabi <mabi@protonmail.ch>:
Hello,
I was wondering if it is ok to have different Ceph version numbers among the nodes in a Ceph cluster?
I am asking because I started setting up a new Octopus cluster with cephadm bootstrap which installed my initial node with version 15.2.10 and added a new mon node today and that new node got installed with 15.2.11...
What is the best practice here?
Best regards, Mabi
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Den fre 23 apr. 2021 kl 08:45 skrev Eugen Block <eblock@nde.ag>:
you can't really avoid having different versions in a cluster, think about an update/upgrade. Usually it's not an issue, but I would recommend to have that state for as short as possible. Just a recently a customer complained about inconsistent objects on a regular basis although all disks were healthy. They run a multi-site RGW and after updating all nodes to the same version they haven't faced those inconsistencies anymore.
There were some bug(s) relating to rgw's mis-caching negative answers in recent Octopus, so you might have just managed to get past that when you upgraded the rgws. Still, I agree that it seldom poses a problem if you do it in the recommended (and most tested) order. -- May the most significant bit of your life be positive.
Thanks to both of you for your answers. So I understand that the best practice would be to keep all nodes on the same Ceph version number. You mention here the "recommended (and most tested) order" which order is that? Using cephadm with containers wouldn't the orchestrator command below take care of everything without having me to worry about any upgraded order/path? ceph orch upgrade start --ceph-version 15.2.11 ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Friday, April 23, 2021 8:52 AM, Janne Johansson <icepic.dz@gmail.com> wrote: > Den fre 23 apr. 2021 kl 08:45 skrev Eugen Block eblock@nde.ag: > > > you can't really avoid having different versions in a cluster, think > > about an update/upgrade. Usually it's not an issue, but I would > > recommend to have that state for as short as possible. Just a recently > > a customer complained about inconsistent objects on a regular basis > > although all disks were healthy. They run a multi-site RGW and after > > updating all nodes to the same version they haven't faced those > > inconsistencies anymore. > > There were some bug(s) relating to rgw's mis-caching negative answers > in recent Octopus, so you might have just managed to get past that > when you upgraded the rgws. > Still, I agree that it seldom poses a problem if you do it in the > recommended (and most tested) order. > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > May the most significant bit of your life be positive. > > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io </icepic.dz@gmail.com>
You mention here the "recommended (and most tested) order" which order is that?
Using cephadm with containers wouldn't the orchestrator command below take care of everything without having me to worry about any upgraded order/path?
Yes, the orchestrator does that for you, but it can't hurt to understand what the order is. Before cephadm managed the update order it was/is like this: 1. Ceph Monitors/Ceph Managers 2. Ceph OSDs 3. Metadata Servers 4. Gateways Zitat von mabi <mabi@protonmail.ch>:
Thanks to both of you for your answers. So I understand that the best practice would be to keep all nodes on the same Ceph version number.
You mention here the "recommended (and most tested) order" which order is that?
Using cephadm with containers wouldn't the orchestrator command below take care of everything without having me to worry about any upgraded order/path?
ceph orch upgrade start --ceph-version 15.2.11
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Friday, April 23, 2021 8:52 AM, Janne Johansson <icepic.dz@gmail.com> wrote:
> Den fre 23 apr. 2021 kl 08:45 skrev Eugen Block eblock@nde.ag: > > > you can't really avoid having different versions in a cluster, think > > about an update/upgrade. Usually it's not an issue, but I would > > recommend to have that state for as short as possible. Just a recently > > a customer complained about inconsistent objects on a regular basis > > although all disks were healthy. They run a multi-site RGW and after > > updating all nodes to the same version they haven't faced those > > inconsistencies anymore. > > There were some bug(s) relating to rgw's mis-caching negative answers > in recent Octopus, so you might have just managed to get past that > when you upgraded the rgws. > Still, I agree that it seldom poses a problem if you do it in the > recommended (and most tested) order. > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > May the most significant bit of your life be positive. > > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io
</icepic.dz@gmail.com>
participants (3)
-
Eugen Block
-
Janne Johansson
-
mabi