Upgrade Luminous to Nautilus on a Debian system
Hi all, I'm planning to upgrade one on my Ceph Cluster currently on Luminous 12.2.13 / Debian Stretch (updated). On this cluster, Luminous is packaged from the official Ceph repo (deb https://download.ceph.com/debian-luminous/ stretch main) I would like to upgrade it with Debian Buster and Nautilus using the croit.io repository (deb https://mirror.croit.io/debian-nautilus/ buster main) I already prepared the steps procedure but I just want to verify one step regarding the upgrade of the ceph packages. Do I have to upgrade ceph in the same time than Debian or do i have to upgrade ceph after the Debian upgrade from Stretch to Buster ? 1) In the first case : * Replace stretch by buster in /etc/apt/sources.list * Modify the ceph.list repo by croit.io one * Upgrade the entire nodes 2) In the second case (upgrade Debian then Ceph) * Replace stretch by buster in /etc/apt/sources.list * keep the /etc/apt/sources.list.d/ceph.list as it is * Upgrade and reboot the nodes * replace the ceph.list file by croit.io * upgrade the ceph packages * restarting the Ceph services (in the right order MON -> MGR -> OSD -> MDS) Thanks a lot for your advices Regards, Hervé
On Wed, Apr 29, 2020 at 11:54 AM Herve Ballans <herve.ballans@ias.u-psud.fr> wrote:
Hi all,
I'm planning to upgrade one on my Ceph Cluster currently on Luminous 12.2.13 / Debian Stretch (updated). On this cluster, Luminous is packaged from the official Ceph repo (deb https://download.ceph.com/debian-luminous/ stretch main)
I would like to upgrade it with Debian Buster and Nautilus using the croit.io repository (deb https://mirror.croit.io/debian-nautilus/ buster main)
I already prepared the steps procedure but I just want to verify one step regarding the upgrade of the ceph packages.
Do I have to upgrade ceph in the same time than Debian or do i have to upgrade ceph after the Debian upgrade from Stretch to Buster ?
1) In the first case :
* Replace stretch by buster in /etc/apt/sources.list * Modify the ceph.list repo by croit.io one * Upgrade the entire nodes
2) In the second case (upgrade Debian then Ceph)
* Replace stretch by buster in /etc/apt/sources.list * keep the /etc/apt/sources.list.d/ceph.list as it is * Upgrade and reboot the nodes * replace the ceph.list file by croit.io * upgrade the ceph packages * restarting the Ceph services (in the right order MON -> MGR -> OSD -> MDS)
Thanks a lot for your advices
Regards, Hervé
Hi Hervé, The one thing I had trouble with (and it's primarily from not reading the docs very carefully) is that you should NOT enable the messenger 2 protocol until all OSDs have been updated. In other words, Ceph will complain about not running msgr2, but you should leave it like that until all OSDs are on Nautilus. Then you run: ceph mon enable-msgr2 ceph osd require-osd-release nautilus Ref: https://docs.ceph.com/docs/master/releases/nautilus/ -- Alex Gorbachev Intelligent Systems Services Inc.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Alex, Thanks a lot for your tips. I note that for my planned upgrade. I take the opportunity here to add a complementary question regarding the require-osd-release functionality (ceph osd require-osd-release nautilus ) I remember that one time I did that (on another cluster, a proxmox one) and it took a very long time and had a strong impact on the ceph performances during this operation (several hours) Did you notice that too on your side ? Thanks again, Hervé On 29/04/2020 20:39, Alex Gorbachev wrote:
On Wed, Apr 29, 2020 at 11:54 AM Herve Ballans <herve.ballans@ias.u-psud.fr <mailto:herve.ballans@ias.u-psud.fr>> wrote:
Hi all,
I'm planning to upgrade one on my Ceph Cluster currently on Luminous 12.2.13 / Debian Stretch (updated). On this cluster, Luminous is packaged from the official Ceph repo (deb https://download.ceph.com/debian-luminous/ stretch main)
I would like to upgrade it with Debian Buster and Nautilus using the croit.io <http://croit.io> repository (deb https://mirror.croit.io/debian-nautilus/ buster main)
I already prepared the steps procedure but I just want to verify one step regarding the upgrade of the ceph packages.
Do I have to upgrade ceph in the same time than Debian or do i have to upgrade ceph after the Debian upgrade from Stretch to Buster ?
1) In the first case :
* Replace stretch by buster in /etc/apt/sources.list * Modify the ceph.list repo by croit.io <http://croit.io> one * Upgrade the entire nodes
2) In the second case (upgrade Debian then Ceph)
* Replace stretch by buster in /etc/apt/sources.list * keep the /etc/apt/sources.list.d/ceph.list as it is * Upgrade and reboot the nodes * replace the ceph.list file by croit.io <http://croit.io> * upgrade the ceph packages * restarting the Ceph services (in the right order MON -> MGR -> OSD -> MDS)
Thanks a lot for your advices
Regards, Hervé
Hi Hervé,
The one thing I had trouble with (and it's primarily from not reading the docs very carefully) is that you should NOT enable the messenger 2 protocol until all OSDs have been updated. In other words, Ceph will complain about not running msgr2, but you should leave it like that until all OSDs are on Nautilus. Then you run: ceph mon enable-msgr2 ceph osd require-osd-release nautilus
Ref: https://docs.ceph.com/docs/master/releases/nautilus/
-- Alex Gorbachev Intelligent Systems Services Inc.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
We run the Luminous/Mimic -> Nautilus upgrade by upgrading Ceph and Debian at the same time, i.e., your first scenario. Didn't encounter any problems with that; the Nautilus upgrade has been very smooth for us and we've migrated almost all of our deployments using our fully automated upgrade assistant; it's just one button that does all the right things in the right order. Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Wed, Apr 29, 2020 at 8:58 PM Herve Ballans <herve.ballans@ias.u-psud.fr> wrote:
Hi Alex,
Thanks a lot for your tips. I note that for my planned upgrade.
I take the opportunity here to add a complementary question regarding the require-osd-release functionality (ceph osd require-osd-release nautilus )
I remember that one time I did that (on another cluster, a proxmox one) and it took a very long time and had a strong impact on the ceph performances during this operation (several hours)
Did you notice that too on your side ?
Thanks again, Hervé
On 29/04/2020 20:39, Alex Gorbachev wrote:
On Wed, Apr 29, 2020 at 11:54 AM Herve Ballans <herve.ballans@ias.u-psud.fr <mailto:herve.ballans@ias.u-psud.fr>> wrote:
Hi all,
I'm planning to upgrade one on my Ceph Cluster currently on Luminous 12.2.13 / Debian Stretch (updated). On this cluster, Luminous is packaged from the official Ceph repo (deb https://download.ceph.com/debian-luminous/ stretch main)
I would like to upgrade it with Debian Buster and Nautilus using the croit.io <http://croit.io> repository (deb https://mirror.croit.io/debian-nautilus/ buster main)
I already prepared the steps procedure but I just want to verify one step regarding the upgrade of the ceph packages.
Do I have to upgrade ceph in the same time than Debian or do i have to upgrade ceph after the Debian upgrade from Stretch to Buster ?
1) In the first case :
* Replace stretch by buster in /etc/apt/sources.list * Modify the ceph.list repo by croit.io <http://croit.io> one * Upgrade the entire nodes
2) In the second case (upgrade Debian then Ceph)
* Replace stretch by buster in /etc/apt/sources.list * keep the /etc/apt/sources.list.d/ceph.list as it is * Upgrade and reboot the nodes * replace the ceph.list file by croit.io <http://croit.io> * upgrade the ceph packages * restarting the Ceph services (in the right order MON -> MGR -> OSD -> MDS)
Thanks a lot for your advices
Regards, Hervé
Hi Hervé,
The one thing I had trouble with (and it's primarily from not reading the docs very carefully) is that you should NOT enable the messenger 2 protocol until all OSDs have been updated. In other words, Ceph will complain about not running msgr2, but you should leave it like that until all OSDs are on Nautilus. Then you run: ceph mon enable-msgr2 ceph osd require-osd-release nautilus
Ref: https://docs.ceph.com/docs/master/releases/nautilus/
-- Alex Gorbachev Intelligent Systems Services Inc.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Paul, So we did the upgrade of our cluster by upgrading Ceph and debian at the same time. And you right, it worked out perfectly ! with no problems at all. We mainly followed the steps described here, so about a dozen steps. Your assistant seems to be powerful as you do that with one button ! ;) Regards, Hervé On 29/04/2020 23:28, Paul Emmerich wrote:
We run the Luminous/Mimic -> Nautilus upgrade by upgrading Ceph and Debian at the same time, i.e., your first scenario.
Didn't encounter any problems with that; the Nautilus upgrade has been very smooth for us and we've migrated almost all of our deployments using our fully automated upgrade assistant; it's just one button that does all the right things in the right order.
Paul
Herve, On Wed, Apr 29, 2020 at 2:57 PM Herve Ballans <herve.ballans@ias.u-psud.fr> wrote:
Hi Alex,
Thanks a lot for your tips. I note that for my planned upgrade.
I take the opportunity here to add a complementary question regarding the require-osd-release functionality (ceph osd require-osd-release nautilus )
I remember that one time I did that (on another cluster, a proxmox one) and it took a very long time and had a strong impact on the ceph performances during this operation (several hours)
Did you notice that too on your side ?
I did not notice any issues with the process. That said, the upgrade was in our lab cluster, which does not do a lot of IO. Things may be different in one of the production clusters, but I will not know for at least a couple of months. -- Alex Gorbachev Intelligent Systems Services Inc.
Thanks again, Hervé On 29/04/2020 20:39, Alex Gorbachev wrote:
On Wed, Apr 29, 2020 at 11:54 AM Herve Ballans < herve.ballans@ias.u-psud.fr> wrote:
Hi all,
I'm planning to upgrade one on my Ceph Cluster currently on Luminous 12.2.13 / Debian Stretch (updated). On this cluster, Luminous is packaged from the official Ceph repo (deb https://download.ceph.com/debian-luminous/ stretch main)
I would like to upgrade it with Debian Buster and Nautilus using the croit.io repository (deb https://mirror.croit.io/debian-nautilus/ buster main)
I already prepared the steps procedure but I just want to verify one step regarding the upgrade of the ceph packages.
Do I have to upgrade ceph in the same time than Debian or do i have to upgrade ceph after the Debian upgrade from Stretch to Buster ?
1) In the first case :
* Replace stretch by buster in /etc/apt/sources.list * Modify the ceph.list repo by croit.io one * Upgrade the entire nodes
2) In the second case (upgrade Debian then Ceph)
* Replace stretch by buster in /etc/apt/sources.list * keep the /etc/apt/sources.list.d/ceph.list as it is * Upgrade and reboot the nodes * replace the ceph.list file by croit.io * upgrade the ceph packages * restarting the Ceph services (in the right order MON -> MGR -> OSD -> MDS)
Thanks a lot for your advices
Regards, Hervé
Hi Hervé,
The one thing I had trouble with (and it's primarily from not reading the docs very carefully) is that you should NOT enable the messenger 2 protocol until all OSDs have been updated. In other words, Ceph will complain about not running msgr2, but you should leave it like that until all OSDs are on Nautilus. Then you run: ceph mon enable-msgr2 ceph osd require-osd-release nautilus
Ref: https://docs.ceph.com/docs/master/releases/nautilus/
-- Alex Gorbachev Intelligent Systems Services Inc.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Alex, Yes you right, indeed no issues encoutered with /'ceph osd //require-osd-release//nautilus/' command. My bad in fact, I got confused that feature with '/ceph osd crush tunables optimal/' ! Oops! But it seems the last one is not necessary when upgrading from Luminous to Nautilus... Cheers, rv On 01/05/2020 07:14, Alex Gorbachev wrote:
Herve,
On Wed, Apr 29, 2020 at 2:57 PM Herve Ballans <herve.ballans@ias.u-psud.fr <mailto:herve.ballans@ias.u-psud.fr>> wrote:
Hi Alex,
Thanks a lot for your tips. I note that for my planned upgrade.
I take the opportunity here to add a complementary question regarding the require-osd-release functionality (ceph osd require-osd-release nautilus )
I remember that one time I did that (on another cluster, a proxmox one) and it took a very long time and had a strong impact on the ceph performances during this operation (several hours)
Did you notice that too on your side ?
I did not notice any issues with the process. That said, the upgrade was in our lab cluster, which does not do a lot of IO. Things may be different in one of the production clusters, but I will not know for at least a couple of months.
-- Alex Gorbachev Intelligent Systems Services Inc.
Thanks again, Hervé
On 29/04/2020 20:39, Alex Gorbachev wrote:
On Wed, Apr 29, 2020 at 11:54 AM Herve Ballans <herve.ballans@ias.u-psud.fr <mailto:herve.ballans@ias.u-psud.fr>> wrote:
Hi all,
I'm planning to upgrade one on my Ceph Cluster currently on Luminous 12.2.13 / Debian Stretch (updated). On this cluster, Luminous is packaged from the official Ceph repo (deb https://download.ceph.com/debian-luminous/ stretch main)
I would like to upgrade it with Debian Buster and Nautilus using the croit.io <http://croit.io> repository (deb https://mirror.croit.io/debian-nautilus/ buster main)
I already prepared the steps procedure but I just want to verify one step regarding the upgrade of the ceph packages.
Do I have to upgrade ceph in the same time than Debian or do i have to upgrade ceph after the Debian upgrade from Stretch to Buster ?
1) In the first case :
* Replace stretch by buster in /etc/apt/sources.list * Modify the ceph.list repo by croit.io <http://croit.io> one * Upgrade the entire nodes
2) In the second case (upgrade Debian then Ceph)
* Replace stretch by buster in /etc/apt/sources.list * keep the /etc/apt/sources.list.d/ceph.list as it is * Upgrade and reboot the nodes * replace the ceph.list file by croit.io <http://croit.io> * upgrade the ceph packages * restarting the Ceph services (in the right order MON -> MGR -> OSD -> MDS)
Thanks a lot for your advices
Regards, Hervé
Hi Hervé,
The one thing I had trouble with (and it's primarily from not reading the docs very carefully) is that you should NOT enable the messenger 2 protocol until all OSDs have been updated. In other words, Ceph will complain about not running msgr2, but you should leave it like that until all OSDs are on Nautilus. Then you run: ceph mon enable-msgr2 ceph osd require-osd-release nautilus
Ref: https://docs.ceph.com/docs/master/releases/nautilus/
-- Alex Gorbachev Intelligent Systems Services Inc.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
participants (3)
-
Alex Gorbachev
-
Herve Ballans
-
Paul Emmerich