1 Feb
2023
1 Feb
'23
5:21 p.m.
Hi to all!
We are running a Ceph cluster (Octopus) on (99%) CentOS 7 (deployed at
the time with ceph-deploy) and we would like to upgrade it. As far as I
know for Pacific (and later releases) there aren't packages for CentOS 7
distribution (at least not on download.ceph.com), so we need to upgrade
(change) not only Ceph but also the distribution.
What is the raccomended path to do so?
We could upgrade (reinstall) all the nodes to Rocky 8 and then upgrade
Ceph to Quincy, but we will "stuck" with "not the latest" distribution
and probably we will have to upgrade (reinstall) again in the near future.
Our second idea is to leverage cephadm (which we would like to
implement) and switch from rpms to containers, but I don't have a clear
vision of how to do it. I was thinking to:
1. install a new monitor/manager with Rocky 9.
2. prepare the node for cephadm.
3. start the manager/monitor containers on that node.
4. repeat for the other monitors.
5. repeat for the OSD servers.
I'm not sure how to execute the point 2 and 3. The documentation says
how to bootstrap a NEW cluster and how to ADOPT an existing one, but our
situation is a hybrid (or in my mind it is...).
I cannot also adopt my current cluster to cephadm because we have 30% of
our OSD still on filestore. My intention was to drain them, reinstall
them and then adopt them. But I would like to avoid (if not necessary)
multiple reinstallations. In my mind all the OSD servers will be drained
before been reinstalled, just to be sure to have a "fresh" start).
Have you any ideas and/or advice to give us?
Thanks a lot!
Iztok
P.S. I saw that the script cephadm doesn't support Rocky, I can modify
it to do so and it should work, but is there a plan to officially
support it?
--
Iztok Gregori
ICT Systems and Services
Elettra - Sincrotrone Trieste S.C.p.A.
Telephone: +39 040 3758948
http://www.elettra.eu
1 Feb
1 Feb
10:41 p.m.
We successfully did ceph-deploy+octopus+centos7 -> (ceph-deploy
unsupported)+octopus+centos8stream (using leap) -> (ceph-deploy
unsupported)+pacific+centos8stream -> cephadm+pacific+centos8stream
Everything in place. Leap was tested repeatedly till the procedure/sideeffects were very
well known.
We also did s/centos8stream/rocky8/ successfully.
Thanks,
Kevin
________________________________________
From: Iztok Gregori <iztok.gregori(a)elettra.eu>
Sent: Wednesday, February 1, 2023 3:51 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Ceph Upgrade path
Check twice before you click! This email originated from outside PNNL.
Hi to all!
We are running a Ceph cluster (Octopus) on (99%) CentOS 7 (deployed at
the time with ceph-deploy) and we would like to upgrade it. As far as I
know for Pacific (and later releases) there aren't packages for CentOS 7
distribution (at least not on download.ceph.com), so we need to upgrade
(change) not only Ceph but also the distribution.
What is the raccomended path to do so?
We could upgrade (reinstall) all the nodes to Rocky 8 and then upgrade
Ceph to Quincy, but we will "stuck" with "not the latest"
distribution
and probably we will have to upgrade (reinstall) again in the near future.
Our second idea is to leverage cephadm (which we would like to
implement) and switch from rpms to containers, but I don't have a clear
vision of how to do it. I was thinking to:
1. install a new monitor/manager with Rocky 9.
2. prepare the node for cephadm.
3. start the manager/monitor containers on that node.
4. repeat for the other monitors.
5. repeat for the OSD servers.
I'm not sure how to execute the point 2 and 3. The documentation says
how to bootstrap a NEW cluster and how to ADOPT an existing one, but our
situation is a hybrid (or in my mind it is...).
I cannot also adopt my current cluster to cephadm because we have 30% of
our OSD still on filestore. My intention was to drain them, reinstall
them and then adopt them. But I would like to avoid (if not necessary)
multiple reinstallations. In my mind all the OSD servers will be drained
before been reinstalled, just to be sure to have a "fresh" start).
Have you any ideas and/or advice to give us?
Thanks a lot!
Iztok
P.S. I saw that the script cephadm doesn't support Rocky, I can modify
it to do so and it should work, but is there a plan to officially
support it?
--
Iztok Gregori
ICT Systems and Services
Elettra - Sincrotrone Trieste S.C.p.A.
Telephone: +39 040 3758948
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.elettr…
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
3 Feb
3 Feb
2:57 a.m.
New subject: [EXTERNAL] Re: Ceph Upgrade path
We had a similar challenge getting from Nautilus (ceph-deploy) / Xenial (ubuntu 16.04) to
Pacific (cephadm) / Focal (ubuntu 20.04), as the pacific packages were not available for
Xenial and Nautilus was not available for Focal. Our method was to upgrade and cephadm
adopt mons/mgrs/rgws all before making any changes to OSD nodes as follows:
1. Rebuild (one at a time) all 5 mon/mgr/rgw hosts as Bionic (ubuntu 18.04) and re-add
to cluster as Nautilus with ceph-deploy
2. Upgrade mon/mgr/rgw services from Nautilus to Pacific with apt (standard package
manager) method
3. Follow cephadm adopt procedures for mon/mgr/rgw hosts -
https://docs.ceph.com/en/pacific/cephadm/adoption/
4. Rebuild (one at a time) all 5 mon/mgr/rgw hosts as Focal and re-add to cluster as
Pacific with orchestrator / cephadm
5. Drain and then rebuild all OSD hosts 1-4 at a time as Focal and re-add to cluster as
Pacific
*
https://docs.ceph.com/en/pacific/rados/operations/bluestore-migration/#migr…
except using orchestrator / cephadm to build the “$NEWHOST”
Thank you,
Josh Beaman
From: Fox, Kevin M <Kevin.Fox(a)pnnl.gov>
Date: Wednesday, February 1, 2023 at 11:11 AM
To: Iztok Gregori <iztok.gregori(a)elettra.eu>eu>, ceph-users(a)ceph.io
<ceph-users(a)ceph.io>
Subject: [EXTERNAL] [ceph-users] Re: Ceph Upgrade path
We successfully did ceph-deploy+octopus+centos7 -> (ceph-deploy
unsupported)+octopus+centos8stream (using leap) -> (ceph-deploy
unsupported)+pacific+centos8stream -> cephadm+pacific+centos8stream
Everything in place. Leap was tested repeatedly till the procedure/sideeffects were very
well known.
We also did s/centos8stream/rocky8/ successfully.
Thanks,
Kevin
________________________________________
From: Iztok Gregori <iztok.gregori(a)elettra.eu>
Sent: Wednesday, February 1, 2023 3:51 AM
To: ceph-users(a)ceph.io
Subject: [ceph-users] Ceph Upgrade path
Check twice before you click! This email originated from outside PNNL.
Hi to all!
We are running a Ceph cluster (Octopus) on (99%) CentOS 7 (deployed at
the time with ceph-deploy) and we would like to upgrade it. As far as I
know for Pacific (and later releases) there aren't packages for CentOS 7
distribution (at least not on download.ceph.com), so we need to upgrade
(change) not only Ceph but also the distribution.
What is the raccomended path to do so?
We could upgrade (reinstall) all the nodes to Rocky 8 and then upgrade
Ceph to Quincy, but we will "stuck" with "not the latest"
distribution
and probably we will have to upgrade (reinstall) again in the near future.
Our second idea is to leverage cephadm (which we would like to
implement) and switch from rpms to containers, but I don't have a clear
vision of how to do it. I was thinking to:
1. install a new monitor/manager with Rocky 9.
2. prepare the node for cephadm.
3. start the manager/monitor containers on that node.
4. repeat for the other monitors.
5. repeat for the OSD servers.
I'm not sure how to execute the point 2 and 3. The documentation says
how to bootstrap a NEW cluster and how to ADOPT an existing one, but our
situation is a hybrid (or in my mind it is...).
I cannot also adopt my current cluster to cephadm because we have 30% of
our OSD still on filestore. My intention was to drain them, reinstall
them and then adopt them. But I would like to avoid (if not necessary)
multiple reinstallations. In my mind all the OSD servers will be drained
before been reinstalled, just to be sure to have a "fresh" start).
Have you any ideas and/or advice to give us?
Thanks a lot!
Iztok
P.S. I saw that the script cephadm doesn't support Rocky, I can modify
it to do so and it should work, but is there a plan to officially
support it?
--
Iztok Gregori
ICT Systems and Services
Elettra - Sincrotrone Trieste S.C.p.A.
Telephone: +39 040 3758948
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.elettr…
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
449
days inactive
450
days old
2 comments
3 participants
participants (3)
-
Beaman, Joshua (Contractor) -
Fox, Kevin M -
Iztok Gregori