For suggestions and best practices on expanding Ceph cluster and removing old nodes
Dear Ceph folks, I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。 I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster The second method would get rid of the procedure to upgrade the old 6-node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster. Any suggestions, advice, or best practice would be highly appreciated. best regards, Samuel huxiaoyu@horebdata.cn
Hi Samuel, While the second method would probably work fine in the happy path, if something goes wrong I think you'll be happier having a uniform release installed. In general, we've found the backfill experience to be better on Nautilus than Luminous, so my vote would be for the first method. Given that your usage is RGW, just note that the OMAP format change that happens between Luminous and Nautilus can sometimes take a while. Josh On Tue, Apr 25, 2023 at 10:31 AM huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> wrote:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the old 6-node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
best regards,
Samuel
huxiaoyu@horebdata.cn _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Get on nautilus first and (perhaps even go to pacific) before expansion. Primarily for the reason that starting in nautilus degraded data recovery will be prioritized over remapped data recovery. As you phase out old hardware and phase in new hardware you will have a very large amount of backfill happening and if you get into a degraded state in the middle of this backfill it will take a much longer time for the degraded data to become clean again. Additionally, you will want to follow the best practice of updating your cluster in order. In short monitors then managers then osds then MDS and RGW then other clients. More details here: https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-l... You dont want to run with a mixed software version cluster longer than a well coordinated upgrade takes. Respectfully, *Wes Dillingham* wes@wesdillingham.com LinkedIn <http://www.linkedin.com/in/wesleydillingham> On Tue, Apr 25, 2023 at 12:31 PM huxiaoyu@horebdata.cn < huxiaoyu@horebdata.cn> wrote:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the old 6-node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
best regards,
Samuel
huxiaoyu@horebdata.cn _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks a lot for the valuable input, Wesley, Josh, and Anthony. It seems the best practice would be upgrade first, and then expand, remove old nodes afterwards. best regards, Samuel huxiaoyu@horebdata.cn From: Wesley Dillingham Date: 2023-04-25 19:55 To: huxiaoyu@horebdata.cn CC: ceph-users Subject: Re: [ceph-users] For suggestions and best practices on expanding Ceph cluster and removing old nodes Get on nautilus first and (perhaps even go to pacific) before expansion. Primarily for the reason that starting in nautilus degraded data recovery will be prioritized over remapped data recovery. As you phase out old hardware and phase in new hardware you will have a very large amount of backfill happening and if you get into a degraded state in the middle of this backfill it will take a much longer time for the degraded data to become clean again. Additionally, you will want to follow the best practice of updating your cluster in order. In short monitors then managers then osds then MDS and RGW then other clients. More details here: https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-l... You dont want to run with a mixed software version cluster longer than a well coordinated upgrade takes. Respectfully, Wes Dillingham wes@wesdillingham.com LinkedIn On Tue, Apr 25, 2023 at 12:31 PM huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> wrote: Dear Ceph folks, I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。 I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster The second method would get rid of the procedure to upgrade the old 6-node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster. Any suggestions, advice, or best practice would be highly appreciated. best regards, Samuel huxiaoyu@horebdata.cn _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I would create a new cluster with Quincy and would migrate the data from the old to the new cluster bucket by bucket. Nautilus is out of support and I would recommend at least to use a ceph version that is receiving Backports. huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> schrieb am Di., 25. Apr. 2023, 18:30:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the old 6-node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
best regards,
Samuel
huxiaoyu@horebdata.cn _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Maybe he is limited by the supported OS
I would create a new cluster with Quincy and would migrate the data from the old to the new cluster bucket by bucket. Nautilus is out of support and I would recommend at least to use a ceph version that is receiving Backports.
huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> schrieb am Di., 25. Apr. 2023, 18:30:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the old 6- node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
I would second Joachim's suggestion - this is exactly what we're in the process of doing for a client, i.e migrating from Luminous to Quincy. However below would also work if you're moving to Nautilus. The only catch with this plan would be if you plan to reuse any hardware - i.e the hosts running rados gateways and mons, etc. If you have enough hardware to spare this is a good plan. My process: 1. Stand a new Quincy cluster and tune the cluster. 2. Migrate user information, secrets and access keys (using radosg-admin in a script). 3. Using a combination of rclone and parallel to push data across from the old cluster to the new cluster. Below is a bash script I used to capture all the user information on the old cluster and I ran it on the new cluster to create users and keep their secrets and keys the same. # for i in $(radosgw-admin user list | jq -r .[]); do USER_INFO=$(radosgw-admin user info --uid=$i) USER_ID=$(echo $USER_INFO | jq -r '.user_id') DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name') EMAIL=$(echo $USER_INFO | jq '.email') MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)') ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key') SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key') echo "radosgw-admin user create --uid=$USER_ID --display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS --access-key=$ACCESS --secret-key=$SECRET" | tee -a generated.radosgw-admin-user-create.sh done # Rclone is a really powerful tool! I lazily set up a backends for each user, by appending below to the for loop in the above script. Below script is not pretty but it does the job: # echo "" >> generated.rclone.conf echo [old-cluster-$USER_ID] >> generated.rclone.conf echo type = s3 >> generated.rclone.conf echo provider = Ceph >> generated.rclone.conf echo env_auth = false >> generated.rclone.conf echo access_key_id = $ACCESS >> generated.rclone.conf echo secret_access_key = $SECRET >> generated.rclone.conf echo endpoint = http://xx.xx.xx.xx:xxxx >> generated.rclone.conf echo acl = public-read >> generated.rclone.conf echo "" >> generated.rclone.conf echo [new-cluster-$USER_ID] >> generated.rclone.conf echo type = s3 >> generated.rclone.conf echo provider = Ceph >> generated.rclone.conf echo env_auth = false >> generated.rclone.conf echo access_key_id = $ACCESS >> generated.rclone.conf echo secret_access_key = $SECRET >> generated.rclone.conf echo endpoint = http://yy.yy.yy.yy:yyyy >> generated.rclone.conf echo acl = public-read >> generated.rclone.conf echo "" >> generated.rclone.conf # Copy the generated.rclone.conf to the node that is going to act as the transfer node (I just used the new rados gateway node) into ~/.config/rclone/rclone.conf Now if you run rclone lsd old-cluser-{user}: (it even tab completes!) you'll get a list of all the buckets for that user. You could even simply rclone sync old-cluser-{user}: new-cluser-{user}: and it should sync all buckets for a user. Catches: - Use the scripts carefully - our buckets for this one user are set public-read - you might want to check each line of the script if you use it. - Quincy bucket naming convention is stricter than Luminous. I've had to catch some '_' and upper cases and fix them in the command line I generate for copying each bucket. - Using rclone will take a long time.Feed a script into parallel sped things up for me: - # parallel -j 10 < sync-script - Watch out for lifecycling! Not sure how to handle this to make sure it's captured correctly. Cheers, Tom On Tue, 25 Apr 2023 at 22:36, Marc <Marc@f1-outsourcing.eu> wrote:
Maybe he is limited by the supported OS
I would create a new cluster with Quincy and would migrate the data from the old to the new cluster bucket by bucket. Nautilus is out of support and I would recommend at least to use a ceph version that is receiving Backports.
huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> schrieb am Di., 25. Apr. 2023, 18:30:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the old 6- node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks Tom, this is a very useful post! I've added our docs guy Zac in cc: IMHO this would be useful in a "Tips & Tricks" section of the docs. -- dan ______________________________ Clyso GmbH | https://www.clyso.com On Wed, Apr 26, 2023 at 7:46 AM Thomas Bennett <thomas@tsolo.io> wrote:
I would second Joachim's suggestion - this is exactly what we're in the process of doing for a client, i.e migrating from Luminous to Quincy. However below would also work if you're moving to Nautilus.
The only catch with this plan would be if you plan to reuse any hardware - i.e the hosts running rados gateways and mons, etc. If you have enough hardware to spare this is a good plan.
My process:
1. Stand a new Quincy cluster and tune the cluster. 2. Migrate user information, secrets and access keys (using radosg-admin in a script). 3. Using a combination of rclone and parallel to push data across from the old cluster to the new cluster.
Below is a bash script I used to capture all the user information on the old cluster and I ran it on the new cluster to create users and keep their secrets and keys the same.
# for i in $(radosgw-admin user list | jq -r .[]); do USER_INFO=$(radosgw-admin user info --uid=$i) USER_ID=$(echo $USER_INFO | jq -r '.user_id') DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name') EMAIL=$(echo $USER_INFO | jq '.email') MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)') ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key') SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key') echo "radosgw-admin user create --uid=$USER_ID --display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS --access-key=$ACCESS --secret-key=$SECRET" | tee -a generated.radosgw-admin-user-create.sh done #
Rclone is a really powerful tool! I lazily set up a backends for each user, by appending below to the for loop in the above script. Below script is not pretty but it does the job: # echo "" >> generated.rclone.conf echo [old-cluster-$USER_ID] >> generated.rclone.conf echo type = s3 >> generated.rclone.conf echo provider = Ceph >> generated.rclone.conf echo env_auth = false >> generated.rclone.conf echo access_key_id = $ACCESS >> generated.rclone.conf echo secret_access_key = $SECRET >> generated.rclone.conf echo endpoint = http://xx.xx.xx.xx:xxxx >> generated.rclone.conf echo acl = public-read >> generated.rclone.conf echo "" >> generated.rclone.conf echo [new-cluster-$USER_ID] >> generated.rclone.conf echo type = s3 >> generated.rclone.conf echo provider = Ceph >> generated.rclone.conf echo env_auth = false >> generated.rclone.conf echo access_key_id = $ACCESS >> generated.rclone.conf echo secret_access_key = $SECRET >> generated.rclone.conf echo endpoint = http://yy.yy.yy.yy:yyyy >> generated.rclone.conf echo acl = public-read >> generated.rclone.conf echo "" >> generated.rclone.conf #
Copy the generated.rclone.conf to the node that is going to act as the transfer node (I just used the new rados gateway node) into ~/.config/rclone/rclone.conf
Now if you run rclone lsd old-cluser-{user}: (it even tab completes!) you'll get a list of all the buckets for that user.
You could even simply rclone sync old-cluser-{user}: new-cluser-{user}: and it should sync all buckets for a user.
Catches:
- Use the scripts carefully - our buckets for this one user are set public-read - you might want to check each line of the script if you use it. - Quincy bucket naming convention is stricter than Luminous. I've had to catch some '_' and upper cases and fix them in the command line I generate for copying each bucket. - Using rclone will take a long time.Feed a script into parallel sped things up for me: - # parallel -j 10 < sync-script - Watch out for lifecycling! Not sure how to handle this to make sure it's captured correctly.
Cheers, Tom
On Tue, 25 Apr 2023 at 22:36, Marc <Marc@f1-outsourcing.eu> wrote:
Maybe he is limited by the supported OS
I would create a new cluster with Quincy and would migrate the data from the old to the new cluster bucket by bucket. Nautilus is out of support and I would recommend at least to use a ceph version that is receiving Backports.
huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> schrieb am Di., 25. Apr. 2023, 18:30:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the old 6- node from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
A pleasure. Hope it helps :) Happy to share if you need any more information Zac. Cheers, Tom On Wed, 26 Apr 2023 at 18:14, Dan van der Ster <dan.vanderster@clyso.com> wrote:
Thanks Tom, this is a very useful post! I've added our docs guy Zac in cc: IMHO this would be useful in a "Tips & Tricks" section of the docs.
-- dan
______________________________ Clyso GmbH | https://www.clyso.com
On Wed, Apr 26, 2023 at 7:46 AM Thomas Bennett <thomas@tsolo.io> wrote:
I would second Joachim's suggestion - this is exactly what we're in the process of doing for a client, i.e migrating from Luminous to Quincy. However below would also work if you're moving to Nautilus.
The only catch with this plan would be if you plan to reuse any hardware
i.e the hosts running rados gateways and mons, etc. If you have enough hardware to spare this is a good plan.
My process:
1. Stand a new Quincy cluster and tune the cluster. 2. Migrate user information, secrets and access keys (using radosg-admin in a script). 3. Using a combination of rclone and parallel to push data across from the old cluster to the new cluster.
Below is a bash script I used to capture all the user information on the old cluster and I ran it on the new cluster to create users and keep
secrets and keys the same.
# for i in $(radosgw-admin user list | jq -r .[]); do USER_INFO=$(radosgw-admin user info --uid=$i) USER_ID=$(echo $USER_INFO | jq -r '.user_id') DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name') EMAIL=$(echo $USER_INFO | jq '.email') MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)') ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key') SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key') echo "radosgw-admin user create --uid=$USER_ID --display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS --access-key=$ACCESS --secret-key=$SECRET" | tee -a generated.radosgw-admin-user-create.sh done #
Rclone is a really powerful tool! I lazily set up a backends for each user, by appending below to the for loop in the above script. Below script is not pretty but it does the job: # echo "" >> generated.rclone.conf echo [old-cluster-$USER_ID] >> generated.rclone.conf echo type = s3 >> generated.rclone.conf echo provider = Ceph >> generated.rclone.conf echo env_auth = false >> generated.rclone.conf echo access_key_id = $ACCESS >> generated.rclone.conf echo secret_access_key = $SECRET >> generated.rclone.conf echo endpoint = http://xx.xx.xx.xx:xxxx >> generated.rclone.conf echo acl = public-read >> generated.rclone.conf echo "" >> generated.rclone.conf echo [new-cluster-$USER_ID] >> generated.rclone.conf echo type = s3 >> generated.rclone.conf echo provider = Ceph >> generated.rclone.conf echo env_auth = false >> generated.rclone.conf echo access_key_id = $ACCESS >> generated.rclone.conf echo secret_access_key = $SECRET >> generated.rclone.conf echo endpoint = http://yy.yy.yy.yy:yyyy >> generated.rclone.conf echo acl = public-read >> generated.rclone.conf echo "" >> generated.rclone.conf #
Copy the generated.rclone.conf to the node that is going to act as the transfer node (I just used the new rados gateway node) into ~/.config/rclone/rclone.conf
Now if you run rclone lsd old-cluser-{user}: (it even tab completes!) you'll get a list of all the buckets for that user.
You could even simply rclone sync old-cluser-{user}: new-cluser-{user}: and it should sync all buckets for a user.
Catches:
- Use the scripts carefully - our buckets for this one user are set public-read - you might want to check each line of the script if you use it. - Quincy bucket naming convention is stricter than Luminous. I've had to catch some '_' and upper cases and fix them in the command line I generate for copying each bucket. - Using rclone will take a long time.Feed a script into parallel sped things up for me: - # parallel -j 10 < sync-script - Watch out for lifecycling! Not sure how to handle this to make sure it's captured correctly.
Cheers, Tom
On Tue, 25 Apr 2023 at 22:36, Marc <Marc@f1-outsourcing.eu> wrote:
Maybe he is limited by the supported OS
I would create a new cluster with Quincy and would migrate the data
from
the old to the new cluster bucket by bucket. Nautilus is out of support and I would recommend at least to use a ceph version that is receiving Backports.
huxiaoyu@horebdata.cn <huxiaoyu@horebdata.cn> schrieb am Di., 25. Apr. 2023, 18:30:
Dear Ceph folks,
I would like to listen to your advice on the following topic: We have a 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph cluster with the new 10 nodes on Nautilus version。
I can think of two ways to achieve the above goal. The first method would be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and then re-balance; 3) After rebalance completes, remove the 6 old nodes from the cluster
The second method would get rid of the procedure to upgrade the
- their old 6-
node
from Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from the cluster.
Any suggestions, advice, or best practice would be highly appreciated.
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (7)
-
Dan van der Ster
-
huxiaoyu＠horebdata.cn
-
Joachim Kraftmayer
-
Josh Baergen
-
Marc
-
Thomas Bennett
-
Wesley Dillingham