Migrating clusters (and versions)
Hi list, I'm in the middle of an OpenStack migration (obviously Ceph backed) and stumble into some huge virtual machines. To ensure downtime is kept to a minimum, I'm thinking of using Ceph's snapshot features using rbd export-diff and import-diff. However, is it safe (or even supported) to do this across versions? The source cluster is running 10.2.11 and destination is 12.2.11 Thanks in advance! Regards, Kees -- https://nefos.nl/contact Nefos IT bv Ambachtsweg 25 (industrienummer 4217) 5627 BZ Eindhoven Nederland KvK 66494931 /Aanwezig op maandag, dinsdag, woensdag en vrijdag/
On 5/8/20 2:32 AM, Kees Meijs wrote:
I'm in the middle of an OpenStack migration (obviously Ceph backed) and stumble into some huge virtual machines.
To ensure downtime is kept to a minimum, I'm thinking of using Ceph's snapshot features using rbd export-diff and import-diff.
However, is it safe (or even supported) to do this across versions?
The source cluster is running 10.2.11 and destination is 12.2.11
Why just not migrate your VM's? k
Hi Konstantin, Thank you very much. That's a good question. The implementations of OpenStack and Ceph and "the other" OpenStack and Ceph are, apart from networking, completely separate. In terms of OpenStack I can recreate the compute instances and storage volumes but obviously need to copy the data as well. This is fine for small to medium volumes but now I stumble into some large ones: up to 10 TiBs in size. My idea (and hope) is to be able to use Ceph's snapshot mechanismn. First: take and copy a snapshot from a certain RBD volume to the other cluster. Then, another snapshot and copy the delta. And then once again but with the compute instance shut off. K. On 14-05-2020 06:24, Konstantin Shalygin wrote:
Why just not migrate your VM's?
Why not use rbd-mirror to handle the volumes?
On May 13, 2020, at 11:27 PM, Kees Meijs <kees@nefos.nl> wrote:
Hi Konstantin,
Thank you very much. That's a good question.
The implementations of OpenStack and Ceph and "the other" OpenStack and Ceph are, apart from networking, completely separate.
In terms of OpenStack I can recreate the compute instances and storage volumes but obviously need to copy the data as well. This is fine for small to medium volumes but now I stumble into some large ones: up to 10 TiBs in size.
My idea (and hope) is to be able to use Ceph's snapshot mechanismn. First: take and copy a snapshot from a certain RBD volume to the other cluster. Then, another snapshot and copy the delta. And then once again but with the compute instance shut off.
K.
On 14-05-2020 06:24, Konstantin Shalygin wrote:
Why just not migrate your VM's?
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
You can also mirror on a per-image basis. Zitat von Kees Meijs <kees@nefos.nl>:
I need to mirror single RBDs while rbd-mirror: "mirroring is configured on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
So?
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
It’s entirely possible — and documented — to mirror individual images. Your proposal to use snapshots is reinventing the wheel, but with less efficiency. https://docs.ceph.com/docs/nautilus/rbd/rbd-mirroring/#image-configuration ISTR that in Octopus the need for RBD journals is gone, but am not positive. For done 1-2 volumes at a time you’ll want to increase two tunables to avoid protracted syncing. Without these I’ve experienced a volume of just a few TB take multiple hours to converge, and some that got increasingly behind over time. rbd_mirror_journal_max_fetch_bytes: section: "client" value: "33554432" rbd_journal_max_payload_bytes: section: "client" value: "8388608"
On May 14, 2020, at 12:23 AM, Kees Meijs <kees@nefos.nl> wrote:
I need to mirror single RBDs while rbd-mirror: "mirroring is configured on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
So?
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks all, I'm going to investigate rbd-mirror further. K. On 14-05-2020 09:30, Anthony D'Atri wrote:
It’s entirely possible — and documented — to mirror individual images. Your proposal to use snapshots is reinventing the wheel, but with less efficiency.
https://docs.ceph.com/docs/nautilus/rbd/rbd-mirroring/#image-configuration
ISTR that in Octopus the need for RBD journals is gone, but am not positive.
For done 1-2 volumes at a time you’ll want to increase two tunables to avoid protracted syncing. Without these I’ve experienced a volume of just a few TB take multiple hours to converge, and some that got increasingly behind over time.
rbd_mirror_journal_max_fetch_bytes: section: "client" value: "33554432"
rbd_journal_max_payload_bytes: section: "client" value: "8388608"
rbd-mirror can work on a single image in the pool. and I did a test on image copy from 13.2 to 14.2. however, the data new in the source image didn't copy to the destination image. I'm not sure if this is normal. Kees Meijs <kees@nefos.nl> 于2020年5月14日周四 下午3:24写道：
I need to mirror single RBDs while rbd-mirror: "mirroring is configured on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
So?
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On 5/14/20 1:27 PM, Kees Meijs wrote:
Thank you very much. That's a good question.
The implementations of OpenStack and Ceph and "the other" OpenStack and Ceph are, apart from networking, completely separate.
Actually I was thinking you perform OpenStack and Ceph upgrade, not migration to other OpenStack and other Ceph. If downtime is acceptable I will suggest to use `qemu-img convert` instead rbd import/export, because: * this copy from one cluster to another; * speed up-to 3Gbit/s (my maximum from qemu network stack in 2018); qemu-img convert -m 16 -W -p -n -f raw -O raw \ rbd:<from_your_pool>/<your_volume>:id=cinder:key=<cinder_key>:mon_host=172.16.16.2 \ rbd:<to_your_pool>/<your_volume>:id=cinder:key=<cinder_key>:mon_host=172.16.17.2 k
participants (5)
-
Anthony D'Atri
-
Eugen Block
-
Kees Meijs
-
Konstantin Shalygin
-
Zhenshi Zhou