8 May
2020
8 May
'20
12:02 a.m.
Hi list,
I'm in the middle of an OpenStack migration (obviously Ceph backed) and
stumble into some huge virtual machines.
To ensure downtime is kept to a minimum, I'm thinking of using Ceph's
snapshot features using rbd export-diff and import-diff.
However, is it safe (or even supported) to do this across versions?
The source cluster is running 10.2.11 and destination is 12.2.11
Thanks in advance!
Regards,
Kees
--
https://nefos.nl/contact
Nefos IT bv
Ambachtsweg 25 (industrienummer 4217)
5627 BZ Eindhoven
Nederland
KvK 66494931
/Aanwezig op maandag, dinsdag, woensdag en vrijdag/
14 May
14 May
8:54 a.m.
On 5/8/20 2:32 AM, Kees Meijs wrote:
I'm in the middle of an OpenStack migration
(obviously Ceph backed) and
stumble into some huge virtual machines.
To ensure downtime is kept to a minimum, I'm thinking of using Ceph's
snapshot features using rbd export-diff and import-diff.
However, is it safe (or even supported) to do this across versions?
The source cluster is running 10.2.11 and destination is 12.2.11
Why just not migrate your VM's?
k
10:57 a.m.
Hi Konstantin,
Thank you very much. That's a good question.
The implementations of OpenStack and Ceph and "the other" OpenStack and
Ceph are, apart from networking, completely separate.
In terms of OpenStack I can recreate the compute instances and storage
volumes but obviously need to copy the data as well. This is fine for
small to medium volumes but now I stumble into some large ones: up to 10
TiBs in size.
My idea (and hope) is to be able to use Ceph's snapshot mechanismn.
First: take and copy a snapshot from a certain RBD volume to the other
cluster. Then, another snapshot and copy the delta. And then once again
but with the compute instance shut off.
K.
On 14-05-2020 06:24, Konstantin Shalygin wrote:
Why just not migrate your VM's?
11:40 a.m.
Why not use rbd-mirror to handle the volumes?
On May 13, 2020, at 11:27 PM, Kees Meijs
<kees(a)nefos.nl> wrote:
Hi Konstantin,
Thank you very much. That's a good question.
The implementations of OpenStack and Ceph and "the other" OpenStack and
Ceph are, apart from networking, completely separate.
In terms of OpenStack I can recreate the compute instances and storage
volumes but obviously need to copy the data as well. This is fine for
small to medium volumes but now I stumble into some large ones: up to 10
TiBs in size.
My idea (and hope) is to be able to use Ceph's snapshot mechanismn.
First: take and copy a snapshot from a certain RBD volume to the other
cluster. Then, another snapshot and copy the delta. And then once again
but with the compute instance shut off.
K.
On 14-05-2020 06:24, Konstantin Shalygin wrote:
Why just not migrate your VM's?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:53 a.m.
I need to mirror single RBDs while rbd-mirror: "mirroring is configured
on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
> So?
11:58 a.m.
You can also mirror on a per-image basis.
Zitat von Kees Meijs <kees(a)nefos.nl>nl>:
I need to mirror single RBDs while rbd-mirror:
"mirroring is configured
on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
So?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
noon
It’s entirely possible — and documented — to mirror individual images. Your proposal to
use snapshots is reinventing the wheel, but with less efficiency.
https://docs.ceph.com/docs/nautilus/rbd/rbd-mirroring/#image-configuration
ISTR that in Octopus the need for RBD journals is gone, but am not positive.
For done 1-2 volumes at a time you’ll want to increase two tunables to avoid protracted
syncing. Without these I’ve experienced a volume of just a few TB take multiple hours to
converge, and some that got increasingly behind over time.
rbd_mirror_journal_max_fetch_bytes:
section: "client"
value: "33554432"
rbd_journal_max_payload_bytes:
section: "client"
value: "8388608"
On May 14, 2020, at 12:23 AM, Kees Meijs
<kees(a)nefos.nl> wrote:
I need to mirror single RBDs while rbd-mirror: "mirroring is configured
on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
So?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1:20 p.m.
Thanks all, I'm going to investigate rbd-mirror further.
K.
On 14-05-2020 09:30, Anthony D'Atri wrote:
> It’s entirely possible — and documented — to mirror individual images. Your proposal
to use snapshots is reinventing the wheel, but with less efficiency.
>
> https://docs.ceph.com/docs/nautilus/rbd/rbd-mirroring/#image-configuration
>
>
> ISTR that in Octopus the need for RBD journals is gone, but am not positive.
>
> For done 1-2 volumes at a time you’ll want to increase two tunables to avoid
protracted syncing. Without these I’ve experienced a volume of just a few TB take
multiple hours to converge, and some that got increasingly behind over time.
>
>
> rbd_mirror_journal_max_fetch_bytes:
> section: "client"
> value: "33554432"
>
> rbd_journal_max_payload_bytes:
> section: "client"
> value: "8388608"
12:05 p.m.
rbd-mirror can work on a single image in the pool.
and I did a test on image copy from 13.2 to 14.2.
however, the data new in the source image didn't
copy to the destination image. I'm not sure if this
is normal.
Kees Meijs <kees(a)nefos.nl> 于2020年5月14日周四 下午3:24写道：
I need to mirror single RBDs while rbd-mirror:
"mirroring is configured
on a per-pool basis" (according documentation).
On 14-05-2020 09:13, Anthony D'Atri wrote:
So?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
12:01 p.m.
On 5/14/20 1:27 PM, Kees Meijs wrote:
Thank you very much. That's a good question.
The implementations of OpenStack and Ceph and "the other" OpenStack and
Ceph are, apart from networking, completely separate.
Actually I was thinking you perform OpenStack and Ceph upgrade, not
migration to other OpenStack and other Ceph.
If downtime is acceptable I will suggest to use `qemu-img convert`
instead rbd import/export, because:
* this copy from one cluster to another;
* speed up-to 3Gbit/s (my maximum from qemu network stack in 2018);
qemu-img convert -m 16 -W -p -n -f raw -O raw \
rbd:<from_your_pool>/<your_volume>:id=cinder:key=<cinder_key>:mon_host=172.16.16.2
\
rbd:<to_your_pool>/<your_volume>:id=cinder:key=<cinder_key>:mon_host=172.16.17.2
k
1463
days inactive
1470
days old
11 comments
5 participants
participants (5)
-
Anthony D'Atri -
Eugen Block -
Kees Meijs -
Konstantin Shalygin -
Zhenshi Zhou