28 Jul
2020
28 Jul
'20
7:50 p.m.
Hi,
As we expand our cluster (adding nodes), we'd like to take advantage of
better EC profiles enabled by higher server/rack counts. I understand, as
Ceph currently exists (15.2.4), there is no way to live-migrate from one EC
profile to another on an existing pool, for example, from 4+2 to 17+3 when
going from 7 nodes to 21. Is this correct?
How are people accomplishing migrations such as this one (or 4+2 to 9+3,
for example) with minimal disruption to services that utilize RBDs sitting
on top of these pools? I found:
https://ceph.io/geen-categorie/ceph-pool-migration/ , which requires
effectively shutting down access during migration (which is doable, but not
ideal) - and from what I've read - has some potential downsides
(specifically referencing the cppool method).
The pools are all EC for data, and an rbd pool exists for metadata.
Thank you!
28 Jul
28 Jul
8:43 p.m.
Den tis 28 juli 2020 kl 18:50 skrev David Orman <ormandj(a)corenode.com>om>:
Hi,
As we expand our cluster (adding nodes), we'd like to take advantage of
better EC profiles enabled by higher server/rack counts. I understand, as
Ceph currently exists (15.2.4), there is no way to live-migrate from one EC
profile to another on an existing pool, for example, from 4+2 to 17+3 when
going from 7 nodes to 21. Is this correct?
..depends on how brave you are, but mostly it's true.
Mostly people stick to the configured EC size(s) on the pools even if you
buy
more nodes, the EC profile doesn't have to be close to the number of nodes,
it
can well be 4+2 even with 21 nodes, it will just choose 6 random nodes out
of
the 21 for each PG and spread out evenly in your new larger cluster.
Also, 17+3 means 20 nodes are involved in every reasonably sized IO, and
that
may not be optimal, both in terms of CPU usage, and of course that if your
previous
speed was ok when 6 nodes were needed to reassemble a piece of data, you
can now do 3 of those 4+2-IOs in parallel if your network allows.
Also, with 17+3 on 21 nodes, then a two node failure means it is
continually degraded*
until a host comes back up, but with k+m being < 15 would mean that the
cluster can
repair itself onto those ~15 nodes and still be in full swing even if up to
6 nodes would
be lost, however improbable such a failure would be.
*) So the +3 normally means three hosts could fail, but having only 19
hosts on a 17+3
would mean it can never reach active+clean for any PG, since there
aren't 20 hosts to
use. You wouldn't lose data, but all PGs would be degraded/undersized
meaning you
run the whole cluster at 17+2 at best.
--
May the most significant bit of your life be positive.
29 Jul
29 Jul
8:46 a.m.
I'm facing the same issue. My cluster will have an expansion and I wanna
modify the ec profile too. What I can think of is to create a new profile
and a new pool, and then migrate the data from the old pool to the new one.
Finally rename the pools as I can use the new pool just like nothing
happened.
And of course, the access will be shut down while migrating. I'm looking
for a more efficient way, which has no need to stop the clients, to deal
with this.
Janne Johansson <icepic.dz(a)gmail.com> 于2020年7月29日周三 上午1:43写道：
Den tis 28 juli 2020 kl 18:50 skrev David Orman
<ormandj(a)corenode.com>om>:
Hi,
As we expand our cluster (adding nodes), we'd like to take advantage of
better EC profiles enabled by higher server/rack counts. I understand, as
Ceph currently exists (15.2.4), there is no way to live-migrate from one
EC
profile to another on an existing pool, for
example, from 4+2 to 17+3
when
going from 7 nodes to 21. Is this correct?
..depends on how brave you are, but mostly it's true.
Mostly people stick to the configured EC size(s) on the pools even if you
buy
more nodes, the EC profile doesn't have to be close to the number of nodes,
it
can well be 4+2 even with 21 nodes, it will just choose 6 random nodes out
of
the 21 for each PG and spread out evenly in your new larger cluster.
Also, 17+3 means 20 nodes are involved in every reasonably sized IO, and
that
may not be optimal, both in terms of CPU usage, and of course that if your
previous
speed was ok when 6 nodes were needed to reassemble a piece of data, you
can now do 3 of those 4+2-IOs in parallel if your network allows.
Also, with 17+3 on 21 nodes, then a two node failure means it is
continually degraded*
until a host comes back up, but with k+m being < 15 would mean that the
cluster can
repair itself onto those ~15 nodes and still be in full swing even if up to
6 nodes would
be lost, however improbable such a failure would be.
*) So the +3 normally means three hosts could fail, but having only 19
hosts on a 17+3
would mean it can never reach active+clean for any PG, since there
aren't 20 hosts to
use. You wouldn't lose data, but all PGs would be degraded/undersized
meaning you
run the whole cluster at 17+2 at best.
--
May the most significant bit of your life be positive.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1372
days inactive
1373
days old
2 comments
3 participants
participants (3)
-
David Orman -
Janne Johansson -
Zhenshi Zhou