25 Feb
2020
25 Feb
'20
3:33 p.m.
Hi Kristof,
just some thoughts/insight on the issue.
First of all it's not clear if you're going to migrate to EC 6+3 only or
downsizing allocation size as well.
Anyway I'd suggest to postpone these modifications for a while if
possible. Ceph core team is aware of both space overhead caused by tons
of small objects and in some scenarios when using EC pools.
For the latter see https://tracker.ceph.com/issues/44213
The general solution for both seems to be downsizing min alloc size for
spinners to 4K but currently this results in a significant performance
drop.
Some benchmarks for standalone BlueStore can be found here:
https://docs.google.com/spreadsheets/d/1DzTEumM2hCnUu9Ceib7GfM035zFmPgPHmXk…
A couple of PRs to avoid performance penalty are on the way:
https://github.com/ceph/ceph/pull/33365
https://github.com/ceph/ceph/pull/33434
Once they get into master we might consider backporting these changes to
Nautilus.
So my general point is that currently the work is in progress and IMO
it's better to wait until its completion.
Thanks,
Igor
On 2/24/2020 11:48 PM, Kristof Coucke wrote:
> Hi all,
>
> A while back, I indicated we had an issue with our cluster filling up too
> fast. After checking everything, we've concluded this was because we had a
> lot of small files and the allocation size on the bluestore was too high
> (64kb).
> We are now recreating the OSD's (2 disk at the same time) but, this will
> take a very long time as we're dealing with 130 OSDs.
> The current process we're following is removing 2 osd's and recreating them.
> We're using erasure coding (6 + 3).
>
> Has anyone some advice on how we can move forward with this? We've already
> increased some parameters to speed up recovery, but even then, it would
> still cost us too much time.
>
> If we could recreate them faster, that would be great... Or adapt the
> allocation on the fly?
>
> Any suggestions are welcome...
>
> Thank you,
>
> Kristof.
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io