16 Feb
2021
16 Feb
'21
4:06 a.m.
Hi Matthew,
Which version are you upgrading from? If recent nautilus, you may have
already completed this conversion.
When we did this fsck (not with octopus, but to a nautilus point
release that had this conversion backported), we first upgraded one
single osd just to see the typical downtime for our data.
On our S3 cluster, the conversion completed within just a couple of
minutes per OSD, so we decided to leave
bluestore_fsck_quick_fix_on_mount at its default true, and did all the
fsck's as we updated the osds.
AFAIK, if you set bluestore_fsck_quick_fix_on_mount to false and
postpone that conversion, you'll just have less-accurate `ceph df` for
omap data.
Cheers, Dan
On Mon, Feb 15, 2021 at 6:00 PM Matthew Vernon <mv3(a)sanger.ac.uk> wrote:
>
> Hi,
>
> Looking at the Octopus upgrade instructions, I see "the first time each
> OSD starts, it will do a format conversion to improve the accounting for
> “omap” data. This may take a few minutes to as much as a few hours (for
> an HDD with lots of omap data)." and that I can disable this by setting
> bluestore_fsck_quick_fix_on_mount to false.
>
> A couple of questions about this:
>
> i) what are the consequences of turning off this "quick fix"? Is it
> possible to have it run in the background or similar?
>
> ii) is there any way to narrow down the time estimate? Our production
> cluster has 3060 OSDs on hdd (with block.db on NVME), and obviously 3000
> lots of "a few hours" is an awful lot of time...
>
> I'll be doing some testing on our test cluster (by putting 10M objects
> into an S3 bucket before trying the upgrade), but it'd be useful to have
> some idea of how this is likely to work at scale...
>
> Thanks,
>
> Matthew
>
>
> --
> The Wellcome Sanger Institute is operated by Genome Research
> Limited, a charity registered in England with number 1021457 and a
> company registered in England with number 2742969, whose registered
> office is 215 Euston Road, London, NW1 2BE.
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io