osd recommended scheduler
Hello everyone, Could some one please let me know what is the recommended modern kernel disk scheduler that should be used for SSD and HDD osds? The information in the manuals is pretty dated and refer to the schedulers which have been deprecated from the recent kernels. Thanks Andrei
Bump ----- Original Message -----
From: "andrei" <andrei@arhont.com> To: "ceph-users" <ceph-users@ceph.io> Sent: Thursday, 28 January, 2021 17:09:23 Subject: [ceph-users] osd recommended scheduler
Hello everyone,
Could some one please let me know what is the recommended modern kernel disk scheduler that should be used for SSD and HDD osds? The information in the manuals is pretty dated and refer to the schedulers which have been deprecated from the recent kernels.
Thanks
Andrei _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On 28/01/2021 18:09, Andrei Mikhailovsky wrote:
Hello everyone,
Could some one please let me know what is the recommended modern kernel disk scheduler that should be used for SSD and HDD osds? The information in the manuals is pretty dated and refer to the schedulers which have been deprecated from the recent kernels.
Afaik noop is usually the one use for Flash devices. CFQ is used on HDDs most of the time as it allows for better scheduling/QoS. Wido
Thanks
Andrei _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi! Interesting. Didn't know that. We are running cfq on the OSDs, and added to ceph.conf:
[osd] osd_disk_thread_ioprio_class = idle osd_disk_thread_ioprio_priority = 7
Since we recently switched from HDD to SSD OSDs, I guess we should also change from CFQ to noop. Is there also something we need to change accordingly in ceph.conf? We simply added to rc.local:
echo cfq > /sys/block/sda/queue/scheduler .... echo cfq > /sys/block/sdf/queue/scheduler
Anything else to do, besides changing cfq to noop in the above..? Thanks for the tip! MJ On 2/2/21 8:44 AM, Wido den Hollander wrote:
On 28/01/2021 18:09, Andrei Mikhailovsky wrote:
Hello everyone,
Could some one please let me know what is the recommended modern kernel disk scheduler that should be used for SSD and HDD osds? The information in the manuals is pretty dated and refer to the schedulers which have been deprecated from the recent kernels.
Afaik noop is usually the one use for Flash devices.
CFQ is used on HDDs most of the time as it allows for better scheduling/QoS.
Wido
Thanks
Andrei _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks for your reply, Wido. Isn't CFQ being deprecated in the latest kernel versions? From what I've read in the Ubuntu support pages, the cfq, deadline and noop are no longer supported since 2019 / kernel version 5.3 and later. There are, however, the following schedulers: bfq, kyber, mq-deadline and none. Could someone please suggest which of these new schedulers does ceph team recommend using for HDD drives and SSD drives? We have both drive types in use. Many thanks Andrei ----- Original Message -----
From: "Wido den Hollander" <wido@42on.com> To: "Andrei Mikhailovsky" <andrei@arhont.com>, "ceph-users" <ceph-users@ceph.io> Sent: Tuesday, 2 February, 2021 07:44:13 Subject: Re: [ceph-users] osd recommended scheduler
On 28/01/2021 18:09, Andrei Mikhailovsky wrote:
Hello everyone,
Could some one please let me know what is the recommended modern kernel disk scheduler that should be used for SSD and HDD osds? The information in the manuals is pretty dated and refer to the schedulers which have been deprecated from the recent kernels.
Afaik noop is usually the one use for Flash devices.
CFQ is used on HDDs most of the time as it allows for better scheduling/QoS.
Wido
Thanks
Andrei _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
cfq and now bfq are the only IO schedulers that implement fair share across processes, and also they are the only schedulers that implement io priorities (e.g. ionice). We run this script via rc.local on all our ceph clusters: https://gist.github.com/dvanders/968d5862f227e0dd988eb5db8fbba203 -- dan On Tue, Feb 2, 2021 at 11:08 AM Andrei Mikhailovsky <andrei@arhont.com> wrote:
Thanks for your reply, Wido.
Isn't CFQ being deprecated in the latest kernel versions? From what I've read in the Ubuntu support pages, the cfq, deadline and noop are no longer supported since 2019 / kernel version 5.3 and later. There are, however, the following schedulers: bfq, kyber, mq-deadline and none. Could someone please suggest which of these new schedulers does ceph team recommend using for HDD drives and SSD drives? We have both drive types in use.
Many thanks
Andrei
----- Original Message -----
From: "Wido den Hollander" <wido@42on.com> To: "Andrei Mikhailovsky" <andrei@arhont.com>, "ceph-users" <ceph-users@ceph.io> Sent: Tuesday, 2 February, 2021 07:44:13 Subject: Re: [ceph-users] osd recommended scheduler
On 28/01/2021 18:09, Andrei Mikhailovsky wrote:
Hello everyone,
Could some one please let me know what is the recommended modern kernel disk scheduler that should be used for SSD and HDD osds? The information in the manuals is pretty dated and refer to the schedulers which have been deprecated from the recent kernels.
Afaik noop is usually the one use for Flash devices.
CFQ is used on HDDs most of the time as it allows for better scheduling/QoS.
Wido
Thanks
Andrei _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (4)
-
Andrei Mikhailovsky
-
Dan van der Ster
-
mj
-
Wido den Hollander