We noticed extremely slow performance when remapping is necessary. We didn't do anything special other than assigning the correct device_class (to ssd). When checking ceph status, we notice the number of objects recovering is around 17-25 (with watch -n 1 -c ceph status). How can we increase the recovery process? There isn't any client load, because we're going to migrate to this cluster in the future, so only a rsync once a while is being executed. [ceph: root@pwsoel12998 /]# ceph status cluster: id: da3ca2e4-ee5b-11ed-8096-0050569e8c3b health: HEALTH_WARN noscrub,nodeep-scrub flag(s) set services: mon: 5 daemons, quorum pqsoel12997,pqsoel12996,pwsoel12994,pwsoel12998,prghygpl03 (age 3h) mgr: pwsoel12998.ylvjcb(active, since 3h), standbys: pqsoel12997.gagpbt mds: 4/4 daemons up, 2 standby osd: 32 osds: 32 up (since 73m), 32 in (since 6d); 10 remapped pgs flags noscrub,nodeep-scrub data: volumes: 2/2 healthy pools: 5 pools, 193 pgs objects: 13.97M objects, 853 GiB usage: 3.5 TiB used, 12 TiB / 16 TiB avail pgs: 755092/55882956 objects misplaced (1.351%) 183 active+clean 10 active+remapped+backfilling io: recovery: 2.3 MiB/s, 20 objects/s
Hello, This is a known issue in quincy which uses mClock scheduler. There is a fix for this which should be available in 17.2.6+ releases. You can confirm the active scheduler type on any osd using: ceph config show osd.0 osd_op_queue If the active scheduler is 'mclock_scheduler', you can try switching the mClock profile to 'high_recovery_ops' on all OSDs to speed up the backfilling using: ceph config set osd osd_mclock_profile high_recovery_ops After the backfilling is complete, you can switch the mClock profile back to the default value using: ceph config rm osd osd_mclock_profile On Tue, May 16, 2023 at 4:46 PM Sake Paulusma <sake1989@hotmail.com> wrote:
We noticed extremely slow performance when remapping is necessary. We didn't do anything special other than assigning the correct device_class (to ssd). When checking ceph status, we notice the number of objects recovering is around 17-25 (with watch -n 1 -c ceph status).
How can we increase the recovery process?
There isn't any client load, because we're going to migrate to this cluster in the future, so only a rsync once a while is being executed.
[ceph: root@pwsoel12998 /]# ceph status cluster: id: da3ca2e4-ee5b-11ed-8096-0050569e8c3b health: HEALTH_WARN noscrub,nodeep-scrub flag(s) set
services: mon: 5 daemons, quorum pqsoel12997,pqsoel12996,pwsoel12994,pwsoel12998,prghygpl03 (age 3h) mgr: pwsoel12998.ylvjcb(active, since 3h), standbys: pqsoel12997.gagpbt mds: 4/4 daemons up, 2 standby osd: 32 osds: 32 up (since 73m), 32 in (since 6d); 10 remapped pgs flags noscrub,nodeep-scrub
data: volumes: 2/2 healthy pools: 5 pools, 193 pgs objects: 13.97M objects, 853 GiB usage: 3.5 TiB used, 12 TiB / 16 TiB avail pgs: 755092/55882956 objects misplaced (1.351%) 183 active+clean 10 active+remapped+backfilling
io: recovery: 2.3 MiB/s, 20 objects/s
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Sridhar Seshasayee Partner Engineer Red Hat <https://www.redhat.com> <https://www.redhat.com>
Hi, The config shows "mclock_scheduler" and I already switched to the high_recovery_ops, this does increase the recovery ops, but only a little. You mention there is a fix in 17.2.6+, but we're running on 17.2.6 (this cluster is created on this version). Any more ideas? Best regards
Just to add: high_client_ops: around 8-13 objects per second high_recovery_ops: around 17-25 objects per second Both observed with "watch - n 1 - c ceph status" Best regards
Hi Sake, We are experiencing the same. I set “osd_mclock_cost_per_byte_usec_hdd” to 0.1 (default is 2.6) and get about 15 times backfill speed, without significant affect client IO. This parameter seems calculated wrongly, from the description 5e-3 should be a reasonable value for HDD (corresponding to 200MB/s). I noticed this default is originally 5.2, then changed to 2.6 to increase the recovery speed. So I suspect the original author just convert the unit wrongly, he may want 5.2e-3 but wrote 5.2 in code. But all this may be not important in the next version. I see the relevant code is rewritten, and this parameter is now removed. high_recovery_ops profile works very poorly for us. It increase the average latency of client IO from 50ms to about 1s. Weiwen Hu 在 2023年5月16日，19:16，Sake Paulusma <sake1989@hotmail.com> 写道： We noticed extremely slow performance when remapping is necessary. We didn't do anything special other than assigning the correct device_class (to ssd). When checking ceph status, we notice the number of objects recovering is around 17-25 (with watch -n 1 -c ceph status). How can we increase the recovery process? There isn't any client load, because we're going to migrate to this cluster in the future, so only a rsync once a while is being executed. [ceph: root@pwsoel12998 /]# ceph status cluster: id: da3ca2e4-ee5b-11ed-8096-0050569e8c3b health: HEALTH_WARN noscrub,nodeep-scrub flag(s) set services: mon: 5 daemons, quorum pqsoel12997,pqsoel12996,pwsoel12994,pwsoel12998,prghygpl03 (age 3h) mgr: pwsoel12998.ylvjcb(active, since 3h), standbys: pqsoel12997.gagpbt mds: 4/4 daemons up, 2 standby osd: 32 osds: 32 up (since 73m), 32 in (since 6d); 10 remapped pgs flags noscrub,nodeep-scrub data: volumes: 2/2 healthy pools: 5 pools, 193 pgs objects: 13.97M objects, 853 GiB usage: 3.5 TiB used, 12 TiB / 16 TiB avail pgs: 755092/55882956 objects misplaced (1.351%) 183 active+clean 10 active+remapped+backfilling io: recovery: 2.3 MiB/s, 20 objects/s _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks for the input! Changing this value we indeed increased the recovery speed from 20 object per second to 500! Now something strange: 1. We needed to change the class for our drives manually to ssd. 2. The setting "osd_mclock_max_capacity_iops_ssd" was set to 0. With osd bench descriped in the documentation, we configured the value 10000 for the ssd parameter. Only nothing changed. 3. But when setting "osd_mclock_max_capacity_iops_hdd" to 10000, the recovery speed also increased dramatically! I don't understand this anymore :( Is the mclock scheduler ignoring the override of the device class?
Did an extra test with shutting down an osd host and force a recovery. Only using the iops setting I got 500 objects a second, but using also the bytes_per_usec setting, I got 1200 objects a second! Maybe there should also be an investigation about this performance issue. Best regards
This is interesting, and it arrived minutes after I had replaced an HDD OSD (with NVME DB/WAL) in a small cluster. With the three profiles i was only seeing objects/second of around 6-8 (high_client_ops), 9-12 (balanced), 12-15 (high_recovery_ops). There was only a very light client load. With a bit of ad-hoc experimenting I ended up setting osd_mclock_cost_per_byte_usec_hdd to 0.4 which seemed to give about a 7-8 times increase in objects/second for each profile, without unduly affecting client response if I used high_client_ops. Values lower than 0.4 did speed up backfill, but disk saturation started to become an issue. These are only very rough-and-ready figures, but they do suggest something rather wrong in the calculations. Chris On 16/05/2023 19:07, 胡 玮文 wrote:
Hi Sake,
We are experiencing the same. I set “osd_mclock_cost_per_byte_usec_hdd” to 0.1 (default is 2.6) and get about 15 times backfill speed, without significant affect client IO. This parameter seems calculated wrongly, from the description 5e-3 should be a reasonable value for HDD (corresponding to 200MB/s). I noticed this default is originally 5.2, then changed to 2.6 to increase the recovery speed. So I suspect the original author just convert the unit wrongly, he may want 5.2e-3 but wrote 5.2 in code.
But all this may be not important in the next version. I see the relevant code is rewritten, and this parameter is now removed.
high_recovery_ops profile works very poorly for us. It increase the average latency of client IO from 50ms to about 1s.
Weiwen Hu
在 2023年5月16日，19:16，Sake Paulusma <sake1989@hotmail.com> 写道：
We noticed extremely slow performance when remapping is necessary. We didn't do anything special other than assigning the correct device_class (to ssd). When checking ceph status, we notice the number of objects recovering is around 17-25 (with watch -n 1 -c ceph status).
How can we increase the recovery process?
There isn't any client load, because we're going to migrate to this cluster in the future, so only a rsync once a while is being executed.
[ceph: root@pwsoel12998 /]# ceph status cluster: id: da3ca2e4-ee5b-11ed-8096-0050569e8c3b health: HEALTH_WARN noscrub,nodeep-scrub flag(s) set
services: mon: 5 daemons, quorum pqsoel12997,pqsoel12996,pwsoel12994,pwsoel12998,prghygpl03 (age 3h) mgr: pwsoel12998.ylvjcb(active, since 3h), standbys: pqsoel12997.gagpbt mds: 4/4 daemons up, 2 standby osd: 32 osds: 32 up (since 73m), 32 in (since 6d); 10 remapped pgs flags noscrub,nodeep-scrub
data: volumes: 2/2 healthy pools: 5 pools, 193 pgs objects: 13.97M objects, 853 GiB usage: 3.5 TiB used, 12 TiB / 16 TiB avail pgs: 755092/55882956 objects misplaced (1.351%) 183 active+clean 10 active+remapped+backfilling
io: recovery: 2.3 MiB/s, 20 objects/s
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On 17/05/2023 03.07, 胡 玮文 wrote:
Hi Sake,
We are experiencing the same. I set “osd_mclock_cost_per_byte_usec_hdd” to 0.1 (default is 2.6) and get about 15 times backfill speed, without significant affect client IO. This parameter seems calculated wrongly, from the description 5e-3 should be a reasonable value for HDD (corresponding to 200MB/s). I noticed this default is originally 5.2, then changed to 2.6 to increase the recovery speed. So I suspect the original author just convert the unit wrongly, he may want 5.2e-3 but wrote 5.2 in code.
But all this may be not important in the next version. I see the relevant code is rewritten, and this parameter is now removed.
high_recovery_ops profile works very poorly for us. It increase the average latency of client IO from 50ms to about 1s.
Weiwen Hu
Thank you for this, that parameter indeed seems completely wrong (assuming it means what it says on the tin). After changing that my Quincy cluster is no recovering at a much more reasonable speed. - Hector
Someone who's got data regarding this should file a bug report, it sounds like a quick fix for defaults if this holds true. On Sat, May 20, 2023, at 00:59, Hector Martin wrote:
On 17/05/2023 03.07, 胡 玮文 wrote:
Hi Sake,
We are experiencing the same. I set “osd_mclock_cost_per_byte_usec_hdd” to 0.1 (default is 2.6) and get about 15 times backfill speed, without significant affect client IO. This parameter seems calculated wrongly, from the description 5e-3 should be a reasonable value for HDD (corresponding to 200MB/s). I noticed this default is originally 5.2, then changed to 2.6 to increase the recovery speed. So I suspect the original author just convert the unit wrongly, he may want 5.2e-3 but wrote 5.2 in code.
But all this may be not important in the next version. I see the relevant code is rewritten, and this parameter is now removed.
high_recovery_ops profile works very poorly for us. It increase the average latency of client IO from 50ms to about 1s.
Weiwen Hu
Thank you for this, that parameter indeed seems completely wrong (assuming it means what it says on the tin). After changing that my Quincy cluster is no recovering at a much more reasonable speed.
- Hector _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
We're on 17.2.5 and had the default value (5.2), but changing it didn't seem to impact recovery speed: root@rdx-00:/# ceph config get osd osd_mclock_cost_per_byte_usec_hdd 5.200000 root@rdx-00:/# ceph config show osd.0 osd_op_queue mclock_scheduler root@rdx-00:/# ceph config set osd osd_mclock_cost_per_byte_usec_hdd 0.4 root@rdx-00:/# ceph config get osd osd_mclock_cost_per_byte_usec_hdd 0.400000 root@rdx-00:/# We have about 500TB of misplaced objects moving about so we are keen to speed it up; was there another parameter that needed to be changed too? thanks. On Mon, 22 May 2023 at 23:24, David Orman <ormandj@corenode.com> wrote:
Someone who's got data regarding this should file a bug report, it sounds like a quick fix for defaults if this holds true.
On 17/05/2023 03.07, 胡 玮文 wrote:
Hi Sake,
We are experiencing the same. I set “osd_mclock_cost_per_byte_usec_hdd” to 0.1 (default is 2.6) and get about 15 times backfill speed, without significant affect client IO. This parameter seems calculated wrongly, from
On Sat, May 20, 2023, at 00:59, Hector Martin wrote: the description 5e-3 should be a reasonable value for HDD (corresponding to 200MB/s). I noticed this default is originally 5.2, then changed to 2.6 to increase the recovery speed. So I suspect the original author just convert the unit wrongly, he may want 5.2e-3 but wrote 5.2 in code.
But all this may be not important in the next version. I see the
relevant code is rewritten, and this parameter is now removed.
high_recovery_ops profile works very poorly for us. It increase the
average latency of client IO from 50ms to about 1s.
Weiwen Hu
Thank you for this, that parameter indeed seems completely wrong (assuming it means what it says on the tin). After changing that my Quincy cluster is no recovering at a much more reasonable speed.
- Hector _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I found that I could get much more aggressive with the osd_mclock_cost_per_byte_usec_hdd values if my osd_mclock_max_capacity_iops_hdd numbers were set to sane values. If I used the (very wrong) default calculated values I would start getting slow ops. Setting all hdd in my environment to 300 seemed to calm that down. I'm running the mclock profile of high_client_ops at the moment. I do get better recovery on balanced but I saw client latency go up enough where it wasn't worth the trade-off in my environment at this time. I'm still slowly walking the cost number down looking for the sweet spot. It seems to be very sensitive. A change from .75 to .70 changed my IOPS/BW from 480/1.48GB/s to 515/1.57GB/s. It's easy to overshoot and hit some other bottleneck. --- Paul Mezzanini Platform Engineer III Research Computing Rochester Institute of Technology “End users is a description, not a goal.” ________________________________________ From: Nigel Williams <nigel.williams@tpac.org.au> Sent: Tuesday, May 23, 2023 12:19 AM To: David Orman Cc: ceph-users@ceph.io Subject: [ceph-users] Re: Slow recovery on Quincy We're on 17.2.5 and had the default value (5.2), but changing it didn't seem to impact recovery speed: root@rdx-00:/# ceph config get osd osd_mclock_cost_per_byte_usec_hdd 5.200000 root@rdx-00:/# ceph config show osd.0 osd_op_queue mclock_scheduler root@rdx-00:/# ceph config set osd osd_mclock_cost_per_byte_usec_hdd 0.4 root@rdx-00:/# ceph config get osd osd_mclock_cost_per_byte_usec_hdd 0.400000 root@rdx-00:/# We have about 500TB of misplaced objects moving about so we are keen to speed it up; was there another parameter that needed to be changed too? thanks. On Mon, 22 May 2023 at 23:24, David Orman <ormandj@corenode.com> wrote:
Someone who's got data regarding this should file a bug report, it sounds like a quick fix for defaults if this holds true.
On 17/05/2023 03.07, 胡 玮文 wrote:
Hi Sake,
We are experiencing the same. I set “osd_mclock_cost_per_byte_usec_hdd” to 0.1 (default is 2.6) and get about 15 times backfill speed, without significant affect client IO. This parameter seems calculated wrongly, from
On Sat, May 20, 2023, at 00:59, Hector Martin wrote: the description 5e-3 should be a reasonable value for HDD (corresponding to 200MB/s). I noticed this default is originally 5.2, then changed to 2.6 to increase the recovery speed. So I suspect the original author just convert the unit wrongly, he may want 5.2e-3 but wrote 5.2 in code.
But all this may be not important in the next version. I see the
relevant code is rewritten, and this parameter is now removed.
high_recovery_ops profile works very poorly for us. It increase the
average latency of client IO from 50ms to about 1s.
Weiwen Hu
Thank you for this, that parameter indeed seems completely wrong (assuming it means what it says on the tin). After changing that my Quincy cluster is no recovering at a much more reasonable speed.
- Hector _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
As someone in this thread noted, the cost related config options are removed in the next version (ceph-17.2.6-45.el9cp). The cost parameters may not work in all cases due to the inherent differences in the underlying device types and other external factors. With the endeavor to achieve a more hands free operation, there are significant improvements in the next version that should help resolve issues described in this thread. I am listing the significant ones below: 1. The mClock profile QoS parameters (reservation and limit) are now simplified and are specified in terms of a fraction of the OSD's IOPS capacity rather than in terms of IOPS. 2. The default mclock profile is now changed to the 'balanced' profile which gives equal priority to client and background OSD operations. 3. The allocations for different types of OSD ops within mClock profiles are changed. 4. The earlier cost related config parameters are removed. The cost is now determined by the OSD based on the underlying device characteristics, i.e., its 4 KiB random IOPS capacity and the device's max sequential bandwidth. - The random IOPS capacity is determined using 'osd bench' as before, but now based on the result, unrealistic values are not considered and reasonable defaults are used if the measurement crosses a threshold governed by *osd_mclock_iops_capacity_threshold_[hdd**|ssd]. *The default IOPS capacity may be overridden by users if not accurate, The thresholds too are configurable. The max sequential bandwidth is defined by *osd_mclock_max_sequential_bandwidth_[hdd|ssd*], and are set to reasonable defaults. Again, these may be modified if not accurate. Therefore, these changes account for inaccuracies and provide good control to the user in terms of specifying accurate OSD characteristics. 5. Degraded recoveries are given more priority than backfills. Therefore, you may observe faster degraded recovery rates compared to backfills with the 'balanced' and 'high_client_ops' profile. But the backfills will still show healthy rates when compared to the slow backfill rates mentioned in this thread. For faster recovery and backfills, the 'high_recovery_ops' profile with modified QoS parameters would help. Please see the latest upstream documentation for more details: https://docs.ceph.com/en/quincy/rados/configuration/mclock-config-ref/ The recommendation is to upgrade when feasible and provide your feedback, questions and suggestions. -Sridhar
I'm on 17.2.6, but the option "osd_mclock_max_sequential_bandwidth_hdd" isn't available when I try to set it via "ceph config set osd.0 osd_mclock_max_sequential_bandwidth_hdd 500Mi". I need to use large numbers for hdd, because it looks like the mclock scheduler isn't using the device class override value. Best regards, Sake ________________________________ From: Sridhar Seshasayee <sseshasa@redhat.com> Sent: Wednesday, May 24, 2023 11:34:02 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Slow recovery on Quincy As someone in this thread noted, the cost related config options are removed in the next version (ceph-17.2.6-45.el9cp). The cost parameters may not work in all cases due to the inherent differences in the underlying device types and other external factors. With the endeavor to achieve a more hands free operation, there are significant improvements in the next version that should help resolve issues described in this thread. I am listing the significant ones below: 1. The mClock profile QoS parameters (reservation and limit) are now simplified and are specified in terms of a fraction of the OSD's IOPS capacity rather than in terms of IOPS. 2. The default mclock profile is now changed to the 'balanced' profile which gives equal priority to client and background OSD operations. 3. The allocations for different types of OSD ops within mClock profiles are changed. 4. The earlier cost related config parameters are removed. The cost is now determined by the OSD based on the underlying device characteristics, i.e., its 4 KiB random IOPS capacity and the device's max sequential bandwidth. - The random IOPS capacity is determined using 'osd bench' as before, but now based on the result, unrealistic values are not considered and reasonable defaults are used if the measurement crosses a threshold governed by *osd_mclock_iops_capacity_threshold_[hdd**|ssd]. *The default IOPS capacity may be overridden by users if not accurate, The thresholds too are configurable. The max sequential bandwidth is defined by *osd_mclock_max_sequential_bandwidth_[hdd|ssd*], and are set to reasonable defaults. Again, these may be modified if not accurate. Therefore, these changes account for inaccuracies and provide good control to the user in terms of specifying accurate OSD characteristics. 5. Degraded recoveries are given more priority than backfills. Therefore, you may observe faster degraded recovery rates compared to backfills with the 'balanced' and 'high_client_ops' profile. But the backfills will still show healthy rates when compared to the slow backfill rates mentioned in this thread. For faster recovery and backfills, the 'high_recovery_ops' profile with modified QoS parameters would help. Please see the latest upstream documentation for more details: https://docs.ceph.com/en/quincy/rados/configuration/mclock-config-ref/ The recommendation is to upgrade when feasible and provide your feedback, questions and suggestions. -Sridhar _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I'm on 17.2.6, but the option "osd_mclock_max_sequential_bandwidth_hdd" > isn't available when I try to set it via "ceph config set osd.0 > osd_mclock_max_sequential_bandwidth_hdd 500Mi". > > Can you paste the output of 1. ceph version 2. ceph config show-with-defaults osd.0 | grep osd_mclock 3. ceph config show osd.0 | grep osd_mclock I just want to verify if your Ceph version has the changes.
If I glance at the commits to the quincy branch, shouldn't the mentioned configurations be included in 17.2.7? The requested command output: [ceph: root@mgrhost1 /]# ceph version ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy (stable) [ceph: root@mgrhost1 /]# ceph config show-with-defaults osd.0 | grep osd_mclock osd_mclock_cost_per_byte_usec 0.000000 default osd_mclock_cost_per_byte_usec_hdd 0.100000 mon osd_mclock_cost_per_byte_usec_ssd 0.100000 mon osd_mclock_cost_per_io_usec 0.000000 default osd_mclock_cost_per_io_usec_hdd 11400.000000 default osd_mclock_cost_per_io_usec_ssd 50.000000 default osd_mclock_force_run_benchmark_on_init false default osd_mclock_iops_capacity_threshold_hdd 500.000000 default osd_mclock_iops_capacity_threshold_ssd 80000.000000 default osd_mclock_max_capacity_iops_hdd 10000.000000 mon osd_mclock_max_capacity_iops_ssd 0.000000 override mon[10000.000000] osd_mclock_override_recovery_settings false mon osd_mclock_profile high_client_ops mon osd_mclock_scheduler_anticipation_timeout 0.000000 default osd_mclock_scheduler_background_best_effort_lim 999999 default osd_mclock_scheduler_background_best_effort_res 500 default osd_mclock_scheduler_background_best_effort_wgt 2 default osd_mclock_scheduler_background_recovery_lim 2000 default osd_mclock_scheduler_background_recovery_res 500 default osd_mclock_scheduler_background_recovery_wgt 1 default osd_mclock_scheduler_client_lim 999999 default osd_mclock_scheduler_client_res 1000 default osd_mclock_scheduler_client_wgt 2 default osd_mclock_skip_benchmark false default [ceph: root@mgrhost1 /]# ceph config show osd.0 | grep osd_mclock osd_mclock_cost_per_byte_usec_hdd 0.100000 mon osd_mclock_cost_per_byte_usec_ssd 0.100000 mon osd_mclock_max_capacity_iops_hdd 10000.000000 mon osd_mclock_max_capacity_iops_ssd 0.000000 override mon[10000.000000] osd_mclock_override_recovery_settings false mon osd_mclock_profile high_client_ops mon osd_mclock_scheduler_background_best_effort_lim 999999 default osd_mclock_scheduler_background_best_effort_res 500 default osd_mclock_scheduler_background_best_effort_wgt 2 default osd_mclock_scheduler_background_recovery_lim 2000 default osd_mclock_scheduler_background_recovery_res 500 default osd_mclock_scheduler_background_recovery_wgt 1 default osd_mclock_scheduler_client_lim 999999 default osd_mclock_scheduler_client_res 1000 default osd_mclock_scheduler_client_wgt 2 default
Yes, the fix should be in the next quincy upstream version. The version I posted was the downstream one.
Thanks, will keep an eye out for this version. Will report back to this thread about these options and the recovery time/number of objects per second for recovery. Again, thank you'll for the information and answers!
participants (8)
-
Chris Palmer
-
David Orman
-
Hector Martin
-
Nigel Williams
-
Paul Mezzanini
-
Sake Paulusma
-
Sridhar Seshasayee
-
胡 玮文