4 Sep
2019
4 Sep
'19
9:44 a.m.
Hi,
I am using ceph version 13.2.6 (mimic) on test setup trying with cephfs.
my ceph health status showing warning.
My current setup:
3 OSD node each with a single disk, recently I added one more disk in one
of the node and ceph cluster status showing warning.
I can see the progress but it was more than 12 hours but still its moving
objects.
How to increase the speed of moving objects?
output from "ceph -s"
cluster:
id: 7c138e13-7b98-4309-b591-d4091a1742b4
health: HEALTH_WARN
834820/7943361 objects misplaced (10.510%)
services:
mon: 1 daemons, quorum mon01
mgr: mon01(active)
mds: cephfs-tst-1/1/1 up {0=mon01=up:active}
osd: 4 osds: 4 up, 4 in; 12 remapped pgs
data:
pools: 2 pools, 64 pgs
objects: 2.65 M objects, 178 GiB
usage: 548 GiB used, 6.7 TiB / 7.3 TiB avail
pgs: 834820/7943361 objects misplaced (10.510%)
52 active+clean
11 active+remapped+backfill_wait
1 active+remapped+backfilling
io:
recovery: 0 B/s, 6 objects/s
output from "ceph osd df "
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
0 hdd 1.81940 1.00000 1.8 TiB 88 GiB 1.7 TiB 4.71 0.64 40
3 hdd 1.81940 1.00000 1.8 TiB 96 GiB 1.7 TiB 5.15 0.70 24
1 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
2 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
TOTAL 7.3 TiB 548 GiB 6.7 TiB 7.36
MIN/MAX VAR: 0.64/1.33 STDDEV: 2.43
regards
Amudhan P
4 Sep
4 Sep
10:41 a.m.
Den ons 4 sep. 2019 kl 11:41 skrev Amudhan P <amudhan83(a)gmail.com>om>:
Hi,
I am using ceph version 13.2.6 (mimic) on test setup trying with cephfs.
my ceph health status showing warning.
My current setup:
3 OSD node each with a single disk, recently I added one more disk in one
of the node and ceph cluster status showing warning.
I can see the progress but it was more than 12 hours but still its moving
objects.
How to increase the speed of moving objects?
If the disks can handle it, increase the max_backfills (seem to be 1 now)
to allow more than one OSD to feed data into the new OSD.
Also, more drives mean more parallelism when moving data around, so the
smaller the cluster is, the less it can spread the load of moving data.
--
May the most significant bit of your life be positive.
5:45 p.m.
Hey hey,
First of all: 10GBps connection.
Then, some magic commands:
# ceph tell 'osd.*' injectargs '--osd-max-backfills 32'
# ceph tell 'osd.*' injectargs '--osd-recovery-max-active 12'
# ceph tell 'osd.*' injectargs '--osd-recovery-op-priority 63'
=D
[]'s
Arthur (aKa Guilherme Geronimo)
On 04/09/2019 06:44, Amudhan P wrote:
Hi,
I am using ceph version 13.2.6 (mimic) on test setup trying with cephfs.
my ceph health status showing warning.
My current setup:
3 OSD node each with a single disk, recently I added one more disk in
one of the node and ceph cluster status showing warning.
I can see the progress but it was more than 12 hours but still its
moving objects.
How to increase the speed of moving objects?
output from "ceph -s"
cluster:
id: 7c138e13-7b98-4309-b591-d4091a1742b4
health: HEALTH_WARN
834820/7943361 objects misplaced (10.510%)
services:
mon: 1 daemons, quorum mon01
mgr: mon01(active)
mds: cephfs-tst-1/1/1 up {0=mon01=up:active}
osd: 4 osds: 4 up, 4 in; 12 remapped pgs
data:
pools: 2 pools, 64 pgs
objects: 2.65 M objects, 178 GiB
usage: 548 GiB used, 6.7 TiB / 7.3 TiB avail
pgs: 834820/7943361 objects misplaced (10.510%)
52 active+clean
11 active+remapped+backfill_wait
1 active+remapped+backfilling
io:
recovery: 0 B/s, 6 objects/s
output from "ceph osd df "
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
0 hdd 1.81940 1.00000 1.8 TiB 88 GiB 1.7 TiB 4.71 0.64 40
3 hdd 1.81940 1.00000 1.8 TiB 96 GiB 1.7 TiB 5.15 0.70 24
1 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
2 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
TOTAL 7.3 TiB 548 GiB 6.7 TiB 7.36
MIN/MAX VAR: 0.64/1.33 STDDEV: 2.43
regards
Amudhan P
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
8:09 p.m.
how about also increasing osd_recovery_threads?
On Wed, Sep 4, 2019 at 10:47 AM Guilherme Geronimo <
guilherme.geronimo(a)gmail.com> wrote:
Hey hey,
First of all: 10GBps connection.
Then, some magic commands:
# ceph tell 'osd.*' injectargs '--osd-max-backfills 32'
# ceph tell 'osd.*' injectargs '--osd-recovery-max-active 12'
# ceph tell 'osd.*' injectargs '--osd-recovery-op-priority 63'
=D
[]'s
Arthur (aKa Guilherme Geronimo)
On 04/09/2019 06:44, Amudhan P wrote:
Hi,
I am using ceph version 13.2.6 (mimic) on test setup trying with cephfs.
my ceph health status showing warning.
My current setup:
3 OSD node each with a single disk, recently I added one more disk in one
of the node and ceph cluster status showing warning.
I can see the progress but it was more than 12 hours but still its moving
objects.
How to increase the speed of moving objects?
output from "ceph -s"
cluster:
id: 7c138e13-7b98-4309-b591-d4091a1742b4
health: HEALTH_WARN
834820/7943361 objects misplaced (10.510%)
services:
mon: 1 daemons, quorum mon01
mgr: mon01(active)
mds: cephfs-tst-1/1/1 up {0=mon01=up:active}
osd: 4 osds: 4 up, 4 in; 12 remapped pgs
data:
pools: 2 pools, 64 pgs
objects: 2.65 M objects, 178 GiB
usage: 548 GiB used, 6.7 TiB / 7.3 TiB avail
pgs: 834820/7943361 objects misplaced (10.510%)
52 active+clean
11 active+remapped+backfill_wait
1 active+remapped+backfilling
io:
recovery: 0 B/s, 6 objects/s
output from "ceph osd df "
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
0 hdd 1.81940 1.00000 1.8 TiB 88 GiB 1.7 TiB 4.71 0.64 40
3 hdd 1.81940 1.00000 1.8 TiB 96 GiB 1.7 TiB 5.15 0.70 24
1 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
2 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
TOTAL 7.3 TiB 548 GiB 6.7 TiB 7.36
MIN/MAX VAR: 0.64/1.33 STDDEV: 2.43
regards
Amudhan P
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
9:32 p.m.
I don't think MIMIC support this attribute:
https://docs.ceph.com/docs/mimic/rados/configuration/osd-config-ref/
[]'s
Arthur (aKa Guilherme Geronimo)
On 04/09/2019 17:09, solarflow99 wrote:
how about also increasing osd_recovery_threads?
On Wed, Sep 4, 2019 at 10:47 AM Guilherme Geronimo
<guilherme.geronimo(a)gmail.com <mailto:guilherme.geronimo@gmail.com>>
wrote:
Hey hey,
First of all: 10GBps connection.
Then, some magic commands:
# ceph tell 'osd.*' injectargs '--osd-max-backfills 32'
# ceph tell 'osd.*' injectargs '--osd-recovery-max-active 12'
# ceph tell 'osd.*' injectargs '--osd-recovery-op-priority 63'
=D
[]'s
Arthur (aKa Guilherme Geronimo)
On 04/09/2019 06:44, Amudhan P wrote:
Hi,
I am using ceph version 13.2.6 (mimic) on test setup trying with
cephfs.
my ceph health status showing warning.
My current setup:
3 OSD node each with a single disk, recently I added one more
disk in one of the node and ceph cluster status showing warning.
I can see the progress but it was more than 12 hours but still
its moving objects.
How to increase the speed of moving objects?
output from "ceph -s"
cluster:
id: 7c138e13-7b98-4309-b591-d4091a1742b4
health: HEALTH_WARN
834820/7943361 objects misplaced (10.510%)
services:
mon: 1 daemons, quorum mon01
mgr: mon01(active)
mds: cephfs-tst-1/1/1 up {0=mon01=up:active}
osd: 4 osds: 4 up, 4 in; 12 remapped pgs
data:
pools: 2 pools, 64 pgs
objects: 2.65 M objects, 178 GiB
usage: 548 GiB used, 6.7 TiB / 7.3 TiB avail
pgs: 834820/7943361 objects misplaced (10.510%)
52 active+clean
11 active+remapped+backfill_wait
1 active+remapped+backfilling
io:
recovery: 0 B/s, 6 objects/s
output from "ceph osd df "
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
0 hdd 1.81940 1.00000 1.8 TiB 88 GiB 1.7 TiB 4.71 0.64 40
3 hdd 1.81940 1.00000 1.8 TiB 96 GiB 1.7 TiB 5.15 0.70 24
1 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
2 hdd 1.81940 1.00000 1.8 TiB 182 GiB 1.6 TiB 9.79 1.33 64
TOTAL 7.3 TiB 548 GiB 6.7 TiB 7.36
MIN/MAX VAR: 0.64/1.33 STDDEV: 2.43
regards
Amudhan P
_______________________________________________
ceph-users mailing list --ceph-users(a)ceph.io <mailto:ceph-users@ceph.io>
To unsubscribe send an email toceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
<mailto:ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
1711
days inactive
1711
days old
4 comments
4 participants
participants (4)
-
Amudhan P -
Guilherme Geronimo -
Janne Johansson -
solarflow99