24 Feb
2020
24 Feb
'20
1:59 p.m.
Hi, I have a Nautilus installation version 14.2.1 with a very unbalanced
cephfs pool, I have 430 osd in the cluster but this pool only have 8 PG
and PGP and 118 TB used :
# ceph -s
cluster:
id: a2269da7-e399-484a-b6ae-4ee1a31a4154
health: HEALTH_WARN
1 nearfull osd(s)
2 pool(s) nearfull
services:
mon: 3 daemons, quorum mon21,mon22,mon23 (age 7M)
mgr: mon23(active, since 8M), standbys: mon22, mon21
mds: cephfs:2 {0=mon21=up:active,1=mon22=up:active} 1 up:standby
osd: 430 osds: 430 up, 430 in
data:
pools: 2 pools, 16 pgs
objects: 10.07M objects, 38 TiB
usage: 118 TiB used, 4.5 PiB / 4.6 PiB avail
pgs: 15 active+clean
1 active+clean+scrubbing+deep
# ceph osd pool get cephfs_data pg_num
pg_num: 8
Due to this bad configuration I have this warning message:
# ceph status
cluster:
id: a2269da7-e399-484a-b6ae-4ee1a31a4154
health: HEALTH_WARN
1 nearfull osd(s)
2 pool(s) nearfull
I've discover that some osd are full:
# ceph osd status
| 113 | osd23 | 9824G | 1351G | 0 | 0 | 0 | 0 |
exists,nearfull,up |
I've tried to reweight this osd:
ceph osd reweight osd.113 0.9
But the process of reweight doesn't start. Otherwise I've tried to
increase the PG and PGP numbers but it' doesn't work.
# ceph osd pool set cephfs_data pg_num 128
set pool 1 pg_num to 128
# ceph osd pool get cephfs_data pg_num
pg_num: 8
What could be the reason for this problem?
24 Feb
24 Feb
2:10 p.m.
Have you tried making a smaller increment instead of jumping from 8 to 128 as that is
quite a big leap?
2:44 p.m.
I have tried to increase to 16, with the same result:
# ceph osd pool set cephfs_data pg_num 16
set pool 1 pg_num to 16
# ceph osd pool get cephfs_data pg_num
pg_num: 8
El 24/2/20 a las 15:10, Gabryel Mason-Williams escribió:
Have you tried making a smaller increment instead of
jumping from 8 to 128 as that is quite a big leap?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
*******************************************************
Andrés Rojas Guerrero
Unidad Sistemas Linux
Area Arquitectura Tecnológica
Secretaría General Adjunta de Informática
Consejo Superior de Investigaciones Científicas (CSIC)
Pinar 19
28006 - Madrid
Tel: +34 915680059 -- Ext. 990059
email: a.rojas(a)csic.es
ID comunicate.csic.es: @50852720l:matrix.csic.es
*******************************************************
25 Feb
25 Feb
2:01 p.m.
I believe you are encountering https://tracker.ceph.com/issues/39570
You should do a "ceph versions" on a mon and ensure all your OSDs are
nautilus and if so set "ceph osd require-osd-release nautilus" then try to
increase pg num. Upgrading to a more recent nautilus release is also
probably a good idea.
Respectfully,
*Wes Dillingham*
wes(a)wesdillingham.com
LinkedIn
<http://www.linkedin.com/in/wesleydillingham>
On Mon, Feb 24, 2020 at 9:44 AM Andres Rojas Guerrero <a.rojas(a)csic.es>
wrote:
I have tried to increase to 16, with the same result:
# ceph osd pool set cephfs_data pg_num 16
set pool 1 pg_num to 16
# ceph osd pool get cephfs_data pg_num
pg_num: 8
El 24/2/20 a las 15:10, Gabryel Mason-Williams escribió:
Have you tried making a smaller increment instead
of jumping from 8 to
128 as that is quite a big leap?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
*******************************************************
Andrés Rojas Guerrero
Unidad Sistemas Linux
Area Arquitectura Tecnológica
Secretaría General Adjunta de Informática
Consejo Superior de Investigaciones Científicas (CSIC)
Pinar 19
28006 - Madrid
Tel: +34 915680059 -- Ext. 990059
email: a.rojas(a)csic.es
ID comunicate.csic.es: @50852720l:matrix.csic.es
*******************************************************
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
2:16 p.m.
That's right!! I will try to update, but now I have the desire PG numbers.
Thank you.
El 25/2/20 a las 15:01, Wesley Dillingham escribió:
I believe you are
encountering https://tracker.ceph.com/issues/39570
You should do a "ceph versions" on a mon and ensure all your OSDs are
nautilus and if so set "ceph osd require-osd-release nautilus" then try
to increase pg num. Upgrading to a more recent nautilus release is also
probably a good idea.
Respectfully,
*Wes Dillingham*
wes(a)wesdillingham.com <mailto:wes@wesdillingham.com>
LinkedIn
<http://www.linkedin.com/in/wesleydillingham>
On Mon, Feb 24, 2020 at 9:44 AM Andres Rojas Guerrero <a.rojas(a)csic.es
<mailto:a.rojas@csic.es>> wrote:
I have tried to increase to 16, with the same result:
# ceph osd pool set cephfs_data pg_num 16
set pool 1 pg_num to 16
# ceph osd pool get cephfs_data pg_num
pg_num: 8
El 24/2/20 a las 15:10, Gabryel Mason-Williams escribió:
--
*******************************************************
Andrés Rojas Guerrero
Unidad Sistemas Linux
Area Arquitectura Tecnológica
Secretaría General Adjunta de Informática
Consejo Superior de Investigaciones Científicas (CSIC)
Pinar 19
28006 - Madrid
Tel: +34 915680059 -- Ext. 990059
email: a.rojas(a)csic.es <mailto:a.rojas@csic.es>
ID comunicate.csic.es <http://comunicate.csic.es>:
@50852720l:matrix.csic.es <http://matrix.csic.es>
*******************************************************
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
<mailto:ceph-users@ceph.io>
To unsubscribe send an email to ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
--
*******************************************************
Andrés Rojas Guerrero
Unidad Sistemas Linux
Area Arquitectura Tecnológica
Secretaría General Adjunta de Informática
Consejo Superior de Investigaciones Científicas (CSIC)
Pinar 19
28006 - Madrid
Tel: +34 915680059 -- Ext. 990059
email: a.rojas(a)csic.es
ID comunicate.csic.es: @50852720l:matrix.csic.es
*******************************************************
Have you tried making a smaller increment instead
of jumping from
8 to 128 as that is quite a big leap?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
<mailto:ceph-users@ceph.io>
To unsubscribe send an email to
ceph-users-leave(a)ceph.io
<mailto:ceph-users-leave@ceph.io>
1541
days inactive
1542
days old
5 comments
3 participants
participants (3)
-
Andres Rojas Guerrero -
Gabryel Mason-Williams -
Wesley Dillingham