21 Jan
2021
21 Jan
'21
2:57 a.m.
I have hell of the question: how to make HEALTH_ERR status for a cluster
without consequences?
I'm working on CI tests and I need to check if our reaction to
HEALTH_ERR is good. For this I need to take an empty cluster with an
empty pool and do something. Preferably quick and reversible.
For HEALTH_WARN the best thing I found is to change pool size to 1, it
raises "1 pool(s) have no replicas configured" warning almost instantly
and it can be reverted very quickly for empty pool.
But HEALTH_ERR is a bit more tricky. Any ideas?
21 Jan
21 Jan
3:02 a.m.
Hi,
For HEALTH_WARN the best thing I found is to change
pool size to 1,
it raises "1 pool(s) have no replicas configured" warning almost
instantly and it can be reverted very quickly for empty pool.
any osd flag (noout, nodeep-scrub etc.) cause health warnings. ;-)
But HEALTH_ERR is a bit more tricky. Any ideas?
I think if you set a very low quota for a pool (e.g. 1000 bytes or so)
and fill it up it should create a HEALTH_ERR status, IIRC.
Zitat von George Shuklin <george.shuklin(a)gmail.com>om>:
> I have hell of the question: how to make HEALTH_ERR status for a
> cluster without consequences?
>
> I'm working on CI tests and I need to check if our reaction to
> HEALTH_ERR is good. For this I need to take an empty cluster with an
> empty pool and do something. Preferably quick and reversible.
>
For HEALTH_WARN the best thing I found is to change
pool size to 1,
it raises "1 pool(s) have no replicas configured" warning almost
instantly and it can be reverted very quickly for empty pool.
>
But HEALTH_ERR is a bit more tricky. Any ideas?
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
3:09 a.m.
On 21/01/2021 13:02, Eugen Block wrote:
Cool idea.
Unfortunately, even with 1 byte quota (and some data in the
pool), it's HEALTH_WARN, 1 pool(s) full
But HEALTH_ERR
is a bit more tricky. Any ideas?
I think if you set a very low quota for a pool (e.g. 1000 bytes or so)
and fill it up it should create a HEALTH_ERR status, IIRC.
4:37 a.m.
On 21/01/2021 12:57, George Shuklin wrote:
I have hell of the question: how to make HEALTH_ERR
status for a
cluster without consequences?
I'm working on CI tests and I need to check if our reaction to
HEALTH_ERR is good. For this I need to take an empty cluster with an
empty pool and do something. Preferably quick and reversible.
For HEALTH_WARN the best thing I found is to change pool size to 1, it
raises "1 pool(s) have no replicas configured" warning almost
instantly and it can be reverted very quickly for empty pool.
But HEALTH_ERR is a bit more tricky. Any ideas?
I found the way:
ceph osd set-full-ratio 0.0
instantly causing
health: HEALTH_ERR
full ratio(s) out of order
even on empty cluster. Problem solved.
4:38 a.m.
Oh really, I thought it would be an error. My bad.
There was an osd flag "full" which is not usable anymore, I never used
it so I just tried it with full OSD which should lead to an error (and
it does):
host:~ # ceph -s
cluster:
id: 8f279f36-811c-3270-9f9d-58335b1bb9c0
health: HEALTH_ERR
1 full osd(s)
22 pool(s) full
A created a pool with 1 pg and size 1, created an rbd image in that
pool and filled that up until the respective OSD was full. I have a
virtual lab cluster with 20GB OSDs so it didn't take that long. If you
try this, make sure to disable pg-autoscaler on that pool or otherwise
it will increase pg-num. Does that help?
Regards,
Eugen
Zitat von George Shuklin <george.shuklin(a)gmail.com>om>:
On 21/01/2021 13:02, Eugen Block wrote:
Cool idea.
Unfortunately, even with 1 byte quota (and some data in
the pool), it's HEALTH_WARN, 1 pool(s) full
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
But
HEALTH_ERR is a bit more tricky. Any ideas?
I think if you set a very low quota for a pool (e.g. 1000 bytes or
so) and fill it up it should create a HEALTH_ERR status, IIRC.
4:51 a.m.
Oh that's better, I had to recreate my OSD because it didn't want to
start anymore :-D
Zitat von George Shuklin <george.shuklin(a)gmail.com>om>:
On 21/01/2021 12:57, George Shuklin wrote:
I have hell of the question: how to make
HEALTH_ERR status for a
cluster without consequences?
I'm working on CI tests and I need to check if our reaction to
HEALTH_ERR is good. For this I need to take an empty cluster with
an empty pool and do something. Preferably quick and reversible.
For HEALTH_WARN the best thing I found is to change pool size to 1,
it raises "1 pool(s) have no replicas configured" warning almost
instantly and it can be reverted very quickly for empty pool.
But HEALTH_ERR is a bit more tricky. Any ideas?
I found the way:
ceph osd set-full-ratio 0.0
instantly causing
health: HEALTH_ERR
full ratio(s) out of order
even on empty cluster. Problem solved.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1189
days inactive
1189
days old
5 comments
2 participants
participants (2)
-
Eugen Block -
George Shuklin