Nautilus upgrade HEALTH_WARN legacy tunables
I just upgraded from Luminous to Nautilus. the cluster was originally hammer or kraken (can't recall), then jewel, on to luminous, and now Nautilus. The message i get from ceph -s is: "health: HEALTH_WARN crush map has legacy tunables (require firefly, min is hammer)" Not sure how to get rid of it. I tried "ceph osd set-require-min-compat-client firefly" per instructions but still have error. When I query osd: ceph osd dump | grep min_compat_client require_min_compat_client firefly min_compat_client firefly Instructions say I can ignore the error, but that seems like a bad idea. Windows VM's are running on qemu/kvm via rbd connection. Ceph and KVM are running CentOS7. Just upgraded all clients to "centos-release-7-8.2003.0.el7.centos.x86_64" I have rebooted all nodes since CentOS upgrade and again after Nautilus upgrade. tried setting "ceph osd set-require-min-compat-client jewel" but then heath_warn was still present and: ceph osd dump | grep min_compat_client require_min_compat_client jewel min_compat_client firefly Any help is appreciated.
min_compat is a different thing entirely. You need to set the tunables as a group. This will cause data to move, so you may wish to throttle recovery, model the PG movement ahead of time, use the upmap trick to control movement etc. https://ceph.io/geen-categorie/set-tunables-optimal-on-ceph-crushmap/
On Jul 4, 2020, at 11:01 AM, jimf@mninc.net wrote:
I just upgraded from Luminous to Nautilus. the cluster was originally hammer or kraken (can't recall), then jewel, on to luminous, and now Nautilus. The message i get from ceph -s is: "health: HEALTH_WARN crush map has legacy tunables (require firefly, min is hammer)" Not sure how to get rid of it. I tried "ceph osd set-require-min-compat-client firefly" per instructions but still have error. When I query osd: ceph osd dump | grep min_compat_client require_min_compat_client firefly min_compat_client firefly Instructions say I can ignore the error, but that seems like a bad idea. Windows VM's are running on qemu/kvm via rbd connection. Ceph and KVM are running CentOS7. Just upgraded all clients to "centos-release-7-8.2003.0.el7.centos.x86_64" I have rebooted all nodes since CentOS upgrade and again after Nautilus upgrade.
tried setting "ceph osd set-require-min-compat-client jewel" but then heath_warn was still present and: ceph osd dump | grep min_compat_client require_min_compat_client jewel min_compat_client firefly
Any help is appreciated. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Den sön 5 juli 2020 kl 00:15 skrev Anthony D'Atri <anthony.datri@gmail.com>:
min_compat is a different thing entirely. You need to set the tunables as a group. This will cause data to move, so you may wish to throttle recovery, model the PG movement ahead of time, use the upmap trick to control movement etc.
https://ceph.io/geen-categorie/set-tunables-optimal-on-ceph-crushmap/
Not sure, but I don't think you can use upmap before your min level is luminous at least. -- May the most significant bit of your life be positive.
Thanks for the help. That page had sort of the answer. I had tried "ceph osd crush tunables optimal" earlier but I got an error: "Error EINVAL: new crush map requires client version jewel but require_min_compat_client is firefly" The links for help on that page are dead, but I did end up finding this page. https://docs.ceph.com/docs/master/rados/operations/upmap/ I am not sure what upmap is, or the upmap trick, but i was able to resolve with the info on this page. First I ran "ceph features" and made sure all clients were listed as luminous. Second I ran "ceph osd set-require-min-compat-client luminous" Then I was able to successfully run "ceph osd crush tunables optimal" without error. My cluster is 19 total TiB theree was 54% objects misplaced. Took about 4 hours to re-balance. Cluster is not really being used much because of Covid and Sunday, so a good time to do it. HEALTH_OK :) Thanks again!
participants (4)
-
Anthony D'Atri
-
Janne Johansson
-
Jim Forde
-
jimf＠mninc.net