`ceph features` on Nautilus still reports "luminous"
Dear Ceph community, on our way towards getting our cluster to a current Ceph release, we updated all hosts and clients to Nautilus 14.2.22. But despite setting `ceph osd set-require-min-compat-client nautilus`, the release reported by `ceph features` is still "luminous". Is this supposed to be like this? If not, does anyone have an idea what might be missing to make the features being reported as "nautilus" as well? ``` ~ # ceph mon dump epoch 66 fsid b67bad36-3273-11e3-a2ed-0200000311bf last_changed 2022-12-12 12:20:39.244333 created 2013-10-11 14:57:32.291514 min_mon_release 14 (nautilus) 0: [v2:172.20.4.10:3300/0,v1:172.20.4.10:6789/0] mon.host1 1: [v2:172.20.4.100:3300/0,v1:172.20.4.100:6789/0] mon.host2 2: [v2:172.20.4.101:3300/0,v1:172.20.4.101:6789/0] mon.host3 dumped monmap epoch 66 ~ # ceph features { "mon": [ { "features": "0x3ffddff8ffecffff", "release": "luminous", "num": 3 } ], "osd": [ { "features": "0x3ffddff8ffecffff", "release": "luminous", "num": 14 } ], "client": [ { "features": "0x3ffddff8ffecffff", "release": "luminous", "num": 137 } ], "mgr": [ { "features": "0x3ffddff8ffecffff", "release": "luminous", "num": 3 } ] } ``` All the best -- Oliver Schmidt · os@flyingcircus.io · Systems Engineer Flying Circus Internet Operations GmbH · http://flyingcircus.io Leipziger Str. 70/71 · 06108 Halle (Saale) · Deutschland HR Stendal HRB 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick
Hi Marc,
I think for an upgrade the rocksdb is necessary. Check this for your monitors
cat /var/lib/ceph/mon/ceph-a/kv_backend
Thanks, but I already had migrated all mons to use rocksdb when upgrading to Luminous. ~ # cat /srv/ceph/mon/ceph-host1/kv_backend rocksdb Is this what you expect here? Best regards Oliver
Fairly confident this is normal. I just checked a pacific cluster and they all report luminous as well. I think some of the backstory of this is luminous is the release where up-maps were released and there hasnt been a reason to increment the features release of subsequent daemons. To be honest I am not confident that "ceph osd set-require-min-compat-client nautilus" is a necessary step for you. What prompted you to run that command? That step is not listed here: https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-l... but its been a bit since ive operated a pre-nautilus release. Respectfully, *Wes Dillingham* wes@wesdillingham.com LinkedIn <http://www.linkedin.com/in/wesleydillingham> On Thu, May 25, 2023 at 3:14 PM Oliver Schmidt <os@flyingcircus.io> wrote:
Hi Marc,
I think for an upgrade the rocksdb is necessary. Check this for your
monitors
cat /var/lib/ceph/mon/ceph-a/kv_backend
Thanks, but I already had migrated all mons to use rocksdb when upgrading to Luminous.
~ # cat /srv/ceph/mon/ceph-host1/kv_backend rocksdb
Is this what you expect here?
Best regards Oliver _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
This is my understanding as well: as with CRUSH tunable sets, features *happen* to be named after releases don't always correlate 1:1.
On May 25, 2023, at 15:49, Wesley Dillingham <wes@wesdillingham.com> wrote:
Fairly confident this is normal. I just checked a pacific cluster and they all report luminous as well. I think some of the backstory of this is luminous is the release where up-maps were released and there hasnt been a reason to increment the features release of subsequent daemons.
To be honest I am not confident that "ceph osd set-require-min-compat-client nautilus" is a necessary step for you. What prompted you to run that command?
That step is not listed here: https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-l...
but its been a bit since ive operated a pre-nautilus release.
Respectfully,
*Wes Dillingham* wes@wesdillingham.com LinkedIn <http://www.linkedin.com/in/wesleydillingham>
On Thu, May 25, 2023 at 3:14 PM Oliver Schmidt <os@flyingcircus.io> wrote:
Hi Marc,
I think for an upgrade the rocksdb is necessary. Check this for your
monitors
cat /var/lib/ceph/mon/ceph-a/kv_backend
Thanks, but I already had migrated all mons to use rocksdb when upgrading to Luminous.
~ # cat /srv/ceph/mon/ceph-host1/kv_backend rocksdb
Is this what you expect here?
Best regards Oliver _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
To be honest I am not confident that "ceph osd set-require-min-compat-client nautilus" is a necessary step for you. What prompted you to run that command?
That step is not listed here: https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-l...
You're correct indeed, it's neither listed in the docs for upgrading to Mimic nor Nautilus. It apparently just slipped over from my Luminous upgrade checklist, which I based the Nautilus upgrade steps upon. Anthony D'Atri <aad@dreamsnake.net>
features *happen* to be named after releases don't always correlate 1:1.
That somehow makes sense, but can be a bit confusing. It's good to hear that apparently I did not miss any steps. I wonder whether this should be documented somewhere, and where the chances of folks actually finding it are the best. (docs, manpage) Kind regards Oliver
Hi Oliver, there is a little bit more to it. The feature flags tell you also what version a client should be at a minimum - a bit as indicated by the command name set-require-*min-compat-client*. All clusters allow old clients to connect, but there is a minimum compatibility cap. If you increase the minimum version requirement, older clients might not be able to connect any more. Therefore, such min-compat-version increases only happen when there is a reason. The new cluster + old client combo is very common, specifically with ceph fs kclients. We have our cluster on octopus and clients reporting as kraken. Your current setting simply says that clients all the way back to luminous will be able to connect and any client older than that might get refused. In other words, there are always 2 versions: those of your daemons (cluster version) and those supported to connect (client versions). You should limit downwards compatibility only if there is a reason for doing so. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Oliver Schmidt <os@flyingcircus.io> Sent: Thursday, May 25, 2023 10:44 PM To: Wesley Dillingham Cc: ceph-users@ceph.io; aad@dreamsnake.net Subject: [ceph-users] Re: `ceph features` on Nautilus still reports "luminous"
To be honest I am not confident that "ceph osd set-require-min-compat-client nautilus" is a necessary step for you. What prompted you to run that command?
That step is not listed here: https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-l...
You're correct indeed, it's neither listed in the docs for upgrading to Mimic nor Nautilus. It apparently just slipped over from my Luminous upgrade checklist, which I based the Nautilus upgrade steps upon. Anthony D'Atri <aad@dreamsnake.net>
features *happen* to be named after releases don't always correlate 1:1.
That somehow makes sense, but can be a bit confusing. It's good to hear that apparently I did not miss any steps. I wonder whether this should be documented somewhere, and where the chances of folks actually finding it are the best. (docs, manpage) Kind regards Oliver _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (5)
-
Anthony D'Atri
-
Frank Schilder
-
Marc
-
Oliver Schmidt
-
Wesley Dillingham