25 May
2023
25 May
'23
9:23 p.m.
Dear Ceph community,
on our way towards getting our cluster to a current Ceph release, we updated all hosts and
clients to Nautilus 14.2.22. But despite setting `ceph osd set-require-min-compat-client
nautilus`, the release reported by `ceph features` is still "luminous".
Is this supposed to be like this? If not, does anyone have an idea what might be missing
to make the features being reported as "nautilus" as well?
```
~ # ceph mon dump
epoch 66
fsid b67bad36-3273-11e3-a2ed-0200000311bf
last_changed 2022-12-12 12:20:39.244333
created 2013-10-11 14:57:32.291514
min_mon_release 14 (nautilus)
0: [v2:172.20.4.10:3300/0,v1:172.20.4.10:6789/0] mon.host1
1: [v2:172.20.4.100:3300/0,v1:172.20.4.100:6789/0] mon.host2
2: [v2:172.20.4.101:3300/0,v1:172.20.4.101:6789/0] mon.host3
dumped monmap epoch 66
~ # ceph features
{
"mon": [
{
"features": "0x3ffddff8ffecffff",
"release": "luminous",
"num": 3
}
],
"osd": [
{
"features": "0x3ffddff8ffecffff",
"release": "luminous",
"num": 14
}
],
"client": [
{
"features": "0x3ffddff8ffecffff",
"release": "luminous",
"num": 137
}
],
"mgr": [
{
"features": "0x3ffddff8ffecffff",
"release": "luminous",
"num": 3
}
]
}
```
All the best
--
Oliver Schmidt · os(a)flyingcircus.io · Systems Engineer
Flying Circus Internet Operations GmbH · http://flyingcircus.io
Leipziger Str. 70/71 · 06108 Halle (Saale) · Deutschland
HR Stendal HRB 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick
10:14 p.m.
Hi Marc,
I think for an upgrade the rocksdb is necessary. Check this for your monitors
cat /var/lib/ceph/mon/ceph-a/kv_backend
Thanks, but I already had migrated all mons to use rocksdb when upgrading to Luminous.
~ # cat /srv/ceph/mon/ceph-host1/kv_backend
rocksdb
Is this what you expect here?
Best regards
Oliver
10:49 p.m.
Fairly confident this is normal. I just checked a pacific cluster and they
all report luminous as well. I think some of the backstory of this is
luminous is the release where up-maps were released and there hasnt been a
reason to increment the features release of subsequent daemons.
To be honest I am not confident that "ceph osd
set-require-min-compat-client nautilus" is a necessary step for you. What
prompted you to run that command?
That step is not listed here:
https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-…
but its been a bit since ive operated a pre-nautilus release.
Respectfully,
*Wes Dillingham*
wes(a)wesdillingham.com
LinkedIn <http://www.linkedin.com/in/wesleydillingham>
On Thu, May 25, 2023 at 3:14 PM Oliver Schmidt <os(a)flyingcircus.io> wrote:
Hi Marc,
I think for an upgrade the rocksdb is necessary. Check this for your
monitors
cat /var/lib/ceph/mon/ceph-a/kv_backend
Thanks, but I already had migrated all mons to use rocksdb when upgrading
to Luminous.
~ # cat /srv/ceph/mon/ceph-host1/kv_backend
rocksdb
Is this what you expect here?
Best regards
Oliver
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:22 p.m.
This is my understanding as well: as with CRUSH tunable sets, features *happen* to be
named after releases don't always correlate 1:1.
On May 25, 2023, at 15:49, Wesley Dillingham
<wes(a)wesdillingham.com> wrote:
Fairly confident this is normal. I just checked a pacific cluster and they
all report luminous as well. I think some of the backstory of this is
luminous is the release where up-maps were released and there hasnt been a
reason to increment the features release of subsequent daemons.
To be honest I am not confident that "ceph osd
set-require-min-compat-client nautilus" is a necessary step for you. What
prompted you to run that command?
That step is not listed here:
https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-…
but its been a bit since ive operated a pre-nautilus release.
Respectfully,
*Wes Dillingham*
wes(a)wesdillingham.com
LinkedIn <http://www.linkedin.com/in/wesleydillingham>
On Thu, May 25, 2023 at 3:14 PM Oliver Schmidt <os(a)flyingcircus.io> wrote:
Hi Marc,
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
I think for an upgrade the rocksdb is necessary. Check this for your
monitors
cat /var/lib/ceph/mon/ceph-a/kv_backend
Thanks, but I already had migrated all mons to use rocksdb when upgrading
to Luminous.
~ # cat /srv/ceph/mon/ceph-host1/kv_backend
rocksdb
Is this what you expect here?
Best regards
Oliver
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:44 p.m.
To be honest I am not confident that "ceph osd
set-require-min-compat-client nautilus" is a necessary step for you. What prompted
you to run that command?
That step is not listed here:
https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-…
You're correct indeed, it's neither listed in the docs for upgrading to Mimic nor
Nautilus. It apparently just slipped over from my Luminous upgrade checklist, which I
based the Nautilus upgrade steps upon.
Anthony D'Atri <aad(a)dreamsnake.net>
features *happen* to be named after releases don't
always correlate 1:1.
That somehow makes sense, but can be a bit confusing. It's good to hear that
apparently I did not miss any steps. I wonder whether this should be documented somewhere,
and where the chances of folks actually finding it are the best. (docs, manpage)
Kind regards
Oliver
26 May
26 May
11:40 a.m.
Hi Oliver,
there is a little bit more to it. The feature flags tell you also what version a client
should be at a minimum - a bit as indicated by the command name
set-require-*min-compat-client*. All clusters allow old clients to connect, but there is a
minimum compatibility cap. If you increase the minimum version requirement, older clients
might not be able to connect any more. Therefore, such min-compat-version increases only
happen when there is a reason. The new cluster + old client combo is very common,
specifically with ceph fs kclients. We have our cluster on octopus and clients reporting
as kraken.
Your current setting simply says that clients all the way back to luminous will be able to
connect and any client older than that might get refused.
In other words, there are always 2 versions: those of your daemons (cluster version) and
those supported to connect (client versions). You should limit downwards compatibility
only if there is a reason for doing so.
Best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14
________________________________________
From: Oliver Schmidt <os(a)flyingcircus.io>
Sent: Thursday, May 25, 2023 10:44 PM
To: Wesley Dillingham
Cc: ceph-users(a)ceph.io; aad(a)dreamsnake.net
Subject: [ceph-users] Re: `ceph features` on Nautilus still reports "luminous"
To be honest I am not confident that "ceph osd
set-require-min-compat-client nautilus" is a necessary step for you. What prompted
you to run that command?
That step is not listed here:
https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-…
You're correct indeed, it's neither listed in the docs for upgrading to Mimic nor
Nautilus. It apparently just slipped over from my Luminous upgrade checklist, which I
based the Nautilus upgrade steps upon.
Anthony D'Atri <aad(a)dreamsnake.net>
features *happen* to be named after releases don't
always correlate 1:1.
That somehow makes sense, but can be a bit confusing. It's good to hear that
apparently I did not miss any steps. I wonder whether this should be documented somewhere,
and where the chances of folks actually finding it are the best. (docs, manpage)
Kind regards
Oliver
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
343
days inactive
344
days old
6 comments
5 participants
participants (5)
-
Anthony D'Atri -
Frank Schilder -
Marc -
Oliver Schmidt -
Wesley Dillingham