Just a quick update: I set auth_allow_insecure_global_id_reclaim to false because all the client sessions we had showed either new_ok or reclaim_ok in global_id_status. No complaints so far. :-) Zitat von Eugen Block <eblock@nde.ag>:
The mon sessions dump also shows the global_id_status, this could help identify the clients to focus on. I would recommend to inspect this output:
ceph tell mon.<MON> sessions | jq -r '.[] | .name,.socket_addr.addr,.global_id_status'
and check each client.
While looking at our own mon sessions, it seems like I could probably disable auth_allow_insecure_global_id_reclaim, looks like all older clients are either new_ok or reclaim_ok. And we still have plenty of "con_features_release" entries with "luminous", just FYI. I don't really recall doing anything special with our clients (also OpenStack servers), tbh. We upgraded them along with the Ceph versions, and now it looks okay. Maybe a reboot after an upgrade suffices, I don't know.
Here's the list of possible global_id_status entries with a short comment:
https://github.com/ceph/ceph/blob/df7b75547aab3bb055fff16588cd697e593dc19a/s...
Zitat von Sergio Rabellino <rabellino@di.unito.it>:
Probably yes, but the list it's too long, so in the middle there's a "..." : that's why I searched for a ceph command able to return all the results for session properties without ellipsing.
For this purpose, in my search I found the command "ceph daemon mon-name sessions" were I saw the "luminous" word that in my mind was "wrong" from my top post of this thread.
Il 16/05/2025 14:56, Eugen Block ha scritto:
If auth_expose_insecure_global_id_reclaim is true, do you see the client in 'ceph health detail'?
Zitat von Sergio Rabellino <rabellino@di.unito.it>:
this is the output by now that allows normal operation. Changing the third breaks client access (and maybe other bad behaviour)
root@juju-XXX7ca-YY62-lxd-0:~# for i in $(ceph config ls | grep global_id); do echo "Config: $i"; ceph config get mon $i; done Config: mon_warn_on_insecure_global_id_reclaim true Config: mon_warn_on_insecure_global_id_reclaim_allowed true Config: auth_allow_insecure_global_id_reclaim true Config: auth_expose_insecure_global_id_reclaim true We could disable the warnings, but is this affordable ?
Il 16/05/2025 13:45, Eugen Block ha scritto:
It's been a while since I dealt with all that global_id stuff:
https://docs.ceph.com/en/octopus/rados/operations/health-checks/#auth-insecu... Can you share this output?
for i in $(ceph config ls | grep global_id); do echo "Config: $i"; ceph config get mon $i; done Config: mon_warn_on_insecure_global_id_reclaim true Config: mon_warn_on_insecure_global_id_reclaim_allowed false Config: auth_allow_insecure_global_id_reclaim true Config: auth_expose_insecure_global_id_reclaim true
We still allow older clients, but we're aware and don't want the warning.
Zitat von Sergio Rabellino <rabellino@di.unito.it>:
Sorry to bother the list with old questions again, I'm a little new into this list.
But if 'ceph daemon mon.name sessions' does not reports the actual protocol used, how I can find this info ?
Both the configs you asked for are "true": are you saying that I'm concerned on null problem ?
If so (and it may be), why if we remove the allow of insecure_global_id on mon(s) all the VMs stop talking with the underlying volumes ?
Bests.
Il 15/05/2025 11:39, Eugen Block ha scritto: > The output of 'ceph features' (or the mon session output) does > not reflect that it's an older client. It just represents a > collection of supported features. This has been discussed > multiple times on this list. This is not unusual. > > Regarding the insecure global_id, does your cluster warn about > it? Can you share these configs? > > ceph config get mon mon_warn_on_insecure_global_id_reclaim_allowed > ceph config get mon mon_warn_on_insecure_global_id_reclaim > > > Zitat von Sergio Rabellino <rabellino@di.unito.it>: > >> Thanks for your prompt reply, dump follows: >> >>> # ceph osd dump |grep require_osd_release >>> require_osd_release octopus >> We are aware of this setting and change it on release change >> (we did in the last days mimic->nautilus->octopus). >> The very strange thing it's that we upgraded luminous->mimic >> more than two years ago. >> >> About librados, I get: >>> # rados -v >>> ceph version 15.2.17 >>> (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable) >> so it seems fine to me. >> >> Il 15/05/2025 11:01, Sinan Polat ha scritto: >>> What is the output of: >>> ceph osd dump | grep require_osd_release >>> >>> Have you upgraded OpenStack (librados) as well? >>> >>> Op do 15 mei 2025 om 10:50 schreef Sergio Rabellino >>> <rabellino@di.unito.it <mailto:rabellino@di.unito.it>>: >>> >>> Dear list, >>> >>> we are upgrading our ceph infrastructure from mimic to octopus >>> (please >>> be kind, we known that we are working with "old" tools, but these >>> ceph >>> releases are tied to our openstack installation needs) and _*all*_ >>> the >>> ceph actors (mon/mgr/osd/rgw - no mds as we do not serve >>> filesystem) >>> were upgraded fine as follow: >>> >>> > # ceph -v >>> > ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) >>> > octopus (stable) >>> We're using a ubuntu/juju orchestrator for managing ceph (and >>> openstack >>> too). >>> >>> It seems all ok, but if I ask a mon to dump the sessions, we get >>> that_all of them_ are on luminous: >>> >>> > "con_features": 4540138292840890367, >>> > "con_features_hex": "3f01cfb8ffedffff", >>> > "con_features_release": "luminous", >>> We found this oddity when we tried to unset the "insecure >>> global_id >>> reclaim" flag and all things got broken, so we had to reactivate >>> the flag. >>> >>> All ceph network layers are "closed", so we're not so >>> urged to remove >>> the flag, but we would like to understand if this is a known >>> problem or >>> an error done by us. >>> >>> Any hints ? >>> >>> thanks in advance. >>> >>> -- ing. Sergio Rabellino >>> >>> Università degli Studi di Torino >>> Dipartimento di Informatica >>> Tecnico di Ricerca >>> Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 >>> C.so Svizzera , 185 - 10149 - Torino >>> >>> <http://www.di.unito.it> >>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> <mailto:ceph-users@ceph.io> >>> To unsubscribe send an email to ceph-users-leave@ceph.io >>> <mailto:ceph-users-leave@ceph.io> >>> >> >> -- >> ing. Sergio Rabellino >> >> Università degli Studi di Torino >> Dipartimento di Informatica >> Tecnico di Ricerca >> Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 >> C.so Svizzera , 185 - 10149 - Torino >> >> <http://www.di.unito.it> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-leave@ceph.io > > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io >
-- ing. Sergio Rabellino
Università degli Studi di Torino Dipartimento di Informatica Tecnico di Ricerca Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 C.so Svizzera , 185 - 10149 - Torino
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- ing. Sergio Rabellino
Università degli Studi di Torino Dipartimento di Informatica Tecnico di Ricerca Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 C.so Svizzera , 185 - 10149 - Torino
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- ing. Sergio Rabellino
Università degli Studi di Torino Dipartimento di Informatica Tecnico di Ricerca Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 C.so Svizzera , 185 - 10149 - Torino
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io