Thanks for your hints, we'll check asap our status and I'll update this thread. Il 19/05/2025 21:14, Eugen Block ha scritto:
Just a quick update: I set auth_allow_insecure_global_id_reclaim to false because all the client sessions we had showed either new_ok or reclaim_ok in global_id_status. No complaints so far. :-)
Zitat von Eugen Block <eblock@nde.ag>:
The mon sessions dump also shows the global_id_status, this could help identify the clients to focus on. I would recommend to inspect this output:
ceph tell mon.<MON> sessions | jq -r '.[] | .name,.socket_addr.addr,.global_id_status'
and check each client.
While looking at our own mon sessions, it seems like I could probably disable auth_allow_insecure_global_id_reclaim, looks like all older clients are either new_ok or reclaim_ok. And we still have plenty of "con_features_release" entries with "luminous", just FYI. I don't really recall doing anything special with our clients (also OpenStack servers), tbh. We upgraded them along with the Ceph versions, and now it looks okay. Maybe a reboot after an upgrade suffices, I don't know.
Here's the list of possible global_id_status entries with a short comment:
https://github.com/ceph/ceph/blob/df7b75547aab3bb055fff16588cd697e593dc19a/s...
Zitat von Sergio Rabellino <rabellino@di.unito.it>:
Probably yes, but the list it's too long, so in the middle there's a "..." : that's why I searched for a ceph command able to return all the results for session properties without ellipsing.
For this purpose, in my search I found the command "ceph daemon mon-name sessions" were I saw the "luminous" word that in my mind was "wrong" from my top post of this thread.
Il 16/05/2025 14:56, Eugen Block ha scritto:
If auth_expose_insecure_global_id_reclaim is true, do you see the client in 'ceph health detail'?
Zitat von Sergio Rabellino <rabellino@di.unito.it>:
this is the output by now that allows normal operation. Changing the third breaks client access (and maybe other bad behaviour)
root@juju-XXX7ca-YY62-lxd-0:~# for i in $(ceph config ls | grep global_id); do echo "Config: $i"; ceph config get mon $i; done Config: mon_warn_on_insecure_global_id_reclaim true Config: mon_warn_on_insecure_global_id_reclaim_allowed true Config: auth_allow_insecure_global_id_reclaim true Config: auth_expose_insecure_global_id_reclaim true We could disable the warnings, but is this affordable ?
Il 16/05/2025 13:45, Eugen Block ha scritto:
It's been a while since I dealt with all that global_id stuff:
https://docs.ceph.com/en/octopus/rados/operations/health-checks/#auth-insecu... Can you share this output?
for i in $(ceph config ls | grep global_id); do echo "Config: $i"; ceph config get mon $i; done Config: mon_warn_on_insecure_global_id_reclaim true Config: mon_warn_on_insecure_global_id_reclaim_allowed false Config: auth_allow_insecure_global_id_reclaim true Config: auth_expose_insecure_global_id_reclaim true
We still allow older clients, but we're aware and don't want the warning.
Zitat von Sergio Rabellino <rabellino@di.unito.it>:
> Sorry to bother the list with old questions again, I'm a little > new into this list. > > But if 'ceph daemon mon.name sessions' does not reports the > actual protocol used, how I can find this info ? > > Both the configs you asked for are "true": are you saying that > I'm concerned on null problem ? > > If so (and it may be), why if we remove the allow of > insecure_global_id on mon(s) all the VMs stop talking with the > underlying volumes ? > > Bests. > > > Il 15/05/2025 11:39, Eugen Block ha scritto: >> The output of 'ceph features' (or the mon session output) does >> not reflect that it's an older client. It just represents a >> collection of supported features. This has been discussed >> multiple times on this list. This is not unusual. >> >> Regarding the insecure global_id, does your cluster warn about >> it? Can you share these configs? >> >> ceph config get mon mon_warn_on_insecure_global_id_reclaim_allowed >> ceph config get mon mon_warn_on_insecure_global_id_reclaim >> >> >> Zitat von Sergio Rabellino <rabellino@di.unito.it>: >> >>> Thanks for your prompt reply, dump follows: >>> >>>> # ceph osd dump |grep require_osd_release >>>> require_osd_release octopus >>> We are aware of this setting and change it on release change >>> (we did in the last days mimic->nautilus->octopus). >>> The very strange thing it's that we upgraded luminous->mimic >>> more than two years ago. >>> >>> About librados, I get: >>>> # rados -v >>>> ceph version 15.2.17 >>>> (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable) >>> so it seems fine to me. >>> >>> Il 15/05/2025 11:01, Sinan Polat ha scritto: >>>> What is the output of: >>>> ceph osd dump | grep require_osd_release >>>> >>>> Have you upgraded OpenStack (librados) as well? >>>> >>>> Op do 15 mei 2025 om 10:50 schreef Sergio Rabellino >>>> <rabellino@di.unito.it <mailto:rabellino@di.unito.it>>: >>>> >>>> Dear list, >>>> >>>> we are upgrading our ceph infrastructure from mimic to >>>> octopus >>>> (please >>>> be kind, we known that we are working with "old" tools, >>>> but these >>>> ceph >>>> releases are tied to our openstack installation needs) and >>>> _*all*_ >>>> the >>>> ceph actors (mon/mgr/osd/rgw - no mds as we do not serve >>>> filesystem) >>>> were upgraded fine as follow: >>>> >>>> > # ceph -v >>>> > ceph version 15.2.17 >>>> (8a82819d84cf884bd39c17e3236e0632ac146dc4) >>>> > octopus (stable) >>>> We're using a ubuntu/juju orchestrator for managing ceph (and >>>> openstack >>>> too). >>>> >>>> It seems all ok, but if I ask a mon to dump the sessions, >>>> we get >>>> that_all of them_ are on luminous: >>>> >>>> > "con_features": 4540138292840890367, >>>> > "con_features_hex": "3f01cfb8ffedffff", >>>> > "con_features_release": "luminous", >>>> We found this oddity when we tried to unset the "insecure >>>> global_id >>>> reclaim" flag and all things got broken, so we had to >>>> reactivate >>>> the flag. >>>> >>>> All ceph network layers are "closed", so we're not so >>>> urged to remove >>>> the flag, but we would like to understand if this is a known >>>> problem or >>>> an error done by us. >>>> >>>> Any hints ? >>>> >>>> thanks in advance. >>>> >>>> -- ing. Sergio Rabellino >>>> >>>> Università degli Studi di Torino >>>> Dipartimento di Informatica >>>> Tecnico di Ricerca >>>> Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 >>>> C.so Svizzera , 185 - 10149 - Torino >>>> >>>> <http://www.di.unito.it> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users@ceph.io >>>> <mailto:ceph-users@ceph.io> >>>> To unsubscribe send an email to ceph-users-leave@ceph.io >>>> <mailto:ceph-users-leave@ceph.io> >>>> >>> >>> -- >>> ing. Sergio Rabellino >>> >>> Università degli Studi di Torino >>> Dipartimento di Informatica >>> Tecnico di Ricerca >>> Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 >>> C.so Svizzera , 185 - 10149 - Torino >>> >>> <http://www.di.unito.it> >>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-leave@ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-leave@ceph.io >> > > -- > ing. Sergio Rabellino > > Università degli Studi di Torino > Dipartimento di Informatica > Tecnico di Ricerca > Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 > C.so Svizzera , 185 - 10149 - Torino > > <http://www.di.unito.it>
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- ing. Sergio Rabellino
Università degli Studi di Torino Dipartimento di Informatica Tecnico di Ricerca Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 C.so Svizzera , 185 - 10149 - Torino
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- ing. Sergio Rabellino
Università degli Studi di Torino Dipartimento di Informatica Tecnico di Ricerca Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 C.so Svizzera , 185 - 10149 - Torino
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- ing. Sergio Rabellino Università degli Studi di Torino Dipartimento di Informatica Tecnico di Ricerca Cel +39-342-529-5409 Tel +39-011-670-6701 Fax +39-011-751603 C.so Svizzera , 185 - 10149 - Torino <http://www.di.unito.it>