rbd map error: couldn't connect to the cluster!

List overview All Threads
Download

newer

older

mons excessive writes to local...

CompleteMultipartUploadResult has...

Thomas Schneider

23 Feb 2023 23 Feb '23

2:12 p.m.

Hello, I'm trying to mount RBD using rbd map, but I get this error message: # rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring rbd: couldn't connect to the cluster! Checking on Ceph server the required permission for relevant keyring exists: # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring [client.VCT] key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== # ceph auth get client.VCT [client.VCT] key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== caps mon = "allow r" caps osd = "allow rwx pool hdb_backup object_prefix rbd_data.b768d4baac048b; allow rwx pool hdb_backup object_prefix rbd_header.b768d4baac048b; allow rx pool hdb_backup object_prefix rbd_id.VCT" exported keyring for client.VCT Can you please advise how to fix this error? THX

Show replies by date

Eugen Block

23 Feb 23 Feb

2:30 p.m.

You don't specify which client in your rbd command:

...

rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring

Have you tried this (not sure about upper-case client names, haven't tried that)? rbd map hdb_backup/VCT --id VCT --keyring /etc/ceph/ceph.client.VCT.keyring Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>:

...

Thomas Schneider

2:53 p.m.

Hm... I'm not sure about the correct rbd command syntax, but I thought it's correct. Anyway, using a different ID fails, too: # rbd map hdb_backup/VCT --id client.VCT --keyring /etc/ceph/ceph.client.VCT.keyring rbd: couldn't connect to the cluster! # rbd map hdb_backup/VCT --id VCT --keyring /etc/ceph/ceph.client.VCT.keyring 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 librbd::image::GetMetadataRequest: 0x7f220c001ef0 handle_metadata_list: failed to retrieve image metadata: (1) Operation not permitted 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 librbd::image::RefreshRequest: failed to retrieve pool metadata: (1) Operation not permitted 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 librbd::image::OpenRequest: failed to refresh image: (1) Operation not permitted 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 librbd::ImageState: 0x5569d8a16ba0 failed to open image: (1) Operation not permitted rbd: error opening image VCT: (1) Operation not permitted Am 23.02.2023 um 15:30 schrieb Eugen Block:

...

You don't specify which client in your rbd command:

rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Eugen Block

3:01 p.m.

For rbd commands you don't specify the "client" prefix for the --id parameter, just the client name, in your case "VCT". Your second approach shows a different error message, so it can connect with "VCT" successfully, but the permissions seem not to be sufficient. Those caps look very restrictive, not sure which prevent the map command though. Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>:

...

You don't specify which client in your rbd command:

rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

3:08 p.m.

I'll delete existing authentication and its caps "VCT" and recreate it. Just to be sure: there's no ingress communication to the client (from Ceph server)? Am 23.02.2023 um 16:01 schrieb Eugen Block:

...

You don't specify which client in your rbd command:

rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Eugen Block

3:40 p.m.

...

I'll delete existing authentication and its caps "VCT" and recreate it. Just to be sure: there's no ingress communication to the client (from Ceph server)? Am 23.02.2023 um 16:01 schrieb Eugen Block:

You don't specify which client in your rbd command: > rbd map hdb_backup/VCT --id client --keyring > /etc/ceph/ceph.client.VCT.keyring Have you tried this (not sure about upper-case client names, haven't tried that)? rbd map hdb_backup/VCT --id VCT --keyring /etc/ceph/ceph.client.VCT.keyring Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > Hello, > > I'm trying to mount RBD using rbd map, but I get this error message: > # rbd map hdb_backup/VCT --id client --keyring > /etc/ceph/ceph.client.VCT.keyring > rbd: couldn't connect to the cluster! > > Checking on Ceph server the required permission for relevant > keyring exists: > # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring > [client.VCT] > key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== > > # ceph auth get client.VCT > [client.VCT] > key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== > caps mon = "allow r" > caps osd = "allow rwx pool hdb_backup object_prefix > rbd_data.b768d4baac048b; allow rwx pool hdb_backup object_prefix > rbd_header.b768d4baac048b; allow rx pool hdb_backup > object_prefix rbd_id.VCT" > exported keyring for client.VCT > > > Can you please advise how to fix this error? > > > THX > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Eugen Block

3:47 p.m.

...

I tried to recreate this restrictive client access, one thing is that the rbd_id is in lower-case. I created a test client named "TEST": storage01:~ # rados -p pool ls | grep -vE "5473cdeb5c62c|1f553ba0f6222" | grep test rbd_id.test But after adding all necessary caps I'm still not allowed to get the image info: client:~ # rbd -p pool info test --id TEST --keyring /etc/ceph/ceph.client.TEST.keyring 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 librbd::mirror::GetInfoRequest: 0x556072a66560 handle_get_mirror_image: failed to retrieve mirroring state: (1) Operation not permitted rbd: info: (1) Operation not permitted And I don't have rbd-mirror enabled in this cluster, so that's kind of strange... I'll try to find out which other caps it requires. I already disabled all image features but to no avail. Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > I'll delete existing authentication and its caps "VCT" and recreate it. > > Just to be sure: there's no ingress communication to the client > (from Ceph server)? > > Am 23.02.2023 um 16:01 schrieb Eugen Block: >> For rbd commands you don't specify the "client" prefix for the >> --id parameter, just the client name, in your case "VCT". Your >> second approach shows a different error message, so it can connect >> with "VCT" successfully, but the permissions seem not to be >> sufficient. Those caps look very restrictive, not sure which >> prevent the map command though. >> >> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >> >>> Hm... I'm not sure about the correct rbd command syntax, but I >>> thought it's correct. >>> >>> Anyway, using a different ID fails, too: >>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> rbd: couldn't connect to the cluster! >>> >>> # rbd map hdb_backup/VCT --id VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>> handle_metadata_list: failed to retrieve image metadata: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::RefreshRequest: failed to retrieve pool metadata: >>> (1) Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::OpenRequest: failed to refresh image: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 librbd::ImageState: >>> 0x5569d8a16ba0 failed to open image: (1) Operation not permitted >>> rbd: error opening image VCT: (1) Operation not permitted >>> >>> >>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>> You don't specify which client in your rbd command: >>>> >>>>> rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> Have you tried this (not sure about upper-case client names, >>>> haven't tried that)? >>>> >>>> rbd map hdb_backup/VCT --id VCT --keyring >>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> >>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>> >>>>> Hello, >>>>> >>>>> I'm trying to mount RBD using rbd map, but I get this error message: >>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> rbd: couldn't connect to the cluster! >>>>> >>>>> Checking on Ceph server the required permission for relevant >>>>> keyring exists: >>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> >>>>> # ceph auth get client.VCT >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> caps mon = "allow r" >>>>> caps osd = "allow rwx pool hdb_backup object_prefix >>>>> rbd_data.b768d4baac048b; allow rwx pool hdb_backup >>>>> object_prefix rbd_header.b768d4baac048b; allow rx pool >>>>> hdb_backup object_prefix rbd_id.VCT" >>>>> exported keyring for client.VCT >>>>> >>>>> >>>>> Can you please advise how to fix this error? >>>>> >>>>> >>>>> THX >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

3:58 p.m.

This is # ceph --version ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable) Am 23.02.2023 um 16:47 schrieb Eugen Block:

...

I tried to recreate this restrictive client access, one thing is that the rbd_id is in lower-case. I created a test client named "TEST": storage01:~ # rados -p pool ls | grep -vE "5473cdeb5c62c|1f553ba0f6222" | grep test rbd_id.test But after adding all necessary caps I'm still not allowed to get the image info: client:~ # rbd -p pool info test --id TEST --keyring /etc/ceph/ceph.client.TEST.keyring 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 librbd::mirror::GetInfoRequest: 0x556072a66560 handle_get_mirror_image: failed to retrieve mirroring state: (1) Operation not permitted rbd: info: (1) Operation not permitted And I don't have rbd-mirror enabled in this cluster, so that's kind of strange... I'll try to find out which other caps it requires. I already disabled all image features but to no avail. Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > I'll delete existing authentication and its caps "VCT" and recreate it. > > Just to be sure: there's no ingress communication to the client > (from Ceph server)? > > Am 23.02.2023 um 16:01 schrieb Eugen Block: >> For rbd commands you don't specify the "client" prefix for the --id >> parameter, just the client name, in your case "VCT". Your second >> approach shows a different error message, so it can connect with >> "VCT" successfully, but the permissions seem not to be sufficient. >> Those caps look very restrictive, not sure which prevent the map >> command though. >> >> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >> >>> Hm... I'm not sure about the correct rbd command syntax, but I >>> thought it's correct. >>> >>> Anyway, using a different ID fails, too: >>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> rbd: couldn't connect to the cluster! >>> >>> # rbd map hdb_backup/VCT --id VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>> handle_metadata_list: failed to retrieve image metadata: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::RefreshRequest: failed to retrieve pool metadata: >>> (1) Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::OpenRequest: failed to refresh image: (1) Operation >>> not permitted >>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 librbd::ImageState: >>> 0x5569d8a16ba0 failed to open image: (1) Operation not permitted >>> rbd: error opening image VCT: (1) Operation not permitted >>> >>> >>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>> You don't specify which client in your rbd command: >>>> >>>>> rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> Have you tried this (not sure about upper-case client names, >>>> haven't tried that)? >>>> >>>> rbd map hdb_backup/VCT --id VCT --keyring >>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> >>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>> >>>>> Hello, >>>>> >>>>> I'm trying to mount RBD using rbd map, but I get this error >>>>> message: >>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> rbd: couldn't connect to the cluster! >>>>> >>>>> Checking on Ceph server the required permission for relevant >>>>> keyring exists: >>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> >>>>> # ceph auth get client.VCT >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> caps mon = "allow r" >>>>> caps osd = "allow rwx pool hdb_backup object_prefix >>>>> rbd_data.b768d4baac048b; allow rwx pool hdb_backup object_prefix >>>>> rbd_header.b768d4baac048b; allow rx pool hdb_backup >>>>> object_prefix rbd_id.VCT" >>>>> exported keyring for client.VCT >>>>> >>>>> >>>>> Can you please advise how to fix this error? >>>>> >>>>> >>>>> THX >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Eugen Block

4:33 p.m.

...

This is # ceph --version ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable) Am 23.02.2023 um 16:47 schrieb Eugen Block:

I tried to recreate this restrictive client access, one thing is that the rbd_id is in lower-case. I created a test client named "TEST": storage01:~ # rados -p pool ls | grep -vE "5473cdeb5c62c|1f553ba0f6222" | grep test rbd_id.test But after adding all necessary caps I'm still not allowed to get the image info: client:~ # rbd -p pool info test --id TEST --keyring /etc/ceph/ceph.client.TEST.keyring 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 librbd::mirror::GetInfoRequest: 0x556072a66560 handle_get_mirror_image: failed to retrieve mirroring state: (1) Operation not permitted rbd: info: (1) Operation not permitted And I don't have rbd-mirror enabled in this cluster, so that's kind of strange... I'll try to find out which other caps it requires. I already disabled all image features but to no avail. Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > I'll delete existing authentication and its caps "VCT" and recreate it. > > Just to be sure: there's no ingress communication to the client > (from Ceph server)? > > Am 23.02.2023 um 16:01 schrieb Eugen Block: >> For rbd commands you don't specify the "client" prefix for the >> --id parameter, just the client name, in your case "VCT". Your >> second approach shows a different error message, so it can >> connect with "VCT" successfully, but the permissions seem not to >> be sufficient. Those caps look very restrictive, not sure which >> prevent the map command though. >> >> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >> >>> Hm... I'm not sure about the correct rbd command syntax, but I >>> thought it's correct. >>> >>> Anyway, using a different ID fails, too: >>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> rbd: couldn't connect to the cluster! >>> >>> # rbd map hdb_backup/VCT --id VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>> handle_metadata_list: failed to retrieve image metadata: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::RefreshRequest: failed to retrieve pool >>> metadata: (1) Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::OpenRequest: failed to refresh image: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 >>> librbd::ImageState: 0x5569d8a16ba0 failed to open image: (1) >>> Operation not permitted >>> rbd: error opening image VCT: (1) Operation not permitted >>> >>> >>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>> You don't specify which client in your rbd command: >>>> >>>>> rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> Have you tried this (not sure about upper-case client names, >>>> haven't tried that)? >>>> >>>> rbd map hdb_backup/VCT --id VCT --keyring >>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> >>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>> >>>>> Hello, >>>>> >>>>> I'm trying to mount RBD using rbd map, but I get this error message: >>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> rbd: couldn't connect to the cluster! >>>>> >>>>> Checking on Ceph server the required permission for relevant >>>>> keyring exists: >>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> >>>>> # ceph auth get client.VCT >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> caps mon = "allow r" >>>>> caps osd = "allow rwx pool hdb_backup object_prefix >>>>> rbd_data.b768d4baac048b; allow rwx pool hdb_backup >>>>> object_prefix rbd_header.b768d4baac048b; allow rx pool >>>>> hdb_backup object_prefix rbd_id.VCT" >>>>> exported keyring for client.VCT >>>>> >>>>> >>>>> Can you please advise how to fix this error? >>>>> >>>>> >>>>> THX >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

4:47 p.m.

Confirmed. # ceph versions { "mon": { "ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable)": 3 }, "mgr": { "ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable)": 3 }, "osd": { "ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable)": 437 }, "mds": { "ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable)": 7 }, "overall": { "ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable)": 450 } } Am 23.02.2023 um 17:33 schrieb Eugen Block:

...

This is # ceph --version ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable) Am 23.02.2023 um 16:47 schrieb Eugen Block:

I tried to recreate this restrictive client access, one thing is that the rbd_id is in lower-case. I created a test client named "TEST": storage01:~ # rados -p pool ls | grep -vE "5473cdeb5c62c|1f553ba0f6222" | grep test rbd_id.test But after adding all necessary caps I'm still not allowed to get the image info: client:~ # rbd -p pool info test --id TEST --keyring /etc/ceph/ceph.client.TEST.keyring 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 librbd::mirror::GetInfoRequest: 0x556072a66560 handle_get_mirror_image: failed to retrieve mirroring state: (1) Operation not permitted rbd: info: (1) Operation not permitted And I don't have rbd-mirror enabled in this cluster, so that's kind of strange... I'll try to find out which other caps it requires. I already disabled all image features but to no avail. Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > I'll delete existing authentication and its caps "VCT" and > recreate it. > > Just to be sure: there's no ingress communication to the client > (from Ceph server)? > > Am 23.02.2023 um 16:01 schrieb Eugen Block: >> For rbd commands you don't specify the "client" prefix for the >> --id parameter, just the client name, in your case "VCT". Your >> second approach shows a different error message, so it can >> connect with "VCT" successfully, but the permissions seem not to >> be sufficient. Those caps look very restrictive, not sure which >> prevent the map command though. >> >> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >> >>> Hm... I'm not sure about the correct rbd command syntax, but I >>> thought it's correct. >>> >>> Anyway, using a different ID fails, too: >>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> rbd: couldn't connect to the cluster! >>> >>> # rbd map hdb_backup/VCT --id VCT --keyring >>> /etc/ceph/ceph.client.VCT.keyring >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>> handle_metadata_list: failed to retrieve image metadata: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::RefreshRequest: failed to retrieve pool metadata: >>> (1) Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>> librbd::image::OpenRequest: failed to refresh image: (1) >>> Operation not permitted >>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 librbd::ImageState: >>> 0x5569d8a16ba0 failed to open image: (1) Operation not permitted >>> rbd: error opening image VCT: (1) Operation not permitted >>> >>> >>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>> You don't specify which client in your rbd command: >>>> >>>>> rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> Have you tried this (not sure about upper-case client names, >>>> haven't tried that)? >>>> >>>> rbd map hdb_backup/VCT --id VCT --keyring >>>> /etc/ceph/ceph.client.VCT.keyring >>>> >>>> >>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>> >>>>> Hello, >>>>> >>>>> I'm trying to mount RBD using rbd map, but I get this error >>>>> message: >>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> rbd: couldn't connect to the cluster! >>>>> >>>>> Checking on Ceph server the required permission for relevant >>>>> keyring exists: >>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> >>>>> # ceph auth get client.VCT >>>>> [client.VCT] >>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>> caps mon = "allow r" >>>>> caps osd = "allow rwx pool hdb_backup object_prefix >>>>> rbd_data.b768d4baac048b; allow rwx pool hdb_backup >>>>> object_prefix rbd_header.b768d4baac048b; allow rx pool >>>>> hdb_backup object_prefix rbd_id.VCT" >>>>> exported keyring for client.VCT >>>>> >>>>> >>>>> Can you please advise how to fix this error? >>>>> >>>>> >>>>> THX >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Eugen Block

4:55 p.m.

And did you already try the other caps? Do those work? Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>:

...

This is # ceph --version ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable) Am 23.02.2023 um 16:47 schrieb Eugen Block:

Which ceph version is this? In a Nautilus cluster it works for me with the lower-case rbd_id, in Pacific it doesn't. I don't have an Octopus cluster at hand. Zitat von Eugen Block <eblock(a)nde.ag>ag>: > I tried to recreate this restrictive client access, one thing is > that the rbd_id is in lower-case. I created a test client named > "TEST": > > storage01:~ # rados -p pool ls | grep -vE > "5473cdeb5c62c|1f553ba0f6222" | grep test > rbd_id.test > > But after adding all necessary caps I'm still not allowed to get > the image info: > > client:~ # rbd -p pool info test --id TEST --keyring > /etc/ceph/ceph.client.TEST.keyring > 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 > librbd::mirror::GetInfoRequest: 0x556072a66560 > handle_get_mirror_image: failed to retrieve mirroring state: (1) > Operation not permitted > rbd: info: (1) Operation not permitted > > And I don't have rbd-mirror enabled in this cluster, so that's > kind of strange... I'll try to find out which other caps it > requires. I already disabled all image features but to no avail. > > Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > >> I'll delete existing authentication and its caps "VCT" and recreate it. >> >> Just to be sure: there's no ingress communication to the client >> (from Ceph server)? >> >> Am 23.02.2023 um 16:01 schrieb Eugen Block: >>> For rbd commands you don't specify the "client" prefix for the >>> --id parameter, just the client name, in your case "VCT". Your >>> second approach shows a different error message, so it can >>> connect with "VCT" successfully, but the permissions seem not >>> to be sufficient. Those caps look very restrictive, not sure >>> which prevent the map command though. >>> >>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>> >>>> Hm... I'm not sure about the correct rbd command syntax, but >>>> I thought it's correct. >>>> >>>> Anyway, using a different ID fails, too: >>>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>>> /etc/ceph/ceph.client.VCT.keyring >>>> rbd: couldn't connect to the cluster! >>>> >>>> # rbd map hdb_backup/VCT --id VCT --keyring >>>> /etc/ceph/ceph.client.VCT.keyring >>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>>> handle_metadata_list: failed to retrieve image metadata: (1) >>>> Operation not permitted >>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>> librbd::image::RefreshRequest: failed to retrieve pool >>>> metadata: (1) Operation not permitted >>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>> librbd::image::OpenRequest: failed to refresh image: (1) >>>> Operation not permitted >>>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 >>>> librbd::ImageState: 0x5569d8a16ba0 failed to open image: (1) >>>> Operation not permitted >>>> rbd: error opening image VCT: (1) Operation not permitted >>>> >>>> >>>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>>> You don't specify which client in your rbd command: >>>>> >>>>>> rbd map hdb_backup/VCT --id client --keyring >>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> >>>>> Have you tried this (not sure about upper-case client names, >>>>> haven't tried that)? >>>>> >>>>> rbd map hdb_backup/VCT --id VCT --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> >>>>> >>>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>>> >>>>>> Hello, >>>>>> >>>>>> I'm trying to mount RBD using rbd map, but I get this error message: >>>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>> rbd: couldn't connect to the cluster! >>>>>> >>>>>> Checking on Ceph server the required permission for >>>>>> relevant keyring exists: >>>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>>> [client.VCT] >>>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>>> >>>>>> # ceph auth get client.VCT >>>>>> [client.VCT] >>>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>>> caps mon = "allow r" >>>>>> caps osd = "allow rwx pool hdb_backup object_prefix >>>>>> rbd_data.b768d4baac048b; allow rwx pool hdb_backup >>>>>> object_prefix rbd_header.b768d4baac048b; allow rx pool >>>>>> hdb_backup object_prefix rbd_id.VCT" >>>>>> exported keyring for client.VCT >>>>>> >>>>>> >>>>>> Can you please advise how to fix this error? >>>>>> >>>>>> >>>>>> THX >>>>>> _______________________________________________ >>>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>>> >>>>> >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> >>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

24 Feb 24 Feb

8:04 a.m.

Actually I didn't try other caps. The setup of RBD images and authorizations is automised with a bash script that worked in the past w/o issues. I need to understand the root cause in order to adapt the script accordingly. Am 23.02.2023 um 17:55 schrieb Eugen Block:

...

And did you already try the other caps? Do those work? Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>:

This is # ceph --version ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) nautilus (stable) Am 23.02.2023 um 16:47 schrieb Eugen Block: > Which ceph version is this? In a Nautilus cluster it works for me > with the lower-case rbd_id, in Pacific it doesn't. I don't have an > Octopus cluster at hand. > > Zitat von Eugen Block <eblock(a)nde.ag>ag>: > >> I tried to recreate this restrictive client access, one thing is >> that the rbd_id is in lower-case. I created a test client named >> "TEST": >> >> storage01:~ # rados -p pool ls | grep -vE >> "5473cdeb5c62c|1f553ba0f6222" | grep test >> rbd_id.test >> >> But after adding all necessary caps I'm still not allowed to get >> the image info: >> >> client:~ # rbd -p pool info test --id TEST --keyring >> /etc/ceph/ceph.client.TEST.keyring >> 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 >> librbd::mirror::GetInfoRequest: 0x556072a66560 >> handle_get_mirror_image: failed to retrieve mirroring state: (1) >> Operation not permitted >> rbd: info: (1) Operation not permitted >> >> And I don't have rbd-mirror enabled in this cluster, so that's >> kind of strange... I'll try to find out which other caps it >> requires. I already disabled all image features but to no avail. >> >> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >> >>> I'll delete existing authentication and its caps "VCT" and >>> recreate it. >>> >>> Just to be sure: there's no ingress communication to the client >>> (from Ceph server)? >>> >>> Am 23.02.2023 um 16:01 schrieb Eugen Block: >>>> For rbd commands you don't specify the "client" prefix for the >>>> --id parameter, just the client name, in your case "VCT". Your >>>> second approach shows a different error message, so it can >>>> connect with "VCT" successfully, but the permissions seem not >>>> to be sufficient. Those caps look very restrictive, not sure >>>> which prevent the map command though. >>>> >>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>> >>>>> Hm... I'm not sure about the correct rbd command syntax, but I >>>>> thought it's correct. >>>>> >>>>> Anyway, using a different ID fails, too: >>>>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> rbd: couldn't connect to the cluster! >>>>> >>>>> # rbd map hdb_backup/VCT --id VCT --keyring >>>>> /etc/ceph/ceph.client.VCT.keyring >>>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>>>> handle_metadata_list: failed to retrieve image metadata: (1) >>>>> Operation not permitted >>>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>>> librbd::image::RefreshRequest: failed to retrieve pool >>>>> metadata: (1) Operation not permitted >>>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>>> librbd::image::OpenRequest: failed to refresh image: (1) >>>>> Operation not permitted >>>>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 >>>>> librbd::ImageState: 0x5569d8a16ba0 failed to open image: (1) >>>>> Operation not permitted >>>>> rbd: error opening image VCT: (1) Operation not permitted >>>>> >>>>> >>>>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>>>> You don't specify which client in your rbd command: >>>>>> >>>>>>> rbd map hdb_backup/VCT --id client --keyring >>>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>> >>>>>> Have you tried this (not sure about upper-case client names, >>>>>> haven't tried that)? >>>>>> >>>>>> rbd map hdb_backup/VCT --id VCT --keyring >>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>> >>>>>> >>>>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I'm trying to mount RBD using rbd map, but I get this error >>>>>>> message: >>>>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>>> rbd: couldn't connect to the cluster! >>>>>>> >>>>>>> Checking on Ceph server the required permission for relevant >>>>>>> keyring exists: >>>>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>>>> [client.VCT] >>>>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>>>> >>>>>>> # ceph auth get client.VCT >>>>>>> [client.VCT] >>>>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>>>> caps mon = "allow r" >>>>>>> caps osd = "allow rwx pool hdb_backup object_prefix >>>>>>> rbd_data.b768d4baac048b; allow rwx pool hdb_backup >>>>>>> object_prefix rbd_header.b768d4baac048b; allow rx pool >>>>>>> hdb_backup object_prefix rbd_id.VCT" >>>>>>> exported keyring for client.VCT >>>>>>> >>>>>>> >>>>>>> Can you please advise how to fix this error? >>>>>>> >>>>>>> >>>>>>> THX >>>>>>> _______________________________________________ >>>>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io > > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Eugen Block

10:38 a.m.

Just one addition from my test, I believe I misinterpreted my results because my test image was named "test" and the client "TEST", so the rbd_id.<IMAGE> is indeed upper case for an image that has an upper-case name. So forget my comment about that. Another question though: does the image you're trying to map actually contain the object_prefix you have in your caps? Can you paste the output of 'rbd info hdb_backup/VCT'? Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>:

...

And did you already try the other caps? Do those work? Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>:

And the ceph cluster has the same version? ‚ceph versions‘ shows all daemons. If the cluster is also 14.2.X the caps should work with lower-case rbd_id. Can you confirm? Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > This is > # ceph --version > ceph version 14.2.22 (877fa256043e4743620f4677e72dee5e738d1226) > nautilus (stable) > > > > Am 23.02.2023 um 16:47 schrieb Eugen Block: >> Which ceph version is this? In a Nautilus cluster it works for >> me with the lower-case rbd_id, in Pacific it doesn't. I don't >> have an Octopus cluster at hand. >> >> Zitat von Eugen Block <eblock(a)nde.ag>ag>: >> >>> I tried to recreate this restrictive client access, one thing >>> is that the rbd_id is in lower-case. I created a test client >>> named "TEST": >>> >>> storage01:~ # rados -p pool ls | grep -vE >>> "5473cdeb5c62c|1f553ba0f6222" | grep test >>> rbd_id.test >>> >>> But after adding all necessary caps I'm still not allowed to >>> get the image info: >>> >>> client:~ # rbd -p pool info test --id TEST --keyring >>> /etc/ceph/ceph.client.TEST.keyring >>> 2023-02-23T16:35:16.740+0100 7faebaffd700 -1 >>> librbd::mirror::GetInfoRequest: 0x556072a66560 >>> handle_get_mirror_image: failed to retrieve mirroring state: >>> (1) Operation not permitted >>> rbd: info: (1) Operation not permitted >>> >>> And I don't have rbd-mirror enabled in this cluster, so that's >>> kind of strange... I'll try to find out which other caps it >>> requires. I already disabled all image features but to no avail. >>> >>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>> >>>> I'll delete existing authentication and its caps "VCT" and >>>> recreate it. >>>> >>>> Just to be sure: there's no ingress communication to the >>>> client (from Ceph server)? >>>> >>>> Am 23.02.2023 um 16:01 schrieb Eugen Block: >>>>> For rbd commands you don't specify the "client" prefix for >>>>> the --id parameter, just the client name, in your case >>>>> "VCT". Your second approach shows a different error message, >>>>> so it can connect with "VCT" successfully, but the >>>>> permissions seem not to be sufficient. Those caps look very >>>>> restrictive, not sure which prevent the map command though. >>>>> >>>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>>> >>>>>> Hm... I'm not sure about the correct rbd command syntax, >>>>>> but I thought it's correct. >>>>>> >>>>>> Anyway, using a different ID fails, too: >>>>>> # rbd map hdb_backup/VCT --id client.VCT --keyring >>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>> rbd: couldn't connect to the cluster! >>>>>> >>>>>> # rbd map hdb_backup/VCT --id VCT --keyring >>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>>>> librbd::image::GetMetadataRequest: 0x7f220c001ef0 >>>>>> handle_metadata_list: failed to retrieve image metadata: >>>>>> (1) Operation not permitted >>>>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>>>> librbd::image::RefreshRequest: failed to retrieve pool >>>>>> metadata: (1) Operation not permitted >>>>>> 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 >>>>>> librbd::image::OpenRequest: failed to refresh image: (1) >>>>>> Operation not permitted >>>>>> 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 >>>>>> librbd::ImageState: 0x5569d8a16ba0 failed to open image: >>>>>> (1) Operation not permitted >>>>>> rbd: error opening image VCT: (1) Operation not permitted >>>>>> >>>>>> >>>>>> Am 23.02.2023 um 15:30 schrieb Eugen Block: >>>>>>> You don't specify which client in your rbd command: >>>>>>> >>>>>>>> rbd map hdb_backup/VCT --id client --keyring >>>>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>>> >>>>>>> Have you tried this (not sure about upper-case client >>>>>>> names, haven't tried that)? >>>>>>> >>>>>>> rbd map hdb_backup/VCT --id VCT --keyring >>>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>>> >>>>>>> >>>>>>> Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> I'm trying to mount RBD using rbd map, but I get this >>>>>>>> error message: >>>>>>>> # rbd map hdb_backup/VCT --id client --keyring >>>>>>>> /etc/ceph/ceph.client.VCT.keyring >>>>>>>> rbd: couldn't connect to the cluster! >>>>>>>> >>>>>>>> Checking on Ceph server the required permission for >>>>>>>> relevant keyring exists: >>>>>>>> # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring >>>>>>>> [client.VCT] >>>>>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>>>>> >>>>>>>> # ceph auth get client.VCT >>>>>>>> [client.VCT] >>>>>>>> key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== >>>>>>>> caps mon = "allow r" >>>>>>>> caps osd = "allow rwx pool hdb_backup >>>>>>>> object_prefix rbd_data.b768d4baac048b; allow rwx pool >>>>>>>> hdb_backup object_prefix rbd_header.b768d4baac048b; allow >>>>>>>> rx pool hdb_backup object_prefix rbd_id.VCT" >>>>>>>> exported keyring for client.VCT >>>>>>>> >>>>>>>> >>>>>>>> Can you please advise how to fix this error? >>>>>>>> >>>>>>>> >>>>>>>> THX >>>>>>>> _______________________________________________ >>>>>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>>>> _______________________________________________ >>>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>>> >>>>> >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users(a)ceph.io >>>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

12:35 p.m.

Please check the output here: # rbd info hdb_backup/VCT rbd image 'VCT': size 800 GiB in 204800 objects order 22 (4 MiB objects) snapshot_count: 0 id: b768d4baac048b block_name_prefix: rbd_data.b768d4baac048b format: 2 features: layering op_features: flags: create_timestamp: Thu Jan 5 15:19:14 2023 access_timestamp: Thu Jan 5 15:19:14 2023 modify_timestamp: Thu Jan 5 15:19:14 2023 Am 24.02.2023 um 11:38 schrieb Eugen Block:

...

rbd info hdb_backup/VCT

Ilya Dryomov

26 Feb 26 Feb

12:51 p.m.

On Fri, Feb 24, 2023 at 9:05 AM Thomas Schneider <74cmonty(a)gmail.com> wrote:

...

Using home-grown scripts to allow a client to access only a particular image this way (object_prefix match for "rbd_data.<image id>" and then playing whack-a-mole to cover everything else that is needed: rbd_info object, rbd_mirroring object, etc -- and that is before someone enables e.g. object-map image feature on the image) is not recommended. Instead, create a namespace, place the image(s) that a client should be allowed to access into the namespace and use the built-in "profile rbd" capability for it. Here is an example: $ rbd namespace create mypool/foo $ rbd namespace create mypool/bar $ ceph auth get-or-create client.foo mon 'profile rbd' osd 'profile rbd pool mypool namespace foo' $ ceph auth get-or-create client.bar mon 'profile rbd' osd 'profile rbd pool mypool namespace bar' $ rbd create --id foo --size 1G mypool/foo/img $ sudo rbd map --id foo mypool/foo/img $ rbd create --id bar --size 1G mypool/bar/img $ sudo rbd map --id bar mypool/bar/img Note that both images are named "img" -- since each resides in its own namespace, this works just fine. An attempt to map using a mismatching client would fail (unless it's sufficiently privileged, of course): $ sudo rbd map --id foo mypool/bar/img rbd: error asserting namespace: (1) Operation not permitted 2023-02-26T07:22:37.420-0500 7f7354ad8740 -1 librbd::api::Namespace: exists: error asserting namespace: (1) Operation not permitted The kernel client supports namespaces since kernel 4.19 (and also CentOS 8.0). Thanks, Ilya

Curt

23 Feb 23 Feb

3:01 p.m.

...

You don't specify which client in your rbd command:

rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

3:05 p.m.

This conclusion is certainly incorrect: # rbd ls -l hdb_backup | grep VCT VCT 800 GiB 2 Am 23.02.2023 um 16:01 schrieb Curt:

...

What does 'rbd ls hbd_backup' return? Or is your pool VCT? Which if that's the case those should be switched. 'rbd map VCT/hdb_backup --id VCT --keyring /etc/ceph/ceph.client.VCT.keyring' On Thu, Feb 23, 2023 at 6:54 PM Thomas Schneider <74cmonty(a)gmail.com> wrote: Hm... I'm not sure about the correct rbd command syntax, but I thought it's correct. Anyway, using a different ID fails, too: # rbd map hdb_backup/VCT --id client.VCT --keyring /etc/ceph/ceph.client.VCT.keyring rbd: couldn't connect to the cluster! # rbd map hdb_backup/VCT --id VCT --keyring /etc/ceph/ceph.client.VCT.keyring 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 librbd::image::GetMetadataRequest: 0x7f220c001ef0 handle_metadata_list: failed to retrieve image metadata: (1) Operation not permitted 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 librbd::image::RefreshRequest: failed to retrieve pool metadata: (1) Operation not permitted 2023-02-23T15:46:16.848+0100 7f222d19d700 -1 librbd::image::OpenRequest: failed to refresh image: (1) Operation not permitted 2023-02-23T15:46:16.848+0100 7f222c99c700 -1 librbd::ImageState: 0x5569d8a16ba0 failed to open image: (1) Operation not permitted rbd: error opening image VCT: (1) Operation not permitted Am 23.02.2023 um 15:30 schrieb Eugen Block:

You don't specify which client in your rbd command:

rbd map hdb_backup/VCT --id client --keyring /etc/ceph/ceph.client.VCT.keyring

Have you tried this (not sure about upper-case client names,

haven't

tried that)? rbd map hdb_backup/VCT --id VCT --keyring /etc/ceph/ceph.client.VCT.keyring Zitat von Thomas Schneider <74cmonty(a)gmail.com>om>: > Hello, > > I'm trying to mount RBD using rbd map, but I get this error

message:

> # rbd map hdb_backup/VCT --id client --keyring > /etc/ceph/ceph.client.VCT.keyring > rbd: couldn't connect to the cluster! > > Checking on Ceph server the required permission for relevant

keyring

exists: # ceph-authtool -l /etc/ceph/ceph.client.VCT.keyring [client.VCT] key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== # ceph auth get client.VCT [client.VCT] key = AQBj3LZjNGn/BhAAG8IqMyH0WLKi4kTlbjiW7g== caps mon = "allow r" caps osd = "allow rwx pool hdb_backup object_prefix rbd_data.b768d4baac048b; allow rwx pool hdb_backup object_prefix rbd_header.b768d4baac048b; allow rx pool hdb_backup object_prefix rbd_id.VCT" exported keyring for client.VCT Can you please advise how to fix this error? THX _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Ilya Dryomov

26 Feb 26 Feb

11:46 a.m.

On Thu, Feb 23, 2023 at 3:53 PM Thomas Schneider <74cmonty(a)gmail.com> wrote:

...

Hi Thomas, At the very least, you are missing a permission for metadata_list method: allow class rbd metadata_list pool hdb_backup Thanks, Ilya

Thomas Schneider

27 Feb 27 Feb

12:55 p.m.

Hi, I get an error running this ceph auth get-or-create syntax: # ceph auth get-or-create client.${rbdName} mon "allow r" osd "allow rwx pool ${rbdPoolName} object_prefix rbd_data.${imageID}; allow rwx pool ${rbdPoolName} object_prefix rbd_header.${imageID}; allow rx pool ${rbdPoolName} object_prefix rbd_id.${rbdName}"; allow class rbd metadata_list pool ${rbdPoolName} -o /etc/ceph/ceph.client.${rbdName}.keyring; [client.VCT] key = AQDGp/xj5EKrFRAArU7SyOVF8NFUC4lRCWwmCQ== -bash: allow: command not found. THX Am 26.02.2023 um 12:46 schrieb Ilya Dryomov:

...

allow class rbd metadata_list pool hdb_backup

Curt

12:59 p.m.

Needs to be inside the " with your other commands. On Mon, Feb 27, 2023, 16:55 Thomas Schneider <74cmonty(a)gmail.com> wrote:

...

allow class rbd metadata_list pool hdb_backup

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

Thomas Schneider

3:59 p.m.

Hi, issue is solved now after executing this command: # ceph auth get-or-create client.${rbdName} mon "allow r" osd "allow rwx pool ${rbdPoolName} object_prefix rbd_data.${imageID}; allow rwx pool ${rbdPoolName} object_prefix rbd_header.${imageID}; allow rx pool ${rbdPoolName} object_prefix rbd_id.${rbdName}; allow class rbd metadata_list pool ${rbdPoolName}" -o /etc/ceph/ceph.client.${rbdName}.keyring; With this newly generated keyring I can map the relevant RBD w/o errors. Many thanks to everybody who contributed to the solution. Am 27.02.2023 um 13:59 schrieb Curt:

...

Needs to be inside the " with your other commands. On Mon, Feb 27, 2023, 16:55 Thomas Schneider <74cmonty(a)gmail.com> wrote: Hi, I get an error running this ceph auth get-or-create syntax: # ceph auth get-or-create client.${rbdName} mon "allow r" osd "allow rwx pool ${rbdPoolName} object_prefix rbd_data.${imageID}; allow rwx pool ${rbdPoolName} object_prefix rbd_header.${imageID}; allow rx pool ${rbdPoolName} object_prefix rbd_id.${rbdName}"; allow class rbd metadata_list pool ${rbdPoolName} -o /etc/ceph/ceph.client.${rbdName}.keyring; [client.VCT] key = AQDGp/xj5EKrFRAArU7SyOVF8NFUC4lRCWwmCQ== -bash: allow: command not found. THX Am 26.02.2023 um 12:46 schrieb Ilya Dryomov:

allow class rbd metadata_list pool hdb_backup

_______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

447

days inactive

451

days old

ceph-users@ceph.io

Manage subscription

20 comments

4 participants

tags (0)

participants (4)

Curt
Eugen Block
Ilya Dryomov
Thomas Schneider