Hi Zac, Any movement on this? We really need to come up with an answer/solution - thanks Dulux-Oz On 19/04/2024 18:03, duluxoz wrote:

Cool!

Thanks for that  :-)

On 19/04/2024 18:01, Zac Dover wrote:

...

I think I understand, after more thought. The second command is expected to work after the first.

I will ask the cephfs team when they wake up.

Zac Dover Upstream Docs Ceph Foundation

On Fri, Apr 19, 2024 at 17:51, duluxoz <duluxoz@gmail.com <mailto:On Fri, Apr 19, 2024 at 17:51, duluxoz <<a href=>> wrote:

...

Hi All,

In reference to this page from the Ceph documentation: https://docs.ceph.com/en/latest/cephfs/client-auth/, down the bottom of that page it says that you can run the following commands:

~~~ ceph fs authorize a client.x /dir1 rw ceph fs authorize a client.x /dir2 rw ~~~

This will allow `client.x` to access both `dir1` and `dir2`.

So, having a use case where we need to do this, we are, HOWEVER, getting the following error on running the 2nd command on a Reef 18.2.2 cluster:

`Error EINVAL: client.x already has fs capabilities that differ from those supplied. To generate a new auth key for client.x, first remove client.x from configuration files, execute 'ceph auth rm client.x', then execute this command again.`

Something we're doing wrong, or is the doco "out of date" (mind you, that's from the "latest" version of the doco, and the "reef" version), or is something else going on?

Thanks in advance for the help

Cheers

Dulux-Oz

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

Hi, I believe the docs [2] are okay, running 'ceph fs authorize' will overwrite the existing caps, it will not add more caps to the client:

Capabilities can be modified by running fs authorize only in the case when read/write permissions must be changed.

If a client already has a capability for file-system name a and path dir1, running fs authorize again for FS name a but path dir2, instead of modifying the capabilities client already holds, a new cap for dir2 will be granted

To add more caps you'll need to use the 'ceph auth caps' command, for example: quincy-1:~ # ceph fs authorize cephfs client.usera /dir1 rw [client.usera] key = AQDOrShmk6XhGxAAwz07ngr0JtPSID06RH8lAw== quincy-1:~ # ceph auth get client.usera [client.usera] key = AQDOrShmk6XhGxAAwz07ngr0JtPSID06RH8lAw== caps mds = "allow rw fsname=cephfs path=/dir1" caps mon = "allow r fsname=cephfs" caps osd = "allow rw tag cephfs data=cephfs" quincy-1:~ # ceph auth caps client.usera mds 'allow rw fsname=cephfs path=/dir1, allow rw fsname=cephfs path=/dir2' mon 'allow r fsname=cephfs' osd 'allow rw tag cephfs data=cephfs' updated caps for client.usera quincy-1:~ # ceph auth get client.usera [client.usera] key = AQDOrShmk6XhGxAAwz07ngr0JtPSID06RH8lAw== caps mds = "allow rw fsname=cephfs path=/dir1, allow rw fsname=cephfs path=/dir2" caps mon = "allow r fsname=cephfs" caps osd = "allow rw tag cephfs data=cephfs" Note that I don't actually have these directories in that cephfs, it's just to demonstrate, so you'll need to make sure your caps actually work. Thanks, Eugen [2] https://docs.ceph.com/en/latest/cephfs/client-auth/#changing-rw-permissions-... Zitat von Zac Dover <zac.dover@proton.me>:

It's in my list of ongoing initiatives. I'll stay up late tonight and ask Venky directly what's going on in this instance.

Sometime later today, I'll create an issue tracking bug and I'll send it to you for review. Make sure that I haven't misrepresented this issue.

Zac

On Wednesday, April 24th, 2024 at 2:10 PM, duluxoz <duluxoz@gmail.com> wrote:

...

Hi Zac,

Any movement on this? We really need to come up with an answer/solution - thanks

Dulux-Oz

On 19/04/2024 18:03, duluxoz wrote:

...

Cool!

Thanks for that :-)

On 19/04/2024 18:01, Zac Dover wrote:

...

I think I understand, after more thought. The second command is expected to work after the first.

I will ask the cephfs team when they wake up.

Zac Dover Upstream Docs Ceph Foundation

On Fri, Apr 19, 2024 at 17:51, duluxoz <[duluxoz@gmail.com](mailto:On Fri, Apr 19, 2024 at 17:51, duluxoz <<a href=)> wrote:

...

Hi All,

In reference to this page from the Ceph documentation: https://docs.ceph.com/en/latest/cephfs/client-auth/, down the bottom of that page it says that you can run the following commands:

~~~ ceph fs authorize a client.x /dir1 rw ceph fs authorize a client.x /dir2 rw ~~~

This will allow `client.x` to access both `dir1` and `dir2`.

So, having a use case where we need to do this, we are, HOWEVER, getting the following error on running the 2nd command on a Reef 18.2.2 cluster:

`Error EINVAL: client.x already has fs capabilities that differ from those supplied. To generate a new auth key for client.x, first remove client.x from configuration files, execute 'ceph auth rm client.x', then execute this command again.`

Something we're doing wrong, or is the doco "out of date" (mind you, that's from the "latest" version of the doco, and the "reef" version), or is something else going on?

Thanks in advance for the help

Cheers

Dulux-Oz

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

---

ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

Hi, I fully agree that there should be a smoother way to update client caps. But regarding the misleading terms, the docs do mention:

This is because the command fs authorize becomes ambiguous

So they are aware of the current state, but I don't know if there's any work in progress to improve the client authorization. Thanks, Eugen Zitat von Frank Schilder <frans@dtu.dk>:

Hi Eugen,

I would ask for a slight change here:

...

If a client already has a capability for file-system name a and path dir1, running fs authorize again for FS name a but path dir2, instead of modifying the capabilities client already holds, a new cap for dir2 will be granted

The formulation "a new cap for dir2 will be granted" is very misleading. I would also read it as that the new cap is in addition to the already existing cap. I tried to modify caps with fs authorize as well in the past, because it will set caps using pool tags and the docu sounded like it will allow to modify caps. In my case, I got the same error and thought that its implementation is buggy and did it with the authtool.

To be honest, when I look at the command sequence

ceph fs authorize a client.x /dir1 rw ceph fs authorize a client.x /dir2 rw

and it goes through without error, I would expect the client to have both permissions as a result - no matter what the documentation says. There is no "revoke caps" instruction anywhere. Revoking caps in this way is a really dangerous side effect and telling people to read the documentation about a command that should follow how other linux tools manage permissions is not the best answer. There is something called parallelism in software engineering and this command line syntax violates this in a highly un-intuitive way. The intuition of the syntax clearly is that it *adds* capabilities, its incremental.

A command like this should follow how existing linux tools work so that context switching will be easier for admins. Here, the choice of the term "authorize" seems to be unlucky. A more explicit command that follows setfacl a bit could be

ceph fs caps set a client.x /dir1 rw ceph fs caps modify a client.x /dir2 rw

or even more parallel

ceph fs setcaps a client.x /dir1 rw ceph fs setcaps -m a client.x /dir2 rw

Such parallel syntax will not only avoid the reported confusion but also make it possible to implement a modify operation in the future without breaking stuff. And you can save time on the documentation, because it works like other stuff.

Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14

________________________________________ From: Eugen Block <eblock@nde.ag> Sent: Wednesday, April 24, 2024 9:02 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: Latest Doco Out Of Date?

Hi,

I believe the docs [2] are okay, running 'ceph fs authorize' will overwrite the existing caps, it will not add more caps to the client:

...

Capabilities can be modified by running fs authorize only in the case when read/write permissions must be changed.

...

If a client already has a capability for file-system name a and path dir1, running fs authorize again for FS name a but path dir2, instead of modifying the capabilities client already holds, a new cap for dir2 will be granted

To add more caps you'll need to use the 'ceph auth caps' command, for example:

quincy-1:~ # ceph fs authorize cephfs client.usera /dir1 rw [client.usera] key = AQDOrShmk6XhGxAAwz07ngr0JtPSID06RH8lAw==

quincy-1:~ # ceph auth get client.usera [client.usera] key = AQDOrShmk6XhGxAAwz07ngr0JtPSID06RH8lAw== caps mds = "allow rw fsname=cephfs path=/dir1" caps mon = "allow r fsname=cephfs" caps osd = "allow rw tag cephfs data=cephfs"

quincy-1:~ # ceph auth caps client.usera mds 'allow rw fsname=cephfs path=/dir1, allow rw fsname=cephfs path=/dir2' mon 'allow r fsname=cephfs' osd 'allow rw tag cephfs data=cephfs' updated caps for client.usera

quincy-1:~ # ceph auth get client.usera [client.usera] key = AQDOrShmk6XhGxAAwz07ngr0JtPSID06RH8lAw== caps mds = "allow rw fsname=cephfs path=/dir1, allow rw fsname=cephfs path=/dir2" caps mon = "allow r fsname=cephfs" caps osd = "allow rw tag cephfs data=cephfs"

Note that I don't actually have these directories in that cephfs, it's just to demonstrate, so you'll need to make sure your caps actually work.

Thanks, Eugen

[2] https://docs.ceph.com/en/latest/cephfs/client-auth/#changing-rw-permissions-...

Zitat von Zac Dover <zac.dover@proton.me>:

...

It's in my list of ongoing initiatives. I'll stay up late tonight and ask Venky directly what's going on in this instance.

Sometime later today, I'll create an issue tracking bug and I'll send it to you for review. Make sure that I haven't misrepresented this issue.

Zac

On Wednesday, April 24th, 2024 at 2:10 PM, duluxoz <duluxoz@gmail.com> wrote:

...

Hi Zac,

Any movement on this? We really need to come up with an answer/solution - thanks

Dulux-Oz

On 19/04/2024 18:03, duluxoz wrote:

...

Cool!

Thanks for that :-)

On 19/04/2024 18:01, Zac Dover wrote:

...

I think I understand, after more thought. The second command is expected to work after the first.

I will ask the cephfs team when they wake up.

Zac Dover Upstream Docs Ceph Foundation

On Fri, Apr 19, 2024 at 17:51, duluxoz <[duluxoz@gmail.com](mailto:On Fri, Apr 19, 2024 at 17:51, duluxoz <<a href=)> wrote:

...

Hi All,

In reference to this page from the Ceph documentation: https://docs.ceph.com/en/latest/cephfs/client-auth/, down the bottom of that page it says that you can run the following commands:

~~~ ceph fs authorize a client.x /dir1 rw ceph fs authorize a client.x /dir2 rw ~~~

This will allow `client.x` to access both `dir1` and `dir2`.

So, having a use case where we need to do this, we are, HOWEVER, getting the following error on running the 2nd command on a Reef 18.2.2 cluster:

`Error EINVAL: client.x already has fs capabilities that differ from those supplied. To generate a new auth key for client.x, first remove client.x from configuration files, execute 'ceph auth rm client.x', then execute this command again.`

Something we're doing wrong, or is the doco "out of date" (mind you, that's from the "latest" version of the doco, and the "reef" version), or is something else going on?

Thanks in advance for the help

Cheers

Dulux-Oz

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

---

ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io