3 Nov
2020
3 Nov
'20
12:32 p.m.
Hi all,
We still have legacy caps on our nautilus rbd cluster. I just wanted
to check if this is totally safe (and to post here ftr because I don't
think this has ever been documented)
Here are the current caps:
[client.images]
key = xxx
caps mgr = "allow r"
caps mon = "allow r, allow command \"osd blacklist\""
caps osd = "allow class-read object_prefix rbd_children, allow rwx pool=images"
[client.volumes]
key = xxx
caps mgr = "allow r"
caps mon = "allow r, allow command \"osd blacklist\""
caps osd = "allow class-read object_prefix rbd_children, allow rwx
pool=volumes, allow rx pool=images, allow rwx pool=cinder-critical"
Now that we upgraded to nautilus we would do:
# ceph auth caps client.images mon 'profile rbd' osd 'profile rbd
pool=images' mgr 'profile rbd pool=images'
# ceph auth caps client.volumes mon 'profile rbd' osd 'profile rbd
pool=volumes, profile rbd-read-only pool=images, profile rbd
pool=cinder-critical' mgr 'profile rbd pool=volumes, profile rbd
pool=cinder-critical'
Does that look correct? Does this apply without impacting any client IOs ?
Thanks!
Dan
3 Nov
3 Nov
2:37 p.m.
On 03/11/2020 10:02, Dan van der Ster wrote:
Hi all,
We still have legacy caps on our nautilus rbd cluster. I just wanted
to check if this is totally safe (and to post here ftr because I don't
think this has ever been documented)
Here are the current caps:
[client.images]
key = xxx
caps mgr = "allow r"
caps mon = "allow r, allow command \"osd blacklist\""
caps osd = "allow class-read object_prefix rbd_children, allow rwx
pool=images"
[client.volumes]
key = xxx
caps mgr = "allow r"
caps mon = "allow r, allow command \"osd blacklist\""
caps osd = "allow class-read object_prefix rbd_children, allow rwx
pool=volumes, allow rx pool=images, allow rwx pool=cinder-critical"
Now that we upgraded to nautilus we would do:
# ceph auth caps client.images mon 'profile rbd' osd 'profile rbd
pool=images' mgr 'profile rbd pool=images'
# ceph auth caps client.volumes mon 'profile rbd' osd 'profile rbd
pool=volumes, profile rbd-read-only pool=images, profile rbd
pool=cinder-critical' mgr 'profile rbd pool=volumes, profile rbd
pool=cinder-critical'
Does that look correct? Does this apply without impacting any client IOs ?
Yes, it looks correct, but what I usually do:
$ ceph auth get client.images -o client.images
$ cp client.images client.images.org
$ edit the client.images file
$ diff -u client.images client.images.org
$ ceph auth import -i client.images
This way I also have a way of reverting quickly if things do go wrong.
What I also did is import the key with a name (eg client.images2) and
test if I could manually perform RBD operations with the 'rbd' cli tool.
Warning: IF you make a mistake (and I have seen this happen!) ceph will
start returning 'Operation Not Permitted' to librados which they causes
I/O errors inside librbd. Your VMs will go into Read-Only as filesystems
break and probably need an fsck to get back.
So triple-check your work before doing this. But if done properly it can
be done online.
Wido
> Thanks!
>
> Dan
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
5:44 p.m.
On Tue, Nov 3, 2020 at 12:07 PM Wido den Hollander <wido(a)42on.com> wrote:
On 03/11/2020 10:02, Dan van der Ster wrote:
Thanks Wido, and good idea with the testing steps beforehand.
We created images2, volumes2, tested that they are all good, then
updated the real users with `ceph auth caps`.
All done!
Cheers, Dan
Hi all,
We still have legacy caps on our nautilus rbd cluster. I just wanted
to check if this is totally safe (and to post here ftr because I don't
think this has ever been documented)
Here are the current caps:
[client.images]
key = xxx
caps mgr = "allow r"
caps mon = "allow r, allow command \"osd blacklist\""
caps osd = "allow class-read object_prefix rbd_children, allow rwx
pool=images"
[client.volumes]
key = xxx
caps mgr = "allow r"
caps mon = "allow r, allow command \"osd blacklist\""
caps osd = "allow class-read object_prefix rbd_children, allow rwx
pool=volumes, allow rx pool=images, allow rwx pool=cinder-critical"
Now that we upgraded to nautilus we would do:
# ceph auth caps client.images mon 'profile rbd' osd 'profile rbd
pool=images' mgr 'profile rbd pool=images'
# ceph auth caps client.volumes mon 'profile rbd' osd 'profile rbd
pool=volumes, profile rbd-read-only pool=images, profile rbd
pool=cinder-critical' mgr 'profile rbd pool=volumes, profile rbd
pool=cinder-critical'
Does that look correct? Does this apply without impacting any client IOs ?
Yes, it looks correct, but what I usually do:
$ ceph auth get client.images -o client.images
$ cp client.images client.images.org
$ edit the client.images file
$ diff -u client.images client.images.org
$ ceph auth import -i client.images
This way I also have a way of reverting quickly if things do go wrong.
What I also did is import the key with a name (eg client.images2) and
test if I could manually perform RBD operations with the 'rbd' cli tool.
Warning: IF you make a mistake (and I have seen this happen!) ceph will
start returning 'Operation Not Permitted' to librados which they causes
I/O errors inside librbd. Your VMs will go into Read-Only as filesystems
break and probably need an fsck to get back.
So triple-check your work before doing this. But if done properly it can
be done online.
1280
days inactive
1280
days old
2 comments
2 participants
participants (2)
-
Dan van der Ster -
Wido den Hollander