Hi Shridhar, As Ilya suggested : Use image names instead of device names, i.e. "rbd unmap myimage" instead of "rbd unmap /dev/rbd0". I think this will solve the problem. You just need to advice your orchestrator/hypervisor to use device names instead of rdb0... About udev rule I guess not, because ceph is creating a brand new device every time, but I'm not sure. On Mon, Apr 20, 2020 at 10:27 PM Void Star Nill <void.star.nill@gmail.com> wrote:
Hi Jason,
Why would you need to map the same image multiple times? Is that just a limitation of your container management system? For example, k8s w/ ceph-csi will only map a PVC once per node even if it's used by multiple containers. When the last container is stopped, it will then unmap the image.
The way we orchestrate the RBD volumes today makes it difficult to do it the way k8s CSI does. Volume map and unmap are currently managed by individual hypervisor threads/subprocesses so we just keep a 1:1 mapping of devices to containers.
Thanks, Shridhar
Is it possible to add custom udev rules to control this behavior?
Thanks, Shridhar
On Mon, 20 Apr 2020 at 01:19, Ilya Dryomov <idryomov@gmail.com> wrote:
On Sat, Apr 18, 2020 at 6:53 AM Void Star Nill < void.star.nill@gmail.com> wrote:
Hello,
How frequently do RBD device names get reused? For instance, when I
map a
volume on a client and it gets mapped to /dev/rbd0 and when it is unmapped, does a subsequent map reuse this name right away?
Yes.
I ask this question, because in our use case, we try to unmap a
volume
we are thinking about adding some retries in case unmap fails for any reason. But I am concerned about race conditions such as the following: 1. thread 1 calls unmap, but the call times out and returns in the
but in the background unmap request does go through and the device gets removed 2. thread 1 does a retry based on the device name.
If between 1 and 2, another thread tries to map another volume and if it gets mapped to same device right after previous unmap was successful,
and process, then
in step 2, we will be trying to unmap a device that doesn't belong to previous map.
So I want to know how frequently do the device names get reused, and if there is a way to keep them using new names until they round back after a max limit.
No, there is no way.
Use image names instead of device names, i.e. "rbd unmap myimage" instead of "rbd unmap /dev/rbd0".
Thanks,
Ilya
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Jason
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io