[ceph-users] Re: Unable to enable RBD-Mirror Snapshot on image when VM is using RBD

Doing some testing it seems this issue only happens about 10% of the time. Haven't really correlated anything that could be the cause. A resync gets things going again. Anyway to further determine what this means? description: failed to copy snapshots from remote to local image From: "adamb" <adamb(a)medent.com> To: "dillaman" <dillaman(a)redhat.com> Cc: "ceph-users" <ceph-users(a)ceph.io> Sent: Friday, January 29, 2021 11:00:01 AM Subject: [ceph-users] Re: Unable to enable RBD-Mirror Snapshot on image when VM is using RBD Spot on, once I upgraded the client to 15.2.8 I was able to enable rbd-mirror snapshots and create them while the VM was running. However, I have noticed that I am also able to break replication when the rbd is being used by a KVM VM. While writing data in the VM, I took a rbd snapshot, then kicked off a mirror snapshot. I took the rbd snapshot and mirror snapshot from one the osd nodes. Primary vm-101-disk-0: global_id: 68fe424e-3683-48af-ba39-2b0b4726d918 state: up+stopped description: local image is primary service: admin on Ccscephmon1 last_update: 2021-01-29 10:44:40 peer_sites: name: bunker state: up+error description: failed to copy snapshots from remote to local image last_update: 2021-01-29 10:44:50 root@Ccscephtest1:~# rbd snap ls --all CephTestPool1/vm-101-disk-0 SNAPID NAME SIZE PROTECTED TIMESTAMP NAMESPACE 1581 .mirror.primary.68fe424e-3683-48af-ba39-2b0b4726d918.3f9da1d2-7f23-4a75-865e-3af062be93be 2 TiB Fri Jan 29 10:28:53 2021 mirror (primary peer_uuids:[35c97c1f-a897-4635-abf6-dbd4085c89bd]) 1584 Test 2 TiB Fri Jan 29 10:37:37 2021 user 1585 .mirror.primary.68fe424e-3683-48af-ba39-2b0b4726d918.24744afa-2d9f-4958-8775-c23b0b7f383c 2 TiB Fri Jan 29 10:37:47 2021 mirror (primary peer_uuids:[35c97c1f-a897-4635-abf6-dbd4085c89bd]) Secondary vm-101-disk-0: global_id: 68fe424e-3683-48af-ba39-2b0b4726d918 state: up+error description: failed to copy snapshots from remote to local image service: admin on Bunkcephtest1 last_update: 2021-01-29 10:47:12 peer_sites: name: ccs state: up+stopped description: local image is primary last_update: 2021-01-29 10:47:10 root@Bunkcephtest1:~# rbd snap ls --all CephTestPool1/vm-101-disk-0 SNAPID NAME SIZE PROTECTED TIMESTAMP NAMESPACE 1552 Test 2 TiB Fri Jan 29 10:28:53 2021 user 1553 .mirror.non_primary.68fe424e-3683-48af-ba39-2b0b4726d918.0cd471db-7ed7-4700-8da0-8a82bc2d23a5 2 TiB Fri Jan 29 10:28:54 2021 mirror (non-primary peer_uuids:[] adcedc94-b1a1-4508-8402-db4ec0dc3085:1581 copied) I tried restarting the rbd-mirror services but no luck. I ended up issuing a resync and that got things moving until I repeated the steps again. I have the option of using KRBD but not sure if that will help in this situation. From: "Jason Dillaman" <jdillama(a)redhat.com> To: "adamb" <adamb(a)medent.com> Cc: "ceph-users" <ceph-users(a)ceph.io> Sent: Friday, January 29, 2021 9:39:28 AM Subject: Re: [ceph-users] Unable to enable RBD-Mirror Snapshot on image when VM is using RBD On Fri, Jan 29, 2021 at 9:34 AM Adam Boyhan <adamb(a)medent.com> wrote: I suspect that you have the exclusive-lock feature enabled and the QEMU hypervisor host has a pre-octopus version of librbd. -- Jason _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io