On Wed, Jun 18, 2025 at 7:29 PM Julien Laurenceau <julien.laurenceau@pepitedata.com> wrote:
Hi,
I have 2 running ceph squid clusters (19.2.2). On each cluster there is an rbd pool named k8s-1 that I want to mirror using the snapshot based mode.
if I configure both clusters in two way mirroring using the mode=image everything works fine. ``` rbd mirror pool enable ${mypool} image rbd mirror image enable ${mypool}/${myimage} snapshot ``` When I take a snapshot of myimage it gets mirrored.
if I configure both clusters in two way mirroring using the mode=pool nothings happen. The snapshot are never mirrored in the second cluster.
Reading the documentation https://docs.ceph.com/en/squid/rbd/rbd-mirroring/, I think that the problem is that the default mode for this command is "journal":
rbd mirror image enable {pool-name}/{image-name} {mode}
But, when enabling mirror on the pool there is no way to define mode=journal/snapshot:
rbd mirror pool enable [--site-name {local-site-name}] {pool-name} {mode}
with mode=pool (default) or image.
Then I think when I do : rbd mirror pool enable ${mypool} image
I think it implicitly do : rbd mirror pool enable ${mypool} image journal # THIS COMMAND DOES NOT WORK.
But I would need to have: rbd mirror pool enable ${mypool} image snapshot # THIS COMMAND DOES NOT WORK.
In addition, when I am configured on mirror mode = pool, if I try to disable mirroring on an image I get the following error : rbd mirror image disable ${mypool}/${myimage} 2025-06-18T15:37:42.177+0000 7f24ec53ce00 -1 librbd::api::Mirror: image_disable: cannot disable mirroring in the current pool mirroring mode
Also error when I am configured on mirror mode = pool, if I try to enable snapshot mirroring on an image : rbd mirror image enable ${mypool}/${myimage} snapshot 2025-06-18T15:39:44.507+0000 7f533df1fe00 -1 librbd::api::Mirror: image_enable: cannot enable mirroring: pool is not in image mirror mode
Could someone please confirm, that this is not supported ?
Hi Julien, Yes, snapshot-based mirroring is possible only in "image" mode. The "pool" mode is limited to journal-based mirroring. It's stated in the documentation, but perhaps not explicitly enough: You can also mirror all images within a given pool or namespace when using journal-based mirroring.
What is the best workaround ? Cron cephadm shell script to enable mirror on all images ?
PS : I do not want to use the journal based mode since the ceph-csi mounter nbd is not prod-ready.
If you are using Ceph CSI, it would be much better to drive mirroring through Ceph CSI -- see VolumeReplication and related resources. Thanks, Ilya