Hi,
Are the two clusters the same size? Pools both replicated? Do the nodes where rbd-mirror is running have the same CPU/RAM resources? Is one more >heavily loaded than the other?
Cluster has to be exactly the same to mirroring work well ?
rbd_mirror_journal_max_fetch_bytes and rbd_journal_max_payload_bytes=8388608
I didn't see this part in the docs, my bad. I added this to ceph.conf and it's working a lot faster One more thing: do you have more info about the Ceph subreddit ban, and if it will be lifted soon? Thanks Anthony and Eugen Vivien ________________________________ De : Eugen Block <eblock@nde.ag> Envoyé : jeudi 23 octobre 2025 07:58:08 À : ceph-users@ceph.io Objet : [ceph-users] Re: Very slow mirroring operation Hi, I haven't dealt with journal based mirroring in quite a while, but this section [0] in the docs seems relevant:
rbd-mirror tunables are set by default to values suitable for mirroring an entire pool. When using rbd-mirror to migrate single volumes between clusters you may achieve substantial performance gains by setting rbd_journal_max_payload_bytes=8388608 within the [client] config section of the local or centralized configuration. Note that this setting may allow rbd-mirror to present a substantial write workload to the destination cluster: monitor cluster performance closely during migrations and test carefully before running multiple migrations in parallel.
# ceph config help rbd_journal_max_payload_bytes rbd_journal_max_payload_bytes - maximum journal payload size before splitting (size, advanced) Default: 16384 Can update at runtime: true Services: [rbd] Regards, Eugen [0] https://docs.ceph.com/en/latest/rbd/rbd-mirroring/#enable-image-journaling-f... Zitat von "GLE, Vivien" <Vivien.GLE@inist.fr>:
Hi,
I'm testing mirroring in ceph and dealt with very slow mirroring operation,
Cluster A (primary) is on a Ceph hyperconverged on proxmox and cluster B (secondary) is a Ceph cluster managed by cephadm. These 2 have been peered via bootstrap.
Moving 60 GiB VM to the mirrored pool in cluster A it took 5-6 hours to fully sync in the cluster B (SSD device only)
Moving 60 GiB VM from cluster A to cluster B in replica 3 SSD pool took 2 minutes (SSD device only)
It looks like for this purpose cluster wont be higher than 6-7 MiB/s, is this a normal behaviour ?
Ceph -s during mirroring output and rbd info below
ceph -s cluster: id: ID health: HEALTH_OK
services: mon: 3 daemons, quorum r620-13-1,r620-13-9,r620-13-4 (age 12d) mgr: r620-13-7.zxzajo(active, since 24h), standbys: r620-13-10.wmzodp mds: 1/1 daemons up osd: 24 osds: 24 up (since 6d), 24 in (since 6d) rbd-mirror: 1 daemon active (1 hosts) rgw: 1 daemon active (1 hosts, 1 zones)
data: volumes: 1/1 healthy pools: 16 pools, 1287 pgs objects: 151.21k objects, 551 GiB usage: 1.6 TiB used, 6.3 TiB / 7.9 TiB avail pgs: 1287 active+clean
io: client: 15 KiB/s rd, 6.5 MiB/s wr, 11 op/s rd, 419 op/s wr
--------------------------------------------
rbd info ceph_proxmox/vm-118-disk-0 rbd image 'vm-118-disk-0': size 60 GiB in 15360 objects order 22 (4 MiB objects) snapshot_count: 0 id: abe8c560f69498 block_name_prefix: rbd_data.abe8c560f69498 format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, journaling op_features: flags: create_timestamp: Wed Oct 22 07:52:31 2025 access_timestamp: Wed Oct 22 07:52:31 2025 modify_timestamp: Wed Oct 22 07:52:31 2025 journal: abe8c560f69498 mirroring state: enabled mirroring mode: journal mirroring global id: id mirroring primary: false
-----------------------------------------
In cluster A ceph.conf
rbd_default_features = 125
Thanks !
Vivien
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io