Changing it to rx-tx seems to do the whole magic.
After restarting rbd-mirror on the backup site the image was synced.
Thanks a lot for your perfect and fast support.
Solving problems this way is fun.
Yves
-----Ursprüngliche Nachricht-----
Von: Jason Dillaman <jdillama(a)redhat.com>
Gesendet: Freitag, 24. Juli 2020 23:11
An: Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
Cc: dillaman <dillaman(a)redhat.com>om>; ceph-users <ceph-users(a)ceph.io>
Betreff: Re: [ceph-users] Re: journal based mirroring works but snapshot based not
On Fri, Jul 24, 2020 at 3:49 PM Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
wrote:
Yes, there are some:
2020-07-24T21:46:25.974+0200 7f932fccd700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_task:
2020-07-24T21:46:25.974+0200 7f93304ce700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 get_mirror_uuid:
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_get_mirror_uuid:
r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_get_mirror_uuid:
remote_mirror_uuid=ce9e8f12-c8e2-4c16-bd74-f28f29d405a3
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 mirror_peer_ping:
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_ping:
r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 mirror_peer_list:
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_list:
r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_list:
remote_mirror_peer_uuid=
Yeah, it's not finding itself (by site-name) in the remote peer cluster. Presuming
this was a log from your "backup" site's rbd-mirror daemon, I think the
issue is that your peer record on "master" for "backup" is listed as a
"tx-only" peer, which means that "backup" will only transmit images to
"master". Can you try changing the peer direction on "master" to
"rx-only" (i.e. it will only receive images from "master") or
"rx-tx" for bi-directional mirroring?
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 schedule_task:
-----Ursprüngliche Nachricht-----
Von: Jason Dillaman <jdillama(a)redhat.com>
Gesendet: Freitag, 24. Juli 2020 21:34
An: Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Betreff: Re: [ceph-users] Re: journal based mirroring works but
snapshot based not
On Fri, Jul 24, 2020 at 3:22 PM <yveskretzschmar(a)web.de> wrote:
Hi Jason,
now I'm a bit confused ...
I recreated the peers again with names clear:
#main site:
rbd mirror pool info testpool
Mode: image
Site Name: master
Peer Sites:
UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294
Name: backup
Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a
Direction: tx-only
#backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1
Name: master
Mirror UUID:
Direction: rx-tx
Client: client.admin
In my opinion this looks ok, but the error seems the same:
2020-07-24T21:14:21.559+0200 7fb9d709a700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init:
failed to retrieve mirror peer uuid from remote pool
2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
Turn your rbd-mirror debug level up from 5 to at least 10 and retry.
Every 30 seconds or so there should be a log message from the
"RemotePoolPoller" which is responsible for pulling the mirror peer uuid from
the remote cluster.
If I disable mirroring on the testimage on main site, the mirror daemon tells me:
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_remote_update_watcher: failed to unregister remote
update watcher: (2) No such file or directory
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_local_update_watcher: failed to unregister local
update watcher: (2) No such file or directory
The created image on the backup site doesn't get removed.
Journal based mirroring works without any problem on the same image.
Thanks for any hints
Yves
-----Ursprüngliche Nachricht-----
Betreff: Re: Re: [ceph-users] Re: journal based mirroring works but
snapshot based not
You need to have unique names for your site -- so "ceph" is not an appropriate
name. Right now, your backup set is named "ceph" and you have a peer named
"ceph".
On Fri, Jul 24, 2020 at 8:43 AM Yves wrote:
Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15
site-name backup"
get's me:
2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror:
peer_site_set_name: failed to update site
'0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists
So I deleted peers again and recreated with same site-name.
On main site it looks this way:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7
Name: ceph
Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da
Direction: tx-only
On backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a
Name: ceph
Mirror UUID:
Direction: rx-tx
Client: client.admin
But situation didn't change.
Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr
Von: "Jason Dillaman"
Cc: "ceph-users" <ceph-users(a)ceph.io>
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
At the backup site, run "rbd mirror pool peer set testpool
0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update
the site-name to match.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe
send an email to ceph-users-leave(a)ceph.io
--
Jason
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
--
Jason
--
Jason