24 Jul
2020
24 Jul
'20
1:48 p.m.
Hi,
i have a working journal based mirror setup initially created with nautilus.
I recently upgraded to octopus (15.2.4) to use snapshot based mirroring.
After that I disabled mirroring for the first image and reenabled it snapshot based.
The remote Image is created but it never starts syncing.
Network usage is only about some kb/s.
And after disabling mirroring on the image the remote images doesn't get deleted.
The same is with a new test pool and new images.
I also remove the peer from the pools and disabled mirroring on the pools and recreated it
from scratch, because I've found this:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/LRBDKQ2YXQN…
bit it didn't resolved it.
Mirroring daemon shows the follwing after enabling snapshot based mirroring on an image:
2020-07-24T13:45:40.253+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images: peer_uuid=3d2164b5-96ca-4ad6-9fe3-4eb1f6c02f3e, added_count=1,
removed_count=0
2020-07-24T13:45:40.253+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images_added: peer_uuid=3d2164b5-96ca-4ad6-9fe3-4eb1f6c02f3e,
global_image_ids=[d7f7d308-d18b-488f-9dcc-87f8e77f05cc]
2020-07-24T13:45:40.253+0200 7f33dae1d700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 add_image: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 start_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=MAP_UPDATE
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 execute_policy_action:
global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc, policy_action=MAP
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 map: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
instance_id=10464477
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_image_mapping: updates=[{global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
instance_id=10464477}], removes=[]
2020-07-24T13:45:41.258+0200 7f33dae1d700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 finish_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=MAP_UPDATE, r=0
2020-07-24T13:45:41.294+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:41.402+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:41.502+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:41.606+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:45:42.258+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 start_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=ACQUIRE
2020-07-24T13:45:42.258+0200 7f33da61c700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
notify_listener_acquire_release_images:
acquire=[{global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc, instance_id=10464477}],
release=[]
2020-07-24T13:45:42.258+0200 7f33da61c700 5 rbd::mirror::NamespaceReplayer:
0x55cfcc4c6000 handle_acquire_image: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
instance_id=10464477
2020-07-24T13:45:42.258+0200 7f33dae1d700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 finish_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=ACQUIRE, r=0
2020-07-24T13:45:42.306+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images: peer_uuid=, added_count=1, removed_count=0
2020-07-24T13:45:42.306+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images_added: peer_uuid=, global_image_ids=[d7f7d308-d18b-488f-9dcc-87f8e77f05cc]
2020-07-24T13:45:42.330+0200 7f33a3daf700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 init: failed to retrieve
mirror peer uuid from remote pool
2020-07-24T13:45:42.330+0200 7f33dae1d700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 get_replay_status: replay
not running
2020-07-24T13:45:46.298+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:46.402+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:46.506+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:46.606+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:45:51.298+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:51.406+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:51.506+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:51.610+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:45:52.330+0200 7f33dae1d700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 get_replay_status: replay
not running
2020-07-24T13:45:56.298+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:56.406+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:56.506+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:56.610+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:46:01.303+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:46:01.407+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:46:01.511+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:46:01.611+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:46:02.331+0200 7f33dae1d700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 get_replay_status: replay
not running
Regards,
Yves
24 Jul
24 Jul
1:58 p.m.
On Fri, Jul 24, 2020 at 7:49 AM <yveskretzschmar(a)web.de> wrote:
Hi,
i have a working journal based mirror setup initially created with nautilus.
I recently upgraded to octopus (15.2.4) to use snapshot based mirroring.
After that I disabled mirroring for the first image and reenabled it snapshot based.
The remote Image is created but it never starts syncing.
Network usage is only about some kb/s.
And after disabling mirroring on the image the remote images doesn't get deleted.
The same is with a new test pool and new images.
I also remove the peer from the pools and disabled mirroring on the pools and recreated
it from scratch, because I've found this:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/LRBDKQ2YXQN…
bit it didn't resolved it.
Mirroring daemon shows the follwing after enabling snapshot based mirroring on an image:
2020-07-24T13:45:40.253+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images: peer_uuid=3d2164b5-96ca-4ad6-9fe3-4eb1f6c02f3e, added_count=1,
removed_count=0
2020-07-24T13:45:40.253+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images_added: peer_uuid=3d2164b5-96ca-4ad6-9fe3-4eb1f6c02f3e,
global_image_ids=[d7f7d308-d18b-488f-9dcc-87f8e77f05cc]
2020-07-24T13:45:40.253+0200 7f33dae1d700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 add_image: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 start_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=MAP_UPDATE
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 execute_policy_action:
global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc, policy_action=MAP
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 map: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
instance_id=10464477
2020-07-24T13:45:41.254+0200 7f33da61c700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_image_mapping: updates=[{global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
instance_id=10464477}], removes=[]
2020-07-24T13:45:41.258+0200 7f33dae1d700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 finish_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=MAP_UPDATE, r=0
2020-07-24T13:45:41.294+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:41.402+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:41.502+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:41.606+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:45:42.258+0200 7f33da61c700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 start_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=ACQUIRE
2020-07-24T13:45:42.258+0200 7f33da61c700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
notify_listener_acquire_release_images:
acquire=[{global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc, instance_id=10464477}],
release=[]
2020-07-24T13:45:42.258+0200 7f33da61c700 5 rbd::mirror::NamespaceReplayer:
0x55cfcc4c6000 handle_acquire_image: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
instance_id=10464477
2020-07-24T13:45:42.258+0200 7f33dae1d700 5 rbd::mirror::image_map::Policy:
0x55cfc8825a40 finish_action: global_image_id=d7f7d308-d18b-488f-9dcc-87f8e77f05cc,
state=ASSOCIATING, action_type=ACQUIRE, r=0
2020-07-24T13:45:42.306+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images: peer_uuid=, added_count=1, removed_count=0
2020-07-24T13:45:42.306+0200 7f33dae1d700 5 rbd::mirror::ImageMap: 0x55cfcc4c0480
update_images_added: peer_uuid=, global_image_ids=[d7f7d308-d18b-488f-9dcc-87f8e77f05cc]
2020-07-24T13:45:42.330+0200 7f33a3daf700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 init: failed to retrieve
mirror peer uuid from remote pool
Can you provide the output from "rbd mirror pool info" on both sides?
The site-names need to align across both sides so that the two sites
know who-is-who.
2020-07-24T13:45:42.330+0200 7f33dae1d700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 get_replay_status: replay
not running
2020-07-24T13:45:46.298+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:46.402+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:46.506+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:46.606+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:45:51.298+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:51.406+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:51.506+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:51.610+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:45:52.330+0200 7f33dae1d700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 get_replay_status: replay
not running
2020-07-24T13:45:56.298+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:45:56.406+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:45:56.506+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:45:56.610+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:46:01.303+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc7c1bd40
handle_acked: instance_ids=[10464466]
2020-07-24T13:46:01.407+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfc9424240
handle_acked: instance_ids=[10464470]
2020-07-24T13:46:01.511+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcac66240
handle_acked: instance_ids=[10464474]
2020-07-24T13:46:01.611+0200 7f33dae1d700 5 rbd::mirror::Instances: 0x55cfcc4c0360
handle_acked: instance_ids=[10464477]
2020-07-24T13:46:02.331+0200 7f33dae1d700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55cfcd28c800 get_replay_status: replay
not running
Regards,
Yves
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Jason
2:01 p.m.
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
2:04 p.m.
On Fri, Jul 24, 2020 at 8:02 AM <yveskretzschmar(a)web.de> wrote:
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
At the backup site, run "rbd mirror pool peer set testpool
0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update the
site-name to match.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Jason
9:21 p.m.
Hi Jason,
now I'm a bit confused ...
I recreated the peers again with names clear:
#main site:
rbd mirror pool info testpool
Mode: image
Site Name: master
Peer Sites:
UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294
Name: backup
Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a
Direction: tx-only
#backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1
Name: master
Mirror UUID:
Direction: rx-tx
Client: client.admin
In my opinion this looks ok, but the error seems the same:
2020-07-24T21:14:21.559+0200 7fb9d709a700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init: failed to retrieve
mirror peer uuid from remote pool
2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 get_replay_status: replay
not running
2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances: 0x56122af49d40
handle_acked: instance_ids=[10464751]
2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances: 0x56122af49d40
handle_acked: instance_ids=[10464751]
2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 get_replay_status: replay
not running
2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances: 0x56122af49d40
handle_acked: instance_ids=[10464751]
If I disable mirroring on the testimage on main site, the mirror daemon tells me:
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_remote_update_watcher: failed to unregister remote update watcher: (2)
No such file or directory
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_local_update_watcher: failed to unregister local update watcher: (2) No
such file or directory
The created image on the backup site doesn't get removed.
Journal based mirroring works without any problem on the same image.
Thanks for any hints
Yves
-----Ursprüngliche Nachricht-----
Betreff: Re: Re: [ceph-users] Re: journal based mirroring works but snapshot based not
You need to have unique names for your site -- so "ceph" is not an appropriate
name. Right now, your backup set is named "ceph" and you have a peer named
"ceph".
On Fri, Jul 24, 2020 at 8:43 AM Yves wrote:
>
> Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15
site-name backup"
> get's me:
> 2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror:
> peer_site_set_name: failed to update site
> '0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists
>
> So I deleted peers again and recreated with same site-name.
>
> On main site it looks this way:
> rbd mirror pool info testpool
> Mode: image
> Site Name: ceph
> Peer Sites:
> UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7
> Name: ceph
> Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da
> Direction: tx-only
>
>
> On backup site:
> rbd --cluster backup mirror pool info testpool
> Mode: image
> Site Name: ceph
> Peer Sites:
> UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a
> Name: ceph
> Mirror UUID:
> Direction: rx-tx
> Client: client.admin
>
>
> But situation didn't change.
>
>
> Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr
> Von: "Jason Dillaman"
> Cc: "ceph-users" <ceph-users(a)ceph.io>
> >
> > Hi,
> >
> > this is the main site:
> >
> > rbd mirror pool info testpool
> > Mode: image
> > Site Name: ceph
> >
> > Peer Sites:
> >
> > UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
> > Name: backup
> > Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
> > Direction: tx-only
> >
> >
> > and the backup site:
> >
> > rbd --cluster backup mirror pool info testpool
> > Mode: image
> > Site Name: backup
> >
> > Peer Sites:
> >
> > UUID: 0622b647-2c24-4840-93d0-efdee180ab15
> > Name: ceph
> > Direction: rx-only
> > Client: client.rbd-mirror-peer
>
> At the backup site, run "rbd mirror pool peer set testpool
> 0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update the
> site-name to match.
>
> > _______________________________________________
> > ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
> > email to ceph-users-leave(a)ceph.io
> >
>
>
> --
> Jason
>
9:33 p.m.
On Fri, Jul 24, 2020 at 3:22 PM <yveskretzschmar(a)web.de> wrote:
Hi Jason,
now I'm a bit confused ...
I recreated the peers again with names clear:
#main site:
rbd mirror pool info testpool
Mode: image
Site Name: master
Peer Sites:
UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294
Name: backup
Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a
Direction: tx-only
#backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1
Name: master
Mirror UUID:
Direction: rx-tx
Client: client.admin
In my opinion this looks ok, but the error seems the same:
2020-07-24T21:14:21.559+0200 7fb9d709a700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init: failed to retrieve
mirror peer uuid from remote pool
2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 get_replay_status: replay
not running
2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances: 0x56122af49d40
handle_acked: instance_ids=[10464751]
2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances: 0x56122af49d40
handle_acked: instance_ids=[10464751]
2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 get_replay_status: replay
not running
2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances: 0x56122af49d40
handle_acked: instance_ids=[10464751]
Turn your rbd-mirror debug level up from 5 to at least 10 and retry.
Every 30 seconds or so there should be a log message from the
"RemotePoolPoller" which is responsible for pulling the mirror peer
uuid from the remote cluster.
If I disable mirroring on the testimage on main site, the mirror daemon tells me:
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_remote_update_watcher: failed to unregister remote update watcher: (2)
No such file or directory
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_local_update_watcher: failed to unregister local update watcher: (2) No
such file or directory
The created image on the backup site doesn't get removed.
Journal based mirroring works without any problem on the same image.
Thanks for any hints
Yves
-----Ursprüngliche Nachricht-----
Betreff: Re: Re: [ceph-users] Re: journal based mirroring works but snapshot based not
You need to have unique names for your site -- so "ceph" is not an appropriate
name. Right now, your backup set is named "ceph" and you have a peer named
"ceph".
On Fri, Jul 24, 2020 at 8:43 AM Yves wrote:
--
Jason
Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15
site-name backup"
get's me:
2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror:
peer_site_set_name: failed to update site
'0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists
So I deleted peers again and recreated with same site-name.
On main site it looks this way:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7
Name: ceph
Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da
Direction: tx-only
On backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a
Name: ceph
Mirror UUID:
Direction: rx-tx
Client: client.admin
But situation didn't change.
Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr
Von: "Jason Dillaman"
Cc: "ceph-users" <ceph-users(a)ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
At the backup site, run "rbd mirror pool peer set testpool
0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update the
site-name to match.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
--
Jason
9:49 p.m.
Yes, there are some:
2020-07-24T21:46:25.974+0200 7f932fccd700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
handle_task:
2020-07-24T21:46:25.974+0200 7f93304ce700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
get_mirror_uuid:
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
handle_get_mirror_uuid: r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
handle_get_mirror_uuid: remote_mirror_uuid=ce9e8f12-c8e2-4c16-bd74-f28f29d405a3
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
mirror_peer_ping:
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
handle_mirror_peer_ping: r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
mirror_peer_list:
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
handle_mirror_peer_list: r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
handle_mirror_peer_list: remote_mirror_peer_uuid=
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller: 0x5628339d92b0
schedule_task:
-----Ursprüngliche Nachricht-----
Von: Jason Dillaman <jdillama(a)redhat.com>
Gesendet: Freitag, 24. Juli 2020 21:34
An: Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Betreff: Re: [ceph-users] Re: journal based mirroring works but snapshot based not
On Fri, Jul 24, 2020 at 3:22 PM <yveskretzschmar(a)web.de> wrote:
Hi Jason,
now I'm a bit confused ...
I recreated the peers again with names clear:
#main site:
rbd mirror pool info testpool
Mode: image
Site Name: master
Peer Sites:
UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294
Name: backup
Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a
Direction: tx-only
#backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1
Name: master
Mirror UUID:
Direction: rx-tx
Client: client.admin
In my opinion this looks ok, but the error seems the same:
2020-07-24T21:14:21.559+0200 7fb9d709a700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init:
failed to retrieve mirror peer uuid from remote pool
2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
Turn your rbd-mirror debug level up from 5 to at least 10 and retry.
Every 30 seconds or so there should be a log message from the "RemotePoolPoller"
which is responsible for pulling the mirror peer uuid from the remote cluster.
If I disable mirroring on the testimage on main site, the mirror daemon tells me:
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_remote_update_watcher: failed to unregister remote
update watcher: (2) No such file or directory
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_local_update_watcher: failed to unregister local
update watcher: (2) No such file or directory
The created image on the backup site doesn't get removed.
Journal based mirroring works without any problem on the same image.
Thanks for any hints
Yves
-----Ursprüngliche Nachricht-----
Betreff: Re: Re: [ceph-users] Re: journal based mirroring works but
snapshot based not
You need to have unique names for your site -- so "ceph" is not an appropriate
name. Right now, your backup set is named "ceph" and you have a peer named
"ceph".
On Fri, Jul 24, 2020 at 8:43 AM Yves wrote:
--
Jason
Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15
site-name backup"
get's me:
2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror:
peer_site_set_name: failed to update site
'0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists
So I deleted peers again and recreated with same site-name.
On main site it looks this way:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7
Name: ceph
Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da
Direction: tx-only
On backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a
Name: ceph
Mirror UUID:
Direction: rx-tx
Client: client.admin
But situation didn't change.
Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr
Von: "Jason Dillaman"
Cc: "ceph-users" <ceph-users(a)ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
At the backup site, run "rbd mirror pool peer set testpool
0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update the
site-name to match.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send
an email to ceph-users-leave(a)ceph.io
--
Jason
11:10 p.m.
On Fri, Jul 24, 2020 at 3:49 PM Yves Kretzschmar-Schwipper
<yveskretzschmar(a)web.de> wrote:
Yes, there are some:
2020-07-24T21:46:25.974+0200 7f932fccd700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_task:
2020-07-24T21:46:25.974+0200 7f93304ce700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 get_mirror_uuid:
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_get_mirror_uuid: r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_get_mirror_uuid:
remote_mirror_uuid=ce9e8f12-c8e2-4c16-bd74-f28f29d405a3
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 mirror_peer_ping:
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_mirror_peer_ping: r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 mirror_peer_list:
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_mirror_peer_list: r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_mirror_peer_list: remote_mirror_peer_uuid=
Yeah, it's not finding itself (by site-name) in the remote peer
cluster. Presuming this was a log from your "backup" site's rbd-mirror
daemon, I think the issue is that your peer record on "master" for
"backup" is listed as a "tx-only" peer, which means that
"backup" will
only transmit images to "master". Can you try changing the peer
direction on "master" to "rx-only" (i.e. it will only receive images
from "master") or "rx-tx" for bi-directional mirroring?
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 schedule_task:
-----Ursprüngliche Nachricht-----
Von: Jason Dillaman <jdillama(a)redhat.com>
Gesendet: Freitag, 24. Juli 2020 21:34
An: Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Betreff: Re: [ceph-users] Re: journal based mirroring works but snapshot based not
On Fri, Jul 24, 2020 at 3:22 PM <yveskretzschmar(a)web.de> wrote:
--
Jason
Hi Jason,
now I'm a bit confused ...
I recreated the peers again with names clear:
#main site:
rbd mirror pool info testpool
Mode: image
Site Name: master
Peer Sites:
UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294
Name: backup
Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a
Direction: tx-only
#backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1
Name: master
Mirror UUID:
Direction: rx-tx
Client: client.admin
In my opinion this looks ok, but the error seems the same:
2020-07-24T21:14:21.559+0200 7fb9d709a700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init:
failed to retrieve mirror peer uuid from remote pool
2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
Turn your rbd-mirror debug level up from 5 to at least 10 and retry.
Every 30 seconds or so there should be a log message from the
"RemotePoolPoller" which is responsible for pulling the mirror peer uuid from
the remote cluster.
If I disable mirroring on the testimage on main site, the mirror daemon tells me:
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_remote_update_watcher: failed to unregister remote
update watcher: (2) No such file or directory
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_local_update_watcher: failed to unregister local
update watcher: (2) No such file or directory
The created image on the backup site doesn't get removed.
Journal based mirroring works without any problem on the same image.
Thanks for any hints
Yves
-----Ursprüngliche Nachricht-----
Betreff: Re: Re: [ceph-users] Re: journal based mirroring works but
snapshot based not
You need to have unique names for your site -- so "ceph" is not an appropriate
name. Right now, your backup set is named "ceph" and you have a peer named
"ceph".
On Fri, Jul 24, 2020 at 8:43 AM Yves wrote:
--
Jason
Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15
site-name backup"
get's me:
2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror:
peer_site_set_name: failed to update site
'0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists
So I deleted peers again and recreated with same site-name.
On main site it looks this way:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7
Name: ceph
Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da
Direction: tx-only
On backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a
Name: ceph
Mirror UUID:
Direction: rx-tx
Client: client.admin
But situation didn't change.
Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr
Von: "Jason Dillaman"
Cc: "ceph-users" <ceph-users(a)ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
At the backup site, run "rbd mirror pool peer set testpool
0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update the
site-name to match.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send
an email to ceph-users-leave(a)ceph.io
--
Jason
11:47 p.m.
Changing it to rx-tx seems to do the whole magic.
After restarting rbd-mirror on the backup site the image was synced.
Thanks a lot for your perfect and fast support.
Solving problems this way is fun.
Yves
-----Ursprüngliche Nachricht-----
Von: Jason Dillaman <jdillama(a)redhat.com>
Gesendet: Freitag, 24. Juli 2020 23:11
An: Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
Cc: dillaman <dillaman(a)redhat.com>om>; ceph-users <ceph-users(a)ceph.io>
Betreff: Re: [ceph-users] Re: journal based mirroring works but snapshot based not
On Fri, Jul 24, 2020 at 3:49 PM Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
wrote:
Yes, there are some:
2020-07-24T21:46:25.974+0200 7f932fccd700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 handle_task:
2020-07-24T21:46:25.974+0200 7f93304ce700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 get_mirror_uuid:
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_get_mirror_uuid:
r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_get_mirror_uuid:
remote_mirror_uuid=ce9e8f12-c8e2-4c16-bd74-f28f29d405a3
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 mirror_peer_ping:
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_ping:
r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10 rbd::mirror::RemotePollPoller:
0x5628339d92b0 mirror_peer_list:
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_list:
r=0
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 handle_mirror_peer_list:
remote_mirror_peer_uuid=
Yeah, it's not finding itself (by site-name) in the remote peer cluster. Presuming
this was a log from your "backup" site's rbd-mirror daemon, I think the
issue is that your peer record on "master" for "backup" is listed as a
"tx-only" peer, which means that "backup" will only transmit images to
"master". Can you try changing the peer direction on "master" to
"rx-only" (i.e. it will only receive images from "master") or
"rx-tx" for bi-directional mirroring?
2020-07-24T21:46:25.978+0200 7f931dca9700 10
rbd::mirror::RemotePollPoller: 0x5628339d92b0 schedule_task:
-----Ursprüngliche Nachricht-----
Von: Jason Dillaman <jdillama(a)redhat.com>
Gesendet: Freitag, 24. Juli 2020 21:34
An: Yves Kretzschmar-Schwipper <yveskretzschmar(a)web.de>
Cc: ceph-users <ceph-users(a)ceph.io>
Betreff: Re: [ceph-users] Re: journal based mirroring works but
snapshot based not
On Fri, Jul 24, 2020 at 3:22 PM <yveskretzschmar(a)web.de> wrote:
--
Jason
Hi Jason,
now I'm a bit confused ...
I recreated the peers again with names clear:
#main site:
rbd mirror pool info testpool
Mode: image
Site Name: master
Peer Sites:
UUID: e7443ebd-b9e2-47ab-bcfe-f40cb75a6294
Name: backup
Mirror UUID: e0d03424-1913-4e0b-90dd-61defabae51a
Direction: tx-only
#backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: a7ffac0b-09e3-427c-a3a1-5f001a55dcf1
Name: master
Mirror UUID:
Direction: rx-tx
Client: client.admin
In my opinion this looks ok, but the error seems the same:
2020-07-24T21:14:21.559+0200 7fb9d709a700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800 init:
failed to retrieve mirror peer uuid from remote pool
2020-07-24T21:14:21.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:22.515+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:27.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
2020-07-24T21:14:31.559+0200 7fb9e28b1700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x56122bb43800
get_replay_status: replay not running
2020-07-24T21:14:32.519+0200 7fb9e28b1700 5 rbd::mirror::Instances:
0x56122af49d40 handle_acked: instance_ids=[10464751]
Turn your rbd-mirror debug level up from 5 to at least 10 and retry.
Every 30 seconds or so there should be a log message from the
"RemotePoolPoller" which is responsible for pulling the mirror peer uuid from
the remote cluster.
If I disable mirroring on the testimage on main site, the mirror daemon tells me:
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_remote_update_watcher: failed to unregister remote
update watcher: (2) No such file or directory
2020-07-24T21:15:58.775+0200 7f3cd6f3b700 -1
rbd::mirror::image_replayer::snapshot::Replayer: 0x55fbad17c800
handle_unregister_local_update_watcher: failed to unregister local
update watcher: (2) No such file or directory
The created image on the backup site doesn't get removed.
Journal based mirroring works without any problem on the same image.
Thanks for any hints
Yves
-----Ursprüngliche Nachricht-----
Betreff: Re: Re: [ceph-users] Re: journal based mirroring works but
snapshot based not
You need to have unique names for your site -- so "ceph" is not an appropriate
name. Right now, your backup set is named "ceph" and you have a peer named
"ceph".
On Fri, Jul 24, 2020 at 8:43 AM Yves wrote:
--
Jason
Runing "rbd mirror pool peer set testpool 0622b647-2c24-4840-93d0-efdee180ab15
site-name backup"
get's me:
2020-07-24T14:07:57.205+0200 7fc414af53c0 -1 librbd::api::Mirror:
peer_site_set_name: failed to update site
'0622b647-2c24-4840-93d0-efdee180ab15': (17) File exists
So I deleted peers again and recreated with same site-name.
On main site it looks this way:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: e68b09de-1d2c-4ec6-9350-a6ccad26e1b7
Name: ceph
Mirror UUID: 4d7f87f4-47be-46dd-85f1-79caa3fa23da
Direction: tx-only
On backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 399e902c-deee-498c-bcdc-1c4f7a59433a
Name: ceph
Mirror UUID:
Direction: rx-tx
Client: client.admin
But situation didn't change.
Gesendet: Freitag, 24. Juli 2020 um 14:04 Uhr
Von: "Jason Dillaman"
Cc: "ceph-users" <ceph-users(a)ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
email to ceph-users-leave(a)ceph.io
Hi,
this is the main site:
rbd mirror pool info testpool
Mode: image
Site Name: ceph
Peer Sites:
UUID: 1f1877cb-5753-4a0e-8b8c-5e5547c0619e
Name: backup
Mirror UUID: e9e2c4a0-1900-4db6-b828-e655be5ed9d8
Direction: tx-only
and the backup site:
rbd --cluster backup mirror pool info testpool
Mode: image
Site Name: backup
Peer Sites:
UUID: 0622b647-2c24-4840-93d0-efdee180ab15
Name: ceph
Direction: rx-only
Client: client.rbd-mirror-peer
At the backup site, run "rbd mirror pool peer set testpool
0622b647-2c24-4840-93d0-efdee180ab15 site-name backup" to update
the site-name to match.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe
send an email to ceph-users-leave(a)ceph.io
--
Jason
1366
days inactive
1366
days old
8 comments
3 participants
participants (3)
-
Jason Dillaman -
Yves Kretzschmar-Schwipper -
yveskretzschmar＠web.de