Both clusters show sync status are up to date or is there any behind shards, how many gw endpoints you configure and what rgw_rados_hander parameters are set? On Mon, Apr 26, 2021 at 8:59 AM 特木勒 &lt;twl007(a)gmail.com&gt; wrote: > Another problem I notice for a new bucket, the first object in the bucket > will not be sync. the sync will start with the second object. I tried to > fix the index on the bucket and manually rerun bucket sync, but the first > object still does not sync with secondary cluster. > > Do you have any ideas for this issue? > > Thanks > > 特木勒 &lt;twl007(a)gmail.com&gt; 于2021年4月26日周一 上午11:16写道： > >> Hi Istvan: >> >> Thanks Amit's suggestion. >> >> I followed his suggestion to fix bucket index and re-do sync on buckets, >> but it still did not work for me. >> >> Then I tried to use bucket rewrite command to rewrite all the objects in >> buckets and it works for me. I think the reason is there's something wrong >> with bucket index and rewrite has rebuilt the index. >> >> Here's the command I use: >> `sudo radosgw-admin bucket rewrite -b BUCKET-NAME --min-rewrite-size 0` >> >> Maybe you can try this to fix the sync issues. >> >> @Amit Ghadge &lt;amitg.b14(a)gmail.com&gt; Thanks for your suggestions. Without >> your suggestions, I will not notice something wrong with index part. >> >> Thanks :) >> >> Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.com&gt; 于2021年4月26日周一 上午9:57写道： >> >>> Hi, >>> >>> >>> >>> No, doesn’t work, now we will write our own sync app for ceph, I gave >>> up. >>> >>> >>> >>> Istvan Szabo >>> Senior Infrastructure Engineer >>> --------------------------------------------------- >>> Agoda Services Co., Ltd. >>> e: istvan.szabo(a)agoda.com >>> --------------------------------------------------- >>> >>> >>> >>> *From:* 特木勒 &lt;twl007(a)gmail.com&gt; >>> *Sent:* Friday, April 23, 2021 7:50 PM >>> *To:* Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.com&gt; >>> *Cc:* ceph-users(a)ceph.io >>> *Subject:* Re: [Suspicious newsletter] [ceph-users] RGW: Multiple Site >>> does not sync olds data >>> >>> >>> >>> Hi Istvan: >>> >>> >>> >>> We just upgraded whole cluster to 15.2.10 and the multiple site still >>> cannot sync whole objects to secondary cluster. 🙁 >>> >>> >>> >>> Do you have any suggestions on this? And I open another issues in ceph >>> tracker site: >>> >>> https://tracker.ceph.com/issues/50474 >>> >>> >>> >>> Hope someone could go to check this issue. >>> >>> >>> >>> Thanks >>> >>> >>> >>> 特木勒 &lt;twl007(a)gmail.xn--com&gt;2021322-6x8vy47pnp0auje 周一下午9:08写道： >>> >>> Thank you~ >>> >>> >>> >>> I will try to upgrade cluster too. Seem like this is the only way for >>> now. 😭 >>> >>> >>> >>> I will let you know once I complete testing. :) >>> >>> >>> >>> Have a good day >>> >>> >>> >>> Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.xn--com&gt;2021322-6x8vy47pnp0auje 周一下午3:38写道： >>> >>> Yeah, doesn't work. Last week they fixed my problem ticket which caused >>> the crashes, and due to the crashes stopped the replication I'll give a try >>> this week again after the update if the daemon doesn't crash, maybe it will >>> work, because if crash hasn't happened, the data was synced. Fingers >>> crossed ;) Don't give up 😄 >>> ------------------------------ >>> >>> *From:* 特木勒 &lt;twl007(a)gmail.com&gt; >>> *Sent:* Monday, March 22, 2021 1:38 PM >>> *To:* Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.com&gt; >>> *Cc:* ceph-users(a)ceph.io &lt;ceph-users(a)ceph.io&gt; >>> >>> >>> *Subject:* Re: [Suspicious newsletter] [ceph-users] RGW: Multiple Site >>> does not sync olds data >>> >>> >>> >>> Hi Istvan: >>> >>> >>> >>> Do you have any update on directional sync? >>> >>> >>> >>> I am trying to upgrade cluster to 15.2.10 to see if the problem is >>> solved. :( >>> >>> >>> >>> Thanks >>> >>> >>> >>> Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.com&gt; 于2021年3月1日周一 上午10:01写道： >>> >>> So-so. I had some interruption so it failed on one site, but the other >>> is kind of working. This is the first time when I saw data caught up in the >>> radosgw-admin data sync status on 1 side. >>> >>> Today will finish the other problematic site, I’ll let you know the >>> result is it working or not. >>> >>> >>> >>> Istvan Szabo >>> Senior Infrastructure Engineer >>> --------------------------------------------------- >>> Agoda Services Co., Ltd. >>> e: istvan.szabo(a)agoda.com >>> --------------------------------------------------- >>> >>> >>> >>> *From:* 特木勒 &lt;twl007(a)gmail.com&gt; >>> *Sent:* Sunday, February 28, 2021 1:34 PM >>> *To:* Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.com&gt; >>> *Cc:* ceph-users(a)ceph.io >>> *Subject:* Re: [Suspicious newsletter] [ceph-users] RGW: Multiple Site >>> does not sync olds data >>> >>> >>> >>> Email received from outside the company. If in doubt don't click links >>> nor open attachments! >>> ------------------------------ >>> >>> Hi Istvan: >>> >>> >>> >>> Thanks for your reply. >>> >>> >>> >>> Does directional sync solve the problem? I tried to run `radosgw-admin >>> sync init`, bit it still did not work. :( >>> >>> >>> >>> Thanks >>> >>> >>> >>> Szabo, Istvan (Agoda) &lt;Istvan.Szabo(a)agoda.com&gt; 于2021年2月26日周五 上午7:47写道： >>> >>> Same for me, 15.2.8 also. >>> I’m trying directional sync now, looks like symmetrical has issue. >>> >>> Istvan Szabo >>> Senior Infrastructure Engineer >>> --------------------------------------------------- >>> Agoda Services Co., Ltd. >>> e: istvan.szabo@agoda.com<mailto:istvan.szabo@agoda.com> >>> --------------------------------------------------- >>> >>> On 2021. Feb 26., at 1:03, 特木勒 &lt;twl007(a)gmail.com&gt; wrote: >>> >>> Email received from outside the company. If in doubt don't click links >>> nor open attachments! >>> ________________________________ >>> >>> Hi all: >>> >>> ceph version: 15.2.7 (88e41c6c49beb18add4fdb6b4326ca466d931db8) >>> >>> I have a strange question, I just create a multiple site for Ceph >>> cluster. >>> But I notice the old data of source cluster is not synced. Only new data >>> will be synced into second zone cluster. >>> >>> Is there anything I need to do to enable full sync for bucket or this >>> is a >>> bug? >>> >>> Thanks >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >>> >>> ________________________________ >>> This message is confidential and is for the sole use of the intended >>> recipient(s). It may also be privileged or otherwise protected by copyright >>> or other legal rules. If you have received it by mistake please let us know >>> by reply email and delete it from your system. It is prohibited to copy >>> this message or disclose its content to anyone. Any confidentiality or >>> privilege is not waived or lost by any mistaken delivery or unauthorized >>> disclosure of the message. All messages sent to and from Agoda may be >>> monitored to ensure compliance with company policies, to protect the >>> company's interests and to remove potential malware. Electronic messages >>> may be intercepted, amended, lost or deleted, or contain viruses. >>> >>> >>> >>> Internal >>> >>>