[Query] Safe to discard bucket lock objects in reshard pool?
Dear Ceph users, We have a slightly dated version of Luminous cluster in which dynamic bucket resharding was accidentally enabled due to a misconfig (we don't use this feature since the number of objects per bucket is capped). This resulted in creation of the RGW reshard pool with lots of bucket reshard lock objects (we have thousands of buckets) which is leading to clutter. Also, we've run into a malloc failure issue (similar to https://tracker.ceph.com/issues/21826 but not the same since we already use tcmalloc) on the OSDs in which these reshard lock objects are located and we'd like to reduce the objects that have to be copied out. My question to the community is: "Is it safe to discard the bucket reshard lock objects if we know that we'll never use the reshard feature on the cluster again?". The RGWs performed resharding several months ago due to a misconfiguration and we already have stale bucket instances which are due for cleanup on this cluster. Thanks, Prasad Krishnan -- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_
On Tue, Jan 19, 2021 at 10:57 AM Prasad Krishnan <prasad.krishnan@flipkart.com> wrote:
Dear Ceph users,
We have a slightly dated version of Luminous cluster in which dynamic bucket resharding was accidentally enabled due to a misconfig (we don't use this feature since the number of objects per bucket is capped).
This resulted in creation of the RGW reshard pool with lots of bucket reshard lock objects (we have thousands of buckets) which is leading to clutter. Also, we've run into a malloc failure issue (similar to https://tracker.ceph.com/issues/21826 but not the same since we already use tcmalloc) on the OSDs in which these reshard lock objects are located and we'd like to reduce the objects that have to be copied out.
My question to the community is: "Is it safe to discard the bucket reshard lock objects if we know that we'll never use the reshard feature on the cluster again?".
yes, they can be safely deleted. those locks just prevent multiple radosgws from trying to reshard at the same time. any such locks would have expired long ago, so removing them should have no observable effect
The RGWs performed resharding several months ago due to a misconfiguration and we already have stale bucket instances which are due for cleanup on this cluster.
Thanks, Prasad Krishnan
--
*-----------------------------------------------------------------------------------------*
*This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.*****
****
*Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.*****
****
*Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.*
_-----------------------------------------------------------------------------------------_ _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Just update your luminous to 12.2.13, user radosgw admin reshard stale instances rm command, leave dynamic reshard feature enabled. In this release reshrding bugs was fixed. Cheers, k Sent from my iPhone
On 19 Jan 2021, at 18:58, Prasad Krishnan <prasad.krishnan@flipkart.com> wrote:
We have a slightly dated version of Luminous cluster in which dynamic bucket resharding was accidentally enabled due to a misconfig (we don't use this feature since the number of objects per bucket is capped).
participants (3)
-
Casey Bodley
-
Konstantin Shalygin
-
Prasad Krishnan