Balancing request between rados gateway nodes
Hi everyone, We’re working on the Multi-site setup using Rados gateway with the Active-standby mode. The master zone processes all requests from the web application through the load balancer (3 Rados Gateway nodes behind). From our testing, we figured out that three Gateway nodes in backup zone makes a lot of requests into the load balancer after the period of time to check status of new objects and pulling them to the backup zone OSDs if needed. Our problem is that the workload on each Gateway nodes is not balance. Normally, one of the Gateway node is overloaded while others quite free (no incoming data, some of them even didn’t make any requests to load balancer for data checking). The load balancer from the master zone also under the high load since it’s receving data from web application and also sending data into backup zone….[cid:image001.png@01D64F0C.1252B3A0] The workload on 3 Gate ways [cid:image002.png@01D64F0C.1252B3A0] How to reduce the workload on master load balancer and balancing requests into gateway nodes in Backup zone? Many thanks! -- Nghia Viet Tran (Mr) mgm technology partners Vietnam Co. Ltd 7 Phan Châu Trinh Đà Nẵng, Vietnam +84 935905659 nghia.viet.tran@mgm-tp.com<mailto:nghia.viet.tran@mgm-tp.com> www.mgm-tp.com<https://www.mgm-tp.com/en/> Visit us on LinkedIn<https://www.linkedin.com/company/mgm-technology-partners-vietnam-co-ltd> and Facebook<https://www.facebook.com/mgmTechnologyPartnersVietnam>! Innovation Implemented. General Director: Frank Müller Registered office: 7 Pasteur, Hải Châu 1, Hải Châu, Đà Nẵng MST/Tax 0401703955
Tran; My experience with RGW multi-site suggests that the clusters don’t need to be exactly mirrored. Here’s what I would suggest: Move on of the gateway nodes from the backup zone to the master zone, but don’t add it to the load balancer Shift the backup zone from the load balancer to the single gateway Possibly; move one of the gateways out from behind the load balancer, and have it help support replication (second load balancer?) Thank you, Dominic L. Hilsbos, MBA Director – Information Technology [Perform Air International, Inc.] DHilsbos@PerformAir.com<mailto:DHilsbos@PerformAir.com> www.PerformAir.com<http://www.performair.com/> From: Nghia Viet Tran [mailto:Nghia.Viet.Tran@mgm-tp.com] Sent: Tuesday, June 30, 2020 4:27 AM To: Ceph users mailing list Subject: [ceph-users] Balancing request between rados gateway nodes Hi everyone, We’re working on the Multi-site setup using Rados gateway with the Active-standby mode. The master zone processes all requests from the web application through the load balancer (3 Rados Gateway nodes behind). From our testing, we figured out that three Gateway nodes in backup zone makes a lot of requests into the load balancer after the period of time to check status of new objects and pulling them to the backup zone OSDs if needed. Our problem is that the workload on each Gateway nodes is not balance. Normally, one of the Gateway node is overloaded while others quite free (no incoming data, some of them even didn’t make any requests to load balancer for data checking). The load balancer from the master zone also under the high load since it’s receving data from web application and also sending data into backup zone….[cid:image001.png@01D64EC2.0B905A80] The workload on 3 Gate ways [cid:image002.png@01D64EC2.0B905A80] How to reduce the workload on master load balancer and balancing requests into gateway nodes in Backup zone? Many thanks! -- Nghia Viet Tran (Mr) mgm technology partners Vietnam Co. Ltd 7 Phan Châu Trinh Đà Nẵng, Vietnam +84 935905659 nghia.viet.tran@mgm-tp.com<mailto:nghia.viet.tran@mgm-tp.com> www.mgm-tp.com<https://www.mgm-tp.com/en/> Visit us on LinkedIn<https://www.linkedin.com/company/mgm-technology-partners-vietnam-co-ltd> and Facebook<https://www.facebook.com/mgmTechnologyPartnersVietnam>! Innovation Implemented. General Director: Frank Müller Registered office: 7 Pasteur, Hải Châu 1, Hải Châu, Đà Nẵng MST/Tax 0401703955
participants (2)
-
DHilsbos@performair.com
-
Nghia Viet Tran