On Fri, Nov 17, 2023 at 11:09:22AM +0100, Boris Behrens wrote:
Hi, I am looking for some experience on how people make their RGW public. What level of fine-grained control do you have over DNS for your environment?
If you can use a very short TTL, and dynamically update DNS rapidly, maybe a DNS-based routing solution would be the quickest win for you? s3.example.com => A/AAAA record that resolves to only the pod(s) that are online AND least loaded with traffic. 10 second TTL on the DNS entry. Right now those pods might be direct RGW, or L7LB+RGW (HAProxy, Envoy). In future, you might iterate the design to be L4LB ingress on those pods, and have the L7LB+RGW pods doing direct server return. If a pod goes offline: 0-TTL seconds: some clients might have to retry on a different IP. TTL+ seconds: failed pod is no longer in the DNS records. A good piece of overall reading is vbernat's load-balancing with Linux page: https://vincent.bernat.ch/en/blog/2018-multi-tier-loadbalancer It doesn't have the above dynamic DNS solution directly in front of pods, because it mostly focuses on what can be done with BGP as a common point. It does however suggest DNS for regional failover. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer E-Mail : robbat2@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136