On Mon, May 26, 2025 at 7:38 AM Michel Jouvin <michel.jouvin@ijclab.in2p3.fr> wrote:
Hi,
We have a Ceph cluster hosting ~10 RGW realms and in our experience, radosgw-admin is making the Ceph admins life very hard... The opposite experience to other Ceph management tools.
In addition to the lack of verifications before applying modifications that leads to the issues describe in another thread were you can have a configuration that cannot be commited and at the same time cannot be fixed, I have the feeling that one of the major issue is that radosgw-admin predates the rgw/zonegroup/zone paradigm and manages these 3 different kinds of objects as completely independent from the others. It is made worst by the fact that radosgw-admin defines a default object in each category, used implicitely (and slilently) if you don't select one specific object. The consequence of these 2 problems is that if you modify a zone, you can endup modifying the zonegroup of another realm than the one the zone belongs to (as there is no concept of a zone belonging to a zonegroup in radosgw-admin). It happens for example if you modify a zone (with zone modify) to make it the master zone (of a zonegroup in fact) and you don't specify the zonegroup the zone belongs to/is associated with: it will in fact update the default zonegroup that can be in another realm or at least not be the zonegroup the zone is associated with. A less impactful but nevertheless unexpected result is if you do something like: 'zonegroup list --rgw-realm my_realm'. You would expect to see only the zonegroups associated with the realm you specified where you'll get the list of all the zonegroups defined.
I can imagine that chaning this is difficult without breaking backward compatibility. IMO, an improvement would be the ability not to have a default realm, zonegroup, zone. At least you would have an error if you don't specify explicitely the object to modify.
thanks Michel, this 'default' stuff was meant to be a convenience for clusters with a single zone each, but has indeed turned out to be a nightmare. prior to squid, it wasn't even possible to leave the 'default realm' unset. quoting the release note:
The radosgw-admin commands 'realm create' and 'realm pull' no longer set the default realm without --default.
there's a 'radosgw-admin realm default rm' command that can clear that default realm, if set. i don't know of a way to unset the default zone or zonegroup, though you at least had to opt in to those with --default
Am I the only one struggling with radosgw-admin? Is there any plan to improve this?
as you say, it's hard to make significant changes here without breaking existing deployments/automation. so there aren't any concrete plans, but we're open to ideas in the meantime, would you suggest any improvements to the documentation? for example, the note about 'default' behavior under https://docs.ceph.com/en/latest/radosgw/multisite/#create-a-realm
Best regards,
Michel _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io