Hi Casey, Thanks for you answer. Good news to know that in Squid it is possible to unset the default realm definition. In fact I was running Reef when I (or my colleagues) had all the last bad experiences but we upgraded to Squid yesterday! Nonetheless, I just add a look at `radosgw-admin --help` and the command `realm default rm` doesn't seems to be documented... Improvements to the documentation are probably the easiest and here are a few suggestions: - Warn that, despite there is normally a hierarchical relationship between realm, zonegroup and zone, radosgw-admin doesn't implement this hierarchy with the result that for a zone or zonegroup specifying a realm will not have any effect to restrict/select one (and the same with zonegroup and zone). - Mention that defining a default realm/zonegroup/zone is discouraged if you have several of thems for the reason above. This can lead to updating an object different from what you had in mind. - Remove the --default from suggested commands. When you are new you tend to add it, it you miss the note or didn't fully understand all the things it implies. As for small improvements to radosgw-admin that may avoid running into troubles, I could suggest: - Have a "default rm" command for zonegroup and zone too and document it in the help for all the objects that have it - Rewrite the help (may be not a small change in fact) so that you could have a hierarchical help giving more details than the summary on a given command (for example `realm zone modify --help` would tell you that if you use --master, this will modify the zonegroup the zone belongs to and that you have to mention it or the default zone will be used) - Do a minimum of validation when modifications are done to prevent have a configuration/period that cannot be commited anymore Really RGW are a very nice service in Ceph, easy to deploy in various scenarios when you use cephadm, but the radosgw-admin part of the configuration is very risky. This is not the case in other parts of Ceph and I think something should be done to put this part at the same level as other Ceph management commands/tools (I don't have in mind any where you can just by a small mistake break your configuration as there is enough validation done to prevent it and the famous --yes-i-really-want-it for the things that can make sense but are potentially dangerous). An issue has been open by Frederic after the discussion in another thread (https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/6SD24SR7JBHF...) around this idea: https://tracker.ceph.com/issues/71406. Best regards, Michel Le 27/05/2025 à 17:53, Casey Bodley a écrit :
Hi,
We have a Ceph cluster hosting ~10 RGW realms and in our experience, radosgw-admin is making the Ceph admins life very hard... The opposite experience to other Ceph management tools.
In addition to the lack of verifications before applying modifications that leads to the issues describe in another thread were you can have a configuration that cannot be commited and at the same time cannot be fixed, I have the feeling that one of the major issue is that radosgw-admin predates the rgw/zonegroup/zone paradigm and manages these 3 different kinds of objects as completely independent from the others. It is made worst by the fact that radosgw-admin defines a default object in each category, used implicitely (and slilently) if you don't select one specific object. The consequence of these 2 problems is that if you modify a zone, you can endup modifying the zonegroup of another realm than the one the zone belongs to (as there is no concept of a zone belonging to a zonegroup in radosgw-admin). It happens for example if you modify a zone (with zone modify) to make it the master zone (of a zonegroup in fact) and you don't specify the zonegroup the zone belongs to/is associated with: it will in fact update the default zonegroup that can be in another realm or at least not be the zonegroup the zone is associated with. A less impactful but nevertheless unexpected result is if you do something like: 'zonegroup list --rgw-realm my_realm'. You would expect to see only the zonegroups associated with the realm you specified where you'll get the list of all the zonegroups defined.
I can imagine that chaning this is difficult without breaking backward compatibility. IMO, an improvement would be the ability not to have a default realm, zonegroup, zone. At least you would have an error if you don't specify explicitely the object to modify.
On Mon, May 26, 2025 at 7:38 AM Michel Jouvin <michel.jouvin@ijclab.in2p3.fr> wrote: thanks Michel,
The radosgw-admin commands 'realm create' and 'realm pull' no longer set the default realm without --default.
this 'default' stuff was meant to be a convenience for clusters with a single zone each, but has indeed turned out to be a nightmare. prior to squid, it wasn't even possible to leave the 'default realm' unset. quoting the release note: there's a 'radosgw-admin realm default rm' command that can clear that default realm, if set. i don't know of a way to unset the default zone or zonegroup, though you at least had to opt in to those with --default
Am I the only one struggling with radosgw-admin? Is there any plan to improve this? as you say, it's hard to make significant changes here without breaking existing deployments/automation. so there aren't any concrete plans, but we're open to ideas
in the meantime, would you suggest any improvements to the documentation? for example, the note about 'default' behavior under https://docs.ceph.com/en/latest/radosgw/multisite/#create-a-realm
Best regards,
Michel _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io