On Thu, Aug 31, 2023 at 1:37 PM Ilya Dryomov <idryomov(a)gmail.com> wrote:
On Thu, Aug 31, 2023 at 9:26 PM Prashant Dhange <pdhange(a)redhat.com> wrote:
Hi Ilya,
We discussed this topic in yesterday's RADOS meeting. Overall sentiments are not to
revert the PR#47560 till we have a viable solution from the RGW and orchestrator side.
Similar problems can be seen with the application built on top of LIBRADOS APIs and fail
to enable application for the pool. The end users may find it difficult to debug the issue
of why pool is not writable.
We believe the solution may lie outside RADOS but the end solution should be less
intrusive and should be backward-compatible. RGW was silently failing to create buckets.
We had to debug the issue through RGW debug logs which was time consuming and not at all
user friendly. Reference :
https://bugzilla.redhat.com/show_bug.cgi?id=2028999. One of
ceph's users had a major production outage for more than 24 hours because the RGW was
failing to create buckets after cluster upgrade due to enforcement of tag in OSD caps.
Alternatively we can mute the POOL_APP_NOT_ENABLED warning in case HEALTH_WARN is a bit
annoying for newly created pools.
Hi Prashant,
Thanks for providing the context.
I can't say I agree with the approach. There are many other ways to
screw up OSD caps (especially if one tries to lock down as tight as
possible) none of which would be similarly highlighted in "ceph
status", so this doesn't address the general lack of user-friendliness
in this area.
Let us know if you would like to join the next RADOS meeting (or separate meeting) to
discuss the feasible solution with the RGW and cephadm team. I will invite all the
stakeholders for the meeting.
That said, I don't really have a stake in the ground here. Given that
creating pools is a rare operation, perhaps a bogus health alert that
shows up only briefly is acceptable.
It is an alert that would only show up when a cluster admin is already
sitting at the keyboard, and we've gone from "pool tags are a
new-fangled thing" to "most of the project won't work right if you
don't have tags set correctly".
I think this is probably okay, although something more elegant would be nice.