For userspace clients I think we can implement a normal config option (rados_localize_reads) and set it to true in the mon config store, and then clients will pick it up. But that would be global to all ops — for it to be pool-based, we’d have to extend the OSDMap with client information and make some more extensive Objecter changes (and kclient ones). That’s not impossible but feels a bit weird. :/ -Greg On Mon, Jul 3, 2023 at 11:36 AM Matt Benjamin <mbenjami@redhat.com> wrote:
is there a global/cluster-wide version of such a flag, or to automate applying it? it seems like the scenarios I hear about are stretched clusters, and that it would be attractive to not have to manage that by exception?
Matt
On Mon, Jul 3, 2023 at 12:25 PM Casey Bodley <cbodley@redhat.com> wrote:
On Tue, Jun 27, 2023 at 7:12 PM Sam Just <sjust@redhat.com> wrote:
It would probably make sense to add support in the objecter for picking up certain flags (such as LOCALIZE_READS) from the pool config. -Sam
thanks Sam. Jiffin, would you be willing to look into this Objecter part? i think it would make your project easier if the feature was transparent to rgw
On Sun, Jun 18, 2023 at 1:34 AM Josh Salomon <jsalomon@redhat.com>
One more point - from a business perspective, this flag is per pool
and not global so the API should be per pool and not global.
Regards,
Josh
On Sat, Jun 17, 2023 at 4:22 PM Ilya Dryomov <idryomov@gmail.com> wrote:
On Fri, Jun 16, 2023 at 5:13 PM Casey Bodley <cbodley@redhat.com>
wrote:
On Fri, Jun 16, 2023 at 3:21 AM Jiffin Thottan <
jthottan@redhat.com> wrote:
> > Hi, > > I am planning to support read localise feature for RGW servers similar to what RBD volumes support. From code reading, it looks to pass "librados::OPERATION_LOCALIZE_READS" before sending the request to RADOS. I have created a tracker issue for this feature https://tracker.ceph.com/issues/61701, this will be per server config
> > Thanks and regards, > Jiffin
thanks Jiffin,
that sounds simple enough, though rgw issues librados reads in many different places that would need to manage this flag
i'm curious why this is a per-op flag, rather than a global
wrote: option. I don't know about any other technical hurdles in implementing this feature. Please share your thoughts on the same. librados
setting. is there a reason that rgw would only want *some* of its reads localized? are there any implications for read-after-write consistency that we need to worry about here?
Hi Casey,
I'm not aware of any read-after-write or other consistency implications. Read-from-replica is safe for general use since Octopus (OSD-side issues were fixed in [1]).
Ultimately all flags are per-op. The reason that there is no global librados setting for BALANCE_READS and LOCALIZE_READS seems to be a combination of a) the fact that these flags could only be used in special circumstances in the past and b) the fact that librados doesn't have a generic API for applying a given flag globally like it does for applying a given snapshot ID/context to all reads/writes.
[1] https://github.com/ceph/ceph/pull/32381
Thanks,
Ilya _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
--
Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A <https://www.google.com/maps/search/315+West+Huron+Street,+Suite+140A+Ann+Arbor,+Michigan+48103?entry=gmail&source=g> Ann Arbor, Michigan 48103 <https://www.google.com/maps/search/315+West+Huron+Street,+Suite+140A+Ann+Arbor,+Michigan+48103?entry=gmail&source=g>
http://www.redhat.com/en/technologies/storage
tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309 _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io