Amazon AWS S3 has two type of policies. One from bucket policy and one form
IAM. I think it could be better to have two policies models in Ceph one
from bucket policy and one form OPA if its enable.
So if you are okay we can change the PR to make bucket policy enabled when
OPA is enabled, too. Because now bucket policies not working when OPA
integration is enabled.
On Tue, Jan 28, 2020 at 2:57 AM Seena Fallah <seenafallah(a)gmail.com> wrote:
Matt When OPA integration is enabled S3 policies
doesn’t work! If you want
them to be worked we should bypass S3 policies to OPA for being applied and
worked.
Here we have conflict in OPA integration with S3 policies!
On Tue, Jan 28, 2020 at 2:52 AM Matt Benjamin <mbenjami(a)redhat.com> wrote:
> My take so far is that this is not a bug, and I'd like not to
> introduce special-case logic to override or suppress processing of
> native policy.
>
> Matt
>
> On Mon, Jan 27, 2020 at 5:24 PM Seena Fallah <seenafallah(a)gmail.com>
> wrote:
> >
> > I think it's very good that Ceph export its authorization and we could
> have external source of truth with it. S3 policies can transport to OPA and
> updates by users set/del policies.
> > But now we have conflict with OPA integration and S3 policies which is
> set when OPA integration is enabled, aren't work.
> >
> > Can you all please help to fix this bug?
> >
> > On Fri, Jan 24, 2020 at 1:05 PM Seena Fallah <seenafallah(a)gmail.com>
> wrote:
> >>
> >> Hi all.
> >>
> >> Any updates here?
> >>
> >> On Tue, Jan 21, 2020 at 2:50 AM Seena Fallah <seenafallah(a)gmail.com>
> wrote:
> >>>
> >>> OPA can be used in companies that uses many services like k8s,
> Ceph,... and want to have one central point for authorizing users so they
> can maintenance their access for each user on each service for example and
> etc. It’s just a use case and so it’s really good to have it. I think this
> is the biggest use case for having OPA in products that gets an option to
> centralize authorizing for all types of services.
> >>>
> >>> Performance for this model is issue like having keystone with Ceph.
> So I think it’s based on users that active this integrations at all.
> >>>
> >>> The model for writing policies to radosgw isn’t really good I think
> because of the reason above if this accrued there is always two copies of
> policies and it doesn’t sounds good for maintaining.
> >>>
> >>> If bucket policy disable, s3 clients like boto3 and etc will not work
> for setting polices but I think when someone is enabling OPA for
> authorizing it will also have an API for his/her OPA server to set/del
> policies and they can call these APIs to set/del policies.
> >>> And for extensions like PublicAccessBlock, it will disable because
> OPA is just authorizing requests and Ceph doesn’t authorize any request
> when OPA integration is enabled so OPA should handle any incoming policies
> were made by S3 policies. So it doesn’t make conflicts and if OPA
> integration is enabled it won’t work as we return 405 on each set/del
> policies requests and if OPA is disabled users can use this policies.
> >>>
> >>>
> >>> On Tue, Jan 21, 2020 at 2:05 AM Casey Bodley <cbodley(a)redhat.com>
> wrote:
> >>>>
> >>>> I am a big fan of the IAM policy documents, both because of the
> >>>> flexibility and expressiveness they provide, and because they're
in a
> >>>> format that all of our s3 clients understand.
> >>>>
> >>>> I'm not familiar enough with OPA to know what extra capabilities
it
> >>>> offers that IAM policy cannot, but I have serious concerns about
the
> >>>> performance and scalability of a model where radosgw has to send
> >>>> blocking RPCs to OPA in order to authorize each and every request.
> >>>>
> >>>> On the other hand, consider a model where a Policy Agent exercises
> its
> >>>> control over authorization by writing IAM documents to radosgw,
> which we
> >>>> use to cheaply authorize requests out of our metadata cache. I
would
> >>>> imagine that this model could cover a lot of interesting use cases,
> >>>> without breaking support for existing s3 applications that rely on
> >>>> bucket policy - as the proposal to reject PutBucketPolicy requests
> would.
> >>>>
> >>>> Is this something that OPA could feasibly do?
> >>>>
> >>>> For use cases that aren't supported by the existing policy
grammar,
> >>>> we're open to maintaining extensions to these documents. We
already
> >>>> implement a number of s3 extensions [1][2] that are easily
accessible
> >>>> via python/boto and the aws cli.
> >>>>
> >>>> But a model where radosgw outsources authorization entirely is a
hard
> >>>> sell, because it conflicts with feature development going forward.
> One
> >>>> example would be support for PublicAccessBlock [3], where radosgw
> needs
> >>>> full visibility into policy to detect cases where public access
> would be
> >>>> granted.
> >>>>
> >>>> [1]
> >>>>
>
https://docs.ceph.com/docs/master/radosgw/s3/python/#using-s3-api-extensions
> >>>>
> >>>> [2]
> >>>>
>
https://github.com/ceph/ceph/blob/master/examples/boto3/service-2.sdk-extra…
> >>>>
> >>>> [3]
https://github.com/ceph/ceph/pull/30033
> >>>>
> >>>> On 1/20/20 12:21 PM, Seena Fallah wrote:
> >>>> > I’m also agree with you Matt that it will free us from
complexity
> of
> >>>> > handling S3 policy or Swift ACL if we save the current state
of
> OPA.
> >>>> > But if we want use this state of OPA we should act for S3
policy
> and
> >>>> > Swift ACL that if user is setting them it shouldn’t be allowed
and
> >>>> > return user that you can’t set them! Because now when OPA
> integration
> >>>> > is enabled and user set bucket policy it returns success but
> actually
> >>>> > it doesn’t work!
> >>>> >
> >>>> > What are your thoughts?
> >>>> >
> >>>> > On Sun, Jan 19, 2020 at 12:33 AM Seena Fallah <
> seenafallah(a)gmail.com
> >>>> > <mailto:seenafallah@gmail.com>> wrote:
> >>>> >
> >>>> > I think the other problem caused when OPA integration is
> enabled
> >>>> > and we set bucket policy is when user wants to get his/her
> bucket
> >>>> > policy. Some policies are set through OPA (for example in
OPA
> >>>> > rules we have user A that has access to user B bucket so
OPA
> >>>> > return true on authorizing request and it acts like bucket
> policy)
> >>>> > and some through bucket policy (s3 clients command). So
when
> user
> >>>> > is getting his/her bucket policy what data should we
return?
> The
> >>>> > policies that are set through bucket policy or OPA rules
for
> that
> >>>> > bucket?
> >>>> >
> >>>> > I fact I think OPA rules are not static and will change in
time
> >>>> > and so there should be a client interface for that OPA
server
> that
> >>>> > users could change their rules for their buckets (giving
> access to
> >>>> > put, get, ... to someone else and etc.). So if the client
> exists
> >>>> > there is no need to bucket policy and we can make it
disable
> (by
> >>>> > returning 405) when OPA integration is enabled (I repeat
that
> >>>> > still now in Ceph latest version when OPA integration is
> enabled
> >>>> > bucket policies aren’t work!) because the policies that are
set
> >>>> > with bucket policy can be check with OPA, too.
> >>>> >
> >>>> > What’s your opinion?
> >>>> >
> >>>> > On Fri, Jan 17, 2020 at 9:40 PM Seena Fallah
> >>>> > <seenafallah(a)gmail.com
<mailto:seenafallah@gmail.com>> wrote:
> >>>> >
> >>>> > I think when OPA integration is enabled the source of
truth
> >>>> > for authorizing should be OPA (it is right now in Ceph
and
> all
> >>>> > requests are authorizing with OPA and Ceph doesn’t
> authorize
> >>>> > any request by it self).
> >>>> > When user is using bucket policy feature he/she wants
to
> get
> >>>> > access to someone else so when he/she is the bucket
owner,
> >>>> > he/she can perform this action and we should apply
this
> policy
> >>>> > for him/her. If we want policies just update within
OPA
> >>>> > server/client and S3 clients (s3cmd, aws, ...) don’t
edit
> >>>> > policies, we should reply to them that set/delpolicy
isn’t
> >>>> > allowed from here (return 405 for example; just for
saying
> >>>> > that the request that user send isn’t successful).
> >>>> >
> >>>> > Yes we can have some process and simplification before
> sending
> >>>> > it to OPA but the s3 policy has a general structure so
OPA
> >>>> > server can decode it by it self.
> >>>> >
> >>>> > On Fri, Jan 17, 2020 at 9:16 PM Matt Benjamin
> >>>> > <mbenjami(a)redhat.com
<mailto:mbenjami@redhat.com>> wrote:
> >>>> >
> >>>> > The larger question, I think, is what OPA is
supposed
> to
> >>>> > do with it.
> >>>> > The larger question I think it asks is whether OPA
or
> Ceph
> >>>> > owns a
> >>>> > particular dimension of policy--or, perhaps, which
owns
> >>>> > policy for
> >>>> > what portions of the namespace (at any particular
> point in
> >>>> > time).
> >>>> >
> >>>> > Without any new interaction, when OPA is
configured,
> OPA
> >>>> > can make a
> >>>> > direct authorization decision with all available
> >>>> > information for
> >>>> > Ceph/RGW, notwithstanding any S3 or Swift ACL or
S3
> policy
> >>>> > that might
> >>>> > exist--including any that might have been stored
prior
> to
> >>>> > turning on
> >>>> > this proposed feature to push policy documents to
> OPA. This
> >>>> > overriding property of the OPA integration when in
use
> >>>> > frees us from a
> >>>> > lot of complexity regarding which system is the
source
> of
> >>>> > truth, and
> >>>> > for what.
> >>>> >
> >>>> > I can see value in more sophisticated integration
that
> >>>> > mutually
> >>>> > comprehends policy--but I'm having trouble with
"send
> >>>> > policy documents
> >>>> > to OPA, maybe it will do something with
them."
> >>>> >
> >>>> > Matt
> >>>> >
> >>>> > On Fri, Jan 17, 2020 at 12:01 PM Seena Fallah
> >>>> > <seenafallah(a)gmail.com
<mailto:seenafallah@gmail.com>>
> wrote:
> >>>> > >
> >>>> > > Hello Ash
> >>>> > >
> >>>> > > With bucket policy user A can get access to
user B
> for
> >>>> > putting object on bucket C. So if this policy sent
to
> Ceph
> >>>> > and OPA integration is enabled it will be discard
> because
> >>>> > this policy isn’t sent to OPA server to be
updated.
> >>>> > > Here is a documentation for bucket policy:
> >>>> > >
>
https://docs.ceph.com/docs/master/radosgw/bucketpolicy/
> >>>> > >
> >>>> > > With this PR when user set bucket policy, the
data of
> >>>> > that policy will sent to OPA server to be applied
and
> so
> >>>> > OPA can get access to user that gets access to
bucket
> via
> >>>> > bucket policy.
> >>>> > >
> >>>> > > On Fri, Jan 17, 2020 at 8:24 PM Ash Narkar
> >>>> > <ash(a)styra.com <mailto:ash@styra.com>>
wrote:
> >>>> > >>
> >>>> > >> Hello Seena,
> >>>> > >>
> >>>> > >> The OPA integration is with the RGW and
the intent
> is
> >>>> > to check if an authenticated user is allowed to
> perform a
> >>>> > particular action on a particular resource. For
> example,
> >>>> > can Bob delete a bucket based on some attribute
like
> his
> >>>> > location. I am not familiar with the internals of
> Ceph's
> >>>> > bucket policy command. It would be great to get
some
> >>>> > context here and discuss if the bucket policy can
be
> >>>> > authorized with OPA which is the intent of your PR
I
> believe.
> >>>> > >>
> >>>> > >> Thanks
> >>>> > >> Ash
> >>>> > >>
> >>>> > >> On Fri, Jan 17, 2020 at 6:33 AM Seena
Fallah
> >>>> > <seenafallah(a)gmail.com
<mailto:seenafallah@gmail.com>>
> wrote:
> >>>> > >>>
> >>>> > >>> So when OPA integration is enabled the
bucket
> policy
> >>>> > from users will not work!
> >>>> > >>> I think it’s about Ceph architecture
not OPA
> because
> >>>> > OPA is for authorizing the requests and bucket
policy
> is
> >>>> > one of the authorizing methods that OPA should
support.
> >>>> > >>>
> >>>> > >>> On Fri, Jan 17, 2020 at 5:56 PM Matt
Benjamin
> >>>> > <mbenjami(a)redhat.com
<mailto:mbenjami@redhat.com>>
> wrote:
> >>>> > >>>>
> >>>> > >>>> Hi Seena,
> >>>> > >>>>
> >>>> > >>>> As I wrote in a comment on your
PR, my current
> >>>> > intuition is that what
> >>>> > >>>> you're doing here isn't
consistent with the
> original
> >>>> > intent of the OPA
> >>>> > >>>> integration we currently have, nor
with the OPA
> model
> >>>> > in general.
> >>>> > >>>>
> >>>> > >>>> That said, I'd really like
some feedback from OPA
> >>>> > architects, CC'd.
> >>>> > >>>>
> >>>> > >>>> regards,
> >>>> > >>>>
> >>>> > >>>> Matt
> >>>> > >>>>
> >>>> > >>>> On Thu, Jan 16, 2020 at 5:04 AM
Seena Fallah
> >>>> > <seenafallah(a)gmail.com
<mailto:seenafallah@gmail.com>>
> wrote:
> >>>> > >>>> >
> >>>> > >>>> > Hi all. In OPA integration
from Ceph there is no
> >>>> > integration for bucket policy.
> >>>> > >>>> > When user is setting bucket
policy to his/her
> >>>> > bucket the OPA server won't get who get's
access to
> that
> >>>> > bucket so after that if the request is coming from
the
> >>>> > user (that gets access to that bucket via bucket
> policy)
> >>>> > to access that bucket (PUT, GET,...), OPA will
reject
> that
> >>>> > because of no data in database.
> >>>> > >>>> > I have create a pull request
for this problem
> so if
> >>>> > user creates a bucket policy for his/her bucket,
the
> >>>> > policy data will send to OPA server to be update on
the
> >>>> > database.
> >>>> > >>>> > I think the main idea of
having OPA is to have
> all
> >>>> > authorization in OPA and Ceph don't authorize
any
> request
> >>>> > by it self.
> >>>> > >>>> > Here is the pull request and
I would be
> thankful to
> >>>> > hear about your comments.
> >>>> > >>>> >
https://github.com/ceph/ceph/pull/32294
> >>>> > >>>> > Thanks.
> >>>> > >>>> >
_______________________________________________
> >>>> > >>>> > Dev mailing list --
dev(a)ceph.io <mailto:
> dev(a)ceph.io>
> >>>> > >>>> > To unsubscribe send an email
to
> dev-leave(a)ceph.io
> >>>> > <mailto:dev-leave@ceph.io>
> >>>> > >>>>
> >>>> > >>>>
> >>>> > >>>>
> >>>> > >>>> --
> >>>> > >>>>
> >>>> > >>>> Matt Benjamin
> >>>> > >>>> Red Hat, Inc.
> >>>> > >>>> 315 West Huron Street, Suite 140A
> >>>> > >>>> Ann Arbor, Michigan 48103
> >>>> > >>>>
> >>>> > >>>>
http://www.redhat.com/en/technologies/storage
> >>>> > >>>>
> >>>> > >>>> tel. 734-821-5101
> >>>> > >>>> fax. 734-769-8938
> >>>> > >>>> cel. 734-216-5309
> >>>> > >>>>
> >>>> >
> >>>> >
> >>>> > --
> >>>> >
> >>>> > Matt Benjamin
> >>>> > Red Hat, Inc.
> >>>> > 315 West Huron Street, Suite 140A
> >>>> > Ann Arbor, Michigan 48103
> >>>> >
> >>>> >
http://www.redhat.com/en/technologies/storage
> >>>> >
> >>>> > tel. 734-821-5101
> >>>> > fax. 734-769-8938
> >>>> > cel. 734-216-5309
> >>>> >
> >>>> >
> >>>> > _______________________________________________
> >>>> > Dev mailing list -- dev(a)ceph.io
> >>>> > To unsubscribe send an email to dev-leave(a)ceph.io
> >>>> _______________________________________________
> >>>> Dev mailing list -- dev(a)ceph.io
> >>>> To unsubscribe send an email to dev-leave(a)ceph.io
>
>
>
> --
>
> Matt Benjamin
> Red Hat, Inc.
> 315 West Huron Street, Suite 140A
> Ann Arbor, Michigan 48103
>
>
http://www.redhat.com/en/technologies/storage
>
> tel. 734-821-5101
> fax. 734-769-8938
> cel. 734-216-5309
>
>