Could AssumeRoleWithOPA become a thing?
On Thu, Feb 20, 2020 at 12:17 Seena Fallah <seenafallah(a)gmail.com> wrote:
In a simple way I should say that OPA can't
get access to something that
radosgw isn't allowing. For example if I want to get access to user A for
my bucket in OPA policies it will reject because radosgw doesn't have this
policy and will reject.
OPA can just reject requests and it's BUG.
Can you please fix this bug with this PR? :)
On Thu, Feb 20, 2020 at 11:43 AM Seena Fallah <seenafallah(a)gmail.com>
wrote:
> I have make this PR to make OPA single source of truth so OPA can get
> access to not owned bucket for user. And also in OPA request we send
> bucket policy, acl, ... to OPA so we can support them when OPA integration
> is enabled. We will authorized user based on bucket policy, acl, ... and
> OPA policies in OPA server.
> Can you please take a look at here?
>
https://github.com/ceph/ceph/pull/32294
>
> On Tue, Feb 18, 2020 at 9:52 AM Seena Fallah <seenafallah(a)gmail.com>
> wrote:
>
>> I have seen another bad scenario that we have two source of truth. If
>> we get access to user for a bucket that he/she doesn't own it in OPA we can
>> perform this action because op->verify_permission() will return -EACCES and
>> so in rgw_process_authenticated function rgw_opa_authorize will not check!
>>
>> I think it's better to have one source of truth when we enabled OPA
>> integration so we can send bucket policy, acl, ... to OPA on each
>> request to be authorized.
>>
>> Do you have any other suggestion?
>>
>> On Tue, Feb 18, 2020 at 1:01 AM Seena Fallah <seenafallah(a)gmail.com>
>> wrote:
>>
>>> I have changed the PR for sending bucket policy, bucket acl, iam
>>> policy and user acl as a field to OPA request so OPA can decision based on
>>> this parameters and we can have a external authorization for our
>>> organization like AWS IAM. So we can have an S3 service that authorize
>>> based on bucket policy and organizations IAM.
>>>
https://github.com/ceph/ceph/pull/32294
>>> Could you please review this?
>>>
>>> On Mon, Feb 17, 2020 at 2:39 PM Seena Fallah <seenafallah(a)gmail.com>
>>> wrote:
>>>
>>>> Also we can add bucket policy result field to OPA request so in OPA
>>>> policies we can act based on bucket policy results.
>>>>
>>>> Are you agree with it?
>>>>
>>>> On Thu, Feb 13, 2020 at 12:17 PM Seena Fallah
<seenafallah(a)gmail.com>
>>>> wrote:
>>>>
>>>>> In bucket policy we have an Effect::Pass in validation result so if
>>>>> we just put OPA authorization in case of Effect::Pass I think it will
be
>>>>> close to what AWS do.
>>>>>
>>>>> On Fri, Feb 7, 2020 at 1:37 PM Seena Fallah
<seenafallah(a)gmail.com>
>>>>> wrote:
>>>>>
>>>>>> In organizations that has many services and want to have a
>>>>>> centralized authorization server this will be a good solution to
have.
>>>>>>
>>>>>> I mean that when we just authorize user ReadOnly in OPA but give
>>>>>> write access via bucket policy, the user can’t write because OPA
is
>>>>>> rejecting.
>>>>>>
>>>>>> I think we can just weight bucket policy upper that OPA so
bucket
>>>>>> policies that apply policies specific than OPA policies can
accept and
>>>>>> reject at first then OPA would authorize that request. I mean
bucket policy
>>>>>> specify policy more specific (On bucket or on object) than OPA
(OPA can set
>>>>>> policy globally too like giving ReadOnly to all buckets) so
it's better to
>>>>>> first check bucket policy then check for OPA. this could be easy
solve this
>>>>>> problem.
>>>>>>
>>>>>> On Thu, Feb 6, 2020 at 10:11 PM Casey Bodley
<cbodley(a)redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> On 2/6/20 11:58 AM, Seena Fallah wrote:
>>>>>>> > The main goal of using OPA like as AWS IAM is having an
external
>>>>>>> > authorization so we can have out own management on
policies from
>>>>>>> > external source of truth (OPA) too.
>>>>>>> >
>>>>>>> > I think it’s better to handle bucket policy with OPA as
well as
>>>>>>> AWS
>>>>>>> > does so we can have a better S3 service :)
>>>>>>>
>>>>>>> Can you expand on why that's better than the model I
suggested
>>>>>>> earlier
>>>>>>> in the thread, where a centralized policy service uses
radosgw's
>>>>>>> existing IAM APIs to manage policy instead of requiring
radosgw to
>>>>>>> call
>>>>>>> out to an external service for every request?
>>>>>>>
>>>>>>> I'd also like to clarify what you mean when you say
"handle bucket
>>>>>>> policy with OPA" - my understanding is that it's not
something
>>>>>>> that OPA
>>>>>>> itself does, but something very specific to your own
product's OPA
>>>>>>> policy script. Am I getting that right? If so, it sounds
like
>>>>>>> you're
>>>>>>> trying to re-engineer our OPA integration in a way that a) is
not
>>>>>>> useful
>>>>>>> to OPA users in general, and b) duplicates functionality
that
>>>>>>> radosgw
>>>>>>> already provides.
>>>>>>>
>>>>>>> For OPA users that just want the ability to write simple
scripts
>>>>>>> to
>>>>>>> customize authorization for their environment, I think our
current
>>>>>>> level
>>>>>>> of OPA integration is sufficient.
>>>>>>>
>>>>>>> >
>>>>>>> > On Thu, Feb 6, 2020 at 8:16 PM Casey Bodley
<cbodley(a)redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>> wrote:
>>>>>>> >
>>>>>>> > Is there an advantage to doing this in OPA over
radosgw?
>>>>>>> Have you
>>>>>>> > looked
>>>>>>> > at using our PutUserPolicy[1] APIs instead? We
support both
>>>>>>> user and
>>>>>>> > bucket policy, and (as far as I know) handle the
>>>>>>> intersection of
>>>>>>> > the two
>>>>>>> > as you'd expect.
>>>>>>> >
>>>>>>> >
>>>>>>>
https://docs.aws.amazon.com/IAM/latest/APIReference/API_PutUserPolicy.html
>>>>>>> >
>>>>>>> > On 2/5/20 9:55 AM, Seena Fallah wrote:
>>>>>>> > > I'm trying to implement AWS IAM with OPA so
I can have
>>>>>>> external
>>>>>>> > > authorization for my S3 service and also have
an active
>>>>>>> bucket
>>>>>>> > ACL and
>>>>>>> > > bucket policy.
>>>>>>> > >
>>>>>>> > > On Wed, Feb 5, 2020 at 6:11 PM Casey Bodley
<
>>>>>>> cbodley(a)redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>>>
>>>>>>> wrote:
>>>>>>> > >
>>>>>>> > > I'm confused by your references to AWS
IAM. Are you
>>>>>>> talking
>>>>>>> > about
>>>>>>> > > radosgw user policy? Or are you trying to
implement
>>>>>>> IAM policy
>>>>>>> > > inside of
>>>>>>> > > OPA?
>>>>>>> > >
>>>>>>> > > On 2/5/20 8:54 AM, Seena Fallah wrote:
>>>>>>> > > > Hi all.
>>>>>>> > > >
>>>>>>> > > > Any updates here? :)
>>>>>>> > > >
>>>>>>> > > > On Sat, Feb 1, 2020 at 10:37 AM Seena
Fallah
>>>>>>> > > <seenafallah(a)gmail.com
<mailto:seenafallah@gmail.com>
>>>>>>> > <mailto:seenafallah@gmail.com
<mailto:seenafallah@gmail.com
>>>>>>> >>
>>>>>>> > > > <mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>>>>
>>>>>>> > > wrote:
>>>>>>> > > >
>>>>>>> > > > I should also mention that if we
get access to
>>>>>>> bucket
>>>>>>> > via bucket
>>>>>>> > > > policy and reject it via AWS IAM,
the request
>>>>>>> will
>>>>>>> > reject so I
>>>>>>> > > > think we should make a new
behavior at what we
>>>>>>> should do
>>>>>>> > > with this
>>>>>>> > > > two source of truth?
>>>>>>> > > >
>>>>>>> > > > On Sat, Feb 1, 2020 at 10:33 AM
Seena Fallah
>>>>>>> > > > <seenafallah(a)gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>
>>>>>>> > <mailto:seenafallah@gmail.com
<mailto:seenafallah@gmail.com
>>>>>>> >>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>
>>>>>>> > <mailto:seenafallah@gmail.com
<mailto:seenafallah@gmail.com>>>>
>>>>>>> wrote:
>>>>>>> > > >
>>>>>>> > > > Yes but the main problem is
when the policy
>>>>>>> isn't
>>>>>>> > set in AWS
>>>>>>> > > > IAM for example a user has
>>>>>>> only AmazonS3ReadOnlyAccess
>>>>>>> > > and we
>>>>>>> > > > give PutObject policy via
bucket policy,
>>>>>>> user can
>>>>>>> > put object
>>>>>>> > > > to that bucket but in radosgw,
OPA will deny
>>>>>>> this
>>>>>>> > process
>>>>>>> > > > because there is only
ReadOnlyAccess to that
>>>>>>> > bucket for user
>>>>>>> > > > and radosgw will not check
bucket policy
>>>>>>> that gave
>>>>>>> > access to
>>>>>>> > > > user.
>>>>>>> > > > I think we should weight
bucket policy over
>>>>>>> OPA so
>>>>>>> > if bucket
>>>>>>> > > > policy accept that request it
doesn't need
>>>>>>> to be
>>>>>>> > checked
>>>>>>> > > with
>>>>>>> > > > OPA BUT if there is no policy
according to
>>>>>>> that
>>>>>>> > request it
>>>>>>> > > > should check by OPA because if
the policy
>>>>>>> > according to that
>>>>>>> > > > request isn't set bucket
policy will reject
>>>>>>> that
>>>>>>> > request
>>>>>>> > > so it
>>>>>>> > > > against failed!
>>>>>>> > > >
>>>>>>> > > > On Fri, Jan 31, 2020 at 1:30
AM Casey Bodley
>>>>>>> > > > <cbodley(a)redhat.com
<mailto:
>>>>>>> cbodley(a)redhat.com>
>>>>>>> > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>
>>>>>>> > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>>>>
>>>>>>> wrote:
>>>>>>> > > >
>>>>>>> > > >
>>>>>>> > > > On 1/30/20 2:18 PM, Seena
Fallah wrote:
>>>>>>> > > > > Hi Casey,
>>>>>>> > > > >
>>>>>>> > > > > The main problem now
is when OPA
>>>>>>> integration is
>>>>>>> > > enabled
>>>>>>> > > > bucket
>>>>>>> > > > > policies aren’t
work!
>>>>>>> > > > > I have checked AWS S3
that what is
>>>>>>> doing
>>>>>>> > when both
>>>>>>> > > > bucket policy and
>>>>>>> > > > > IAM policy (the
policy is set with AWS
>>>>>>> panel
>>>>>>> > in IAM
>>>>>>> > > > section) is set it
>>>>>>> > > > > will OR between two
of them so now in
>>>>>>> > radosgw S3 we
>>>>>>> > > > don’t have this
>>>>>>> > > > > feature and bucket
policies won’t work
>>>>>>> when OPA
>>>>>>> > > > integration is enabled.
>>>>>>> > > > >
>>>>>>> > > > > So I think it’s
better to active this
>>>>>>> > feature and
>>>>>>> > > > enabled bucket
>>>>>>> > > > > policy when OpA
integration is enabled.
>>>>>>> > > > >
>>>>>>> > > > > There is two
solutions here in this
>>>>>>> > discussion for
>>>>>>> > > > enabling bucket
>>>>>>> > > > > policy on OPA
integration:
>>>>>>> > > > > 1. Send bucket policy
on set/del
>>>>>>> actions to
>>>>>>> > OPA server
>>>>>>> > > > to be apply on
>>>>>>> > > > > OPA policy rules so
in this case the
>>>>>>> source
>>>>>>> > of truth
>>>>>>> > > > will be OPA (the
>>>>>>> > > > > state that we have
now in OPA
>>>>>>> integration)
>>>>>>> > and so
>>>>>>> > > these
>>>>>>> > > > policies that
>>>>>>> > > > > sent from bucket
policy will be
>>>>>>> applied.
>>>>>>> > > > > 2. OR between bucket
policy and OPA
>>>>>>> policy like
>>>>>>> > > AWS S3.
>>>>>>> > > > So there is
>>>>>>> > > > > two source of truth
in this case and
>>>>>>> if any of
>>>>>>> > > them deny
>>>>>>> > > > the request,
>>>>>>> > > > > the request will be
denied.
>>>>>>> > > >
>>>>>>> > > > What you described here in
2. is exactly
>>>>>>> how it
>>>>>>> > > currently
>>>>>>> > > > works.
>>>>>>> > > >
>>>>>>> > > > >
>>>>>>> > > > > Do have any other
solutions we have
>>>>>>> here and
>>>>>>> > which of
>>>>>>> > > > these solutions
>>>>>>> > > > > do you prefer to
have?
>>>>>>> > > > >
>>>>>>> > > > > On Thu, Jan 30, 2020
at 10:21 PM Casey
>>>>>>> Bodley
>>>>>>> > > > <cbodley(a)redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>
>>>>>>> > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>>>
>>>>>>> > > > >
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>
>>>>>>> > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>>>>
>>>>>>> > > > wrote:
>>>>>>> > > > >
>>>>>>> > > > > Hi Seena,
>>>>>>> > > > >
>>>>>>> > > > > I think it would
probably help if
>>>>>>> you could
>>>>>>> > > describe
>>>>>>> > > > your use case
>>>>>>> > > > > here,
>>>>>>> > > > > and what role you
want OPA to play
>>>>>>> in the
>>>>>>> > > > interpretation of these
>>>>>>> > > > > bucket
>>>>>>> > > > > policies. In
other words, what is
>>>>>>> it
>>>>>>> > that your OPA
>>>>>>> > > > policy is doing
>>>>>>> > > > > with
>>>>>>> > > > > these bucket
policy documents that
>>>>>>> shouldn't
>>>>>>> > > be done
>>>>>>> > > > within radosgw?
>>>>>>> > > > >
>>>>>>> > > > > On 1/30/20 1:09
AM, Seena Fallah
>>>>>>> wrote:
>>>>>>> > > > > > So Matt what
should we have done
>>>>>>> with
>>>>>>> > bucket
>>>>>>> > > > policy if we enable
>>>>>>> > > > > OPA
>>>>>>> > > > > >
integration?
>>>>>>> > > > > >
>>>>>>> > > > > > On Thu, Jan
30, 2020 at 1:45 AM
>>>>>>> Matt
>>>>>>> > Benjamin
>>>>>>> > > > >
<mbenjami(a)redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>>
>>>>>>> > > > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com
>>>>>>> >
<mailto:mbenjami@redhat.com>>>>>> wrote:
>>>>>>> > > > > >
>>>>>>> > > > > > I think
we should not be
>>>>>>> > introducing new
>>>>>>> > > > special case
>>>>>>> > > > > behavior, nor
>>>>>>> > > > > > sending
policy documents to
>>>>>>> OPA,
>>>>>>> > which from
>>>>>>> > > > what we have
>>>>>>> > > > > heard and
>>>>>>> > > > > > read,
intends to make no use
>>>>>>> of them.
>>>>>>> > > > > >
>>>>>>> > > > > > Matt
>>>>>>> > > > > >
>>>>>>> > > > > > On Wed,
Jan 29, 2020 at 4:45
>>>>>>> PM
>>>>>>> > Seena Fallah
>>>>>>> > > > > >
<seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>>
<mailto:
>>>>>>> seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>>
>>>>>>> > > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> >
<mailto:seenafallah@gmail.com>>>>>> wrote:
>>>>>>> > > > > > >
>>>>>>> > > > > > > I
think it’s better to OR
>>>>>>> > between two
>>>>>>> > > of the
>>>>>>> > > > bucket
>>>>>>> > > > > policies and
>>>>>>> > > > > > OPA
policies. So if one of
>>>>>>> them reject
>>>>>>> > > certain
>>>>>>> > > > access the
>>>>>>> > > > > request
>>>>>>> > > > > > will
reject as AWS do on its
>>>>>>> IAM
>>>>>>> > and bucket
>>>>>>> > > > policy.
>>>>>>> > > > > > > Are
you okay with this
>>>>>>> idea?
>>>>>>> > > > > > >
>>>>>>> > > > > > > On
Wed, Jan 29, 2020 at
>>>>>>> 11:13 PM
>>>>>>> > Casey
>>>>>>> > > Bodley
>>>>>>> > > > > >
<cbodley(a)redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>
>>>>>>> > > >
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>>
>>>>>>> > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>
>>>>>>> > > >
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>>>>
>>>>>>> > > > >
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>
>>>>>>> > > >
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>>
>>>>>>> > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>
>>>>>>> > > >
<mailto:cbodley@redhat.com
>>>>>>> > <mailto:cbodley@redhat.com>
>>>>>>> > > <mailto:cbodley@redhat.com
<mailto:cbodley@redhat.com
>>>>>>> >>>>>>
>>>>>>> > wrote:
>>>>>>> > > > > >
>>
>>>>>>> > > > > >
>>
>>>>>>> > > > > > >>
On 1/28/20 2:45 PM,
>>>>>>> Matthias Muench
>>>>>>> > > wrote:
>>>>>>> > > > > > >>
> Hi,
>>>>>>> > > > > > >>
> I think making Ceph
>>>>>>> special
>>>>>>> > to what the
>>>>>>> > > > rest of the clients
>>>>>>> > > > > > in the
>>>>>>> > > > > > >>
> world would expect
>>>>>>> would be a bit
>>>>>>> > > off the
>>>>>>> > > > idea of providing
>>>>>>> > > > > > S3 like
>>>>>>> > > > > > >>
> service.
>>>>>>> > > > > > >>
> To my understanding,
>>>>>>> setting
>>>>>>> > OPA to be
>>>>>>> > > > the source of
>>>>>>> > > > > truth would
>>>>>>> > > > > > >>
> introduce latency
>>>>>>> (based on
>>>>>>> > Casey’s
>>>>>>> > > > comments) and will not
>>>>>>> > > > > > allow
to
>>>>>>> > > > > > >>
> set policies (based on
>>>>>>> Seena).
>>>>>>> > > > > > >>
> The first one brings us
>>>>>>> > towards harder
>>>>>>> > > > latency and
>>>>>>> > > > > especially
>>>>>>> > > > > > >>
> depending on extern
>>>>>>> systems
>>>>>>> > resource
>>>>>>> > > > capability (assume
>>>>>>> > > > > central
>>>>>>> > > > > > >>
> resource as the idea is
>>>>>>> and
>>>>>>> > > therefor not
>>>>>>> > > > necessarily really
>>>>>>> > > > > > “in
reach”
>>>>>>> > > > > > >>
> within an acceptable
>>>>>>> latency,
>>>>>>> > > routing in
>>>>>>> > > > addition,
>>>>>>> > > > > etc.). The
>>>>>>> > > > > > second
>>>>>>> > > > > > >>
> one says simply that
>>>>>>> this would
>>>>>>> > > break any
>>>>>>> > > > existing
>>>>>>> > > > > >
compatibility with
>>>>>>> > > > > > >>
> clients and use cases.
>>>>>>> To me it
>>>>>>> > > looks not
>>>>>>> > > > that good to
>>>>>>> > > > > loose
>>>>>>> > > > > > on both
ends.
>>>>>>> > > > > >
>>
>>>>>>> > > > > > >>
Agreed. Even if one has to
>>>>>>> > opt-in to this
>>>>>>> > > > broken s3
>>>>>>> > > > > >
compatiblity, I'm
>>>>>>> > > > > > >>
skeptical that users will
>>>>>>> find this
>>>>>>> > > to be a
>>>>>>> > > > compelling target
>>>>>>> > > > > > for
their
>>>>>>> > > > > > >>
applications.
>>>>>>> > > > > >
>>
>>>>>>> > > > > > >>
The existing prototype of
>>>>>>> OPA
>>>>>>> > integration
>>>>>>> > > > sends this
>>>>>>> > > > > authorization
>>>>>>> > > > > > >>
request to OPA -in
>>>>>>> addition to-
>>>>>>> > radosgw's
>>>>>>> > > > own authorization
>>>>>>> > > > > > logic,
where
>>>>>>> > > > > > >>
we consult any of our
>>>>>>> user/bucket
>>>>>>> > > policies
>>>>>>> > > > or ACLs that
>>>>>>> > > > > apply.
>>>>>>> > > > > > In this
>>>>>>> > > > > > >>
model, OPA is not the only
>>>>>>> > source of
>>>>>>> > > truth.
>>>>>>> > > > It just has the
>>>>>>> > > > > >
opportunity
>>>>>>> > > > > > >>
to deny access that we
>>>>>>> would
>>>>>>> > otherwise
>>>>>>> > > > grant, so it doesn't
>>>>>>> > > > > > require
that
>>>>>>> > > > > > >>
we break compatibility
>>>>>>> with any S3
>>>>>>> > > features
>>>>>>> > > > that conflict
>>>>>>> > > > > with
>>>>>>> > > > > >
OPA's
>>>>>>> > > > > > >>
view of policy.
>>>>>>> > > > > >
>>
>>>>>>> > > > > > >>
Were we to change this so
>>>>>>> that OPA
>>>>>>> > > was the
>>>>>>> > > > only source of
>>>>>>> > > > > > truth,
then
>>>>>>> > > > > > >>
we'd be left with two bad
>>>>>>> > options: either
>>>>>>> > > > reject all requests
>>>>>>> > > > > > to
modify
>>>>>>> > > > > > >>
policy and break existing
>>>>>>> > > applications, or
>>>>>>> > > > send all
>>>>>>> > > > > policy/ACL
>>>>>>> > > > > > >>
information to OPA and
>>>>>>> require
>>>>>>> > every OPA
>>>>>>> > > > policy script to
>>>>>>> > > > > implement
>>>>>>> > > > > > >>
s3-compatible enforcement
>>>>>>> of
>>>>>>> > them. I also
>>>>>>> > > > don't see any
>>>>>>> > > > > benefit
>>>>>>> > > > > > to this
>>>>>>> > > > > > >>
model - why, if an client
>>>>>>> wants
>>>>>>> > to use s3
>>>>>>> > > > policy to
>>>>>>> > > > > restrict a
>>>>>>> > > > > > certain
>>>>>>> > > > > > >>
access, would OPA want to
>>>>>>> override
>>>>>>> > > that and
>>>>>>> > > > grant access
>>>>>>> > > > > instead?
>>>>>>> > > > > >
>>
>>>>>>> > > > > > >>
> I could live more with
>>>>>>> the
>>>>>>> > latency
>>>>>>> > > issue
>>>>>>> > > > but wouldn’t
>>>>>>> > > > > like it.
>>>>>>> > > > > > >>
> For the second, I can
>>>>>>> understand
>>>>>>> > > the idea
>>>>>>> > > > of having
>>>>>>> > > > > >
simplification for
>>>>>>> > > > > > >>
> auditing the access but
>>>>>>> I’m
>>>>>>> > not that
>>>>>>> > > > convinced to take the
>>>>>>> > > > > > burden
of
>>>>>>> > > > > > >>
> being “the special” one
>>>>>>> that
>>>>>>> > nobody
>>>>>>> > > wants
>>>>>>> > > > to work with.
>>>>>>> > > > > So, I
>>>>>>> > > > > > would
>>>>>>> > > > > > >>
> love to see the full
>>>>>>> fledged
>>>>>>> > support of
>>>>>>> > > > setting the
>>>>>>> > > > > policy by
>>>>>>> > > > > >
clients,
>>>>>>> > > > > > >>
> no matter what the
>>>>>>> result
>>>>>>> > would be in
>>>>>>> > > > terms of
>>>>>>> > > > > implementing it
to
>>>>>>> > > > > > >>
> interact with OPA.
>>>>>>> Instead,
>>>>>>> > having an
>>>>>>> > > > additional
>>>>>>> > > > > requirement to
>>>>>>> > > > > > >>
> implement additional
>>>>>>> handling
>>>>>>> > to set
>>>>>>> > > > policies different
>>>>>>> > > > > from
>>>>>>> > > > > > what S3
>>>>>>> > > > > > >>
> actually provides would
>>>>>>> > require special
>>>>>>> > > > clients first and
>>>>>>> > > > > > secondly
an
>>>>>>> > > > > > >>
> additional path to OPA
>>>>>>> with
>>>>>>> > all the
>>>>>>> > > > additional burden
>>>>>>> > > > > to tweak
>>>>>>> > > > > > >>
> security to allow this
>>>>>>> path
>>>>>>> > to OPA. I
>>>>>>> > > > feel that the first
>>>>>>> > > > > >
wouldn’t
>>>>>>> > > > > > >>
> happen (special
>>>>>>> clients) and
>>>>>>> > the second
>>>>>>> > > > in practice not
>>>>>>> > > > > > either
because
>>>>>>> > > > > > >>
> of security constraints
>>>>>>> by
>>>>>>> > the OPA
>>>>>>> > > admin
>>>>>>> > > > folks.
>>>>>>> > > > > > >>
>
>>>>>>> > > > > > >>
> G,
>>>>>>> > > > > > >>
> -matt
>>>>>>> > > > > > >>
>
>>>>>>> > > > > > >>
> ——————————————————
>>>>>>> > > > > > >>
> Matthias Muench
>>>>>>> > > > > > >>
> Senior Specialist
>>>>>>> Solution
>>>>>>> > Architect
>>>>>>> > > > > > >>
> EMEA Storage Specialist
>>>>>>> > > > > > >>
>
>>>>>>> matthias.muench(a)redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>>
>>>>>>> > > >
<mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>>>
>>>>>>> > > > >
<mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>>
>>>>>>> > > >
<mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> >
<mailto:matthias.muench@redhat.com>>>>
>>>>>>> > > > > >
<mailto:
>>>>>>> matthias.muench(a)redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>>
>>>>>>> > > >
<mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>>>
>>>>>>> > > > >
<mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>>
>>>>>>> > > >
<mailto:matthias.muench@redhat.com
>>>>>>> > <mailto:matthias.muench@redhat.com>
>>>>>>> > > <mailto:matthias.muench@redhat.com
>>>>>>> >
<mailto:matthias.muench@redhat.com>>>>>
>>>>>>> > > >
<mailto:mmuench@redhat.com
>>>>>>> > <mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>
>>>>>>> > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>>
>>>>>>> > > > >
<mailto:mmuench@redhat.com
>>>>>>> > <mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>
>>>>>>> > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>>>
>>>>>>> > > > > >
<mailto:mmuench@redhat.com
>>>>>>> > <mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>
>>>>>>> > > >
<mailto:mmuench@redhat.com
>>>>>>> > <mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>>
>>>>>>> > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>
>>>>>>> > > >
<mailto:mmuench@redhat.com
>>>>>>> > <mailto:mmuench@redhat.com>
>>>>>>> > > <mailto:mmuench@redhat.com
<mailto:mmuench@redhat.com
>>>>>>> >>>>>>
>>>>>>> > > > > > >>
> Phone: +49-160-92654111
>>>>>>> > > >
<tel:+49-160-92654111>
>>>>>>> > > > > > >>
>
>>>>>>> > > > > > >>
> Red Hat GmbH
>>>>>>> > > > > > >>
> Werner-von-Siemens-Ring
>>>>>>> 14
>>>>>>>
<https://www.google.com/maps/search/Werner-von-Siemens-Ring+14+%0D%0A+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+x?entry=gmail&source=g>
>>>>>>>
>>>>>>>
<https://www.google.com/maps/search/Werner-von-Siemens-Ring+14+%0D%0A+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+x?entry=gmail&source=g>
>>>>>>> >
>>>>>>>
<https://www.google.com/maps/search/Werner-von-Siemens-Ring+14+%0D%0A+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+x?entry=gmail&source=g>
>>>>>>> >
>>>>>>>
<https://www.google.com/maps/search/Werner-von-Siemens-Ring+14+%0D%0A+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+x?entry=gmail&source=g>>
>>>>>>>
>>>>>>>
<https://www.google.com/maps/search/Werner-von-Siemens-Ring+14+%0D%0A+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+x?entry=gmail&source=g>
>>>>>>> <x
>>>>>>>
<https://www.google.com/maps/search/Werner-von-Siemens-Ring+14+%0D%0A+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+%C2%A0+x?entry=gmail&source=g>
>>>>>>> -apple-data-detectors://2/1>
>>>>>>> > > > > > >>
> 85630 Grasbrunn
>>>>>>> > > > <x-apple-data-detectors://2/1>
>>>>>>> > > > > > >>
> Germany
>>>>>>> > <x-apple-data-detectors://2/1>
>>>>>>> > > > > > >>
>
>>>>>>> > > > > >
>>>>>>> > > > >
>>>>>>> > > >
>>>>>>> > >
>>>>>>> >
>>>>>>>
_______________________________________________________________________
>>>>>>> > > > > > >>
> Red Hat GmbH,
>>>>>>> >
http://www.de.redhat.com
>>>>>>> > > > >
<http://de.redhat.com/> ·
>>>>>>> > > > > > >>
> Registered seat:
>>>>>>> Grasbrunn,
>>>>>>> > Commercial
>>>>>>> > > > register:
>>>>>>> > > > > Amtsgericht
>>>>>>> > > > > >
Muenchen
>>>>>>> > > > > > >>
> HRB 153243 · Managing
>>>>>>> Directors:
>>>>>>> > > Charles
>>>>>>> > > > Cachera, Michael
>>>>>>> > > > > >
O'Neill, Tom
>>>>>>> > > > > > >>
> Savage, Eric Shander
>>>>>>> > > > > > >>
>
>>>>>>> > > > > > >>
>> On Jan 28, 2020, at
>>>>>>> 15:02, Seena
>>>>>>> > > Fallah
>>>>>>> > > > > >
<seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>>
<mailto:
>>>>>>> seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>>
>>>>>>> > > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> >
<mailto:seenafallah@gmail.com>>>>>> wrote:
>>>>>>> > > > > > >>
>>
>>>>>>> > > > > > >>
>>
>>>>>>> > > > > > >>
>> Amazon AWS S3 has two
>>>>>>> type of
>>>>>>> > > policies.
>>>>>>> > > > One from bucket
>>>>>>> > > > > > policy
and
>>>>>>> > > > > > >>
>> one form IAM. I think
>>>>>>> it
>>>>>>> > could be
>>>>>>> > > better
>>>>>>> > > > to have two
>>>>>>> > > > > > policies
models
>>>>>>> > > > > > >>
>> in Ceph one from bucket
>>>>>>> > policy and one
>>>>>>> > > > form OPA if its
>>>>>>> > > > > enable.
>>>>>>> > > > > > >>
>> So if you are okay we
>>>>>>> can change
>>>>>>> > > the PR
>>>>>>> > > > to make bucket
>>>>>>> > > > > > policy
enabled
>>>>>>> > > > > > >>
>> when OPA is enabled,
>>>>>>> too.
>>>>>>> > Because now
>>>>>>> > > > bucket policies not
>>>>>>> > > > > > working
>>>>>>> > > > > > >>
>> when OPA integration
>>>>>>> is enabled.
>>>>>>> > > > > > >>
>>
>>>>>>> > > > > > >>
>> On Tue, Jan 28, 2020
>>>>>>> at 2:57 AM
>>>>>>> > > Seena Fallah
>>>>>>> > > > > >
<seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>>
<mailto:
>>>>>>> seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>>
>>>>>>> > > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>>
<mailto:
>>>>>>> seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>>>
>>>>>>> > > > > > >>
>>
>>>>>>> > <mailto:seenafallah@gmail.com
<mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>>
>>>>>>> > > > > >
<mailto:
>>>>>>> seenafallah(a)gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>>
>>>>>>> > > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
<mailto:
>>>>>>> seenafallah(a)gmail.com>>
>>>>>>> > > >
<mailto:seenafallah@gmail.com
>>>>>>> > <mailto:seenafallah@gmail.com>
>>>>>>> > > <mailto:seenafallah@gmail.com
>>>>>>> >
<mailto:seenafallah@gmail.com>>>>>>> wrote:
>>>>>>> > > > > > >>
>>
>>>>>>> > > > > > >>
>> Matt When OPA
>>>>>>> integration is
>>>>>>> > > enabled
>>>>>>> > > > S3 policies
>>>>>>> > > > > doesn’t
>>>>>>> > > > > > work!
If
>>>>>>> > > > > > >>
>> you want them to be
>>>>>>> > worked we
>>>>>>> > > should
>>>>>>> > > > bypass S3
>>>>>>> > > > > policies
>>>>>>> > > > > > to OPA
>>>>>>> > > > > > >>
>> for being applied
>>>>>>> and
>>>>>>> > worked.
>>>>>>> > > > > > >>
>> Here we have
>>>>>>> conflict in OPA
>>>>>>> > > > integration with S3
>>>>>>> > > > > policies!
>>>>>>> > > > > > >>
>>
>>>>>>> > > > > > >>
>> On Tue, Jan 28,
>>>>>>> 2020 at 2:52
>>>>>>> > > AM Matt
>>>>>>> > > > Benjamin
>>>>>>> > > > > > >>
>> <
>>>>>>> mbenjami(a)redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>>
>>>>>>> > > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>>>
>>>>>>> > > > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>>
>>>>>>> > > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>>
>>>>>>> > <mailto:mbenjami@redhat.com
<mailto:mbenjami@redhat.com>
>>>>>>> > > <mailto:mbenjami@redhat.com <mailto:
>>>>>>> mbenjami(a)redhat.com>>
>>>>>>> > > >
<mailto:mbenjami@redhat.com
>>>>>>> > <mailto:mbenjami@redhat.com>
>>>>>>> >
>>>>>>
>>>>>> _______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io