17 Feb
2021
17 Feb
'21
6:52 p.m.
I am bumping this email to hopefully get some more eyes on it.
We are continuing to have this problem. Unfortunately the cluster is very lightly used
currently until we go full production so we do not have the level of traffic that would
generate a lot of statistics.
We did update to 14.2.16 from 14.2.10 on Feb 1, 2021 and this seems to correlate with when
the errors started popping up.
Our current plan is to roll back the version to 14.2.10 again and rerun the test that
causes the issue.
I noted there was another email thread regarding latencies for a user who also updated to
14.2.16 recently and I'm not sure if this could be related or not to my issue.
Any suggestions you may have are very welcomed.
Cheers,
--
Mike Cave
On 2021-02-11, 8:37 AM, "Mike Cave" <mcave(a)uvic.ca> wrote:
So, as the subject states I have an issue with buckets returning a 404 error when they
are listed immediately after being created; as well the bucket fails to be deleted if you
try to delete it immediately after creation.
The behaviour is intermittent.
If I leave the bucket in place for a few minutes, the bucket behaves normally. I’m
thinking this is a metadata issue or something along those lines but I’m out of my depth
now.
To the best of our knowledge the cluster has not changed in any way since the same
tests were run in December with no errors.
We are running Ceph 14.2.16 on all parts of the cluster.
I am using the python-swift client for the connection on a CentOS7 machine.
Can replicate the results from the mons or an external client as well.
I’m willing to share my test script as well if you would like to see how I’m
generating the error.
Here is a piece of the logs in case I missed something in the interpretation (log
level at 20):
14:23:17.069 7faba00df700 1 ====== starting new request req=0x55fb7a138700 =====
14:23:17.069 7faba00df700 2 req 148 0.000s initializing for trans_id =
tx000000000000000000094-0060245cd5-2b8949-default
14:23:17.069 7faba00df700 10 rgw api priority: s3=8 s3website=7
14:23:17.069 7faba00df700 10 host=<NameRemoved>
14:23:17.069 7faba00df700 20 subdomain= domain= in_hosted_domain=0
in_hosted_domain_s3website=0
14:23:17.069 7faba00df700 -1 res_query() failed
14:23:17.069 7faba00df700 20 final domain/bucket subdomain= domain= in_hosted_domain=0
in_hosted_domain_s3website=0 s->info.domain= s->info.request_uri=/swift/v1/404test
14:23:17.069 7faba00df700 10 ver=v1 first=404test req=
14:23:17.069 7faba00df700 10 handler=28RGWHandler_REST_Bucket_SWIFT
14:23:17.069 7faba00df700 2 req 148 0.000s getting op 2
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket scheduling with
dmclock client=3 cost=1
14:23:17.069 7faba00df700 10 op=30RGWDeleteBucket_ObjStore_SWIFT
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying requester
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::TempURLEngine
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::TempURLEngine denied with reason=-13
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::SignedTokenEngine
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
swift_user=xmcc:swift
14:23:17.069 7faba00df700 20 build_token
token=0a000000786d63633a73776966748960ea4653df708a55ae2560e58acf01
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::SignedTokenEngine granted access
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket normalizing buckets
and tenants
14:23:17.069 7faba00df700 10 s->object=<NULL> s->bucket=404test
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket init permissions
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137770
obj=default.rgw.meta:root:404test state=0x55fb7a060ac0 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get: name=default.rgw.meta+root+404test : hit
(negative entry)
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137130
obj=default.rgw.meta:users.uid:xmcc state=0x55fb7a060f40 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get: name=default.rgw.meta+users.uid+xmcc : hit
(requested=0x6, cached=0x17)
14:23:17.069 7faba00df700 20 get_system_obj_state: s->obj_tag was set empty
14:23:17.069 7faba00df700 20 Read xattr: user.rgw.idtag
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137130
obj=default.rgw.meta:users.uid:xmcc state=0x55fb7a060f40 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get: name=default.rgw.meta+users.uid+xmcc : hit
(requested=0x6, cached=0x17)
14:23:17.069 7faba00df700 20 get_system_obj_state: s->obj_tag was set empty
14:23:17.069 7faba00df700 20 Read xattr: user.rgw.idtag
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket recalculating target
14:23:17.069 7faba00df700 10 Starting retarget
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket reading permissions
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket init op
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying op mask
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket required_mask= 4
user.op_mask=7
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying op
permissions
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket -- Getting permissions
begin with perm_mask=50
14:23:17.069 7faba00df700 5 req 148 0.000s swift:delete_bucket Searching permissions
for identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier ->
rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift, perm_mask=15,
is_admin=0) mask=50
14:23:17.069 7faba00df700 5 Searching permissions for uid=xmcc
14:23:17.069 7faba00df700 5 Found permission: 15
14:23:17.069 7faba00df700 5 Searching permissions for group=1 mask=50
14:23:17.069 7faba00df700 5 Permissions for group not found
14:23:17.069 7faba00df700 5 Searching permissions for group=2 mask=50
14:23:17.069 7faba00df700 5 Permissions for group not found
14:23:17.069 7faba00df700 5 req 148 0.000s swift:delete_bucket -- Getting permissions
done for identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier
-> rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift, perm_mask=15,
is_admin=0), owner=xmcc, perm=2
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier ->
rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift, perm_mask=15,
is_admin=0) requested perm (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying op params
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket pre-executing
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket executing
14:23:17.069 7faba00df700 0 req 148 0.000s swift:delete_bucket ERROR: bucket 404test
not found
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket completing
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket op status=-2002
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket http status=404
14:23:17.069 7faba00df700 1 ====== req done req=0x55fb7a138700 op status=-2002
http_status=404 latency=0s ======
--
Mike Cave
I acknowledge and respect the Lekwungen-speaking Peoples on whose traditional
territories the university stands and the Songhees, Esquimalt and WSANEC peoples whose
historical relationships with the land continue to this day.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
17 Feb
17 Feb
8:51 p.m.
New subject: RGW/Swift 404 error when listing/deleting a newly created empty bucket
Mike, do you create ticket for this issue, especially with logs and reproducer?
k
Sent from my iPhone
On 17 Feb 2021, at 20:54, Mike Cave
<mcave(a)uvic.ca> wrote:
I am bumping this email to hopefully get some more eyes on it.
We are continuing to have this problem. Unfortunately the cluster is very lightly used
currently until we go full production so we do not have the level of traffic that would
generate a lot of statistics.
We did update to 14.2.16 from 14.2.10 on Feb 1, 2021 and this seems to correlate with
when the errors started popping up.
Our current plan is to roll back the version to 14.2.10 again and rerun the test that
causes the issue.
I noted there was another email thread regarding latencies for a user who also updated to
14.2.16 recently and I'm not sure if this could be related or not to my issue.
Any suggestions you may have are very welcomed.
9:40 p.m.
New subject: RGW/Swift 404 error when listing/deleting a newly created empty bucket
Hi Konstantin, thank you for your response.
I have not created a bug report yet, as I was not sure if it was a bug or if I had a
configuration issue.
A new piece of information is that if I create a bucket, then try to delete it, it fails
with a 404.
Now if I create a bucket, restart the rgw service, and then try to delete the bucket, it
deletes just fine.
I'll include this in the bug report.
Thank you,
--
Mike Cave
I acknowledge and respect the Lekwungen-speaking Peoples on whose traditional territories
the university stands and the Songhees, Esquimalt and WSANEC peoples whose historical
relationships with the land continue to this day.
On 2021-02-17, 11:51 AM, "Konstantin Shalygin" <k0ste(a)k0ste.ru> wrote:
Notice: This message was sent from outside the University of Victoria email system.
Please be cautious with links and sensitive information.
Mike, do you create ticket for this issue, especially with logs and reproducer?
k
Sent from my iPhone
On 17 Feb 2021, at 20:54, Mike Cave
<mcave(a)uvic.ca> wrote:
I am bumping this email to hopefully get some more eyes on it.
We are continuing to have this problem. Unfortunately the cluster is very lightly used
currently until we go full production so we do not have the level of traffic that would
generate a lot of statistics.
We did update to 14.2.16 from 14.2.10 on Feb 1, 2021 and this seems to correlate with
when the errors started popping up.
Our current plan is to roll back the version to 14.2.10 again and rerun the test that
causes the issue.
I noted there was another email thread regarding latencies for a user who also updated to
14.2.16 recently and I'm not sure if this could be related or not to my issue.
Any suggestions you may have are very welcomed.
9:52 p.m.
New subject: RGW/Swift 404 error when listing/deleting a newly created empty bucket
Hey Mike,
Maybe it's this? https://tracker.ceph.com/issues/48632
Cheers, Dan
On Wed, Feb 17, 2021, 6:53 PM Mike Cave <mcave(a)uvic.ca> wrote:
I am bumping this email to hopefully get some more
eyes on it.
We are continuing to have this problem. Unfortunately the cluster is very
lightly used currently until we go full production so we do not have the
level of traffic that would generate a lot of statistics.
We did update to 14.2.16 from 14.2.10 on Feb 1, 2021 and this seems to
correlate with when the errors started popping up.
Our current plan is to roll back the version to 14.2.10 again and rerun
the test that causes the issue.
I noted there was another email thread regarding latencies for a user who
also updated to 14.2.16 recently and I'm not sure if this could be related
or not to my issue.
Any suggestions you may have are very welcomed.
Cheers,
--
Mike Cave
On 2021-02-11, 8:37 AM, "Mike Cave" <mcave(a)uvic.ca> wrote:
So, as the subject states I have an issue with buckets returning a 404
error when they are listed immediately after being created; as well the
bucket fails to be deleted if you try to delete it immediately after
creation.
The behaviour is intermittent.
If I leave the bucket in place for a few minutes, the bucket behaves
normally. I’m thinking this is a metadata issue or something along those
lines but I’m out of my depth now.
To the best of our knowledge the cluster has not changed in any way
since the same tests were run in December with no errors.
We are running Ceph 14.2.16 on all parts of the cluster.
I am using the python-swift client for the connection on a CentOS7
machine.
Can replicate the results from the mons or an external client as well.
I’m willing to share my test script as well if you would like to see
how I’m generating the error.
Here is a piece of the logs in case I missed something in the
interpretation (log level at 20):
14:23:17.069 7faba00df700 1 ====== starting new request
req=0x55fb7a138700 =====
14:23:17.069 7faba00df700 2 req 148 0.000s initializing for trans_id
= tx000000000000000000094-0060245cd5-2b8949-default
14:23:17.069 7faba00df700 10 rgw api priority: s3=8 s3website=7
14:23:17.069 7faba00df700 10 host=<NameRemoved>
14:23:17.069 7faba00df700 20 subdomain= domain= in_hosted_domain=0
in_hosted_domain_s3website=0
14:23:17.069 7faba00df700 -1 res_query() failed
14:23:17.069 7faba00df700 20 final domain/bucket subdomain= domain=
in_hosted_domain=0 in_hosted_domain_s3website=0 s->info.domain=
s->info.request_uri=/swift/v1/404test
14:23:17.069 7faba00df700 10 ver=v1 first=404test req=
14:23:17.069 7faba00df700 10 handler=28RGWHandler_REST_Bucket_SWIFT
14:23:17.069 7faba00df700 2 req 148 0.000s getting op 2
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
scheduling with dmclock client=3 cost=1
14:23:17.069 7faba00df700 10 op=30RGWDeleteBucket_ObjStore_SWIFT
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
verifying requester
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::TempURLEngine
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::TempURLEngine denied with reason=-13
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::DefaultStrategy: trying
rgw::auth::swift::SignedTokenEngine
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
swift_user=xmcc:swift
14:23:17.069 7faba00df700 20 build_token
token=0a000000786d63633a73776966748960ea4653df708a55ae2560e58acf01
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::SignedTokenEngine granted access
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
normalizing buckets and tenants
14:23:17.069 7faba00df700 10 s->object=<NULL> s->bucket=404test
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket init
permissions
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137770
obj=default.rgw.meta:root:404test state=0x55fb7a060ac0 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get:
name=default.rgw.meta+root+404test : hit (negative entry)
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137130
obj=default.rgw.meta:users.uid:xmcc state=0x55fb7a060f40 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get:
name=default.rgw.meta+users.uid+xmcc : hit (requested=0x6, cached=0x17)
14:23:17.069 7faba00df700 20 get_system_obj_state: s->obj_tag was set
empty
14:23:17.069 7faba00df700 20 Read xattr: user.rgw.idtag
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137130
obj=default.rgw.meta:users.uid:xmcc state=0x55fb7a060f40 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get:
name=default.rgw.meta+users.uid+xmcc : hit (requested=0x6, cached=0x17)
14:23:17.069 7faba00df700 20 get_system_obj_state: s->obj_tag was set
empty
14:23:17.069 7faba00df700 20 Read xattr: user.rgw.idtag
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
recalculating target
14:23:17.069 7faba00df700 10 Starting retarget
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
reading permissions
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket init op
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
verifying op mask
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
required_mask= 4 user.op_mask=7
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
verifying op permissions
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket --
Getting permissions begin with perm_mask=50
14:23:17.069 7faba00df700 5 req 148 0.000s swift:delete_bucket
Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier() ->
rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=xmcc,
acct_name=xmcc, subuser=swift, perm_mask=15, is_admin=0) mask=50
14:23:17.069 7faba00df700 5 Searching permissions for uid=xmcc
14:23:17.069 7faba00df700 5 Found permission: 15
14:23:17.069 7faba00df700 5 Searching permissions for group=1 mask=50
14:23:17.069 7faba00df700 5 Permissions for group not found
14:23:17.069 7faba00df700 5 Searching permissions for group=2 mask=50
14:23:17.069 7faba00df700 5 Permissions for group not found
14:23:17.069 7faba00df700 5 req 148 0.000s swift:delete_bucket --
Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier()
-> rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=xmcc,
acct_name=xmcc, subuser=swift, perm_mask=15, is_admin=0), owner=xmcc, perm=2
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier
-> rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift,
perm_mask=15, is_admin=0) requested perm (type)=2, policy perm=2,
user_perm_mask=2, acl perm=2
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
verifying op params
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
pre-executing
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
executing
14:23:17.069 7faba00df700 0 req 148 0.000s swift:delete_bucket ERROR:
bucket 404test not found
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket
completing
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket op
status=-2002
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket http
status=404
14:23:17.069 7faba00df700 1 ====== req done req=0x55fb7a138700 op
status=-2002 http_status=404 latency=0s ======
--
Mike Cave
I acknowledge and respect the Lekwungen-speaking Peoples on whose
traditional territories the university stands and the Songhees, Esquimalt
and WSANEC peoples whose historical relationships with the land continue to
this day.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
11:32 p.m.
New subject: RGW/Swift 404 error when listing/deleting a newly created empty bucket
Thanks for that link Dan.
I had searched for a bug but did not find that one.
Rolling back to 14.2.10 (our previous version) has resolved our issue for now. I’ll keep
an eye out for an update with this bug fix.
Cheers,
--
Mike Cave
From: Dan van der Ster <dan(a)vanderster.com>
Date: Wednesday, February 17, 2021 at 12:53 PM
To: Cave Mike <mcave(a)uvic.ca>
Cc: Ceph Users <ceph-users(a)ceph.io>
Subject: Re: [ceph-users] Re: RGW/Swift 404 error when listing/deleting a newly created
empty bucket
Notice: This message was sent from outside the University of Victoria email system. Please
be cautious with links and sensitive information.
Hey Mike,
Maybe it's this? https://tracker.ceph.com/issues/48632
Cheers, Dan
On Wed, Feb 17, 2021, 6:53 PM Mike Cave <mcave@uvic.ca<mailto:mcave@uvic.ca>>
wrote:
I am bumping this email to hopefully get some more eyes on it.
We are continuing to have this problem. Unfortunately the cluster is very lightly used
currently until we go full production so we do not have the level of traffic that would
generate a lot of statistics.
We did update to 14.2.16 from 14.2.10 on Feb 1, 2021 and this seems to correlate with when
the errors started popping up.
Our current plan is to roll back the version to 14.2.10 again and rerun the test that
causes the issue.
I noted there was another email thread regarding latencies for a user who also updated to
14.2.16 recently and I'm not sure if this could be related or not to my issue.
Any suggestions you may have are very welcomed.
Cheers,
--
Mike Cave
On 2021-02-11, 8:37 AM, "Mike Cave"
<mcave@uvic.ca<mailto:mcave@uvic.ca>> wrote:
So, as the subject states I have an issue with buckets returning a 404 error when they
are listed immediately after being created; as well the bucket fails to be deleted if you
try to delete it immediately after creation.
The behaviour is intermittent.
If I leave the bucket in place for a few minutes, the bucket behaves normally. I’m
thinking this is a metadata issue or something along those lines but I’m out of my depth
now.
To the best of our knowledge the cluster has not changed in any way since the same
tests were run in December with no errors.
We are running Ceph 14.2.16 on all parts of the cluster.
I am using the python-swift client for the connection on a CentOS7 machine.
Can replicate the results from the mons or an external client as well.
I’m willing to share my test script as well if you would like to see how I’m
generating the error.
Here is a piece of the logs in case I missed something in the interpretation (log
level at 20):
14:23:17.069 7faba00df700 1 ====== starting new request req=0x55fb7a138700 =====
14:23:17.069 7faba00df700 2 req 148 0.000s initializing for trans_id =
tx000000000000000000094-0060245cd5-2b8949-default
14:23:17.069 7faba00df700 10 rgw api priority: s3=8 s3website=7
14:23:17.069 7faba00df700 10 host=<NameRemoved>
14:23:17.069 7faba00df700 20 subdomain= domain= in_hosted_domain=0
in_hosted_domain_s3website=0
14:23:17.069 7faba00df700 -1 res_query() failed
14:23:17.069 7faba00df700 20 final domain/bucket subdomain= domain= in_hosted_domain=0
in_hosted_domain_s3website=0 s->info.domain= s->info.request_uri=/swift/v1/404test
14:23:17.069 7faba00df700 10 ver=v1 first=404test req=
14:23:17.069 7faba00df700 10 handler=28RGWHandler_REST_Bucket_SWIFT
14:23:17.069 7faba00df700 2 req 148 0.000s getting op 2
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket scheduling with
dmclock client=3 cost=1
14:23:17.069 7faba00df700 10 op=30RGWDeleteBucket_ObjStore_SWIFT
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying requester
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::TempURLEngine
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::TempURLEngine denied with reason=-13
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::SignedTokenEngine
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
swift_user=xmcc:swift
14:23:17.069 7faba00df700 20 build_token
token=0a000000786d63633a73776966748960ea4653df708a55ae2560e58acf01
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket
rgw::auth::swift::SignedTokenEngine granted access
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket normalizing buckets
and tenants
14:23:17.069 7faba00df700 10 s->object=<NULL> s->bucket=404test
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket init permissions
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137770
obj=default.rgw.meta:root:404test state=0x55fb7a060ac0 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get: name=default.rgw.meta+root+404test : hit
(negative entry)
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137130
obj=default.rgw.meta:users.uid:xmcc state=0x55fb7a060f40 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get: name=default.rgw.meta+users.uid+xmcc : hit
(requested=0x6, cached=0x17)
14:23:17.069 7faba00df700 20 get_system_obj_state: s->obj_tag was set empty
14:23:17.069 7faba00df700 20 Read xattr: user.rgw.idtag
14:23:17.069 7faba00df700 20 get_system_obj_state: rctx=0x55fb7a137130
obj=default.rgw.meta:users.uid:xmcc state=0x55fb7a060f40 s->prefetch_data=0
14:23:17.069 7faba00df700 10 cache get: name=default.rgw.meta+users.uid+xmcc : hit
(requested=0x6, cached=0x17)
14:23:17.069 7faba00df700 20 get_system_obj_state: s->obj_tag was set empty
14:23:17.069 7faba00df700 20 Read xattr: user.rgw.idtag
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket recalculating target
14:23:17.069 7faba00df700 10 Starting retarget
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket reading permissions
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket init op
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying op mask
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket required_mask= 4
user.op_mask=7
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying op
permissions
14:23:17.069 7faba00df700 20 req 148 0.000s swift:delete_bucket -- Getting permissions
begin with perm_mask=50
14:23:17.069 7faba00df700 5 req 148 0.000s swift:delete_bucket Searching permissions
for identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier ->
rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift, perm_mask=15,
is_admin=0) mask=50
14:23:17.069 7faba00df700 5 Searching permissions for uid=xmcc
14:23:17.069 7faba00df700 5 Found permission: 15
14:23:17.069 7faba00df700 5 Searching permissions for group=1 mask=50
14:23:17.069 7faba00df700 5 Permissions for group not found
14:23:17.069 7faba00df700 5 Searching permissions for group=2 mask=50
14:23:17.069 7faba00df700 5 Permissions for group not found
14:23:17.069 7faba00df700 5 req 148 0.000s swift:delete_bucket -- Getting permissions
done for identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier
-> rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift, perm_mask=15,
is_admin=0), owner=xmcc, perm=2
14:23:17.069 7faba00df700 10 req 148 0.000s swift:delete_bucket
identity=rgw::auth::ThirdPartyAccountApplier() -> rgw::auth::SysReqApplier ->
rgw::auth::LocalApplier(acct_user=xmcc, acct_name=xmcc, subuser=swift, perm_mask=15,
is_admin=0) requested perm (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket verifying op params
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket pre-executing
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket executing
14:23:17.069 7faba00df700 0 req 148 0.000s swift:delete_bucket ERROR: bucket 404test
not found
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket completing
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket op status=-2002
14:23:17.069 7faba00df700 2 req 148 0.000s swift:delete_bucket http status=404
14:23:17.069 7faba00df700 1 ====== req done req=0x55fb7a138700 op status=-2002
http_status=404 latency=0s ======
--
Mike Cave
I acknowledge and respect the Lekwungen-speaking Peoples on whose traditional
territories the university stands and the Songhees, Esquimalt and WSANEC peoples whose
historical relationships with the land continue to this day.
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io<mailto:ceph-users@ceph.io>
To unsubscribe send an email to
ceph-users-leave@ceph.io<mailto:ceph-users-leave@ceph.io>
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io<mailto:ceph-users@ceph.io>
To unsubscribe send an email to
ceph-users-leave@ceph.io<mailto:ceph-users-leave@ceph.io>
1173
days inactive
1173
days old
4 comments
3 participants
participants (3)
-
Dan van der Ster -
Konstantin Shalygin -
Mike Cave