March 2020 - ceph-users - lists.ceph.io

by Yoann Moulin

Hello, I have a Nautilus cluster with a cephfs volume, on grafana, it shows that cephfs_data pool is almost full[1] but if I give a look to the pool usage, it looks like I have plenty of space. Which metrics are used by grafana? 1. https://framapic.org/5r7J86s55x6k/jGSIsjEUPYMU.png pool usage: > artemis@icitsrv5:~$ ceph df detail > RAW STORAGE: > CLASS SIZE AVAIL USED RAW USED %RAW USED > hdd 662 TiB 296 TiB 366 TiB 366 TiB 55.32 > TOTAL 662 TiB 296 TiB 366 TiB 366 TiB 55.32 > > POOLS: > POOL ID STORED OBJECTS USED %USED MAX AVAIL QUOTA OBJECTS QUOTA BYTES DIRTY USED COMPR UNDER COMPR > .rgw.root 3 8.1 KiB 15 2.8 MiB 0 63 TiB N/A N/A 15 0 B 0 B > default.rgw.control 4 0 B 8 0 B 0 63 TiB N/A N/A 8 0 B 0 B > default.rgw.meta 5 26 KiB 85 16 MiB 0 63 TiB N/A N/A 85 0 B 0 B > default.rgw.log 6 0 B 207 0 B 0 63 TiB N/A N/A 207 0 B 0 B > cephfs_data 7 113 TiB 139.34M 186 TiB 49.47 138 TiB N/A N/A 139.34M 0 B 0 B > cephfs_metadata 8 54 GiB 10.21M 57 GiB 0.03 63 TiB N/A N/A 10.21M 0 B 0 B > default.rgw.buckets.data 9 122 TiB 54.57M 173 TiB 47.70 138 TiB N/A N/A 54.57M 0 B 0 B > default.rgw.buckets.index 10 2.6 GiB 19.97k 2.6 GiB 0 63 TiB N/A N/A 19.97k 0 B 0 B > default.rgw.buckets.non-ec 11 67 MiB 186 102 MiB 0 63 TiB N/A N/A 186 0 B 0 B > device_health_metrics 12 1.2 MiB 145 1.2 MiB 0 63 TiB N/A N/A 145 0 B 0 B Best, -- Yoann Moulin EPFL IC-IT

4 years, 1 month

1
0
0 0

Radosgw dynamic sharding jewel -> luminous

by Robert LeBlanc

I just upgraded a cluster that I inherited from Jewel to Luminous and trying to work through the new warnings/errors. I got the message about 3 OMAP objects being too big, all of them in the default.rgw.buckets.index pool. I expected that dynamic sharding should kick in, but no luck after several days. I looked at $ radosgw-admin reshard list [2020-03-02 04:27:22.303601 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000000 2020-03-02 04:27:22.305403 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000001 2020-03-02 04:27:22.307038 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000002 2020-03-02 04:27:22.317932 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000003 2020-03-02 04:27:22.348383 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000004 2020-03-02 04:27:22.349212 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000005 2020-03-02 04:27:22.349853 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000006 2020-03-02 04:27:22.350490 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000007 2020-03-02 04:27:22.351256 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000008 2020-03-02 04:27:22.351843 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000009 2020-03-02 04:27:22.353225 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000010 2020-03-02 04:27:22.353910 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000011 2020-03-02 04:27:22.367161 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000012 2020-03-02 04:27:22.367741 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000013 2020-03-02 04:27:22.368498 7f8bb58b8e40 -1 ERROR: failed to list reshard log entries, oid=reshard.0000000014 ] And searching on the Internet indicates tha Luminous added a new "reshard" namespace that the radosgw user needs access to. I'm not sure which pool this namespace was added to (being from Jewel there are a slew of rgw pools) and I'm not sure which rados-gw user it's talking about. I can't find a keyring for rados-admin, but it works so I assume that it is using the admin key ring. The permissions are open. I even appended the namespace option to the admin caps like follows: [client.admin] key = SECRET caps mds = "allow *" caps mon = "allow *" caps: [osd] allow * namespace="*" But I get a new error: $ radosgw-admin reshard list 2020-03-02 05:31:40.875642 7fdb983f7e40 0 failed reading realm info: ret -1 (1) Operation not permitted Any nudge in the right direction would be helpful. I manually sharded the indexes, but I'd really like to have it done automatically from now on so I don't have to worry about it. Thank you, Robert LeBlanc ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1

4 years, 1 month

2
4
0 0

Monitors' election failed on VMs : e4 handle_auth_request failed to assign global_id

by Yoann Moulin

Hello, On a Nautilus cluster, I'd like to move monitors from bare metal servers to VMs to prepare a migration. I have added 3 new monitors on 3 VMs and I'd like to stop the 3 old monitors daemon. But I soon as I stop the 3rd old monitor, the cluster stuck because the election of a new monitor fails. The 3 old monitors are in 14.2.4-1xenial The 3 new monitors are in 14.2.7-1bionic > 2020-03-09 16:06:00.167 7fc4a3138700 1 mon.icvm0017(a)3(peon).paxos(paxos active c 20918592..20919120) lease_timeout -- calling new election > 2020-03-09 16:06:02.143 7fc49f931700 1 mon.icvm0017@3(probing) e4 handle_auth_request failed to assign global_id Did I miss something? In attachment : some logs and ceph.conf Thanks for your help. Best, -- Yoann Moulin EPFL IC-IT

4 years, 1 month

3
3
0 0

Clear health warning

by Peter Eisch

Hi, (nautilus, 14.2.8, whole cluster) I doodled with adding a second cephfs and the project got canceled. I removed the unused cephfs with "ceph fs rm dream --yes-i-really-mean-it" and that worked as expected. I have a lingering health warning though which won't clear. The original cephfs1 volume exists and is healthy: [root@cephmon-03]# ceph fs ls name: cephfs1, metadata pool: stp.cephfs_metadata, data pools: [stp.cephfs_data ] [root@cephmon-03]# ceph mds stat cephfs1:3 {0=cephmon-03=up:active,1=cephmon-02=up:active,2=cephmon-01=up:active} [root@cephmon-03]# ceph health detail HEALTH_WARN insufficient standby MDS daemons available MDS_INSUFFICIENT_STANDBY insufficient standby MDS daemons available have 0; want 1 more [root@cephmon-03]# I have not yet deleted the pools for 'dream', the second cephfs definition. (There is nothing in it.) Before deleting the pools is there a command to clear this warning? Thanks! peter Peter Eisch Senior Site Reliability Engineer T1.612.445.5135 virginpulse.com |virginpulse.com/global-challenge Australia | Bosnia and Herzegovina | Brazil | Canada | Singapore | Switzerland | United Kingdom | USA Confidentiality Notice: The information contained in this e-mail, including any attachment(s), is intended solely for use by the designated recipient(s). Unauthorized use, dissemination, distribution, or reproduction of this message by anyone other than the intended recipient(s), or a person designated as responsible for delivering such messages to the intended recipient, is strictly prohibited and may be unlawful. This e-mail may contain proprietary, confidential or privileged information. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Virgin Pulse, Inc. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. v2.64

4 years, 1 month

2
4
0 0

Link to Nautilus upgrade

by Peter Eisch

Hi, When upgrading a cluster from Luminous to Nautilus I followed a page on ceph.com. I need to do another cluster and while I tagged the link, the page no longer exists. https://docs.ceph.com/master/releases/nautilus/#nautilus-old-upgrade Might anyone have either an updated link or point me to how to find the contents of best steps? Thanks, peter Peter Eisch Senior Site Reliability Engineer T1.612.445.5135 virginpulse.com |virginpulse.com/global-challenge Australia | Bosnia and Herzegovina | Brazil | Canada | Singapore | Switzerland | United Kingdom | USA Confidentiality Notice: The information contained in this e-mail, including any attachment(s), is intended solely for use by the designated recipient(s). Unauthorized use, dissemination, distribution, or reproduction of this message by anyone other than the intended recipient(s), or a person designated as responsible for delivering such messages to the intended recipient, is strictly prohibited and may be unlawful. This e-mail may contain proprietary, confidential or privileged information. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Virgin Pulse, Inc. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. v2.64

4 years, 1 month

2
2
0 0

Hardware feedback before purchasing for a PoC

by Ignacio Ocampo

Hi team, I'm planning to invest in hardware for a PoC and I would like your feedback before the purchase: The goal is to deploy a *16TB* storage cluster, with *3 replicas* thus *3 nodes*. System configuration: https://pcpartpicker.com/list/cfDpDx ($400 USD per node) Some notes about the configuration: - 4-core processor for 4 OSD daemons - 8GB RAM for the first 4TB of storage, that will increase to 16GB of RAM when 16TB of storage. - Motherboard: - 4 x SATA 6 Gb/s (one per each OSD disk) - 2 x PCI-E x1 Slots (1 will be used for an additional Gigabit Ethernet) - 1 x M.2 Slots for the host OS - Ram can increase up-to 32 GB, and another SATA 6b/s controller can be added on PCI-E x1 for growth up to *32TB* As noted, the plan is to deploy nodes with *4TB* and gradually add *12TB* as needed, memory also should be increased to *16GB* after *8TB* threshold. Edit Questions to validate before the purchase 1. Does the hardware components make sense for the *16TB* growth projection? 2. Is it easy to gradually add more capacity to each node (*4TB* each time per node)? Thanks for your support! -- Ignacio Ocampo

4 years, 1 month

4
5
0 0

A fast tool to export/copy a pool

by Vladimir Brik

I am wondering if there exists a tool, faster than "rados export", that can copy and restore read-only pools (to/from another pool or file system). It looks like "rados export" is very slow because it is single threaded (the best I can tell, --workers doesn't make a difference). Vlad

4 years, 1 month

1
0
0 0

ceph df hangs

by Rebecca CH

Hello When I run 'ceph df' on one of osd nodes, the command get hanged and no output to the screen. How to resolve it? Thanks.

4 years, 1 month

2
1
0 0

Re: ceph rbd volumes/images IO details

by M Ranga Swami Reddy

On Fri, Mar 6, 2020 at 1:06 AM M Ranga Swami Reddy <swamireddy(a)gmail.com> wrote: > Hello, > Can we get the IOPs of any rbd image/volume? > > For ex: I have created volumes via OpenStack Cinder. Want to know > the IOPs of these volumes. > > In general - we can get pool stats, but not seen the per volumes stats. > > Any hint here? Appreciated. >

4 years, 1 month

3
3
0 0

Ceph (version 14.2.7) RGW STS AccessDenied

by 曹海旺

Hi ,I want to use the sts to get a temporary credentials,but I got 403 . The user info : { "user_id": "sr", "display_name": "shiran", "email": "", "suspended": 0, "max_buckets": 1000, "subusers": [], "keys": [ { "user": "sr", "access_key": "FBGYU7CUBUACP2OYBE76", "secret_key": "RoovZBBNdlOvVS4LzuuuCsZ1QPA6W81mbNXvonYD" } ], "swift_keys": [], "caps": [ { "type": "roles", "perm": "*" }, { "type": "user-policy", "perm": "*" } ], "op_mask": "read, write, delete", "default_placement": "", "default_storage_class": "", "placement_tags": [], "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "temp_url_keys": [], "type": "rgw", "mfa_ids": [] } and the role info : { "id": "c983c0df-a49c-4e36-957d-7d709f65e292", "name": "S3Access", "path": "/application_abc/component_xyz/", "arn": "arn:aws:iam:::role/application_abc/component_xyz/S3Access", "create_date": "2020-03-04T08:19:41.24Z", "max_session_duration": 3600, "assume_role_policy_document": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":[\"arn:aws:iam:::user/sr\"]},\"Action\":[\"sts:AssumeRole\"]}]}" }, I use AWA java sdk to AssumeRole ,code like: public static final String ACCESSKEY_STRING = “xxxxxxxxxxxxxx"; public static final String BUCKETNAME_STRING = "osstest"; public static final String SECRETKEY_STRING = “xxxxxxxxxxxxxxxxxxxxx"; public static final String ENDPOINT_STRING = "http://192.168.3.250”;// haproxy String clientRegion = ENDPOINT_STRING; String roleARN = "arn:aws:iam:::role/application_abc/component_xyz/S3Access"; String roleSessionName = "bob"; String bucketName = ENDPOINT_STRING; AWSCredentials credentials = new BasicAWSCredentials(ACCESSKEY_STRING, SECRETKEY_STRING); ClientConfiguration clientConfig = new ClientConfiguration(); clientConfig.setProtocol(Protocol.HTTP); clientConfig.setSignerOverride("AWS3SignerType"); AWSSecurityTokenService sts_client = AWSSecurityTokenServiceClientBuilder.standard() .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(ENDPOINT_STRING, "")) .withCredentials(new AWSStaticCredentialsProvider(credentials)).withClientConfiguration(clientConfig) .build(); GetSessionTokenRequest session_token_request = new GetSessionTokenRequest(); AssumeRoleRequest roleRequest = new AssumeRoleRequest() .withRoleArn(roleARN) .withRoleSessionName(roleSessionName).withTokenCode("sts"); session_token_request.setDurationSeconds(3600); AssumeRoleResult roleResponse = sts_client.assumeRole(roleRequest); Credentials sessionCredentials = roleResponse.getCredentials(); I got the response: <?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><RequestId>tx000000000000000000016-005e64fa9e-b5b47-default</RequestId><HostId>b5b47-default-default</HostId></Error>" The debug info in the log file like this : 1 ====== starting new request req=0x5592abefc910 ===== 2020-03-08 22:01:02.984 7f2548f95700 2 req 22 0.000s initializing for trans_id = tx000000000000000000016-005e64fa9e-b5b47-default 2020-03-08 22:01:02.984 7f2548f95700 10 rgw api priority: s3=7 s3website=6 2020-03-08 22:01:02.984 7f2548f95700 10 host=rgw1 2020-03-08 22:01:02.984 7f2548f95700 20 subdomain= domain= in_hosted_domain=0 in_hosted_domain_s3website=0 2020-03-08 22:01:02.984 7f2548f95700 20 final domain/bucket subdomain= domain= in_hosted_domain=0 in_hosted_domain_s3website=0 s->info.domain= s->info.request_uri=/ 2020-03-08 22:01:02.984 7f2548f95700 10 meta>> HTTP_X_AMZ_DATE 2020-03-08 22:01:02.984 7f2548f95700 10 meta>> HTTP_X_AMZN_AUTHORIZATION 2020-03-08 22:01:02.984 7f2548f95700 10 x>> x-amz-date:Sun, 08 Mar 2020 14:01:02 GMT 2020-03-08 22:01:02.984 7f2548f95700 10 x>> x-amzn-authorization:AWS3 AWSAccessKeyId=FBGYU7CUBUACP2OYBE76,Algorithm=HmacSHA256,SignedHeaders=Host;X-Amz-Date,Signature=D7p74AFsJn4J823e6Up4Zo2eRV/n5PFmyQ2/w+kOaqY= 2020-03-08 22:01:02.984 7f2548f95700 20 get_handler handler=26RGWHandler_REST_Service_S3 2020-03-08 22:01:02.984 7f2548f95700 10 handler=26RGWHandler_REST_Service_S3 2020-03-08 22:01:02.984 7f2548f95700 2 req 22 0.000s getting op 4 2020-03-08 22:01:02.984 7f2548f95700 10 Content of POST: Action=AssumeRole&Version=2011-06-15&RoleArn=arn%3Aaws%3Aiam%3A%3A%3Arole%2Fapplication_abc%2Fcomponent_xyz%2FS3Access&RoleSessionName=bob&SerialNumber=sdfsdf&TokenCode=sts 2020-03-08 22:01:02.984 7f2548f95700 10 Key: ActionValue: AssumeRole 2020-03-08 22:01:02.984 7f2548f95700 10 Key: VersionValue: 2011-06-15 2020-03-08 22:01:02.984 7f2548f95700 10 Key: RoleArnValue: arn:aws:iam:::role/application_abc/component_xyz/S3Access 2020-03-08 22:01:02.984 7f2548f95700 10 Key: RoleSessionNameValue: bob 2020-03-08 22:01:02.984 7f2548f95700 10 Key: SerialNumberValue: sdfsdf 2020-03-08 22:01:02.984 7f2548f95700 10 Key: TokenCodeValue: sts 2020-03-08 22:01:02.984 7f2548f95700 10 req 22 0.000s sts:assume_role scheduling with dmclock client=0 cost=1 2020-03-08 22:01:02.984 7f2548f95700 10 op=16RGWSTSAssumeRole 2020-03-08 22:01:02.984 7f2548f95700 2 req 22 0.000s sts:assume_role verifying requester 2020-03-08 22:01:02.984 7f2548f95700 20 req 22 0.000s sts:assume_role rgw::auth::StrategyRegistry::s3_main_strategy_t: trying rgw::auth::s3::AWSAuthStrategy 2020-03-08 22:01:02.984 7f2548f95700 20 req 22 0.000s sts:assume_role rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::S3AnonymousEngine 2020-03-08 22:01:02.984 7f2548f95700 20 req 22 0.000s sts:assume_role rgw::auth::s3::S3AnonymousEngine granted access 2020-03-08 22:01:02.984 7f2548f95700 20 req 22 0.000s sts:assume_role rgw::auth::s3::AWSAuthStrategy granted access 2020-03-08 22:01:02.984 7f2548f95700 2 req 22 0.000s sts:assume_role normalizing buckets and tenants 2020-03-08 22:01:02.984 7f2548f95700 10 s->object=<NULL> s->bucket= 2020-03-08 22:01:02.984 7f2548f95700 2 req 22 0.000s sts:assume_role init permissions 2020-03-08 22:01:02.985 7f2548f95700 20 get_system_obj_state: rctx=0x5592abefb400 obj=default.rgw.meta:users.uid:anonymous state=0x5592aa8d99c0 s->prefetch_data=0 2020-03-08 22:01:02.985 7f2548f95700 10 cache get: name=default.rgw.meta+users.uid+anonymous : type miss (requested=0x6, cached=0x0) 2020-03-08 22:01:02.985 7f2548f95700 20 WARNING: blocking librados call 2020-03-08 22:01:02.985 7f2548f95700 1 -- 192.168.3.247:0/1491423941 --> [v2:192.168.3.244:6800/2853,v1:192.168.3.244:6801/2853] -- osd_op(unknown.0.0:159377 3.0 3:0ab27964:users.uid::anonymous:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e282) v8 -- 0x5592abea3b80 con 0x5592ab6acc00 2020-03-08 22:01:02.985 7f2593ab4700 1 -- 192.168.3.247:0/1491423941 <== osd.5 v2:192.168.3.244:6800/2853 4 ==== osd_op_reply(159377 anonymous [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 195+0+0 (crc 0 0 0) 0x5592abda1400 con 0x5592ab6acc00 2020-03-08 22:01:02.985 7f2548f95700 10 cache put: name=default.rgw.meta+users.uid+anonymous info.flags=0x0 2020-03-08 22:01:02.985 7f2548f95700 10 moving default.rgw.meta+users.uid+anonymous to cache LRU end 2020-03-08 22:01:02.985 7f2548f95700 2 req 22 0.001s sts:assume_role recalculating target 2020-03-08 22:01:02.985 7f2548f95700 2 req 22 0.001s sts:assume_role reading permissions 2020-03-08 22:01:02.985 7f2548f95700 2 req 22 0.001s sts:assume_role init op 2020-03-08 22:01:02.985 7f2548f95700 2 req 22 0.001s sts:assume_role verifying op mask 2020-03-08 22:01:02.985 7f2548f95700 20 req 22 0.001s sts:assume_role required_mask= 0 user.op_mask=7 2020-03-08 22:01:02.985 7f2548f95700 2 req 22 0.001s sts:assume_role verifying op permissions 2020-03-08 22:01:02.985 7f2548f95700 20 get_system_obj_state: rctx=0x5592abefb7f0 obj=default.rgw.meta:roles:role_names.S3Access state=0x5592aa8d99c0 s->prefetch_data=0 2020-03-08 22:01:02.985 7f2548f95700 10 cache get: name=default.rgw.meta+roles+role_names.S3Access : type miss (requested=0x6, cached=0x3) 2020-03-08 22:01:02.985 7f2548f95700 20 WARNING: blocking librados call 2020-03-08 22:01:02.985 7f2548f95700 1 -- 192.168.3.247:0/1491423941 --> [v2:192.168.3.245:6808/2863,v1:192.168.3.245:6809/2863] -- osd_op(unknown.0.0:159378 3.3 3:c86e03bb:roles::role_names.S3Access:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e282) v8 -- 0x5592abf18000 con 0x5592ab6ad400 2020-03-08 22:01:02.986 7f2593ab4700 1 -- 192.168.3.247:0/1491423941 <== osd.8 v2:192.168.3.245:6808/2863 3 ==== osd_op_reply(159378 role_names.S3Access [getxattrs,stat] v0'0 uv4 ondisk = 0) v8 ==== 205+0+20 (crc 0 0 0) 0x5592abda1400 con 0x5592ab6ad400 2020-03-08 22:01:02.986 7f2548f95700 10 cache put: name=default.rgw.meta+roles+role_names.S3Access info.flags=0x6 2020-03-08 22:01:02.986 7f2548f95700 10 moving default.rgw.meta+roles+role_names.S3Access to cache LRU end 2020-03-08 22:01:02.986 7f2548f95700 20 get_system_obj_state: s->obj_tag was set empty 2020-03-08 22:01:02.986 7f2548f95700 10 cache get: name=default.rgw.meta+roles+role_names.S3Access : hit (requested=0x1, cached=0x7) 2020-03-08 22:01:02.986 7f2548f95700 20 get_system_obj_state: rctx=0x5592abefb7f0 obj=default.rgw.meta:roles:roles.c983c0df-a49c-4e36-957d-7d709f65e292 state=0x5592aa8d99c0 s->prefetch_data=0 2020-03-08 22:01:02.986 7f2548f95700 10 cache get: name=default.rgw.meta+roles+roles.c983c0df-a49c-4e36-957d-7d709f65e292 : type miss (requested=0x6, cached=0x3) 2020-03-08 22:01:02.986 7f2548f95700 20 WARNING: blocking librados call 2020-03-08 22:01:02.986 7f2548f95700 1 -- 192.168.3.247:0/1491423941 --> [v2:192.168.3.244:6800/2853,v1:192.168.3.244:6801/2853] -- osd_op(unknown.0.0:159379 3.0 3:0e42bcf0:roles::roles.c983c0df-a49c-4e36-957d-7d709f65e292:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e282) v8 -- 0x5592abf182c0 con 0x5592ab6acc00 2020-03-08 22:01:02.986 7f2593ab4700 1 -- 192.168.3.247:0/1491423941 <== osd.5 v2:192.168.3.244:6800/2853 5 ==== osd_op_reply(159379 roles.c983c0df-a49c-4e36-957d-7d709f65e292 [getxattrs,stat] v0'0 uv4 ondisk = 0) v8 ==== 228+0+20 (crc 0 0 0) 0x5592abda1400 con 0x5592ab6acc00 2020-03-08 22:01:02.986 7f2548f95700 10 cache put: name=default.rgw.meta+roles+roles.c983c0df-a49c-4e36-957d-7d709f65e292 info.flags=0x6 2020-03-08 22:01:02.986 7f2548f95700 10 moving default.rgw.meta+roles+roles.c983c0df-a49c-4e36-957d-7d709f65e292 to cache LRU end 2020-03-08 22:01:02.986 7f2548f95700 20 get_system_obj_state: s->obj_tag was set empty 2020-03-08 22:01:02.986 7f2548f95700 10 cache get: name=default.rgw.meta+roles+roles.c983c0df-a49c-4e36-957d-7d709f65e292 : hit (requested=0x1, cached=0x7) 2020-03-08 22:01:02.986 7f2548f95700 20 op->ERRORHANDLER: err_no=-1 new_err_no=-1 2020-03-08 22:01:02.986 7f2548f95700 2 req 22 0.002s sts:assume_role op status=0 2020-03-08 22:01:02.986 7f2548f95700 2 req 22 0.002s sts:assume_role http status=403

4 years, 1 month

2
1
0 0

2024

2023

2022

2021

2020

2019

ceph-users March 2020