- ceph-users - lists.ceph.io

SOLVED: How to Limit S3 Access to One Subuser
by Ansgar Jazdzewski 03 Sep '24

03 Sep '24

Hi folks, I found countless questions but no real solution on how to have multiple subusers and buckets in one account while limiting access to a bucket to just one specific subuser. Here’s how I managed to make it work: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "DenyAllUsersButOne", "Effect": "Deny", "Action": "s3:*", "Resource": [ "arn:aws:s3:::test-a", "arn:aws:s3:::test-a/*" ], "NotPrincipal": { "AWS": "arn:aws:iam:::user/<UID>:<ACCESSKEY>" } } ] } ``` I hope this might be useful for others as well. Best regards, Ansgar

2 1

rgw: swift tempurl in 18.2.1 cannot handle list_bucket correctly
by Henry Zhang （张搏航） 03 Sep '24

03 Sep '24

Dear team, hi,I met a swift tempurl problem in my prod ceph cluster, I found rgw in 18.2.1 cannot handle list_bucket correctly and led to rgw crash I created an issue: https://tracker.ceph.com/issues/67825 Can someone help to solve the issue? Best regards Henry

1 0

ceph-ansible installation error
by Michel Niyoyita 02 Sep '24

02 Sep '24

Dear team , I configuring ceph cluster using ceph-ansible , ubuntu OS 20.04 . my previous production cluster have been configured using the same configuration and it is working perfectly , now I am trying to build the new cluster using the same configurations , but I am facing the following errors: ansible-playbook site.yml Traceback (most recent call last): File "/usr/bin/ansible-playbook", line 66, in <module> from ansible.utils.display import Display, initialize_locale ImportError: cannot import name 'initialize_locale' from 'ansible.utils.display' (/usr/local/lib/python3.8/dist-packages/ansible/utils/display.py) Can someone help to solve the issue? Best regards Michel

6 11

Large volume of rgw requests on idle multisite.
by Adam Prycki 01 Sep '24

01 Sep '24

Hello, I've been testing rgw sync policies with bucket sync policies on reef 18.2.2 and discovered something that looks like a bug. My setup is really simple. I have 2 ceph clusters with RGW. Main cluster A and backup cluster B. Sync policy allows user to enable replication from A to B. I've enabled replication on few buckets via `aws s3api put-bucket-replication`. Data seems to replicate between cluster without major issues. But after sync completes and there is no data waiting for replication cluster's B rgw starts to spam cluster's A RWG with thousands of requests per second generating unnecessary load on cluster's A rgw.log pool and huge ammounts of system logs. It looks like RGW on cluster B seems to constantly check metadata shard updates. Is there a way to throttle this behavior? Log example from RGW B: 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]payload request hash = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]canonical request = GET /admin/log id=60&max-entries=100&period=579115d4-b9df-435a-99f1-40c7eb03da28&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55&type=metadata date:Sun, 01 Sep 2024 13:45:00 +0000 host:10.0.6.10:80 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20240901T134500Z date;host;x-amz-content-sha256;x-amz-date e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]canonical request hash = a89e3cdb353e45bbaa5356d1b35cd1bbfd5cb4118da8a42677b8456931b16d13 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]string to sign = AWS4-HMAC-SHA256 20240901T134500Z 20240901/home/s3/aws4_request a89e3cdb353e45bbaa5356d1b35cd1bbfd5cb4118da8a42677b8456931b16d13 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]date_k = 665f3152284a19f7eb31f7feb891577b045da2ebe65a12675d7b9b549bd7aa55 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]region_k = 81abe23f881a8ff25c0352f5f61a873bcdac805fba60c0001dedba940aac60a7 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]service_k = 49e4c3a8bce4feed131eccf6df5280dacd75d0a7fec6405a1d41e31804c4d742 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]signing_k = 2578ef828af150f3b303e7f02b87663ed9b6f783bec697048a760cc589491d86 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=60&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]generated signature = 1291a73a183e4fabac1494e1124bde56647e5f63618d1cd91c1ab894757e8da1 2024-09-01T15:45:00.757+0200 7fd0a280b6c0 10 http_client[GET/http://10.0.6.10:80/admin/log?type=metadata&id=47&period=579115d4-b9df-435a-99f1-40c7eb03da28&max-entries=100&&rgwx-zonegroup=3e543ae3-24ed-4568-be71-fa50791ecf55]canonical headers format = date:Sun, 01 Sep 2024 13:45:00 +0000 host:10.0.6.10:80 x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 x-amz-date:20240901T134500Z Best regards Adam Prycki

1 0

How many MDS & MON servers are required
by s.dhivagar.cse＠gmail.com 01 Sep '24

01 Sep '24

Hi, We are using raw cephfs data 182 TB replica 2 and single MDS seemed to regularly run around 4002 req/s, So how many MDS & MON servers are required? Also mentioned current ceph cluster servers Client : 60 MDS: 3 (2 Active + 1 Standby) MON: 4 MGR: 3 (1 Active + 2 Standby) OSD: 52 PG : auto scale

2 1

How to know is ceph is ready?
by Alfredo Rezinovsky 30 Aug '24

30 Aug '24

I have a proxmox cluster using an external CEPH cluster. Sometimes due to blackouts the servers need to restart. If proxmox starts before CEPH is ready the VMs fail to boot. I want to add a dependency in proxmox to wait for ceph to be ready. I can work with a HEALTH_WARN as long the RBD pool is usable. ceph status exit status doesn´t helps Should I grep for "pgs not active" in ceph status or for "inactive" pgs in ceph health or is there something more direct to know if everything is alright? -- Alfrenovsky

5 6

ceph orch host drain daemon type
by Eugen Block 30 Aug '24

30 Aug '24

Hi, is there anything on the road map to be able to choose a specific daemon type to be entirely removed from a host instead of all cephadm managed daemons? I just did a quick search in tracker and github, but it may be "hidden" somewhere else. I was thinking about colocated daemons on a host, for example MON, MGR, OSDs, node-exporter, crash. That's quite common, but if I just wanted to drain all OSDs (maybe mark them as "destroyed" in order to replace the drives), the usual 'ceph orch host drain <host>' command would remove all daemons. That seems unnecessary if I'm going to add the OSDs back. Since there are a couple of other daemon types which can be deployed multiple times per host, e. g. MDS, RGW, it doesn't only make sense for OSDs but for other daemons as well. And those other daemons usually have some cryptic suffix, we wouldn't need that in order to get rid of them, it doesn't save that much time, but it could be a nice enhancement. Should I create a tracker issue in the enhancement section for that? Thanks! Eugen

3 6

squid release codename
by Alfredo Rezinovsky 30 Aug '24

30 Aug '24

I think is a very bad idea to name a release with the name of the most popular http cache. It will difficult googling. -- Alfrenovsky

13 15

how to distinguish SED/OPAL and non SED/OPAL disks in orchestrator?
by Boris 29 Aug '24

29 Aug '24

Hi, is it possible to somehow distinguish self encrypting drives from drives that are lacking the support in the orchestrator. So I don't encrypt them twice :) -- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groÃƒ¼en Saal.

3 3

Found a way to clean ceph device ls
by Alfredo Rezinovsky 28 Aug '24

28 Aug '24

Ceph device ls shows every disk. Even removed ones with ceph config-key ls I can find the old disk key and remove it with ceph config-key rm -- Alfrenovsky

1 0