- ceph-users - lists.ceph.io

Re: Recommended settings for PostgreSQL
by Gencer W. Genç 19 Oct '20

19 Oct '20

Yes, I had to tune some settings on PostgreSQL. Especially on: synchronous_commit = off I have a default RBD settings. Do you have any recommendation? Thanks, Gencer. On 19.10.2020 12:49:51, Marc Roos <m.roos(a)f1-outsourcing.eu> wrote: > In the past I see some good results (benchmark & latencies) for MySQL and PostgreSQL. However, I've always used > 4MB object size. Maybe i can get much better performance on smaller object size. Haven't tried actually. Did you tune mysql / postgres for this setup? Did you have a default ceph rbd setup?

1 0

RGW with HAProxy
by Seena Fallah 19 Oct '20

19 Oct '20

Hi When I use haproxy with keep-alive mode to rgws, haproxy gives many responses like this! Is there any problem with keep-alive mode in rgw? Using nautilus 14.2.9 with beast frontend.

1 0

Module 'cephadm' has failed: cephadm exited with an error code: 2, stderr:usage: rm-daemon [-h] --name NAME --fsid FSID [--force] [--force-delete-data]
by 周凡夫 19 Oct '20

19 Oct '20

Hello, I had ERRs in using Cephadm for my company. This an important question for me and my company. Hope you can post it. Thanks a lot. Fanfu

1 0

Re: multiple OSD crash, unfound objects
by Chad William Seys 16 Oct '20

16 Oct '20

This problem may also be related to the below unsolved issue, which specifically mentions 'unfound' objects. Sadly, there is probably nothing in the report which will help with your troubleshooting. https://tracker.ceph.com/issues/44286 C.

1 0

radosgw bucket subdomain with tls
by Marc Roos 15 Oct '20

15 Oct '20

I enabled a certificate on my radosgw, but I think I am running into the problem that the s3 clients are accessing the buckets like bucket.rgw.domain.com. Which fails my cert rgw.domain.com. Is there any way to configure that only rgw.domain.com is being used?

3 3

Cannot ingest data after quota full and modify quota
by St-Germain, Sylvain (SSC/SPC) 15 Oct '20

15 Oct '20

Good morning all, I don't know if this happened to someone, but recently a user ran out of their quota for the number of objects in a bucket. The only sign I could see in the logs (tail -f /var/log/ceph/ceph-rgw-dao-wkr-01.rgw0.log) was the following: 2020-10-01T03:07:02.098+0000 7fd872bf6700 1 ====== req done req=0x7fd8cb13a8a0 op status=-2026 http_status=413 latency=17.304220706s ====== When I removed the User limits the error went from 413 to 422 in the logs. radosgw-admin quota set --quota-scope=bucket --uid=smithj --max-objects=-1 2020-10-15T11:58:41.661+0000 7f0c79b51700 1 ====== req done req=0x7f0cb7f5f8a0 op status=-2018 http_status=422 latency=0.001000013s ====== I have tried a lot of radosgw-admin command, but so far the error persists. Do you have any ideas ? Thx ! Sylvain

1 0

Using crushtool reclassify to insert device class into existing crush map
by Mathias Lindberg 15 Oct '20

15 Oct '20

Hi, We are trying to introduce SSD/NvME OSD’s and to prevent data moving from current (hdd based) OSD’s while also having erasure coded pools we could not just simply change the erasure coding profile or create a new one and just apply it to the pool. Reading this list and other posts on forums it was suggested to use the crushtool reclassify to insert device classes into the current crush rules and load that crush map manually. Not having edited the crush map in this or any other fashion earlier I would appreciate very much if I someone could verify if I have done the reclassify correctly. Thank you, [root@cephyr-mon1 crushtest]# crushtool -i crush_comp.c --reclassify --reclassify-root default hdd -o crush_comp_corr.c classify_root default (-1) as hdd renumbering bucket -1 -> -29 renumbering bucket -27 -> -30 renumbering bucket -25 -> -31 renumbering bucket -23 -> -32 renumbering bucket -21 -> -33 renumbering bucket -19 -> -34 renumbering bucket -17 -> -35 renumbering bucket -15 -> -36 renumbering bucket -13 -> -37 renumbering bucket -11 -> -38 renumbering bucket -9 -> -39 renumbering bucket -7 -> -40 renumbering bucket -5 -> -41 renumbering bucket -3 -> -42 [root@cephyr-mon1 crushtest]# crushtool -i crush_comp.c --compare crush_comp_corr.c rule 0 had 0/10240 mismatched mappings (0) rule 6 had 0/10240 mismatched mappings (0) rule 7 had 0/4096 mismatched mappings (0) rule 8 had 0/4096 mismatched mappings (0) rule 9 had 0/4096 mismatched mappings (0) rule 10 had 0/4096 mismatched mappings (0) rule 11 had 0/4096 mismatched mappings (0) rule 12 had 0/4096 mismatched mappings (0) rule 13 had 0/4096 mismatched mappings (0) maps appear equivalent Regards, Mathias Lindberg Tel: +46 (0)31 7723059 Mob: +46 (0)723 526107 Mathias Lindberg mathlin(a)chalmers.se

2 1

TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES
by Seena Fallah 14 Oct '20

14 Oct '20

Hi all, Is TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES configured just for filestore or can be used for bluestore, too? https://github.com/ceph/ceph/blob/master/etc/default/ceph#L7 Thanks.

2 2

Proxmox+Ceph Benchmark 2020
by Alwin Antreich 14 Oct '20

14 Oct '20

Hello fellow Ceph users, we have released our new Ceph benchmark paper [0]. The used platform and Hardware is Proxmox VE 6.2 with Ceph Octopus on a new AMD Epyc Zen2 CPU with U.2 SSDs (details in the paper). The paper should illustrate the performance that is possible with a 3x node cluster without significant tuning. I welcome everyone to share their experience and add to the discussion, perferred on our forum [1] thread with our fellow Proxmox VE users. -- Cheers, Alwin [0] https://proxmox.com/en/downloads/item/proxmox-ve-ceph-benchmark-2020-09 [1] https://forum.proxmox.com/threads/proxmox-ve-ceph-benchmark-2020-09-hyper-c…

6 8

ceph orch apply rgw - rgw fails to boot
by Karsten Nielsen 14 Oct '20

14 Oct '20

Hi, I have been setting up a new cluster with a combination of cephadm and ceph orch. I have run in to a problem with rgw daemons that does not start. I have been following the documentation: https://docs.ceph.com/en/latest/cephadm/install/ - the RGW section ceph orch apply rgw ikea cn-dc9-1 --port=8080 --placement="3 itcnchn-bb4065 itcnchn-bb4066 itcnchn-bb4067" Then wait for everything to settle and then rgw's was expected to start on nodes itcnchn-bb4065 itcnchn-bb4066 itcnchn-bb4067 but that was not the case. ceph orch ps itcnchn-bb4065 NAME HOST STATUS REFRESHED AGE VERSION IMAGE NAME IMAGE ID CONTAINER ID alertmanager.itcnchn-bb4065 itcnchn-bb4065 running (3w) 2m ago 4w 0.20.0 prom/alertmanager:v0.20.0 0881eb8f169f c595826b8867 crash.itcnchn-bb4065 itcnchn-bb4065 running (4w) 2m ago 4w 15.2.4 docker.io/ceph/ceph:v15 852b28cb10de 4598d37bf91c grafana.itcnchn-bb4065 itcnchn-bb4065 running (4w) 2m ago 4w 6.6.2 ceph/ceph-grafana:latest 87a51ecf0b1c ad8e16d12ef8 mgr.itcnchn-bb4065.jxtcer itcnchn-bb4065 running (4w) 2m ago 4w 15.2.4 docker.io/ceph/ceph:v15 852b28cb10de d9278a7e772c mon.itcnchn-bb4065 itcnchn-bb4065 running (4w) 2m ago 4w 15.2.4 docker.io/ceph/ceph:v15 852b28cb10de e0edd4a9038a prometheus.itcnchn-bb4065 itcnchn-bb4065 running (3w) 2m ago 4w 2.18.1 prom/prometheus:v2.18.1 de242295e225 1103b62e191b rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict itcnchn-bb4065 error 2m ago 22h <unknown> docker.io/ceph/ceph:v15 <unknown> <unknown> I have logged into itcnchn-bb4065 and tried to start the rgw container my self, the command from (/var/lib/ceph/0aba4c3a-f735-11ea-af33-02423ee04865/rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict/unit.run) /usr/bin/docker run --rm --net=host --ipc=host --name ceph-0aba4c3a-f735-11ea-af33-02423ee04865-rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15 -e NODE_NAME=itcnchn-bb4065 -v /var/run/ceph/0aba4c3a-f735-11ea-af33-02423ee04865:/var/run/ceph:z -v /var/log/ceph/0aba4c3a-f735-11ea-af33-02423ee04865:/var/log/ceph:z -v /var/lib/ceph/0aba4c3a-f735-11ea-af33-02423ee04865/crash:/var/lib/ceph/crash:z -v /var/lib/ceph/0aba4c3a-f735-11ea-af33-02423ee04865/rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict:/var/lib/ceph/radosgw/ceph-rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict:z -v /var/lib/ceph/0aba4c3a-f735-11ea-af33-02423ee04865/rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict/config:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/radosgw docker.io/ceph/ceph:v15 -n client.rgw.ikea.cn-dc9-1.itcnchn-bb4065.xpwict -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix="debug " debug 2020-10-14T06:55:54.835+0000 7f87075b1280 0 deferred set uid:gid to 167:167 (ceph:ceph) debug 2020-10-14T06:55:54.835+0000 7f87075b1280 0 ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable), process radosgw, pid 1 debug 2020-10-14T06:55:54.835+0000 7f87075b1280 0 framework: beast debug 2020-10-14T06:55:54.835+0000 7f87075b1280 0 framework conf key: port, val: 8080 debug 2020-10-14T06:55:54.835+0000 7f87075b1280 1 radosgw_Main not setting numa affinity debug 2020-10-14T06:55:54.875+0000 7f87075b1280 1 Cannot find zone id= (name=cn-dc9-1), switching to local zonegroup configuration debug 2020-10-14T06:55:54.875+0000 7f87075b1280 -1 Cannot find zone id= (name=cn-dc9-1) debug 2020-10-14T06:55:54.875+0000 7f87075b1280 0 ERROR: failed to start notify service ((22) Invalid argument debug 2020-10-14T06:55:54.875+0000 7f87075b1280 0 ERROR: failed to init services (ret=(22) Invalid argument) debug 2020-10-14T06:55:54.879+0000 7f87075b1280 -1 Couldn't init storage provider (RADOS) I am not really sure how to get past this point. Any pointers / help would be very appreciated. Thanks, - Karsten

1 1