- ceph-users - lists.ceph.io

ssd suggestion
by mj 23 Nov '20

23 Nov '20

Hi, We are going to replace our spinning SATA 4GB filestore disks with new 4GB SSD bluestore disks. Our cluster is reading far more than writing. Comparing options, I found the interesting and cheap Micron 5210 ION 3,84TB SSDs. The way we understand it, there is a performance hit, when it comes to continuous writing speeds. But costwise those SSD's are very interesting. (only 450 euro each) Our cluster is only small: consisting of three servers, in a 3/2 redundancy config. I was planning to replace the 8 OSDs on one server, and then take some time to checkout how well (or not...) they perform. We just wanted to ask here: anyone with suggestions on alternative SSDs we should consider? Or other tips we should take into consideration..? Thanks, MJ

2 1

PGs undersized for no reason?
by Frank Schilder 23 Nov '20

23 Nov '20

Hi all, I'm upgrading ceph mimic 13.2.8 to 13.2.10 and make a strange observation. When restarting OSDs on the new version, the PGs come back as undersized. They are missing 1 OSD and I get a lot of objects degraded/misplaced. I have only the noout flag set. Can anyone help me out why the PGs don't peer until they are all complete? Is there a flag I can set to get complete PGs before starting backfill/recovery? Ceph is currently rebuilding objects even though all data should still be there. Hence, the update takes an unreasonable amount of time now and I remember that with the update from 13.2.2 to 13.2.8 PGs came back complete really fast. There was no such extended period with incomplete PGs and degraded redundancy. Thanks and best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14

1 1

HA_proxy setup
by Szabo, Istvan (Agoda) 23 Nov '20

23 Nov '20

Hi, I wonder is there anybody have a setup like I want to setup? 1st subnet: 10.118.170.0/24 (FE users) 2nd subnet: 10.192.150.0/24 (BE users) The users are coming from these subnets, and I want that the FE users will come on the 1st interface on the loadbalancer, the BE users will come one the 2nd interface of the HA_Proxy loadbalancer, so somehow need to create 2 backends maybe in the HA_Proxy config? Both users would go to the same rados gateways and to the same ceph cluster. Somehow I want to create static routes on the loadbalancer, but not sure how can I define in the HA_Proxy config, that go to that specific interface ? Thanks ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

1 0

Sizing radosgw and monitor
by Szabo, Istvan (Agoda) 23 Nov '20

23 Nov '20

Hi, I haven't really find any documentation about how to size radosgw. One redhat doc says we need to decide the ratio like 1:50 or 1:100 osd / rgw. I had an issue earlier where I had a user who source loadbalanced so always went to the same radosgateway and 1 time just maxed out. So the question is, how to monitor rgw, what kind of values or ... ? How to size RGW? Thank you ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

1 0

using fio tool in ceph development cluster (vstart.sh)
by Bobby 22 Nov '20

22 Nov '20

Hi, I am using the Ceph development cluster through vstart.sh script. I would like to measure/benchmark read and write performance (benchmark ceph at a low level). For that I want to use the fio tool. Can we use fio on the development cluster? AFAIK, we can..... I have seen the fio option in the CMakeLists.txt of the Ceph source code. Thanks in advance. BR

2 1

v15.2.6 Octopus released
by David Galloway 21 Nov '20

21 Nov '20

This is the 6th backport release in the Octopus series. This releases fixes a security flaw affecting Messenger V2 for Octopus & Nautilus. We recommend users to update to this release. Notable Changes --------------- * CVE 2020-25660: Fix a regression in Messenger V2 replay attacks Getting Ceph ------------ * Git at git://github.com/ceph/ceph.git * Tarball at http://download.ceph.com/tarballs/ceph-15.2.6.tar.gz * For packages, see http://docs.ceph.com/docs/master/install/get-packages/ * Release git sha1: cb8c61a60551b72614257d632a574d420064c17a

3 2

question about rgw index pool
by Adrian Nicolae 21 Nov '20

21 Nov '20

Hi guys, I'll have a future Ceph deployment with the following setup : - 7 powerful nodes running Ceph 15.2.x with mon, rgw and osd daemons colocated - 100+ SATA drives with EC 4+2 - every OSD will have a large NVME partition (300GB) for rocksdb - the storage will be dedicated for rgw traffic using Swift (no cephfs, no rbd) - we will probably have a lot of 4MB ceph objects (more than 400k millions) after the first year. Does it still matter to have the rgw index pool on dedicated SSD/NVME drives or it's good enough to spread it on many SATA drives with 3x replication and a large PG number ? Thanks.

1 0

Manual bucket resharding problem
by Mateusz Skała 21 Nov '20

21 Nov '20

Hello Community. I need Your help. Few days ago I started manual resharding of one bucket with large objects. Unfortunately I interrupted this by Ctrl+c. At now I can’t start this process again. There is message: # radosgw-admin bucket reshard --bucket objects --num-shards 2 ERROR: the bucket is currently undergoing resharding and cannot be added to the reshard list at this time But list of reshard process is empty: # radosgw-admin reshard list [] # radosgw-admin reshard status --bucket objects [ { "reshard_status": "not-resharding", "new_bucket_instance_id": "", "num_shards": -1 } ] How can I fix this situation ? How to restore possibility resharding this bucket? And BTW is resharding process locking writes/reads on bucket? Regards Mateusz Skała

1 0

Problems with mon
by Mateusz Skała 21 Nov '20

21 Nov '20

Hello Community, I have problems with ceph-mons in docker. Docker pods are starting but I got a lot of messages "e6 handle_auth_request failed to assign global_id” in log. 2 mons are up but I can’t send any ceph commands. Regards Mateusz

2 4

Multisite design details
by Girish Aher 20 Nov '20

20 Nov '20

Hello All, I am looking to understand some of the internal details on how multisite is architected. On the Ceph user list, I see mentions of metadata logs, bucket index shard logs etc. but there is just no documentation anywhere I could find on how multisite works using these. Could someone please point me in the right direction here? Apart from the code, is there any resource that could help me with understanding the multisite internals? --Girish

1 0