- ceph-users - lists.ceph.io

Re: [doc] drivegroups advanced case
by Jan Fajerski 24 Aug '20

24 Aug '20

On Fri, Aug 21, 2020 at 06:52:19PM +0000, Tony Liu wrote: >Hi, > >Regarding to the YAML in this section, >https://ceph.readthedocs.io/en/latest/cephadm/drivegroups/#the-advanced-case > >Is the "rotational" supposed to be "1" meaning spinning HDD? Correct. This corresponds to /sys/block/<your block device>/queue/rotational. > > >Thanks! >Tony >_______________________________________________ >ceph-users mailing list -- ceph-users(a)ceph.io >To unsubscribe send an email to ceph-users-leave(a)ceph.io > -- Jan Fajerski Senior Software Engineer Enterprise Storage SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer

1 0

OSD Crash, high RAM usage
by Cloud Guy 23 Aug '20

23 Aug '20

Hello TL;DR We have a Nautilus cluster which has been operating without issue for quite some time. Recently one OSD experienced a relatively slow and painful death. The OSD was purged (via dashboard), replaced and added as a new OSD (same ID). Upon rebuild, we notice the node hosting the replaced OSD to experience extraordinary memory consumption (upwards of 95%). Remaining nodes healthy, operating normally. No adverse performance observations within the cluster. Looking for community assistance in helping troubleshoot this anomaly and resolving. Detail - Ceph 14.2.11 - Originally Deployed using ceph-ansible 4.0 stable; - OSD nodes run CentOS 7, have 10 drives, 256G RAM, no swap; - Problematic OSD ramps memory consumption to 95-99% available RAM; - Problematic OSD node appears to ramp RAM utilization until OSDs crash - and restart – resembling a contiguous saw-tooth pattern / memory leak??; - Other OSD nodes average around 30-40G steady state consumption. Cluster Logs: 2020-08-23 08:36:59.458072 [INF] Cluster is now healthy 2020-08-23 08:36:59.458032 [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2274696/100424847 objects degraded (2.265%), 85 pgs degraded) 2020-08-23 08:36:56.352104 [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 13 pgs inactive, 4 pgs down, 2 pgs peering, 15 pgs incomplete) 2020-08-23 08:36:56.352062 [WRN] Health check update: Degraded data redundancy: 2320455/100424847 objects degraded (2.311%), 87 pgs degraded (PG_DEGRADED) 2020-08-23 08:36:55.401933 [INF] osd.16 [v2:10.1.10.7:6831/204520,v1:10.1.10.7:6833/204520] boot 2020-08-23 08:36:54.744590 [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2020-08-23 08:36:53.688833 [WRN] Health check update: 1 osds down (OSD_DOWN) 2020-08-23 08:36:49.714790 [WRN] Health check update: Degraded data redundancy: 6617521/100424847 objects degraded (6.590%), 298 pgs degraded (PG_DEGRADED) 2020-08-23 08:36:49.714769 [WRN] Health check update: Reduced data availability: 17 pgs inactive, 5 pgs down, 2 pgs peering, 41 pgs incomplete (PG_AVAILABILITY) 2020-08-23 08:36:49.701966 [INF] osd.32 [v2:10.1.10.7:6812/204477,v1:10.1.10.7:6813/204477] boot 2020-08-23 08:36:49.701868 [INF] osd.28 [v2:10.1.10.7:6820/204478,v1:10.1.10.7:6821/204478] boot 2020-08-23 08:36:48.613586 [INF] osd.24 [v2:10.1.10.7:6836/204523,v1:10.1.10.7:6837/204523] boot 2020-08-23 08:36:48.540225 [WRN] Health check update: 3 osds down (OSD_DOWN) 2020-08-23 08:36:43.755645 [WRN] Health check failed: Degraded data redundancy: 5700895/100424847 objects degraded (5.677%), 266 pgs degraded (PG_DEGRADED) 2020-08-23 08:36:43.755611 [WRN] Health check failed: Reduced data availability: 12 pgs inactive, 5 pgs down, 37 pgs incomplete (PG_AVAILABILITY) 2020-08-23 08:36:40.049275 [INF] osd.16 failed (root=default,host=otr2817e02stor23) (connection refused reported by osd.39) 2020-08-23 08:36:40.047058 [INF] osd.24 failed (root=default,host=otr2817e02stor23) (connection refused reported by osd.6) 2020-08-23 08:36:39.452024 [WRN] Health check failed: 2 osds down (OSD_DOWN) 2020-08-23 08:36:39.399920 [INF] osd.32 failed (root=default,host=otr2817e02stor23) (connection refused reported by osd.33) 2020-08-23 08:36:39.399255 [INF] osd.28 failed (root=default,host=otr2817e02stor23) (connection refused reported by osd.18) 2020-08-23 08:04:08.089438 [INF] Cluster is now healthy 2020-08-23 08:04:08.089399 [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1549004/100424847 objects degraded (1.542%), 62 pgs degraded) 2020-08-23 08:04:05.154839 [WRN] Health check update: Degraded data redundancy: 1873878/100424847 objects degraded (1.866%), 68 pgs degraded (PG_DEGRADED) 2020-08-23 08:04:02.728402 [INF] osd.24 [v2:10.1.10.7:6820/202960,v1:10.1.10.7:6821/202960] boot 2020-08-23 08:04:02.572426 [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2020-08-23 08:03:53.138693 [WRN] Health check update: Degraded data redundancy: 2507272/100424847 objects degraded (2.497%), 88 pgs degraded (PG_DEGRADED) 2020-08-23 08:03:48.049569 [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 59 pgs peering) 2020-08-23 08:03:45.170204 [WRN] Health check failed: Degraded data redundancy: 281695/100424847 objects degraded (0.281%), 15 pgs degraded (PG_DEGRADED) 2020-08-23 08:03:45.170175 [WRN] Health check failed: Reduced data availability: 7 pgs inactive, 59 pgs peering (PG_AVAILABILITY) 2020-08-23 08:03:41.355393 [WRN] Health check failed: 1 osds down (OSD_DOWN)

2 1

SEO Services Company
by kseoagency＠gmail.com 23 Aug '20

23 Aug '20

K SEO Agency an SEO Company in Bangalore focused on providing full-service Search Engine Optimization Solutions that include specialized Local SEO, National SEO, Ecommerce SEO services for businesses in Bangalore, India. https://kseo.agency/

1 0

seo for small business
by kseoagency＠gmail.com 23 Aug '20

23 Aug '20

K SEO Agency an SEO Company in Bangalore focused on providing full-service Search Engine Optimization Solutions that include specialized Local SEO, National SEO, Ecommerce SEO services for businesses in Bangalore, India. https://kseo.agency/

1 0

ISO Certification in Kolkata
by kseoagency＠gmail.com 23 Aug '20

23 Aug '20

ISO Certification in Kolkata – Genveritas a global ISO Certification Consulting firm represents considerable authority in tweaked and result-situated answers for assisting organizations to actualize change and improve business execution. https://genveritas.in/iso-certification-in-kolkata/

1 0

How to change wal block in bluestore?
by Xu Xiao 23 Aug '20

23 Aug '20

Hi, I want to change my wal device to another device. I search much time and write a shell script, but finally, when I start the osd, the process is crashed. I think something is wrong, please help me check the script. Thanks! ceph version: ceph version 14.2.9 (bed944f8c45b9c98485e99b70e11bbcec6f6659a) nautilus (stable). The shell script is below: #! /bin/bash id=$1 # osd id vg=$2 # vg name osdPath="/var/lib/ceph/osd/ceph-$id" osdService="ceph-osd(a)${id}.service" blockPath=$(ls -lh ${osdPath}/block|awk '{print $11}') if [ -z "$blockPath" ]; then echo "no block path" exit 1; fi walPath=$(ls -lh ${osdPath}/block.wal|awk '{print $11}') if [ -z "$walPath" ]; then echo "no wal path" exit 2; fi walSize=$(blockdev --getsize64 $walPath) if [ -z "$walSize" ]; then echo "no wal size" exit 3; fi walUUID=$(lvdisplay ${walPath}|grep "LV UUID"|awk '{print $3}') if [ -z "$walUUID" ]; then echo "no wal uuid" exit 3; fi newWalLv="journal-osd${id}" newWalPath="/dev/${vg}/${newWalLv}" lvcreate -n "${newWalLv}" -L${walSize}b $vg || exit 4 newWalUUID=$(lvdisplay ${newWalPath}|grep "LV UUID"|awk '{print $3}') if [ -z "$newWalUUID" ]; then echo "no new wal uuid, new wal path:${newWalPath}" exit 3; fi ceph osd set noout || exit 5 systemctl stop ${osdService} || exit 6 ceph-osd -i $id --flush-journal || exit 7 dd if=${walPath} of=${newWalPath} bs=4M || exit 8 chown -h ceph:ceph $(realpath ${newWalPath}) || exit 9 ceph-bluestore-tool bluefs-bdev-migrate --path ${osdPath} --devs-source ${osdPath}/block.wal --dev-target ${newWalPath} chown -h ceph:ceph ${osdPath}/block.wal || exit 11 # device lvchange --deltag "ceph.wal_device=${walPath}" ${blockPath} || exit 12 lvchange --addtag "ceph.wal_device=${newWalPath}" ${blockPath} || exit 13 # for uuid lvchange --deltag "ceph.wal_uuid=${walUUID}" ${blockPath} || exit 12 lvchange --addtag "ceph.wal_uuid=${newWalUUID}" ${blockPath} || exit 13 # for wal lvs for tag in $(lvs -o lv_tags ${blockPath}|grep ceph|sed 's/,/\n/g') do lvchange --addtag "$tag" ${newWalPath} done lvchange --deltag "ceph.type=block" ${newWalPath} lvchange --addtag "ceph.type=wal" ${newWalPath} systemctl start ceph-osd(a)${id}.service || exit 14 ceph osd unset noout || exit 15

1 0

Add OSD with primary on HDD, WAL and DB on SSD
by Tony Liu 23 Aug '20

23 Aug '20

Hi, I am able to add OSD with primary on HDD, WAL and DB on SSD on dashboard. LVs are created on SSD for OSDs. How can I achieve the same by CLI? I tried with drivegroups, which adds OSD, but not WAL and DB. Should I handle LV for WAL and DB separately? Anything I am missing here? Thanks! Tony

1 0

How to Find the Cheap & Best Assignment Help Sydney Available Online?
by maxwillor＠gmail.com 22 Aug '20

22 Aug '20

I have been a part of Myassignmenthelpau I am working as a fully functional academic consultant with the team. We offer affordable pricing solutions for assignments and homework across all course subjects. I hold excellent writing skills and offer Assignment help Sydney of the best form. https://www.myassignmenthelpau.com/assignment-help-sydney

1 0

does ceph RBD have the ability to load balance?
by 差很多先生 22 Aug '20

22 Aug '20

Hello Cephers, I am a newcomer to Ceph.I know that RBD is a distributed block storage device and libvirt supports rbd pool type. Suppose I have a ceph cluster and use it to build a rbd pool to store virtual machine images. When multiple qemu-kvm clients use the image in the rbd pool to start virtual machines at the same time, will rbd perform load balancing? That is, different hosts in the cluster would send images to different clients at the same time. Or only one host will send images, and the other hosts will only send images when the current host is down? Would someone explain it to me please? Thanks consteven

2 1

Get Instant HELP From Epson support For Epson Error Code 0x97.
by mary smith 22 Aug '20

22 Aug '20

The Epson Error Code 0x97 is a kind of caution at whatever point you experience an inside equipment issue with your printer. It very well may be a result of a motherboard glitch or some other sort of inner segment breakdown. You should choose the guide from Epson support , straightforwardly from comfort of your home. https://www.epsonprintersupportpro.net/epson-error-code-0x97/

1 0