- ceph-users - lists.ceph.io

luks / disk encryption best practice
by Marc Roos 20 Aug '20

20 Aug '20

I still need to move from ceph disk to ceph volume. When doing this, I wanted to also start using disk encryption. I am not really interested in encryption offered by the hdd vendors. Is there a best practice or advice what encryption to use ciphers/hash? Stick to the default of CentOS7 or maybe choose what is default in CentOS or something else? Different settings for ssd / hdd?

1 0

Ceph on windows?
by Stolte, Felix 20 Aug '20

20 Aug '20

Hey guys, it seems like there was a presentation called “ceph on windows” at the Cephalocon 2020, but I cannot find any information on that topic. Is there a video from the presentation out there or any other information? I only found https://ceph2020.sched.com/event/ZDUK/ceph-on-windows-alessandro-pilotti-cl… Would be great to mount rbds directly instead using an iscsi gateway. IT-Services Telefon 02461 61-9243 E-Mail: f.stolte(a)fz-juelich.de ------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt ------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------

3 2

Ceph mon crash, many osd down
by hoannv46＠gmail.com 20 Aug '20

20 Aug '20

Hi all. My cluster has many log mon scrub mon data 2020-08-20 13:12:16.393 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {auth=100} crc {auth=3066031631}) 2020-08-20 13:12:16.395 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {auth=100} crc {auth=221313478}) 2020-08-20 13:12:16.401 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {auth=15,config=2,health=10,logm=73} crc {auth=2119885989,config=3307175017,health=67914304,logm=3854202346}) 2020-08-20 13:12:16.404 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {logm=100} crc {logm=3116621380}) 2020-08-20 13:12:16.408 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {logm=100} crc {logm=767596958}) 2020-08-20 13:12:16.411 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {logm=100} crc {logm=3982727178}) 2020-08-20 13:12:16.414 7fe89becc700 0 log_channel(cluster) log [DBG] : scrub ok on 0,1,2,3: ScrubResult(keys {logm=100} crc {logm=4144183080}) after 900 seconds, mon mark osd down 2020-08-20 13:23:32.546 7fe89e6d1700 0 log_channel(cluster) log [INF] : osd.112 marked down after no beacon for 904.106586 seconds 2020-08-20 13:23:32.546 7fe89e6d1700 -1 mon.ceph-mon-1(a)0(leader).osd e2960665 no beacon from osd.112 since 2020-08-20 13:08:28.441052, 904.106586 seconds ago. marking down 2020-08-20 13:23:32.551 7fe89e6d1700 0 log_channel(cluster) log [WRN] : Health check failed: 1 osds down (OSD_DOWN) 2020-08-20 13:24:07.899 7fe89e6d1700 0 log_channel(cluster) log [INF] : osd.263 marked down after no beacon for 901.445447 seconds 2020-08-20 13:24:07.899 7fe89e6d1700 -1 mon.ceph-mon-1(a)0(leader).osd e2960666 no beacon from osd.263 since 2020-08-20 13:09:06.454891, 901.445447 seconds ago. marking down 2020-08-20 13:24:07.902 7fe89e6d1700 0 log_channel(cluster) log [WRN] : Health check update: 2 osds down (OSD_DOWN) 2020-08-20 13:24:13.020 7fe89e6d1700 0 log_channel(cluster) log [INF] : osd.384 marked down after no beacon for 900.132560 seconds 2020-08-20 13:24:13.020 7fe89e6d1700 -1 mon.ceph-mon-1(a)0(leader).osd e2960667 no beacon from osd.384 since 2020-08-20 13:09:12.888844, 900.132560 seconds ago. marking down 2020-08-20 13:24:13.020 7fe89e6d1700 0 log_channel(cluster) log [INF] : osd.614 marked down after no beacon for 901.359447 seconds 2020-08-20 13:24:13.020 7fe89e6d1700 -1 mon.ceph-mon-1(a)0(leader).osd e2960667 no beacon from osd.614 since 2020-08-20 13:09:11.661958, 901.359447 seconds ago. marking down 2020-08-20 13:24:13.026 7fe89e6d1700 0 log_channel(cluster) log [WRN] : Health check update: 4 osds down (OSD_DOWN) 2020-08-20 13:24:18.084 7fe89e6d1700 0 log_channel(cluster) log [INF] : osd.34 marked down after no beacon for 903.818250 seconds is this bug of ceph mon. Thanks.

1 0

cephadm not working with non-root user
by Amudhan P 20 Aug '20

20 Aug '20

Hi, I am trying to install ceph 'octopus' using cephadm. In bootstrap command, I have specified a non-root user account as ssh-user. cephadm bootstrap --mon-ip xx.xxx.xx.xx --ssh-user non-rootuser when bootstrap about to complete it threw an error stating. """" INFO:cephadm:Non-zero exit code 2 from /usr/bin/podman run --rm --net=host --ipc=host -e CONTAINER_IMAGE=docker.io/ceph/ceph:v15 -e NODE_NAME=node1 - v /var/log/ceph/ae4ed114-e145-11ea-9c1f-0025900a8ebe:/var/log/ceph:z -v /tmp/ceph-tmpm22k9j9w:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpe 1ltigk8:/etc/ceph/ceph.conf:z --entrypoint /usr/bin/ceph docker.io/ceph/ceph:v15 orch host add node1 INFO:cephadm:/usr/bin/ceph:stderr Error ENOENT: Failed to connect to node1 (node1). INFO:cephadm:/usr/bin/ceph:stderr Check that the host is reachable and accepts connections using the cephadm SSH key INFO:cephadm:/usr/bin/ceph:stderr INFO:cephadm:/usr/bin/ceph:stderr you may want to run: INFO:cephadm:/usr/bin/ceph:stderr > ceph cephadm get-ssh-config > ssh_config INFO:cephadm:/usr/bin/ceph:stderr > ceph config-key get mgr/cephadm/ssh_identity_key > key INFO:cephadm:/usr/bin/ceph:stderr > ssh -F ssh_config -i key root@node1 """"" In the above steps, it's trying to connect as root to the node and when I downloaded ssh_config file it was also specified as 'root' inside. so, I modified the config file and uploaded but same to ceph but still ssh to node1 is not working. To confirm if I have used the right command been used during bootstrap. I have tried the below command. " ceph config-key dump mgr/cephadm/ssh_user" { "mgr/cephadm/ssh_user": "non-rootuser" } and the output shows the user I have used during bootstrap "non-rootuser" but at the same time when I run cmd " ceph cephadm get-user " the output still shows 'root' as the user. Why the change is not affecting? do anyone faced a similar issue in bootstrap? Is there any way to avoid using container with cephadm? regards Amudhan

1 1

Ceph Snapshot Children not exists / children relation broken
by Torsten Ennenbach 20 Aug '20

20 Aug '20

Hello all, I`ve a problem with an undeletable Image. Let my try to explain. There is a storage with a Snapshot, and the snapshot thinks he has a Children: rbd snap unprotect delete-me-please@995cc2e3-c636-4c43-87c3-dbc729173c09 2020-07-30 09:25:08.492 7f15f77fe700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [f907bc6b8b4567] in pool 'rbd' 2020-07-30 09:25:08.492 7f15f77fe700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy 2020-07-30 09:25:08.492 7f15f77fe700 -1 librbd::SnapshotUnprotectRequest: 0x559ca6dd7b80 should_complete_error: ret_val=-16 2020-07-30 09:25:08.496 7f15f77fe700 -1 librbd::SnapshotUnprotectRequest: 0x559ca6dd7b80 should_complete_error: ret_val=-16rbd: unprotecting snap failed: (16) Device or resource busy rbd children delete-me-please@995cc2e3-c636-4c43-87c3-dbc729173c09 2020-07-30 09:19:36.650 7f39563deb00 -1 librbd::api::Image: list_descendants: error looking up name for image id f907bc6b8b4567 in pool rbd rbd: listing children failed: (2) No such file or directory So i am not able to unprotect and delete the Snapshot. Is there a way to fix this issue? I know this happens if deep-flatten is not enabled, and this is the root cause I think. We`ve activated this feature now but this s artifact from a time where it was disabled. Best regards & thx for your help. Torsten

3 8

CEPH FS is always showing the status as creating
by Alokkumar Mahajan 20 Aug '20

20 Aug '20

Hello, We have created CEPH FS but it is always showing the status as creating. ceph fs get returns below output:- =========================================== Filesystem 'cephfs' (4) fs_name cephfs epoch 2865929 flags 12 created 2020-08-07 05:05:58.033824 modified 2020-08-14 03:15:49.727680 tableserver 0 root 0 session_timeout 60 session_autoclose 300 max_file_size 1099511627776 min_compat_client -1 (unspecified) last_failure 0 last_failure_osd_epoch 652 compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} max_mds 1 in 0 up {0=1494099} failed damaged stopped data_pools [32] metadata_pool 33 inline_data disabled balancer standby_count_wanted 0 1494099: 10.18.97.47:6800/2041780514 <http://10.128.97.247:6800/2041780514> 'vl-pun-qa' mds.0.2748022 up:creating seq 149289 ==================================================== We are CEPH 13.2.6 MIMIC Version. I am new to CEPH so i am really not sure where to start checking this, any help will be greatly appreciated. Thanks, -alok

3 3

When do I need to update my GPS maps?
by jacknick1996＠gmail.com 20 Aug '20

20 Aug '20

I am a traveler and using GPS technology for finding the unknown locations or places. I am using this technology from the last three months, so I cannot say that I have complete knowledge about Garmin GPS technology. I know that I should update my Garmin GPS unit on a regular basis. I don’t know when I need to update my Garmin GPS unit. Due to insufficient knowledge, I am not able to update Garmin GPS unit. Really, it is a very difficult job for me for Garmin updates. Therefore anyone can recommend the best ways to update my Garmin GPS maps. Read More:- https://www.garmingpsmap-update.com/

1 0

When do I need to update my GPS maps?
by jacknick1996＠gmail.com 20 Aug '20

20 Aug '20

I am a traveler and using GPS technology for finding the unknown locations or places. I am using this technology from the last three months, so I cannot say that I have complete knowledge about Garmin GPS technology. I know that I should update my Garmin GPS unit on a regular basis. I don’t know when I need to update my Garmin GPS unit. Due to insufficient knowledge, I am not able to update Garmin GPS unit. Really, it is a very difficult job for me for Garmin updates. Therefore anyone can recommend the best ways to update my Garmin GPS maps. Read More:-https://www.garmingpsmap-update.com/

1 0

When do I need to update my GPS maps?
by jacknick1996＠gmail.com 20 Aug '20

20 Aug '20

I am a traveler and using GPS technology for finding the unknown locations or places. I am using this technology from the last three months, so I cannot say that I have complete knowledge about Garmin GPS technology. I know that I should update my Garmin GPS unit on a regular basis. I don’t know when I need to update my Garmin GPS unit. Due to insufficient knowledge, I am not able to update Garmin GPS unit. Really, it is a very difficult job for me for [url=https://www.garmingpsmap-update.com/]Garmin updates[/url]. Therefore anyone can recommend the best ways to update my Garmin GPS maps.

1 0

Re: RGW Lifecycle Processing and Promote Master Process
by Casey Bodley 19 Aug '20

19 Aug '20

On Fri, Aug 14, 2020 at 9:25 AM Alex Hussein-Kershaw <Alex.Hussein-Kershaw(a)metaswitch.com> wrote: > > Hi, > > I've previously discussed some issues I've had with the RGW lifecycle processing. I've discovered that the root cause of my problem is that: > > * I'm running a multisite configuration > * Life cycle processing is done on the master site each night. `radosgw-admin lc list` correctly returns all buckets with lc config. > * I simulate the master site being destroyed from my VM host. > * I promote the secondary site to master following the instructions here: https://docs.ceph.com/docs/master/radosgw/multisite/ > * The new master site isn't doing any lifecycle processing. `radosgw-admin lc list` returns empty. > * I recreate a cluster and pair it with the new master site to get back to having multisite redundancy. > * Neither site is doing any lifecycle processing. `radosgw-admin lc list` returns empty. > So in the process of failover/recovery I have gone from having two paired clusters performing lifecycle processing, to two paired clusters NOT performing lifecycle processing. > > Is this behaviour expected? I've found `radosgw-admin lc reshard fix` will "remind" the cluster that I run it on that it needs to do lifecycle processing. Although I found no mention of having to use this in the docs, for that command the docs state it's only relevant on earlier Ceph versions. I'm running Nautilus 14.2.9. > > In addition, if I have two healthy clusters paired in a multisite system, and swap the master cluster by promoting the non-master, the demoted cluster seems to still continue doing lifecycle processing, while the promote does not. If I run `radosgw-admin lc reshard fix` on the promoted cluster, then both clusters seem to claim they are doing the processing. Is this a happy state to be in? > > Does anyone have any experience with this? > > Thanks, > Alex > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > There's a defect in metadata sync (https://tracker.ceph.com/issues/44268) which prevents buckets with lifecycle policies from being indexed for lifecycle processing on non-master zones. It sounds like the 'lc reshard fix' command is adding it back to that index for processing. The intent is for lifecycle processing to occur independently on every zone. That's the only way to guarantee the correct result now that we have PutBucketReplication (and specifically the Filter policy) where any given zone may only hold a subset of the objects from its source.

2 1