- ceph-users - lists.ceph.io

Re: Ceph survey results
by Satoru Takeuchi 06 Jul '21

06 Jul '21

Hi, 2021年7月4日(日) 0:17 changcheng.liu <changcheng.liu(a)aliyun.com>: > > Hi all, > I'm reading the ceph survey results: https://ceph.io/community/2021-ceph-user-survey-results. > Do we have the data about which type of AsyncMessenger is used? TCP/RDMA/DPDK. > What's the reason that RDMA & DPDK isn't often used? > You can find both the report and raw data below. > > 2021 Ceph User Survey Results > 2021 Ceph User Survey Raw Data The above two links are 404. Could you fix them? Thanks, Satoru > > B.R. > Jerry > > > _______________________________________________ > Dev mailing list -- dev(a)ceph.io > To unsubscribe send an email to dev-leave(a)ceph.io

1 0

Re: XFS on RBD on EC painfully slow
by Sebastian Knust 05 Jul '21

05 Jul '21

Hi Reed, To add to this command by Weiwen: On 28.05.21 13:03, 胡玮文 wrote: > Have you tried just start multiple rsync process simultaneously to transfer different directories? Distributed system like ceph often benefits from more parallelism. When I migrated from XFS on iSCSI (legacy system, no Ceph) to CephFS a few months ago, I used msrsync [1] and was quite happy with the speed. For your use case, I would start with -p 12 but might experiment with up to -p 24 (as you only have 6C/12T in your CPU). With many small files, you also might want to increase -s from the default 1000. Note that msrsync does not work with the --delete rsync flag. As I was syncing a live system, I ended up with this workflow: - Initial sync with msrsync (something like ./msrsync -p 12 --progress --stats --rsync "-aS --numeric-ids" ...) - Second sync with msrsync (to sync changes during the first sync) - Take old storage off-line for users / read-only - Final rsync with --delete (i.e. rsync -aS --numeric-ids --delete ...) - Mount cephfs at location of old storage, adjust /etc/exports with fsid entries where necessary, turn system back on-line / read-write Cheers Sebastian [1] https://github.com/jbd/msrsync

3 2

Graphics in ceph dashboard
by Fyodor Ustinov 05 Jul '21

05 Jul '21

Hi! I installed fresh cluster 16.2.4 as described in https://docs.ceph.com/en/latest/cephadm/#cephadm Everything works except for one thing: there are only graphics in the hosts / overall performance, (only the CPU and the network). In all other places the inscription "no data". What could I have done wrong? WBR, Fyodor.

2 2

Haproxy config, multilple RGW on the same node with different ports haproxy ignore
by Szabo, Istvan (Agoda) 05 Jul '21

05 Jul '21

Hi, I have this config: https://jpst.it/2yBsD What I'm missing from the backend part to make it able to balance on the same server but different port? Thank you ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

1 0

cephadm shell fails to start due to missing config files?
by Vladimir Brik 05 Jul '21

05 Jul '21

Hello I am getting an error on one node in my cluster (other nodes are fine) when trying to run "cephadm shell". Historically this machine has been used as the primary Ceph management host, so it would be nice if this could be fixed. ceph-1 ~ # cephadm -v shell container_init=False Inferring fsid 79656e6e-21e2-4092-ac04-d536f25a435d Inferring config /var/lib/ceph/79656e6e-21e2-4092-ac04-d536f25a435d/mon.ceph-1/config Running command: /usr/bin/podman images --filter label=ceph=True --filter dangling=false --format {{.Repository}}(a){{.Digest}} /usr/bin/podman: stdout docker.io/ceph/daemon-base@sha256:0810dc7db854150bc48cf8fc079875e28b3138d070990a630b8fb7cec7cd2ced /usr/bin/podman: stdout docker.io/ceph/ceph@sha256:54e95ae1e11404157d7b329d0bef866ebbb214b195a009e87aae4eba9d282949 /usr/bin/podman: stdout docker.io/ceph/ceph@sha256:16d37584df43bd6545d16e5aeba527de7d6ac3da3ca7b882384839d2d86acc7d Using recent ceph image docker.io/ceph/daemon-base@sha256:0810dc7db854150bc48cf8fc079875e28b3138d070990a630b8fb7cec7cd2ced Running command: /usr/bin/podman run --rm --ipc=host --net=host --entrypoint stat -e CONTAINER_IMAGE=docker.io/ceph/daemon-base@sha256:0810dc7db854150bc48cf8fc079875e28b3138d070990a630b8fb7cec7cd2ced -e NODE_NAME=ceph-1 docker.io/ceph/daemon-base@sha256:0810dc7db854150bc48cf8fc079875e28b3138d070990a630b8fb7cec7cd2ced -c %u %g /var/lib/ceph stat: stdout 167 167 Running command (timeout=None): /usr/bin/podman run --rm --ipc=host --net=host --privileged --group-add=disk -it -e LANG=C -e PS1=[ceph: \u@\h \W]\$ -e CONTAINER_IMAGE=docker.io/ceph/daemon-base@sha256:0810dc7db854150bc48cf8fc079875e28b3138d070990a630b8fb7cec7cd2ced -e NODE_NAME=ceph-1 -v /var/run/ceph/79656e6e-21e2-4092-ac04-d536f25a435d:/var/run/ceph:z -v /var/log/ceph/79656e6e-21e2-4092-ac04-d536f25a435d:/var/log/ceph:z -v /var/lib/ceph/79656e6e-21e2-4092-ac04-d536f25a435d/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/79656e6e-21e2-4092-ac04-d536f25a435d/mon.ceph-1/config:/etc/ceph/ceph.conf:z -v /etc/ceph/ceph.client.admin.keyring:/etc/ceph/ceph.keyring:z -v /var/lib/ceph/79656e6e-21e2-4092-ac04-d536f25a435d/home:/root --entrypoint bash docker.io/ceph/daemon-base@sha256:0810dc7db854150bc48cf8fc079875e28b3138d070990a630b8fb7cec7cd2ced Error: error checking path "/var/lib/ceph/79656e6e-21e2-4092-ac04-d536f25a435d/mon.ceph-1/config": stat /var/lib/ceph/79656e6e-21e2-4092-ac04-d536f25a435d/mon.ceph-1/config: no such file or directory The machine in question doesn't run a mon daemon (but it did a long time ago), so I am not sure why "cephadm shell" on this particular machine is looking for mon.ceph-1/config Can anybody help? Thanks, Vlad

2 1

Objectstore user IO and operations monitoring
by Szabo, Istvan (Agoda) 05 Jul '21

05 Jul '21

Hi, I'm looking for this long time ago, I have a lot of users and when 1 user can take down the cluster I want to know which one, but there isn't any bucket stats that could help. Anyone knows anything? Istvan Szabo Senior Infrastructure Engineer --------------------------------------------------- Agoda Services Co., Ltd. e: istvan.szabo(a)agoda.com<mailto:istvan.szabo@agoda.com> --------------------------------------------------- ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

1 0

Remove objectstore from a RBD RGW cluster
by Szabo, Istvan (Agoda) 05 Jul '21

05 Jul '21

Hi, I want to remove all the objectstore related things from my cluster and keep only for RBD. I've uninstalled the RGW services. Removed the haproxy config related to that. When I try to delete realm, zone, zonegroup it is finished but after coupe of minutes something recreate another zonegroup. I can't figure out what. What I miss? PS: The pools are still there, that will be the last step, hope I don't miss any necessary step. Istvan Szabo Senior Infrastructure Engineer --------------------------------------------------- Agoda Services Co., Ltd. e: istvan.szabo(a)agoda.com<mailto:istvan.szabo@agoda.com> --------------------------------------------------- ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

2 1

how to compare setting differences between two rbd images
by Philip Brown 01 Jul '21

01 Jul '21

I have two rbd images. running fio on them, one performs very well, and one performs poorly. I'd like to get more insight on why. I know that rbd info xx/yyy gives me a small amount of information. but I cant seem to find a detailed info dump, in the way you can do things like ceph daemon osd.9 config show I cant even get useful google hits on things like "ceph find rbd image stripe-unit" all the pages seem to detail how to set the value, but not how to query it. Can anyone point me in the right direction? -- Philip Brown| Sr. Linux System Administrator | Medata, Inc. 5 Peters Canyon Rd Suite 250 Irvine CA 92606 Office 714.918.1310| Fax 714.918.1325 pbrown(a)medata.com| www.medata.com

1 0

rbd: map failed: rbd: sysfs write failed -- (108) Cannot send after transport endpoint shutdown
by Oliver Dzombic 01 Jul '21

01 Jul '21

Hi, mapping of rbd volumes fails clusterwide. The volumes that are mapped, are ok, but new volumes wont map. Receiving errors liks: (108) Cannot send after transport endpoint shutdown or executing: #rbd -p CEPH map vm-120-disk-0 will show: rbd: sysfs write failed In some cases useful info is found in syslog - try "dmesg | tail". rbd: map failed: (108) Cannot send after transport endpoint shutdown I added some more debugging for the logs and got this: rbd: sysfs write failed 2021-07-01T10:28:44.258+0200 7f803c4a33c0 5 librbd::io::ImageRequestWQ: 0x7f802c0011f0 ImageRequestWQ: ictx=0x55846e936c40 2021-07-01T10:28:44.258+0200 7f803c4a33c0 20 librbd::ImageState: 0x7f802c001420 open 2021-07-01T10:28:44.258+0200 7f803c4a33c0 10 librbd::ImageState: 0x7f802c001420 0x7f802c001420 send_open_unlock 2021-07-01T10:28:44.258+0200 7f803c4a33c0 10 librbd::image::OpenRequest: 0x55846e979620 send_v2_detect_header 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: handle_v2_detect_header: r=0 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 send_v2_get_id 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: handle_v2_get_id: r=0 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 send_v2_get_initial_metadata 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: handle_v2_get_initial_metadata: r=0 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 send_v2_get_create_timestamp 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 handle_v2_get_create_timestamp: r=0 2021-07-01T10:28:44.262+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 send_v2_get_access_modify_timestamp 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 handle_v2_get_access_modify_timestamp: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 send_v2_get_data_pool 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 handle_v2_get_data_pool: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::ImageCtx: init_layout stripe_unit 4194304 stripe_count 1 object_size 4194304 prefix rbd_data.21f6f3a7ec4663 format rbd_data.21f6f3a7ec4663.%016llx 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::OpenRequest: 0x55846e979620 send_refresh 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::ConfigWatcher: init: 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 send_v2_get_mutable_metadata 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 handle_v2_get_mutable_metadata: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 send_v2_get_parent: legacy=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 handle_v2_get_parent: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 send_v2_get_metadata 2021-07-01T10:28:44.266+0200 7f80297fa700 15 librbd::image::GetMetadataRequest: 0x7f80100126a0 metadata_list: start_key=conf_ 2021-07-01T10:28:44.266+0200 7f80297fa700 15 librbd::image::GetMetadataRequest: 0x7f80100126a0 handle_metadata_list: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 15 librbd::image::GetMetadataRequest: 0x7f80100126a0 finish: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 handle_v2_get_metadata: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 send_v2_get_pool_metadata 2021-07-01T10:28:44.266+0200 7f80297fa700 15 librbd::image::GetMetadataRequest: 0x7f8010012c10 metadata_list: start_key=conf_ 2021-07-01T10:28:44.266+0200 7f80297fa700 15 librbd::image::GetMetadataRequest: 0x7f8010012c10 handle_metadata_list: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 15 librbd::image::GetMetadataRequest: 0x7f8010012c10 finish: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 handle_v2_get_pool_metadata: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 20 librbd::ImageCtx: apply_metadata 2021-07-01T10:28:44.266+0200 7f80297fa700 5 librbd::ImageCtx: 0x55846e936c40: disabling zero-copy writes 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 send_v2_get_group 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 handle_v2_get_group: r=0 2021-07-01T10:28:44.266+0200 7f80297fa700 10 librbd::image::RefreshRequest: 0x7f8010011870 send_v2_apply 2021-07-01T10:28:44.266+0200 7f8028ff9700 10 librbd::image::RefreshRequest: 0x7f8010011870 handle_v2_apply 2021-07-01T10:28:44.266+0200 7f8028ff9700 20 librbd::image::RefreshRequest: 0x7f8010011870 apply 2021-07-01T10:28:44.266+0200 7f8028ff9700 10 librbd::image::OpenRequest: handle_refresh: r=0 2021-07-01T10:28:44.266+0200 7f8028ff9700 10 librbd::image::OpenRequest: send_parent_cache: r=0 2021-07-01T10:28:44.266+0200 7f8028ff9700 10 librbd::image::OpenRequest: 0x55846e979620 send_init_cache 2021-07-01T10:28:44.266+0200 7f8028ff9700 5 librbd::cache::WriteAroundObjectDispatch: 0x7f80180020d0 init: 2021-07-01T10:28:44.266+0200 7f8028ff9700 5 librbd::io::ObjectDispatcher: 0x55846e979670 register_object_dispatch: object_dispatch_layer=1 2021-07-01T10:28:44.266+0200 7f8028ff9700 10 librbd::ImageState: 0x7f802c001420 0x7f802c001420 handle_open: r=0 In some cases useful info is found in syslog - try "dmesg | tail". 2021-07-01T10:28:44.266+0200 7f803c4a33c0 20 librbd::ImageState: 0x7f802c001420 close 2021-07-01T10:28:44.266+0200 7f803c4a33c0 10 librbd::ImageState: 0x7f802c001420 0x7f802c001420 send_close_unlock 2021-07-01T10:28:44.266+0200 7f803c4a33c0 10 librbd::ConfigWatcher: shut_down: 2021-07-01T10:28:44.266+0200 7f803c4a33c0 10 librbd::image::CloseRequest: 0x55846e97a510 send_block_image_watcher 2021-07-01T10:28:44.270+0200 7f803c4a33c0 10 librbd::ImageWatcher: 0x7f8010003e00 block_notifies 2021-07-01T10:28:44.270+0200 7f803c4a33c0 5 librbd::Watcher: 0x7f8010003e00 block_notifies: blocked_count=1 2021-07-01T10:28:44.270+0200 7f803c4a33c0 10 librbd::image::CloseRequest: 0x55846e97a510 handle_block_image_watcher: r=0 2021-07-01T10:28:44.270+0200 7f803c4a33c0 10 librbd::image::CloseRequest: 0x55846e97a510 send_shut_down_update_watchers 2021-07-01T10:28:44.270+0200 7f803c4a33c0 20 librbd::ImageState: 0x7f802c001420 shut_down_update_watchers 2021-07-01T10:28:44.270+0200 7f803c4a33c0 20 librbd::ImageState: 0x55846e80a230 ImageUpdateWatchers::shut_down 2021-07-01T10:28:44.270+0200 7f803c4a33c0 20 librbd::ImageState: 0x55846e80a230 ImageUpdateWatchers::shut_down: completing shut down 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_shut_down_update_watchers: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 send_shut_down_io_queue 2021-07-01T10:28:44.270+0200 7f8028ff9700 5 librbd::io::ImageRequestWQ: 0x7f802c0011f0 shut_down: shut_down: in_flight=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AsyncOperation: 0x7f80180024c8 start_op 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::ImageRequest: 0x7f8028ff3d30 send: aio_flush: ictx=0x55846e936c40, completion=0x7f80180023d0 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AioCompletion: 0x7f80180023d0 set_request_count: pending=1 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::ObjectDispatcher: 0x55846e979670 send: object_dispatch_spec=0x7f8018002520 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::cache::WriteAroundObjectDispatch: 0x7f80180020d0 flush: 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AioCompletion: 0x7f80180023d0 complete_request: cb=1, pending=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AioCompletion: 0x7f80180023d0 finalize: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_shut_down_io_queue: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 send_flush 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AsyncOperation: 0x7f8018002718 start_op 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::ImageRequest: 0x7f8028ff3c70 send: aio_flush: ictx=0x55846e936c40, completion=0x7f8018002620 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AioCompletion: 0x7f8018002620 set_request_count: pending=1 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AsyncOperation: 0x7f80180024c8 finish_op 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AsyncOperation: completed flush: 0x7f801000e6d0 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::ObjectDispatcher: 0x55846e979670 send: object_dispatch_spec=0x7f8018002520 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::cache::WriteAroundObjectDispatch: 0x7f80180020d0 flush: 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AioCompletion: 0x7f8018002620 complete_request: cb=1, pending=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AioCompletion: 0x7f8018002620 finalize: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_flush: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 send_unregister_image_watcher 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::ImageWatcher: 0x7f8010003e00 unregistering image watcher 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::Watcher: 0x7f8010003e00 unregister_watch: 2021-07-01T10:28:44.270+0200 7f8028ff9700 20 librbd::io::AsyncOperation: 0x7f8018002718 finish_op 2021-07-01T10:28:44.270+0200 7f8022ffd700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_unregister_image_watcher: r=0 2021-07-01T10:28:44.270+0200 7f8022ffd700 10 librbd::image::CloseRequest: 0x55846e97a510 send_flush_readahead 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_flush_readahead: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 send_shut_down_object_dispatcher 2021-07-01T10:28:44.270+0200 7f8028ff9700 5 librbd::io::ObjectDispatcher: 0x55846e979670 shut_down: 2021-07-01T10:28:44.270+0200 7f8028ff9700 5 librbd::io::ObjectDispatch: 0x55846e97b430 shut_down: 2021-07-01T10:28:44.270+0200 7f8028ff9700 5 librbd::cache::WriteAroundObjectDispatch: 0x7f80180020d0 shut_down: 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_shut_down_object_dispatcher: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 send_flush_op_work_queue 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_flush_op_work_queue: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::image::CloseRequest: 0x55846e97a510 handle_flush_image_watcher: r=0 2021-07-01T10:28:44.270+0200 7f8028ff9700 10 librbd::ImageState: 0x7f802c001420 0x7f802c001420 handle_close: r=0 rbd: map failed: (108) Cannot send after transport endpoint shutdown ... which does actually not tell me too much honestly. For now, i am out of idea's, can someone lead a way to analyse / fix this ? New volumes can be created and removed without issues. But they wont map. Thank you! -- Mit freundlichen Gruessen / Best regards Oliver Dzombic Layer7 Networks mailto:info@layer7.net Anschrift: Layer7 Networks GmbH Zum Sonnenberg 1-3 63571 Gelnhausen HRB 96293 beim Amtsgericht Hanau Geschäftsführung: Oliver Dzombic UST ID: DE259845632

2 1

configure fuse in fstab
by Simon Sutter 01 Jul '21

01 Jul '21

Hello Everyone! I'm trying to mount the ceph, with the fuse client under debian 9 (ceph-fuse 10.2.11-2). Ceph is on the latest Octopus release. The direct command is working, but writing it in fstab does not. Command I use: ceph-fuse --id dev.wsc -k /etc/ceph/ceph.clinet.dev.wsc.keyring -r /testing/dev.wsc/userdata /mnt/ceph /etc/ceph.conf: [global] fsid = [fsid] mon_host = 10.99.15.1 10.99.15.2 10.99.15.3 10.99.15.4 10.99.15.5 /etc/ceph/ceph.clinet.dev.wsc.keyring: [client.dev.wsc] key = [key] fstab: none /mnt/ceph fuse.ceph ceph.id=dev.wsc,ceph.conf=/etc/ceph/ceph.conf,ceph.client_mountpoint=/testing/dev.wsc/userdata,_netdev,defaults 0 0 With mount -a it tells me "ceph mount failed with (1) Operation not permitted". Where can I find authentication logs on the cluster? Thanks in advance and kind regards, Simon

2 2