On a Pacific cluster I have the same error message: ---snip--- 023-08-25T11:56:47.222407+02:00 ses7-host1 conmon[1383161]: debug (LUN.add_dev_to_lio) Adding image 'iscsi-pool/image3' to LIO backstore rbd 2023-08-25T11:56:47.714375+02:00 ses7-host1 kernel: [12861743.121824] rbd: rbd1: capacity 5368709120 features 0x3d 2023-08-25T11:56:47.746390+02:00 ses7-host1 conmon[1383161]: /dev/rbd1 2023-08-25T11:56:47.764072+02:00 ses7-host1 conmon[1383161]: debug failed to add iscsi-pool/image3 to LIO - error([Errno 22] Invalid argument) 2023-08-25T11:56:47.764563+02:00 ses7-host1 conmon[1383161]: debug LUN alloc problem - failed to add iscsi-pool/image3 to LIO - error([Errno 22] Invalid argument) 2023-08-25T11:56:47.766292+02:00 ses7-host1 kernel: [12861743.172626] target_core_rbd: RBD: emulate_legacy_capacity must be disabled for RBD_FEATURE_OBJECT_MAP images 2023-08-25T11:56:47.766304+02:00 ses7-host1 kernel: [12861743.172653] target_core_rbd: RBD: emulate_legacy_capacity must be disabled for RBD_FEATURE_OBJECT_MAP images 2023-08-25T11:56:47.770242+02:00 ses7-host1 conmon[1383161]: debug ::ffff:127.0.0.1 - - [25/Aug/2023 09:56:47] "#033[35m#033[1mPUT /api/_disk/iscsi-pool/image3 HTTP/1.1#033[0m" 500 - 2023-08-25T11:56:47.770640+02:00 ses7-host1 conmon[1383161]: ::ffff:127.0.0.1 - - [25/Aug/2023 09:56:47] "#033[35m#033[1mPUT /api/_disk/iscsi-pool/image3 HTTP/1.1#033[0m" 500 - 2023-08-25T11:56:47.772916+02:00 ses7-host1 conmon[1383161]: debug _disk change on localhost failed with 500 2023-08-25T11:56:47.773753+02:00 ses7-host1 conmon[1383161]: debug ::ffff:192.168.168.81 - - [25/Aug/2023 09:56:47] "#033[35m#033[1mPUT /api/disk/iscsi-pool/image3 HTTP/1.1#033[0m" 500 - 2023-08-25T11:56:47.776186+02:00 ses7-host1 conmon[1741104]: iscsi REST API failed PUT req status: 500 2023-08-25T11:56:47.777479+02:00 ses7-host1 conmon[1741104]: Error while calling Task(ns=iscsi/target/edit, md={'target_iqn': 'iqn.2001-07.com.ceph:1692955254223'}) 2023-08-25T11:56:47.777689+02:00 ses7-host1 conmon[1741104]: Traceback (most recent call last): 2023-08-25T11:56:47.777900+02:00 ses7-host1 conmon[1741104]: File "/usr/share/ceph/mgr/dashboard/controllers/iscsi.py", line 789, in _create 2023-08-25T11:56:47.778333+02:00 ses7-host1 conmon[1741104]: controls=controls) 2023-08-25T11:56:47.778725+02:00 ses7-host1 conmon[1741104]: File "/usr/share/ceph/mgr/dashboard/rest_client.py", line 535, in func_wrapper 2023-08-25T11:56:47.778879+02:00 ses7-host1 conmon[1741104]: **kwargs) 2023-08-25T11:56:47.779103+02:00 ses7-host1 conmon[1741104]: File "/usr/share/ceph/mgr/dashboard/services/iscsi_client.py", line 126, in create_disk 2023-08-25T11:56:47.779264+02:00 ses7-host1 conmon[1741104]: 'wwn': wwn 2023-08-25T11:56:47.779412+02:00 ses7-host1 conmon[1741104]: File "/usr/share/ceph/mgr/dashboard/rest_client.py", line 324, in __call__ 2023-08-25T11:56:47.779549+02:00 ses7-host1 conmon[1741104]: data, raw_content, headers) 2023-08-25T11:56:47.779715+02:00 ses7-host1 conmon[1741104]: File "/usr/share/ceph/mgr/dashboard/rest_client.py", line 453, in do_request 2023-08-25T11:56:47.779861+02:00 ses7-host1 conmon[1741104]: resp.content) 2023-08-25T11:56:47.780025+02:00 ses7-host1 conmon[1741104]: dashboard.rest_client.RequestException: iscsi REST API failed request with status code 500 2023-08-25T11:56:47.780189+02:00 ses7-host1 conmon[1741104]: (b'{"message":"disk create/update failed on ses7-host1. LUN all' 2023-08-25T11:56:47.780332+02:00 ses7-host1 conmon[1741104]: b'ocation failure"}\n') ---snip--- I'm wondering why on Pacific I have the rbd image mapped while on Reef I don't. But isn't iscsi-gw deprecated [1]?
The iSCSI gateway is in maintenance as of November 2022. This means that it is no longer in active development and will not be updated to add new features.
Not sure if it makes sense to dig deeper here... [1] https://docs.ceph.com/en/quincy/rbd/iscsi-overview/ Zitat von Eugen Block <eblock@nde.ag>:
Hi,
that's quite interesting, I tried to reproduce with 18.2.0 but it worked for me. The cluster runs on openSUSE Leap 15.4. There are two things that seem to differ in my attempt. 1. I had to run 'modprobe iscsi_target_mod' to get rid of this error message:
(b'{"message":"iscsi target \'init\' process failed for iqn.2001-07.com.ceph:' b'1692952282206 - Could not load module: iscsi_target_mod"}\n')
2. I don't have tcmu runner up which is reported every 10 seconds but it seems to work anyway:
debug there is no tcmu-runner data available
In a Pacific test cluster I do see that a tcmu runner is deployed alongside the iscsi gw. Anyway, this is my output:
---snip--- [...] o- disks ......................................................................................................... [10G, Disks: 1] | o- test-pool ................................................................................................. [test-pool (10G)] | o- image1 .................................................................................. [test-pool/image1 (Unknown, 10G)] o- iscsi-targets ............................................................................... [DiscoveryAuth: None, Targets: 1] o- iqn.2001-07.com.ceph:1692952282206 ................................................................ [Auth: None, Gateways: 1] o- disks .......................................................................................................... [Disks: 1] | o- test-pool/image1 ............................................................. [Owner: reef01, Lun: 0] o- gateways ............................................................................................ [Up: 1/1, Portals: 1] | o- reef01 ........................................................................ [>IP> (UP)] o- host-groups .................................................................................................. [Groups : 0] o- hosts ...................................................................................... [Auth: ACL_DISABLED, Hosts: 0] ---snip---
This is my exact ceph version:
"ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3
Zitat von Opánszki Gábor <gabor.opanszki@acetelecom.hu>:
Hi folks,
we deployed new reef cluster to our lab.
all of the nodes are up and running, but we can't allocate lun to target.
on the gui we got "disk create/update failed on ceph-iscsigw0. LUN allocation failure" message.
We created images on gui
do you have any idea?
Thanks
root@ceph-mgr0:~# ceph -s *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** cluster: id: ad0aede2-4100-11ee-bc14-1c40244f5c21 health: HEALTH_OK
services: mon: 5 daemons, quorum ceph-mgr0,ceph-mgr1,ceph-osd5,ceph-osd7,ceph-osd6 (age 28h) mgr: ceph-mgr0.sapbav(active, since 45h), standbys: ceph-mgr1.zwzyuc osd: 44 osds: 44 up (since 4h), 44 in (since 4h) tcmu-runner: 1 portal active (1 hosts)
data: pools: 5 pools, 3074 pgs objects: 27 objects, 453 KiB usage: 30 GiB used, 101 TiB / 101 TiB avail pgs: 3074 active+clean
io: client: 2.7 KiB/s rd, 2 op/s rd, 0 op/s wr
root@ceph-mgr0:~#
root@ceph-mgr0:~# rados lspools .mgr ace1 1T-r3-01 ace0 x root@ceph-mgr0:~# rbd ls 1T-r3-01 111 aaaa bb pool2 teszt root@ceph-mgr0:~# rbd ls x x-a root@ceph-mgr0:~#
root@ceph-mgr0:~# rbd info 1T-r3-01/111 rbd image '111': size 1 GiB in 256 objects order 22 (4 MiB objects) snapshot_count: 0 id: 5f927ce161de block_name_prefix: rbd_data.5f927ce161de format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten op_features: flags: create_timestamp: Thu Aug 24 17:33:37 2023 access_timestamp: Thu Aug 24 17:33:37 2023 modify_timestamp: Thu Aug 24 17:33:37 2023 root@ceph-mgr0:~# rbd info 1T-r3-01/aaaa rbd image 'aaaa': size 1 GiB in 256 objects order 22 (4 MiB objects) snapshot_count: 0 id: 5f926a0e299f block_name_prefix: rbd_data.5f926a0e299f format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten op_features: flags: create_timestamp: Thu Aug 24 17:18:06 2023 access_timestamp: Thu Aug 24 17:18:06 2023 modify_timestamp: Thu Aug 24 17:18:06 2023 root@ceph-mgr0:~# rbd info x/x-a rbd image 'x-a': size 1 GiB in 256 objects order 22 (4 MiB objects) snapshot_count: 0 id: 5f922dbdf6c6 block_name_prefix: rbd_data.5f922dbdf6c6 format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten op_features: flags: create_timestamp: Thu Aug 24 17:48:28 2023 access_timestamp: Thu Aug 24 17:48:28 2023 modify_timestamp: Thu Aug 24 17:48:28 2023 root@ceph-mgr0:~#
root@ceph-mgr0:~# ceph orch ls --service_type iscsi *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** NAME PORTS RUNNING REFRESHED AGE PLACEMENT iscsi.gw-1 ?:5000 2/2 4m ago 6m ceph-iscsigw0;ceph-iscsigw1 root@ceph-mgr0:~#
GW:
root@ceph-iscsigw0:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d677a8abd2d8 quay.io/ceph/ceph "/usr/bin/rbd-target…" 6 seconds ago Up 5 seconds ceph-ad0aede2-4100-11ee-bc14-1c40244f5c21-iscsi-gw-1-ceph-iscsigw0-fmuyhi ead503586cdd quay.io/ceph/ceph "/usr/bin/tcmu-runner" 6 seconds ago Up 5 seconds ceph-ad0aede2-4100-11ee-bc14-1c40244f5c21-iscsi-gw-1-ceph-iscsigw0-fmuyhi-tcmu 3ae0014bcc41 quay.io/ceph/ceph "/usr/bin/ceph-crash…" About an hour ago Up About an hour ceph-ad0aede2-4100-11ee-bc14-1c40244f5c21-crash-ceph-iscsigw0 1a7bc044ed8a quay.io/ceph/ceph "/usr/bin/ceph-expor…" About an hour ago Up About an hour ceph-ad0aede2-4100-11ee-bc14-1c40244f5c21-ceph-exporter-ceph-iscsigw0 c746a4da2bbb quay.io/prometheus/node-exporter:v1.5.0 "/bin/node_exporter …" About an hour ago Up About an hour ceph-ad0aede2-4100-11ee-bc14-1c40244f5c21-node-exporter-ceph-iscsigw0 root@ceph-iscsigw0:~# docker exec -it d677a8abd2d8 /bin/bash [root@ceph-iscsigw0 /]# gwcli ls o- / ......................................................................................................................... [...] o- cluster ......................................................................................................... [Clusters: 1] | o- ceph ............................................................................................................ [HEALTH_OK] | o- pools .......................................................................................................... [Pools: 5] | | o- .mgr .................................................................. [(x3), Commit: 0.00Y/33602764M (0%), Used: 3184K] | | o- 1T-r3-01 ................................................................ [(x3), Commit: 0.00Y/5793684M (0%), Used: 108K] | | o- ace0 ................................................................... [(2+1), Commit: 0.00Y/11587368M (0%), Used: 24K] | | o- ace1 ................................................................... [(2+1), Commit: 0.00Y/55665220M (0%), Used: 12K] | | o- x ....................................................................... [(x3), Commit: 0.00Y/33602764M (0%), Used: 36K] | o- topology ............................................................................................... [OSDs: 44,MONs: 5] o- disks ....................................................................................................... [0.00Y, Disks: 0] o- iscsi-targets ............................................................................... [DiscoveryAuth: None, Targets: 1] o- iqn.2001-07.com.ceph:1692892702115 ................................................................ [Auth: None, Gateways: 2] o- disks .......................................................................................................... [Disks: 0] o- gateways ............................................................................................ [Up: 2/2, Portals: 2] | o- ceph-iscsigw0 ............................................................................ [10.202.5.21,10.202.4.21 (UP)] | o- ceph-iscsigw1 ............................................................................ [10.202.3.21,10.202.2.21 (UP)] o- host-groups .................................................................................................. [Groups : 0] o- hosts ....................................................................................... [Auth: ACL_ENABLED, Hosts: 0] [root@ceph-iscsigw0 /]#
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io