On 4/28/20 2:21 AM, Simone Lazzaris wrote:
In data lunedì 27 aprile 2020 18:46:09 CEST, Mike Christie ha scritto:
[snip]
Are you using the ceph-iscsi tools with tcmu-runner or did you setup
tcmu-runner directly with targetcli?
I followed this guide: https://docs.ceph.com/docs/master//rbd/iscsi-target-cli/ and configured the target with gwcli, so I think I'm using ceph-iscsi tools.
[snip]
You would see these:
1. when paths are discovered initially. The initiator is sending IO to
all paths at the same time, so the lock is bouncing between all the paths.
Ok, but the nodes are already configured and all path discovered. So that's not the case.
You should only see this for 10-60 seconds depending on how many paths
you have, number of nodes, etc. When the multipath layer kicks in and
adds the paths to the dm-multipath device then they should stop.
I have NO such logs when the system is running unless I start USING the luns
Could you send me: 1. The /var/log/messages for the initiator when you do IO and see those lock messages. 2. The output of From one of the gateways: # gwcli ls From the initiator node you send the /var/log/messages for: # iscsiadm -m session -P 3 # multipath -ll 3. version info: # uname -a If you using rpm do: # rpm -q ceph-iscsi # rpm -q tcmu-runner # rpm -q python-rtslib
2. during failover/failback when the multipath layer switches paths and
one path takes the lock from the previously used one.
No failover/failback is occuring.
Or, if you exported a disk to multiple initiator nodes, and some
initiator nodes can't reach the active optimized path, so some
initiators are using the optimized path and some are using the
non-optimized path.
I do have exported the disk to multiple initiator nodes. How can I tell if they are using all the active path?
For linux's multipath-tools prio=50 is the active optimized (AO) path. prio=10 is the active non-optimized (ANO) path. The prio=50 one should be the one with status=active. To map that to an iscsi gateway then you can do the following. If sdb is the AO one, then run iscsiadm -m session -P 3 Here you can see the sdXYZ name to iscsi session mapping. The iscsi session/connection's target IP address from that command should match to the gateway that is listed as the "owner" of the LUN in the "gwcli ls" output.