Hi all, First of all I apologize if I've not done things correctly but these are some tests results. 1) I've compiled the main branch in a fresh podman container (Alma Linux 8) and installed. Successfull! 2) I have done a copy of the /etc/ceph directory of the host (member of the ceph cluster in Pacific 16.2.14) in this container (good or bad idea ?) 3) "ceph-volume inventory" works but with some error messages: [root@74285dcfa91f etc]# ceph-volume inventory stderr: Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected. stderr: Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected. stderr: Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected. stderr: Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected. stderr: Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected. Device Path Size Device nodes rotates available Model name /dev/sdc 232.83 GB sdc True True SAMSUNG HE253GJ /dev/sda 232.83 GB sda True False SAMSUNG HE253GJ /dev/sdb 465.76 GB sdb True False WDC WD5003ABYX-1 4) ceph version show: [root@74285dcfa91f etc]# ceph -v ceph version 18.0.0-6846-g2706ecac4a9 (2706ecac4a90447420904e42d6e0445134dff2be) reef (dev) 5) lsblk works (container launched with "--privileged" flag) [root@74285dcfa91f etc]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 1 232.9G 0 disk |-sda1 8:1 3.9G 0 part |-sda2 8:2 1 3.9G 0 part [SWAP] `-sda3 8:3 1 225G 0 part sdb 8:16 1 465.8G 0 disk sdc 8:32 1 232.9G 0 disk But some commands do not works (my setup or ceph ?) [root@74285dcfa91f etc]# ceph orch device zap mostha1.legi.grenoble-inp.fr /dev/sdc --force Error EINVAL: Device path '/dev/sdc' not found on host 'mostha1.legi.grenoble-inp.fr' [root@74285dcfa91f etc]# [root@74285dcfa91f etc]# ceph orch device ls [root@74285dcfa91f etc]# Patrick Le 24/10/2023 à 22:43, Zack Cerza a écrit :
That's correct - it's the removable flag that's causing the disks to be excluded.
I actually just merged this PR last week: https://github.com/ceph/ceph/pull/49954
One of the changes it made was to enable removable (but not USB) devices, as there are vendors that report hot-swappable drives as removable. Patrick, it looks like this may resolve your issue as well.
On Tue, Oct 24, 2023 at 5:57 AM Eugen Block <eblock@nde.ag> wrote:
Hi,
May be because they are hot-swappable hard drives. yes, that's my assumption as well.
Zitat von Patrick Begou <Patrick.Begou@univ-grenoble-alpes.fr>:
Hi Eugen,
Yes Eugen, all the devices /dev/sd[abc] have the removable flag set to 1. May be because they are hot-swappable hard drives.
I have contacted the commit author Zack Cerza and he asked me for some additional tests too this morning. I add him in copy to this mail.
Patrick
Le 24/10/2023 à 12:57, Eugen Block a écrit :
Hi,
just to confirm, could you check that the disk which is *not* discovered by 16.2.11 has a "removable" flag?
cat /sys/block/sdX/removable
I could reproduce it as well on a test machine with a USB thumb drive (live distro) which is excluded in 16.2.11 but is shown in 16.2.10. Although I'm not a developer I tried to understand what changes were made in https://github.com/ceph/ceph/pull/46375/files#diff-330f9319b0fe352dff0486f66... and there's this line:
if get_file_contents(os.path.join(_sys_block_path, dev, 'removable')) == "1": continue The thumb drive is removable, of course, apparently that is filtered here.
Regards, Eugen
Zitat von Patrick Begou <Patrick.Begou@univ-grenoble-alpes.fr>:
Le 23/10/2023 à 03:04, 544463199@qq.com a écrit :
I think you can try to roll back this part of the python code and wait for your good news :)
Not so easy 😕
[root@e9865d9a7f41 ceph]# git revert 4fc6bc394dffaf3ad375ff29cbb0a3eb9e4dbefc Auto-merging src/ceph-volume/ceph_volume/tests/util/test_device.py CONFLICT (content): Merge conflict in src/ceph-volume/ceph_volume/tests/util/test_device.py Auto-merging src/ceph-volume/ceph_volume/util/device.py CONFLICT (content): Merge conflict in src/ceph-volume/ceph_volume/util/device.py Auto-merging src/ceph-volume/ceph_volume/util/disk.py CONFLICT (content): Merge conflict in src/ceph-volume/ceph_volume/util/disk.py error: could not revert 4fc6bc394df... ceph-volume: Optionally consume loop devices
Patrick _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io