If you want to be directly setting up the OSDs using ceph-volume commands (I'll pretty much always recommend following https://docs.ceph.com/en/latest/cephadm/services/osd/#dedicated-wal-db over manual ceph-volume stuff in cephadm deployments unless what you're doing can't be done with the spec files), you probably actually want to use `cephadm ceph-volume -- ...` rather than `cephadm shell`. The `cephadm ceph-volume` command mounts the provided keyring (or whatever keyring it infers) at `/var/lib/ceph/bootstrap-osd/ceph.keyring` inside the container, whereas the shell will not. So in theory you could try `cephadm ceph-volume -- lvm prepare --bluestore --data ceph-block-0/block-0 --block.db ceph-db-0/db-0` and that might get you past the keyring issue it seems to be complaining about. On Wed, Mar 6, 2024 at 2:10 PM <service.plant@ya.ru> wrote:

Hi all! I;ve faced an issue I couldnt even google. Trying to create OSD with two separate LVM for data.db and data, gives me intresting error

``` root@ceph-uvm2:/# ceph-volume lvm prepare --bluestore --data ceph-block-0/block-0 --block.db ceph-db-0/db-0 --> Incompatible flags were found, some values may get ignored --> Cannot use None (None) with --bluestore (bluestore) --> Incompatible flags were found, some values may get ignored --> Cannot use --bluestore (bluestore) with --block.db (bluestore) Running command: /usr/bin/ceph-authtool --gen-print-key Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new e3b7b9e5-6399-4e61-8634-41a991bb1948 stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 auth: unable to find a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or directory stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 AuthRegistry(0x7f71bc064978) no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 auth: unable to find a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or directory stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 AuthRegistry(0x7f71bc067fd0) no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx stderr: 2024-03-04T11:45:52.267+0000 7f71c3a89700 -1 auth: unable to find a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or directory stderr: 2024-03-04T11:45:52.267+0000 7f71c3a89700 -1 AuthRegistry(0x7f71c3a87ea0) no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx stderr: 2024-03-04T11:45:52.267+0000 7f71c1024700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1] stderr: 2024-03-04T11:45:52.267+0000 7f71c2026700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1] stderr: 2024-03-04T11:45:52.267+0000 7f71c1825700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1] stderr: 2024-03-04T11:45:52.267+0000 7f71c3a89700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication stderr: [errno 13] RADOS permission denied (error connecting to the cluster) --> RuntimeError: Unable to create a new OSD id ```

And here is output which shows my created volumes

``` root@ceph-uvm2:/# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert block-0 ceph-block-0 -wi-a----- 16.37t

db-0 ceph-db-0 -wi-a----- 326.00g ```

CEPH was rolled out by using cephadm and all the commands I do under cephadm shell

I have no idea what is the reason of that error and that is why I am here.

Any help is appreciated. Thanks in advance. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io