Hi All I am facing some issue related to osd config in ceph-ansible I have 2 nvme disks where i have wal and db configure and i configure lvm like this in osd.yamL osd.yaml devices: - /dev/sdb - /dev/sdc - /dev/sdd - /dev/sde - /dev/sdf - /dev/sdg - /dev/sdh - /dev/sdi - /dev/sdj - /dev/sdk - /dev/sdl - /dev/sdm lvm_volumes: - data: /dev/sdb wal: wal-lv1 wal_vg: vg1 db: journal-lv1 db_vg: vg1 - data: /dev/sdb wal: wal-lv2 wal_vg: vg1 db: journal-lv2 db_vg: vg1 - data: /dev/sdb wal: wal-lv3 wal_vg: vg1 db: journal-lv3 db_vg: vg1 - data: /dev/sdb wal: wal-lv4 wal_vg: vg1 db: journal-lv4 db_vg: vg1 - data: /dev/sdb wal: wal-lv5 wal_vg: vg1 db: journal-lv5 db_vg: vg1 - data: /dev/sdc wal: wal-lv1 wal_vg: vg2 db: journal-lv1 db_vg: vg2 - data: /dev/sdc wal: wal-lv2 wal_vg: vg2 db: journal-lv2 db_vg: vg2 - data: /dev/sdc wal: wal-lv3 wal_vg: vg2 db: journal-lv3 db_vg: vg2 - data: /dev/sdc wal: wal-lv4 wal_vg: vg2 db: journal-lv4 db_vg: vg2 - data: /dev/sdc wal: wal-lv5 wal_vg: vg2 db: journal-lv5 db_vg: vg2 - data: /dev/sdd - data: /dev/sde - data: /dev/sdf - data: /dev/sdg - data: /dev/sdh - data: /dev/sdi - data: /dev/sdj - data: /dev/sdk - data: /dev/sdl - data: /dev/sdm I have also setup wal size and db size respectively in all.yaml all.yaml ntp_service_enabled: false mon_group_name: mons osd_group_name: osds mgr_group_name: mgrs ceph_origin: repository ceph_repository: community ceph_stable_release: octopus public_network: 10.18.0.0/22 cluster_network: 10.25.0.0/24 monitor_interface: ens3 configure_firewall: False ceph_test: true cephx: true ip_version: ipv4 osd_objectstore: bluestore dashboard_enabled: False grafana_server_group_name: grafana-server ceph_conf_overrides: global: bluestore block db size: 85899345920 (80GB) bluestore_block_wal_size: 1073741824 (1GB) i have also tried to to set values of db_size and journal_size in OSD options journal_size: 1024 # OSD journal size in MB block_db_size: 85899345920 But my osd partitions are not creating as I want and taking full disk space sdb 8:16 0 447.1G 0 disk └─ceph--3e81a883--86cd--4ddb--b0b0--be5892f6afe8-osd--block--b981ea8d--410d--4e2f--86f5--509112ff5ebf 253:0 0 447.1G 0 lvm sdc 8:32 0 447.1G 0 disk └─ceph--d3d82887--a7ad--40a4--9c3d--440cd9057fd9-osd--block--de6cae7d--daef--4144--a4bc--077f6044ef22 253:1 0 447.1G 0 lvm sdd 8:48 0 5.5T 0 disk └─ceph--e5aa3641--263c--45df--90d0--cd7e26772e69-osd--block--989fd7b2--d7b7--4170--8b33--cf4c8a29f5c1 253:2 0 5.5T 0 lvm sde 8:64 0 5.5T 0 disk └─ceph--286034b5--1c55--4fb6--8983--4ec6ef482dbd-osd--block--7cdc5ef4--8a88--4ff8--a08e--278985f4764c 253:3 0 5.5T 0 lvm sdf 8:80 0 5.5T 0 disk └─ceph--3bd8b866--b99e--41c4--a993--d69e85c1b658-osd--block--d32e42f4--1133--4c0d--aaf6--821b724fd220 253:4 0 5.5T 0 lvm sdg 8:96 0 5.5T 0 disk └─ceph--972ae2d1--2098--436f--a448--9015edfc4dc5-osd--block--79411ef4--2469--4243--9545--89cd266719f1 253:5 0 5.5T 0 lvm sdh 8:112 0 5.5T 0 disk └─ceph--92cd1c5f--3c67--420e--8a4d--c9ab0b75705b-osd--block--e643c7b3--cb10--49ab--86a8--31b60f4525dc 253:6 0 5.5T 0 lvm sdi 8:128 0 5.5T 0 disk └─ceph--0a27a900--0e81--4ef9--9b2c--692d0a609e7e-osd--block--30eddf48--5e12--4c2f--aed0--e06fb403ed33 253:7 0 5.5T 0 lvm sdj 8:144 0 5.5T 0 disk └─ceph--90156988--e0a3--4699--ae18--ad1f743521bb-osd--block--eb879f94--f289--4dd9--ba2e--48ff06a14708 253:8 0 5.5T 0 lvm sdk 8:160 0 5.5T 0 disk └─ceph--cdcabf84--e713--42c2--b2f0--f1bd6bd54924-osd--block--067418cf--1bd3--46a3--bdb9--b23216d6e8e3 253:9 0 5.5T 0 lvm sdl 8:176 0 5.5T 0 disk └─ceph--60829d44--ca66--44b3--b4be--05e8fe0c154e-osd--block--49a34fa3--c8ba--4fd9--ba3e--a6c4a889ecd5 253:10 0 5.5T 0 lvm sdm 8:192 0 5.5T 0 disk └─ceph--bd0a95e3--4d83--4c9e--a5fe--9f28414015fd-osd--block--abf5c032--eaf8--47b4--9966--67526009f68f 253:11 0 5.5T 0 lvm so after running playbook i get this result of osds with error failed: [10.11.2.202] (item={u'wal_vg': u'vg2', u'data': u'/dev/sdc', u'wal': u'wal-lv5', u'db': u'journal-lv5', u'db_vg': u'vg2'}) => changed=true ansible_loop_var: item cmd: - ceph-volume - --cluster - ceph - lvm - create - --bluestore - --data - /dev/sdc - --block.db - vg2/journal-lv5 - --block.wal - vg2/wal-lv5 delta: '0:00:03.260879' end: '2020-04-20 15:49:51.407784' item: data: /dev/sdc db: journal-lv5 db_vg: vg2 wal: wal-lv5 wal_vg: vg2 msg: non-zero return code rc: 1 start: '2020-04-20 15:49:48.146905' stderr: |- Running command: /usr/bin/ceph-authtool --gen-print-key Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new b26655b0-0859-4f8b-9471-0bafddfc36ef Running command: /sbin/lvcreate --yes -l 100%FREE -n osd-block-b26655b0-0859-4f8b-9471-0bafddfc36ef ceph-b7e18568-6735-4a1a-9aa1-95094189e859 stderr: Calculated size of logical volume is 0 extents. Needs to be larger. --> Was unable to complete a new OSD, will rollback changes Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it stderr: purged osd.0 Traceback (most recent call last): File "/usr/sbin/ceph-volume", line 11, in <module> load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')() File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 39, in __init__ self.main(self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 150, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 42, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 42, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 246, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 327, in prepare block_lv = self.prepare_data_device('block', osd_fsid) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 223, in prepare_data_device **kwargs) File "/usr/lib/python3/dist-packages/ceph_volume/api/lvm.py", line 1185, in create_lv process.run(command) File "/usr/lib/python3/dist-packages/ceph_volume/process.py", line 153, in run raise RuntimeError(msg) RuntimeError: command returned non-zero exit status: 5 So some one could please help me to figure solution for this issue . May be i am doing somewhere something wrong. If yes then please tell me. Also how sizing can be done for osds i mean how we can allow ceph-ansible to not to use default sizing? -- Thanks and Regards, Hemant Sonawane