5 Feb
2021
5 Feb
'21
8:15 a.m.
Here is the log from ceph-volume.
```
[2021-02-05 04:03:17,000][ceph_volume.process][INFO ] Running command: /usr/sbin/vgcreate
--force --yes ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64 /dev/sdd
[2021-02-05 04:03:17,134][ceph_volume.process][INFO ] stdout Physical volume
"/dev/sdd" successfully created.
[2021-02-05 04:03:17,166][ceph_volume.process][INFO ] stdout Volume group
"ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64" successfully created
[2021-02-05 04:03:17,189][ceph_volume.process][INFO ] Running command: /usr/sbin/vgs
--noheadings --readonly --units=b --nosuffix --separator=";" -S
vg_name=ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64 -o
vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size
[2021-02-05 04:03:17,229][ceph_volume.process][INFO ] stdout
ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64";"1";"0";"wz--n-";"572317";"572317";"4194304
[2021-02-05 04:03:17,229][ceph_volume.api.lvm][DEBUG ] size was passed: 2.18 TB ->
572318
[2021-02-05 04:03:17,235][ceph_volume.process][INFO ] Running command: /usr/sbin/lvcreate
--yes -l 572318 -n osd-block-b05c3c90-b7d5-4f13-8a58-f72761c1971b
ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64
[2021-02-05 04:03:17,244][ceph_volume.process][INFO ] stderr Volume group
"ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64" has insufficient free space (572317
extents): 572318 required.
```
size was passed: 2.18 TB -> 572318
How is this calculated?
Thanks!
Tony
> -----Original Message-----
> From: Tony Liu <tonyliu0592(a)hotmail.com>
> Sent: Thursday, February 4, 2021 8:34 PM
> To: ceph-users(a)ceph.io
> Subject: [ceph-users] replace OSD failed
>
> Hi,
>
> With 15.2.8, run "ceph orch rm osd 12 --replace --force", PGs on osd.12
> are remapped, osd.12 is removed from "ceph osd tree", the daemon is
> removed from "ceph orch ps", the device is "available"
> in "ceph orch device ls". Everything seems good at this point.
>
> Then dry-run service spec.
> ```
> # cat osd-spec.yaml
> service_type: osd
> service_id: osd-spec
> placement:
> hosts:
> - ceph-osd-1
> data_devices:
> rotational: 1
> db_devices:
> rotational: 0
>
> # ceph orch apply osd -i osd-spec.yaml --dry-run
> +---------+----------+------------+----------+----------+-----+
> |SERVICE |NAME |HOST |DATA |DB |WAL |
> +---------+----------+------------+----------+----------+-----+
> |osd |osd-spec |ceph-osd-3 |/dev/sdd |/dev/sdb |- |
> +---------+----------+------------+----------+----------+-----+
> ```
> It looks as expected.
>
> Then "ceph orch apply osd -i osd-spec.yaml".
> Here is the log of cephadm.
> ```
> /bin/docker:stderr --> relative data size: 1.0 /bin/docker:stderr -->
> passed block_db devices: 1 physical, 0 LVM /bin/docker:stderr Running
> command: /usr/bin/ceph-authtool --gen-print-key /bin/docker:stderr
> Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-
> osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd tree -f json
> /bin/docker:stderr Running command: /usr/bin/ceph --cluster ceph --name
> client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
> -i - osd new b05c3c90-b7d5-4f13-8a58-f72761c1971b 12 /bin/docker:stderr
> Running command: /usr/sbin/vgcreate --force --yes ceph-a3886f74-3de9-
> 4e6e-a983-8330eda0bd64 /dev/sdd /bin/docker:stderr stdout: Physical
> volume "/dev/sdd" successfully created.
> /bin/docker:stderr stdout: Volume group "ceph-a3886f74-3de9-4e6e-a983-
> 8330eda0bd64" successfully created /bin/docker:stderr Running command:
> /usr/sbin/lvcreate --yes -l 572318 -n osd-block-b05c3c90-b7d5-4f13-8a58-
> f72761c1971b ceph-a3886f74-3de9-4e6e-a983-8330eda0bd64
> /bin/docker:stderr stderr: Volume group "ceph-a3886f74-3de9-4e6e-a983-
> 8330eda0bd64" has insufficient free space (572317 extents): 572318
> required.
> /bin/docker:stderr --> Was unable to complete a new OSD, will rollback
> changes ``` Q1, why VG name (ceph-<id>) is different from others (ceph-
> block-<id>)?
> Q2, where is that 572318 from? Since all HDDs are the same model, VG
> "Total PE" of all HDDs is 572317.
> Has anyone seen similar issues? Anything I am missing?
>
>
> Thanks!
> Tony
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
> email to ceph-users-leave(a)ceph.io