cephadm - How to deploy ceph cluster with a partition on SSD for block.db
I'm trying to deploy a ceph cluster with a cephadm tool. I've already successfully done all steps except adding OSDs. My testing equipment consists of three hosts. Each host has SSD storage, where OS is installed into. On that storage I created partition, which can be used as a ceph block.db. Hosts have also 2 additional HDs (spinning drives) for OSD data. On docs I couldn't find how to deploy such configuration. Do you have any hints, how to do that? Thanks for help!
You'll need to specify drive_groups in a yaml file if you don't deploy standalone OSDs: https://docs.ceph.com/docs/master/cephadm/drivegroups/ Zitat von klemen@psi-net.si:
I'm trying to deploy a ceph cluster with a cephadm tool. I've already successfully done all steps except adding OSDs. My testing equipment consists of three hosts. Each host has SSD storage, where OS is installed into. On that storage I created partition, which can be used as a ceph block.db. Hosts have also 2 additional HDs (spinning drives) for OSD data. On docs I couldn't find how to deploy such configuration. Do you have any hints, how to do that? Thanks for help! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Is there a possibility to specify partitions? I only see whole discs/devices chosen by vendor or model name. Regards, Lars Am Thu, 18 Jun 2020 09:52:36 +0000 schrieb Eugen Block <eblock@nde.ag>:
You'll need to specify drive_groups in a yaml file if you don't deploy standalone OSDs:
https://docs.ceph.com/docs/master/cephadm/drivegroups/
Zitat von klemen@psi-net.si:
I'm trying to deploy a ceph cluster with a cephadm tool. I've already successfully done all steps except adding OSDs. My testing equipment consists of three hosts. Each host has SSD storage, where OS is installed into. On that storage I created partition, which can be used as a ceph block.db. Hosts have also 2 additional HDs (spinning drives) for OSD data. On docs I couldn't find how to deploy such configuration. Do you have any hints, how to do that? Thanks for help! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Informationstechnologie Berlin-Brandenburgische Akademie der Wissenschaften Jägerstraße 22-23 10117 Berlin Tel.: +49 30 20370-352 http://www.bbaw.de
I overread the partition part, I don't think drive_groups are ready for that (yet?). In that case I would deploy the OSDs manually with 'ceph-volume lvm create --data {vg name/lv name} --journal /path/to/device' or '... --journal {vg name/lv name}' on your OSD nodes. Zitat von Lars Täuber <taeuber@bbaw.de>:
Is there a possibility to specify partitions? I only see whole discs/devices chosen by vendor or model name.
Regards, Lars
Am Thu, 18 Jun 2020 09:52:36 +0000 schrieb Eugen Block <eblock@nde.ag>:
You'll need to specify drive_groups in a yaml file if you don't deploy standalone OSDs:
https://docs.ceph.com/docs/master/cephadm/drivegroups/
Zitat von klemen@psi-net.si:
I'm trying to deploy a ceph cluster with a cephadm tool. I've already successfully done all steps except adding OSDs. My testing equipment consists of three hosts. Each host has SSD storage, where OS is installed into. On that storage I created partition, which can be used as a ceph block.db. Hosts have also 2 additional HDs (spinning drives) for OSD data. On docs I couldn't find how to deploy such configuration. Do you have any hints, how to do that? Thanks for help! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Informationstechnologie Berlin-Brandenburgische Akademie der Wissenschaften Jägerstraße 22-23 10117 Berlin Tel.: +49 30 20370-352 http://www.bbaw.de _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks for your prompt reply. I was afraid it's not ready yet. Hope it will be soon. The next thing I missed on drive_groups is to be able to specify storage devices directly (/dev/XXX). Probably that would also solve my original problem.
I found out that it's already possible to specify storage path in OSD service specification yaml. It works for data_devices, but unfortunately not for db_devices and wal_devices, at least not in my case. service_type: osd service_id: osd_spec_default placement: host_pattern: '*' data_devices: paths: - /dev/vdb1 db_devices: paths: - /dev/vdb2 wal_devices: paths: - /dev/vdb3
On Tue, Sep 08, 2020 at 07:14:16AM -0000, klemen@psi-net.si wrote:
I found out that it's already possible to specify storage path in OSD service specification yaml. It works for data_devices, but unfortunately not for db_devices and wal_devices, at least not in my case.
Aside from the question whether db/wal/journal_devices should accept paths as a filter, I'd like to point out that partitions are only a valid argument when calling `ceph-volume lvm prepare/create`. OSD service specs are quite tightly coupled to the batch subcommand, which has no support partitions. The batch subcommand will soon gain support for handle logical volumes too. I'll explore if we can extend osd service specs accordingly. Until then I'm afraid you're stuck to use the create or prepare subcommand for "uncommon" deployments like this (db devices collocated with root device).
service_type: osd service_id: osd_spec_default placement: host_pattern: '*' data_devices: paths: - /dev/vdb1 db_devices: paths: - /dev/vdb2 wal_devices: paths: - /dev/vdb3 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (5)
-
Dimitri Savineau
-
Eugen Block
-
Jan Fajerski
-
klemen@psi-net.si
-
Lars Täuber