Hi, that is expected since the orchestrator didn't create those OSDs, just adopted them. It won't be different in any other release. You can create a new spec file that reflects your current setup, the docs contain a couple of examples, here's a simple one: ---snip--- # cat osd-specs.yaml service_type: osd service_id: osd-hdd-ssd service_name: osd.osd-hdd-ssd placement: hosts: - host1 - host2 - host3 spec: data_devices: rotational: 1 size: 15G db_devices: rotational: 0 size: 13G:16G filter_logic: AND objectstore: bluestore ---snip--- Note that the above spec is a different one than in the following example. If you're sure that your new spec will match, you can apply it, but nothing will happen to the existing OSDs. If there are unused block devices that would apply to the specs, the orchestrator would try to deploy new OSDs. Once you have a different osd spec in place, you can set the appropriate string in the unit.meta file of the OSD: # grep service /var/lib/ceph/FSID/osd.0/unit.meta "service_name": "osd.osd-hdd-ssd", After a couple of minutes, the orchestrator will refresh its view on OSDs (transition from the unmanaged service to the managed one): # cat osd-standalone.yaml service_type: osd service_name: osd.standalone service_id: osd.standalone placement: label: osd spec: data_devices: rotational: 1 filter_logic: AND objectstore: bluestore After chaning the first OSD's unit.meta file NAME PORTS RUNNING REFRESHED AGE PLACEMENT osd 5 2s ago - <unmanaged> osd.osd.standalone 1 2s ago 5m label:osd As soon as all OSDs have been transitioned, the unmanaged service will vanish: NAME PORTS RUNNING REFRESHED AGE PLACEMENT osd.osd.standalone 6 0s ago 8m label:osd I'm not sure if there's a smoother way, but that's how I've done it in the past because it's relatively quick (although it's been a while since I last adopted a prod cluster to cephadm). Regards, Eugen Zitat von Chris Palmer via ceph-users <ceph-users@ceph.io>:

I would appreciate advice on the "best" way to manage adopted OSDs.

This is a test squid 19.2.3 cluster, originally installed using centos 9 packages. I am testing the procedure for adopting, then will use cephadm to update to tentacle. Once I am happy with the whole process I will repeat on a small production cluster.

I have successfully adopted mon, mgr, osd, rgw & mds as described in the docs. That left rgw & mds managed, but mon, mgr & osds unmanaged. Setting the mon & mgr services managed was easy by tweaking the yml and applying. But I am unclear how to best handle osds. That leaves me in this state:

[root@c4 ~]# ceph orch ls NAME        PORTS  RUNNING  REFRESHED  AGE  PLACEMENT mds.cephfs             2/2  51s ago    21h  c1;c2 mgr                    3/3  52s ago    21h  c1;c2;c4 mon                    3/3  51s ago    21h  c1;c2;c3 osd                     12  51s ago    -    <unmanaged> rgw.rgw     ?:80       2/2  51s ago    21h  c1;c2

The exported yml had the following for osds, which had to be commented out as it would not parse (missing placement, missing data_devices etc):

service_type: osd service_name: osd unmanaged: true spec:   filter_logic: AND   objectstore: bluestore

My questions are:

* With the OSDs unmanaged, what functionality am I missing? In particular, things like using cephadm to replace a failed disk... * These are only small clusters, so I am happy to have the OSDs enumerated in the yml, and not have ceph try to find and automatically use available disks. Is it possible to have the full functionality except from automatically grabbing available disks? * How can I generate yml that reflects the current state of the OSDs? We currently have pools (per application) for NVMe, SSD, and HDD with shared NVMe DB/WAL. * I can't find documentation for transitioning OSDs from unmanaged to managed. Have I just overlooked it? * Is any part of this easier on tentacle than squid? If so I can upgrade the package squid to tentacle before adopting. (But unless there is a good reason, I would rather have familiarised myself with the cephadm upgrade process before starting any of this on the production cluster...)

Many thanks for any insights.

Chris _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io