Hello Michael, You can try this: 1/ check that the host shows up on ceph orch ls with the right label 'osds' 2/ check that the host is OK with ceph cephadm check-host <hostname>. It should look like: <hostname> (None) ok podman (/usr/bin/podman) version 4.6.1 is present systemctl is present lvcreate is present Unit chronyd.service is enabled and running Hostname "<hostname>" matches what is expected. Host looks OK 3/ double check you service_type 'osd' with ceph orch ls --service-type osd --export It should show the correct placement and spec (drives size, etc.) 4/ enable debugging with ceph config set mgr mgr/cephadm/log_to_cluster_level debug 5/ open a terminal and observe ceph -W cephadm --watch-debug 6/ ceph mgr fail 7/ ceph orch device ls --hostname=<hostname> --wide --refresh (should show local bloc devices as Available and trigger the creation of the OSDs) If your service_type 'osd' is correct, the orchestrator should deploy OSDs on the node. If it does not then look for the reason why in ceph -W cephadm --watch-debug output. Regards, Frédéric. ----- Le 24 Avr 24, à 3:22, Michael Baer ceph@mikesoffice.com a écrit :
Hi,
This problem started with trying to add a new storage server into a quincy v17.2.6 ceph cluster. Whatever I did, I could not add the drives on the new host as OSDs: via dashboard, via cephadm shell, by setting osd unmanaged to false.
But what I started realizing is that orchestrator will also no longer automatically manage services. I.e. if a service is set to manage by labels, removing and adding labels to different hosts for that service has no affect. Same if I set a service to be manage via hostnames. Same if I try to drain a host (the services/podman containers just keep running). Although, I am able to add/rm services via 'cephadm shell ceph orch daemon add/rm'. But Ceph will not manage automatically using labels/hostnames.
This apparently includes OSD daemons. I can not create and on the new host either automatically or manually, but I'm hoping the services/OSD issues are related and not two issues.
I haven't been able to find any obvious errors in /var/log/ceph, /var/log/syslog, logs <container>, etc. I have been able to get 'slow ops' errors on monitors by trying to add OSDs manually (and having to restart the monitor). I've also gotten cephadm shell to hang. And had to restart managers. I'm not an expert and it could be something obvious, but I haven't been able to figure out a solution. If anyone has any suggestions, I would greatly appreciate them.
Thanks, Mike
-- Michael Baer ceph@mikesoffice.com _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io