Hi, I am having issues trying to deploy ceph 19.2.3 on rocky linux 9. 1 - I cannot install ceph-common : it needs openssl 3.4, rocky 9 offers only openssl 3.2 2 - I bootstraped my cluster like this : cephadm bootstrap \ --mon-ip 10.200.15.10 \ --cluster-network 10.210.0.0/16 \ --initial-dashboard-user admin \ --initial-dashboard-password 'admin' \ --ssh-private-key /root/.ssh/id_rsa \ --ssh-public-key /root/.ssh/id_rsa.pub it did bootstrap, but when I try adding other hosts, I cannot, it has something to do with ssh. I did copy id.rsa.pub on the other nodes via ssh-copy-id, and I can ssh without password to them. but I cannot ssh to them via cephadm shell. and when i try to add a host I get : Aug 11 15:29:06 hyper01 ceph-mgr[35633]: [cephadm ERROR cephadm.ssh] Traceback (most recent call last): File "/lib/python3.9/site-packages/asyncssh/connection.py", line 440, in _connect await options.waiter asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/lib64/python3.9/asyncio/tasks.py", line 490, in wait_for return fut.result() asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/ssh.py", line 197, in redirect_log yield File "/usr/share/ceph/mgr/cephadm/ssh.py", line 174, in _remote_connection conn = await asyncssh.connect(addr, username=self.mgr.ssh_user, client_keys=[ self.mgr.tkey.name], File "/lib/python3.9/site-packages/asyncssh/connection.py", line 8170, in connect return await asyncio.wait_for( File "/lib64/python3.9/asyncio/tasks.py", line 492, in wait_for raise exceptions.TimeoutError() from exc asyncio.exceptions.TimeoutError Aug 11 15:29:06 hyper01 ceph-mgr[35633]: log_channel(cephadm) log [ERR] : Traceback (most recent call last): File "/lib/python3.9/site-packages/asyncssh/connection.py", line 440, in _connect await options.waiter asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/lib64/python3.9/asyncio/tasks.py", line 490, in wait_for return fut.result() asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/ssh.py", line 197, in redirect_log yield File "/usr/share/ceph/mgr/cephadm/ssh.py", line 174, in _remote_connection conn = await asyncssh.connect(addr, username=self.mgr.ssh_user, client_keys=[ self.mgr.tkey.name], File "/lib/python3.9/site-packages/asyncssh/connection.py", line 8170, in connect return await asyncio.wait_for( File "/lib64/python3.9/asyncio/tasks.py", line 492, in wait_for raise exceptions.TimeoutError() from exc asyncio.exceptions.TimeoutError Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: 2025-08-11T14:29:06.509+0000 7f27b44ed640 -1 log_channel(cephadm) log [ERR] : Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Traceback (most recent call last): Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: File "/lib/python3.9/site-packages/asyncssh/connection.py", line 440, in _connect Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: await options.waiter Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: asyncio.exceptions.CancelledError Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: During handling of the above exception, another exception occurred: Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Traceback (most recent call last): Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: File "/lib64/python3.9/asyncio/tasks.py", line 490, in wait_for Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: return fut.result() Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: asyncio.exceptions.CancelledError Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: The above exception was the direct cause of the following exception: Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: Traceback (most recent call last): Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 197, in redirect_log Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: yield Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 174, in _remote_connection Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: conn = await asyncssh.connect(addr, username=self.mgr.ssh_user, client_keys=[self.mgr.tkey.name], Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: File "/lib/python3.9/site-packages/asyncssh/connection.py", line 8170, in connect Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: return await asyncio.wait_for( Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: File "/lib64/python3.9/asyncio/tasks.py", line 492, in wait_for Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: raise exceptions.TimeoutError() from exc Aug 11 15:29:06 hyper01 ceph-42cab8b0-76bc-11f0-b9ed-7cc255beecde-mgr-hyper01-rzwgry[35624]: asyncio.exceptions.TimeoutError Aug 11 15:29:06 hyper01 ceph-mgr[35633]: [cephadm ERROR orchestrator._interface] Failed to connect to hyper02 (10.200.15.11): TimeoutError() Log: Opening SSH connection to 10.200.15.11, port 22 [conn=7] Connected to SSH server at 10.200.15.11, port 22 [conn=7] Local address: 10.200.15.10, port 53326 [conn=7] Peer address: 10.200.15.11, port 22 [conn=6, chan=10] Requesting new SSH session [conn=6, chan=10] Command: which python3 [conn=6, chan=10] Received exit status 0 [conn=6, chan=10] Received channel close [conn=6, chan=10] Channel closed [conn=6, chan=11] Requesting new SSH session [conn=6, chan=11] Command: /usr/bin/python3 /var/lib/ceph/42cab8b0-76bc-11f0-b9ed-7cc255beecde/cephadm.1a8853661a9c1798390b8e8d13c27688c1b1327a075745af2ee40ac466f0ac36 --timeout 895 gather-facts [conn=6, chan=11] Received exit status 0 [conn=6, chan=11] Received channel close [conn=6, chan=11] Channel closed [conn=6, chan=12] Requesting new SSH session [conn=6, chan=12] Command: ls /etc/sysctl.d [conn=6, chan=12] Received exit status 0 [conn=6, chan=12] Received channel close [conn=6, chan=12] Channel closed [conn=7] Aborting connection [conn=7] Connection closed Traceback (most recent call last): File "/lib/python3.9/site-packages/asyncssh/connection.py", line 440, in _connect await options.waiter asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/lib64/python3.9/asyncio/tasks.py", line 490, in wait_for return fut.result() asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/ssh.py", line 197, in redirect_log yield File "/usr/share/ceph/mgr/cephadm/ssh.py", line 174, in _remote_connection conn = await asyncssh.connect(addr, username=self.mgr.ssh_user, client_keys=[ self.mgr.tkey.name], File "/lib/python3.9/site-packages/asyncssh/connection.py", line 8170, in connect return await asyncio.wait_for( File "/lib64/python3.9/asyncio/tasks.py", line 492, in wait_for raise exceptions.TimeoutError() from exc asyncio.exceptions.TimeoutError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/module.py", line 1775, in _check_valid_addr out, err, code = self.wait_async(CephadmServe(self)._run_cephadm( File "/usr/share/ceph/mgr/cephadm/module.py", line 815, in wait_async return self.event_loop.get_result(coro, timeout) File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result return future.result(timeout) File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result return self.__get_result() File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result raise self._exception File "/usr/share/ceph/mgr/cephadm/serve.py", line 1595, in _run_cephadm await self.mgr.ssh._remote_connection(host, addr) File "/usr/share/ceph/mgr/cephadm/ssh.py", line 182, in _remote_connection raise File "/lib64/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/share/ceph/mgr/cephadm/ssh.py", line 214, in redirect_log raise HostConnectionError( cephadm.ssh.HostConnectionError: Failed to connect to hyper02 (10.200.15.11): TimeoutError() Log: Opening SSH connection to 10.200.15.11, port 22 [conn=7] Connected to SSH server at 10.200.15.11, port 22 [conn=7] Local address: 10.200.15.10, port 53326 [conn=7] Peer address: 10.200.15.11, port 22 [conn=6, chan=10] Requesting new SSH session [conn=6, chan=10] Command: which python3 [conn=6, chan=10] Received exit status 0 [conn=6, chan=10] Received channel close [conn=6, chan=10] Channel closed [conn=6, chan=11] Requesting new SSH session [conn=6, chan=11] Command: /usr/bin/python3 /var/lib/ceph/42cab8b0-76bc-11f0-b9ed-7cc255beecde/cephadm.1a8853661a9c1798390b8e8d13c27688c1b1327a075745af2ee40ac466f0ac36 --timeout 895 gather-facts [conn=6, chan=11] Received exit status 0 [conn=6, chan=11] Received channel close [conn=6, chan=11] Channel closed [conn=6, chan=12] Requesting new SSH session [conn=6, chan=12] Command: ls /etc/sysctl.d [conn=6, chan=12] Received exit status 0 [conn=6, chan=12] Received channel close [conn=6, chan=12] Channel closed [conn=7] Aborting connection [conn=7] Connection closed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 138, in wrapper return OrchResult(f(*args, **kwargs)) File "/usr/share/ceph/mgr/cephadm/module.py", line 3502, in apply results.append(self._apply(spec)) File "/usr/share/ceph/mgr/cephadm/module.py", line 3251, in _apply return self._add_host(cast(HostSpec, spec)) File "/usr/share/ceph/mgr/cephadm/module.py", line 1800, in _add_host ip_addr = self._check_valid_addr(spec.hostname, spec.addr) File "/usr/share/ceph/mgr/cephadm/module.py", line 1789, in _check_valid_addr raise OrchestratorError(str(e)) orchestrator._interface.OrchestratorError: Failed to connect to hyper02 (10.200.15.11): TimeoutError() Log: Opening SSH connection to 10.200.15.11, port 22 [conn=7] Connected to SSH server at 10.200.15.11, port 22 [conn=7] Local address: 10.200.15.10, port 53326 [conn=7] Peer address: 10.200.15.11, port 22 [conn=6, chan=10] Requesting new SSH session [conn=6, chan=10] Command: which python3 [conn=6, chan=10] Received exit status 0 [conn=6, chan=10] Received channel close [conn=6, chan=10] Channel closed [conn=6, chan=11] Requesting new SSH session [conn=6, chan=11] Command: /usr/bin/python3 /var/lib/ceph/42cab8b0-76bc-11f0-b9ed-7cc255beecde/cephadm.1a8853661a9c1798390b8e8d13c27688c1b1327a075745af2ee40ac466f0ac36 --timeout 895 gather-facts [conn=6, chan=11] Received exit status 0 [conn=6, chan=11] Received channel close [conn=6, chan=11] Channel closed [conn=6, chan=12] Requesting new SSH session [conn=6, chan=12] Command: ls /etc/sysctl.d [conn=6, chan=12] Received exit status 0 [conn=6, chan=12] Received channel close [conn=6, chan=12] Channel closed [conn=7] Aborting connection [conn=7] Connection closed Aug 11 15:29:06 hyper01 ceph-mgr[35633]: log_channel(cephadm) log [ERR] : Failed to connect to hyper02 (10.200.15.11): TimeoutError() Log: Opening SSH connection to 10.200.15.11, port 22 [conn=7] Connected to SSH server at 10.200.15.11, port 22 [conn=7] Local address: 10.200.15.10, port 53326 [conn=7] Peer address: 10.200.15.11, port 22 [conn=6, chan=10] Requesting new SSH session [conn=6, chan=10] Command: which python3 [conn=6, chan=10] Received exit status 0 [conn=6, chan=10] Received channel close [conn=6, chan=10] Channel closed [conn=6, chan=11] Requesting new SSH session [conn=6, chan=11] Command: /usr/bin/python3 /var/lib/ceph/42cab8b0-76bc-11f0-b9ed-7cc255beecde/cephadm.1a8853661a9c1798390b8e8d13c27688c1b1327a075745af2ee40ac466f0ac36 --timeout 895 gather-facts [conn=6, chan=11] Received exit status 0 [conn=6, chan=11] Received channel close [conn=6, chan=11] Channel closed [conn=6, chan=12] Requesting new SSH session [conn=6, chan=12] Command: ls /etc/sysctl.d [conn=6, chan=12] Received exit status 0 [conn=6, chan=12] Received channel close [conn=6, chan=12] Channel closed [conn=7] Aborting connection [conn=7] Connection closed Traceback (most recent call last): File "/lib/python3.9/site-packages/asyncssh/connection.py", line 440, in _connect await options.waiter asyncio.exceptions.CancelledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/lib64/python3.9/asyncio/tasks.py", line 490, in wait_for return fut.result() asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/ssh.py", line 197, in redirect_log yield File "/usr/share/ceph/mgr/cephadm/ssh.py", line 174, in _remote_connection conn = await asyncssh.connect(addr, username=self.mgr.ssh_user, client_keys=[ self.mgr.tkey.name], File "/lib/python3.9/site-packages/asyncssh/connection.py", line 8170, in connect return await asyncio.wait_for( File "/lib64/python3.9/asyncio/tasks.py", line 492, in wait_for raise exceptions.TimeoutError() from exc asyncio.exceptions.TimeoutError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/module.py", line 1775, in _check_valid_addr out, err, code = self.wait_async(CephadmServe(self)._run_cephadm( File "/usr/share/ceph/mgr/cephadm/module.py", line 815, in wait_async return self.event_loop.get_result(coro, timeout) File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result return future.result(timeout) File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result return self.__get_result() File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result raise self._exception File "/usr/share/ceph/mgr/cephadm/serve.py", line 1595, in _run_cephadm await self.mgr.ssh._remote_connection(host, addr) File "/usr/share/ceph/mgr/cephadm/ssh.py", line 182, in _remote_connection raise File "/lib64/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/share/ceph/mgr/cephadm/ssh.py", line 214, in redirect_log raise HostConnectionError( cephadm.ssh.HostConnectionError: Failed to connect to hyper02 (10.200.15.11): TimeoutError() Log: Opening SSH connection to 10.200.15.11, port 22 [conn=7] Connected to SSH server at 10.200.15.11, port 22 [conn=7] Local address: 10.200.15.10, port 53326 [conn=7] Peer address: 10.200.15.11, port 22 [conn=6, chan=10] Requesting new SSH session [conn=6, chan=10] Command: which python3 [conn=6, chan=10] Received exit status 0 [conn=6, chan=10] Received channel close [conn=6, chan=10] Channel closed [conn=6, chan=11] Requesting new SSH session [conn=6, chan=11] Command: /usr/bin/python3 /var/lib/ceph/42cab8b0-76bc-11f0-b9ed-7cc255beecde/cephadm.1a8853661a9c1798390b8e8d13c27688c1b1327a075745af2ee40ac466f0ac36 --timeout 895 gather-facts [conn=6, chan=11] Received exit status 0 [conn=6, chan=11] Received channel close [conn=6, chan=11] Channel closed [conn=6, chan=12] Requesting new SSH session [conn=6, chan=12] Command: ls /etc/sysctl.d [conn=6, chan=12] Received exit status 0 [conn=6, chan=12] Received channel close [conn=6, chan=12] Channel closed [conn=7] Aborting connection [conn=7] Connection closed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 138, in wrapper return OrchResult(f(*args, **kwargs)) File "/usr/share/ceph/mgr/cephadm/module.py", line 3502, in apply results.append(self._apply(spec)) File "/usr/share/ceph/mgr/cephadm/module.py", line 3251, in _apply return self._add_host(cast(HostSpec, spec)) File "/usr/share/ceph/mgr/cephadm/module.py", line 1800, in _add_host ip_addr = self._check_valid_addr(spec.hostname, spec.addr) File "/usr/share/ceph/mgr/cephadm/module.py", line 1789, in _check_valid_addr raise OrchestratorError(str(e)) orchestrator._interface.OrchestratorError: Failed to connect to hyper02 (10.200.15.11): TimeoutError() Log: Opening SSH connection to 10.200.15.11, port 22 [conn=7] Connected to SSH server at 10.200.15.11, port 22 [conn=7] Local address: 10.200.15.10, port 53326 [conn=7] Peer address: 10.200.15.11, port 22 [conn=6, chan=10] Requesting new SSH session [conn=6, chan=10] Command: which python3 [conn=6, chan=10] Received exit status 0 [conn=6, chan=10] Received channel close [conn=6, chan=10] Channel closed [conn=6, chan=11] Requesting new SSH session [conn=6, chan=11] Command: /usr/bin/python3 /var/lib/ceph/42cab8b0-76bc-11f0-b9ed-7cc255beecde/cephadm.1a8853661a9c1798390b8e8d13c27688c1b1327a075745af2ee40ac466f0ac36 --timeout 895 gather-facts [conn=6, chan=11] Received exit status 0 [conn=6, chan=11] Received channel close [conn=6, chan=11] Channel closed [conn=6, chan=12] Requesting new SSH session [conn=6, chan=12] Command: ls /etc/sysctl.d [conn=6, chan=12] Received exit status 0 [conn=6, chan=12] Received channel close [conn=6, chan=12] Channel closed [conn=7] Aborting connection [conn=7] Connection closed Regards.