Hi Shashi, I just ran into this myself, and I thought I'd share the solution/workaround that I applied. On 15/05/2023 22:08, Shashi Dahal wrote:
Hi, I followed this documentation:
https://docs.ceph.com/en/pacific/cephadm/adoption/
This is the error I get when trying to enable cephadm.
ceph mgr module enable cephadm
Error ENOENT: module 'cephadm' reports that it cannot run on the active manager daemon: loading remoto library:No module named 'remoto' (pass --force to force enablement)
When I import remoto, it imports just fine.
OS is ubuntu 20.04 focal
As far as I can see, this issue applies to non-containerized Ceph Pacific deployments — such as ones orchestrated with ceph-ansible — running on Debian or Ubuntu. There is no python3-remoto package on those platforms, so you can't install remoto by "regular" installation means (that is, apt/apt-get). It looks to me like this issue was introduced in Pacific, and then went away in Quincy because that release dropped remoto and replaced it with asyncssh (for which a Debian/Ubuntu package does exist). If you start out on Octopus with ceph-ansible and do the Cephadm migration *then*, you're apparently fine too, and you can subsequently use Cephadm to upgrade to Pacific and Quincy. I think it's just this particular combination — (a) run on Debian/Ubuntu, (b) deploy non-containerized, *and* (c) start your deployment on Pacific, where Cephadm adoption breaks. The problem has apparently been known for a while (see https://tracker.ceph.com/issues/43415), but the recommendation appears to have been "just run mgr on a different OS then", which is frequently not a viable option. I tried (like you did, I assume) to just pip-install remoto, and if I opened a Python console and typed "import remoto" it imported just fine, but apparently the cephadm mgr module didn't like that. I've now traced this down to the following line that shows up in the ceph-mgr log if you bump "debug mgr" to 10/10: 2023-06-26T10:01:34.799+0000 7fb0979ba500 10 mgr[py] Computed sys.path '/usr/share/ceph/mgr:/local/lib/python3.8/dist-packages:/lib/python3/dist-packages:/lib/python3.8/dist-packages:lib/python38.zip:/lib/python3.8:/lib/python3.8/lib-dynload' Note the /local/lib/python3.8/dist-packages path, which does not exist on Ubuntu Focal. It's properly /usr/local/lib/python3.8/dist-packages, and this is where "pip install", when run as root outside a virtualenv, installs packages to. I think the incorrect sys.path may actually be a build or packaging bug in the community packages built for Debian/Ubuntu, but I'm not 100% certain. At any rate, the combined workaround for this issue, for me, is: (1) pip install remoto (this installs remoto into /usr/local/lib/python3.8/dist-packages) (2) ln -s /usr/local/lib/python3.8/dist-packages /local/lib/python3.8/dist-packages (this makes pip-installed packages available to ceph-mgr) (3) restart all ceph-mgr instances (4) ceph mgr module enable cephadm Cheers, Florian