Hey Jeremi, I'm not sure how ceph-ansible will handle a hybrid system. You'll need to make sure that you have the same info in the ceph-ansible "fetch" directory or it will create a separate cluster. I'm not sure if you can somehow force this without causing some issues. I'm also not sure what other issues might arise. Perhaps a way forward would be to start the osds by hand? Something like: sudo apt update sudo apt install ceph Get a copy of the /etc/ceph/ceph.conf and /var/lib/ceph/bootstrap-osd/ceph.keyring from another node. sudo ceph-volume lvm zap /dev/sd? sudo ceph-volume lvm create --data /dev/sd? On Mon, Nov 11, 2019 at 1:29 PM Jeremi Avenant <jeremi@idia.ac.za> wrote:

Good day

We currently have 12 nodes in 4 Racks (3x4) and getting another 3 nodes to complete the 5th rack on Version 12.2.12, using ceph-ansible & docker containers.

With the 3 new nodes (1 rack bucket) we would like to make use of a non-containerised setup since our long term plan is to completely move away from OSD containers.

How would one go forward running a hybrid environment in the interim until we've rebuilt all our existing osd nodes?

I can assume that i would require a separate /ceph-ansible directory with site.yml to my existing site-docker.yml

Would I require separate inventory files, one for the existing 12 nodes & one for the 3 new nodes? How would I connect the new nodes to the current controller (mds,mgr, mon) container nodes?

Regards

--

*Jeremi-Ernst Avenant, Mr.*Cloud Infrastructure Specialist Inter-University Institute for Data Intensive Astronomy 5th Floor, Department of Physics and Astronomy, University of Cape Town

Tel: 021 959 4137 <0219592327> Web: www.idia.ac.za <http://www.uwc.ac.za/> E-mail (IDIA): jeremi@idia.ac.za <mfundo@idia.ac.za> Rondebosch, Cape Town, 7600, South Africa _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

-- Thomas Bennett Storage Engineer at SARAO