Hey Jeremi,

I'm not sure how ceph-ansible will handle a hybrid system. You'll need to make sure that you have the same info in the ceph-ansible "fetch" directory or it will create a separate cluster.

 I'm not sure if you can somehow force this without causing some issues. I'm also not sure what other issues might arise.

Perhaps a way forward would be to start the osds by hand?

Something like:
sudo apt update
sudo apt install ceph

Get a copy of the /etc/ceph/ceph.conf and /var/lib/ceph/bootstrap-osd/ceph.keyring  from another node.

sudo ceph-volume lvm zap /dev/sd?
sudo ceph-volume lvm create --data /dev/sd?



On Mon, Nov 11, 2019 at 1:29 PM Jeremi Avenant <jeremi@idia.ac.za> wrote:
Good day

We currently have 12 nodes in 4 Racks (3x4) and getting another 3 nodes to complete the 5th rack on Version 12.2.12, using ceph-ansible & docker containers.

With the 3 new nodes (1 rack bucket) we would like to make use of a non-containerised setup since our long term plan is to completely move away from OSD containers.

How would one go forward running a hybrid environment in the interim until we've rebuilt all our existing osd nodes?

I can assume that i would require a separate /ceph-ansible directory with site.yml to my existing site-docker.yml

Would I require separate inventory files, one for the existing 12 nodes & one for the 3 new nodes? How would I connect the new nodes to the current controller (mds,mgr, mon) container nodes?

Regards


--



Jeremi-Ernst Avenant, Mr.
Cloud Infrastructure Specialist
Inter-University Institute for Data Intensive Astronomy
5th Floor, Department of Physics and Astronomy, 
University of Cape Town

E-mail (IDIA): jeremi@idia.ac.za
Rondebosch, Cape Town, 7600, South Africa
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io


--
Thomas Bennett

Storage Engineer at SARAO