Dear Cephalopodians, as we all know, ceph-deploy is on its demise since a while and essentially in "maintenance mode". We've been eyeing the "ssh orchestrator" which was in Nautilus as the "successor in spirit" of ceph-deploy. While we have not tried it out just yet, I find this module seems to be gone without a trace in Octopus. There's still an Orchestrator module, but this seems to work "only" with containers. Is this true, or is there still an SSH orchestrator capable of bare-metal operation in Octopus (or are there plans to have something like this)? While I see many advantages of containers in many areas, and certainly also for smaller setups or test setups with Ceph, as any technology, they come with their own problems. Example issues (which all can be solved, but require extra work from the administrator) are: - Operation on machines without connectivity to the internet (you'd need to mirror the containers or run your own registry), - Ensuring automated security updates both outside the containers and inside the containers, or re-pull them regularly (and monitor that), - Integrate with existing logging and configuration management systems, - Potential hardware issues, such das InfiniBand RDMA. There's surely more (and there are also as many benefits), and as I said, all can be solved; the point I want to make is: Containers are not the best solution in all environments and also not for all admins. So my question is: Is there something like the SSH orchestrator still available? I guess essentially the cephadm orchestrator does something similar behind the screnes, with the added bells and whistles to manage the containers. Of course, a reduced feature-set would be expected (e.g. no "ceph orch upgrade"), but it would jump into the hole ceph-deploy has left. Maybe this is as easy as setting a configuration knob? Or is it also possible to switch to a "bare-metal edition" of cephadm (which might rely on users or existing configuration management to install the packages, e.g.)? Cheers, Oliver
Am 02.07.20 um 19:57 schrieb Oliver Freyermuth:
Dear Cephalopodians,
as we all know, ceph-deploy is on its demise since a while and essentially in "maintenance mode".
We've been eyeing the "ssh orchestrator" which was in Nautilus as the "successor in spirit" of ceph-deploy. While we have not tried it out just yet, I find this module seems to be gone without a trace in Octopus. There's still an Orchestrator module, but this seems to work "only" with containers.
Is this true, or is there still an SSH orchestrator capable of bare-metal operation in Octopus (or are there plans to have something like this)?
While I see many advantages of containers in many areas, and certainly also for smaller setups or test setups with Ceph, as any technology, they come with their own problems. Example issues (which all can be solved, but require extra work from the administrator) are: - Operation on machines without connectivity to the internet (you'd need to mirror the containers or run your own registry), - Ensuring automated security updates both outside the containers and inside the containers, or re-pull them regularly (and monitor that), - Integrate with existing logging and configuration management systems, - Potential hardware issues, such das InfiniBand RDMA.
There's surely more (and there are also as many benefits), and as I said, all can be solved; the point I want to make is: Containers are not the best solution in all environments and also not for all admins.
So my question is: Is there something like the SSH orchestrator still available? I guess essentially the cephadm orchestrator does something similar behind the screnes, with the added bells and whistles to manage the containers. Of course, a reduced feature-set would be expected (e.g. no "ceph orch upgrade"), but it would jump into the hole ceph-deploy has left.
we're renamed the SSH orchestrator into cephadm. See https://github.com/ceph/ceph/pull/32193 for the corresponding pull request. Hope that helps, Sebastian
Maybe this is as easy as setting a configuration knob? Or is it also possible to switch to a "bare-metal edition" of cephadm (which might rely on users or existing configuration management to install the packages, e.g.)?
Cheers, Oliver
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg). Geschäftsführer: Felix Imendörffer
Am 03.07.20 um 10:00 schrieb Sebastian Wagner:
Am 02.07.20 um 19:57 schrieb Oliver Freyermuth:
Dear Cephalopodians,
as we all know, ceph-deploy is on its demise since a while and essentially in "maintenance mode".
We've been eyeing the "ssh orchestrator" which was in Nautilus as the "successor in spirit" of ceph-deploy. While we have not tried it out just yet, I find this module seems to be gone without a trace in Octopus. There's still an Orchestrator module, but this seems to work "only" with containers.
Is this true, or is there still an SSH orchestrator capable of bare-metal operation in Octopus (or are there plans to have something like this)?
While I see many advantages of containers in many areas, and certainly also for smaller setups or test setups with Ceph, as any technology, they come with their own problems. Example issues (which all can be solved, but require extra work from the administrator) are: - Operation on machines without connectivity to the internet (you'd need to mirror the containers or run your own registry), - Ensuring automated security updates both outside the containers and inside the containers, or re-pull them regularly (and monitor that), - Integrate with existing logging and configuration management systems, - Potential hardware issues, such das InfiniBand RDMA.
There's surely more (and there are also as many benefits), and as I said, all can be solved; the point I want to make is: Containers are not the best solution in all environments and also not for all admins.
So my question is: Is there something like the SSH orchestrator still available? I guess essentially the cephadm orchestrator does something similar behind the screnes, with the added bells and whistles to manage the containers. Of course, a reduced feature-set would be expected (e.g. no "ceph orch upgrade"), but it would jump into the hole ceph-deploy has left.
we're renamed the SSH orchestrator into cephadm. See
https://github.com/ceph/ceph/pull/32193 for the corresponding pull request.
Thanks, that explains where it went :-). However, from the linked PR, it seems also the SSH orchestrator was using containers behind the scenes (it just was not clarified in the docs of good old Nautilus). So my main question remains — is there a successor in spirit to ceph-deploy for those who'd prefer to run bare-metal (containers are an answer to many problems, but not all, and they bring their own)? Cheers, Oliver
Hope that helps,
Sebastian
Maybe this is as easy as setting a configuration knob? Or is it also possible to switch to a "bare-metal edition" of cephadm (which might rely on users or existing configuration management to install the packages, e.g.)?
Cheers, Oliver
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
You can try to use ceph-ansible which supports baremetal and containerized deployment. https://github.com/ceph/ceph-ansible
Am 03.07.20 um 20:29 schrieb Dimitri Savineau:
You can try to use ceph-ansible which supports baremetal and containerized deployment.
Thanks for the pointer! I know about ceph-ansible. The problem is that our full infrastructure is Puppet-based, so mixing in a different configuration management will increase complexity (while ceph-deploy is really good at filling the gaps, ceph-ansible seems overkill for us). Additionally, all existing users of ceph-ansible I have talked to and asked about their experiences have responded with a heavy sigh and painful face, mentioning there were regular issues during usage, so I am reluctant to try it out with zero ansible experience myself. Of course, no solution ever is perfect, but I really believe a classic "unix-style" solution (i.e. being modular) which can be combined with others is essential for any project. If there are only full-fledged "all-in-one" solutions such as Ansible or container orchestration, this may be good for many cases, but can never cover all user requirements, since it can't be integrated into existing workflows and infrastructures as well as a lightweight module I can either use in full or only use parts of. To me, that's part of the Open Source freedom. Cheers, Oliver
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
+1 from me I also hope for a bare metal solution for the upcoming versions. At the moment it is a show stopper for an upgrade to Octopus. Thanks everybody involved for the great storage solution! Cheers, Lars Am Fri, 3 Jul 2020 20:44:02 +0200 schrieb Oliver Freyermuth <freyermuth@physik.uni-bonn.de>:
Of course, no solution ever is perfect, but I really believe a classic "unix-style" solution (i.e. being modular) which can be combined with others is essential for any project. If there are only full-fledged "all-in-one" solutions such as Ansible or container orchestration, this may be good for many cases, but can never cover all user requirements, since it can't be integrated into existing workflows and infrastructures as well as a lightweight module I can either use in full or only use parts of. To me, that's part of the Open Source freedom.
Cheers, Oliver
-- Informationstechnologie Berlin-Brandenburgische Akademie der Wissenschaften Jägerstraße 22-23 10117 Berlin Tel.: +49 30 20370-352 http://www.bbaw.de
Hello, Ceph users, Lars Täuber wrote: : +1 from me : : I also hope for a bare metal solution for the upcoming versions. At the moment it is a show stopper for an upgrade to Octopus. Also +1. Not having a unix-style deployment tool is a show-stopper for me as well. So far I keep my servers on Nautilus for this reason. -Yenya : Am Fri, 3 Jul 2020 20:44:02 +0200 : schrieb Oliver Freyermuth <freyermuth@physik.uni-bonn.de>: : : > Of course, no solution ever is perfect, but I really believe a classic "unix-style" solution (i.e. being modular) which can be combined with others : > is essential for any project. : > If there are only full-fledged "all-in-one" solutions such as Ansible or container orchestration, this may be good for many cases, : > but can never cover all user requirements, since it can't be integrated into existing workflows and infrastructures as well as a lightweight module : > I can either use in full or only use parts of. To me, that's part of the Open Source freedom. : > : > Cheers, : > Oliver : : : : -- : Informationstechnologie : Berlin-Brandenburgische Akademie der Wissenschaften : Jägerstraße 22-23 10117 Berlin : Tel.: +49 30 20370-352 http://www.bbaw.de : _______________________________________________ : ceph-users mailing list -- ceph-users@ceph.io : To unsubscribe send an email to ceph-users-leave@ceph.io -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | IORING_OP_NOP ... the benefits of doing nothing asynchronously are minimal, but sometimes a placeholder is useful. --Jonathan Corbet at LWN
Hi, On 03/07/2020 19:44, Oliver Freyermuth wrote:
Am 03.07.20 um 20:29 schrieb Dimitri Savineau:
You can try to use ceph-ansible which supports baremetal and containerized deployment.
Thanks for the pointer! I know about ceph-ansible. The problem is that our full infrastructure is Puppet-based, so mixing in a different configuration management will increase complexity (while ceph-deploy is really good at filling the gaps, ceph-ansible seems overkill for us).
Additionally, all existing users of ceph-ansible I have talked to and asked about their experiences have responded with a heavy sigh and painful face, mentioning there were regular issues during usage, so I am reluctant to try it out with zero ansible experience myself.
We're using ceph-ansible at the Sanger Institute, and are still pretty happy with it (by which I mean, we're not looking to change at least for our move to Nautilus); I gave a talk about our setup at Barcelona Cephalocon which is on YT somewhere... Regards, Matthew -- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.
participants (6)
-
Dimitri Savineau
-
Jan Kasprzak
-
Lars Täuber
-
Matthew Vernon
-
Oliver Freyermuth
-
Sebastian Wagner