On Wednesday, October 30, 2024 2:00:56 PM EDT Darrell Enns wrote:
Is there a simple way to deploy a custom (in-house) mgr module to an orchestrator managed cluster? I assume the module code would need to be included in the mgr container image. However, there doesn't seem to be a straightforward way to do this without having the module merged to upstream ceph (not possible for a custom/in-house solution) or maintaining an in-house container repository and custom container images (a lot of extra maintenance overhead).
Also, what's the best way to handle testing during development? Custom scripts to push the code into to a mgr container in a dev cluster?
There's a tool in the ceph tree designed for this: src/script/cpatch.py (there's also an older shell based version in the same dir but that is not maintained WRT python changes as far as I know). There are some downsides to how this script works, as it creates many layers, but it's intended for development and in these cases having extra container layers is not usually a big deal. If you are working on a python based mgr module there's flags you can pass to have the script only create images with your local version of src/pybind/mgr (or whatnot, see the --help for more info). If you need extra help with the script, just ask here on the list. I use it frequently and have been maintaining it. When you build an image you can then push it to a private or public registry and configure ceph(adm) to use it. ( See https://docs.ceph.com/en/latest/ cephadm/install/#deployment-in-an-isolated-environment for some hints)