Hi, On 5/16/24 17:44, Matthew Vernon wrote:

cephadm --image docker-registry.wikimedia.org/ceph shell

...but is there a good way to arrange for cephadm to use the already-downloaded image without having to remember to specify --image each time?

You could create a shell alias: alias cephshell="cephadm --image docker-registry.wikimedia.org/ceph shell"

* OS reimages

We do OS upgrades by reimaging the server (which doesn't touch the storage disks); on an old-style deployment you could then use ceph-volume to re-start the OSDs and away you went; how does one do this in a cephadm cluster? [I presume involves telling cephadm to download a new image for podman to use and suchlike]

cephadm osd activate HOST would re-activate the OSDs. Before doing maintenance on a host run ceph orch host maintenance enter HOST and the orchestrator will stop the OSDs and set them to noout and will try to move other services away from the host if possible. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin

At least for the current up-to-date reef branch (not sure what reef version you're on) when --image is not provided to the shell, it should try to infer the image in this order 1. from the CEPHADM_IMAGE env. variable 2. if you pass --name with a daemon name to the shell command, it will try to get the image that daemon uses 3. next it tries to find the image being used by any ceph container on the host 4. The most recently built ceph image it can find on the host (by CreatedAt metadata field for the image) There is a `ceph cephadm osd activate <host>` command that is meant to do something similar in terms of OSD activation. If I'm being honest I haven't looked at it in some time, but it does have some CI test coverage via https://github.com/ceph/ceph/blob/main/qa/suites/orch/cephadm/osds/2-ops/rmd... On Thu, May 16, 2024 at 11:45 AM Matthew Vernon <mvernon@wikimedia.org> wrote:

Hi,

I've some experience with Ceph, but haven't used cephadm much before, and am trying to configure a pair of reef clusters with cephadm. A couple of newbie questions, if I may:

* cephadm shell image

I'm in an isolated environment, so pulling from a local repository. I bootstrapped OK with cephadm --image docker-registry.wikimedia.org/ceph bootstrap ...

And that worked nicely, but if I want to run cephadm shell (to do any sort of admin), then I have to specify cephadm --image docker-registry.wikimedia.org/ceph shell

(otherwise it just hangs failing to talk to quay.io).

I found the docs, which refer to setting lots of other images, but not the one that cephadm uses:

https://docs.ceph.com/en/reef/cephadm/install/#deployment-in-an-isolated-env...

I found an old tracker in this area: https://tracker.ceph.com/issues/47274

...but is there a good way to arrange for cephadm to use the already-downloaded image without having to remember to specify --image each time?

* OS reimages

We do OS upgrades by reimaging the server (which doesn't touch the storage disks); on an old-style deployment you could then use ceph-volume to re-start the OSDs and away you went; how does one do this in a cephadm cluster? [I presume involves telling cephadm to download a new image for podman to use and suchlike]

Would the process be smoother if we arranged to leave /var/lib/ceph intact between reimages?

Thanks,

Matthew _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io