On 03/01 21:41, Dave Hall wrote: Yep, the systemd units are usually named 'ceph-<daemon>@<id>', for example 'ceph-osd@45' would be the systemd unit for osd.45. Also yes, and if you use some configuration management or similar that might bring them up automatically you might want to disable it temporarily too. Yes, that is correct. Monitor would be the MONs, and admin the MGRs. Can you share a bit more about your setup? Are you using replicas? How many? Erasure coding? (a ceph osd pool ls detail , ceph osd status or similar can help too). I would recommend trying to get the hand of the process in a test environment first. Cheers! -- David Caro SRE - Cloud Services Wikimedia Foundation <https://wikimediafoundation.org/> PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3 "Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment."

Hello Dave, I recommend you read this docu: https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/a… Regards, Joachim ___________________________________ Clyso GmbH - Ceph Foundation Member Am 02.03.2021 um 13:55 schrieb Dave Hall: > Dave, > > Just to be certain of the terminology, > > ----- > Step before Step 4: Quiesce client systems using Ceph > > Step 4: Turn off everything that's not a MGR, MON, or OSD. > > Step 5: Turn off OSDs > > Step 6: Turn off MONs > > Step 7: Turn off MGRs > > If any of the above are running on the the same nodes (i.e. mixed nodes), > use OS capabilities (systemd) to stop and disable so nothing auto-starts > when the hardware is powered back on. > ---- > > Regarding my cluster: Currently 3 nodes with 10GB front and back networks, > 8 x 12 TB HDDs per node with Samsung 1.6TB PCIe NVMe cards. The NVMe was > provisioned to allow adding 4 more HDDs per node, but the RocksDBs are > proving to be a bit too small. > > We will shortly increase to 6 OSD nodes plus 3 separate nodes for MGRs, > MONs, MDSs, RGWs, etc. We will also add Enterprise M.2 drives to the > original nodes to allow us to increase the size of the caches. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall(a)binghamton.edu > > On Tue, Mar 2, 2021 at 4:06 AM David Caro &lt;dcaro(a)wikimedia.org&gt; wrote: > >> On 03/01 21:41, Dave Hall wrote: >>> Hello, >>> >>> I've had a look at the instructions for clean shutdown given at >>> https://ceph.io/planet/how-to-do-a-ceph-cluster-maintenance-shutdown/, >> but >>> I'm not clear about some things on the steps about shutting down the >>> various Ceph components. >>> >>> For my current 3-node cluster I have MONs, MDSs, MGRs, and OSDs all >> running >>> on the same nodes. Also, this is a non-container installation. >>> >>> Since I don't have separate dedicated nodes, as described in the >> referenced >>> web page, I think the instructions mean that I need to issue SystemD >>> commands to stop the corresponding services/targets on each node for the >>> Ceph components mentioned in each step. >> Yep, the systemd units are usually named 'ceph-<daemon>@<id>', for example >> 'ceph-osd@45' would be the systemd unit for osd.45. >> >>> Since we want to bring services up in the right order, I should also use >>> SystemD commands to disable these services/targets so they don't >>> automatically restart when I power the nodes back on. After power-on, I >>> would then re-enable and manually start services/targets in the order >>> described. >> Also yes, and if you use some configuration management or similar that >> might >> bring them up automatically you might want to disable it temporarily too. >> >>> One other specific question: For step 4 it says to shut down my service >>> nodes. Does this mean my MDSs? (I'm not running any Object Gateways or >>> NFS, but I think these would go in this step as well?) >> Yes, that is correct. Monitor would be the MONs, and admin the MGRs. >> >>> Please let me know if I've got this right. The cluster contains 200TB >> of a >>> researcher's data that has taken a year to collect, so caution is needed. >> Can you share a bit more about your setup? Are you using replicas? How >> many? >> Erasure coding? (a ceph osd pool ls detail , ceph osd status or similar can >> help too). >> >> >> I would recommend trying to get the hand of the process in a test >> environment >> first. >> >> Cheers! >> >>> Thanks. >>> >>> -Dave >>> >>> -- >>> Dave Hall >>> Binghamton University >>> kdhall(a)binghamton.edu >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users(a)ceph.io >>> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> -- >> David Caro >> SRE - Cloud Services >> Wikimedia Foundation <https://wikimediafoundation.org/> >> PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3 >> >> "Imagine a world in which every single human being can freely share in the >> sum of all knowledge. That's our commitment." >> > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io