Perfect. Thanks! (adding a standby) peter Peter Eisch Senior Site Reliability Engineer T1.612.445.5135 virginpulse.com |virginpulse.com/global-challenge Australia | Bosnia and Herzegovina | Brazil | Canada | Singapore | Switzerland | United Kingdom | USA Confidentiality Notice: The information contained in this e-mail, including any attachment(s), is intended solely for use by the designated recipient(s). Unauthorized use, dissemination, distribution, or reproduction of this message by anyone other than the intended recipient(s), or a person designated as responsible for delivering such messages to the intended recipient, is strictly prohibited and may be unlawful. This e-mail may contain proprietary, confidential or privileged information. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Virgin Pulse, Inc. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. v2.64 From: Nathan Fish <lordcirth@gmail.com> Date: Monday, March 9, 2020 at 2:30 PM To: Peter Eisch <peter.eisch@virginpulse.com> Cc: "ceph-users@ceph.io" <ceph-users@ceph.io> Subject: Re: [ceph-users] Clear health warning Right, so you have 3 active MDS's and 0 on standby, which is generating a (correct) health warning. You need to either add more MDS' to be standbys, or reduce the fs to 2 active MDS'. On Mon, Mar 9, 2020 at 3:26 PM Peter Eisch <peter.eisch@virginpulse.com<mailto:peter.eisch@virginpulse.com>> wrote: Three MDS’ total, all serving the primary/original fs: [root@cephmon-s03 ]# ceph fs status cephfs1 - 8 clients ======= +------+--------+-------------+---------------+-------+-------+ | Rank | State | MDS | Activity | dns | inos | +------+--------+-------------+---------------+-------+-------+ | 0 | active | cephmon-03 | Reqs: 0 /s | 160 | 163 | | 1 | active | cephmon-02 | Reqs: 0 /s | 10 | 13 | | 2 | active | cephmon-01 | Reqs: 0 /s | 10 | 13 | +------+--------+-------------+---------------+-------+-------+ +---------------------+----------+-------+-------+ | Pool | type | used | avail | +---------------------+----------+-------+-------+ | stp.cephfs_metadata | metadata | 344M | 24.8T | | stp.cephfs_data | data | 11.1T | 24.8T | +---------------------+----------+-------+-------+ +-------------+ | Standby MDS | +-------------+ +-------------+ MDS version: ceph version 14.2.8 (2d095e947a02261ce61424021bb43bd3022d35cb) nautilus (stable) [root@cephmon-03 ]# Sorry for email formatting, I’m stuck with formatting options. peter Peter Eisch Senior Site Reliability Engineer T 1.612.445.5135<tel:1.612.445.5135> [Facebook]<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2FVirginPulse&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509805651&sdata=jAILHrtacLUNXXS%2FvONwnTkxE5GLNrWCoUVn%2FrotET0%3D&reserved=0> [LinkedIn]<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fvirgin-pulse&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509805651&sdata=j5fYA%2FTZsSQob3GN9wmQpAuJDZmyajQqQSJu2fDEfIo%3D&reserved=0> [Twitter]<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fvirginpulse&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509815645&sdata=gvi310d8OK7CQEfTOAyf7T6EIfMXjmdbMb%2B6cv0Dy2k%3D&reserved=0> virginpulse.com<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.virginpulse.com%2F&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509815645&sdata=f1d6lLvlgH8aZ8WM3P%2F%2BVpIxpKKZs9h2ddBYiUZ8ZIg%3D&reserved=0> | virginpulse.com/global-challenge<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.virginpulse.com%2Fen-gb%2Fglobal-challenge%2F&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509825645&sdata=6x4j6%2FdoaWr9V4F0r2tWqAQi2JUCsdgNmA%2BUHplQaDM%3D&reserved=0> Australia | Bosnia and Herzegovina | Brazil | Canada | Singapore | Switzerland | United Kingdom | USA Confidentiality Notice: The information contained in this e-mail, including any attachment(s), is intended solely for use by the designated recipient(s). Unauthorized use, dissemination, distribution, or reproduction of this message by anyone other than the intended recipient(s), or a person designated as responsible for delivering such messages to the intended recipient, is strictly prohibited and may be unlawful. This e-mail may contain proprietary, confidential or privileged information. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Virgin Pulse, Inc. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. v2.64 On Mon, Mar 9, 2020 at 2:17 PM Nathan Fish <lordcirth@gmail.com<mailto:lordcirth@gmail.com>> wrote: How many MDS's do you have total, and how are they assigned? 'ceph fs status'. On Mon, Mar 9, 2020 at 3:14 PM Peter Eisch <peter.eisch@virginpulse.com<mailto:peter.eisch@virginpulse.com>> wrote: Hi, (nautilus, 14.2.8, whole cluster) I doodled with adding a second cephfs and the project got canceled. I removed the unused cephfs with "ceph fs rm dream --yes-i-really-mean-it" and that worked as expected. I have a lingering health warning though which won't clear. The original cephfs1 volume exists and is healthy: [root@cephmon-03]# ceph fs ls name: cephfs1, metadata pool: stp.cephfs_metadata, data pools: [stp.cephfs_data ] [root@cephmon-03]# ceph mds stat cephfs1:3 {0=cephmon-03=up:active,1=cephmon-02=up:active,2=cephmon-01=up:active} [root@cephmon-03]# ceph health detail HEALTH_WARN insufficient standby MDS daemons available MDS_INSUFFICIENT_STANDBY insufficient standby MDS daemons available have 0; want 1 more [root@cephmon-03]# I have not yet deleted the pools for 'dream', the second cephfs definition. (There is nothing in it.) Before deleting the pools is there a command to clear this warning? Thanks! peter Peter Eisch Senior Site Reliability Engineer T 1.612.445.5135<tel:1.612.445.5135> [Facebook]<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2FVirginPulse&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509825645&sdata=45HnT91kxFoIvDJVwtiQ4Uey%2F1qOL4B8YB3AXzwqsv8%3D&reserved=0> [LinkedIn]<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fvirgin-pulse&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509835632&sdata=YMZk2rozrpYmfxmytZMwQr%2BENXc8Wx2ez61Cuybu6jc%3D&reserved=0> [Twitter]<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fvirginpulse&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509835632&sdata=9CP7Z5%2FM3kWl9YKAH3FlVcNSUZAztc5N%2BNGzsROjsOs%3D&reserved=0> virginpulse.com<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.virginpulse.com%2F&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509845627&sdata=gpikwWHNeyMIJgxVfvepevJSlpPZotFmP4P7Y9Sydrk%3D&reserved=0> | virginpulse.com/global-challenge<https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.virginpulse.com%2Fen-gb%2Fglobal-challenge%2F&data=02%7C01%7Cpeter.eisch%40virginpulse.com%7Ccb1e3b8211894fabee9308d7c4605e82%7Cb123a16e892b4cf6a55a6f8c7606a035%7C0%7C0%7C637193790509845627&sdata=rhbZN3Gvsoto%2B%2FYBa%2F56879vzVITtzVUwCxN0BJbYDw%3D&reserved=0> Australia | Bosnia and Herzegovina | Brazil | Canada | Singapore | Switzerland | United Kingdom | USA Confidentiality Notice: The information contained in this e-mail, including any attachment(s), is intended solely for use by the designated recipient(s). Unauthorized use, dissemination, distribution, or reproduction of this message by anyone other than the intended recipient(s), or a person designated as responsible for delivering such messages to the intended recipient, is strictly prohibited and may be unlawful. This e-mail may contain proprietary, confidential or privileged information. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Virgin Pulse, Inc. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. v2.64 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io<mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io<mailto:ceph-users-leave@ceph.io>
