As mon should be deployed in odd numbers, and I have a fourth node, can I deploy a fourth mds only? - 14.2.7
Presently I have about 1.2B objects (400M w/3 Replicas) and I'm finding the PG scrubbing and deep scrubbing are not completing. There is only 1 client accessing the data, a Samba server. I found large disparities in PG Distribution and Drive utilization. I enabled pg_autoscaler and found that it was reducing the number of PGs per OSD from 116 to 104.5 at this time, but it wasn't helping with space consumption equalization. I found loadbalancer and enabled that and it is in the process of evening out. As we were also having mds crashes even after increasing mds memory, i tried enabling multimds with rank:2 I currently have only 1 spare and would like to potentially enable the mds component on the fourth node (no mon present) but am having some difficulty. Is mon a requirement? I tried ceph-deploy mds create node4 but am having errors. I tried manually creating the /var/lib/ceph/mds/node4 directory and the command to create the keyring but still no joy. What am i missing? Thanks,
Please show the exact errors you are getting. You should not need a mon on the same node to create an mds, though you will want an /etc/ceph/ceph.conf that points to the mons. On Fri, Feb 7, 2020 at 2:58 PM <marcopizzolo@gmail.com> wrote:
Presently I have about 1.2B objects (400M w/3 Replicas) and I'm finding the PG scrubbing and deep scrubbing are not completing. There is only 1 client accessing the data, a Samba server. I found large disparities in PG Distribution and Drive utilization. I enabled pg_autoscaler and found that it was reducing the number of PGs per OSD from 116 to 104.5 at this time, but it wasn't helping with space consumption equalization. I found loadbalancer and enabled that and it is in the process of evening out. As we were also having mds crashes even after increasing mds memory, i tried enabling multimds with rank:2
I currently have only 1 spare and would like to potentially enable the mds component on the fourth node (no mon present) but am having some difficulty.
Is mon a requirement?
I tried ceph-deploy mds create node4 but am having errors. I tried manually creating the /var/lib/ceph/mds/node4 directory and the command to create the keyring but still no joy.
What am i missing?
Thanks, _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thanks Nathan, Having worked on this a bit since I did make some progress: [prdceph04][DEBUG ] connected to host: prdceph04 [prdceph04][DEBUG ] detect platform information from remote host [prdceph04][DEBUG ] detect machine type [ceph_deploy.mds][INFO ] Distro info: CentOS Linux 7.7.1908 Core [ceph_deploy.mds][DEBUG ] remote host will use systemd [ceph_deploy.mds][DEBUG ] deploying mds bootstrap to prdceph04 [prdceph04][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [prdceph04][WARNIN] mds keyring does not exist yet, creating one [prdceph04][DEBUG ] create a keyring file [prdceph04][DEBUG ] create path if it doesn't exist [prdceph04][INFO ] Running command: ceph --cluster ceph --name client.bootstrap-mds --keyring /var/lib/ceph/bootstrap-mds/ceph.keyring auth get-or-create mds.prdceph04 osd allow rwx mds allow mon allow profile mds -o /var/lib/ceph/mds/ceph-prdceph04/keyring [prdceph04][INFO ] Running command: systemctl enable ceph-mds@prdceph04 [prdceph04][WARNIN] Created symlink from /etc/systemd/system/ceph-mds.target.wants/ceph-mds@prdceph04.service to /usr/lib/systemd/system/ceph-mds@.service. [prdceph04][INFO ] Running command: systemctl start ceph-mds@prdceph04 [prdceph04][INFO ] Running command: systemctl enable ceph.target However i don't see it in dashboard or in ceph -s: health: HEALTH_WARN 1 pools have many more objects per pg than average Degraded data redundancy: 1351072/1169191146 objects degraded (0.116%), 39 pgs degraded, 39 pgs undersized 108 pgs not deep-scrubbed in time 12 pgs not scrubbed in time services: mon: 3 daemons, quorum prdceph01,prdceph02,prdceph03 (age 2h) mgr: prdceph01(active, since 2h), standbys: prdceph03, prdceph04, prdceph02 mds: ArchiveRepository:2 {0=prdceph03=up:active,1=prdceph02=up:active} 1 up:standby osd: 240 osds: 240 up (since 2h), 240 in; 323 remapped pgs data: pools: 7 pools, 8383 pgs objects: 389.73M objects, 460 TiB usage: 1.4 PiB used, 763 TiB / 2.1 PiB avail pgs: 1351072/1169191146 objects degraded (0.116%) 11416275/1169191146 objects misplaced (0.976%) 8054 active+clean 242 active+remapped+backfill_wait 42 active+remapped+backfilling 29 active+undersized+degraded+remapped+backfill_wait 10 active+undersized+degraded+remapped+backfilling 6 active+clean+scrubbing+deep
I think i have it working now... I'm sure i missed something early on that would have made this simpler. Thanks Nathan all the same.
participants (3)
-
Marco Pizzolo
-
marcopizzoloï¼ gmail.com
-
Nathan Fish