The default placement policy is to separate replicas across hosts, and you only have one host. You can create a new rule that replicas across osds with 'ceph osd crush rule create-replicated ...' and switch your data pool to that with 'ceph osd pool set crush_rule $pool $rulename'. s On Thu, 24 Oct 2019, Liu, Changcheng wrote:
Hi all, I'm using ceph-deploy tool to deploy single node cluster based on Ceph master branch(head id: bf09a04d2). The PGs are inactive and undersized. -bash-4.2$ ceph -s cluster: id: 93e5ff9c-1dee-4bd8-9b0a-318b917dfd8c health: HEALTH_WARN Reduced data availability: 4 pgs inactive Degraded data redundancy: 4 pgs undersized
services: mon: 1 daemons, quorum rdmarhel0 (age 30m) mgr: rdmarhel0(active, since 29m) osd: 3 osds: 3 up (since 25m), 3 in (since 25m)
data: pools: 1 pools, 4 pgs objects: 0 objects, 0 B usage: 3.0 GiB used, 834 GiB / 837 GiB avail pgs: 100.000% pgs not active 4 undersized+peered
-bash-4.2$
Other info is attached.
I apply below patch based on head-id:bf09a04d2. It's for Ceph/RDMA deployment. The above problem is hit whatever for RDMA-messenger deployment or Posix-TCP-messenger deployment. diff --git a/systemd/ceph-fuse@.service.in b/systemd/ceph-fuse@.service.in index d603042..ff2e907 100644 --- a/systemd/ceph-fuse@.service.in +++ b/systemd/ceph-fuse@.service.in @@ -12,6 +12,7 @@ ExecStart=/usr/bin/ceph-fuse -f --cluster ${CLUSTER} %I LockPersonality=true MemoryDenyWriteExecute=true NoNewPrivileges=true +LimitMEMLOCK=infinity # ceph-fuse requires access to /dev fuse device PrivateDevices=no ProtectControlGroups=true diff --git a/systemd/ceph-mds@.service.in b/systemd/ceph-mds@.service.in index 39a2e63..0e58dfe 100644 --- a/systemd/ceph-mds@.service.in +++ b/systemd/ceph-mds@.service.in @@ -14,7 +14,8 @@ ExecReload=/bin/kill -HUP $MAINPID LockPersonality=true MemoryDenyWriteExecute=true NoNewPrivileges=true -PrivateDevices=yes +LimitMEMLOCK=infinity +PrivateDevices=no ProtectControlGroups=true ProtectHome=true ProtectKernelModules=true diff --git a/systemd/ceph-mgr@.service.in b/systemd/ceph-mgr@.service.in index c98f637..682c7ec 100644 --- a/systemd/ceph-mgr@.service.in +++ b/systemd/ceph-mgr@.service.in @@ -18,7 +18,8 @@ LockPersonality=true MemoryDenyWriteExecute=false
NoNewPrivileges=true -PrivateDevices=yes +LimitMEMLOCK=infinity +PrivateDevices=no ProtectControlGroups=true ProtectHome=true ProtectKernelModules=true diff --git a/systemd/ceph-mon@.service.in b/systemd/ceph-mon@.service.in index c95fcab..51854fa 100644 --- a/systemd/ceph-mon@.service.in +++ b/systemd/ceph-mon@.service.in @@ -21,7 +21,8 @@ LockPersonality=true MemoryDenyWriteExecute=true # Need NewPrivileges via `sudo smartctl` NoNewPrivileges=false -PrivateDevices=yes +LimitMEMLOCK=infinity +PrivateDevices=no ProtectControlGroups=true ProtectHome=true ProtectKernelModules=true diff --git a/systemd/ceph-osd@.service.in b/systemd/ceph-osd@.service.in index 1b5c9c8..06c20d7 100644 --- a/systemd/ceph-osd@.service.in +++ b/systemd/ceph-osd@.service.in @@ -16,6 +16,8 @@ LockPersonality=true MemoryDenyWriteExecute=true # Need NewPrivileges via `sudo smartctl` NoNewPrivileges=false +LimitMEMLOCK=infinity +PrivateDevices=no ProtectControlGroups=true ProtectHome=true ProtectKernelModules=true diff --git a/systemd/ceph-radosgw@.service.in b/systemd/ceph-radosgw@.service.in index 7e3ddf6..fe1a6b9 100644 --- a/systemd/ceph-radosgw@.service.in +++ b/systemd/ceph-radosgw@.service.in @@ -13,7 +13,8 @@ ExecStart=/usr/bin/radosgw -f --cluster ${CLUSTER} --name client.%i --setuser ce LockPersonality=true MemoryDenyWriteExecute=true NoNewPrivileges=true -PrivateDevices=yes +LimitMEMLOCK=infinity +PrivateDevices=no ProtectControlGroups=true ProtectHome=true ProtectKernelModules=true diff --git a/systemd/ceph-volume@.service b/systemd/ceph-volume@.service index c21002c..e2d1f67 100644 --- a/systemd/ceph-volume@.service +++ b/systemd/ceph-volume@.service @@ -9,6 +9,7 @@ KillMode=none Environment=CEPH_VOLUME_TIMEOUT=10000 ExecStart=/bin/sh -c 'timeout $CEPH_VOLUME_TIMEOUT /usr/sbin/ceph-volume-systemd %i' TimeoutSec=0 +LimitMEMLOCK=infinity
[Install] WantedBy=multi-user.target -- 1.8.3.1