7 Aug
2019
7 Aug
'19
11:16 p.m.
Hi everyone,
When I started setting up a test cluster, I created a cephfs_data pool
since I thought I was going to use the replicated type for the pool.
This turned out not to be to most ideal choice when it comes to usable
disk space. So I decided to create another erasure coded pool with
k=2;m=2 which I did. Please see the attached log file. I also attached
the output of the pools from the dashboard.
The problem is this. Whenever an OSD needs to be brought down, the
cluster is going to rebalance and enter the degraded mode. The rebalance
of the cluster takes a while since all the PG's need to be
redistributed. Now I could use something like:
|ceph osd ||set| |noout && ceph osd ||set| |norebalance|
However, I want to be able to have the cluster recovered much faster but
cephfs_data pool which has a lot of PG's which I cannot reduce (not able
in mimic) is keeping the cluster busy. This pool isn't used for storing
any data and I am not sure if I can remove this pool without effecting
the th-ec22 EC pool.
Anyone any thoughs on this?
In case I provided insufficient or missing information, please let me
know. I'd gladly share those with you.
--
Met vriendelijke groeten, Kind regards
Valentin Bajrami
Target Holding
8 Aug
8 Aug
12:13 a.m.
On Wed, Aug 7, 2019 at 11:46 AM Valentin Bajrami
<valentin.bajrami(a)target-holding.nl> wrote:
Hi everyone,
When I started setting up a test cluster, I created a cephfs_data pool since I thought I
was going to use the replicated type for the pool. This turned out not to be to most ideal
choice when it comes to usable disk space. So I decided to create another erasure coded
pool with k=2;m=2 which I did. Please see the attached log file. I also attached the
output of the pools from the dashboard.
The problem is this. Whenever an OSD needs to be brought down, the cluster is going to
rebalance and enter the degraded mode. The rebalance of the cluster takes a while since
all the PG's need to be redistributed. Now I could use something like:
ceph osd set noout && ceph osd set norebalance
However, I want to be able to have the cluster recovered much faster but cephfs_data pool
which has a lot of PG's which I cannot reduce (not able in mimic) is keeping the
cluster busy. This pool isn't used for storing any data and I am not sure if I can
remove this pool without effecting the th-ec22 EC pool.
Anyone any thoughs on this?
Is the cephfs_data pool associated with the CephFS at all? From the
"ceph df detail" it looks like it is, which means you unfortunately
can't delete it — the "root" CephFS data pool stores backpointers for
all the files in the system and those are needed to resolve hard
links, support disaster recovery, and do a few other things.
You can check this by looking at the FSMap in detail and seeing which
pools are which.
-Greg
>
> In case I provided insufficient or missing information, please let me know. I'd
gladly share those with you.
>
> --
> Met vriendelijke groeten, Kind regards
>
> Valentin Bajrami
> Target Holding
>
> _______________________________________________
> Dev mailing list -- dev(a)ceph.io
> To unsubscribe send an email to dev-leave(a)ceph.io
1:23 a.m.
Hi Gregory,
Thanks for the quick reply. I'll have to wait until nautilus is
realeased so I can reduce the PGs on this cephfs_data pool. I thought
there was another way of making this recover / PG rebalance on the
cluster much faster. I was planning to perform the following actions to
change the structure of my cluster.
1) Move data from the th-ec22 pool to a local storage and remove this
pool. Mainly because I want to move from crush-failure-domain=host to
crush-failure-domain=rack
2) Remove cephfs_data pool and re-create the pool with less PG's
3) Re-create th-ec22 pool with crush-failure-domain=rack so I can lose
one Datacenter where my OSD's are hosted.
4) Move data from local storage back to th-ec22 pool
Is this the proper course of action or are there easier / better ways to
achieve this?
Thanks again!
On 8/7/19 9:43 PM, Gregory Farnum wrote:
On Wed, Aug 7, 2019 at 11:46 AM Valentin Bajrami
<valentin.bajrami(a)target-holding.nl> wrote:
--
Met vriendelijke groeten,
Valentin Bajrami
Target Holding
Hi everyone,
When I started setting up a test cluster, I created a cephfs_data pool since I thought I
was going to use the replicated type for the pool. This turned out not to be to most ideal
choice when it comes to usable disk space. So I decided to create another erasure coded
pool with k=2;m=2 which I did. Please see the attached log file. I also attached the
output of the pools from the dashboard.
The problem is this. Whenever an OSD needs to be brought down, the cluster is going to
rebalance and enter the degraded mode. The rebalance of the cluster takes a while since
all the PG's need to be redistributed. Now I could use something like:
ceph osd set noout && ceph osd set norebalance
However, I want to be able to have the cluster recovered much faster but cephfs_data pool
which has a lot of PG's which I cannot reduce (not able in mimic) is keeping the
cluster busy. This pool isn't used for storing any data and I am not sure if I can
remove this pool without effecting the th-ec22 EC pool.
Anyone any thoughs on this?
Is the cephfs_data pool associated with the CephFS at
all? From the
"ceph df detail" it looks like it is, which means you unfortunately
can't delete it — the "root" CephFS data pool stores backpointers for
all the files in the system and those are needed to resolve hard
links, support disaster recovery, and do a few other things.
You can check this by looking at the FSMap in detail and seeing which
pools are which.
-Greg
> In case I provided insufficient or missing information, please let me know. I'd
gladly share those with you.
>
> --
> Met vriendelijke groeten, Kind regards
>
> Valentin Bajrami
> Target Holding
>
> _______________________________________________
> Dev mailing list -- dev(a)ceph.io
> To unsubscribe send an email to dev-leave(a)ceph.io
1734
days inactive
1734
days old
2 comments
2 participants
participants (2)
-
Gregory Farnum -
Valentin Bajrami