Hi, I have been curiuos about ceph for long time and now I started to experiment to find out, how it works. The idea I like most is, that ceph can provide growing storage without the need to move from storage x to storage y on consumer side. I started with a 3 node cluster where each node got one OSD (2x 500GB, 1x 1TB). I created the pool and the filesystem, mounted the filesystem and filled it with data. All three OSDs filled up evenly and when the pool reached 70%, I added two 1TB OSDs (each on the node with 500GB). Now I expected some activity to rebalance the fill level of the OSDs, but nothing special happened. The new OSDs got some data, but the 500GB OSDs ran into 95%, the pool reached 100%, everything got stuck with 2 OSDs filled up just 20% and left 80% free space? I searched the documentation how to initiate the rebalance/redistribution by hand but was unable to find anything. Can someone please point me to the docs how I can expand the capacity of the pool without such problems. Thanks in advance Thomas
On 5/30/24 08:53, tpDev Tester wrote:
Can someone please point me to the docs how I can expand the capacity of the pool without such problems.
Please show the output of ceph status ceph df ceph osd df tree ceph osd crush rule dump ceph osd pool ls detail Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin
Hi, Am 30.05.2024 um 08:58 schrieb Robert Sander:
... Please show the output of ceph ...
sorry for PM. Short update, I started from scratch with new Cluster, Reef instead of Quincy and this time I used a RBD instead of Filesystem for the first test and the rebalancing took place as expected. I will continue testing with a filesystem this afternoon, we will see how it works. Kind regards Thomas
participants (2)
-
Robert Sander
-
tpDev Tester