Migrate Jewel from leveldb to rocksdb
In the last release of Jewel [0] it mentions that omap data can be stored in rocksdb instead of leveldb. We are seeing high latencies from compaction of leveldb on our Jewel cluster (can't upgrade at this time). I installed the latest version, but apparently that is not enough to do the conversion. Is there a way to move OSDs from leveldb to rocksdb without trashing and rebuilding all the osds? [0] https://docs.ceph.com/docs/master/releases/jewel/ Thanks, Robert LeBlanc ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
Den ons 22 jan. 2020 kl 16:30 skrev Robert LeBlanc <robert@leblancnet.us>:
In the last release of Jewel [0] it mentions that omap data can be stored in rocksdb instead of leveldb. We are seeing high latencies from compaction of leveldb on our Jewel cluster (can't upgrade at this time). I installed the latest version, but apparently that is not enough to do the conversion. Is there a way to move OSDs from leveldb to rocksdb without trashing and rebuilding all the osds?
If you are running something else than CentOS there seems to be a way. For some reason CentOS jewel lacks snappy compression for its rocksdb, but the tool will make such a rocksdb anyhow. https://ceph-users.ceph.narkive.com/Znl1HyKq/any-backfill-in-our-cluster-mak... -- May the most significant bit of your life be positive.
Hi, There is no need to rebuild all ODS. You can follow the procedure described by RedHat[0] to convert the DB and tell the OSD to use rocksdb. Couldn't find this documented elsewhere. You may need a RedHat account to access the content, but you could create a developer one for free IIRC. [0] https://access.redhat.com/solutions/3210951 --- Alex Cucu On Wed, Jan 22, 2020 at 5:30 PM Robert LeBlanc <robert@leblancnet.us> wrote:
In the last release of Jewel [0] it mentions that omap data can be stored in rocksdb instead of leveldb. We are seeing high latencies from compaction of leveldb on our Jewel cluster (can't upgrade at this time). I installed the latest version, but apparently that is not enough to do the conversion. Is there a way to move OSDs from leveldb to rocksdb without trashing and rebuilding all the osds?
[0] https://docs.ceph.com/docs/master/releases/jewel/
Thanks, Robert LeBlanc ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On Wed, Jan 22, 2020 at 9:01 AM Alexandru Cucu <me@alexcucu.ro> wrote:
Hi,
There is no need to rebuild all ODS. You can follow the procedure described by RedHat[0] to convert the DB and tell the OSD to use rocksdb. Couldn't find this documented elsewhere. You may need a RedHat account to access the content, but you could create a developer one for free IIRC.
Thanks guys, I was able to convert some OSDs to rocksdb just fine (the ceph-kvstore-tool was in the ceph-test debian package). I'm going to let that run for a few days on one host before converting the others. I really hope this helps with the heavy write/delete load kicking out OSDs. ---------------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
participants (3)
-
Alexandru Cucu
-
Janne Johansson
-
Robert LeBlanc