August 2020 - Dev - lists.ceph.io

OSDs fail to start due to permissions on block.db device (Nautilus 14.2.11)

by Wyll Ingersoll

I have a cluster (14.2.11) with bluestore osds that have the block.DB on an SSD partition that is separate from the primary osd device. On some of my storage servers, the osd processes fail to start up at boot time because the permissions on the block.db device are not being changed from root:root, or are being reset by udev after the ceph-volume-systemd has run successfully. The problem only occurs on a couple of the storage servers, though all of them are configured the same and are running the same software versions. I suspect a race condition or conflict with the udev rules, but I have not been successful in identifying where the problem lies and udev is a complete nightmare to debug and diagnose. One workaround solution is to update /usr/lib/ceph/ceph-osd-prestart.sh so that it checks (and corrects) the permissions on the block.db device so that the osd can start correctly. This particular script looks like is hasn't been updated to support bluestore, so I added a some lines to address the problem, which works for me. Has anyone else seen a similar issue and found a different solution? Here is the code I added to the ceph-osd-prestart.sh script: ... blockdb="$data/block.db" if [ -L "$blockdb" -a -e "$blockdb" ]; then dev_db=`readlink -f $blockdb` owner=`stat -c %U $dev_db` if [ $owner != 'ceph' ]; then echo "ceph-osd(${cluster:-ceph}-$id): bluestore DB ($dev_db) has incorrect permissions, fixing." 1>&2 chown ceph:ceph $dev_db fi fi ... thanks, Wyllys Ingersoll

3 years, 8 months

1
0
0 0

Seastar cryptopp not found

by Abhinav Singh

Hello everyone, I m building ceph cmake command >cd build/ >cmake -DWITH_SEASTAR=ON .. but it is giving error with following message "could not find crptopp_LIBRARY and crytopp_INCLUDE" crytopp is defined in "SeastarDependencies.cmake" with other dependencies like protobuf(this is also not building), yaml-cpp etc I built this branch on latest ceph repo, any kind of hint will be helpful. Thanks. Regards

3 years, 8 months

1
1
0 0

RGW tracing

by Abhinav Singh

Hello everyone, My PR failing ceph-API test with message 2020-08-22T01:16:11.341+0000 7f1be492b540 -1 error: monitor data filesystem reached concerning levels of available storage space (available: 3% 13 GiB)you may adjust 'mon data avail crit' to a lower value to make this go away (default: 5%) What does it mean? and any suggestions how to overcome this? Thanks

3 years, 8 months

1
0
0 0

lua submodule

by Yuval Lifshitz

dear community, does anyone know why we have lua5.1 as a ceph submodule [1]? the latest lua is available for many linux distros (centos/rhel, ubuntu, fedora, opensuse, etc.), why not add it to install-deps.sh? Yuval [1] https://github.com/ceph/lua

3 years, 8 months

4
4
0 0

update crush map item weight

by Liu, Changcheng

Hi all, Could one item(device or bucket) under different bucket in one crushmap? Which case do we need this kind of configuration? I'm reading code to know how to create OSDMap. Some of the algorithm makes me confused. B.R. Changcheng

3 years, 8 months

1
0
0 0

RGW find_yaml:cpp target missing

by Abhinav Singh

Hello everyone, I have a flag WITH_JAEGER when true it loads library yaml-cpp and when not set it does not load yaml-cpp, when I m pushing this then jenkins ceph-make check fails with message CMake Error at src/seastar/CMakeLists.txt:411 (add_library): Target "seastar" links to target "yaml-cpp::yaml-cpp" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing? CMake Error at src/crimson/CMakeLists.txt:174 (add_library): Target "crimson" links to target "yaml-cpp::yaml-cpp" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing? I m not able to understand why is this happening, because these submmodules have their own cmake and their findyaml-cpp.cmake, so they should be independent with the flags set in global cmake isnt it? Also one thing I observed is that git not showing any changes in my submodules src/seastr but when I m opening CMakeLists.txt inside of this submodule in master branch in github they it is different then my local version of CMake, strange Thanks

3 years, 8 months

1
0
0 0

08/20/2020 perf meeting is on at 8AM PST!

by Mark Nelson

Hi Folks, The weekly performance meeting will be starting in about 15 minutes! Today, we are going to continue discussing the research paper Avnum Hanukov presented last week regarding the drawbacks of the CRUSH algorithm and the "MapX" crush extension. You can download the paper here: https://www.usenix.org/system/files/fast20-wang_li.pdf Hope to see you there! Etherpad: https://pad.ceph.com/p/performance_weekly Bluejeans: https://bluejeans.com/908675367 Thanks, Mark

3 years, 8 months

2
1
0 0

Jenkins Test Make Check failure FLEX

by Abhinav Singh

Hello everyone, I get this error in jenkins when I included thirft installation, ``` Could NOT find FLEX (missing: FLEX_EXECUTABLE) Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) /usr/share/cmake-3.10/Modules/FindFLEX.cmake:217 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) compiler/cpp/CMakeLists.txt:32 (find_package) ``` in my machine the build is successful, but not in jenkins

3 years, 8 months

1
0
0 0

RGWTracing

by Abhinav Singh

Hello everyone, I was rebasing my branch with remote ceph master and I found out that there is some changes in src/seaster submodule due to this my jenkins test is not passing, so I did ``` cd src/seaster git checkout master && git pull cd ../ git add git commit ``` but this is not working, the difference is still there, another way I was doing was ``` git checkout sr/seaster ceph/maser ``` this is also not working, how do I solve this problem? Thanks

3 years, 8 months

2
1
0 0

RocksDB in Ceph: column families, levels' size and spillover

by Kajetan Janiak

Hi everyone, We would like to share with you our thoughts on RocksDB in Ceph with main focus on efficient use of fast storage. Storage space provisioning for RocksDB is quite complex in Ceph as it includes several layers of abstraction. We highlight some problems while explaining how things are working under the hood. At the end, we propose several solutions to alleviate these problems. RocksDB in Ceph: column families, levels' size and spillover https://cf2.cloudferro.com:8080/swift/v1/AUTH_5e376cddf8a94f9294259b5f48d7b… We'd love to hear your feedback. Thanks -- *Kajetan Janiak* C++ Engineer CloudFerro sp z o.o. office: Fabryczna 5A *m:* +48 500 191 166 00-446 Warszawa, Poland *e:* kjaniak(a)cloudferro.com <https://cloudferro.com/>

3 years, 8 months

1
0
0 0

2024

2023

2022

2021

2020

2019

Dev August 2020