August 2023 - Dev - lists.ceph.io

by Casey Bodley

hey Gal and Eric, in today's standup, we discussed the version of our apache arrow submodule. it's currently pinned at 6.0.1, which was tagged in nov. 2021. the centos9 builds are using the system package libarrow-devel-9.0.0. arrow's upstream recently tagged an 11.0.0 release as far as i know, there still aren't any system packages for ubuntu, so we're likely to be stuck with the submodule for quite a while. how do guys want to handle these updates? is it worth trying to update before the reef release?

2 months, 2 weeks

4
7
0 0

Re: [ceph-users] Re: Debian/bullseye build for reef

by Josh Durgin

We weren't targeting bullseye once we discovered the compiler version problem, the focus shifted to bookworm. If anyone would like to help maintaining debian builds, or looking into these issues, it would be welcome: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1030129 https://tracker.ceph.com/issues/61845 On Mon, Aug 21, 2023 at 7:50 AM Matthew Darwin <bugs(a)mdarwin.ca> wrote: > Thanks for the link to the issue. Any reason it wasn't added to the > release notes (for bullseye). > > I am also waiting for this to be available to start testing. > On 2023-08-21 10:25, Josh Durgin wrote: > > There was difficulty building on bullseye due to the older version of GCC > available: https://tracker.ceph.com/issues/61845 > > On Mon, Aug 21, 2023 at 3:01 AM Chris Palmer <chris.palmer(a)idnet.com> <chris.palmer(a)idnet.com> wrote: > > > I'd like to try reef, but we are on debian 11 (bullseye). > In the ceph repos, there is debian-quincy/bullseye and > debian-quincy/focal, but under reef there is only focal & jammy. > > Is there a reason why there is no reef/bullseye build? I had thought > that the blocker only affected debian-bookworm builds. > > Thanks, Chris > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > >

7 months, 3 weeks

2
2
0 0

"issue pool application warning even if pool is empty" change

by Ilya Dryomov

Hello, What was the motivation for the "issue pool application warning even if pool is empty" change [1]? This didn't occur to me what I saw the PR, but it basically makes it impossible to create a pool without HEALTH_WARN popping up. The reason is that pool creation and pool application enablement are separate monitor commands: there is no way to create a pool and enable an application on a pool atomically. And this is why this health check has always been limited to in-use pools (also pointed out by Greg when it was being introduced [2]). [1] https://github.com/ceph/ceph/pull/47560 [2] https://github.com/ceph/ceph/pull/15763#discussion_r123084421 Thanks, Ilya

7 months, 4 weeks

4
10
0 0

v16.2.14 Pacific released

by Yuri Weinstein

We're happy to announce the 14th backport release in the Pacific series. https://ceph.io/en/news/blog/2023/v16-2-14-pacific-released/ Notable Changes --------------- * CEPHFS: After recovering a Ceph File System post following the disaster recovery procedure, the recovered files under lost+found directory can now be deleted. * ceph mgr dump command now displays the name of the mgr module that registered a RADOS client in the name field added to elements of the active_clients array. Previously, only the address of a module's RADOS client was shown in the active_clients array. Getting Ceph ------------ * Git at git://github.com/ceph/ceph.git * Tarball at https://download.ceph.com/tarballs/ceph-16.2.14.tar.gz * Containers at https://quay.io/repository/ceph/ceph * For packages, see https://docs.ceph.com/en/latest/install/get-packages/ * Release git sha1: 238ba602515df21ea7ffc75c88db29f9e5ef12c9

8 months

1
0
0 0

CLT Meeting minutes 2023-08-30

by Nizamudeen A

Hello, Finish v18.2.0 upgrade on LRC? It seems to be running v18.1.3 not much of a difference in code commits news on teuthology jobs hanging? cephfs issues because of network troubles Its resolved by Patrick User council discussion follow-up Detailed info on this pad: https://pad.ceph.com/p/user_dev_relaunch First topic will come from David's team 16.2.14 release Pushing to release by this week. Regards, Nizam -- Nizamudeen A Software Engineer Red Hat <https://www.redhat.com/> <https://www.redhat.com/>

8 months

1
0
0 0

bluestore_compression_min_blob_size vs. bluestore_compression_min_blob_size_<type>

by Tony Liu

Hi, With Pacific and afterwards, which of the following two settings takes precedence? bluestore_compression_min_blob_size (default 0) bluestore_compression_min_blob_size_<type> (default 8192) I'd assume the more specific setting takes precedence? What's this "blob"? Is it write block? Then, there is no compression for 4K write block? I was not able to find it out from doc, hope someone could help to clarify. Thanks! Tony

8 months

3
7
0 0

16.2.14 pacific QE validation status

by Yuri Weinstein

Details of this release are summarized here: https://tracker.ceph.com/issues/62527#note-1 Release Notes - TBD Seeking approvals for: smoke - Venky rados - Radek, Laura rook - Sébastien Han cephadm - Adam K dashboard - Ernesto rgw - Casey rbd - Ilya krbd - Ilya fs - Venky, Patrick upgrade/pacific-p2p - Laura powercycle - Brad (SELinux denials) Thx YuriW

8 months

10
17
0 0

rbd export with export-format 2 exports all snapshots?

by Tony Liu

Hi, Say, source image has snapshot s1, s2 and s3. I expect "export" behaves the same as "deep cp", when specify a snapshot, with "--export-format 2", only the specified snapshot and all snapshots earlier than that will be exported. What I see is that, no matter which snapshot I specify, "export" with "--export-format 2" always exports the whole image with all snapshots. Is this expected? Could anyone help to clarify? Thanks! Tony

8 months

2
2
0 0

rbd export-diff/import-diff hangs

by Tony Liu

Hi, I'm using rbd import and export to copy image from one cluster to another. Also using import-diff and export-diff to update image in remote cluster. For example, "rbd --cluster local export-diff ... | rbd --cluster remote import-diff ...". Sometimes, the whole command is stuck. I can't tell it's stuck on which end of the pipe. I did some search, [1] seems the same issue and [2] is also related. Wonder if there is any way to identify where it's stuck and get more debugging info. Given [2], I'd suspect the import-diff is stuck, cause rbd client is importing to the remote cluster. Networking latency could be involved here? Ping latency is 7~8 ms. Any comments is appreciated! [1] https://bugs.launchpad.net/cinder/+bug/2031897 [2] https://stackoverflow.com/questions/69858763/ceph-rbd-import-hangs Thanks! Tony

8 months

1
1
0 0

mgr down : OSDMap.cc: 5155: FAILED ceph_assert(osd_weight.count(oid)

by xxbirds＠naver.com

When ceph osd reweight <osd> 0 is executed, mgrs go down and after several retries, mgrs eventually go down. In mgr.log and crashinfo, the following logs are displayed. BUILD/ceph-17.2.3/src/osd/OSDMap.cc: 5155: FAILED ceph_assert(osd_weight.count(oid)) There seems to be a problem with the ceph_assert(osd_weight.count(oid)) part. Anyone helps me. # ceph crash info { "assert_condition": "osd_weight.count(oid)", "assert_file": "/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.3/rpm/el8/BUILD/ceph-17.2.3/src/osd/OSDMap.cc", "assert_func": "float OSDMap::calc_deviations(ceph::common::CephContext*, const std::map<int, std::set<pg_t> >&, const std::map<int, float>&, float, std::map<int, float>&, std::multimap<float, int>&, float&)", "assert_line": 5155, "assert_msg": "/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.3/rpm/el8/BUILD/ceph-17.2.3/src/osd/OSDMap.cc: In function 'float OSDMap::calc_deviations(ceph::common::CephContext*, const std::map<int, std::set<pg_t> >&, const std::map<int, float>&, float, std::map<int, float>&, std::multimap<float, int>&, float&)' thread 7febe3a6f700 time 2023-08-22T10:27:57.207040+0000\n/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.3/rpm/el8/BUILD/ceph-17.2.3/src/osd/OSDMap.cc: 5155: FAILED ceph_assert(osd_weight.count(oid))\n", "assert_thread_name": "balancer", "backtrace": [ "/lib64/libpthread.so.0(+0x12ce0) [0x7fec4fb2bce0]", "gsignal()", "abort()", "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fec50d0ccd2]", "/usr/lib64/ceph/libceph-common.so.2(+0x283e95) [0x7fec50d0ce95]", "(OSDMap::calc_deviations(ceph::common::CephContext*, std::map<int, std::set<pg_t, std::less<pg_t>, std::allocator<pg_t> >, std::less<int>, std::allocator<std::pair<int const, std::set<pg_t, std::less<pg_t>, std::allocator<pg_t> > > > > const&, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > > const&, float, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > >&, std::multimap<float, int, std::less<float>, std::allocator<std::pair<float const, int> > >&, float&)+0xe0) [0x7fec511d9490]", "(OSDMap::calc_pg_upmaps(ceph::common::CephContext*, unsigned int, int, std::set<long, std::less<long>, std::allocator<long> > const&, OSDMap::Incremental*, unsigned int*)+0x389) [0x7fec511dd829]", "/usr/bin/ceph-mgr(+0x299406) [0x55e77ab23406]", "/lib64/libpython3.6m.so.1.0(+0x19d0d7) [0x7fec51c400d7]", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.6m.so.1.0(+0xf9984) [0x7fec51b9c984]", "/lib64/libpython3.6m.so.1.0(+0x17a030) [0x7fec51c1d030]", "/lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fec51c40377]", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.6m.so.1.0(+0x179e48) [0x7fec51c1ce48]", "/lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fec51c40377]", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.6m.so.1.0(+0x179e48) [0x7fec51c1ce48]", "/lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fec51c40377]", "_PyEval_EvalFrameDefault()", "/lib64/libpython3.6m.so.1.0(+0xfa2f6) [0x7fec51b9d2f6]", "_PyFunction_FastCallDict()", "_PyObject_FastCallDict()", "/lib64/libpython3.6m.so.1.0(+0x10db30) [0x7fec51bb0b30]", "_PyObject_FastCallDict()", "PyObject_CallMethod()", "(PyModuleRunner::serve()+0x66) [0x55e77ab1ee06]", "(PyModuleRunner::PyModuleRunnerThread::entry()+0x3e3) [0x55e77ab20443]", "/lib64/libpthread.so.0(+0x81ca) [0x7fec4fb211ca]", "clone()" ], "ceph_version": "17.2.3", # mgr.log 2023-08-22T11:06:56.450+0000 7fbf43d5e700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.3/rpm/el8/BUILD/ceph-17.2.3/src/osd/OSDMap.cc: In function 'float OSDMap::calc_deviations(ceph::common::CephContext*, const std::map<int, std::set<pg_t> >&, const std::map<int, float>&, float, std::map<int, float>&, std::multimap<float, int>&, float&)' thread 7fbf43d5e700 time 2023-08-22T11:06:56.450708+0000 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.3/rpm/el8/BUILD/ceph-17.2.3/src/osd/OSDMap.cc: 5155: FAILED ceph_assert(osd_weight.count(oid)) ceph version 17.2.3 (dff484dfc9e19a9819f375586300b3b79d80034d) quincy (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x7fbfadff5c74] 2: /usr/lib64/ceph/libceph-common.so.2(+0x283e95) [0x7fbfadff5e95] 3: (OSDMap::calc_deviations(ceph::common::CephContext*, std::map<int, std::set<pg_t, std::less<pg_t>, std::allocator<pg_t> >, std::less<int>, std::allocator<std::pair<int const, std::set<pg_t, std::less<pg_t>, std::allocator<pg_t> > > > > const&, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > > const&, float, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > >&, std::multimap<float, int, std::less<float>, std::allocator<std::pair<float const, int> > >&, float&)+0xe0) [0x7fbfae4c2490] 4: (OSDMap::calc_pg_upmaps(ceph::common::CephContext*, unsigned int, int, std::set<long, std::less<long>, std::allocator<long> > const&, OSDMap::Incremental*, unsigned int*)+0x389) [0x7fbfae4c6829] 5: /usr/bin/ceph-mgr(+0x299406) [0x55e50358a406] 6: /lib64/libpython3.6m.so.1.0(+0x19d0d7) [0x7fbfaef290d7] 7: _PyEval_EvalFrameDefault() 8: /lib64/libpython3.6m.so.1.0(+0xf9984) [0x7fbfaee85984] 9: /lib64/libpython3.6m.so.1.0(+0x17a030) [0x7fbfaef06030] 10: /lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fbfaef29377] 11: _PyEval_EvalFrameDefault() 12: /lib64/libpython3.6m.so.1.0(+0x179e48) [0x7fbfaef05e48] 13: /lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fbfaef29377] 14: _PyEval_EvalFrameDefault() 15: /lib64/libpython3.6m.so.1.0(+0x179e48) [0x7fbfaef05e48] 16: /lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fbfaef29377] 17: _PyEval_EvalFrameDefault() 18: /lib64/libpython3.6m.so.1.0(+0xfa2f6) [0x7fbfaee862f6] 19: _PyFunction_FastCallDict() 20: _PyObject_FastCallDict() 21: /lib64/libpython3.6m.so.1.0(+0x10db30) [0x7fbfaee99b30] 22: _PyObject_FastCallDict() 23: PyObject_CallMethod() 24: (PyModuleRunner::serve()+0x66) [0x55e503585e06] 25: (PyModuleRunner::PyModuleRunnerThread::entry()+0x3e3) [0x55e503587443] 26: /lib64/libpthread.so.0(+0x81ca) [0x7fbface0a1ca] 27: clone() 2023-08-22T11:06:56.450+0000 7fbf43d5e700 -1 *** Caught signal (Aborted) ** in thread 7fbf43d5e700 thread_name:balancer ceph version 17.2.3 (dff484dfc9e19a9819f375586300b3b79d80034d) quincy (stable) 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fbface14ce0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fbfadff5cd2] 5: /usr/lib64/ceph/libceph-common.so.2(+0x283e95) [0x7fbfadff5e95] 6: (OSDMap::calc_deviations(ceph::common::CephContext*, std::map<int, std::set<pg_t, std::less<pg_t>, std::allocator<pg_t> >, std::less<int>, std::allocator<std::pair<int const, std::set<pg_t, std::less<pg_t>, std::allocator<pg_t> > > > > const&, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > > const&, float, std::map<int, float, std::less<int>, std::allocator<std::pair<int const, float> > >&, std::multimap<float, int, std::less<float>, std::allocator<std::pair<float const, int> > >&, float&)+0xe0) [0x7fbfae4c2490] 7: (OSDMap::calc_pg_upmaps(ceph::common::CephContext*, unsigned int, int, std::set<long, std::less<long>, std::allocator<long> > const&, OSDMap::Incremental*, unsigned int*)+0x389) [0x7fbfae4c6829] 8: /usr/bin/ceph-mgr(+0x299406) [0x55e50358a406] 9: /lib64/libpython3.6m.so.1.0(+0x19d0d7) [0x7fbfaef290d7] 10: _PyEval_EvalFrameDefault() 11: /lib64/libpython3.6m.so.1.0(+0xf9984) [0x7fbfaee85984] 12: /lib64/libpython3.6m.so.1.0(+0x17a030) [0x7fbfaef06030] 13: /lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fbfaef29377] 14: _PyEval_EvalFrameDefault() 15: /lib64/libpython3.6m.so.1.0(+0x179e48) [0x7fbfaef05e48] 16: /lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fbfaef29377] 17: _PyEval_EvalFrameDefault() 18: /lib64/libpython3.6m.so.1.0(+0x179e48) [0x7fbfaef05e48] 19: /lib64/libpython3.6m.so.1.0(+0x19d377) [0x7fbfaef29377] 20: _PyEval_EvalFrameDefault() 21: /lib64/libpython3.6m.so.1.0(+0xfa2f6) [0x7fbfaee862f6] 22: _PyFunction_FastCallDict() 23: _PyObject_FastCallDict() 24: /lib64/libpython3.6m.so.1.0(+0x10db30) [0x7fbfaee99b30] 25: _PyObject_FastCallDict() 26: PyObject_CallMethod() 27: (PyModuleRunner::serve()+0x66) [0x55e503585e06] 28: (PyModuleRunner::PyModuleRunnerThread::entry()+0x3e3) [0x55e503587443] 29: /lib64/libpthread.so.0(+0x81ca) [0x7fbface0a1ca] 30: clone() NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

8 months

2
1
0 0

2024

2023

2022

2021

2020

2019

Dev August 2023