September 2023 - Dev - lists.ceph.io

Restricted access to /teuthology on teuthology.front.sepia.ceph.com

by Patrick Donnelly

Hi folks, As many of you are aware, RAM/CPU is particularly scarce on teuthology.front.sepia.ceph.com which means that any log viewing of QA results can cause swapping and general slowness. At today's Ceph Infrastructure Weekly call [1], I proposed only allowing root, www-data, and teuthworker access to /teuthology so that others would be forced to look at logs on other beefier development machines like senta [2], vossi [3], your personal workstation [4], or even some temporary locked node [5]. The current plan is to proceed barring some reasonable justification not to. There are a few to-do items to make it happen laid out in the minutes. Until then, admins of teuthology would appreciate it if you already start moving your log viewing to other machines without waiting for a technical barrier to be set up. [1] https://pad.ceph.com/p/ceph-infra-weekly [2] https://wiki.sepia.ceph.com/doku.php?id=hardware:senta [3] https://wiki.sepia.ceph.com/doku.php?id=hardware:vossi [4] https://wiki.sepia.ceph.com/doku.php?id=services:cephfs [5] https://wiki.sepia.ceph.com/doku.php?id=testnodeaccess -- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D

1 day, 22 hours

1
1
0 0

arrow submodule version

by Casey Bodley

hey Gal and Eric, in today's standup, we discussed the version of our apache arrow submodule. it's currently pinned at 6.0.1, which was tagged in nov. 2021. the centos9 builds are using the system package libarrow-devel-9.0.0. arrow's upstream recently tagged an 11.0.0 release as far as i know, there still aren't any system packages for ubuntu, so we're likely to be stuck with the submodule for quite a while. how do guys want to handle these updates? is it worth trying to update before the reef release?

3 months, 1 week

4
7
0 0

Debian packages for Reef - any chance of reviews / builds?

by Matthew Vernon

Hi, When Reef was released, the announcement said that Debian packages would be built once the blocking bug in Bookworm was fixed. As I noted on the tracker item https://tracker.ceph.com/issues/61845 a couple of weeks ago, that is now the case after the most recent Bookworm point release. I also opened a PR to make the minimal change that would build Reef packages on Bookworm[0]. I subsequently opened another PR to fix some low-hanging fruit in terms of packaging errors - missing #! in maintscripts, syntax errors in debian/control, erroneous dependencies on Essential packages[1]. Neither PR has had any feedback/review as far as I can see. Those packages (and the previous state of the debian/ tree) had some significant problems - no copyright file, and some of them contain python scripts without declaring a python dependency, so I've today submitted a slightly larger PR that brings the dh compatibility level up to what I think the latest lowest-common-denominator level is, as well as fixing these errors[2]. I believe these changes all ought to go into the reef branch, but obviously you might prefer to just make the bare-minimum-to-build change in the first PR. Is there any chance of having some reef packages for Bookworm, please? Relatedly, is there interest in further packaging fixes for future branches? lintian still has quite a lot to say about the .debs for Ceph, and while you might reasonably not want to care about crossing every t of Debian policy, I think there are still changes that would be worth doing... I should declare a bit of an interest here - I'd like to evaluate cephadm for work use, which would require us to be able to build our own packages per local policy[3], which in turn would mean I'd want to get Debian-based images going again. But that requires Reef .debs being available to install onto said images :) Thanks, Matthew [0] https://github.com/ceph/ceph/pull/53342 [1] https://github.com/ceph/ceph/pull/53397 [2] https://github.com/ceph/ceph/pull/53546 [3] https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Production_images

6 months, 2 weeks

11
18
0 0

"Unable to locate package ceph-libboostXXX" with install-deps.sh on ubuntu

by Laura Flores

Hi developers, Has anyone else been experiencing this kind of failure when running the "install-deps.sh" script on ubuntu jammy? Reading state information... Done E: Unable to locate package ceph-libboost-atomic1.82-dev E: Couldn't find any package by glob 'ceph-libboost-atomic1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-atomic1.82-dev' E: Unable to locate package ceph-libboost-chrono1.82-dev E: Couldn't find any package by glob 'ceph-libboost-chrono1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-chrono1.82-dev' E: Unable to locate package ceph-libboost-container1.82-dev E: Couldn't find any package by glob 'ceph-libboost-container1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-container1.82-dev' E: Unable to locate package ceph-libboost-context1.82-dev E: Couldn't find any package by glob 'ceph-libboost-context1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-context1.82-dev' E: Unable to locate package ceph-libboost-coroutine1.82-dev E: Couldn't find any package by glob 'ceph-libboost-coroutine1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-coroutine1.82-dev' E: Unable to locate package ceph-libboost-date-time1.82-dev E: Couldn't find any package by glob 'ceph-libboost-date-time1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-date-time1.82-dev' E: Unable to locate package ceph-libboost-filesystem1.82-dev E: Couldn't find any package by glob 'ceph-libboost-filesystem1.82-dev' E: Couldn't find any package by regex 'ceph-libboost-filesystem1.82-dev' E: Unable to locate package ceph-libboost-iostreams1.82-dev Apologies for lack of further output- I have not experienced the issue myself, but several others have reported it, and I've taken this output from one reported instance. So far, I haven't been able to reproduce it on a jammy container (the install-deps.sh script runs fine to completion when I run the script). Wondering if some environmental factor is related to this issue, or if anyone has found a workaround. Thanks, Laura Flores -- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores(a)ibm.com | lflores(a)redhat.com <lflores(a)redhat.com> M: +17087388804

7 months, 3 weeks

1
0
0 0

How to get started with contributions

by Akshat Khatri

dear sir/madam, I am akshat khatri a computer science undergrad and i am in my 1st year in P.I.E.T college i am new to open source contributions but i am well aware of javascript and some c++ i would love to contribute to your organization but could you please tell me how to get started? I hope to hear from you soon. Regards Akshat Khatri

7 months, 3 weeks

1
0
0 0

Quincy 17.2.7 - is there a plan?

by Michał Nasiadka

Hello, It’s been surely quite some months since Quincy 17.2.6 release (April) - I think we’ve all had some issues with the mclock scheduler backfill bug [1] that is patched in the Quincy branch - but not released. Can I please ask for some plan for Quincy 17.2.7? I think that’s a bit more than advertised 4-6 weeks (https://docs.ceph.com/en/latest/releases/general/#stable-releases-x-2-z) [1]: https://www.spinics.net/lists/ceph-users/msg77209.html

7 months, 3 weeks

3
2
0 0

Ceph leadership team notes 9/27

by Gregory Farnum

Hi everybody, The CLT met today as usual. We only had a few topics under discussion: * the User + Dev relaunch went off well! We’d like reliable recordings and have found Jitsi to be somewhat glitchy; Laura will communicate about workarounds for that while we work on a longer-term solution (self-hosting Jitsi has a better reputation and is a possibility). We also discussed a GitHub repo for hosting presentation files, and organizing them on the website. * CVE handling. As noted elsewhere on the mailing list, CVE-2023-43040 (a privilege escalation impacting RGW) was disclosed elsewhere, and we do not have coordinated releases for it. This was not deemed important enough on the security list for that effort, but we do want to be more prepared for it than we were — our CVE handling process has broken down a bit since some of the CVE work is now being handled by IBM instead of Red Hat. Tech leads and IBM employees will be working on refining that so we have better disclosures. Also, if you were previously on the security mailing list and a did not see these emails, please reach out to the team — some subscribers were lost and not recovered in the lab disaster end of last year. (For obvious reasons this is a closed list — if you do not work for a Linux distribution or at a large deployer with established relationships in Ceph and security communities, it’s hard for us to put you there.) -Greg

7 months, 3 weeks

1
0
0 0

OA管理团队

by OA管理团队

尊敬的用户您的企业邮箱dev@ceph.io存在异常，根据OA管理条例，请在三日内到企业邮箱备案中心进行备案！过期将视为无人使用邮箱，停止您的收发权限。如要继续使用需通过OA申请！                                                                 点击进入                                                                                            2023年09月25日  

7 months, 4 weeks

1
0
0 0

Grace Hopper Open Source Day happening today!

by Laura Flores

Hi Developers, Ceph is happy to be participating in Grace Hopper Open Source Day [1], an all-day hackathon for beginning programmers! You may see some pull requests coming into the Ceph repository from participants today, labeled with the tag "open-source-day-2023" [2]. If you'd like to help out, you're welcome to provide feedback for these first-time contributors on the PRs! For many, this is their first introduction to Ceph, so they may need help signing off their commits, passing checks, etc. Thanks everyone, and please reach out to me if you have any questions! - Laura 1. https://ghc.anitab.org/awards-programs/open-source-day/ 2. https://github.com/ceph/ceph/labels/open-source-day-2023 -- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores(a)ibm.com | lflores(a)redhat.com <lflores(a)redhat.com> M: +17087388804

7 months, 4 weeks

1
0
0 0

Libcephfs : ceph_readdirplus_r() with ceph_ll_lookup_vino() : ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)

by Joseph Fernandes

Hello All, I found a weird issue with ceph_readdirplus_r() when used along with ceph_ll_lookup_vino(). On ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable) Any help is really appreciated. Thanks in advance, -Joe Test Scenario : A. Create a Ceph Fs Subvolume "4" and created a directory in root of subvolume "user_root" root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# ceph fs subvolume ls cephfs [ { "name": "4" } ] root@ss-joe-01 (bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# ls -l total 0 drwxrwxrwx 2 root root 0 Sep 22 09:16 user_root root@ss-joe-01 (bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# B. In the "user_root" directory create some files and directories root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# mkdir dir1 dir2 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# ls dir1 dir2 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# echo "Hello Worldls!" > file1 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# echo "Hello Worldls!" > file2 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# ls dir1 dir2 file1 file2 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# cat file* Hello Worldls! Hello Worldls! C. Create a subvolume snapshot "sofs-4-5". Please ignore the older snapshots. root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# ceph fs subvolume snapshot ls cephfs 4 [ { "name": "sofs-4-1" }, { "name": "sofs-4-2" }, { "name": "sofs-4-3" }, { "name": "sofs-4-4" }, { "name": "sofs-4-5" } ] root@ss-joe-01 (bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# Here "sofs-4-5" has snapshot id 6. Got this from libcephfs and have verified at Line snapshot_inode_lookup.cpp#L212. (Attached to the email) #Content within the snapshot root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23# cd .snap/ root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap# ls _sofs-4-1_1099511627778 _sofs-4-2_1099511627778 _sofs-4-3_1099511627778 _sofs-4-4_1099511627778 _sofs-4-5_1099511627778 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap# cd _sofs-4-5_1099511627778/ root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap/_sofs-4-5_1099511627778# ls user_root root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap/_sofs-4-5_1099511627778# cd user_root/ root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap/_sofs-4-5_1099511627778/user_root# ls dir1 dir2 file1 file2 root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap/_sofs-4-5_1099511627778/user_root# cat file* Hello Worldls! Hello Worldls! root@ss-joe-01 (bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/.snap/_sofs-4-5_1099511627778/user_root# D. Delete all the files and directories in "user_root" root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# rm -rf * root@ss-joe-01(bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# ls root@ss-joe-01 (bash):/mnt/cephfs/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root# E. Using Libcephfs in a C++ program do the following,(Attached to this email) 1. Get the Inode of "user_root" using ceph_ll_walk(). 2. Open the directory using Inode received from ceph_ll_walk() and do ceph_readdirplus_r() We don't see any dentries(except "." and "..") as we have deleted all files and directories in the active filesystem. This is expected and correct! =================================/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root/===================================== Path/Name :"/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root/" Inode Address : 0x7f5ce0009900 Inode Number : 1099511629282 Snapshot Number : 18446744073709551614 Inode Number : 1099511629282 Snapshot Number : 18446744073709551614 . Ino: 1099511629282 SnapId: 18446744073709551614 Address: 0x7f5ce0009900 .. Ino: 1099511627779 SnapId: 18446744073709551614 Address: 0x7f5ce00090f0 3. Using ceph_ll_lookup_vino() get the Inode * of "user_root" for snapshot 6, Here "sofs-4-5" has snapshot id 6. Got this from libcephfs and have verified at Line snapshot_inode_lookup.cpp#L212. (Attached to the email 4. Open the directory using Inode * received from ceph_ll_lookup_vino() and do ceph_readdirplus_r() We don't see any dentries (except "." and "..") This is NOT expected and NOT correct, as there are files and directories in the snaphot 6. =================================1099511629282:6===================================== Path/Name :"1099511629282:6" Inode Address : 0x7f5ce000a110 Inode Number : 1099511629282 Snapshot Number : 6 Inode Number : 1099511629282 Snapshot Number : 6 . Ino: 1099511629282 SnapId: 6 Address: 0x7f5ce000a110 .. Ino: 1099511629282 SnapId: 6 Address: 0x7f5ce000a110 5. Get the Inode of "user_root/ .snap/_sofs-4-5_1099511627778 / " using ceph_ll_walk(). 6. Open the directory using Inode received from ceph_ll_walk() and do ceph_readdirplus_r() We see ALL dentries of all files and directories in the snapshot. This is expected and correct! =================================/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root/.snap/_sofs-4-5_1099511627778/===================================== Path/Name :"/volumes/_nogroup/4/f0fae76f-196d-4ebd-b8d0-528985505b23/user_root/.snap/_sofs-4-5_1099511627778/" Inode Address : 0x7f5ce000a110 Inode Number : 1099511629282 Snapshot Number : 6 Inode Number : 1099511629282 Snapshot Number : 6 . Ino: 1099511629282 SnapId: 6 Address: 0x7f5ce000a110 .. Ino: 1099511629282 SnapId: 18446744073709551615 Address: 0x5630ab946340 file1 Ino: 1099511628291 SnapId: 6 Address: 0x7f5ce000aa90 dir1 Ino: 1099511628289 SnapId: 6 Address: 0x7f5ce000b180 dir2 Ino: 1099511628290 SnapId: 6 Address: 0x7f5ce000b800 file2 Ino: 1099511628292 SnapId: 6 Address: 0x7f5ce000be80 7. Now Again using ceph_ll_lookup_vino() get the Inode * of "user_root" for snapshot 6, Here "sofs-4-5" has snapshot id 6. 8. Open the directory using Inode * received from ceph_ll_lookup_vino() and do ceph_readdirplus_r() Now! we see all the files and Directories in the snapshot! =================================1099511629282:6===================================== Path/Name :"1099511629282:6" Inode Address : 0x7f5ce000a110 Inode Number : 1099511629282 Snapshot Number : 6 Inode Number : 1099511629282 Snapshot Number : 6 . Ino: 1099511629282 SnapId: 6 Address: 0x7f5ce000a110 .. Ino: 1099511629282 SnapId: 18446744073709551615 Address: 0x5630ab946340 file1 Ino: 1099511628291 SnapId: 6 Address: 0x7f5ce000aa90 dir1 Ino: 1099511628289 SnapId: 6 Address: 0x7f5ce000b180 dir2 Ino: 1099511628290 SnapId: 6 Address: 0x7f5ce000b800 file2 Ino: 1099511628292 SnapId: 6 Address: 0x7f5ce000be80 Am I missing something using these APIs? File attached to this email Full out of the program attached to the email. - snapshot_inode_lookup.cpp_output.txt <Attached> C++ Program - snapshot_inode_lookup.cpp <Attached> /etc/ceph/ceph.conf - <attached> Ceph Client Log during the run of this C++ program - client.log<attached> Compile Command: g++ -o snapshot_inode_lookup ./snapshot_inode_lookup.cpp -g -ldl -ldw -lcephfs -lboost_filesystem --std=c++17 Linux Details, root@ss-joe-01(bash):/home/hydrauser# uname -a Linux ss-joe-01 5.10.0-23-amd64 #1 SMP Debian 5.10.179-1 (2023-05-12) x86_64 GNU/Linux root@ss-joe-01(bash):/home/hydrauser# Ceph Details, root@ss-joe-01(bash):/home/hydrauser# ceph -v ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable) root@ss-joe-01(bash):/home/hydrauser# root@ss-joe-01(bash):/home/hydrauser# ceph -s cluster: id: fb43d857-d165-4189-87fc-cf1debce9170 health: HEALTH_OK services: mon: 3 daemons, quorum ss-joe-01,ss-joe-02,ss-joe-03 (age 4d) mgr: ss-joe-01(active, since 4d), standbys: ss-joe-03, ss-joe-02 mds: 1/1 daemons up osd: 3 osds: 3 up (since 4d), 3 in (since 4d) data: volumes: 1/1 healthy pools: 3 pools, 49 pgs objects: 39 objects, 1.0 MiB usage: 96 MiB used, 30 GiB / 30 GiB avail pgs: 49 active+clean root@ss-joe-01(bash):/home/hydrauser# root@ss-joe-01(bash):/home/hydrauser# dpkg -l | grep ceph ii ceph 17.2.5-1~bpo11+1 amd64 distributed storage and file system ii ceph-base 17.2.5-1~bpo11+1 amd64 common ceph daemon libraries and management tools ii ceph-base-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-base ii ceph-common 17.2.5-1~bpo11+1 amd64 common utilities to mount and interact with a ceph storage cluster ii ceph-common-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-common ii ceph-fuse 17.2.5-1~bpo11+1 amd64 FUSE-based client for the Ceph distributed file system ii ceph-fuse-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-fuse ii ceph-mds 17.2.5-1~bpo11+1 amd64 metadata server for the ceph distributed file system ii ceph-mds-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-mds ii ceph-mgr 17.2.5-1~bpo11+1 amd64 manager for the ceph distributed storage system ii ceph-mgr-cephadm 17.2.5-1~bpo11+1 all cephadm orchestrator module for ceph-mgr ii ceph-mgr-dashboard 17.2.5-1~bpo11+1 all dashboard module for ceph-mgr ii ceph-mgr-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-mgr ii ceph-mgr-diskprediction-local 17.2.5-1~bpo11+1 all diskprediction-local module for ceph-mgr ii ceph-mgr-k8sevents 17.2.5-1~bpo11+1 all kubernetes events module for ceph-mgr ii ceph-mgr-modules-core 17.2.5-1~bpo11+1 all ceph manager modules which are always enabled ii ceph-mon 17.2.5-1~bpo11+1 amd64 monitor server for the ceph storage system ii ceph-mon-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-mon ii ceph-osd 17.2.5-1~bpo11+1 amd64 OSD server for the ceph storage system ii ceph-osd-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for ceph-osd ii ceph-volume 17.2.5-1~bpo11+1 all tool to facilidate OSD deployment ii cephadm 17.2.5-1~bpo11+1 amd64 cephadm utility to bootstrap ceph daemons with systemd and containers ii libcephfs2 17.2.5-1~bpo11+1 amd64 Ceph distributed file system client library ii libcephfs2-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for libcephfs2 ii libsqlite3-mod-ceph 17.2.5-1~bpo11+1 amd64 SQLite3 VFS for Ceph ii libsqlite3-mod-ceph-dbg 17.2.5-1~bpo11+1 amd64 debugging symbols for libsqlite3-mod-ceph ii python3-ceph-argparse 17.2.5-1~bpo11+1 all Python 3 utility libraries for Ceph CLI ii python3-ceph-common 17.2.5-1~bpo11+1 all Python 3 utility libraries for Ceph ii python3-cephfs 17.2.5-1~bpo11+1 amd64 Python 3 libraries for the Ceph libcephfs library ii python3-cephfs-dbg 17.2.5-1~bpo11+1 amd64 Python 3 libraries for the Ceph libcephfs library root@ss-joe-01(bash):/home/hydrauser#

8 months

1
0
0 0

2024

2023

2022

2021

2020

2019

Dev September 2023