- ceph-users - lists.ceph.io

No recovery after removing node - active+undersized+degraded-- removed osd using purge...
by Devender Singh 25 Jan '25

25 Jan '25

Hello all Urgent help needed. No recovery happening. Tried repairing pg and redeploy or create. Rebooted cluster but no luck.. data: volumes: 2/2 healthy pools: 18 pools, 817 pgs objects: 6.06M objects, 20 TiB usage: 30 TiB used, 302 TiB / 332 TiB avail pgs: 2846742/29091911 objects degraded (9.785%) 2654404/29091911 objects misplaced (9.124%) 516 active+clean 105 active+undersized+degraded Regards Dev

2 1

FS design question around subvolumes vs dirs
by Jesse Galley 24 Jan '25

24 Jan '25

Hello, We are building a ~2PiB 200 OSD cluster that will be used entirely by an FS for hosting data storage. I have a question regarding whether there is any specific reason/advantage to use subvolumes vs plain directories for segregating different departments, envs, etc. Currently, the plan is to create subtrees like: /web/home/env1 /web/logs/env1 /web/logs/env2 /mail/home/env1 ..etc Clients will be path-restricted to the bottom directory with MDS caps, and namespace restricted with OSD caps. Namespace layouts will be set on the mounted dir when new users/dirs are provisioned. Snapshots will be managed by storage admins from the top of the trees. Static MDS pinning will be used throughout. As far as I understand, if I create subvolume groups and subvolumes, I would have to flatten the <department>/<service>/<env> structure a bit to make it work, but it may simplify the manual management of snapshots, caps, and namespaces. Are there any other reasons to use subvolumes via manual dir management? Are subvolumes treated any differently by the cluster than dirs? I am especially concerned with io performance, as well as snap mirroring, as we are building an identical cluster to receive mirrors. Appreciate any insight. Thank you. *Jesse Galley* *Director, Mail Platform & Migrations, Hostopia* *C: 416.816.0866* *jesse.galley(a)hostpapa.com <jesse.galley(a)hostpapa.com>*

1 0

January Ceph Science Virtual User Group
by Kevin Hrpcek 24 Jan '25

24 Jan '25

Hey All, We will be having a Ceph science/research/big cluster call on Tuesday January 28th. If anyone wants to discuss something specific they can add it to the pad linked below. If you have questions or comments you can contact me. This is an informal open call of community members mostly from hpc/htc/research/big cluster environments, though anyone is welcome, where we discuss whatever is on our minds regarding ceph. Updates, outages, features, maintenance, etc...there is no single set presenter for the length of the meeting, but it is somewhat moderated to keep the conversation flowing. A change for this year is that we will start each meeting with a brief 10-15 minute talk given by a community member. If you're interested in signing up to give a short talk for a meeting please let me know! This month Mattia will present regarding balancing CephFS performance and reliability. Pad URL: https://pad.ceph.com/p/Ceph_Science_User_Group_20250128 Virtual event details: January 28th, 2025 15:00 UTC 4pm Central European 9am Central US Main pad for discussions: https://pad.ceph.com/p/Ceph_Science_User_Group_Index Meetings will be recorded and posted to the Ceph Youtube channel. To join the meeting on a computer or mobile phone: https://meet.jit.si/ceph-science-wg Ceph Community Calendar. Go to Jan 28 and you can quickly add it to your calendar as well https://ceph.io/en/community/meetups/ Kevin

1 0

Help needed, ceph fs down due to large stray dir
by Frank Schilder 24 Jan '25

24 Jan '25

Hi all, we seem to have a serious issue with our file system, ceph version is pacific latest. After a large cleanup operation we had an MDS rank with 100Mio stray entries (yes, one hundred million). Today we restarted this daemon, which cleans up the stray entries. It seems that this leads to a restart loop due to OOM. The rank becomes active and then starts pulling in DNS and INOS entries until all memory is exhausted. I have no idea if there is at least progress removing the stray items or if it starts from scratch every time. If it needs to pull as many DNS/INOS into cache as there are stray items, we don't have a server at hand with enough RAM. Q1: Is the MDS at least making progress in every restart iteration? Q2: If not, how do we get this rank up again? Q3: If we can't get this rank up soon, can we at least move directories away from this rank by pinning it to another rank? Currently, the rank in question reports .mds_cache.num_strays=0 in perf dump. ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14

7 34

MDS hung in purge_stale_snap_data after populating cache
by Frank Schilder 23 Jan '25

23 Jan '25

Hi all, this post is related to "Help needed, ceph fs down due to large stray dir" (https://www.spinics.net/lists/ceph-users/msg85394.html) We finally got the MDS up and could record what it is doing before and after getting hung. We opened tracker https://tracker.ceph.com/issues/69547 . The uploads were too large to attach. For a quick overview of the key findings see the 3 screen shots posted here: https://imgur.com/a/RF7ExSP . Apparently, the MDS gets hung in CInode::purge_stale_snap_data after populating the cache. I hope that the data uploaded will give someone a clue what is happening and how we can get out of it. Putting my hopes on tomorrow after sleeping a bit (well, at least trying to). Thanks for all your help with this critical situation. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14

3 16

malformed osd ID
by Joffrey 23 Jan '25

23 Jan '25

Hi, After a failed OSD during the upgrade to 19.2.0, I renamed a folder. Now, I have an OSD with the name osd.60.oldf. This bug impacts various commands and services. The host is offline (but it is not — another OSD, osd.67, is up and in). Some commands return the following error: cephadm shell -- ceph cephadm get-ssh-config: Error EIO: Module 'cephadm' has experienced an error and cannot handle commands: invalid literal for int() with base 10: '60.oldf' ceph osd status: Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1864, in _handle_command return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf) File "/usr/share/ceph/mgr/mgr_module.py", line 499, in call return self.func(mgr, **kwargs) File "/usr/share/ceph/mgr/status/module.py", line 337, in handle_osd_status assert metadata AssertionError I tried to remove the daemon: ceph orch daemon rm osd.60.oldf --force but received: TypeError: __init__() missing 2 required positional arguments: 'hostname' and 'addr' because the host is offline, and get-ssh-config does not work... I have removed all data related to osd.60 and osd.60.oldf on the host hosting this OSD. How can I remove this daemon entry osd.60.oldf from Ceph's store? osd.60.oldf only appears in orch ps, not in osd ls... Thank you.

2 1

Bad/strange performance on a new cluster
by Jan 23 Jan '25

23 Jan '25

Hi all, we have a new Ceph cluster and have some very bad/strange performance behavior. I really don't understand what I'm doing wrong here and would be more than happy if anyone has an idea. Even a hint on what to look at would be helpful. Some Information: Machines (8 Nodes) per Node: - CPU 2x Intel(R) Xeon(R) Gold 6258R CPU @ 2.70GHz (28 Cores) - 384 GB RAM - 20x Dell Ent NVMe AGN RI U.2 7.68TB (for OSDs) - 4x 25G LACP Backend - 2x 25G LACP Frontend - OS: - Ubuntu 22.04 - Kernel: 5.15.0 - Ceph: - Version 18.2.4 - 160 osds - 4096 PGs for the VM pool I took some fio benchmarks from the Proxmox Ceph Performance Paper: https://www.proxmox.com/images/download/pve/docs/Proxmox-VE_Ceph-Benchmark-… The First test should have about 1500 IOPS (Proxmox Paper: 1806). We only get 170. root@ceph001:/mnt# fio --ioengine=psync --filename=test_fio --size=9G --time_based --name=fio --group_reporting --runtime=60 --direct=1 --sync=1 --rw=write --bs=4K --numjobs=1 --iodepth=1 fio: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.28 Starting 1 process fio: Laying out IO file (1 file / 9216MiB) Jobs: 1 (f=1): [W(1)][100.0%][w=680KiB/s][w=170 IOPS][eta 00m:00s] fio: (groupid=0, jobs=1): err= 0: pid=174797: Wed Jan 22 20:19:19 2025 write: IOPS=202, BW=811KiB/s (831kB/s)(47.5MiB/60003msec); 0 zone resets clat (usec): min=2185, max=20081, avg=4925.43, stdev=931.63 lat (usec): min=2186, max=20082, avg=4926.19, stdev=931.63 clat percentiles (usec): | 1.00th=[ 3425], 5.00th=[ 3818], 10.00th=[ 3982], 20.00th=[ 4293], | 30.00th=[ 4490], 40.00th=[ 4686], 50.00th=[ 4817], 60.00th=[ 5014], | 70.00th=[ 5211], 80.00th=[ 5407], 90.00th=[ 5800], 95.00th=[ 6063], | 99.00th=[ 8586], 99.50th=[ 9503], 99.90th=[12256], 99.95th=[13304], | 99.99th=[19006] bw ( KiB/s): min= 672, max= 1000, per=100.00%, avg=813.11, stdev=73.18, samples=119 iops : min= 168, max= 250, avg=203.28, stdev=18.29, samples=119 lat (msec) : 4=10.24%, 10=89.43%, 20=0.32%, 50=0.01% cpu : usr=0.25%, sys=2.57%, ctx=36503, majf=0, minf=15 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,12167,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=811KiB/s (831kB/s), 811KiB/s-811KiB/s (831kB/s-831kB/s), io=47.5MiB (49.8MB), run=60003-60003msec Disk stats (read/write): rbd0: ios=0/24296, merge=0/2, ticks=0/56351, in_queue=56351, util=99.97% Bandwith and IOPs with more IO depth look ok form me: fio --filename=/mnt/testingfio1 --size=50GB --direct=1 --rw=randrw --bs=4k --ioengine=libaio --iodepth=256 --runtime=150 --numjobs=1 --time_based \ --group_reporting --name=iops-test-job --eta-newline=1 iops-test-job: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=256 fio-3.28 iops-test-job: (groupid=0, jobs=1): err= 0: pid=146931: Wed Jan 22 19:43:14 2025 read: IOPS=20.0k, BW=78.0MiB/s (81.8MB/s)(11.4GiB/150006msec) slat (nsec): min=1245, max=7415.7k, avg=22636.97, stdev=224525.03 clat (usec): min=238, max=32714, avg=5620.53, stdev=2255.97 lat (usec): min=243, max=32721, avg=5643.32, stdev=2258.28 clat percentiles (usec): | 1.00th=[ 1876], 5.00th=[ 2311], 10.00th=[ 2671], 20.00th=[ 3654], | 30.00th=[ 4146], 40.00th=[ 4752], 50.00th=[ 5342], 60.00th=[ 6128], | 70.00th=[ 6915], 80.00th=[ 7635], 90.00th=[ 8717], 95.00th=[ 9896], | 99.00th=[10683], 99.50th=[10945], 99.90th=[11863], 99.95th=[12649], | 99.99th=[14615] bw ( KiB/s): min=63254, max=98432, per=100.00%, avg=79914.04, stdev=6290.69, samples=299 iops : min=15813, max=24608, avg=19978.36, stdev=1572.70, samples=299 write: IOPS=19.9k, BW=77.9MiB/s (81.7MB/s)(11.4GiB/150006msec); 0 zone resets slat (nsec): min=1349, max=8871.7k, avg=23250.80, stdev=225370.53 clat (usec): min=629, max=81108, avg=7160.58, stdev=2338.33 lat (usec): min=633, max=81114, avg=7183.98, stdev=2348.98 clat percentiles (usec): | 1.00th=[ 2900], 5.00th=[ 3982], 10.00th=[ 4293], 20.00th=[ 5014], | 30.00th=[ 5735], 40.00th=[ 6325], 50.00th=[ 6980], 60.00th=[ 7570], | 70.00th=[ 8225], 80.00th=[ 9110], 90.00th=[10421], 95.00th=[11207], | 99.00th=[13435], 99.50th=[14353], 99.90th=[16581], 99.95th=[17957], | 99.99th=[21365] bw ( KiB/s): min=61755, max=98813, per=100.00%, avg=79877.64, stdev=6336.82, samples=299 iops : min=15438, max=24703, avg=19969.22, stdev=1584.22, samples=299 lat (usec) : 250=0.01%, 500=0.03%, 750=0.07%, 1000=0.08% clat (usec): min=238, max=32714, avg=5620.53, stdev=2255.97 [141/1761] lat (usec): min=243, max=32721, avg=5643.32, stdev=2258.28 clat percentiles (usec): | 1.00th=[ 1876], 5.00th=[ 2311], 10.00th=[ 2671], 20.00th=[ 3654], | 30.00th=[ 4146], 40.00th=[ 4752], 50.00th=[ 5342], 60.00th=[ 6128], | 70.00th=[ 6915], 80.00th=[ 7635], 90.00th=[ 8717], 95.00th=[ 9896], | 99.00th=[10683], 99.50th=[10945], 99.90th=[11863], 99.95th=[12649], | 99.99th=[14615] bw ( KiB/s): min=63254, max=98432, per=100.00%, avg=79914.04, stdev=6290.69, samples=299 iops : min=15813, max=24608, avg=19978.36, stdev=1572.70, samples=299 write: IOPS=19.9k, BW=77.9MiB/s (81.7MB/s)(11.4GiB/150006msec); 0 zone resets slat (nsec): min=1349, max=8871.7k, avg=23250.80, stdev=225370.53 clat (usec): min=629, max=81108, avg=7160.58, stdev=2338.33 lat (usec): min=633, max=81114, avg=7183.98, stdev=2348.98 clat percentiles (usec): | 1.00th=[ 2900], 5.00th=[ 3982], 10.00th=[ 4293], 20.00th=[ 5014], | 30.00th=[ 5735], 40.00th=[ 6325], 50.00th=[ 6980], 60.00th=[ 7570], | 70.00th=[ 8225], 80.00th=[ 9110], 90.00th=[10421], 95.00th=[11207], | 99.00th=[13435], 99.50th=[14353], 99.90th=[16581], 99.95th=[17957], | 99.99th=[21365] bw ( KiB/s): min=61755, max=98813, per=100.00%, avg=79877.64, stdev=6336.82, samples=299 iops : min=15438, max=24703, avg=19969.22, stdev=1584.22, samples=299 lat (usec) : 250=0.01%, 500=0.03%, 750=0.07%, 1000=0.08% lat (msec) : 2=0.65%, 4=15.15%, 10=75.45%, 20=8.55%, 50=0.01% lat (msec) : 100=0.01% cpu : usr=7.36%, sys=18.55%, ctx=155494, majf=0, minf=9200 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1% issued rwts: total=2993949,2992429,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=256 Run status group 0 (all jobs): READ: bw=78.0MiB/s (81.8MB/s), 78.0MiB/s-78.0MiB/s (81.8MB/s-81.8MB/s), io=11.4GiB (12.3GB), run=150006-150006msec WRITE: bw=77.9MiB/s (81.7MB/s), 77.9MiB/s-77.9MiB/s (81.7MB/s-81.7MB/s), io=11.4GiB (12.3GB), run=150006-150006msec Disk stats (read/write): rbd0: ios=2989470/2987987, merge=0/1, ticks=9760844/13043854, in_queue=22804699, util=100.00% We have 4096 PG on the tested pool. root@ceph001:/mnt# ceph -s cluster: id: health: HEALTH_OK services: mon: 5 daemons, quorum ceph001,ceph002,ceph003,ceph005,ceph006 (age 52m) mgr: ceph002.hgppdu(active, since 2d), standbys: ceph001.ooznoq osd: 160 osds: 160 up (since 5w), 160 in (since 5M) rgw: 2 daemons active (2 hosts, 1 zones) data: pools: 11 pools, 8449 pgs objects: 7.80M objects, 19 TiB usage: 56 TiB used, 1.0 PiB / 1.1 PiB avail root@ceph001:~# ceph config get osd osd_memory_target 4294967296 root@ceph001:~# ceph config get osd WHO MASK LEVEL OPTION VALUE ... osd advanced osd_memory_target_autotune true ... We would like to use the cluster with OpenStack Cinder. But the tests were made directly on the cluster nodes with rbd. The metrics on VMs are similar. Thanks in advance. Jan

1 0

osd won't restart
by Dominique Ramaekers 23 Jan '25

23 Jan '25

Hi, Strange thing just happened (ceph v19.2.0). I added two disks to a host. Kernel recognized nicely the two disks and they appeared as available devices in ceph. After 15 minutes osd's were not created, so I looked at the logs: /usr/bin/docker: stderr --> Creating keyring file for osd.36 /usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-36/keyring /usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-36/ /usr/bin/docker: stderr Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 36 --monmap /var/lib/ceph/osd/ceph-36/activate.monmap --keyfile - --osdspec-affinity all-available-devices --osd-data /var/lib/ceph/osd/ceph-36/ --osd-uuid 41675779-943d-4dca-baa3-3a4f6ace004a --setuser ceph --setgroup ceph /usr/bin/docker: stderr stderr: 2025-01-20T08:21:01.979+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-36//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3] /usr/bin/docker: stderr stderr: 2025-01-20T08:21:01.979+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-36//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3] /usr/bin/docker: stderr stderr: 2025-01-20T08:21:01.980+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-36//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3] /usr/bin/docker: stderr stderr: 2025-01-20T08:21:01.980+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36/) _read_fsid unparsable uuid /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.075+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.076+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.076+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.076+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.076+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.076+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.076+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.506+0000 79e9fea34640 -1 bluestore(/var/lib/ceph/osd/ceph-36//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.506+0000 79e9fea34640 -1 bdev(0x566a21f14a80 /var/lib/ceph/osd/ceph-36//block) open open got: (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.506+0000 79e9fea34640 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied /usr/bin/docker: stderr stderr: 2025-01-20T08:21:02.506+0000 79e9fea34640 -1 [0;31m ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-36/: (13) Permission denied /usr/bin/docker: stderr --> Was unable to complete a new OSD, will rollback changes As it sad "Permission denied" and I have already osd's running, I thought it was an issue that docker may be updated but not restarted. So I did 'systemcrl restart docker.service'. Now none of the managed osd's are going back online!!! 'systemctl start ceph-dd4b0610-b4d2-11ec-bb58-d1b32ae31585(a)osd.18.service<mailto:ceph-dd4b0610-b4d2-11ec-bb58-d1b32ae31585@osd.18.service>' fails with not much explanation... Only unmanaged osd have no issue... I didn't payed much attention to the log entry '_read_fsid unparsable uuid'... So I think there is more going on. Permission denied is logical if the path is wrong... see dubbel slash in '_read_bdev_label failed to open /var/lib/ceph/osd/ceph-36//block' Is this a bug like https://github.com/rook/rook/issues/10219 ? Can I get around this without recreating these osd as unmanaged? Thanks in advance. Dominique.

2 4

shell fautly command
by Dominique Ramaekers 22 Jan '25

22 Jan '25

Hi, I was removing config keys of osd's that were purged... I accidently entered the following command: #ceph config-key rm osd Feedback "key deleted" :-( Did I do something very stupid??? The output of my osd keys: # ceph config-key ls | grep 'osd' | grep -v history "config/global/osd_pool_default_pg_autoscale_mode", "config/osd.0/osd_mclock_max_capacity_iops_hdd", "config/osd.0/osd_mclock_max_capacity_iops_ssd", "config/osd.1/osd_mclock_max_capacity_iops_hdd", "config/osd.1/osd_mclock_max_capacity_iops_ssd", "config/osd.10/osd_mclock_max_capacity_iops_ssd", "config/osd.11/osd_mclock_max_capacity_iops_ssd", "config/osd.12/osd_mclock_max_capacity_iops_ssd", "config/osd.13/osd_mclock_max_capacity_iops_ssd", "config/osd.14/osd_mclock_max_capacity_iops_ssd", "config/osd.15/osd_mclock_max_capacity_iops_ssd", "config/osd.16/osd_mclock_max_capacity_iops_ssd", "config/osd.17/osd_mclock_max_capacity_iops_ssd", "config/osd.2/osd_mclock_max_capacity_iops_hdd", "config/osd.2/osd_mclock_max_capacity_iops_ssd", "config/osd.20/osd_mclock_max_capacity_iops_ssd", "config/osd.21/osd_mclock_max_capacity_iops_ssd", "config/osd.22/osd_mclock_max_capacity_iops_ssd", "config/osd.23/osd_mclock_max_capacity_iops_ssd", "config/osd.24/osd_mclock_max_capacity_iops_ssd", "config/osd.25/osd_mclock_max_capacity_iops_ssd", "config/osd.26/osd_mclock_max_capacity_iops_ssd", "config/osd.27/osd_mclock_max_capacity_iops_ssd", "config/osd.28/osd_mclock_max_capacity_iops_ssd", "config/osd.29/osd_mclock_max_capacity_iops_ssd", "config/osd.3/osd_mclock_max_capacity_iops_hdd", "config/osd.3/osd_mclock_max_capacity_iops_ssd", "config/osd.30/osd_mclock_max_capacity_iops_ssd", "config/osd.31/osd_mclock_max_capacity_iops_ssd", "config/osd.32/osd_mclock_max_capacity_iops_ssd", "config/osd.33/osd_mclock_max_capacity_iops_ssd", "config/osd.34/osd_mclock_max_capacity_iops_ssd", "config/osd.35/osd_mclock_max_capacity_iops_ssd", "config/osd.37/osd_mclock_max_capacity_iops_ssd", "config/osd.38/osd_mclock_max_capacity_iops_ssd", "config/osd.4/osd_mclock_max_capacity_iops_hdd", "config/osd.4/osd_mclock_max_capacity_iops_ssd", "config/osd.5/osd_mclock_max_capacity_iops_hdd", "config/osd.5/osd_mclock_max_capacity_iops_ssd", "config/osd.6/osd_mclock_max_capacity_iops_ssd", "config/osd.7/osd_mclock_max_capacity_iops_ssd", "config/osd.8/osd_mclock_max_capacity_iops_ssd", "config/osd.9/osd_mclock_max_capacity_iops_ssd", "config/osd/host:hvs001/osd_memory_target", "config/osd/host:hvs002/osd_memory_target", "config/osd/host:hvs003/osd_memory_target", "config/osd/host:hvs005/osd_memory_target", "config/osd/osd_memory_target_autotune", "mgr/cephadm/osd_remove_queue", "mgr/cephadm/spec.osd.all-available-devices",

2 1

January Ceph Science Virtual User Group
by Kevin Hrpcek 22 Jan '25

22 Jan '25

Hey All, We will be having a Ceph science/research/big cluster call on Tuesday January 28th. If anyone wants to discuss something specific they can add it to the pad linked below. If you have questions or comments you can contact me. This is an informal open call of community members mostly from hpc/htc/research/big cluster environments, though anyone is welcome, where we discuss whatever is on our minds regarding ceph. Updates, outages, features, maintenance, etc...there is no single set presenter for the length of the meeting, but it is somewhat moderated to keep the conversation flowing. A change for this year is that we will start each meeting with a brief 10-15 minute talk given by a community member. If you're interested in signing up to give a short talk for a meeting please let me know! This month Mattia will present regarding balancing CephFS performance and reliability. Pad URL: https://pad.ceph.com/p/Ceph_Science_User_Group_20250128 Virtual event details: January 28th, 2025 15:00 UTC 4pm Central European 9am Central US Main pad for discussions: https://pad.ceph.com/p/Ceph_Science_User_Group_Index Meetings will be recorded and posted to the Ceph Youtube channel. To join the meeting on a computer or mobile phone: https://meet.jit.si/ceph-science-wg Ceph Community Calendar. Go to Jan 28 and you can quickly add it to your calendar as well https://ceph.io/en/community/meetups/ Kevin

1 0