- ceph-users - lists.ceph.io

Announcing go-ceph v0.31.0
by Anoop C S 18 Dec '24

18 Dec '24

Hi, We are happy to announce another release of the go-ceph API library. This is a regular release following our every-two-months release cadence. https://github.com/ceph/go-ceph/releases/tag/v0.31.0 More details are available at the link above. The library includes bindings that aim to play a similar role to the "pybind" python bindings in the ceph tree but for the Go language. The library also includes additional APIs that can be used to administer cephfs, rbd, and rgw subsystems. There are already a few consumers of this library in the wild, including the ceph-csi project. Anoop C S.

1 0

Update host operating system - Ceph version 18.2.4 reef
by alessandro＠universonet.com.br 18 Dec '24

18 Dec '24

Ceph version 18.2.4 reef (cephadm) Hello, We have a cluster running with 6 Ubuntu 20.04 servers and we would like to add another host but with Ubuntu 22.04, will we have any problems? We would like to add new HOST with Ubuntu 22.04 and deactivate the Ubuntu 20.04 ones, our idea would be to update the hosts from Ubuntu 20.04 to 22.04. If anyone can give us some guidance, thank you very much. Best regards, Alessandro

4 4

ceph network acl: multiple network prefixes possible?
by Stefan Kooman 18 Dec '24

18 Dec '24

Hi List, Is it possible to specify multiple network prefixes for a given ceph user? I.e. osd 'allow {access-spec} [{match-spec}] [network {network1/prefix} {network2/prefix} {network3/prefix}] ' osd 'profile {name} [pool={pool-name} [namespace={namespace-name}]] [network {network1/prefix} {network2/prefix} {network3/prefix}' Or in some other syntax (comma separated, multiple network tags, etc.? I have tried some different formats but they either result in "1 auth entities have invalid capabilities" or only one of three prefixed being used. Gr. Stefan [1]: https://docs.ceph.com/en/reef/rados/operations/user-management/#authorizati…

1 0

Erasure coding best practice
by Szabo, Istvan (Agoda) 17 Dec '24

17 Dec '24

Hi, Couple of time I saw some use-case in the mail list where people using erasure code on a "weird" way. Normally we use more data chunks than coding chunks, like k=4 m=2 or k=8 m=3. What I mean "weird" I saw someone using k=3 and m=8 (I might remember wrong). I'm trying to understand what is the benefit of the higher coding chunks? You can use smaller object size? Let's say 4:2 the minimum object size should be 24K at least or with 8:3 it would be 44K because nothing will be stored on smaller space. In case of k=3 m=8, smallest object can be 12K, but you can lose 8 nodes (pgs) and data still there? Ty

5 6

stray host with daemons
by Chris Webb 17 Dec '24

17 Dec '24

Hi, I managed to install our 15th server with a fully qualified hostname. I rectified this after adding the disks to the cluster (17.2.6) by changing the hostname. 1) ceph orch host ls - returns the correct (shrot) hostnames 2) ceph orch ps - returns the correct (short) hostname for all demons 3) ceph health detail returns the following ( despite failing the mgr ) [root@cephp15 ~]# ceph health detail HEALTH_WARN 1 stray host(s) with 12 daemon(s) not managed by cephadm [WRN] CEPHADM_STRAY_HOST: 1 stray host(s) with 12 daemon(s) not managed by cephadm stray host cephp15.xxx.xxx.xxx has 12 stray daemons: ['osd.168', 'osd.169', 'osd.170', 'osd.171', 'osd.172', 'osd.173', 'osd.174', 'osd.175', 'osd.176', 'osd.177', 'osd.178', 'osd.179'] [root@cephp15 ~]# 4) ceph device ls - returns the incorrect fully qualified hostname ( despite rebooting the host ) 5) I've tried a "ceph host rescan", this does not appear to make any difference. Does anyone have any suggestions on how to resolve this in a non-destructive way? Thanks, Chris

3 3

Erasure coding issue
by Deba Dey 17 Dec '24

17 Dec '24

See I have total host 5, each host holding 24 HDD and each HDD is of size 9.1TiB. So, a total of 1.2PiB out of which i am getting 700TiB. I did erasure coding 3+2 and placement group 128. But, the issue i am facing is when I turn off one node write is completely disabled. Erasure coding 3+2 can handle two nodes failure but it's not working in my case. I request this community to help me tackle this issue. The min size is 3 and 4 pools are there.

3 2

Tracing Ceph with LTTng-UST issue
by IslamChakib Kedadsa 17 Dec '24

17 Dec '24

Hello, We are writing to you regarding an issue we encountered while attempting to trace Ceph with LTTng-UST. Below are the steps we have followed so far: 1. *Compiling Ceph with LTTng Support*: - We modified the debian/rules file to enable LTTng support using the following flags: -DWITH_LTTNG=ON and -DWITH_OSD_INSTRUMENT_FUNCTIONS=ON. - The build steps can be found in our GitHub <https://github.com/KedadsaChakib/ceph-configuration/blob/main/build/README.…> repository. 2. *Deploying a Ceph Cluster*: - We deployed the cluster manually (1MON,1MGR,2OSDs). Please find the script here <https://github.com/KedadsaChakib/ceph-configuration/blob/main/cluster/clust…> . 3. *Attempting to Trace with LTTng-UST*: - We tried listing the tracepoints using the following command : # lttng list -u - This command did not return any Ceph-related tracepoints: UST events: ------------- NONE 4. *Preloading liblttng-ust-fork.so*: - We attempted to preload liblttng-ust-fork.so before starting the Ceph daemons (MON, OSD, MGR) as follows: # systemctl edit ceph-mon@debian - We added the following configuration: [Service] Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/liblttng-ust-fork.so - After restarting the daemons, we executed lttng list -u, which displayed tracepoints related to the libraries (lttng_ust_lib:*), trace logs (lttng_ust_tracelog:*), and state dump events ( lttng_ust_statedump:*), instead of the expected Ceph-related tracepoints. *Expected Result*: UST events: ------------- PID: 100859 - Name: /path/to/ceph-osd pg:queue_op (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) osd:do_osd_op_post (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) osd:do_osd_op_pre_unknown (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) osd:do_osd_op_pre_copy_from (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) osd:do_osd_op_pre_copy_get (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) ... *Actual Result*: UST events: ------------- lttng_ust_lib:* lttng_ust_tracelog:* lttng_ust_statedump:* We would greatly appreciate any help you could provide to resolve this problem.

1 0

Correct way to replace working OSD disk keeping the same OSD ID
by Nicola Mori 17 Dec '24

17 Dec '24

Dear Ceph users, I'm struggling to unbderstand what is the correct way to remove a working disk and replace it (e.g. for a disk upgrade) while keeping the same OSD ID. I did this several times following this procedure: https://docs.ceph.com/en/reef/cephadm/services/osd/#replacing-an-osd every time facing a different issue. This time it happened that after removal the OSD was marked as destroyed, so I went on with disk replacement and then with: ceph orch apply osd --all-available-devices but this created a new OSD with a new ID instead of reusing the old one, which remained in destroyed state. I must be missing something but I cannot figure out what. Thanks, Nicola

5 6

we cannot read the prometheus Metrics
by Giovanna Ratini 17 Dec '24

17 Dec '24

Hello, we cannot read the prometheus Metrics anymore. ceph config get mgr mgr/cephadm/container_image_node_exporter quay.io/prometheus/node-exporter:v1.5.0 curl localhost:9926/metrics --> works curl 10.244.62.35:9926/metrics --> works (IP = POD IP) same namespace another pod ( rook-tools ) curl 10.244.62.35:9926/metrics --> timeout Any Idea? kubectl describe svc rook-ceph-exporter -n rook-ceph Name: rook-ceph-exporter Namespace: rook-ceph Labels: app=rook-ceph-exporter rook_cluster=rook-ceph Annotations: <none> Selector: app=rook-ceph-exporter,rook_cluster=rook-ceph Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv4 IP: 10.108.230.88 IPs: 10.108.230.88 Port: ceph-exporter-http-metrics 9926/TCP TargetPort: 9926/TCP Endpoints: 10.244.115.233:9926,10.244.132.1:9926,10.244.139.103:9926 + 11 more... Session Affinity: None Events: <none> Best whishes, Gio kubectl get endpoints rook-ceph-exporter -n rook-ceph NAME ENDPOINTS AGE rook-ceph-exporter 10.244.115.233:9926,10.244.132.1:9926,10.244.139.103:9926 + 11 more... 273d

1 0

doc: https://docs.ceph.com/ root URL still redirects to Reef
by Florian Haas 16 Dec '24

16 Dec '24

Hi everyone, A little while back I noticed that the docs.ceph.com root URL still redirects to the docs for Reef: $ curl -I https://docs.ceph.com/ HTTP/2 302 date: Fri, 06 Dec 2024 13:04:15 GMT content-type: text/html; charset=utf-8 content-length: 0 location: https://docs.ceph.com/en/reef/ This is still the case as of today. Considering the current major release is Squid, I am guessing we'll want this to point to https://docs.ceph.com/en/squid/ instead. Does that sound like a good suggestion to others too? Cheers, Florian

1 0