Hello Kilian, I am not entirely sure this is the same issue, but I know Prometheus doesn’t export some performance metrics by default anymore in Reef. This can be worked around by setting: ceph config set mgr mgr/prometheus/exclude_perf_counters false A proper fix would be to switch to using a per-host ceph-exporter. Best regards, Pierre Riteau On Mon, 25 Nov 2024 at 13:45, Kilian Ries <mail@kilian-ries.de> wrote:
Prometheus metrics seem to be broken, too:
ceph_osd_op_r_latency_sum
ceph_osd_op_w_latency_sum
Both of them for example are not reported by the ceph mgr metrics exporter:
curl http://192.168.XXX.XXX:9283/metrics |grep ceph_osd_
I get some merics like "ceph_osd_commit_latency_ms" or "ceph_osd_apply_latency_ms" but none of the "op_r / op_w" metrics. Do the lable names have switched and my grafana dashboard is outdated? Or are they missing at the exporter level?
Thanks
________________________________ Von: Kilian Ries Gesendet: Montag, 25. November 2024 13:37:16 An: ceph-users@ceph.io Betreff: AW: Ceph OSD perf metrics missing
Any ideas? Still facing the problem ...
________________________________ Von: Kilian Ries Gesendet: Mittwoch, 23. Oktober 2024 13:59:06 An: ceph-users@ceph.io Betreff: Ceph OSD perf metrics missing
Hi,
i'm running a Ceph v18.2.4 cluster. I'm trying to build some latency monitoring with the
ceph daemon osd.4 perf dump
cli command. On most of the OSDs i get all the metrics i need. On some OSDs i only get zero values:
osd.op_latency.avgcount: 0
I already tried restarting the OSD process which didn't help. I also tried to reset the metrics via
ceph daemon osd.4 perf reset all
but that didn't help either. How can that be that some OSDs don't show values here? How can i fix that?
Thanks
Regards,
Kilian _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io