27 Feb
2020
27 Feb
'20
9:01 a.m.
Hi all,
A similar question would be if it is possible to let passive mgr do the data
collection!?
We run 14.2.6 on a medium 2.5PB cluster with over 900M objects (rbd and mainly S3) . At
the moment, we face an issue with the prometheus exporter while it has high load. (e.g.
while we insert a new host or a rebalance is active) The CPU usage of the active mgr
daemon raises over 100% (this issue is still not completely solved, see discussion on
ceph-users in "High CPU usage by ceph-mgr in 14.2.6") what leads to the fact
that the daemon cannot answer the scrape request.
We already reduced the scrape frequency radically (I think it is at 30s and was already at
60s) with no effect. Also a curl on the active mgr doesn't respond. Once the load goes
away the mgr can answer the requests again. But somehow, the monitoring is especially
interesting while there is high load so this is not ideal. Did anybody experience a
similar issue?
Thanks in advance,
Michael
On 27.02.20, 08:58, "黄明友" <hmy(a)v.photos> wrote:
thank you ,you are right.
------------------------------------------------------------------
发件人：Jan Fajerski <jfajerski(a)suse.com>
发送时间：2020年2月27日(星期四) 15:34
收件人：ceph-users <ceph-users(a)ceph.io>
主 题：[ceph-users] Re: ceph prometheus module no export content
On Thu, Feb 27, 2020 at 10:20:07AM +0800, 黄明友 wrote:
hi,all
I had enable the prometheus module on my ceph cluster , the
ceph version
is ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nauti
lus (stable).
when I eanble this modle, can get exported content from the prometheus
module.
but , when I restart all the ceph cluster's node , the module can't
expor any content , curl http://host/metrics ; get blan;
the tcp/9283 port is listen; I had disable and enable this module,but
still no content export.
You're probably querying a standby mgr daemon. Check ceph -s for the
current
active and query that one. For a prometheus server you always want to scrape all
mgr daemons. Also see https://docs.ceph.com/docs/master/mgr/prometheus/
黄明友
IT基础架构部经理
V.Photos 云摄影
移动电话: +86 13540630430
客服电话：400 - 806 - 5775
电子邮件: hmy(a)v.photos
官方网址: www.v.photos
上海 黄浦区中山东二路88号外滩SOHO3Q F栋 2层
北京 朝阳区光华路9号光华路SOHO二期南二门SOHO3Q 1层
广州 天河区林和中路136号天誉花园二期3Wcoffice 天誉青创社区
深圳 南山区蛇口网谷科技大厦二期A座102网谷双创街 1层
成都 成华区建设路世贸广场 7层
[cid:__aliyun158277000719672821]
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Jan Fajerski
Senior Software Engineer Enterprise Storage
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io