thank you ,you are right. ------------------------------------------------------------------ 发件人：Jan Fajerski <jfajerski@suse.com> 发送时间：2020年2月27日(星期四) 15:34 收件人：ceph-users <ceph-users@ceph.io> 主　题：[ceph-users] Re: ceph prometheus module no export content On Thu, Feb 27, 2020 at 10:20:07AM +0800, 黄明友 wrote:

hi,all I had enable the prometheus module on my ceph cluster , the ceph version is ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nauti lus (stable). when I eanble this modle, can get exported content from the prometheus module. but , when I restart all the ceph cluster's node , the module can't expor any content , curl http://host/metrics ; get blan; the tcp/9283 port is listen; I had disable and enable this module,but still no content export. You're probably querying a standby mgr daemon. Check ceph -s for the current active and query that one. For a prometheus server you always want to scrape all mgr daemons. Also see https://docs.ceph.com/docs/master/mgr/prometheus/ 黄明友 IT基础架构部经理 V.Photos 云摄影 移动电话: +86 13540630430 客服电话：400 - 806 - 5775 电子邮件: hmy@v.photos 官方网址: www.v.photos 上海 黄浦区中山东二路88号外滩SOHO3Q F栋 2层 北京 朝阳区光华路9号光华路SOHO二期南二门SOHO3Q 1层 广州 天河区林和中路136号天誉花园二期3Wcoffice 天誉青创社区 深圳 南山区蛇口网谷科技大厦二期A座102网谷双创街 1层 成都 成华区建设路世贸广场 7层 [cid:__aliyun158277000719672821]

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

-- Jan Fajerski Senior Software Engineer Enterprise Storage SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

Alternatively, it might be handy to have the passive mgrs issue an HTTP redirect to the active mgr.  Then a single DNS name pointing to all mgrs would always work, even when the active mgr fails over. Going a step further with some HA strategies, the cluster could have a separate, floating IP/DNS for the active mgr.   When an active mgr fails over (or goes down), the new active mgr enables this floating IP and starts responding to mgr requests.  Any passive mgrs would, of course, turn this IP off to assure that requests go to the active mgr. Just a thought... -Dave Dave Hall Binghamton University kdhall@binghamton.edu On 2/27/2020 3:01 AM, Michael Bisig wrote:

Hi all,

A similar question would be if it is possible to let passive mgr do the data collection!?

We run 14.2.6 on a medium 2.5PB cluster with over 900M objects (rbd and mainly S3) . At the moment, we face an issue with the prometheus exporter while it has high load. (e.g. while we insert a new host or a rebalance is active) The CPU usage of the active mgr daemon raises over 100% (this issue is still not completely solved, see discussion on ceph-users in "High CPU usage by ceph-mgr in 14.2.6") what leads to the fact that the daemon cannot answer the scrape request. We already reduced the scrape frequency radically (I think it is at 30s and was already at 60s) with no effect. Also a curl on the active mgr doesn't respond. Once the load goes away the mgr can answer the requests again. But somehow, the monitoring is especially interesting while there is high load so this is not ideal. Did anybody experience a similar issue?

Thanks in advance, Michael

On 27.02.20, 08:58, "黄明友" <hmy@v.photos> wrote:

thank you ,you are right.

------------------------------------------------------------------ 发件人：Jan Fajerski <jfajerski@suse.com> 发送时间：2020年2月27日(星期四) 15:34 收件人：ceph-users <ceph-users@ceph.io> 主　题：[ceph-users] Re: ceph prometheus module no export content

On Thu, Feb 27, 2020 at 10:20:07AM +0800, 黄明友 wrote: > hi,all > I had enable the prometheus module on my ceph cluster , the > ceph version > is ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nauti > lus (stable). > when I eanble this modle, can get exported content from the prometheus > module. > but , when I restart all the ceph cluster's node , the module can't > expor any content , curl http://host/metrics ; get blan; > the tcp/9283 port is listen; I had disable and enable this module,but > still no content export. You're probably querying a standby mgr daemon. Check ceph -s for the current active and query that one. For a prometheus server you always want to scrape all mgr daemons. Also see https://docs.ceph.com/docs/master/mgr/prometheus/ > 黄明友 > IT基础架构部经理 > V.Photos 云摄影 > 移动电话: +86 13540630430 > 客服电话：400 - 806 - 5775 > 电子邮件: hmy@v.photos > 官方网址: www.v.photos > 上海 黄浦区中山东二路88号外滩SOHO3Q F栋 2层 > 北京 朝阳区光华路9号光华路SOHO二期南二门SOHO3Q 1层 > 广州 天河区林和中路136号天誉花园二期3Wcoffice 天誉青创社区 > 深圳 南山区蛇口网谷科技大厦二期A座102网谷双创街 1层 > 成都 成华区建设路世贸广场 7层 > [cid:__aliyun158277000719672821]

>_______________________________________________ >ceph-users mailing list -- ceph-users@ceph.io >To unsubscribe send an email to ceph-users-leave@ceph.io

-- Jan Fajerski Senior Software Engineer Enterprise Storage SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io