Zabbix module Octopus 15.2.3
Trying to configure Zabbix module in Octopus 15.2.3. CentOS 8.1 environment. Installed zabbix40-agent for CentOS 8.1 (from epel repository). This will also install zabbix_sender. After enabling the Zabbix module in Ceph, I configured my Zabbix host and Zabbix identifier. # ceph zabbix config-set zabbix_host <zabbix-fqdn> # ceph zabbix config-set zabbix_identifier <ident> # ceph zabbix config-show Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1153, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/zabbix/module.py", line 407, in handle_command return 0, json.dumps(self.config, index=4, sort_keys=True), '' File "/lib64/python3.6/json/__init__.py", line 238, in dumps **kw).encode(obj) TypeError: __init__() got an unexpected keyword argument 'index' # ceph -v ceph version 15.2.3 (d289bbdec69ed7c1f516e0a093594580a76b78d0) octopus (stable) # ceph health detail HEALTH_OK Anyone found a solution? rgds, -gw
I can vaguely remember having an argument with zabbix developers on how inefficient they were putting metrics in the mysql database, causing unnecessary high load. I guess this was the time of mysql 3. I never can let go the feeling that if people are applying a type of logics, they will continue to do so. Anyway yesterday, I set up a prometheus instance and downloaded the ceph dashboards (am using influx currently). Looks promising. I am interested to see if I can create a setup that dynamically increases/decreases metrics sampling on specific alerts and if downsampling is better than with influx. -----Original Message----- Subject: [ceph-users] Zabbix module Octopus 15.2.3 Trying to configure Zabbix module in Octopus 15.2.3. CentOS 8.1 environment. Installed zabbix40-agent for CentOS 8.1 (from epel repository). This will also install zabbix_sender. After enabling the Zabbix module in Ceph, I configured my Zabbix host and Zabbix identifier. # ceph zabbix config-set zabbix_host <zabbix-fqdn> # ceph zabbix config-set zabbix_identifier <ident> # ceph zabbix config-show Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1153, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/zabbix/module.py", line 407, in handle_command return 0, json.dumps(self.config, index=4, sort_keys=True), '' File "/lib64/python3.6/json/__init__.py", line 238, in dumps **kw).encode(obj) TypeError: __init__() got an unexpected keyword argument 'index' # ceph -v ceph version 15.2.3 (d289bbdec69ed7c1f516e0a093594580a76b78d0) octopus (stable) # ceph health detail HEALTH_OK Anyone found a solution? rgds, -gw _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I'm going to resurrect this thread to throw my hat in the ring as I am having this issue as well. I just moved to 15.2.4 on Ubuntu 18.04/bionic, and Zabbix is 5.0.2.
$ ceph zabbix config-show Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1167, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/zabbix/module.py", line 407, in handle_command return 0, json.dumps(self.config, index=4, sort_keys=True), '' File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps **kw).encode(obj) TypeError: __init__() got an unexpected keyword argument 'index'
Which looks to be exactly the same as your error. I also appear to be seeing some weird issue with the Zabbix template as it pertains to the discovery and daemon/pool stats. I made sure that I pulled the latest XML from the ceph repo here: https://github.com/ceph/ceph/blob/master/src/pybind/mgr/zabbix/zabbix_templa... <https://github.com/ceph/ceph/blob/master/src/pybind/mgr/zabbix/zabbix_template.xml> When I run the discovery command, it looks like it runs ok?
$ ceph zabbix discovery Sending discovery data to Zabbix
And if I pass the --verbose flag I get this at the end
better match: 1.5 > 0.5: zabbix config-set <key> <value> better match: 1.5 > 1.5: zabbix config-show better match: 1.5 > 1.5: zabbix send better match: 2.5 > 1.5: zabbix discovery bestcmds_sorted: [{'flags': 8, 'help': 'Discovering Zabbix data', 'module': 'mgr', 'perm': 'r', 'sig': [argdesc(<class 'ceph_argparse.CephPrefix'>, req=True, name=prefix, n=1, numseen=0, prefix=zabbix), argdesc(<class 'ceph_argparse.CephPrefix'>, req=True, name=prefix, n=1, numseen=0, prefix=discovery)]}] Submitting command: {'prefix': 'zabbix discovery', 'target': ('mon-mgr', '')} submit ['{"prefix": "zabbix discovery", "target": ["mon-mgr", ""]}'] to mon-mgr Sending discovery data to Zabbix
The pools get discovered correctly, however the pool % used doesn't work, and the R/W iops/BW are off by a magnitude of 10^9 I think. [fs-metadata] Pool Percent Used 07/27/2020 06:17:22 PM 0% [fs-metadata] Pool RAW Used 07/28/2020 05:07:22 PM 180 Gb +2.52 Mb [fs-metadata] Pool Read bandwidth 07/28/2020 05:07:22 PM 4.3 Tbytes +560.13 Kbytes [fs-metadata] Pool Read operations 07/28/2020 05:07:22 PM 460.18 Mops +170 ops [fs-metadata] Pool Used 07/28/2020 05:07:22 PM 180 Gb +2.52 Mb [fs-metadata] Pool Write bandwidth 07/28/2020 05:07:22 PM 1.43 Tbytes +337.92 Kbytes [fs-metadata] Pool Write operations 07/28/2020 05:07:22 PM 62.04 Mops +99 ops However, typical patterns are this:
pool fs-metadata id 16 client io 3.9 KiB/s rd, 3.3 KiB/s wr, 2 op/s rd, 0 op/s wr
And Zabbix shows this error for the "Pool Percent Used" Item.
Value of type "string" is not suitable for value type "Numeric (unsigned)". Value "0.014200450852513313" So it looks like this should be a float, and thats a pretty easy change.
The other, much bigger, issue I am seeing is with the discovery for OSD's. It appears like its descending the crush tree and selecting the crush failure domains. For one tree, it is chassis, for another it is host. And the OSD values it is showing "[osd.-##]" correspond directly to the crush ID number. Table of the Items below. Ceph OSD discovery: [osd.-18] OSD fill Triggers 2 ceph.[osd.-18,osd_fill] 90d 365d Zabbix trapper Ceph CRUSH [ssd] Enabled Ceph OSD discovery: [osd.-18] OSD in ceph.[osd.-18,in] 90d 365d Zabbix trapper Ceph CRUSH [ssd] Enabled Ceph OSD discovery: [osd.-18] OSD latency apply ceph.[osd.-18,osd_latency_apply] 90d 365d Zabbix trapper Ceph CRUSH [ssd] Enabled Ceph OSD discovery: [osd.-18] OSD latency commit ceph.[osd.-18,osd_latency_commit] 90d 365d Zabbix trapper Ceph CRUSH [ssd] Enabled Ceph OSD discovery: [osd.-18] OSD PGs ceph.[osd.-18,num_pgs] 90d 365d Zabbix trapper Ceph CRUSH [ssd] Enabled Ceph OSD discovery: [osd.-18] OSD up Triggers 1 ceph.[osd.-18,up] 90d 365d Zabbix trapper Ceph CRUSH [ssd] Enabled Ceph OSD discovery: [osd.-55] OSD fill Triggers 2 ceph.[osd.-55,osd_fill] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-55] OSD in ceph.[osd.-55,in] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-55] OSD latency apply ceph.[osd.-55,osd_latency_apply] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-55] OSD latency commit ceph.[osd.-55,osd_latency_commit] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-55] OSD PGs ceph.[osd.-55,num_pgs] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-55] OSD up Triggers 1 ceph.[osd.-55,up] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-56] OSD fill Triggers 2 ceph.[osd.-56,osd_fill] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-56] OSD in ceph.[osd.-56,in] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-56] OSD latency apply ceph.[osd.-56,osd_latency_apply] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-56] OSD latency commit ceph.[osd.-56,osd_latency_commit] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-56] OSD PGs ceph.[osd.-56,num_pgs] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled Ceph OSD discovery: [osd.-56] OSD up Triggers 1 ceph.[osd.-56,up] 90d 365d Zabbix trapper Ceph CRUSH [default] Enabled One last "oddity" in my environment setup is that I'm not running containerized, I'm just running "legacy" as cephadm states, but hopefully this isn't the root cause of this. Reed
On Jun 6, 2020, at 3:24 AM, Gert Wieberdink <gert.wieberdink@ziggo.nl> wrote:
Trying to configure Zabbix module in Octopus 15.2.3. CentOS 8.1 environment. Installed zabbix40-agent for CentOS 8.1 (from epel repository). This will also install zabbix_sender. After enabling the Zabbix module in Ceph, I configured my Zabbix host and Zabbix identifier. # ceph zabbix config-set zabbix_host <zabbix-fqdn> # ceph zabbix config-set zabbix_identifier <ident> # ceph zabbix config-show Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1153, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/zabbix/module.py", line 407, in handle_command return 0, json.dumps(self.config, index=4, sort_keys=True), '' File "/lib64/python3.6/json/__init__.py", line 238, in dumps **kw).encode(obj) TypeError: __init__() got an unexpected keyword argument 'index'
# ceph -v ceph version 15.2.3 (d289bbdec69ed7c1f516e0a093594580a76b78d0) octopus (stable) # ceph health detail HEALTH_OK
Anyone found a solution? rgds, -gw _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Reed, Gert, list, On 28/07/2020 23:42, Reed Dier wrote:
I'm going to resurrect this thread to throw my hat in the ring as I am having this issue as well.
Did not see any solutions to this thread (be it this one, or any thread more recently) so forgive me for re-resurrecting this thread :-).
I just moved to 15.2.4 on Ubuntu 18.04/bionic, and Zabbix is 5.0.2.
$ ceph zabbix config-show Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1167, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/zabbix/module.py", line 407, in handle_command return 0, json.dumps(self.config, index=4, sort_keys=True), '' File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps **kw).encode(obj) TypeError: __init__() got an unexpected keyword argument 'index'
Which looks to be exactly the same as your error.
As I ran into this exact error as well, I dug a bit deeper in the source code. Currently running ceph-mgr-modules-core on Octopus 15.2.6 / Ubuntu 20.04: # ceph zabbix config-show Error EINVAL: Traceback (most recent call last): File "/usr/share/ceph/mgr/mgr_module.py", line 1177, in _handle_command return self.handle_command(inbuf, cmd) File "/usr/share/ceph/mgr/zabbix/module.py", line 407, in handle_command return 0, json.dumps(self.config, index=4, sort_keys=True), '' File "/lib/python3.8/json/__init__.py", line 234, in dumps return cls( TypeError: __init__() got an unexpected keyword argument 'index' This bug was introduced here <https://github.com/ceph/ceph/commit/0e9303d0599c92e575aae7acca87328ed8ee9527>, where `index=4` was introduced, but it had to be `indent=4`. It has been fixed in <https://github.com/ceph/ceph/commit/ddceba1647096455a506a72d3bfe911e7e6e828a> for pacific, as well as backported to octopus: <https://github.com/ceph/ceph/commit/c40d97ae23d912532ad0c9330b0ed96b1477f20c> via <https://github.com/ceph/ceph/pull/37128>. Relevant bug tracker links: * <https://tracker.ceph.com/issues/47426> * <https://tracker.ceph.com/issues/47404> On an updated git clone of <https://github.com/ceph/ceph>: $ git tag --contains c40d97ae23d912532ad0c9330b0ed96b1477f20c | sort -V v15.2.8 v15.2.9 v15.2.10 v15.2.11 v15.2.12 So, you could upgrade to at least 15.2.8 to have this bug fixed. Or: simply patch the affected file /usr/share/ceph/mgr/zabbix/module.py (see the backported commit <https://github.com/ceph/ceph/commit/c40d97ae23d912532ad0c9330b0ed96b1477f20c>), disable and enable the zabbix module: # ceph mgr module disable zabbix # ceph mgr module enable zabbix # ceph zabbix config-show { "discovery_interval": 100, "identifier": "ceph.redacted.local", "interval": 60, "log_level": "", "log_to_cluster": false, "log_to_cluster_level": "info", "log_to_file": false, "zabbix_host": "zabbix.redacted.local", "zabbix_port": 10051, "zabbix_sender": "/usr/bin/zabbix_sender" } Hope this helps. Best regards, Gerdriaan Mulder
participants (5)
-
Gerdriaan Mulder
-
Gert Wieberdink
-
Marc Roos
-
oladamats4@gmail.com
-
Reed Dier