Monitoring Ceph Bucket and overall ceph cluster remaining space
Is there an easy way to poll the ceph cluster buckets in a way to see how much space is remaining? And is it possible to see how much ceph cluster space is remaining overall? I am trying to extract the data from our Ceph cluster and put it into a format that our SolarWinds can understand in whole number integers, so we can monitor bucket allocated space and overall cluster space in the cluster as a whole. Via Canonical support, the said I can do something like "sudo ceph df -f json-pretty" to pull the information, but what is it I need to look at from the output (see below) to display over to SolarWinds? { "stats": { "total_bytes": 960027263238144, "total_avail_bytes": 403965214187520, "total_used_bytes": 556062049050624, "total_used_raw_bytes": 556062049050624, "total_used_raw_ratio": 0.57921481132507324, "num_osds": 48, "num_per_pool_osds": 48, "num_per_pool_omap_osds": 48 }, "stats_by_class": { "ssd": { "total_bytes": 960027263238144, "total_avail_bytes": 403965214187520, "total_used_bytes": 556062049050624, "total_used_raw_bytes": 556062049050624, "total_used_raw_ratio": 0.57921481132507324 } }, And a couple of data pools... { "name": "default.rgw.jv-va-pool.data", "id": 65, "stats": { "stored": 4343441915904, "objects": 17466616, "kb_used": 12774490932, "bytes_used": 13081078714368, "percent_used": 0.053900588303804398, "max_avail": 76535973281792 } }, { "name": "default.rgw.jv-va-pool.index", "id": 66, "stats": { "stored": 42533675008, "objects": 401, "kb_used": 124610380, "bytes_used": 127601028363, "percent_used": 0.00055542576592415571, "max_avail": 76535973281792 } }, This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments.
Hi, For RGW usage statistics you can use radosgw_usage_exporter [1] k [1] https://github.com/blemmenes/radosgw_usage_exporter Sent from my iPhone
On 6 Mar 2024, at 00:21, Michael Worsham <mworsham@datadimensions.com> wrote: Is there an easy way to poll the ceph cluster buckets in a way to see how much space is remaining? And is it possible to see how much ceph cluster space is remaining overall? I am trying to extract the data from our Ceph cluster and put it into a format that our SolarWinds can understand in whole number integers, so we can monitor bucket allocated space and overall cluster space in the cluster as a whole.
Via Canonical support, the said I can do something like "sudo ceph df -f json-pretty" to pull the information, but what is it I need to look at from the output (see below) to display over to SolarWinds?
{ "stats": { "total_bytes": 960027263238144, "total_avail_bytes": 403965214187520, "total_used_bytes": 556062049050624, "total_used_raw_bytes": 556062049050624, "total_used_raw_ratio": 0.57921481132507324, "num_osds": 48, "num_per_pool_osds": 48, "num_per_pool_omap_osds": 48 }, "stats_by_class": { "ssd": { "total_bytes": 960027263238144, "total_avail_bytes": 403965214187520, "total_used_bytes": 556062049050624, "total_used_raw_bytes": 556062049050624, "total_used_raw_ratio": 0.57921481132507324 } },
And a couple of data pools... { "name": "default.rgw.jv-va-pool.data", "id": 65, "stats": { "stored": 4343441915904, "objects": 17466616, "kb_used": 12774490932, "bytes_used": 13081078714368, "percent_used": 0.053900588303804398, "max_avail": 76535973281792 } }, { "name": "default.rgw.jv-va-pool.index", "id": 66, "stats": { "stored": 42533675008, "objects": 401, "kb_used": 124610380, "bytes_used": 127601028363, "percent_used": 0.00055542576592415571, "max_avail": 76535973281792 } }, This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
This looks interesting, but instead of Prometheus, could the data be exported for SolarWinds? The intent is to have SW watch the available storage space allocated and then to alert when a certain threshold is reached (75% remaining for a warning; 95% remaining for a critical). -- Michael ________________________________ From: Konstantin Shalygin <k0ste@k0ste.ru> Sent: Tuesday, March 5, 2024 11:17:10 PM To: Michael Worsham <mworsham@datadimensions.com> Cc: ceph-users@ceph.io <ceph-users@ceph.io> Subject: Re: [ceph-users] Monitoring Ceph Bucket and overall ceph cluster remaining space This is an external email. Please take care when clicking links or opening attachments. When in doubt, check with the Help Desk or Security. Hi, For RGW usage statistics you can use radosgw_usage_exporter [1] k [1] https://github.com/blemmenes/radosgw_usage_exporter Sent from my iPhone On 6 Mar 2024, at 00:21, Michael Worsham <mworsham@datadimensions.com> wrote: Is there an easy way to poll the ceph cluster buckets in a way to see how much space is remaining? And is it possible to see how much ceph cluster space is remaining overall? I am trying to extract the data from our Ceph cluster and put it into a format that our SolarWinds can understand in whole number integers, so we can monitor bucket allocated space and overall cluster space in the cluster as a whole. Via Canonical support, the said I can do something like "sudo ceph df -f json-pretty" to pull the information, but what is it I need to look at from the output (see below) to display over to SolarWinds? { "stats": { "total_bytes": 960027263238144, "total_avail_bytes": 403965214187520, "total_used_bytes": 556062049050624, "total_used_raw_bytes": 556062049050624, "total_used_raw_ratio": 0.57921481132507324, "num_osds": 48, "num_per_pool_osds": 48, "num_per_pool_omap_osds": 48 }, "stats_by_class": { "ssd": { "total_bytes": 960027263238144, "total_avail_bytes": 403965214187520, "total_used_bytes": 556062049050624, "total_used_raw_bytes": 556062049050624, "total_used_raw_ratio": 0.57921481132507324 } }, And a couple of data pools... { "name": "default.rgw.jv-va-pool.data", "id": 65, "stats": { "stored": 4343441915904, "objects": 17466616, "kb_used": 12774490932, "bytes_used": 13081078714368, "percent_used": 0.053900588303804398, "max_avail": 76535973281792 } }, { "name": "default.rgw.jv-va-pool.index", "id": 66, "stats": { "stored": 42533675008, "objects": 401, "kb_used": 124610380, "bytes_used": 127601028363, "percent_used": 0.00055542576592415571, "max_avail": 76535973281792 } }, This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments.
Hi, Don't aware about what is SW, but if this software works with Prometheus metrics format - why not. Anyway the exporters are open source, you can modify the existing code for your environment k Sent from my iPhone
On 6 Mar 2024, at 07:58, Michael Worsham <mworsham@datadimensions.com> wrote:
This looks interesting, but instead of Prometheus, could the data be exported for SolarWinds?
The intent is to have SW watch the available storage space allocated and then to alert when a certain threshold is reached (75% remaining for a warning; 95% remaining for a critical).
SW is SolarWinds (www.soparwinds.com), a network and application monitoring and alerting platform. It's not very open source at all, but it's what we use for monitoring all of our physical and virtual servers, network switches, SAN and NAS devices, and anything else with a network card in it. ________________________________ From: Konstantin Shalygin <k0ste@k0ste.ru> Sent: Wednesday, March 6, 2024 1:39:43 AM To: Michael Worsham <mworsham@datadimensions.com> Cc: ceph-users@ceph.io <ceph-users@ceph.io> Subject: Re: [ceph-users] Re: Monitoring Ceph Bucket and overall ceph cluster remaining space This is an external email. Please take care when clicking links or opening attachments. When in doubt, check with the Help Desk or Security. Hi, Don't aware about what is SW, but if this software works with Prometheus metrics format - why not. Anyway the exporters are open source, you can modify the existing code for your environment k Sent from my iPhone
On 6 Mar 2024, at 07:58, Michael Worsham <mworsham@datadimensions.com> wrote:
This looks interesting, but instead of Prometheus, could the data be exported for SolarWinds?
The intent is to have SW watch the available storage space allocated and then to alert when a certain threshold is reached (75% remaining for a warning; 95% remaining for a critical).
This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments.
participants (2)
-
Konstantin Shalygin
-
Michael Worsham