Grafana vulnerability - cephadm deployment
With cephadm reef (v18.2.4), is there a way to upgrade the Grafana that is incorporated into the docker build? Currently it's showing Grafana v9.4.7 (quay.io/ceph/ceph-grafana:9.4.7) and that is actually an open vulnerability per CVE-2023-1387 (Grafana 9.1.0 < 9.2.17, 9.3.x < 9.3.13, 9.4.x < 9.4.9 Information Disclosure Vulnerability). Grafana shows that it's fixed with version 9.4.9 and above (https://grafana.com/security/security-advisories/cve-2023-1387/). -- Michael This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments.
This is covered in the docs: https://docs.ceph.com/en/reef/cephadm/services/monitoring/#using-custom-imag...
On Apr 16, 2025, at 2:55 PM, Michael Worsham <mworsham@datadimensions.com> wrote:
With cephadm reef (v18.2.4), is there a way to upgrade the Grafana that is incorporated into the docker build?
Currently it's showing Grafana v9.4.7 (quay.io/ceph/ceph-grafana:9.4.7) and that is actually an open vulnerability per CVE-2023-1387 (Grafana 9.1.0 < 9.2.17, 9.3.x < 9.3.13, 9.4.x < 9.4.9 Information Disclosure Vulnerability). Grafana shows that it's fixed with version 9.4.9 and above (https://grafana.com/security/security-advisories/cve-2023-1387/).
-- Michael
This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi, Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
https://docs.ceph.com/en/reef/cephadm/services/monitoring/#using-custom-imag...
There is a newer Grafana container available at quay.io/ceph/ceph-grafana:9.4.12 You can use it with # ceph config set mgr mgr/cephadm/container_image_grafana quay.io/ceph/ceph-grafana:9.4.12 # ceph orch redeploy grafana Regards -- Robert Sander Linux Consultant Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin
But Grafana 9.4 is EOL for a long time. Shouldn't it be time to upgrade the image? Kind regards, Sake
Op 17-04-2025 09:14 CEST schreef Robert Sander <r.sander@heinlein-support.de>:
Hi,
Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
https://docs.ceph.com/en/reef/cephadm/services/monitoring/#using-custom-imag...
There is a newer Grafana container available at quay.io/ceph/ceph-grafana:9.4.12
You can use it with
# ceph config set mgr mgr/cephadm/container_image_grafana quay.io/ceph/ceph-grafana:9.4.12 # ceph orch redeploy grafana
Regards -- Robert Sander Linux Consultant
Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
https://www.heinlein-support.de
Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-grafana should be upgraded to 10.4 or later because it is not compatible with the latest prometheus alertmanager (0.27 or later) which only support the alertmanager V2 API. Is there an issue to track this? ________________________________ From: Sake Ceph <ceph@paulusma.eu> Sent: Thursday, April 17, 2025 9:35 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment But Grafana 9.4 is EOL for a long time. Shouldn't it be time to upgrade the image? Kind regards, Sake
Op 17-04-2025 09:14 CEST schreef Robert Sander <r.sander@heinlein-support.de>:
Hi,
Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
There is a newer Grafana container available at quay.io/ceph/ceph-grafana:9.4.12
You can use it with
# ceph config set mgr mgr/cephadm/container_image_grafana quay.io/ceph/ceph-grafana:9.4.12 # ceph orch redeploy grafana
Regards -- Robert Sander Linux Consultant
Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I was able to upgrade to v9.4.12, so thanks for that btw. However, it seems that v9.4.12 is still at risk per the vulnerability scans. CVE-2023-3128 (https://nvd.nist.gov/vuln/detail/CVE-2023-3128) CVE-2024-1442 (https://nvd.nist.gov/vuln/detail/CVE-2024-1442) Affects: Grafana 6.7.x < 8.5.27, 9.x < 9.2.20, 9.3.x < 9.3.16, 9.4.x < 9.4.13, 9.5.x < 9.5.5, 10.x < 10.0.1 Authentication Bypass Vulnerability When I looked at the Ceph-Grafana Quay.io site "https://quay.io/repository/ceph/ceph-grafana?tab=tags", it seems the last build was over a year ago. Is anyone actually working on this? Even the Quay Security Scanner has detected 586 vulnerabilities, patches are available for 283 vulnerabilities. I was looking for a place to raise an issue on it, but the quay.io site doesn't show a link any place. -- Michael ________________________________ From: Wyll Ingersoll Sent: Thursday, April 17, 2025 9:40 AM To: Sake Ceph; ceph-users@ceph.io Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment This is an external email. Please take care when clicking links or opening attachments. When in doubt, check with the Help Desk or Security. ceph-grafana should be upgraded to 10.4 or later because it is not compatible with the latest prometheus alertmanager (0.27 or later) which only support the alertmanager V2 API. Is there an issue to track this? ________________________________ From: Sake Ceph <ceph@paulusma.eu> Sent: Thursday, April 17, 2025 9:35 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment But Grafana 9.4 is EOL for a long time. Shouldn't it be time to upgrade the image? Kind regards, Sake
Op 17-04-2025 09:14 CEST schreef Robert Sander <r.sander@heinlein-support.de>:
Hi,
Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
There is a newer Grafana container available at quay.io/ceph/ceph-grafana:9.4.12
You can use it with
# ceph config set mgr mgr/cephadm/container_image_grafana quay.io/ceph/ceph-grafana:9.4.12 # ceph orch redeploy grafana
Regards -- Robert Sander Linux Consultant
Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
https://www.heinlein-support.de/<https://www.heinlein-support.de/>
Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments.
`main` tells me: ./src/python-common/ceph/cephadm/images.py: GRAFANA = _create_image('quay.io/ceph/grafana:10.4.16', 'grafana') The `reef` branch: ./src/cephadm/cephadm.py:DEFAULT_GRAFANA_IMAGE = 'quay.io/ceph/ceph-grafana:9.4.7' YMMV, but looking at the CVE I’m not panicking - one has to enable a non-default option, and then you’re still only vulnerable to insiders, unless you leave your Grafana endpoint exposed on a non-ACL’d routable address. 9.4.x may be EOL, but it wasn’t when Reef was released. A quick search on tracker.ceph.com <http://tracker.ceph.com/> does not find a hit for CVE-2023-1387 I suggest opening an issue, this is a simple one-line fix but it’s not immediately clear to me how to properly open a PR against the reef branch. TL;DR: PROMETHEUS = _create_image('quay.io/prometheus/prometheus:v2.51.0', 'prometheus') LOKI = _create_image('docker.io/grafana/loki:3.0.0', 'loki') PROMTAIL = _create_image('docker.io/grafana/promtail:3.0.0', 'promtail') NODE_EXPORTER = _create_image('quay.io/prometheus/node-exporter:v1.7.0', 'node_exporter') ALERTMANAGER = _create_image('quay.io/prometheus/alertmanager:v0.27.0', 'alertmanager') GRAFANA = _create_image('quay.io/ceph/grafana:10.4.16', 'grafana') HAPROXY = _create_image('quay.io/ceph/haproxy:2.3', 'haproxy') KEEPALIVED = _create_image('quay.io/ceph/keepalived:2.2.4', 'keepalived') NVMEOF = _create_image('quay.io/ceph/nvmeof:1.5', 'nvmeof') SNMP_GATEWAY = _create_image('docker.io/maxwo/snmp-notifier:v1.2.1', 'snmp_gateway') ELASTICSEARCH = _create_image('quay.io/omrizeneva/elasticsearch:6.8.23', 'elasticsearch') JAEGER_COLLECTOR = _create_image('quay.io/jaegertracing/jaeger-collector:1.29', 'jaeger_collector') JAEGER_AGENT = _create_image('quay.io/jaegertracing/jaeger-agent:1.29', 'jaeger_agent') JAEGER_QUERY = _create_image('quay.io/jaegertracing/jaeger-query:1.29', 'jaeger_query') SAMBA = _create_image('quay.io/samba.org/samba-server:devbuilds-centos-amd64', 'samba') SAMBA_METRICS = _create_image('quay.io/samba.org/samba-metrics:latest', 'samba_metrics') NGINX = _create_image('quay.io/ceph/nginx:sclorg-nginx-126', 'nginx') OAUTH2_PROXY = _create_image('quay.io/oauth2-proxy/oauth2-proxy:v7.6.0', 'oauth2_proxy’)
On Apr 17, 2025, at 9:40 AM, Wyll Ingersoll <wyllys.ingersoll@keepertech.com> wrote:
ceph-grafana should be upgraded to 10.4 or later because it is not compatible with the latest prometheus alertmanager (0.27 or later) which only support the alertmanager V2 API.
Is there an issue to track this?
________________________________ From: Sake Ceph <ceph@paulusma.eu> Sent: Thursday, April 17, 2025 9:35 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment
But Grafana 9.4 is EOL for a long time. Shouldn't it be time to upgrade the image?
Kind regards, Sake
Op 17-04-2025 09:14 CEST schreef Robert Sander <r.sander@heinlein-support.de>:
Hi,
Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
There is a newer Grafana container available at quay.io/ceph/ceph-grafana:9.4.12
You can use it with
# ceph config set mgr mgr/cephadm/container_image_grafana quay.io/ceph/ceph-grafana:9.4.12 # ceph orch redeploy grafana
Regards -- Robert Sander Linux Consultant
Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py I couldn't find any issues, but the company really doesn't like old software with know issues :)
Op 17-04-2025 16:07 CEST schreef Anthony D'Atri <anthony.datri@gmail.com>:
`main` tells me:
./src/python-common/ceph/cephadm/images.py: GRAFANA = _create_image('quay.io/ceph/grafana:10.4.16', 'grafana')
The `reef` branch:
./src/cephadm/cephadm.py:DEFAULT_GRAFANA_IMAGE = 'quay.io/ceph/ceph-grafana:9.4.7'
YMMV, but looking at the CVE I’m not panicking - one has to enable a non-default option, and then you’re still only vulnerable to insiders, unless you leave your Grafana endpoint exposed on a non-ACL’d routable address.
9.4.x may be EOL, but it wasn’t when Reef was released.
A quick search on tracker.ceph.com <http://tracker.ceph.com/> does not find a hit for CVE-2023-1387
I suggest opening an issue, this is a simple one-line fix but it’s not immediately clear to me how to properly open a PR against the reef branch.
TL;DR:
PROMETHEUS = _create_image('quay.io/prometheus/prometheus:v2.51.0', 'prometheus') LOKI = _create_image('docker.io/grafana/loki:3.0.0', 'loki') PROMTAIL = _create_image('docker.io/grafana/promtail:3.0.0', 'promtail') NODE_EXPORTER = _create_image('quay.io/prometheus/node-exporter:v1.7.0', 'node_exporter') ALERTMANAGER = _create_image('quay.io/prometheus/alertmanager:v0.27.0', 'alertmanager') GRAFANA = _create_image('quay.io/ceph/grafana:10.4.16', 'grafana') HAPROXY = _create_image('quay.io/ceph/haproxy:2.3', 'haproxy') KEEPALIVED = _create_image('quay.io/ceph/keepalived:2.2.4', 'keepalived') NVMEOF = _create_image('quay.io/ceph/nvmeof:1.5', 'nvmeof') SNMP_GATEWAY = _create_image('docker.io/maxwo/snmp-notifier:v1.2.1', 'snmp_gateway') ELASTICSEARCH = _create_image('quay.io/omrizeneva/elasticsearch:6.8.23', 'elasticsearch') JAEGER_COLLECTOR = _create_image('quay.io/jaegertracing/jaeger-collector:1.29', 'jaeger_collector') JAEGER_AGENT = _create_image('quay.io/jaegertracing/jaeger-agent:1.29', 'jaeger_agent') JAEGER_QUERY = _create_image('quay.io/jaegertracing/jaeger-query:1.29', 'jaeger_query') SAMBA = _create_image('quay.io/samba.org/samba-server:devbuilds-centos-amd64', 'samba') SAMBA_METRICS = _create_image('quay.io/samba.org/samba-metrics:latest', 'samba_metrics') NGINX = _create_image('quay.io/ceph/nginx:sclorg-nginx-126', 'nginx') OAUTH2_PROXY = _create_image('quay.io/oauth2-proxy/oauth2-proxy:v7.6.0', 'oauth2_proxy’)
On Apr 17, 2025, at 9:40 AM, Wyll Ingersoll <wyllys.ingersoll@keepertech.com> wrote:
ceph-grafana should be upgraded to 10.4 or later because it is not compatible with the latest prometheus alertmanager (0.27 or later) which only support the alertmanager V2 API.
Is there an issue to track this?
________________________________ From: Sake Ceph <ceph@paulusma.eu> Sent: Thursday, April 17, 2025 9:35 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment
But Grafana 9.4 is EOL for a long time. Shouldn't it be time to upgrade the image?
Kind regards, Sake
Op 17-04-2025 09:14 CEST schreef Robert Sander <r.sander@heinlein-support.de>:
Hi,
Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
There is a newer Grafana container available at quay.io/ceph/ceph-grafana:9.4.12
You can use it with
# ceph config set mgr mgr/cephadm/container_image_grafana quay.io/ceph/ceph-grafana:9.4.12 # ceph orch redeploy grafana
Regards -- Robert Sander Linux Consultant
Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
We deprecated [1] the use of quay.io/ceph/*ceph-grafana* images from squid onwards due to the pain of building and maintaining those images by ourselves. The main reason we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image. But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/*grafana *image and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah <afrahman@redhat.com> here: https://github.com/ceph/ceph/pull/62827. We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config. Will check both of them next week after the holidays. [1] https://docs.ceph.com/en/latest/releases/squid/#monitoring Regards, Nizam On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote:
Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old software with know issues :)
Op 17-04-2025 16:07 CEST schreef Anthony D'Atri <anthony.datri@gmail.com :
`main` tells me:
./src/python-common/ceph/cephadm/images.py: GRAFANA = _create_image(' quay.io/ceph/grafana:10.4.16', 'grafana')
The `reef` branch:
./src/cephadm/cephadm.py:DEFAULT_GRAFANA_IMAGE = ' quay.io/ceph/ceph-grafana:9.4.7'
YMMV, but looking at the CVE I’m not panicking - one has to enable a non-default option, and then you’re still only vulnerable to insiders, unless you leave your Grafana endpoint exposed on a non-ACL’d routable address.
9.4.x may be EOL, but it wasn’t when Reef was released.
A quick search on tracker.ceph.com <http://tracker.ceph.com/> does not find a hit for CVE-2023-1387
I suggest opening an issue, this is a simple one-line fix but it’s not immediately clear to me how to properly open a PR against the reef branch.
TL;DR:
PROMETHEUS = _create_image('quay.io/prometheus/prometheus:v2.51.0', 'prometheus') LOKI = _create_image('docker.io/grafana/loki:3.0.0', 'loki') PROMTAIL = _create_image('docker.io/grafana/promtail:3.0.0', 'promtail') NODE_EXPORTER = _create_image(' quay.io/prometheus/node-exporter:v1.7.0', 'node_exporter') ALERTMANAGER = _create_image(' quay.io/prometheus/alertmanager:v0.27.0', 'alertmanager') GRAFANA = _create_image('quay.io/ceph/grafana:10.4.16', 'grafana') HAPROXY = _create_image('quay.io/ceph/haproxy:2.3', 'haproxy') KEEPALIVED = _create_image('quay.io/ceph/keepalived:2.2.4', 'keepalived') NVMEOF = _create_image('quay.io/ceph/nvmeof:1.5', 'nvmeof') SNMP_GATEWAY = _create_image('docker.io/maxwo/snmp-notifier:v1.2.1', 'snmp_gateway') ELASTICSEARCH = _create_image(' quay.io/omrizeneva/elasticsearch:6.8.23', 'elasticsearch') JAEGER_COLLECTOR = _create_image(' quay.io/jaegertracing/jaeger-collector:1.29', 'jaeger_collector') JAEGER_AGENT = _create_image(' quay.io/jaegertracing/jaeger-agent:1.29', 'jaeger_agent') JAEGER_QUERY = _create_image(' quay.io/jaegertracing/jaeger-query:1.29', 'jaeger_query') SAMBA = _create_image(' quay.io/samba.org/samba-server:devbuilds-centos-amd64', 'samba') SAMBA_METRICS = _create_image(' quay.io/samba.org/samba-metrics:latest', 'samba_metrics') NGINX = _create_image('quay.io/ceph/nginx:sclorg-nginx-126', 'nginx') OAUTH2_PROXY = _create_image(' quay.io/oauth2-proxy/oauth2-proxy:v7.6.0', 'oauth2_proxy’)
On Apr 17, 2025, at 9:40 AM, Wyll Ingersoll < wyllys.ingersoll@keepertech.com> wrote:
ceph-grafana should be upgraded to 10.4 or later because it is not compatible with the latest prometheus alertmanager (0.27 or later) which only support the alertmanager V2 API.
Is there an issue to track this?
________________________________ From: Sake Ceph <ceph@paulusma.eu> Sent: Thursday, April 17, 2025 9:35 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment
But Grafana 9.4 is EOL for a long time. Shouldn't it be time to upgrade the image?
Kind regards, Sake
Op 17-04-2025 09:14 CEST schreef Robert Sander < r.sander@heinlein-support.de>:
Hi,
Am 4/16/25 um 21:11 schrieb Anthony D'Atri:
This is covered in the docs:
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.ceph.com%2Fen%2Freef%2Fcephadm%2Fservices%2Fmonitoring%2F%23using-custom-images&data=05%7C02%7Cwyllys.ingersoll%40keepertech.com%7Cea14dd831bc8452956d908dd7db4df15%7Ca0e92bbdedfc4a2faf16799792ef0c87%7C0%7C0%7C638804939107001287%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=xN0ENm1GGSU8gadc6gqRO4KP21%2F1KHm%2FA3Hn4pcR7jg%3D&reserved=0 < https://docs.ceph.com/en/reef/cephadm/services/monitoring/#using-custom-imag...
There is a newer Grafana container available at
quay.io/ceph/ceph-grafana:9.4.12
You can use it with
# ceph config set mgr mgr/cephadm/container_image_grafana
quay.io/ceph/ceph-grafana:9.4.12
# ceph orch redeploy grafana
Regards -- Robert Sander Linux Consultant
Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.heinlein-support.de%2F&data=05%7C02%7Cwyllys.ingersoll%40keepertech.com%7Cea14dd831bc8452956d908dd7db4df15%7Ca0e92bbdedfc4a2faf16799792ef0c87%7C0%7C0%7C638804939107047712%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=rCO%2BmKAhv4X7XZUR2uEMAiSr6uLYcPtBLjmesxWnhfE%3D&reserved=0 <https://www.heinlein-support.de/>
Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Nizamudeen A Sr. Software Engineer - IBM Partner Engineer IBM and Red Hat Ceph Storage Red Hat <https://www.redhat.com/> <https://www.redhat.com/>
Really great you moved to the default grafana image and moving the customization to cephadm. This makes upgrading indeed easier. The problem is that Grafana has a stranger release cycle and therefor not really compatible with release cycle of Ceph. This would mean one or two update(s) of the supported Grafana minor version. And maybe with every Ceph release, update to the latest bugfix release? I guess a bugfix version should always work with the same minor version of Grafana. The user can configure this via the provided tools for setting custom images. We're only talking about Grafana, but this applicable for all the extra container images. Thank you for all the hard work you're doing for the project! These are just my 2 cents :D Kind regards, Sake
Op 17-04-2025 19:43 CEST schreef Nizamudeen A <nia@redhat.com>:
We deprecated [1] the use of quay.io/ceph/ (http://quay.io/ceph/)ceph-grafana images from squid onwards due to the pain of building and maintaining those images by ourselves. The main reason we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image.
But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/ (http://quay.io/ceph/)grafanaimage and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah here: https://github.com/ceph/ceph/pull/62827.
We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config.
Will check both of them next week after the holidays.
[1] https://docs.ceph.com/en/latest/releases/squid/#monitoring
Regards, Nizam
On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote:
Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old software with know issues :)
FWIW - Grafana 12 is going to be GA soon. 11.6 or later is probably where you want to target for “stable” releases for the moment. -- Dan Brown
On Apr 18, 2025, at 10:47, Sake Ceph <ceph@paulusma.eu> wrote:
Really great you moved to the default grafana image and moving the customization to cephadm. This makes upgrading indeed easier.
The problem is that Grafana has a stranger release cycle and therefor not really compatible with release cycle of Ceph. This would mean one or two update(s) of the supported Grafana minor version. And maybe with every Ceph release, update to the latest bugfix release? I guess a bugfix version should always work with the same minor version of Grafana. The user can configure this via the provided tools for setting custom images.
We're only talking about Grafana, but this applicable for all the extra container images.
Thank you for all the hard work you're doing for the project! These are just my 2 cents :D
Kind regards, Sake
Op 17-04-2025 19:43 CEST schreef Nizamudeen A <nia@redhat.com>:
We deprecated [1] the use of quay.io/ceph/ (http://quay.io/ceph/)ceph-grafana images from squid onwards due to the pain of building and maintaining those images by ourselves. The main reason we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image.
But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/ (http://quay.io/ceph/)grafanaimage and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah here: https://github.com/ceph/ceph/pull/62827.
We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config.
Will check both of them next week after the holidays.
[1] https://docs.ceph.com/en/latest/releases/squid/#monitoring
Regards, Nizam
On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote: Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old software with know issues :)
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
This thread is only about Grafana, but what about the other services like Prometheus, Loki, Promtail, alertmanager or node-exporter?
Op 19-04-2025 15:18 CEST schreef Daniel Brown <daniel.h.brown@thermify.cloud>:
FWIW - Grafana 12 is going to be GA soon. 11.6 or later is probably where you want to target for “stable” releases for the moment.
-- Dan Brown
On Apr 18, 2025, at 10:47, Sake Ceph <ceph@paulusma.eu> wrote:
Really great you moved to the default grafana image and moving the customization to cephadm. This makes upgrading indeed easier.
The problem is that Grafana has a stranger release cycle and therefor not really compatible with release cycle of Ceph. This would mean one or two update(s) of the supported Grafana minor version. And maybe with every Ceph release, update to the latest bugfix release? I guess a bugfix version should always work with the same minor version of Grafana. The user can configure this via the provided tools for setting custom images.
We're only talking about Grafana, but this applicable for all the extra container images.
Thank you for all the hard work you're doing for the project! These are just my 2 cents :D
Kind regards, Sake
Op 17-04-2025 19:43 CEST schreef Nizamudeen A <nia@redhat.com>:
We deprecated [1] the use of quay.io/ceph/ (http://quay.io/ceph/)ceph-grafana images from squid onwards due to the pain of building and maintaining those images by ourselves. The main reason we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image.
But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/ (http://quay.io/ceph/)grafanaimage and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah here: https://github.com/ceph/ceph/pull/62827.
We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config.
Will check both of them next week after the holidays.
[1] https://docs.ceph.com/en/latest/releases/squid/#monitoring
Regards, Nizam
On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote: Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old software with know issues :)
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Good point, we are still on v2 of prometheus <https://github.com/ceph/ceph/blob/main/src/python-common/ceph/cephadm/images.py#L27> (2.51.0) and I see v3 <https://github.com/prometheus/prometheus/releases> is there so a bump is needed there. We will take care of it along with the other dependent ones. As for loki and promtail, we are moving away from promtail and going to use alloy <https://github.com/ceph/ceph/pull/62960>, since promtail is officially deprecated by grafana. Once the migration is done we will bump loki and test them together. I am not sure if we will backport the alloy thing to squid (could break stuffs), but we can say more when we test it. And as for grafana it'll be 11.6.0 which is what we will use: https://github.com/ceph/ceph/pull/62827 Regards, Nizam On Fri, Apr 25, 2025 at 8:37 PM Sake Ceph <ceph@paulusma.eu> wrote:
This thread is only about Grafana, but what about the other services like Prometheus, Loki, Promtail, alertmanager or node-exporter?
Op 19-04-2025 15:18 CEST schreef Daniel Brown <daniel.h.brown@thermify.cloud>:
FWIW - Grafana 12 is going to be GA soon. 11.6 or later is probably where you want to target for “stable” releases for the moment.
-- Dan Brown
On Apr 18, 2025, at 10:47, Sake Ceph <ceph@paulusma.eu> wrote:
Really great you moved to the default grafana image and moving the customization to cephadm. This makes upgrading indeed easier.
The problem is that Grafana has a stranger release cycle and therefor not really compatible with release cycle of Ceph. This would mean one or two update(s) of the supported Grafana minor version. And maybe with every Ceph release, update to the latest bugfix release? I guess a bugfix version should always work with the same minor version of Grafana. The user can configure this via the provided tools for setting custom images.
We're only talking about Grafana, but this applicable for all the extra container images.
Thank you for all the hard work you're doing for the project! These are just my 2 cents :D
Op 17-04-2025 19:43 CEST schreef Nizamudeen A <nia@redhat.com>:
We deprecated [1] the use of quay.io/ceph/ ( http://quay.io/ceph/)ceph-grafana images from squid onwards due to the
Kind regards, Sake pain of building and maintaining those images by ourselves. The main reason
we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image.
But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/ (http://quay.io/ceph/)grafanaimage and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah here: https://github.com/ceph/ceph/pull/62827.
We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config.
Will check both of them next week after the holidays.
[1] https://docs.ceph.com/en/latest/releases/squid/#monitoring
Regards, Nizam
On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote: Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?).
https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old
software with know issues :)
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Nizamudeen A Sr. Software Engineer - IBM Partner Engineer IBM and Red Hat Ceph Storage Red Hat <https://www.redhat.com/> <https://www.redhat.com/>
A little late response, I had some time off. Thank you for updating the monitoring stack. I understand you can't always backport everything, but good to hear you're considering it!
Op 29-04-2025 06:57 CEST schreef Nizamudeen A <nia@redhat.com>:
Good point, we are still on v2 of prometheus (https://github.com/ceph/ceph/blob/main/src/python-common/ceph/cephadm/images...) (2.51.0) and I see v3 (https://github.com/prometheus/prometheus/releases) is there so a bump is needed there. We will take care of it along with the other dependent ones.
As for loki and promtail, we are moving away from promtail and going to use alloy (https://github.com/ceph/ceph/pull/62960), since promtail is officially deprecated by grafana. Once the migration is done we will bump loki and test them together.
I am not sure if we will backport the alloy thing to squid (could break stuffs), but we can say more when we test it.
And as for grafana it'll be 11.6.0 which is what we will use: https://github.com/ceph/ceph/pull/62827
Regards, Nizam
On Fri, Apr 25, 2025 at 8:37 PM Sake Ceph <ceph@paulusma.eu> wrote:
This thread is only about Grafana, but what about the other services like Prometheus, Loki, Promtail, alertmanager or node-exporter?
Op 19-04-2025 15:18 CEST schreef Daniel Brown <daniel.h.brown@thermify.cloud>:
FWIW - Grafana 12 is going to be GA soon. 11.6 or later is probably where you want to target for “stable” releases for the moment.
-- Dan Brown
On Apr 18, 2025, at 10:47, Sake Ceph <ceph@paulusma.eu> wrote:
Really great you moved to the default grafana image and moving the customization to cephadm. This makes upgrading indeed easier.
The problem is that Grafana has a stranger release cycle and therefor not really compatible with release cycle of Ceph. This would mean one or two update(s) of the supported Grafana minor version. And maybe with every Ceph release, update to the latest bugfix release? I guess a bugfix version should always work with the same minor version of Grafana. The user can configure this via the provided tools for setting custom images.
We're only talking about Grafana, but this applicable for all the extra container images.
Thank you for all the hard work you're doing for the project! These are just my 2 cents :D
Kind regards, Sake
Op 17-04-2025 19:43 CEST schreef Nizamudeen A <nia@redhat.com>:
We deprecated [1] the use of quay.io/ceph/ (http://quay.io/ceph/) (http://quay.io/ceph/)ceph-grafana images from squid onwards due to the pain of building and maintaining those images by ourselves. The main reason we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image.
But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/ (http://quay.io/ceph/) (http://quay.io/ceph/)grafanaimage and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah here: https://github.com/ceph/ceph/pull/62827.
We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config.
Will check both of them next week after the holidays.
[1] https://docs.ceph.com/en/latest/releases/squid/#monitoring
Regards, Nizam
On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote: Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old software with know issues :)
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Nizamudeen A Sr. Software Engineer - IBM Partner Engineer
IBM and Red Hat Ceph Storage Red Hat (https://www.redhat.com/)
Our CTO wants us to move to the latest version of Ceph (aka squid), but with the number of bugs around encryption and stability, it pretty much prevents us from moving forward as we rely at least 95% of it for our government business needs. Get Outlook for Android<https://aka.ms/AAb9ysg> ________________________________ From: Sake Ceph <ceph@paulusma.eu> Sent: Tuesday, May 6, 2025 5:10:24 AM To: Nizamudeen A <nia@redhat.com> Cc: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Grafana vulnerability - cephadm deployment This is an external email. Please take care when clicking links or opening attachments. When in doubt, check with the Help Desk or Security. A little late response, I had some time off. Thank you for updating the monitoring stack. I understand you can't always backport everything, but good to hear you're considering it!
Op 29-04-2025 06:57 CEST schreef Nizamudeen A <nia@redhat.com>:
Good point, we are still on v2 of prometheus (https://github.com/ceph/ceph/blob/main/src/python-common/ceph/cephadm/images...) (2.51.0) and I see v3 (https://github.com/prometheus/prometheus/releases) is there so a bump is needed there. We will take care of it along with the other dependent ones.
As for loki and promtail, we are moving away from promtail and going to use alloy (https://github.com/ceph/ceph/pull/62960), since promtail is officially deprecated by grafana. Once the migration is done we will bump loki and test them together.
I am not sure if we will backport the alloy thing to squid (could break stuffs), but we can say more when we test it.
And as for grafana it'll be 11.6.0 which is what we will use: https://github.com/ceph/ceph/pull/62827
Regards, Nizam
On Fri, Apr 25, 2025 at 8:37 PM Sake Ceph <ceph@paulusma.eu> wrote:
This thread is only about Grafana, but what about the other services like Prometheus, Loki, Promtail, alertmanager or node-exporter?
Op 19-04-2025 15:18 CEST schreef Daniel Brown <daniel.h.brown@thermify.cloud>:
FWIW - Grafana 12 is going to be GA soon. 11.6 or later is probably where you want to target for “stable” releases for the moment.
-- Dan Brown
On Apr 18, 2025, at 10:47, Sake Ceph <ceph@paulusma.eu> wrote:
Really great you moved to the default grafana image and moving the customization to cephadm. This makes upgrading indeed easier.
The problem is that Grafana has a stranger release cycle and therefor not really compatible with release cycle of Ceph. This would mean one or two update(s) of the supported Grafana minor version. And maybe with every Ceph release, update to the latest bugfix release? I guess a bugfix version should always work with the same minor version of Grafana. The user can configure this via the provided tools for setting custom images.
We're only talking about Grafana, but this applicable for all the extra container images.
Thank you for all the hard work you're doing for the project! These are just my 2 cents :D
Kind regards, Sake
Op 17-04-2025 19:43 CEST schreef Nizamudeen A <nia@redhat.com>:
We deprecated [1] the use of quay.io/ceph/ (http://quay.io/ceph/) (http://quay.io/ceph/)ceph-grafana images from squid onwards due to the pain of building and maintaining those images by ourselves. The main reason we were building a custom grafana image was to bundle the grafana dashboards and plugins inside the official grafana image.
But with https://github.com/ceph/ceph/pull/55615, we offloaded that to cephadm at runtime which removed the burden of building and maintaining it which is why we are using the quay.io/ceph/ (http://quay.io/ceph/) (http://quay.io/ceph/)grafanaimage and its simply a mirror from the docker registry. And we are upgrading the grafana to 11.x in tentacle which is in progress by @Afreen Misbah here: https://github.com/ceph/ceph/pull/62827.
We have two options for reef since we are still supporting reef actively. 1. we can try and see if backporting that PR is possible, last time I couldn't because of cephadm binary differences but I could look again 2. Just tag and continue to build a newer ceph-grafana:10.4.x image in quay. Once the image is there anyone can just go ahead and use it by updating the container image config.
Will check both of them next week after the holidays.
[1] https://docs.ceph.com/en/latest/releases/squid/#monitoring
Regards, Nizam
On Thu, Apr 17, 2025 at 7:43 PM Sake Ceph <ceph@paulusma.eu> wrote: Squid is already on 10.4.0 and it looks like it using the default grafana container images (correct?). https://github.com/ceph/ceph/blob/squid/src/cephadm/cephadmlib/constants.py
I couldn't find any issues, but the company really doesn't like old software with know issues :)
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Nizamudeen A Sr. Software Engineer - IBM Partner Engineer
IBM and Red Hat Ceph Storage Red Hat (https://www.redhat.com/)
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io This message and its attachments are from Data Dimensions and are intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately and permanently delete the original email and destroy any copies or printouts of this email as well as any attachments.
participants (7)
-
Anthony D'Atri
-
Daniel Brown
-
Michael Worsham
-
Nizamudeen A
-
Robert Sander
-
Sake Ceph
-
Wyll Ingersoll