- ceph-users - lists.ceph.io

v20.2.3 tentacle QE validation status
by Yuri Weinstein 27 Jul '26

27 Jul '26

Details of this release are summarized here: https://tracker.ceph.com/issues/77256#note-5 Release Notes - TBD LRC upgrade - TBD *PLEASE ADD YOUR COMMENTS/APPROVALS TO THE TRACKER!* Seeking approvals/reviews for: smoke: PASSED rados: Kamoltat, Afreen, Travis rgw: Adam Emerson fs: Venky orch/ceph-volume: Adam King & Guillaume rbd: Ilya krbd: Ilya nvmeof: Aviv, Vallari upgrade: Kamoltat, Patrick upgrade-clients:client-upgrade-reef-tentacle-tentacle: PASSED upgrade-clients:client-upgrade-squid-tentacle-tentacle: PASSED perf-basic: PASSED crimson-rados: N/A powercycle: PASSED Please reply with your approval or flag any concerns.

1 0

ingress service vrrp_interface_network bad effect
by Robert Sander 27 Jul '26

27 Jul '26

Hi, I am trying to deploy an ingress service with this spec: service_type: ingress service_id: rgw.default.default placement: label: ingress spec: backend_service: rgw.default.default virtual_ip: 10.166.166.205/24 frontend_port: 80 monitor_port: 8081 virtual_interface_networks: - 10.166.166.0/24 vrrp_interface_network: 10.104.130.0/24 The orchestrator tells me this: 2026-07-27T12:16:36.145458+0000 mgr.ceph103.xxazgi [INF] 10.166.166.205 is in 10.166.166.0/24 on ceph203 interface bondaccess.166 2026-07-27T12:16:36.145729+0000 mgr.ceph103.xxazgi [INF] 10.166.166.205 is in 10.166.166.0/24 on ceph204 interface bondaccess.166 2026-07-27T12:16:36.145809+0000 mgr.ceph103.xxazgi [INF] vrrp will be configured on ceph203 interface ['bondceph.1530'] (which is in subnet 10.104.130.0/24) 2026-07-27T12:16:36.147088+0000 mgr.ceph103.xxazgi [INF] Deploying daemon keepalived.rgw.default.default.ceph203.ztemnc on ceph203 2026-07-27T12:16:39.683591+0000 mgr.ceph103.xxazgi [INF] 10.166.166.205 is in 10.166.166.0/24 on ceph204 interface bondaccess.166 2026-07-27T12:16:39.683858+0000 mgr.ceph103.xxazgi [INF] 10.166.166.205 is in 10.166.166.0/24 on ceph203 interface bondaccess.166 2026-07-27T12:16:39.683958+0000 mgr.ceph103.xxazgi [INF] vrrp will be configured on ceph204 interface ['bondceph.1530'] (which is in subnet 10.104.130.0/24) 2026-07-27T12:16:39.684823+0000 mgr.ceph103.xxazgi [INF] Deploying daemon keepalived.rgw.default.default.ceph204.cpzqgk on ceph204 But the keepalived.conf contains this: interface bondceph.1530 unicast_src_ip 10.166.166.103 unicast_peer { 10.166.166.104 } which are the hosts' IPs in the network of the virtual IP. Why does the orchestrator not configure the IPs of the bondceph.1530 interface for VRRP? This results in both nodes having the virtual IP online as the backup keepalived does not get any VRRP packets from the first node. Regards -- Robert Sander Linux Consultant Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: +49 30 405051 - 0 Fax: +49 30 405051 - 19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin

1 0

Re: problem uploading dashboard certificates
by Eugen Block 27 Jul '26

27 Jul '26

Hi, the dashboard cert and key files are stored in: ceph config-key get mgr/dashboard/crt ceph config-key get mgr/dashboard/key Or something similar, maybe per host entries like (the "random" string is cephadm specific and won't apply in your case, of course): ceph config-key get mgr/dashboard/host1.random/crt ceph config-key get mgr/dashboard/host1.random/key You should be able to remove those faulty entries with ceph config-key rm mgr/dashboard/host1.random/crt ceph config-key rm mgr/dashboard/host1.random/key Then fail the mgr and inject your correct cert/key files again. I think that should work, if not, let us know. Regards, Eugen Zitat von Simon Oosthoek via ceph-users <ceph-users(a)ceph.io>: > Sorry, forgot to add info about the installation; this is ceph 17.2.x on > ubuntu 22.04.x, no cephadm and the cluster has about 360 OSDs of 12TB HDD. > > On Mon, 27 Jul 2026 at 11:50, Simon Oosthoek via ceph-users < > ceph-users(a)ceph.io> wrote: > >> Hi all, >> we've had some issues when Harica revoked a bunch of our certificates, I >> then accidentally uploaded mismatching key/pem files. When I wanted to fix >> that with the correct ones I got >> >> # ceph dashboard set-ssl-certificate -i cephmon.science.ru.nl.pem >> Error EIO: Module 'dashboard' has experienced an error and cannot handle >> commands: [('x509 certificate routines', '', 'key values mismatch')] >> >> Is there another way to fix this? >> >> Cheers, >> >> /Simon >> -- >> I'm using my gmail.com address, because the gmail.com dmarc policy is >> "none", some mail servers will reject this (microsoft?) others will instead >> allow this when I send mail to a mailling list which has not yet been >> configured to send mail "on behalf of" the sender, but rather do a kind of >> "forward". The latter situation causes dkim/dmarc failures and the dmarc >> policy will be applied. see https://wiki.list.org/DEV/DMARC for more >> details >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> > > > -- > I'm using my gmail.com address, because the gmail.com dmarc policy is > "none", some mail servers will reject this (microsoft?) others will instead > allow this when I send mail to a mailling list which has not yet been > configured to send mail "on behalf of" the sender, but rather do a kind of > "forward". The latter situation causes dkim/dmarc failures and the dmarc > policy will be applied. see https://wiki.list.org/DEV/DMARC for more details > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Re: Ec pool min size
by pg＠ceph.list.sabi.co.UK 27 Jul '26

27 Jul '26

> We have a 9 hosts (3 per pod) cluster with 1 pool in EC 5+3 > (host domain failure), That configuration is optimal for maximizing recovery times and write latency: since each EC set will span 8 hosts of 9, every time a host becames unavailable virtually all PGs will become degraded, and even with all host available any write will involve nearly all hosts. This kind of optimal configuration is popular with those who think that the original design assumption of Ceph of having many, many more hosts and OSDs than the replication factor or stripe width was wasteful and cowardly. :-) > I expected that shutting down 1 pod Is there any particular reason why the failure domain is host but pods with 3 hosts get shut down? > wouldn't affect our pg but I just discovered that the EC pool > is in min_size 6 while trying it. Is min_size 6 important in > this case ? Depends for what. But it is difficult to say without knowing the expected data and workload profiles. I might opine that in most cases it is as unimportant as having 'min_size=2' with 'size=3'. :-) > Could I safely lower it to 5 "Safely" and 'k=5' with 'min_size=5' is an interesting association :-). > or using another EC k+m would suit better? Perhaps 'k=2,m=2' or for the very brave 'k=4,m=2'. But again it all depends on expected data and workload profiles.

1 0

Re: problem uploading dashboard certificates
by Simon Oosthoek 27 Jul '26

27 Jul '26

Sorry, forgot to add info about the installation; this is ceph 17.2.x on ubuntu 22.04.x, no cephadm and the cluster has about 360 OSDs of 12TB HDD. On Mon, 27 Jul 2026 at 11:50, Simon Oosthoek via ceph-users < ceph-users(a)ceph.io> wrote: > Hi all, > we've had some issues when Harica revoked a bunch of our certificates, I > then accidentally uploaded mismatching key/pem files. When I wanted to fix > that with the correct ones I got > > # ceph dashboard set-ssl-certificate -i cephmon.science.ru.nl.pem > Error EIO: Module 'dashboard' has experienced an error and cannot handle > commands: [('x509 certificate routines', '', 'key values mismatch')] > > Is there another way to fix this? > > Cheers, > > /Simon > -- > I'm using my gmail.com address, because the gmail.com dmarc policy is > "none", some mail servers will reject this (microsoft?) others will instead > allow this when I send mail to a mailling list which has not yet been > configured to send mail "on behalf of" the sender, but rather do a kind of > "forward". The latter situation causes dkim/dmarc failures and the dmarc > policy will be applied. see https://wiki.list.org/DEV/DMARC for more > details > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > -- I'm using my gmail.com address, because the gmail.com dmarc policy is "none", some mail servers will reject this (microsoft?) others will instead allow this when I send mail to a mailling list which has not yet been configured to send mail "on behalf of" the sender, but rather do a kind of "forward". The latter situation causes dkim/dmarc failures and the dmarc policy will be applied. see https://wiki.list.org/DEV/DMARC for more details

1 0

problem uploading dashboard certificates
by Simon Oosthoek 27 Jul '26

27 Jul '26

Hi all, we've had some issues when Harica revoked a bunch of our certificates, I then accidentally uploaded mismatching key/pem files. When I wanted to fix that with the correct ones I got # ceph dashboard set-ssl-certificate -i cephmon.science.ru.nl.pem Error EIO: Module 'dashboard' has experienced an error and cannot handle commands: [('x509 certificate routines', '', 'key values mismatch')] Is there another way to fix this? Cheers, /Simon -- I'm using my gmail.com address, because the gmail.com dmarc policy is "none", some mail servers will reject this (microsoft?) others will instead allow this when I send mail to a mailling list which has not yet been configured to send mail "on behalf of" the sender, but rather do a kind of "forward". The latter situation causes dkim/dmarc failures and the dmarc policy will be applied. see https://wiki.list.org/DEV/DMARC for more details

1 0

Re: subvolume in _nogroup
by Neeraj Pratap Singh 27 Jul '26

27 Jul '26

Hey Stephane, ---------- Forwarded message --------- From: Venky Shankar <vshankar(a)redhat.com> Date: Fri, Jul 24, 2026 at 11:19 PM Subject: Re: [ceph-users] subvolume in _nogroup To: Patrick Donnelly <pdonnell(a)redhat.com> Cc: Kotresh Hiremath Ravishankar <khiremat(a)redhat.com>, ceph-fs-team < ceph-fs-team(a)redhat.com>, Jos Collin <jcollin(a)redhat.com> It's possible the subvolume create command was invoked with an empty subvolume name. The subvolume group gets marked with ceph.dir.subvolume xattr. See: https://tracker.ceph.com/issues/68687 Workaround is to remove the xattr from the group directory. To check dump the group directory inode and check `is_subvolume`. Feel free to pass this along to the user, otherwise I will respond tomorrow. On Fri, Jul 24, 2026 at 10:56 PM Patrick Donnelly <pdonnell(a)redhat.com> wrote: > > Anyone have an idea on this issue and can help this poster? > > ---------- Forwarded message --------- > From: Stéphane Delmotte via ceph-users <ceph-users(a)ceph.io> > Date: Fri, Jul 24, 2026 at 9:35 AM > Subject: [ceph-users] subvolume in _nogroup > To: <ceph-users(a)ceph.io> > > > Hello, > > I find myself with a list of subvolumes in the|no_group|subvolume group, > which, if I understand correctly, is a special subvolume group. > > Actually, Manila created all these subvolumes without specifying a > subvolume group. > > Following an unexpected power outage and a restart, it no longer works. > > Manila can no longer create subvolumes, and I can no longer manipulate > my subvolumes. > > For example: > > content_copyCopy code > > |cephfssubvolumermcephfsbidulestef > ErrorEINVAL:invalidvaluespecifiedforceph.dir.subvolume| > > However, the state of my cluster is not critical. I only have: > > * 248 PGs not deep-scrubbed in time > * 517 PGs not scrubbed in time > > My Ceph version is: > > content_copyCopy code > > |cephversion19.3.0-4637-g42c1c3d4(42c1c3d42aaa0b6c9bcbccb51c8b397fb980aa9f) > squid (dev)| > > My CephFS mounts remain functional. > > If anyone has information about this, I’d appreciate it. Alternatively, > how can I continue creating subvolumes without specifying a subvolume group? > > Thanks, > > Stéphane > > > -- > Delmotte Stéphane > Laboratoire de Biométrie et Biologie Évolutive. UMR CNRS 5558 > Campus de La Doua - Université Claude Bernard - Lyon 1 > Bâtiment Grégoire Mendel - 1er étage > 16 rue Raphael Dubois > Villeurbanne 69100 > Tel : +33 06 82 87 93 59 > > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > > > > -- > Patrick Donnelly, Ph.D. > He / Him / His > Red Hat Partner Engineer > IBM, Inc. > GPG: 19F28A586F808C2402351B93C3301A3E258DD79D > -- Cheers, Venky On Fri, Jul 24, 2026 at 7:05 PM Stéphane Delmotte via ceph-users < ceph-users(a)ceph.io> wrote: > Hello, > > I find myself with a list of subvolumes in the|no_group|subvolume group, > which, if I understand correctly, is a special subvolume group. > > Actually, Manila created all these subvolumes without specifying a > subvolume group. > > Following an unexpected power outage and a restart, it no longer works. > > Manila can no longer create subvolumes, and I can no longer manipulate > my subvolumes. > > For example: > > content_copyCopy code > > |cephfssubvolumermcephfsbidulestef > ErrorEINVAL:invalidvaluespecifiedforceph.dir.subvolume| > > However, the state of my cluster is not critical. I only have: > > * 248 PGs not deep-scrubbed in time > * 517 PGs not scrubbed in time > > My Ceph version is: > > content_copyCopy code > > |cephversion19.3.0-4637-g42c1c3d4(42c1c3d42aaa0b6c9bcbccb51c8b397fb980aa9f) > > squid (dev)| > > My CephFS mounts remain functional. > > If anyone has information about this, I’d appreciate it. Alternatively, > how can I continue creating subvolumes without specifying a subvolume > group? > > Thanks, > > Stéphane > > > -- > Delmotte Stéphane > Laboratoire de Biométrie et Biologie Évolutive. UMR CNRS 5558 > Campus de La Doua - Université Claude Bernard - Lyon 1 > Bâtiment Grégoire Mendel - 1er étage > 16 rue Raphael Dubois > Villeurbanne 69100 > Tel : +33 06 82 87 93 59 > > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > >

1 0

Unable to setup cloud sync to AWS eu-north-1 region
by Ieva Astašauskė 27 Jul '26

27 Jul '26

Hi, Ceph cluster: Multisite over 2 regions Per region: 6 OSD nodes, same nodes run RGW with rgw_run_sync_thread=true 1 dedicated RGW node serving client traffic (rgw_run_sync_thread=false) Trying to add a tier-type=cloud zone that would sync to AWS (based on https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/ and https://oneuptime.com/blog/post/2026-03-31-rook-cloud-sync-rgw-aws-s3/view ) Switched RGW on one OSD to sync that cloud zone, added IP address of that RGW node as zone endpoint sudo radosgw-admin zonegroup get { "id": "c1565bb4-acf9-4def-bfe1-7f189136c07b", "name": "poc-ms", "api_name": "poc-ms", "is_master": true, "endpoints": [ "http://ceph-cluster-poc1.ceph-radosgw.service.region1.consul:7480" ], "hostnames": [], "hostnames_s3website": [], "master_zone": "6e1e00de-a43d-4da9-994a-081041f6afb8", "zones": [ { "id": "3838f112-6804-4712-8af5-ab57a886e64c", "name": "poc2-region2", "endpoints": [ " http://ceph-cluster-poc2.ceph-radosgw.service.region2.consul:7480" ], "log_meta": false, "log_data": true, "bucket_index_max_shards": 11, "read_only": false, "tier_type": "", "sync_from_all": true, "sync_from": [], "redirect_zone": "", "supported_features": [ "compress-encrypted", "notification_v2", "resharding" ] }, { "id": "6e1e00de-a43d-4da9-994a-081041f6afb8", "name": "poc1-region1", "endpoints": [ " http://ceph-cluster-poc1.ceph-radosgw.service.region1.consul:7480" ], "log_meta": false, "log_data": true, "bucket_index_max_shards": 11, "read_only": false, "tier_type": "", "sync_from_all": true, "sync_from": [], "redirect_zone": "", "supported_features": [ "compress-encrypted", "notification_v2", "resharding" ] }, { "id": "8c988f68-975f-4c39-ab53-520f9fc6ee9e", "name": "poc-cloud-backup", "endpoints": [ "http://rgw_node_ip:7480" ], "log_meta": false, "log_data": true, "bucket_index_max_shards": 11, "read_only": false, "tier_type": "cloud", "sync_from_all": true, "sync_from": [], "redirect_zone": "", "supported_features": [ "compress-encrypted", "notification_v2", "resharding" ] } ], "placement_targets": [ { "name": "default-placement", "tags": [], "storage_classes": [ "STANDARD", "STANDARD_IA" ] }, { "name": "ec-only", "tags": [], "storage_classes": [ "STANDARD", "STANDARD_IA" ] } ], "default_placement": "default-placement", "realm_id": "939ac8b7-8237-4a50-bea5-fed4b1dfd74d", "sync_policy": { "groups": [ { "id": "bucket-sync", "data_flow": { "symmetrical": [ { "id": "sym-region1-region2", "zones": [ "3838f112-6804-4712-8af5-ab57a886e64c", "6e1e00de-a43d-4da9-994a-081041f6afb8" ] } ], "directional": [ { "source_zone": "6e1e00de-a43d-4da9-994a-081041f6afb8", "dest_zone": "8c988f68-975f-4c39-ab53-520f9fc6ee9e" } ] }, "pipes": [ { "id": "pipe-all", "source": { "bucket": "*", "zones": [ "*" ] }, "dest": { "bucket": "*", "zones": [ "*" ] }, "params": { "source": { "filter": { "tags": [] } }, "dest": {}, "priority": 0, "mode": "system" } } ], "status": "enabled" } ] }, "enabled_features": [ "notification_v2", "resharding" ] } Verified that: - Cloud zone has correct AWS credentials (can PUT object directly to the bucket with same credentials) sudo radosgw-admin zone get --rgw-zone=poc-cloud-backup { "id": "8c988f68-975f-4c39-ab53-520f9fc6ee9e", "name": "poc-cloud-backup", "domain_root": "poc-cloud-backup.rgw.meta:root", "control_pool": "poc-cloud-backup.rgw.control", "dedup_pool": "poc-cloud-backup.rgw.dedup", "gc_pool": "poc-cloud-backup.rgw.log:gc", "lc_pool": "poc-cloud-backup.rgw.log:lc", "log_pool": "poc-cloud-backup.rgw.log", "intent_log_pool": "poc-cloud-backup.rgw.log:intent", "usage_log_pool": "poc-cloud-backup.rgw.log:usage", "roles_pool": "poc-cloud-backup.rgw.meta:roles", "reshard_pool": "poc-cloud-backup.rgw.log:reshard", "user_keys_pool": "poc-cloud-backup.rgw.meta:users.keys", "user_email_pool": "poc-cloud-backup.rgw.meta:users.email", "user_swift_pool": "poc-cloud-backup.rgw.meta:users.swift", "user_uid_pool": "poc-cloud-backup.rgw.meta:users.uid", "otp_pool": "poc-cloud-backup.rgw.otp", "notif_pool": "poc-cloud-backup.rgw.log:notif", "topics_pool": "poc-cloud-backup.rgw.meta:topics", "account_pool": "poc-cloud-backup.rgw.meta:accounts", "group_pool": "poc-cloud-backup.rgw.meta:groups", "bucket_logging_pool": "poc-cloud-backup.rgw.log:logging", "system_key": { "access_key": "XXX", "secret_key": "XXXXX" }, "placement_pools": [ { "key": "default-placement", "val": { "index_pool": "poc-cloud-backup.rgw.buckets.index", "storage_classes": { "STANDARD": { "data_pool": "poc-cloud-backup.rgw.buckets.data-r3" }, "STANDARD_IA": { "data_pool": "poc-cloud-backup.rgw.buckets.data-ec42" } }, "data_extra_pool": "poc-cloud-backup.rgw.buckets.non-ec", "index_type": 0, "inline_data": true } }, { "key": "ec-only", "val": { "index_pool": "poc-cloud-backup.rgw.buckets.index", "storage_classes": { "STANDARD": {}, "STANDARD_IA": { "data_pool": "poc-cloud-backup.rgw.buckets.data-ec42" } }, "data_extra_pool": "poc-cloud-backup.rgw.buckets.non-ec", "index_type": 0, "inline_data": true } } ], "tier_config": { "connection": { "access_key": "XXX", "endpoint": "https://s3.eu-north-1.amazonaws.com", "host_style": "virtual", "region": "eu-north-1", "secret": "XXXXX" }, "target_path": "cloud-sync-test" }, "realm_id": "939ac8b7-8237-4a50-bea5-fed4b1dfd74d", "restore_pool": "poc-cloud-backup.rgw.log:restore" } Sync status seems OK sudo radosgw-admin bucket sync status --bucket=sync-test01 --rgw-zone=poc-cloud-backup realm 939ac8b7-8237-4a50-bea5-fed4b1dfd74d (default) zonegroup c1565bb4-acf9-4def-bfe1-7f189136c07b (poc-ms) zone 8c988f68-975f-4c39-ab53-520f9fc6ee9e (poc-cloud-backup) bucket :sync-test01[6e1e00de-a43d-4da9-994a-081041f6afb8.16963179.1 ]) current time 2026-07-24T13:48:45Z source zone 6e1e00de-a43d-4da9-994a-081041f6afb8 (poc1-region1) source bucket :sync-test01[6e1e00de-a43d-4da9-994a-081041f6afb8.16963179.1 ]) incremental sync on 11 shards bucket is caught up with source source zone 6e1e00de-a43d-4da9-994a-081041f6afb8 (poc1-region1) source bucket :sync-test01[6e1e00de-a43d-4da9-994a-081041f6afb8.16963179.1 ]) incremental sync on 11 shards bucket is caught up with source source zone 6e1e00de-a43d-4da9-994a-081041f6afb8 (poc1-region1) source bucket :sync-test01[6e1e00de-a43d-4da9-994a-081041f6afb8.16963179.1 ]) incremental sync on 11 shards bucket is caught up with source However, nothing appears on AWS And can’t find anything meaningful in RGW logs I keep adding new objects to the master zone every time I change some configs

1 0

Re: v21.1.0 Umbrella RC0 released
by Stefan Möding 27 Jul '26

27 Jul '26

Patrick Donnelly via ceph-users <ceph-users(a)ceph.io> writes: > Well that's unfortunate. I'm not going to make this become Ceph's > problem by retroactively supporting new Ubuntu releases on release > branches. It means multiplying our testing and dev workload when > maintaining a predictable cadence is already difficult. I'd rather > that we work on getting the new releases out on schedule which would > support the new Ubuntu release. I can understand that. I’m thinking about either migrating the cluster to Debian or using Podman. Thanks! -- Stefan

1 0

Re: v21.1.0 Umbrella RC0 released
by Patrick Donnelly 26 Jul '26

26 Jul '26

On Wed, Jul 22, 2026 at 10:33 AM Stefan Möding via ceph-users <ceph-users(a)ceph.io> wrote: > > Patrick Donnelly via ceph-users <ceph-users(a)ceph.io> writes: > > > Why is Canonical not updating/fixing these packages when the standard > > EOL for 24.04 is May 2029? > > The Qt5 packages are part of the Universe repo. This is not covered by > the standard (viz. free) 5 year. LTS covers only the Main repo. > > It requires an ESM contract to get support for the Universe repo. So > with the release of Ubuntu 26.04 the ongoing effort to keep the Universe > repo up to date has effectively been dropped. Well that's unfortunate. I'm not going to make this become Ceph's problem by retroactively supporting new Ubuntu releases on release branches. It means multiplying our testing and dev workload when maintaining a predictable cadence is already difficult. I'd rather that we work on getting the new releases out on schedule which would support the new Ubuntu release. -- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D

1 0