Where are you running Ceph?
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on? We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions. If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you. Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 [image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/>
Lately I don't have a production cluster anymore. But my homelab is ceph with Rook. I'm running it since the alpha versions. Awesome piece of software. You'll need Kubernetes familiarity though. H On Wed, Jul 2, 2025, 18:17 Anthony Fecarotta <anthony@linehaul.ai> wrote:
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on?
We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions.
If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you.
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 <https://www.google.com/maps/search/1+Mid+America+Plz+Flr+3+Oakbrook+Terrace,+IL+60181?entry=gmail&source=g>
[image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
hello, many k8s cluster in production with 20/30 nodes and around 200to each with rook as storage (block, fs, s3) backend for web app and postgres Incredible stack :) oau Le mercredi 02 juillet 2025 à 20:36 +0200, Hans van den Bogert a écrit :
Lately I don't have a production cluster anymore. But my homelab is ceph with Rook. I'm running it since the alpha versions. Awesome piece of software. You'll need Kubernetes familiarity though.
H
On Wed, Jul 2, 2025, 18:17 Anthony Fecarotta <anthony@linehaul.ai> wrote:
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on?
We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions.
If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you.
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 < https://www.google.com/maps/search/1+Mid+America+Plz+Flr+3+Oakbrook+ Terrace,+IL+60181?entry=gmail&source=g>
[image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thank you. From what I can tell, after being on the mailing list for six months, it seems most users are running Kubernetes. I have been able to get by without implementing Kubernetes. I was looking into OpenStack or OpenNebula, although many have deterred me from migrating to OpenStack, citing unneeded complexity. With all that said, GPU time-slicing (Nvidia MIG) is on our roadmap, and I have read the easiest way to achieve that is through the Kubernetes GPU Operator. It just seems unnecessary to implement currently. Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 [image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> On Wed Jul 2, 2025, 09:01 PM GMT, Olivier AUDRY <mailto:olivier@nmlq.fr> wrote:
hello,
many k8s cluster in production with 20/30 nodes and around 200to each with rook as storage (block, fs, s3) backend for web app and postgres
Incredible stack :)
oau
Le mercredi 02 juillet 2025 à 20:36 +0200, Hans van den Bogert a écrit :
Lately I don't have a production cluster anymore. But my homelab is ceph with Rook. I'm running it since the alpha versions. Awesome piece of software. You'll need Kubernetes familiarity though.
H
On Wed, Jul 2, 2025, 18:17 Anthony Fecarotta <anthony@linehaul.ai> wrote:
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on?
We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions.
If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you.
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 < https://www.google.com/maps/search/1+Mid+America+Plz+Flr+3+Oakbrook+ Terrace,+IL+60181?entry=gmail&source=g>
[image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
We deploy ceph in smaller clusters (200TB at a time, customer driven). Though we have a much larger cluster growing. We deploy with cephadm, on bare metal. We use a lot of Openstack with these clusters. We've looked at rook for a very specific "edge" styled deployment, but haven't made any decisions as to whether to move to it or not. Cheers Michael On Wed, Jul 2, 2025 at 6:05 PM Anthony Fecarotta <anthony@linehaul.ai> wrote:
Thank you. From what I can tell, after being on the mailing list for six months, it seems most users are running Kubernetes.
I have been able to get by without implementing Kubernetes. I was looking into OpenStack or OpenNebula, although many have deterred me from migrating to OpenStack, citing unneeded complexity. With all that said, GPU time-slicing (Nvidia MIG) is on our roadmap, and I have read the easiest way to achieve that is through the Kubernetes GPU Operator. It just seems unnecessary to implement currently.
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
[image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/>
On Wed Jul 2, 2025, 09:01 PM GMT, Olivier AUDRY <mailto:olivier@nmlq.fr> wrote:
hello,
many k8s cluster in production with 20/30 nodes and around 200to each with rook as storage (block, fs, s3) backend for web app and postgres
Incredible stack :)
oau
Le mercredi 02 juillet 2025 à 20:36 +0200, Hans van den Bogert a écrit :
Lately I don't have a production cluster anymore. But my homelab is ceph with Rook. I'm running it since the alpha versions. Awesome piece of software. You'll need Kubernetes familiarity though.
H
On Wed, Jul 2, 2025, 18:17 Anthony Fecarotta <anthony@linehaul.ai> wrote:
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on?
We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions.
If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you.
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 < https://www.google.com/maps/search/1+Mid+America+Plz+Flr+3+Oakbrook+ Terrace,+IL+60181?entry=gmail&source=g>
[image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On bare “metal.” cephadm from admin node(s), cssh for nodes' OS maintenance. -- Šarūnas Burdulis Dartmouth Mathematics https://math.dartmouth.edu/~sarunas · https://useplaintext.email ·
On 7/3/25 00:02, Anthony Fecarotta wrote:
Thank you. From what I can tell, after being on the mailing list for six months, it seems most users are running Kubernetes.
The last Ceph survey we did (2022) showed a different picture: https://ceph.io/en/news/blog/2022/ceph-user-survey-results-2022/ https://ceph.io/en/news/blog/2022/ceph-user-survey-results-2022/images/ceph-... (see page 74). It's mostly used in VM environments. I think this still holds true in 2025, although the balance might have shifted a bit towards Kubernetes in the mean time.
I have been able to get by without implementing Kubernetes. I was looking into OpenStack or OpenNebula, although many have deterred me from migrating to OpenStack, citing unneeded complexity. With all that said, GPU time-slicing (Nvidia MIG) is on our roadmap, and I have read the easiest way to achieve that is through the Kubernetes GPU Operator. It just seems unnecessary to implement currently.
We are using OpenNebula. They have support for various GPU configurations [1]. We have Ceph clusters running as packages on bare metal, and also (newer clusters) with containers (Cephadm). You definitely don't need Kubernetes to achieve a high level of automation with Ceph, but I guess it makes sense when you use Kubernetes extensively anyway. Just pick what you feel comfortable with. Gr. Stefan [1]: https://opennebula.io/blog/product/gpu-and-vgpu-in-opennebula/
2025年7月3日(木) 1:17 Anthony Fecarotta <anthony@linehaul.ai>:
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on?
We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions.
If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you.
The result of user survey is useful. https://ceph.io/en/users/survey/ Best, Satoru
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
[image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thank you, Satoru! Exactly what I was looking for! 🙇🏻♂️ Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181 [image] www.linehaul.ai <http://www.linehaul.ai/> [image] <http://www.linehaul.ai/> [image] <https://www.linkedin.com/in/anthony-fec/> On Fri Jul 4, 2025, 12:14 AM GMT, Satoru Takeuchi <mailto:satoru.takeuchi@gmail.com> wrote:
2025年7月3日(木) 1:17 Anthony Fecarotta <anthony@linehaul.ai <mailto:anthony@linehaul.ai>>:
Hello, I was wondering if there are any statistics on which platforms users/organizations are running Ceph on?
We have been running Proxmox since day one, but are interested in migrating to something else that supports Ceph. This choice is mainly because of the lack of RGW UI, as well as the incompatibility with the Ceph dashboard for the last few versions.
If there aren't any statistics available, I would be interested in hearing anyone's feedback and recommendations. Thank you.
The result of user survey is useful.
https://ceph.io/en/users/survey/ <https://ceph.io/en/users/survey/>
Best, Satoru
Regards, [image] Anthony Fecarotta Founder & President [image] anthony@linehaul.ai <mailto:anthony@linehaul.ai> <mailto:anthony@linehaul.ai <mailto:anthony@linehaul.ai>> [image] 224-339-1182 [image] (855) 625-0300 [image] 1 Mid America Plz Flr 3 Oakbrook Terrace, IL 60181
[image] www.linehaul.ai <http://www.linehaul.ai> <http://www.linehaul.ai/ <http://www.linehaul.ai/>> [image] <http://www.linehaul.ai/ <http://www.linehaul.ai/>> [image] <https://www.linkedin.com/in/anthony-fec/ <https://www.linkedin.com/in/anthony-fec/>> _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
participants (8)
-
Anthony Fecarotta
-
Hans van den Bogert
-
Michael Knox
-
Olivier AUDRY
-
quaglio＠bol.com.br
-
Satoru Takeuchi
-
Stefan Kooman
-
Šarūnas Burdulis