CephFS maturity on Squid?
Hi, For those running CephFS on Squid: how mature is the release in your experience? I'm especially interested in those using CephFS in a shared hosting scenario, with many directories containing many small files. I'm extra wary of new releases nowadays, as we've ran into several production-impacting Ceph bugs in the past months post-upgrade, across clusters. (Most notably, OSDs failing causing the MDS to OOM (17.2.7 -> 17.2.8) [1], and major performance issues after upgrading to Reef caused by the aligned 64k blocks constraint [2] (RBD though, not CephFS).) With kind regards, William David Edwards [1]: https://tracker.ceph.com/issues/69764 [2]: https://github.com/ceph/ceph/pull/54772
Hi, since we upgraded to Squid just yesterday morning, it's probably a bit early for a reliable response. But the upgrade itself went great, no hickups at all. We also haven't had any complaints (yet), so I guess we're good (for now). We use CephFS for home directories and software development, RBD for OpenStack and RGW for k8s backups (quite low usage). I assume we would notice issues relatively quick. We haven't heard any complaints from customers yet either, but not all of them are on Squid yet, so hard to say. Some new Squid deployments on customer clusters also seem stable (we haven't had emergency calls yet wrt CephFS). This probably doesn't help you a lot but I wanted to comment anyway. :-) Regards, Eugen Zitat von William David Edwards <wedwards@cyberfusion.nl>:
Hi,
For those running CephFS on Squid: how mature is the release in your experience? I'm especially interested in those using CephFS in a shared hosting scenario, with many directories containing many small files.
I'm extra wary of new releases nowadays, as we've ran into several production-impacting Ceph bugs in the past months post-upgrade, across clusters. (Most notably, OSDs failing causing the MDS to OOM (17.2.7 -> 17.2.8) [1], and major performance issues after upgrading to Reef caused by the aligned 64k blocks constraint [2] (RBD though, not CephFS).)
With kind regards,
William David Edwards
[1]: https://tracker.ceph.com/issues/69764 [2]: https://github.com/ceph/ceph/pull/54772 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I don´t have the largest data farm around, but I do have a fairly eclectic set of files and directories. I have seen no differences in cephfs moving to squid, though I do have some issues that are common to Pacific and Squid. 1. cephs cannot gracefully handle clients who go off/online. Specifically sleeping clients. It causes ceph resources to leak and ceph to whine. The ceph-side resources have to be closed out manually. 2. I have some clients where I've tried using ceph-nfs where some resources do not appear properly. In particular, files that return no data even though in cephfs itself the files are not empty. Some recent posts on the list here have potential to maybe fix #2, but I haven't tried them yet. For permanent cephfs native clients I have no complaints. Tim On 9/17/25 09:37, William David Edwards wrote:
Hi,
For those running CephFS on Squid: how mature is the release in your experience? I'm especially interested in those using CephFS in a shared hosting scenario, with many directories containing many small files.
I'm extra wary of new releases nowadays, as we've ran into several production-impacting Ceph bugs in the past months post-upgrade, across clusters. (Most notably, OSDs failing causing the MDS to OOM (17.2.7 -> 17.2.8) [1], and major performance issues after upgrading to Reef caused by the aligned 64k blocks constraint [2] (RBD though, not CephFS).)
With kind regards,
William David Edwards
[1]: https://tracker.ceph.com/issues/69764 [2]: https://github.com/ceph/ceph/pull/54772 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On Thu, 18 Sept 2025 at 08:00, Tim Holloway <timh@mousetech.com> wrote:
2. I have some clients where I've tried using ceph-nfs where some resources do not appear properly. In particular, files that return no data even though in cephfs itself the files are not empty.
Hi Tim, Could you elaborate on your observation please: "...files that return no data..." Is this an interaction between NFS and Ceph or how clients are interacting with NFS? thanks.
If I mount via NFS, I can see a file like "hello.txt", but if I try to display its contents, nothing displays. I also cannot write to such files. On 9/17/25 18:59, Nigel Williams wrote:
On Thu, 18 Sept 2025 at 08:00, Tim Holloway <timh@mousetech.com> wrote:
2. I have some clients where I've tried using ceph-nfs where some resources do not appear properly. In particular, files that return no data even though in cephfs itself the files are not empty.
Hi Tim,
Could you elaborate on your observation please: "...files that return no data..."
Is this an interaction between NFS and Ceph or how clients are interacting with NFS?
thanks. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (4)
-
Eugen Block
-
Nigel Williams
-
Tim Holloway
-
William David Edwards