- ceph-users - lists.ceph.io

Re: Questions about dm-cache
by Anthony D'Atri 13 Apr '26

13 Apr '26

> > On that we agree: WAL+DB dead is a dead OSD. I've seen a speculative outline on recovering the OSD but have not seen someone try. > And it's indeed a fair > tradeoff: I'd expect the bad backend drives to die long before Spinners often have a lower TBW and AFR than even low-endurance SSDs. > the enterprise write-intensive drives Do you find that you strictly need high endurance SSDs? 99% of enterprise SSDs never use more than 15% of their rated endurance. These days write-intensive usually means Optane-replacement SLC. More common here is mixed-use, which is typically the very same hardware as a "read intensive" SSD just with more overprovisioning. If you really need more endurance than comes out of the box with an RI device, you can often adjust the OP yourself at much lower cost. > I have for wal+db. (or, in the case of > spinning rust, I'd expect a mechanical failure long before a flash > failure). Most SSD failures are firmware and can be fixed in-situ. I've had to replace 4000 HDDs with a design flaw - scattered around the world. Nightmare > Thus, in the end, it's no different from a dedicated wal+db iff wal+db > stays dedicated: I've seen people suggest to keep wal+db colocated if > there's a fast flash in front of the drive, as wal+db would essentially > always be hot in the cache. Indeed I've read studies that claim a superset of benefits from a transparent cache compared to just WAL+DB. > (note: I know that the OSD thinks it directly talks to the crap drives, Reads are always served from the slow device, and writes over a certain size go there directly. I've seen use-cases where the hassle of WAL+DB offload doesn't actually make an appreciable improvement.

2 3

CSC meeting minutes 2026-04-13
by Casey Bodley 13 Apr '26

13 Apr '26

minutes from today's meeting of the Ceph Steering Committee: Discuss Umbrella release schedule - https://docs.ceph.com/en/latest/dev/release-checklists/#dev-kickoff - Based on release engineering meeting discussion - Feature freeze - end of May 2026 - RC phase June - Aug 2026 - GA September 2026 - where best to track features? trello/redmine/gh projects -- Core Trello - https://trello.com/b/ugTc2QFH/ceph-backlog -- redmine can list features with filter "Tracker" is "Feature" drop.ceph.com is unreachable post lab migration (https://tracker.ceph.com/issues/74517) - Infra team is aware, but cannot fix it for awhile - Is ceph-post-file still being used by people? If not, we may decide to remove the test. - ssh is blocked in new lab, perhaps a web form for uploads CSC Election - Stage two will run from April 15th to April 29th. This voting round will cover the following topics: -- CSC Co-Chairs -- Adding a removal policy to the CSC Governance -- CSC Representative, as stated in the CSC Charter - Candidates' speeches 1-2 minutes each. we heard from Dan van der Ster, Ernesto Puerta, Gregory Farnum, Neha Ojha, and Patrick Donnelly. Joseph Mundackal will email the csc list today - how does voting guarantee that no company has all 3 co-chairs? consensus: pick the first 2, and fall back to the next highest votes from a different company - co-chairs will select board rep amongst themselves should we enable immutable releases in the Ceph repo? Hardware gaps? CapEx planning.

1 0

Re: Status of iSCSI-Gateway
by Alexander Schreiber 13 Apr '26

13 Apr '26

On Mon, Apr 13, 2026 at 09:34:39AM +0000, Ernesto Puerta via ceph-users wrote: > Hey Eugen, > > AFAIK the Ceph NVMe-oF Gateway superseded the iSCSI one: it's actively > maintained, more performant, scalable, secure and better integrated with > the Ceph management & observability framework. > > I'm curious: what's the target platform/use case for iSCSI that NVMe/TCP > cannot fulfill? As someone who is running pure spinning rust Ceph clusters, I assume the NVMe-oF gateway requires actual NVMe devices to be used for backing, given that the docs mention sending NMVe commands to targets, correct? Handing out iSCSI devices sounds useful as backing storage for VMs (and some can even boot off it). Kind regards, Alex. -- "Opportunity is missed by most people because it is dressed in overalls and looks like work." -- Thomas A. Edison

2 1

Re: Status of iSCSI-Gateway
by Anthony D'Atri 13 Apr '26

13 Apr '26

> On Apr 13, 2026, at 8:32 AM, Alexander Schreiber via ceph-users <ceph-users(a)ceph.io> wrote: > > On Mon, Apr 13, 2026 at 09:34:39AM +0000, Ernesto Puerta via ceph-users wrote: >> Hey Eugen, >> >> AFAIK the Ceph NVMe-oF Gateway superseded the iSCSI one: it's actively >> maintained, more performant, scalable, secure and better integrated with >> the Ceph management & observability framework. The iscsi-gw has lacked a maintainer for quite a while. If someone really really needs to run iSCSI, they might check out https://github.com/frednass/ceph_petasan >> I'm curious: what's the target platform/use case for iSCSI that NVMe/TCP cannot fulfill? Clients that can't talk NVMe/TCP? E.g. some BIOS implementations? Certain VMware releases? > > As someone who is running pure spinning rust Ceph clusters, I assume the > NVMe-oF gateway requires actual NVMe devices to be used for backing, given > that the docs mention sending NMVe commands to targets, correct? I don't *think* that's the case, though I'd be curious re the use-case for the protocol backed by slow media. > > Handing out iSCSI devices sounds useful as backing storage for VMs > (and some can even boot off it). RBD through libvirt is an all-around better solution for that. Fewer moving parts, less overhead, lower resource utilization on the Ceph side. > > Kind regards, > Alex. > -- > "Opportunity is missed by most people because it is dressed in overalls and > looks like work." -- Thomas A. Edison > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Re: RGW: S3 bucket replication with account‑owned buckets on Ceph Squid 19.2.3
by Ashok Kumar M 13 Apr '26

13 Apr '26

Hello, I am testing S3 bucket replication on Ceph Squid 19.2.3 (RGW multisite with one realm, one zonegroup, and two zones: source and destination). Replication works as expected when: Source and destination buckets are owned by a regular RGW user. However, using the same replication configuration with account‑owned buckets (owner is an RGW account ID like RGWxxxxxxxxxxxxxxx) shows different behavior: The replication configuration is accepted. But new objects written to the source bucket are not replicated to the destination bucket. Additional observation: The S3 replication API is working for user buckets, but it is not working for account-owned buckets My questions Is S3 PutBucketReplication officially supported for account‑owned buckets in Ceph Squid 19.2.3? Are there any known limitations 0r recommended configuration or workaround to account‑owned bucket replication? Any guidance, references to documentation, or known issues would be very helpful. Thank you, Ashok MP Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

1 0

Re: Status of iSCSI-Gateway
by Aviv Caro 13 Apr '26

13 Apr '26

Note that NVMe-oF is GA'd as a part of tentacle. Earlier releases are limited/might not work. Regards, Aviv -----Original Message----- From: Eugen Block via dev <dev(a)ceph.io> Sent: Monday, 13 April 2026 13:26 To: Ernesto Puerta <epuertat(a)redhat.com> Cc: dev(a)ceph.io; ceph-users(a)ceph.io Subject: [EXTERNAL] Re: [ceph-users] Status of iSCSI-Gateway Yeah that's my first suspicion as well (HugePages), I'm gonna look into it. Thanks a lot! Zitat von Ernesto Puerta <epuertat(a)redhat.com>: > Maybe you're hitting issues configuring HugePages inside a VM? In any > case, there's an #nvmeof Slack channel and the team is always quick to > respond over there. > > Kind Regards, > Ernesto > > > On Mon, Apr 13, 2026 at 11:36 AM Eugen Block <eblock(a)nde.ag> wrote: > >> Hi, >> >> thanks for your quick response. >> >> The reason I'm asking about iSCSI is because I have been able to >> deploy and use it in earlier releases (the last one was probably >> Pacific), but I'm struggling with deploying NVMe-oF in a virtual lab >> cluster. I haven't had the time to really dig into it yet, so there's >> definitely a knowledge gap I need to close. Until I made some >> progress with NVMe-oF I thought maybe iSCSI could still be used. The >> use case is to grant access to an air-gapped Ceph cluster. >> >> Thanks! >> Eugen >> >> Zitat von Ernesto Puerta <epuertat(a)redhat.com>: >> >> > Hey Eugen, >> > >> > AFAIK the Ceph NVMe-oF Gateway superseded the iSCSI one: it's >> > actively maintained, more performant, scalable, secure and better >> > integrated with the Ceph management & observability framework. >> > >> > I'm curious: what's the target platform/use case for iSCSI that >> > NVMe/TCP cannot fulfill? >> > >> > Kind Regards, >> > Ernesto >> > >> > >> > On Mon, Apr 13, 2026 at 8:58 AM Eugen Block via ceph-users < >> > ceph-users(a)ceph.io> wrote: >> > >> >> Hi, >> >> >> >> could someone clarify if using iscsi via cephadm is still >> >> supported or not? According to [0] it's "in maintenance" since >> >> November 2022, so no active development. But it's still documented >> >> in all active releases and can be deployed in Squid or Tentacle >> >> (though I haven't checked if it actually works). >> >> Is it still supported to use iSCSI? >> >> >> >> Thanks! >> >> Eugen >> >> >> >> [0] >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.ceph.com >> >> _en_latest_rbd_iscsi-2Doverview_-23ceph-2Discsi&d=DwIFaQ&c=BSDicqB >> >> QBDjDI9RkVyTcHQ&r=Im4omk4NdceLPvgb6sEXw3r6y71Z9JDuTGk4GMFulW0&m=VD >> >> MpTn9kP-I6POYWFAW6K418IPSCrPIGEyC5M22ibV2sgB49EZVv3TE0VYFRXwsN&s=S >> >> pUmSLpyOcc2_lfh9pDoVoVxJCEwgOwDDvLgQah7Sl4&e= >> >> _______________________________________________ >> >> ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send >> >> an email to ceph-users-leave(a)ceph.io >> >> >> >> >> >> >> >>

1 0

Re: Status of iSCSI-Gateway
by Ernesto Puerta 13 Apr '26

13 Apr '26

Hey Eugen, AFAIK the Ceph NVMe-oF Gateway superseded the iSCSI one: it's actively maintained, more performant, scalable, secure and better integrated with the Ceph management & observability framework. I'm curious: what's the target platform/use case for iSCSI that NVMe/TCP cannot fulfill? Kind Regards, Ernesto On Mon, Apr 13, 2026 at 8:58 AM Eugen Block via ceph-users < ceph-users(a)ceph.io> wrote: > Hi, > > could someone clarify if using iscsi via cephadm is still supported or > not? According to [0] it's "in maintenance" since November 2022, so no > active development. But it's still documented in all active releases > and can be deployed in Squid or Tentacle (though I haven't checked if > it actually works). > Is it still supported to use iSCSI? > > Thanks! > Eugen > > [0] https://docs.ceph.com/en/latest/rbd/iscsi-overview/#ceph-iscsi > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > >

2 3

Re: Has anyone been able to use NFS in tentacle or it is really broken?
by Eugen Block 13 Apr '26

13 Apr '26

Hm, the host where I try to deploy nfs (openSUSE Leap 16 virtual machine) has rpcbind installed and it still fails: storage:~ # rpm -qa | grep rpcbind rpcbind-1.2.7-160000.2.2.x86_64 Regards, Eugen Zitat von Robert Sander via ceph-users <ceph-users(a)ceph.io>: > Hi, > > On 4/10/26 17:31, Salatiel Filho via ceph-users wrote: >> It is broken. >> https://tracker.ceph.com/issues/74307 >> https://tracker.ceph.com/issues/74191 > > Thanks for the links. > > It looks like installing rpcbind fixes the issue. > > Why does the orchestrator does not install rpcbind on the node where > an NFS daemon instance is to be deployed? > > Regards > -- > Robert Sander > Heinlein Consulting GmbH > Schwedter Str. 8/9b, 10119 Berlin > > http://www.heinlein-support.de > > Tel: 030 / 405051-43 > Fax: 030 / 405051-19 > > Zwangsangaben lt. §35a GmbHG: > HRB 220009 B / Amtsgericht Berlin-Charlottenburg, > Geschäftsführer: Peer Heinlein -- Sitz: Berlin > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Status of iSCSI-Gateway
by Eugen Block 13 Apr '26

13 Apr '26

Hi, could someone clarify if using iscsi via cephadm is still supported or not? According to [0] it's "in maintenance" since November 2022, so no active development. But it's still documented in all active releases and can be deployed in Squid or Tentacle (though I haven't checked if it actually works). Is it still supported to use iSCSI? Thanks! Eugen [0] https://docs.ceph.com/en/latest/rbd/iscsi-overview/#ceph-iscsi

1 0

Re: Has anyone been able to use NFS in tentacle or it is really broken?
by Robert Sander 12 Apr '26

12 Apr '26

Hi, On 4/10/26 17:31, Salatiel Filho via ceph-users wrote: > It is broken. > https://tracker.ceph.com/issues/74307 > https://tracker.ceph.com/issues/74191 Thanks for the links. It looks like installing rpcbind fixes the issue. Why does the orchestrator does not install rpcbind on the node where an NFS daemon instance is to be deployed? Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 220009 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

1 0