Hi Thomas, I don't really have the time to help maintaining it, sorry. But regarding ceph for 32bit: please drop it. It might compile, but it's not supported by upstream and as long as you can't run the full test suite it doesn't make sense to risk it. Last time I've checked it there were lots assumptions on 64bit in the code that were not even detected by the compiler on 32bit. It will also save you a lot of work... Bernd 28.09.2023 09:11:32 Thomas Goirand <zigo(a)debian.org>rg>: > On 9/27/23 17:05, Gregory Farnum wrote: >> Also, I’d like to connect you and Thomas, who has been working on >> Debian builds in irc again and maintains Ceph packages in Debian >> proper. >> -Greg > > Hi there! > > Thanks Gregory for getting in touch. > > I also CC-ed Bzed, who's co-maintaining Ceph with me (even though I was alone for the last 2 years or so, I still have hope that Bzed helps). > > My mission is probably a bit different from what Matthew is doing. Trying to maintain Ceph in Debian has to be done in Unstable/Testing, which is a way more challenging than building for one's own unofficial backport repository for amd64 alone. For example, I'm currently fighting to get Ceph Pacific fixed for SID and RiscV (we believe we have a solution for it, ie: using -latomic). See here, it still doesn't build: > https://buildd.debian.org/status/package.php?p=ceph&suite=sid > > At the same time, I'm trying to get Reef to build in Experimental: > https://buildd.debian.org/status/package.php?p=ceph&suite=experimental > > As you may see, it currently fails under all 32 bits arch. The issue is, we need Ceph for these arch too, so that Qemu can be built against librbd, otherwise all 32 bits arch wont get RBD support. Lucky, Mipsel was removed, so at least, we have 3GB of RAM to build, not 2. > > I'm not so much interested in packaging in non-official Debian, or even upstream. In fact, I'd be for removing the Debian folder from the Ceph upstream master branch, as this is causing issues on the downstream distributions (ie: I have to remove the debian folder from the orig tarball). Best would be if we could all (wikimedia, Ubuntu, Debian) share the same packaging branch on top of master or stable branches of Ceph, and somehow build a CI to automatically build packages. I have enough experience to write such a CI (which I do already for OpenStack). Though it'd probably be nicer to have this done in the upstream infrastructure, so that we could build Debian and Ubuntu packages for stable AND unstable (Ubuntu/Debian, but also Ceph), aso we see things break. > > My Debian packaging is also a way more Debian policy compliant than what I saw from upstream. For example, I noticed that during the build, Ceph build Arrow, which in its turn, download xsimd (and of course, downloading at build time is strictly forbidden in Debian), so I had to add "extraopts += -DWITH_RADOSGW_SELECT_PARQUET=OFF" until we get arrow properly packaged in Debian. I also fixed many of the things you fixed in the merge request from this week-end. > > I very much would love to get more help on the packaging, and do it with you Matthew. Note that I already work with Arturo Borrero Gonzalez from Wikimedia as well, as I am also the package maintainer for all things OpenStack (which is why I am interested in Ceph). > > My current plan is to finish the packaging of Ceph Reef in Experimental, which means, make it build for all arch in the Debian buildd, before opening a transition bug to the release team and upload to Unstable (at which point, the advise is to attempt rebuilding all reverse depends and file bugs as necessary). Once the package reaches Unstable, and then Testing/Trixie, I want to also upload Reef to official bookworm-backports. > > Our current production plan is to run Ceph on bookworm+arm64 for a new region of our public cloud. Hopefully, this can be done on Reef, using the official bookworm backports I described above. I can forecast a lot of fun testing all that in a real production environment... :) > FYI, we want to use HPE's RL300 servers (though if anyone has other suggestions for ARM hardware and Ceph, please let me know). > > The Ceph packaging is maintained in Salsa, like almost all of Debian: > > https://salsa.debian.org/ceph-team/ceph/ > > There's also a lot of work that should be done in order to de-embed many of the Ceph included libraries: > - jerasure + isal (I already packaged this in Debian) > - arrow (there's a start of a package from the Debian science team [1]) > - uring (Ceph Reef currently fails with the Debian version) > - boost (same thing... we need to transition to 1.8.2 in Unstable) > > This is a lot of work, and really, too much for only myself, who's also in charge of OpenStack, OpenVSwitch/OVN, RabbitMQ, and many more in Debian. > > So, Matthew, would you like to do packaging with me directly in Debian? If so, I warmly welcome you in the #debian-openstack IRC channel in OFTC if you would like to kickstart this (or we continue by email if you prefer). > > Cheers, > > Thomas Goirand (zigo) > > [1] See https://bugs.debian.org/970021 and https://salsa.debian.org/science-team/arrow > This start of packaging would need to be upgraded to version 9.0.0 *at least* (we got to figure out if Ceph can cope with an even newer version), and must make sure it doesn't download xsimd at build time. Probably, some of the arrow (build-)depends should also be packaged separately. > > P.S: I'm not registered to this list (I'm registered to a way too many lists...), so please do CC me.

On 9/29/23 14:53, Matthew Vernon wrote: Sure! But I've been disappointed by it multiple times. Like when the Ceph upstream started to use feature of GCC that were not available at the time in Stable, and then Debian packages were given-up. It gave me the feeling that it couldn't be trusted. This time, it happened again for the Reef packages, that were not really usable in Debian. I'm really motivated to fix both the packages in Debian itself, and upstream too. I'm convince that we should work on both. I don't agree with this, especially considering we need librbd for many packages in Debian. I counted 10 packages in Debian that have build-depends on librbd-dev or librados-dev. If Ceph gets removed from Debian, then the support for Ceph by those packages is gone too... Definitively, I'm all for getting all of the work I did merged at some point, and lower the differences as much as possible. I also think it's a waste of time if we work on things twice (upstream, like you did, and downstream, like I did). Hopefully, we can get to a state where the diff between the Debian package and the upstream Ceph one is very small. Great! The question is: where to start. What I envisioned was fixing all I could on the Debian side of things first, as this wouldn't depend on having patches merged upstream, like you experimented (as I saw it, it looked kind of frustrating to see nobody seemed to care at first). I don't think I should change my workflow for the moment, and probably polishing things in Debian will get faster to the end result. But I'm opened to do differently if you think I should. Cheers, Thomas Goirand (zigo)

Sent from Workspace ONE Boxer On Sep 29, 2023 6:07 PM, Bernd Zeimetz <bzed(a)debian.org> wrote: > I don't have much time to reply right now, but today, I wrote bug reports to ask for 32 bits removal of Ceph support on the 10 packages that had librbd or librados build-depends. So I agree... I also don't have skills and time to reasonably build only libs for 32 bits. Contribs welcome... Not entirely, though I did some work, diffing the 2, so that Debian and upstream packaging converges even more. At this point, IMO, I must invest some time on upstream packaging. Cheers, Thomas Goirand

On Fri, Sep 29, 2023 at 8:04 PM Bernd Zeimetz <bzed(a)debian.org> wrote: If krbd doesn't work, let me know! Like the rest of Ceph, it's not tested in 32-bit environments, but, given that krbd is self-contained, bugs (if any) should be trivial to fix. Owning to being part of the kernel, I assume that it at least builds cleanly. If someone wanted to contribute a test to the krbd test suite (e.g. involving a 32-bit VM image, based on Debian or otherwise), they would be very much welcome ;) Thanks, Ilya

On Mon, Oct 2, 2023 at 3:17 PM Thomas Goirand <thomas(a)goirand.fr> wrote: QEMU doesn't need to be built specially. krbd gives you a /dev/rbd* block device which can be fed to QEMU just like e.g. /dev/sd*. Thanks, Ilya

Thanks again for bringing this to our attention. We resolved the build issue with tcmalloc in Ubuntu packages. The regression was introduced when Ubuntu enabled the crimson backend for testing [0]. This regression only impacted Ubuntu, not Debian. [0] https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2016845/comments/4 On Mon, Oct 2, 2023 at 7:24 PM Mark Nelson <mark.nelson(a)clyso.com> wrote: > > On 9/29/23 11:01, Bernd Zeimetz wrote: > > When I started looking into ceph I've removed lots of stuff from the > > debian folder that the ubuntu people added for unknown reasons, its > > much closer to upstream these days then it was before. Actually close > > enough that it is no problem to switch to the upstream packages at all. > > > Mark > > -- > Best Regards, > Mark Nelson > Head of Research and Development > > Clyso GmbH > p: +49 89 21552391 12 | a: Minnesota, USA > w: https://clyso.com | e: mark.nelson(a)clyso.com > > We are hiring: https://www.clyso.com/jobs/ > _______________________________________________ > Dev mailing list -- dev(a)ceph.io > To unsubscribe send an email to dev-leave(a)ceph.io >

Just to be clear:  Ceph still uses tcmalloc on RHEL/CentOS, but we can no longer rely on built-in tcmalloc packages and have to do it ourselves.  That's why qemu no longer uses tcmalloc (but Ceph does). See: https://bugzilla.redhat.com/show_bug.cgi?id=1717414 Mark On 11/6/23 11:20, Mark Nelson wrote: -- Best Regards, Mark Nelson Head of Research and Development Clyso GmbH p: +49 89 21552391 12 | a: Minnesota, USA w: https://clyso.com | e: mark.nelson(a)clyso.com We are hiring: https://www.clyso.com/jobs/

We currently anticipate shipping them with the 18.2.1 point release. Yuri tried building Debian packages on 18.2.0 but as that doesn’t include your packaging fixes it didn’t go well. ;) -Greg On Fri, Oct 13, 2023 at 5:50 AM Matthew Vernon <mvernon(a)wikimedia.org> wrote: