On Wednesday, May 17, 2023 6:00:01 PM EDT Dan Mick wrote:
A long long time ago when this first came up, I suggested we have instructions that mechanized finding the package from download.ceph.com and downloading/extracting the zipapp from it. I felt strongly at the time that we should try our best to avoid having two sources of truth for the Real Tested Binary, and packages were the obvious answer, but using the actual package management software was not suitable, so wrote the extraction code.
We didn't ever release that code for extracting from the package, but the technique found its way into the build jobs in Oct 2021:
At present, at least for the CI builds, the cephadm binary/zipapp is extracted from packages during the build and placed, separately, in the package repo as a plain file, unmanaged by the package managers. See https://github.com/ceph/ceph-build/blob/main/ceph-dev-new-build/build/build_ rpm#L59-L63 and https://github.com/ceph/ceph-build/blob/main/scripts/build_utils.sh#L984-L98 8 (the asymmetry was there when I got there).
OK, that's a good start. :-)
That code is also present in the "ceph-dev-build" and "ceph-build" jobs (which do candidate builds for named releases and final release builds, respectively), and indeed cephadm is present in the rpm repos (for example, http://download.ceph.com/rpm-17.2.6/el8/noarch/cephadm) but I don't see the cephadm binary on download.ceph.com for deb repos. (It's probably the "sync to download.ceph.com" step being unaware it should copy anything besides .deb.)
Unfortunately, I don't think this RPM derived cephadm is appropriate for general uses. RPM mangles the shebang line to something RH-distro specific so this `cephadm` wouldn't work properly on distros like debian, ubuntu, etc. I think it does this to the zipapp too - but I would need to verify it. I'm aware of chacra.ceph.com (and 1.chacra.ceph.com) but all I can get from it is JSON and I would like to inspect the actual binary. I'm probably not using the proper interface but I don't know where to look. If the file extracted from the ubuntu deb package does not get mangled and the shebang simply refers to `python3` then we'd be good there once the issue with the deb release process is resolved.
My suggestion would be to debug the issue with the deb release process, and document that the binary can be found in the appropriate path corresponding to the release in download.ceph.com.
Sounds like a plan, is that something you expect us to do, or is it something you are planning to do? I'm happy to help but I'm afraid I wouldn't know where to start so at least I'd need some initial "hand holding". Thanks!
On 5/16/2023 10:10 AM, John Mulligan wrote:
Hi Dan & others,
A few months back we (the Orchestration team working on cephadm), discussed the new compiled [1] cephadm in a CLT call and we briefly discussed it on the list [2]. I wanted to revisit that conversation as Reef's release is quickly coming up.
To summarize: Previous versions of Ceph contained a single python source file for cephadm. Upstream users were instructed to copy this source file directly out of the Ceph git tree and execute the file using python. Now, the ceph build process creates an executable python zipapp from the source file and the old instructions are out of date.
We would like to request assistance with making it possible for users who are planning on bootstrapping clusters to download an already compiled version of cephadm from a canonical location. A secondary goal would also be to sign that binary. However, I would treat that as a nice to have since the current workflow doesn't have this.
I'd love to discuss the technical aspects of this and get some code to implement this in place. I'd be happy to continue the conversation here - or if you prefer - in the discussion forum of your choice. Thanks very much!
[1] - It is still python, and it's not compiled to native binaries but is rather a zipapp - https://docs.python.org/3/library/zipapp.html
[2] - I wanted to link to the previous discussion but neither the Archived-At header url or search turn it up. So maybe it got lost in the infra issues a while back. If you need references to the previous thread, I can forward them because I still have them on my mailserver.
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io