Reworked container builds (for centos9 CI builds)
I've just merged https://github.com/ceph/ceph-build/pull/2280, which changes the ceph-dev-new family of Jenkins jobs (the jobs that build branches pushed to ceph-ci.git) to use code from ceph.git to build containers. Ideally all you should need to know about this is nothing; containers are still built for CI branches and pushed with the same tags as they always have been to quay.ceph.io. Of course there may be unforeseen bugs; if a build fails while building containers, and the failure is not clear, please contact me ASAP. Details: Until today, a separate workspace, ceph-container.git, held the code for building Ceph containers. This code is complex and hard to understand and maintain, and supplies much more flexibility than we've ever used for Ceph. The newer strategy is much simpler, and will make container building much more understandable and maintainable, as well as keeping the container sources nearby the code they're used to package. ceph.git now has a container/ subdirectory that holds the parameterized Containerfile and a build script, build.sh. These changes to ceph-build change the Jenkins job to use those new files for building containers, independently from ceph-container.git. Release containers are still handled by ceph-container.git, but that will also change soon.
Hi Dan, Thanks for PSA, I have a question: On Tue, Oct 8, 2024 at 8:32 PM Dan Mick <dmick@redhat.com> wrote:
I've just merged https://github.com/ceph/ceph-build/pull/2280, which changes the ceph-dev-new family of Jenkins jobs (the jobs that build branches pushed to ceph-ci.git) to use code from ceph.git to build containers. Ideally all you should need to know about this is nothing; containers are still built for CI branches and pushed with the same tags as they always have been to quay.ceph.io. Of course there may be unforeseen bugs; if a build fails while building containers, and the failure is not clear, please contact me ASAP.
Details: Until today, a separate workspace, ceph-container.git, held the code for building Ceph containers.
Should we archive / mark read-only ceph-container.git?
This code is complex and hard to understand and maintain, and supplies much more flexibility than we've ever used for Ceph. The newer strategy is much simpler, and will make container building much more understandable and maintainable, as well as keeping the container sources nearby the code they're used to package. ceph.git now has a container/ subdirectory that holds the parameterized Containerfile and a build script, build.sh. These changes to ceph-build change the Jenkins job to use those new files for building containers, independently from ceph-container.git.
Release containers are still handled by ceph-container.git, but that will also change soon.
Ah, so we cannot archive it yet. What is the ultimate goal of this work? Will developers ever run this code to make containers or is it simply for CI testing? -- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
On 10/8/2024 6:05 PM, Patrick Donnelly wrote:
Hi Dan,
Thanks for PSA, I have a question:
On Tue, Oct 8, 2024 at 8:32 PM Dan Mick <dmick@redhat.com> wrote:
I've just merged https://github.com/ceph/ceph-build/pull/2280, which changes the ceph-dev-new family of Jenkins jobs (the jobs that build branches pushed to ceph-ci.git) to use code from ceph.git to build containers. Ideally all you should need to know about this is nothing; containers are still built for CI branches and pushed with the same tags as they always have been to quay.ceph.io. Of course there may be unforeseen bugs; if a build fails while building containers, and the failure is not clear, please contact me ASAP.
Details: Until today, a separate workspace, ceph-container.git, held the code for building Ceph containers.
Should we archive / mark read-only ceph-container.git?
Soon. As you note below, not quite yet.
This code is complex and hard to understand and maintain, and supplies much more flexibility than we've ever used for Ceph. The newer strategy is much simpler, and will make container building much more understandable and maintainable, as well as keeping the container sources nearby the code they're used to package. ceph.git now has a container/ subdirectory that holds the parameterized Containerfile and a build script, build.sh. These changes to ceph-build change the Jenkins job to use those new files for building containers, independently from ceph-container.git.
Release containers are still handled by ceph-container.git, but that will also change soon.
Ah, so we cannot archive it yet. What is the ultimate goal of this work? Will developers ever run this code to make containers or is it simply for CI testing?
There's no reason developers can't run this code, nor was there any reason they couldn't run ceph-container (other than its complexity/learning curve). The intent is to simplify and make maintainable the CI and release processes, but a side benefit could well be more ease of flexibility for the developer. The more-standard and more-easily-used the code is, the more consumers there should be. This should also help the downstream build process, and should streamline further build plans should we move to some other build system.
One thing I should have noted: You'll need to make sure you've rebased on main, or done something else to ensure that you have the changes to ceph.git in https://github.com/ceph/ceph/pull/59868 (the new src/container directory and contents). Also, I forgot this would need backporting to prior releases, so I've just filed backport PRs for reef, squid, and quincy: https://github.com/ceph/ceph/pull/60228 https://github.com/ceph/ceph/pull/60229 https://github.com/ceph/ceph/pull/60230 ...and once those are merged, you may need to rebase your branch on the appropriate corresponding branch. Sorry for the oversight. On 10/8/2024 5:31 PM, Dan Mick wrote:
I've just merged https://github.com/ceph/ceph-build/pull/2280, which changes the ceph-dev-new family of Jenkins jobs (the jobs that build branches pushed to ceph-ci.git) to use code from ceph.git to build containers. Ideally all you should need to know about this is nothing; containers are still built for CI branches and pushed with the same tags as they always have been to quay.ceph.io. Of course there may be unforeseen bugs; if a build fails while building containers, and the failure is not clear, please contact me ASAP.
Details: Until today, a separate workspace, ceph-container.git, held the code for building Ceph containers. This code is complex and hard to understand and maintain, and supplies much more flexibility than we've ever used for Ceph. The newer strategy is much simpler, and will make container building much more understandable and maintainable, as well as keeping the container sources nearby the code they're used to package. ceph.git now has a container/ subdirectory that holds the parameterized Containerfile and a build script, build.sh. These changes to ceph-build change the Jenkins job to use those new files for building containers, independently from ceph-container.git.
Release containers are still handled by ceph-container.git, but that will also change soon.
participants (2)
-
Dan Mick
-
Patrick Donnelly