All,

I’m happy to announce a new Jenkins pipeline: Ceph PR Pipeline

This consists of two new jobs, ceph-pr-pipeline-trigger, and ceph-pr-pipeline. This disables the following jobs: ceph-pr-api, ceph-pr-commits, ceph-pr-submodules, ceph-pull-requests-arm64, ceph-pull-requests, ceph-windows-pull-requests.

Benefits:

All of the same Checks, same names, etc.
Ceph binaries built once per PR instead of twice

make check and the ceph API testshave historically each compiled Ceph from scratch at the expense of builder time. Now, binaries are built once and fanned out to multiple checks. This will not speed up the wall clock time waiting for compilation and testing (Windows has to compile separately and the job still takes 2 hours, so it's still the last check to finish) but the ceph API tests result will show up about an hour sooner since that job no longer does its own build. During busy periods (e.g., Monday morning), there will be about a 10% reduction in builder contention (i.e., time spent waiting for your checks to start), which also indirectly helps anything else sharing those builders, including dev package builds from ceph-ci.git.
Test/Check re-runs no longer need to recompile

Once the binaries for a given sha1 are built, they are shipped to s3 storage and stored for 21 days. In the event of an infrastructure outage or flaky test, you can re-run the same test and won’t need to wait for binary rebuild. Again, this both speeds up the re-check itself and reduces the amount of builder hours spent unnecessarily recompiling the same binaries.
External contributors get a clearer path

In conjunction with this new GitHub Action, Pull Request authors who do not yet have write access to the Ceph repo will receive a comment (one, instead of fourteen) thanking them for their contribution and a note that a needs-ci-approval label is added to the PR. Ceph Org members may trigger the PR Checks by adding a ci-approved label to the PR. The needs-ci-approval label will be removed by the Jenkins trigger and the jobs will proceed to run normally. The “ok to test” comment no longer functions. Force pushes result in the needs-ci-approval label being re-added.
Retires the long-ago-deprecated GitHub Pull Request Builder Plugin for six jobs

All of the usual comments to retrigger a Check/job work. e.g., “jenkins retest, jenkins test make check, etc.”

The pipeline was enabled at 10am Eastern time. I’ll keep an eye on new PRs throughout the day and fix up any edge cases.

David Galloway

Ceph Engineering Labs – Infrastructure Architect

david.galloway@ibm.com

IBM