ceph-dashboard-pr-backend overhead on PRs
Hello, I have a question about "ceph dashboard backend API tests" status and its ceph-dashboard-pr-backend job. It appears to be completely standalone which means that we are building everything twice for each PR: once for "make check" and once for "ceph dashboard backend API tests". The only difference between these builds is that for "make check" WITH_SEASTAR=ON is added. Can ceph-pull-requests job which is responsible for "make check" be changed to run run-backend-api-tests.sh at the end and report two statuses instead of one? That is: - run make - if successful, run ctest and report "make check" status - if successful, run run-backend-api-tests.sh and report "ceph dashboard backend API tests" status Thanks, Ilya
On 6/29/20 11:12 AM, Ilya Dryomov wrote:
Hello,
I have a question about "ceph dashboard backend API tests" status and its ceph-dashboard-pr-backend job. It appears to be completely standalone which means that we are building everything twice for each PR: once for "make check" and once for "ceph dashboard backend API tests". The only difference between these builds is that for "make check" WITH_SEASTAR=ON is added.
Can ceph-pull-requests job which is responsible for "make check" be changed to run run-backend-api-tests.sh at the end and report two statuses instead of one? That is:
- run make - if successful, run ctest and report "make check" status - if successful, run run-backend-api-tests.sh and report "ceph dashboard backend API tests" status
This is doable. I'll work on it.
On 6/29/20 11:45 AM, David Galloway wrote:
On 6/29/20 11:12 AM, Ilya Dryomov wrote:
Hello,
I have a question about "ceph dashboard backend API tests" status and its ceph-dashboard-pr-backend job. It appears to be completely standalone which means that we are building everything twice for each PR: once for "make check" and once for "ceph dashboard backend API tests". The only difference between these builds is that for "make check" WITH_SEASTAR=ON is added.
Can ceph-pull-requests job which is responsible for "make check" be changed to run run-backend-api-tests.sh at the end and report two statuses instead of one? That is:
- run make - if successful, run ctest and report "make check" status - if successful, run run-backend-api-tests.sh and report "ceph dashboard backend API tests" status
This is doable. I'll work on it.
I'm not sure what the best way to go about this is. I wanted to have a job similar to the -trigger job that would sequentially kick off ceph-pull-requests, then ceph-dashboard-pr-backend but as far as I can tell, there's no way to use the same slave. It's way too much data (40GB!) to repeatedly upload to Jenkins then to another slave using the copyartifact plugin. This is originally what I had in mind: https://github.com/ceph/ceph-build/compare/wip-combine-db-prs I think we're just going to have to combine the jobs. We can still use the function I created in scripts/build_utils.sh. It's just going to take some time to figure out the conditional-step stuff to get the PR status updated using it. I'll try some more tomorrow but I'm on PTO July 1 - 8 so it may have to wait.
On Tue, Jun 30, 2020 at 1:12 AM David Galloway <dgallowa@redhat.com> wrote:
On 6/29/20 11:45 AM, David Galloway wrote:
On 6/29/20 11:12 AM, Ilya Dryomov wrote:
Hello,
I have a question about "ceph dashboard backend API tests" status and its ceph-dashboard-pr-backend job. It appears to be completely standalone which means that we are building everything twice for each PR: once for "make check" and once for "ceph dashboard backend API tests". The only difference between these builds is that for "make check" WITH_SEASTAR=ON is added.
Can ceph-pull-requests job which is responsible for "make check" be changed to run run-backend-api-tests.sh at the end and report two statuses instead of one? That is:
- run make - if successful, run ctest and report "make check" status - if successful, run run-backend-api-tests.sh and report "ceph dashboard backend API tests" status
This is doable. I'll work on it.
I'm not sure what the best way to go about this is. I wanted to have a job similar to the -trigger job that would sequentially kick off ceph-pull-requests, then ceph-dashboard-pr-backend but as far as I can tell, there's no way to use the same slave. It's way too much data (40GB!) to repeatedly upload to Jenkins then to another slave using the copyartifact plugin.
This is originally what I had in mind: https://github.com/ceph/ceph-build/compare/wip-combine-db-prs
I think we're just going to have to combine the jobs. We can still use the function I created in scripts/build_utils.sh. It's just going to take some time to figure out the conditional-step stuff to get the PR status updated using it. I'll try some more tomorrow but I'm on PTO July 1 - 8 so it may have to wait.
Yes, a single job that runs on a single worker and reports two statuses instead of one is what I had mind. Copying artifacts is faster than producing them again from scratch of course, but still unnecessary work. Thanks, Ilya
participants (2)
-
David Galloway
-
Ilya Dryomov