Boost on main updated to v1.82
Hello, As of yesterday, the main branch now uses version 1.82 of Boost. The main driver of the change is improved functionality in Boost.Asio. C++20 coroutines are the biggest motivator, but there are other nice things like a type-erased handler class. A minor caveat: Due to an unfixed ODR violation introduced in Boost.Phoenix 1.81 (https://github.com/boostorg/phoenix/issues/111), we disable Phoenix's tuple support, also introduced in 1.81. Hopefully the bug will be fixed in the next version. Thank you.
Hi Adam, I started seen these issues in jenkins lately: " apt-get error Unable to locate package ceph-libboost-atomic1.82-dev " is it related to the change? any idea on how to fix that? (note that I see no issues when building locally on fedora, centos and rhel) Yuval On Wed, Jul 12, 2023 at 7:57 PM Adam Emerson <aemerson@redhat.com> wrote:
Hello,
As of yesterday, the main branch now uses version 1.82 of Boost.
The main driver of the change is improved functionality in Boost.Asio. C++20 coroutines are the biggest motivator, but there are other nice things like a type-erased handler class.
A minor caveat: Due to an unfixed ODR violation introduced in Boost.Phoenix 1.81 (https://github.com/boostorg/phoenix/issues/111), we disable Phoenix's tuple support, also introduced in 1.81. Hopefully the bug will be fixed in the next version.
Thank you. _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
On Mon, Jul 17, 2023 at 4:43 PM Yuval Lifshitz <ylifshit@redhat.com> wrote:
Hi Adam, I started seen these issues in jenkins lately: " apt-get error
Unable to locate package ceph-libboost-atomic1.82-dev " is it related to the change? any idea on how to fix that? (note that I see no issues when building locally on fedora, centos and rhel)
Also raised here (I think it's the same issue): https://github.com/ceph/ceph/pull/52427#issuecomment-1638203028 Thanks, Ilya
Yuval
On Wed, Jul 12, 2023 at 7:57 PM Adam Emerson <aemerson@redhat.com> wrote:
Hello,
As of yesterday, the main branch now uses version 1.82 of Boost.
The main driver of the change is improved functionality in Boost.Asio. C++20 coroutines are the biggest motivator, but there are other nice things like a type-erased handler class.
A minor caveat: Due to an unfixed ODR violation introduced in Boost.Phoenix 1.81 (https://github.com/boostorg/phoenix/issues/111), we disable Phoenix's tuple support, also introduced in 1.81. Hopefully the bug will be fixed in the next version.
Thank you. _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
On 17/07/2023, Yuval Lifshitz wrote:
Hi Adam, I started seen these issues in jenkins lately: " apt-get error
Unable to locate package ceph-libboost-atomic1.82-dev " is it related to the change? any idea on how to fix that? (note that I see no issues when building locally on fedora, centos and rhel)
Sorry about that. It looks (based on preliminary investigation and asking people) that there's a prune job that removed the upload. I've uploaded again; please give it another shot.
thanks Adam! i don't see that issue in latest runs On Mon, Jul 17, 2023 at 11:07 PM Adam Emerson <aemerson@redhat.com> wrote:
On 17/07/2023, Yuval Lifshitz wrote:
Hi Adam, I started seen these issues in jenkins lately: " apt-get error
Unable to locate package ceph-libboost-atomic1.82-dev " is it related to the change? any idea on how to fix that? (note that I see no issues when building locally on fedora, centos and rhel)
Sorry about that. It looks (based on preliminary investigation and asking people) that there's a prune job that removed the upload.
I've uploaded again; please give it another shot.
On Wed, Jul 12, 2023 at 6:57 PM Adam Emerson <aemerson@redhat.com> wrote:
Hello,
As of yesterday, the main branch now uses version 1.82 of Boost.
The main driver of the change is improved functionality in Boost.Asio. C++20 coroutines are the biggest motivator, but there are other nice things like a type-erased handler class.
A minor caveat: Due to an unfixed ODR violation introduced in Boost.Phoenix 1.81 (https://github.com/boostorg/phoenix/issues/111), we disable Phoenix's tuple support, also introduced in 1.81. Hopefully the bug will be fixed in the next version.
Hi Adam, "make check" fails to build on pacific and quincy: common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x0): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x2): multiple definition of `boost::phoenix::placeholders::uarg2'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x2): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x4): multiple definition of `boost::phoenix::placeholders::uarg3'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x4): first defined here ... Example on pacific: https://jenkins.ceph.com/job/ceph-pull-requests/118551/ Example on quincy: https://jenkins.ceph.com/job/ceph-pull-requests/118552/ Could you please take a look? Thanks, Ilya
On 19/07/2023, Ilya Dryomov wrote:
Hi Adam,
"make check" fails to build on pacific and quincy:
common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x0): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x2): multiple definition of `boost::phoenix::placeholders::uarg2'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x2): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x4): multiple definition of `boost::phoenix::placeholders::uarg3'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x4): first defined here ...
Example on pacific: https://jenkins.ceph.com/job/ceph-pull-requests/118551/
Example on quincy: https://jenkins.ceph.com/job/ceph-pull-requests/118552/
Could you please take a look?
Yes. There are three things going on here: 1. It looks like things aren't completely cleaned up between make check runs, and stuff installed for one hangs around for others. This surprises me. 2. It looks like quincy's install-deps.sh wasn't updated when we switched to jammy, and it was just using whatever version of Boost was lying around from previous builds. When this was 1.79, that didn't cause problems. For 1.82, it does. 3. Proximately, the workaround for the Boost.Phoenix bug isn't in quincy. I have a candidate fix in https://github.com/ceph/ceph/pull/52559 Long-term, I think we should also address issue 1, the state of the bulder being overly dependendent on what was built previously.
On Thu, Jul 20, 2023 at 5:08 AM Adam Emerson <aemerson@redhat.com> wrote:
On 19/07/2023, Ilya Dryomov wrote:
Hi Adam,
"make check" fails to build on pacific and quincy:
common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x0): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x2): multiple definition of `boost::phoenix::placeholders::uarg2'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x2): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x4): multiple definition of `boost::phoenix::placeholders::uarg3'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x4): first defined here ...
Example on pacific: https://jenkins.ceph.com/job/ceph-pull-requests/118551/
Example on quincy: https://jenkins.ceph.com/job/ceph-pull-requests/118552/
Could you please take a look?
Yes.
There are three things going on here:
1. It looks like things aren't completely cleaned up between make check runs, and stuff installed for one hangs around for others. This surprises me.
2. It looks like quincy's install-deps.sh wasn't updated when we switched to jammy, and it was just using whatever version of Boost was lying around from previous builds. When this was 1.79, that didn't cause problems. For 1.82, it does.
3. Proximately, the workaround for the Boost.Phoenix bug isn't in quincy.
I have a candidate fix in https://github.com/ceph/ceph/pull/52559
Long-term, I think we should also address issue 1, the state of the bulder being overly dependendent on what was built previously.
This is what Ernesto and John have been cooperating on. Some preliminary work landed in https://github.com/ceph/ceph/pull/48697, the goal is to fully containerize everything that's running in Jenkins. John, any news on picking up https://github.com/ceph/ceph/pull/46071? Thanks, Ilya
On Thursday, July 20, 2023 5:04:36 AM EDT Ilya Dryomov wrote:
On Thu, Jul 20, 2023 at 5:08 AM Adam Emerson <aemerson@redhat.com> wrote:
On 19/07/2023, Ilya Dryomov wrote:
Hi Adam,
"make check" fails to build on pacific and quincy:
common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x0): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x2): multiple definition of `boost::phoenix::placeholders::uarg2'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x2): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x4): multiple definition of `boost::phoenix::placeholders::uarg3'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x4): first defined here ...
Example on pacific: https://jenkins.ceph.com/job/ceph-pull-requests/118551/
Example on quincy: https://jenkins.ceph.com/job/ceph-pull-requests/118552/
Could you please take a look?
Yes.
There are three things going on here:
1. It looks like things aren't completely cleaned up between make check
runs, and stuff installed for one hangs around for others. This surprises me.
2. It looks like quincy's install-deps.sh wasn't updated when we
switched to jammy, and it was just using whatever version of Boost was lying around from previous builds. When this was 1.79, that didn't cause problems. For 1.82, it does.
3. Proximately, the workaround for the Boost.Phoenix bug isn't in
quincy.
I have a candidate fix in https://github.com/ceph/ceph/pull/52559
Long-term, I think we should also address issue 1, the state of the bulder being overly dependendent on what was built previously.
This is what Ernesto and John have been cooperating on. Some preliminary work landed in https://github.com/ceph/ceph/pull/48697, the goal is to fully containerize everything that's running in Jenkins.
John, any news on picking up https://github.com/ceph/ceph/pull/46071?
Hi there, thanks for following up on this. This thread is timely because I was just talking to Adam King about how I haven't spent much time on that lately. I still occasionally update my WIP branch: https://github.com/phlogistonjohn/ ceph/tree/jjm-containerized-build - mainly when I need to use it from time to time. Unfortunately my experience with PR#48697 was a bit discouraging because it was very hard to get reviews on that and find someone willing to merge it (see: https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/ HMD6SXLYFRPEGW4GVKY46IMXO4GB373D/#RIDNTMSFY2ZB6HAOABXSX4QQ6H3ENTE4 ). It seems to me that a lot of these build scripts fall between the cracks in the Ceph team structure. The remaining changes in the branch are a mix of assorted fixes and cleanups to make things compile or run in the container env and more build scripts to make it "easy" to kick off a containerized build. I should stop being lazy and at least file PRs for some of the small fixes. However, I wonder if there is a good forum for discussion around topics like the builds scripts. I looked at the Ceph community calendar to see if there was a meeting that fit the topic. I saw a biweekly "infrastructure" meeting that I thought I might listen in on to see if it would be appropriate to discuss the build scripts and related topics. Since I'm replying in the thread now I will also raise it here: is there a good forum to have more detailed discussions wrt design and implementation of containerized builds, the builders, the scripts, etc? Thanks! -- John M. phlogistonjohn@asynchrono.us jmulligan@redhat.com
On Thu, Jul 20, 2023 at 1:44 PM John Mulligan <phlogistonjohn@asynchrono.us> wrote:
On Thursday, July 20, 2023 5:04:36 AM EDT Ilya Dryomov wrote:
On Thu, Jul 20, 2023 at 5:08 AM Adam Emerson <aemerson@redhat.com> wrote:
On 19/07/2023, Ilya Dryomov wrote:
Hi Adam,
"make check" fails to build on pacific and quincy:
common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x0): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x2): multiple definition of `boost::phoenix::placeholders::uarg2'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x2): first defined here /usr/bin/ld: CMakeFiles/common-objs.dir/mon/MonCap.cc.o:(.bss+0x4): multiple definition of `boost::phoenix::placeholders::uarg3'; common/CMakeFiles/common-common-objs.dir/ConfUtils.cc.o:(.bss+0x4): first defined here ...
Example on pacific: https://jenkins.ceph.com/job/ceph-pull-requests/118551/
Example on quincy: https://jenkins.ceph.com/job/ceph-pull-requests/118552/
Could you please take a look?
Yes.
There are three things going on here:
1. It looks like things aren't completely cleaned up between make check
runs, and stuff installed for one hangs around for others. This surprises me.
2. It looks like quincy's install-deps.sh wasn't updated when we
switched to jammy, and it was just using whatever version of Boost was lying around from previous builds. When this was 1.79, that didn't cause problems. For 1.82, it does.
3. Proximately, the workaround for the Boost.Phoenix bug isn't in
quincy.
I have a candidate fix in https://github.com/ceph/ceph/pull/52559
Long-term, I think we should also address issue 1, the state of the bulder being overly dependendent on what was built previously.
This is what Ernesto and John have been cooperating on. Some preliminary work landed in https://github.com/ceph/ceph/pull/48697, the goal is to fully containerize everything that's running in Jenkins.
John, any news on picking up https://github.com/ceph/ceph/pull/46071?
Hi there, thanks for following up on this. This thread is timely because I was just talking to Adam King about how I haven't spent much time on that lately. I still occasionally update my WIP branch: https://github.com/phlogistonjohn/ ceph/tree/jjm-containerized-build - mainly when I need to use it from time to time.
Unfortunately my experience with PR#48697 was a bit discouraging because it was very hard to get reviews on that and find someone willing to merge it (see: https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/ HMD6SXLYFRPEGW4GVKY46IMXO4GB373D/#RIDNTMSFY2ZB6HAOABXSX4QQ6H3ENTE4 ). It seems to me that a lot of these build scripts fall between the cracks in the Ceph team structure. The remaining changes in the branch are a mix of assorted
Yes, unfortunately this has been the case with David's and Kefu's departures. For help with advancing jjm-containerized-build work, I would reach out to Adam, Dan and Laura (CCed, please shout if I'm wrong and you shouldn't be on this list!).
fixes and cleanups to make things compile or run in the container env and more build scripts to make it "easy" to kick off a containerized build.
I should stop being lazy and at least file PRs for some of the small fixes. However, I wonder if there is a good forum for discussion around topics like the builds scripts. I looked at the Ceph community calendar to see if there was a meeting that fit the topic. I saw a biweekly "infrastructure" meeting that I thought I might listen in on to see if it would be appropriate to discuss the build scripts and related topics. Since I'm replying in the thread now I will also raise it here: is there a good forum to have more detailed discussions wrt design and implementation of containerized builds, the builders, the scripts, etc?
I think the "Ceph Infrastructure" meeting on Thursdays is the place. Even though this is more of a build/ops item, that meeting should be attended by the right people. Thanks, Ilya
participants (4)
-
Adam Emerson
-
Ilya Dryomov
-
John Mulligan
-
Yuval Lifshitz