Allow Ceph to compile and run on riscv64
Hi there, I'm ffgan. My GitHub account is github.com/ffgan. I'm a RISC-V enthusiast and hope to contribute to making Ceph run smoothly on riscv64 (RV64). Does Ceph have any plans to support RV64 in the near future? If so, I'd be very happy to provide relevant assistance. Please feel free to contact me anytime. I made some modifications based on Ceph v19.2.2. When building v19.2.2, there were some missing third-party dependencies like ceph-boost. A few days ago, I asked @adamemerson about his views on ceph-boost supporting RV, but didn't receive a reply (by email), so I'm posting this thread here. v19.2.2 can basically run on Ubuntu 22 RV, and here are the related modifications. https://github.com/ffgan/ceph/commits/feature/allow_riscv64_compile/ My main work was porting-related, aimed at making Ceph run on RV64. I first modified install-deps.sh - when running this script, some third-party dependencies didn't support RV, so I packaged them to make the script run on U22 RV. Then I also modified some CMakeLists.txt files and some header files to support compilation on RV. For the dashboard, there were some issues, so I disabled the related compilation and building. After completing the modifications, to conveniently demonstrate the build and test results, I set up an RV64 server and used GitHub Actions to execute the related processes step by step, connecting to the server via SSH to execute commands and get the relevant output. Here are the related CI scripts and running results. https://github.com/ffgan/ceph-riscv/actions/runs/16620228074 The results show that the execution was basically smooth. althrought some test failed. What suggestions do you have regarding these modifications, CI scripts, and test results? I noticed you use Jenkins for building and testing. For RV, there might be troublesome CI pipeline setup issues. If you need assistance with related CI pipeline setup, I'd be very happy to help. For pipeline setup, if there are no RV64 physical machines available, that's no problem - QEMU virtual machines can be used to run the related builds and tests.
FWIW, Fedora has F42 for RiscV in their alt arches. And most or all the packages that are available in regular Fedora are available for RiscV too, including Ceph. See https://riscv-koji.fedoraproject.org/koji/packageinfo?packageID=1637 CentOS also has CentOS Stream 10 for RiscV in their alt arches. Ceph packages for CentOS are built in the Storage SIG, and AFAIK there isn't a way for Storage SIG to do builds for alt arches, but I might be wrong about that. I think it would be fairly straight forward to build Ceph for RiscV. Fedora and CentOS also support aarch64 (a.k.a. arm64) and ppc64le, and Ceph packages are available for those as well. I don't speak for the Ceph project, but AFAICT they have their hands full with officially supporting Ceph on x86_64 on CentOS and Ubuntu. I don't think anyone has the bandwidth, and they certainly don't have the hardware to support other arches. On Thu, Jul 31, 2025 at 12:08 PM <sudoemt@gmail.com> wrote:
Hi there, I'm ffgan. My GitHub account is github.com/ffgan. I'm a RISC-V enthusiast and hope to contribute to making Ceph run smoothly on riscv64 (RV64).
Does Ceph have any plans to support RV64 in the near future? If so, I'd be very happy to provide relevant assistance. Please feel free to contact me anytime.
I made some modifications based on Ceph v19.2.2. When building v19.2.2, there were some missing third-party dependencies like ceph-boost. A few days ago, I asked @adamemerson about his views on ceph-boost supporting RV, but didn't receive a reply (by email), so I'm posting this thread here.
v19.2.2 can basically run on Ubuntu 22 RV, and here are the related modifications. https://github.com/ffgan/ceph/commits/feature/allow_riscv64_compile/
My main work was porting-related, aimed at making Ceph run on RV64. I first modified install-deps.sh - when running this script, some third-party dependencies didn't support RV, so I packaged them to make the script run on U22 RV. Then I also modified some CMakeLists.txt files and some header files to support compilation on RV. For the dashboard, there were some issues, so I disabled the related compilation and building. After completing the modifications, to conveniently demonstrate the build and test results, I set up an RV64 server and used GitHub Actions to execute the related processes step by step, connecting to the server via SSH to execute commands and get the relevant output. Here are the related CI scripts and running results. https://github.com/ffgan/ceph-riscv/actions/runs/16620228074 The results show that the execution was basically smooth. althrought some test failed. What suggestions do you have regarding these modifications, CI scripts, and test results?
I noticed you use Jenkins for building and testing. For RV, there might be troublesome CI pipeline setup issues. If you need assistance with related CI pipeline setup, I'd be very happy to help. For pipeline setup, if there are no RV64 physical machines available, that's no problem - QEMU virtual machines can be used to run the related builds and tests. _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
-- Kaleb
On Wednesday, July 30, 2025 9:20:59 AM Eastern Daylight Time sudoemt@gmail.com wrote:
Hi there, I'm ffgan. My GitHub account is github.com/ffgan. I'm a RISC-V enthusiast and hope to contribute to making Ceph run smoothly on riscv64 (RV64).
Does Ceph have any plans to support RV64 in the near future? If so, I'd be very happy to provide relevant assistance. Please feel free to contact me anytime.
If you're interested in making this happen, I suggested getting involved in the various sub-teams around Ceph. It's very hard for the current dev team members to remember one mailing list post where someone once volunteered on this topic, but if you're regularly present in meetings or slack or the mailing list it can make a difference.
I made some modifications based on Ceph v19.2.2. When building v19.2.2, there were some missing third-party dependencies like ceph-boost. A few days ago, I asked @adamemerson about his views on ceph-boost supporting RV, but didn't receive a reply (by email), so I'm posting this thread here.
v19.2.2 can basically run on Ubuntu 22 RV, and here are the related modifications. https://github.com/ffgan/ceph/commits/feature/allow_riscv64_compile/
I suggest trying to break up your work into as many small incremental patches & PRs as you can. Make it clear on each PR what you are trying to do - 'fix compilation error XYZ on riscv64' for example. Making small Patches and small PRs makes it easier to discuss each change with the teams responsible for each area. Note: starting with a big disorganized basket of changes is fine, I do it all the time - it's just that when it comes time to show others it helps to make each patch as small as possible.
My main work was porting-related, aimed at making Ceph run on RV64. I first modified install-deps.sh - when running this script, some third-party dependencies didn't support RV, so I packaged them to make the script run on U22 RV. Then I also modified some CMakeLists.txt files and some header files to support compilation on RV. For the dashboard, there were some issues, so I disabled the related compilation and building. After completing the modifications, to conveniently demonstrate the build and test results, I set up an RV64 server and used GitHub Actions to execute the related processes step by step, connecting to the server via SSH to execute commands and get the relevant output. Here are the related CI scripts and running results. https://github.com/ffgan/ceph-riscv/actions/runs/16620228074 The results show that the execution was basically smooth. althrought some test failed. What suggestions do you have regarding these modifications, CI scripts, and test results?
I noticed you use Jenkins for building and testing. For RV, there might be troublesome CI pipeline setup issues. If you need assistance with related CI pipeline setup, I'd be very happy to help. For pipeline setup, if there are no RV64 physical machines available, that's no problem - QEMU virtual machines can be used to run the related builds and tests.
Yes, ceph uses jenkins and it can be a pain at times. However, it's not clear to me why there'd be specific issues to riscv64, the system has arm64 builders and those work largely OK. You might consider joining the Ceph infrastructure meetings if you want to try and make riscv64 part of set of supported architectures, however that'd probably involve getting donated hardware at some point - I have no idea if you're in a position to help push that. But it's also a fine place to discuss virtualized builders (although I worry about the performance there - but I won't let myself get more sidetracked into details now). You can find this meeting on the ceph community calendar (every two weeks on Thu). https:// ceph.io/en/community/meetups/ We have also recently started a Build SIG (special interest group) recently. We discuss online in #ceph-build-sig on ceph slack. This is a good place to discuss things like CI, jenkins jobs, packaging dependencies and changes to the build scripts. Hope this was helpful!
thanks for your all reply. The following is my response to your forum post: ## 1. Are there any differences or issues with the build process between ARM64 and RV64? This is more a matter of improving the current RV64 ecosystem, not compiling Ceph itself. Rather, it's a matter of issues with the third-party dependencies required to compile Ceph under RV64. Using `install-deps.sh` as an example, when running this script on Ubuntu 22.04 RV64, there are generally two missing dependencies: issues with the deb/rpm packages and issues with the PyPI wheels. One issue is that when building `ceph-build-deps`, two packages are reported missing: `ceph-boost` and `valgrind`. I've built and tested ceph-boost on RV64, and the results are acceptable. I used a GitHub action to fully display the build and test process. For details, see https://github.com/ffgan/ceph-boost-riscv/actions/runs/16500969083/job/46659.... The final product can be found directly in the corresponding release repository. The situation with valgrind is more complicated. Simply put, valgrind 3.25.1 just added support for RV64, and Debian hasn't yet packaged it for this version. For convenience, I used the Debian packaging script from 3.24 to create the deb package. This deb package hasn't been fully tested, but it can be used to install and run `install-deps.sh` successfully. On Fedora, during the `install-depsd` command, RPM packages, including but not limited to Valgrind, were also missing. I didn't perform further build tests. The second issue was with Python dependencies. These third-party dependencies lacked support for the RV64 architecture. PyPI recently added hosting support for RV64 packages [link](https://github.com/pypi/warehouse/pull/18390), so I've been busy adding RV64 support to PyPI for packages missing from the `install-deps` command, such as maturin and rpds, and more recently, pyyaml and lxml. Currently, they don't fully support RV64, so configuring a toolchain like `Rustup` is required to successfully complete the Python installation in the `install-deps` command. Another issue arises with RV64: the Node.js version, which can lead to issues like memory out-of-bounds access and illegal instructions. These minor issues are incredibly frustrating to resolve. Despite this, I've still managed to complete a relatively complete project, from installing dependencies to building, testing, starting the cluster, and accessing the dashboard in a browser. So, overall, Ceph's support for RV64 is feasible. I think it's appropriate to start by resolving these small issues piecemeal. There's no need to make major changes to current build scripts or processes; simply gradually resolve these dependency issues, and you'll be able to run smoothly on RV64. ## 2. Split large changes into multiple smaller ones Yes, this will facilitate discussion and facilitate PR merging. I'll briefly outline the possible work process I believe is possible: 1. For now, consider only supporting Ubuntu 22 or Ubuntu 24, choosing which one to support first. 2. Consider adding RV64 support to `Ceph-Boost` and provide a RV64 deb package at chacra.ceph.com. 3. Discuss packaging issues for `Valgrind 3.25`, assisting with upstream packaging on Debian or performing test-level packaging at Ceph. 4. Add a new Jenkins pipeline for RV64 (Jenkins already supports RV64). For the Jenkins agent, I recommend using an x86 server running QEMU to create a virtual RV64 agent. 5. Modify install-deps.sh. When building for the RV64 architecture, you'll need to install the Rust toolchain and upgrade pip,and so on. 6. Change the Node.js version; 20.28.1 may be work (the issue occurred in QEMU or Docker; I can't remember exactly; it's related to the CPU instruction set). 7. Run the pipeline and check the errors and output. 8. Run the test. I've run this complete process on my own machine using QEMU and obtained the test results. Interested users can check out this link. 1 https://github.com/ffgan/ceph-riscv/actions/runs/16637799881/job/47082102410 2 https://github.com/ffgan/ceph-riscv/actions/runs/16591687390/job/46930034291 The test results are generally good. ## 3. Lack of RV64 Hardware As I mentioned above, at this stage, you can use an x86 machine to run a QEMU server, virtualize a RV64 node, and then run the Jenkins agent on it. Once the Jenkins pipeline is successfully set up and running, I'd be happy to assist in contacting RV64 open source vendors to obtain donations of RV64 hardware. This requires further communication with the Ceph community and is not the current priority. The priority is to enable Ceph to run on RV. ## 4. done Thank you again for your detailed reply
participants (4)
-
ffgan ffgan
-
John Mulligan
-
Kaleb Keithley
-
sudoemt＠gmail.com