4 Jan
2024
4 Jan
'24
6:39 a.m.
Is it possible to use Ceph as a root filesystem for a pxe booted host?
Thanks
4 Jan
4 Jan
12:37 p.m.
Hello,
Jeremy Hansen wrote:
: Is it possible to use Ceph as a root filesystem for a pxe booted host?
I am not sure about CephFS, but as for Ceph RBD, it should definitely
be possible to get an IP address and run "rbd map" from the initrd,
and then use /dev/rbd0 as a root device.
-Yenya
--
| Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}>
|
| https://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 |
We all agree on the necessity of compromise. We just can't agree on
when it's necessary to compromise. --Larry Wall
7:33 p.m.
A while ago I had done this with iSCSI, using iBFT, the advantage being the
NIC itself supports such boot, the disadvantage is you need to use iSCSI.
I recall PXE can bring an image over TFTP, so assuming you want to load
that image into an RBD device for persistence? Looks like this was
discussed here: https://www.spinics.net/lists/ceph-users/msg66370.html
--
Alex Gorbachev
iss-integration.com
On Wed, Jan 3, 2024 at 10:10 PM Jeremy Hansen <jeremy(a)skidrow.la> wrote:
Is it possible to use Ceph as a root filesystem for a
pxe booted host?
Thanks
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
7:46 p.m.
We also had a look at this a few years ago when we flashed almost all
servers to boot iPXE from the nic directly. The idea was as follows:
iPXE -> http -> get kernel + initramfs -> mount rootfs
Our idea was to use RBD as a disk for the server, but to the last of my
knowledge, there is no support in the linux kernel for something like
linux root=ceph://mon1,mon2,mon3,key=keytouse/rbd/pool-x/image-y
Which would be needed for a ceph based rootfs.
It would be amazing to have that though, a decentralised storage system
that is able to be booted from. Above description would actually be more
suitable to the requested cephfs booting, as NFS root also often splits
the kernel+initramfs.
The RBD case would potentially require boot loader support, if we assume
that the RBD image contains something like grub - so a tiny bit more
complicated.
BR and wishing you all the best for this approach,
Nico
Alex Gorbachev <ag(a)iss-integration.com> writes:
A while ago I had done this with iSCSI, using iBFT,
the advantage being the
NIC itself supports such boot, the disadvantage is you need to use iSCSI.
I recall PXE can bring an image over TFTP, so assuming you want to load
that image into an RBD device for persistence? Looks like this was
discussed here: https://www.spinics.net/lists/ceph-users/msg66370.html
--
Sustainable and modern Infrastructures by ungleich.ch
6 Feb
6 Feb
9:02 p.m.
My €0.02 for what it's worth(less).
I've been doing RBD-based VMs under libvirt with no problem. In that
particular case, the ceph RBD base images are being overlaid cloud-
style with a an instance-specific qcow2 image and the RBD is just part
of my storage pools.
For a physical machine, I'd probably be content to use the Ceph NFS
share for the OS and PXE boot itself. I could always mount a ceph
filesystem once the OS was up if I wanted to.
If I'm not mistaken, getting ceph to boot directly would require a
custom ramdisk with ceph driver installed, whereas an NFS-supporting
ramdisk is pretty much off-the-shelf. Since ceph-ganesha NFS seems
pretty reliable and performant, I'd be happy with that.
Tim
On Thu, 2024-01-04 at 17:16 +0100, Nico Schottelius wrote:
We also had a look at this a few years ago when we flashed almost all
servers to boot iPXE from the nic directly. The idea was as follows:
iPXE -> http -> get kernel + initramfs -> mount rootfs
Our idea was to use RBD as a disk for the server, but to the last of
my
knowledge, there is no support in the linux kernel for something like
linux root=ceph://mon1,mon2,mon3,key=keytouse/rbd/pool-x/image-y
Which would be needed for a ceph based rootfs.
It would be amazing to have that though, a decentralised storage
system
that is able to be booted from. Above description would actually be
more
suitable to the requested cephfs booting, as NFS root also often
splits
the kernel+initramfs.
The RBD case would potentially require boot loader support, if we
assume
that the RBD image contains something like grub - so a tiny bit more
complicated.
BR and wishing you all the best for this approach,
Nico
Alex Gorbachev <ag(a)iss-integration.com> writes:
A while ago I had done this with iSCSI, using
iBFT, the advantage
being the
NIC itself supports such boot, the disadvantage is you need to use
iSCSI.
I recall PXE can bring an image over TFTP, so assuming you want to
load
that image into an RBD device for persistence? Looks like this was
discussed here:
https://www.spinics.net/lists/ceph-users/msg66370.html
--
Sustainable and modern Infrastructures by ungleich.ch
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
104
days inactive
137
days old
4 comments
5 participants
participants (5)
-
Alex Gorbachev -
Jan Kasprzak -
Jeremy Hansen -
Nico Schottelius -
Tim Holloway