Hi, I need to create manually few VM with KVM. I would like to know if they are any difference between using a libvirt module and kernel module to access a ceph cluster. Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 12 févr. 2025 16:14:15 CET
I don't really understand what you mean with libvirt module. afaik this how you add add disk <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='xxxxx.xxx'> <secret type='ceph' uuid='b924bdbb-5462-4e9d-8b1e-2ad1cbed7f5b'/> </auth> <source protocol='rbd' name='rbd/aaaaaa> <host name='192.168.x.x' port='6789'/> <host name='192.168.x.x' port='6789'/> <host name='192.168.x.x' port='6789'/> </source> <target dev='sda' bus='scsi'/> <boot order='2'/> <alias name='scsi0-0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk>
I need to create manually few VM with KVM. I would like to know if they are any difference between using a libvirt module and kernel module to access a ceph cluster.
Regards
-- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 12 févr. 2025 16:14:15 CET _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi, I'm guessing you are deciding between librbd and krbd. I personally use krbd as in my original tests it was a bit faster. I think there are some cases where librbd is faster, but I don't remember those edge cases off the top of my head. That's my two cents. On Wed, Feb 12, 2025, 17:28 Albert Shih <Albert.Shih@obspm.fr> wrote:
Hi,
I need to create manually few VM with KVM. I would like to know if they are any difference between using a libvirt module and kernel module to access a ceph cluster.
Regards
-- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 12 févr. 2025 16:14:15 CET _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Le 12/02/2025 à 15:35:53+0000, Marc a écrit Hi,
I don't really understand what you mean with libvirt module.
It's exactly that.
afaik this how you add add disk
<disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='xxxxx.xxx'> <secret type='ceph' uuid='b924bdbb-5462-4e9d-8b1e-2ad1cbed7f5b'/> </auth> <source protocol='rbd' name='rbd/aaaaaa> <host name='192.168.x.x' port='6789'/> <host name='192.168.x.x' port='6789'/> <host name='192.168.x.x' port='6789'/> </source> <target dev='sda' bus='scsi'/> <boot order='2'/> <alias name='scsi0-0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk>
but I also can mount the disk directly with /etc/ceph/rbdmap at the boot the disk will appear somewhere in /dev/sd* on the kvm server and then use it in kvm as a «normal» disk. Don't know if they are any difference or just a preference. Regards. -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 12 févr. 2025 16:47:37 CET
The kernel code and Ceph proper are separate codebases, so if you’re running an old enough kernel KRBD may lack certain features, including compatibility with pg-upmap. I’m not sure about compatibility with RADOS / RBD namespaces. Last I knew KRBD did not have support for RBD QoS, so if you need to throttle client ops for noisy neighbor reasons, that would favor librbd, as would having a super old kernel, like on CentOS 7.
Hi,
I need to create manually few VM with KVM. I would like to know if they are any difference between using a libvirt module and kernel module to access a ceph cluster.
Regards
-- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 12 févr. 2025 16:14:15 CET _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On 12-02-2025 16:49, Curt wrote:
Hi,
I'm guessing you are deciding between librbd and krbd. I personally use krbd as in my original tests it was a bit faster. I think there are some cases where librbd is faster, but I don't remember those edge cases off the top of my head.
That's my two cents.
You might find this benchmark made by 42on interesting: https://www.youtube.com/watch?v=iVCPjNZa7N0 Gr. Stefan
but I also can mount the disk directly with
/etc/ceph/rbdmap
at the boot the disk will appear somewhere in /dev/sd* on the kvm server and then use it in kvm as a «normal» disk. Don't know if they are any difference or just a preference.
If you mount with KRBD and the ceph cluster acts up, the KVM host might stop serving IO, which would affect lots more than if KVM uses librbd to mount it for the guest only, in the second case, the host and possibly other guests might run on undisturbed. -- May the most significant bit of your life be positive.
participants (6)
-
Albert Shih
-
Anthony D'Atri
-
Curt
-
Janne Johansson
-
Marc
-
Stefan Kooman