Hi, can you share the auth caps for your k8s client? ceph auth get client.<K8S_CLIENT> And maybe share the yaml files as well (redact sensitive data) so we can get a full picture. Zitat von Kushagr Gupta <kushagrguptasps.mun@gmail.com>:
Hi Team,
Components: Kubernetes, Ceph
Problem statement: We are trying to integrate Ceph with kubernetes. We are unable to utilize the block volume mode in the pod.
Description: OS: Almalinux 8.8 Ceph version: 18.2 Kubernetes version: 1.28.2
We have deployed a single node kubernetes cluster and a 3 node ceph cluster. We are trying to integrate kubernetes with ceph using csi-rbd plugin. We have used the following link to do the same: https://docs.ceph.com/en/reef/rbd/rbd-kubernetes/
We are strugling with the usage of the pvc we created in the block mode. Kindly refer the "raw-block-pvc-4.yaml". We created a pod using the file "raw-block-pod-4.yaml".
The pod was created successfully. The device path we used was /dev/xvda. We are unable to use this path inside the pod. Its not gettting shown in the lsblk of the pod and its not even shown in the df -kh of the pod. When we try to mount /dev/xvda on /tmp but we get permission denied error. We get a similar error when we try to make partiotion of the disk. The partition gets created but partiotion table is not updated. At the same time we are able to format the disk using the command mkfs.ext4 /dev/xvda. Kindly refer the output: " [root@kube-cluster-1 ceph_provisioner]# k exec -it pod-with-raw-block-volume-2 sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. sh-4.4# hostname pod-with-raw-block-volume-2 sh-4.4# df -kh Filesystem Size Used Avail Use% Mounted on overlay 70G 14G 57G 19% / tmpfs 64M 0 64M 0% /dev tmpfs 32G 0 32G 0% /sys/fs/cgroup /dev/mapper/almalinux-root 70G 14G 57G 19% /etc/hosts shm 64M 0 64M 0% /dev/shm tmpfs 63G 12K 63G 1% /run/secrets/ kubernetes.io/serviceaccount tmpfs 32G 0 32G 0% /proc/acpi tmpfs 32G 0 32G 0% /proc/scsi tmpfs 32G 0 32G 0% /sys/firmware sh-4.4# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 1G 0 loop loop1 7:1 0 500G 0 loop sda 8:0 0 446.6G 0 disk |-sda1 8:1 200M 0 part |-sda2 8:2 1M 0 part |-sda3 8:3 1G 0 part `-sda4 8:4 0 445.4G 0 part sdb 8:16 0 893.3G 0 disk rbd0 252:0 0 1G 0 disk rbd1 252:16 0 500G 0 disk rbd2 252:32 0 100G 0 disk sh-4.4# ls /dev core fd full mqueue null ptmx pts random shm stderr stdin stdout termination-log tty urandom xvda zero sh-4.4# "
Altough we are able to see a volume corresponding to the PV. But we are not able to use the block device.
Could anyone help me out.
Thanks and Regards, Kushagra Gupta _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io