Dear Ceph Community, Maybe you can help me with this I have inconsistencies with the number of subvolumes in Cephfs in my Ceph Cluster and the number of Persistent Volumes of StorageClass Cephfs in my K8S cluster I have K8S 1.25 version on my kubernetes cluster and use the Ceph CSI CephFs 3.8.1, my ceph cluster is 16.2.15 pacific release. I have 35 Persistent Volumes of type CephFS $ kubectl get pv -o custom-columns=name:.metadata.name,subvolume:.spec.csi.volumeAttributes.subv olumeName,storageclass:.spec.storageClassName | grep cephfs pvc-0062e845-4a87-4a32-8f19-cbfff3b2789d csi-vol-919a2302-f964-11ed-b5e0-b615b8ae8847 csi-cephfs-sc pvc-0298fc13-bc5f-489a-b88e-64d7cdc61f1e csi-vol-7ce2f152-d96c-4f54-b335-9ede672ce320 csi-cephfs-sc pvc-09991b1e-b9f6-4a7c-b710-25131433c6e9 csi-vol-ef2e7be0-8044-11ec-b79c-fa4d817fb9f0 csi-cephfs-sc pvc-19b8f683-b3e8-4476-abc3-9d7d7e4d8941 csi-vol-5a9ce4b8-1edb-11ed-8f9a-e6ec00c2f5e5 csi-cephfs-sc pvc-2fc90e06-22b4-4e80-b33a-c4c5dd6a0baa csi-vol-760f34d1-4e26-11ed-af16-d67e0f01e63a csi-cephfs-sc pvc-316aa495-3d52-45a5-aac3-4f64bf598aa3 csi-vol-9d135b18-8b43-402b-a606-a2c426527b20 csi-cephfs-sc pvc-342b66f9-f6d5-413c-8634-e51a25d638a8 csi-vol-7e3aca09-8d76-11eb-95b9-ee71e85dde57 csi-cephfs-sc pvc-48c664a6-073e-4498-9370-d4a718f11729 csi-vol-72d71655-9518-11ee-a81d-b2c17fca41c9 csi-cephfs-sc pvc-5349d4ec-7e62-48b7-bd8c-8f718a13906b csi-vol-9748018f-4df9-11ed-af16-d67e0f01e63a csi-cephfs-sc pvc-5db458a4-2dfb-46d7-8b23-2bde45a9bf46 csi-vol-bd87eade-e726-4a5d-993c-56dbece42420 csi-cephfs-sc pvc-5fd1f3a5-d120-4283-8e2c-b6e1e3e4cbd1 csi-vol-e7ec3159-9d1c-11eb-82b9-aaa0cbaeebfa csi-cephfs-sc pvc-669c35f9-9927-4e44-94d2-a9ddd0eda914 csi-vol-ef9539f6-9c5a-40d8-a89b-087ad7c6e9b3 csi-cephfs-sc pvc-6909652a-2da6-4a13-a111-59a21ed685b2 csi-vol-a9a1fff7-550a-43a1-aa52-4f6710e1ca9d csi-cephfs-sc pvc-72c2f70e-2976-4188-8d1a-f2928745097b csi-vol-51045501-f17b-4f4d-af08-100ea3db3db3 csi-cephfs-sc pvc-73b1d3f3-f861-44d7-bbd0-43fc16f9e7a9 csi-vol-439769d7-9b94-4350-bfa2-fe5e9a182650 csi-cephfs-sc pvc-76b04010-1e00-4fea-82b7-855ff83ab820 csi-vol-28852a9b-79c0-4eb7-afcf-b443ff9f867d csi-cephfs-sc pvc-7e0b08ea-9d31-45a7-a098-588563c18f19 csi-vol-4a5086d9-62ad-11ee-80a7-323a25995550 csi-cephfs-sc pvc-83853b0b-ab55-46be-ae3a-78f8d431e780 csi-vol-ef2e919c-8044-11ec-b79c-fa4d817fb9f0 csi-cephfs-sc pvc-89c85899-c26a-4efd-91dd-cc891d303ad6 csi-vol-e8b89f50-fdf9-11ec-8f9a-e6ec00c2f5e5 csi-cephfs-sc pvc-8a4c12b1-b7d8-4337-b218-7de2bd6eadde csi-vol-718d31a6-9b4a-11ee-a81d-b2c17fca41c9 csi-cephfs-sc pvc-8b56ae01-1809-4552-9192-930bd1db95f8 csi-vol-a72f2d4c-1f8b-4a71-a846-2b7a6febbf82 csi-cephfs-sc pvc-97b80840-fd60-436a-8223-5cf41bfe56ef csi-vol-0289ed46-4df6-11ed-af16-d67e0f01e63a csi-cephfs-sc pvc-984eadce-c908-4e21-a558-70d12bcff3c2 csi-vol-ef2e8a6b-8044-11ec-b79c-fa4d817fb9f0 csi-cephfs-sc pvc-9de21d9d-80ae-49f8-bea0-2a265945cfa6 csi-vol-c7b4f302-4d71-4777-b6c0-82720ebe8950 csi-cephfs-sc pvc-9f5651b5-0cb2-47d2-889e-69b09d96fe36 csi-vol-9ba650df-3fa5-43f0-abea-5231643fb1bf csi-cephfs-sc pvc-a2607a68-62a9-47a0-8f15-871359460556 csi-vol-028a40c4-4df6-11ed-af16-d67e0f01e63a csi-cephfs-sc pvc-a84158f8-1d21-475a-9987-a0e8a3347c42 csi-vol-22835b9b-c31c-46c6-aec2-67b554c683d3 csi-cephfs-sc pvc-b3d81d90-eaec-4d8a-8065-790d9c0e6ea7 csi-vol-c908a216-cf78-11eb-a202-52b7fddc54da csi-cephfs-sc pvc-bcc5979b-f9cb-4c10-ab64-24424749856b csi-vol-e8dd361e-b6fb-4a19-94af-7cae07d4702c csi-cephfs-sc pvc-bd09916e-a47d-429f-9371-ac779d3959b3 csi-vol-cc9573bc-6733-11ed-af16-d67e0f01e63a csi-cephfs-sc pvc-dac9a949-cd28-4448-9c66-7bab365c3d25 csi-vol-21b0d822-6fc3-11ec-96cb-dec0a2aa44f8 csi-cephfs-sc pvc-e0783e40-f49c-4d7c-a404-4554ce35341c csi-vol-0289e975-4df6-11ed-af16-d67e0f01e63a csi-cephfs-sc pvc-f660eb28-2af4-45de-b00b-72f73e08745d csi-vol-4b6e4fd4-9e4e-4b0d-a022-305b8e1d8e84 csi-cephfs-sc pvc-fd994df6-9ca5-47d5-af64-afec037fd5d1 csi-vol-ef2e42e3-8044-11ec-b79c-fa4d817fb9f0 csi-cephfs-sc pvc-fe72f0a2-c260-4a40-acca-50f47d31c11a csi-vol-6c0efe87-d02c-11eb-a202-52b7fddc54da csi-cephfs-sc I have 985 subvolumes in Cephfs csi volume , i have no idea on why i have this difference. $ ceph fs subvolume ls cephfs csi | awk '{if ($2 != "") print $2}' | sed 's/"//g' | wc -l 1020 If i check the number of Volume Snapshots of StorageClass Cephfs via the command "kubectl get volumesnapshot -A | grep cephfs | wc -l" i have 177 If i check from the subvolumes of Cephfs that are mapped to the K8S PV i have 178 via the command "for subvolume in $(cat list.txt | xargs);do ceph fs subvolume snapshot ls cephfs $subvolume csi | awk '{if ($2 != "") print $2}' | sed 's/"//g' ;done | wc -l" I have 178 Which seems to be correct and consistent. Are my checks valid to identifies subvolumes that are not anymore used by my K8S workload ? Can i safely delete the subvolumes as they seems not associated with a persistent volume in Kubernetes ? Thanks in advance for the help Have a nice day Best Regards, Edouard Fazenda. Edouard FAZENDA Technical Support <https://www.csti.ch/> www.csti.ch
