- ceph-users - lists.ceph.io

Re: Version compatibility with QEMU/KVM
by Janne Johansson 10 Dec '25

10 Dec '25

> We've got some QEMU/KVM work load linked to librbd 14.2.21 (with > backported patches), being complicated to "just upgrade" for the time being. > > With regard to compatibility, I'm curious about cluster versions. Are > there specific features or other caveats to look out for when connecting > such work load to an Octopus, a Pacific, a Quincy or even a Reef cluster? My guess is that it probably never would be a problem to run a newer client for an RBD image made on an older cluster, since they do set feature flags on the images, and an old release would only set flags it know about, and the later versions would of course handle them, but the opposite might not be equal, so it a Reef client creates an RBD image it might ask for flags that a later Oct client doesn't understand what to do with. And as far as ceph stuff goes, RBD is (when used with mostly defaults) is quite simple and the protocol or stored data isn't changing much between releases. It does develop new features like being able to be stored on EC pools, fast diffs and things like that, but the basic read and write operations are quite the same. -- May the most significant bit of your life be positive.

2 1

[mgmt-gateway] monitoring problems, origin not allowed in grafana
by Gilles Mocellin 10 Dec '25

10 Dec '25

Hello ! Does someone manage to use mgmt-gateway, with a FQDN, and the monitoring stack working ? In my LAB, after upgrading from 10.2.3 to 20.2.0, My dashboard was working fine, with grafana dashboards. I decided to test mgmt-gateway. I use a VIP with a specific DNS name pointing at that VIP. I have a certificate generated with our internal PKI, valid for the DNS name and the VIP. Dashboard works, but no monitoring anymore. - Popups concerning alermanager : Failed to execute prometheus Error parsing Prometheus Alertmanager response: Expecting value 9/12/25 04:07 PM - No graphs in dashboard, with errors "origin not allowed" in each panel If I open Grafana directly, I see : - that the VIP is used, not my DNS name, but none works. - Grafana works but I can't see dashboards, and if I put a URL that should work (https://$VIP/grafana/d/edtb0oxdq/ceph-cluster), I have a message : Failed to load dashboard Unauthorized Some more infos : https://$DNS/grafana/dashboards => show the list of Dashboards Open a dashboard show errors 'origin not allowed" on panels https://$VIP/grafana/dashboards => cannot list the Dashboards, error Unauthorized Concerning the Altermanger errors, I see thoses XHR calls in my browser, ending in HTTP error 400 Bad Request : https://$DNS/api/prometheus?cluster_filter=true I opened an issue here : https://tracker.ceph.com/issues/74164 -- Gilles

1 0

Re: SMB on ARM
by John Mulligan 10 Dec '25

10 Dec '25

On Monday, December 8, 2025 11:31:48 AM Eastern Standard Time Robert Sander via ceph-users wrote: > Hi, > > Am 08.12.25 um 4:02 PM schrieb Daniel Brown via ceph-users: > > > > > I did try to setup SMB > > The SMB service and the SMB MGR module are far from being production > ready. I only get errors when trying to use them on AMD64. > > Regards > -- Do you have a moment to elaborate on what happened? What kind of errors did you get? Was this on a tentacle branch build or something else?

2 5

Re: Cannot list or access SMB shares as any specified user, anonymous can list shares
by Michael Sudnick 10 Dec '25

10 Dec '25

Thanks for following up and trying to reproduce the error. I've tried the commands you posted: root@ceph0:~# cephadm enter --name smb.smb.4.0.ceph0.xcbqff smbclient -d 10 -U 'media%media' //localhost/media -c ls <trimmed output> gensec_update_send: ntlmssp[0x5579a2320a20]: subreq: 0x5579a233aec0 gensec_update_send: spnego[0x5579a2324c40]: subreq: 0x5579a233b2e0 gensec_update_done: ntlmssp[0x5579a2320a20]: NT_STATUS_OK tevent_req[0x5579a233aec0/../../auth/ntlmssp/ntlmssp.c:189]: state[2] error[0 (0x0)] state[struct gensec_ntlmssp_update_state (0x5579a233b0a0)] timer[(nil)] finish[../../auth/ntlmssp/ntlmssp.c:231] gensec_update_done: spnego[0x5579a2324c40]: NT_STATUS_MORE_PROCESSING_REQUIRED tevent_req[0x5579a233b2e0/../../auth/gensec/spnego.c:1614]: state[2] error[0 (0x0)] state[struct gensec_spnego_update_state (0x5579a233b4c0)] timer[(nil)] finish[../../auth/gensec/spnego.c:2109] SPNEGO login failed: The attempted logon is invalid. This is either due to a bad username or authentication information. Freeing parametrics: session setup failed: NT_STATUS_LOGON_FAILURE root@ceph0:~# cephadm enter --name smb.smb.4.0.ceph0.xcbqff getent passwd media Inferring fsid aca834ef-5617-47fd-be18-283faba1f0b1 media:x:1000:1000::/invalid:/bin/false root@ceph0:~# cephadm enter --name smb.smb.4.0.ceph0.xcbqff pdbedit -L Inferring fsid aca834ef-5617-47fd-be18-283faba1f0b1 (no output after pbdedit, just the next prompt line) I sadly can't for the life of me figure out where to find the logs for the samba server. -Michael Sudnick Sent from Proton Mail for Android. -------- Original Message -------- On Tuesday, 12/09/25 at 14:45 John Mulligan via ceph-users <ceph-users(a)ceph.io> wrote: On Tuesday, December 9, 2025 11:45:43 AM Eastern Standard Time Michael Sudnick via ceph-users wrote: > Sorry for the wall of text. I'm getting an error when trying to access smb > shares (or even list shares as an authenticated user. An anonymous user can > list shares) from a Linux client. Tests are performed with: podman run --rm > -it quay.io/samba.org/samba-client:latest > > # smbclient -N -L 10.0.150.77 > Anonymous login successful > > Sharename Type Comment > --------- ---- ------- > IPC$ IPC IPC Service (Samba 4.23.2) > media Disk > SMB1 disabled -- no workgroup available > > However when I attempt to specify a user: > I get the following error from the client with the following command: > # smbclient -d 10 -U media%media -L 10.0.150.77 > > gensec_update_send: ntlmssp[0x55a593b774c0]: subreq: 0x55a593b72b80 > gensec_update_send: spnego[0x55a593b75800]: subreq: 0x55a593b8b1e0 > gensec_update_done: ntlmssp[0x55a593b774c0]: NT_STATUS_OK > tevent_req[0x55a593b72b80/../../auth/ntlmssp/ntlmssp.c:189]: state[2] > error[0 (0x0)] state[struct gensec_ntlmssp_up date_state (0x55a593b72d60)] > timer[(nil)] finish[../../auth/ntlmssp/ntlmssp.c:231] gensec_update_done: > spnego[0x55a593b75800]: NT_STATUS_MORE_PROCESSING_REQUIRED > tevent_req[0x55a593b8b1e0/../../auth/gensec/spnego.c:1614]: state[2] > error[0 (0x0)] state[stru ct gensec_spnego_update_state (0x55a593b8b3c0)] > timer[(nil)] finish[../../auth/gensec/spnego.c:2109] SPNEGO login failed: > The attempted logon is invalid. This is either due to a bad username or > authentication information. session setup failed: NT_STATUS_LOGON_FAILURE > Freeing parametrics: > > My cluster and share definitions are as follows. It looks like there are a > few remnants of various attempts are getting it working: # ceph smb show > { > "resources": [ > { > "resource_type": "ceph.smb.cluster", > "cluster_id": "smb", > "auth_mode": "user", > "intent": "present", > "user_group_settings": [ > { > "source_type": "resource", > "ref": "smbeskkuhxm" > } > ], > "placement": { > "count": 5 > }, > "clustering": "always", > "public_addrs": [ > { > "address": "10.0.150.77/16" > } > ] > }, > { > "resource_type": "ceph.smb.share", > "cluster_id": "smb", > "share_id": "media", > "intent": "present", > "name": "media", > "readonly": false, > "browseable": true, > "cephfs": { > "volume": "cephfs", > "path": "/", > "subvolumegroup": "smb", > "subvolume": "media", > "provider": "samba-vfs" > }, > "login_control": [ > { > "name": "media", > "category": "user", > "access": "admin" > } > ] > }, > { > "resource_type": "ceph.smb.usersgroups", > "users_groups_id": "smbeskkuhxm", > "intent": "present", > "values": { > "users": [ > { > "name": "media", > "password": "media" > } > ], > "groups": [] > }, > "linked_to_cluster": "smb" > } > ] > } > > -Michael Sudnick Thank you for trying the SMB support out! Also, thanks for providing the JSON so I know what your configuration generally looks like. I adapted the JSON to one of my own clusters (changed the IPs and placement) and deployed it. I was able to connect to the share on my cluster: smbclient -U 'media%media' //192.168.76.202/media Try "help" to get a list of possible commands. smb: \> So I'm not entirely sure what has happened on your cluster. If I had reproduced the error I would have first tried again to see if using the smbclient inside the smb container image produced the same result, and then I would have looked to see if the users got created in the container image correctly. Some example commands: (on a ceph cluster node) # cephadm enter -i smb smbclient -U 'media%media' //localhost/media -c ls # cephadm enter -i smb getent passwd media # cephadm enter -i smb pdbedit -L You could also try to redploy with smbd logging cranked way up and see if anything interesting appears in the logs. For now you can add: {"custom_smb_global_options": { "log level": "10", "_allow_customization: "i- take-responsibility-for-all-samba-configuration-errors"}} to the cluster resource json (We're going to make enabling debug logging easier in future versions FWIW) _______________________________________________ ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to ceph-users-leave(a)ceph.io

1 0

Re: Cannot list or access SMB shares as any specified user, anonymous can list shares
by John Mulligan 09 Dec '25

09 Dec '25

On Tuesday, December 9, 2025 11:45:43 AM Eastern Standard Time Michael Sudnick via ceph-users wrote: > Sorry for the wall of text. I'm getting an error when trying to access smb > shares (or even list shares as an authenticated user. An anonymous user can > list shares) from a Linux client. Tests are performed with: podman run --rm > -it quay.io/samba.org/samba-client:latest > > # smbclient -N -L 10.0.150.77 > Anonymous login successful > > Sharename Type Comment > --------- ---- ------- > IPC$ IPC IPC Service (Samba 4.23.2) > media Disk > SMB1 disabled -- no workgroup available > > However when I attempt to specify a user: > I get the following error from the client with the following command: > # smbclient -d 10 -U media%media -L 10.0.150.77 > > gensec_update_send: ntlmssp[0x55a593b774c0]: subreq: 0x55a593b72b80 > gensec_update_send: spnego[0x55a593b75800]: subreq: 0x55a593b8b1e0 > gensec_update_done: ntlmssp[0x55a593b774c0]: NT_STATUS_OK > tevent_req[0x55a593b72b80/../../auth/ntlmssp/ntlmssp.c:189]: state[2] > error[0 (0x0)] state[struct gensec_ntlmssp_up date_state (0x55a593b72d60)] > timer[(nil)] finish[../../auth/ntlmssp/ntlmssp.c:231] gensec_update_done: > spnego[0x55a593b75800]: NT_STATUS_MORE_PROCESSING_REQUIRED > tevent_req[0x55a593b8b1e0/../../auth/gensec/spnego.c:1614]: state[2] > error[0 (0x0)] state[stru ct gensec_spnego_update_state (0x55a593b8b3c0)] > timer[(nil)] finish[../../auth/gensec/spnego.c:2109] SPNEGO login failed: > The attempted logon is invalid. This is either due to a bad username or > authentication information. session setup failed: NT_STATUS_LOGON_FAILURE > Freeing parametrics: > > My cluster and share definitions are as follows. It looks like there are a > few remnants of various attempts are getting it working: # ceph smb show > { > "resources": [ > { > "resource_type": "ceph.smb.cluster", > "cluster_id": "smb", > "auth_mode": "user", > "intent": "present", > "user_group_settings": [ > { > "source_type": "resource", > "ref": "smbeskkuhxm" > } > ], > "placement": { > "count": 5 > }, > "clustering": "always", > "public_addrs": [ > { > "address": "10.0.150.77/16" > } > ] > }, > { > "resource_type": "ceph.smb.share", > "cluster_id": "smb", > "share_id": "media", > "intent": "present", > "name": "media", > "readonly": false, > "browseable": true, > "cephfs": { > "volume": "cephfs", > "path": "/", > "subvolumegroup": "smb", > "subvolume": "media", > "provider": "samba-vfs" > }, > "login_control": [ > { > "name": "media", > "category": "user", > "access": "admin" > } > ] > }, > { > "resource_type": "ceph.smb.usersgroups", > "users_groups_id": "smbeskkuhxm", > "intent": "present", > "values": { > "users": [ > { > "name": "media", > "password": "media" > } > ], > "groups": [] > }, > "linked_to_cluster": "smb" > } > ] > } > > -Michael Sudnick Thank you for trying the SMB support out! Also, thanks for providing the JSON so I know what your configuration generally looks like. I adapted the JSON to one of my own clusters (changed the IPs and placement) and deployed it. I was able to connect to the share on my cluster: smbclient -U 'media%media' //192.168.76.202/media Try "help" to get a list of possible commands. smb: \> So I'm not entirely sure what has happened on your cluster. If I had reproduced the error I would have first tried again to see if using the smbclient inside the smb container image produced the same result, and then I would have looked to see if the users got created in the container image correctly. Some example commands: (on a ceph cluster node) # cephadm enter -i smb smbclient -U 'media%media' //localhost/media -c ls # cephadm enter -i smb getent passwd media # cephadm enter -i smb pdbedit -L You could also try to redploy with smbd logging cranked way up and see if anything interesting appears in the logs. For now you can add: {"custom_smb_global_options": { "log level": "10", "_allow_customization: "i- take-responsibility-for-all-samba-configuration-errors"}} to the cluster resource json (We're going to make enabling debug logging easier in future versions FWIW)

1 0

Cannot list or access SMB shares as any specified user, anonymous can list shares
by Michael Sudnick 09 Dec '25

09 Dec '25

Sorry for the wall of text. I'm getting an error when trying to access smb shares (or even list shares as an authenticated user. An anonymous user can list shares) from a Linux client. Tests are performed with: podman run --rm -it quay.io/samba.org/samba-client:latest # smbclient -N -L 10.0.150.77 Anonymous login successful Sharename Type Comment --------- ---- ------- IPC$ IPC IPC Service (Samba 4.23.2) media Disk SMB1 disabled -- no workgroup available However when I attempt to specify a user: I get the following error from the client with the following command: # smbclient -d 10 -U media%media -L 10.0.150.77 gensec_update_send: ntlmssp[0x55a593b774c0]: subreq: 0x55a593b72b80 gensec_update_send: spnego[0x55a593b75800]: subreq: 0x55a593b8b1e0 gensec_update_done: ntlmssp[0x55a593b774c0]: NT_STATUS_OK tevent_req[0x55a593b72b80/../../auth/ntlmssp/ntlmssp.c:189]: state[2] error[0 (0x0)] state[struct gensec_ntlmssp_up date_state (0x55a593b72d60)] timer[(nil)] finish[../../auth/ntlmssp/ntlmssp.c:231] gensec_update_done: spnego[0x55a593b75800]: NT_STATUS_MORE_PROCESSING_REQUIRED tevent_req[0x55a593b8b1e0/../../auth/gensec/spnego.c:1614]: state[2] error[0 (0x0)] state[stru ct gensec_spnego_update_state (0x55a593b8b3c0)] timer[(nil)] finish[../../auth/gensec/spnego.c:2109] SPNEGO login failed: The attempted logon is invalid. This is either due to a bad username or authentication information. session setup failed: NT_STATUS_LOGON_FAILURE Freeing parametrics: My cluster and share definitions are as follows. It looks like there are a few remnants of various attempts are getting it working: # ceph smb show { "resources": [ { "resource_type": "ceph.smb.cluster", "cluster_id": "smb", "auth_mode": "user", "intent": "present", "user_group_settings": [ { "source_type": "resource", "ref": "smbeskkuhxm" } ], "placement": { "count": 5 }, "clustering": "always", "public_addrs": [ { "address": "10.0.150.77/16" } ] }, { "resource_type": "ceph.smb.share", "cluster_id": "smb", "share_id": "media", "intent": "present", "name": "media", "readonly": false, "browseable": true, "cephfs": { "volume": "cephfs", "path": "/", "subvolumegroup": "smb", "subvolume": "media", "provider": "samba-vfs" }, "login_control": [ { "name": "media", "category": "user", "access": "admin" } ] }, { "resource_type": "ceph.smb.usersgroups", "users_groups_id": "smbeskkuhxm", "intent": "present", "values": { "users": [ { "name": "media", "password": "media" } ], "groups": [] }, "linked_to_cluster": "smb" } ] } -Michael Sudnick

1 0

Version compatibility with QEMU/KVM
by Kees Meijs | Nefos 09 Dec '25

09 Dec '25

Hi list, We've got some QEMU/KVM work load linked to librbd 14.2.21 (with backported patches), being complicated to "just upgrade" for the time being. With regard to compatibility, I'm curious about cluster versions. Are there specific features or other caveats to look out for when connecting such work load to an Octopus, a Pacific, a Quincy or even a Reef cluster? Sharing real world experience would be really, really appreciated. Cheers, Kees

1 0

Re: SMB on ARM
by Anthony D'Atri 09 Dec '25

09 Dec '25

> Don’t suppose there’s a way to update that for the correct architecture? And or to document it if it isn’t going to be possible? I defer to the nonpariel JM re daemons / services available on ARM, but FWIW I've added a note to the docs: https://github.com/ceph/ceph/pull/66560/files -- aad

2 1

Re: dashboard dial tcp connection refused
by Nizamudeen A 09 Dec '25

09 Dec '25

Hi, Was that cluster ever configured with ssl before? Maybe the initial alertmanager webhook config had the ssl enabled url added to it and its now failing to detect it. Have you tried to redeploy the alertmanager to probably refresh its config? It might be worth checking the alertmanager configuration to see what urls its pointing to. Also it might help more if you can share some info about your cluster like the version it is in right now. I remember there were some bugs around the standby server and alertmanager configuration but they seem to have been fixed in some version of ceph. Regards, Nizam On Mon, Dec 8, 2025 at 5:22 PM lejeczek via ceph-users <ceph-users(a)ceph.io> wrote: > Hi guys. > > I've no _ssl_ used by _dashboard_ > > -> $ ceph mgr services > { > "dashboard": "http://10.1.1.63:8080/", > "prometheus": "http://10.1.1.63:9283/" > } > -> $ ceph config get mgr mgr/dashboard/ssl > false > > and the 'standby' servers logs: > ... > Dec 08 11:33:39 > ceph-9f4f9dba-72c7-11f0-8052-525400519d29-alertmanager-podster1[2503]: > ts=2025-12-08T11:33:39.781Z caller=dispatch.go:353 > level=error component=dispatcher msg="Notify for alerts > failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify > retry canceled after 7 attempts: Post \"<redacted>\": dial > tcp 10.1.1.63:8443: connect: connection refused; > ceph-dashboard/webhook[1]: notify retry canceled after 7 > attempts: Post \"<redacted>\": dial tcp 10.1.1.61:8443: > connect: connection refused" > Dec 08 11:33:39 > ceph-9f4f9dba-72c7-11f0-8052-525400519d29-alertmanager-podster1[2503]: > ts=2025-12-08T11:33:39.781Z caller=notify.go:848 level=warn > component=dispatcher receiver=ceph-dashboard > integration=webhook[0] > aggrGroup="{}/{}:{alertname=\"CephHealthWarning\"}" > msg="Notify attempt failed, will retry later" attempts=1 > err="Post \"<redacted>\": dial tcp 10.1.1.63:8443: connect: > connection refused" > ... > with the snippet of the logs above _ceph_ says only: > -> $ ceph health detail > HEALTH_WARN mon podster1 is low on available space > [WRN] MON_DISK_LOW: mon podster1 is low on available space > mon.podster1 has 12% avail > > nothing listens on 8443 anywhere, which is as it should be - > why standby server wants that - would anybody know? > 10.1.1.61 is standby's own IP. > > many thanks, L. > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io > -- Nizamudeen A Sr. Software Engineer - IBM Partner Engineer IBM and Red Hat Ceph Storage Red Hat <https://www.redhat.com/> <https://www.redhat.com/>

1 0

Size or IOPS for DB+WAL?
by Andrew Klaassen 09 Dec '25

09 Dec '25

Hi, I'm looking at converting an old Nutanix+VMware cluster to Ceph+Proxmox. It'll host a mix of a couple of hundred VMs - web servers, Active Directory, Elasticsearch clusters, postgres and mysql databases, software deployment servers, log forwarders, some stuff that I don't even know what it does - basically a grab bag of workloads. It'll be a 9 node cluster. CPU+RAM is a mix from 12 core Intel Haswell with 512GB to 18 core Intel Ice Lake with 1TB. NICs will be two dual-port 10GbE (total four ports) per node. Disks per node will be a mix of: HDD: 2x12TB + 1x8TB SSD: 2x3.84TB HDD: 1x12TB + 2x6TB + 1x8TB SSD: 1x7.68TB This will leave me with one slot in each node for a DB+WAL drive. Is either of these a clear winner? Smaller and faster, or bigger and slower? Micron 7450 MTFDKCB960TFR - 960GB - 85K random write IOPS - 2% of raw storage KIOXIA KCD6XLUL3T84 - 3.84TB - 60K random write IOPS - 8% of raw storage If there is a clear winner, are there any setup or tuning options I should know about to take full advantage of them? Thanks. Andrew

1 0