- ceph-users - lists.ceph.io

2024-11-21 Perf meeting cancelled!
by Matt Vandermeulen 21 Nov '24

21 Nov '24

Hi folks, the perf meeting will be cancelled today, Mark is flying from a conference! Thanks, Matt

1 0

Lifetime for ceph
by Steve Brasier 21 Nov '24

21 Nov '24

The octopus repos disappeared a couple of days ago - no argument with that given its marked as out of support. However I see from https://docs.ceph.com/en/latest/releases/ that quincy is also marked as out of support, but currently the repos are still there. Is there any guesstimate of when the quincy repos might disappear please? many thanks Steve Brasier http://stackhpc.com/ Please note I work Tuesday to Friday.

1 0

CephFS subvolumes not inheriting ephemeral distributed pin
by Rajmohan Ramamoorthy 21 Nov '24

21 Nov '24

CephFS subvolumes not inheriting ephemeral distributed pin configured on the subvolumegroups. References: https://github.com/ceph/ceph/pull/35759 https://github.com/ceph/ceph/pull/43896 # Environmet setup Ceph version: 18.2.4 and 19.2.0 CephFS with multiple MDS (2 active + 1 standby MDS) RHEL 9.4 Linux myserver 5.14.0-427.37.1.el9_4.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Sep 13 12:41:50 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux # Ceph FS configuration ``` ceph config set mds mds_export_ephemeral_distributed true ceph fs set midline-a balance_automate false ``` # Subvolume group (csi) creation ``` ceph fs subvolumegroup create midline-a csi ``` # Subvolume group (csi) pinning (tried both methods) ``` ceph fs subvolumegroup pin midline-a csi distributed 1 setfattr -n ceph.dir.pin.distributed -v 1 setfattr -n ceph.dir.pin.distributed -v 1 /mnt/cephfs/volumes/csi/ ``` # Subvolume creation ``` ceph fs subvolume create midline-a subvol1 csi ceph fs subvolume create midline-a subvol2 csi ``` # Get attribute (subvolume group) ``` getfattr -n ceph.dir.pin.distributed /mnt/cephfs/volumes/csi getfattr: Removing leading '/' from absolute path names # file: mnt/cephfs/volumes/csi ceph.dir.pin.distributed="1" ``` # Get attribute (subvolume) - Subvolumes not inheriting the distributed pin attribute ``` getfattr -n ceph.dir.pin.distributed /mnt/cephfs/volumes/csi/subvol1 getfattr: Removing leading '/' from absolute path names # file: mnt/cephfs/volumes/csi/subvol1 ceph.dir.pin.distributed="0" ``` ``` getfattr -n ceph.dir.pin.distributed /mnt/cephfs/volumes/csi/subvol2 getfattr: Removing leading '/' from absolute path names # file: mnt/cephfs/volumes/csi/subvol2 ceph.dir.pin.distributed="0" ``` # MDS logs (subtree information) ``` ceph tell mds.abc123 get subtrees | jq '.[] | [.dir.path, .auth_first, .export_pin, .export_pin_target]' [ "~mds1", 1, -1, -1 ] [ "", 0, -1, -1 ] [ "/volumes/csi", 1, -1, 1 ] ``` ``` ceph tell mds.xyz123 get subtrees | jq '.[] | [.dir.path, .auth_first, .export_pin, .export_pin_target]' [ "/volumes/csi", 1, -1, 1 ] [ "", 0, -1, -1 ] [ "~mds0", 0, -1, -1 ] [ "/volumes/csi", 0, -1, 0 ] ``` Regards, Rajmohan R

2 5

Squid: regression in rgw multisite replication from Quincy/Reef clusters
by Casey Bodley 20 Nov '24

20 Nov '24

Recent multisite testing has uncovered a regression on Squid that happens when secondary zones are upgraded to Squid before the metadata master zone. User metadata replicates incorrectly in this configuration, such that their access keys are "inactive". As a result, these users are denied access to API requests on that secondary zone. This is tracked in https://tracker.ceph.com/issues/68985, and we'll prioritize the backport for 19.2.1. In the meantime, we strongly recommend upgrading the metadata master zone before others.

1 0

Join us for today's User + Developer Monthly Meetup!
by Laura Flores 20 Nov '24

20 Nov '24

Hi all, Please join us for today's User + Developer Monthly Meetup at 10:00 AM ET. RSVP here! https://www.meetup.com/ceph-user-group/events/304636936 Thanks, Laura Flores -- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores(a)ibm.com | lflores(a)redhat.com <lflores(a)redhat.com> M: +17087388804

1 0

CephFS maximum filename length
by Naumann, Thomas 20 Nov '24

20 Nov '24

Hi at all, we use proxmoxcluster (v8.2.8) with ceph (v18.2.4) an EC-pools (all ceph options on default). One pool is exported with CephFS as backend storage for nextcloud servers. At the moment there is data migration from old S3 storage to CephFS pool. There are many files with huge filename length. In Wikipedia (see: https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits) there is CephFS max filename length to 255 characters and no limit for pathname length. Is this true? What is maximum filename length for CephFS (reef)? Which CephFS options could be altered to increase filnename length? What is the best practice to handle files with too long filename? Has anyone experience about this setting? best regards -- Thomas Naumann

1 0

Centos 9 updates break Reef MGR
by Chris Palmer 19 Nov '24

19 Nov '24

I've just applied routine Centos 9 updates to one node of a Reef 18.2.4 system (package install). They include some python3 updates that break the MGR in at least two ways. When the MGR starts in standby, it immediately logs the following two crashes. Only the dashboard and restful services are enabled (prometheus is not enabled). We have mgr/dashboard/FEATURE_TOGGLE_DASHBOARD = false. { "archived": "2024-11-19 10:30:46.031489", "backtrace": [ " File \"/usr/share/ceph/mgr/dashboard/__init__.py\", line 60, in <module>\n from .module import Module, StandbyModule # noqa: F401", " File \"/usr/share/ceph/mgr/dashboard/module.py\", line 51, in <module>\n patch_cherrypy(cherrypy.__version__)", " File \"/usr/share/ceph/mgr/dashboard/cherrypy_backports.py\", line 197, in patch_cherrypy\n accept_exceptions_from_builtin_ssl(ver)", " File \"/usr/share/ceph/mgr/dashboard/cherrypy_backports.py\", line 113, in accept_exceptions_from_builtin_ssl\n patch_builtin_ssl_wrap(v, accept_ssl_errors)", " File \"/usr/share/ceph/mgr/dashboard/cherrypy_backports.py\", line 75, in patch_builtin_ssl_wrap\n from cherrypy.wsgiserver.ssl_builtin import BuiltinSSLAdapter as builtin_ssl", "ModuleNotFoundError: No module named 'cherrypy.wsgiserver'" ], "ceph_version": "18.2.4", "crash_id": "2024-11-19T09:19:40.427015Z_f96f4d70-2112-47b5-96f7-5e9bd463e8eb", "entity_name": "mgr.ceph1", "mgr_module": "dashboard", "mgr_module_caller": "PyModule::load_subclass_of", "mgr_python_exception": "ModuleNotFoundError", "os_id": "centos", "os_name": "CentOS Stream", "os_version": "9", "os_version_id": "9", "process_name": "ceph-mgr", "stack_sig": "9c98ebf1b6831bfca2823f54c9e6be01306090c2a7749def2ed8a15167fc527a", "timestamp": "2024-11-19T09:19:40.427015Z", "utsname_hostname": "ceph1.xxxxxx", "utsname_machine": "x86_64", "utsname_release": "6.1.112-1.el9.elrepo.x86_64", "utsname_sysname": "Linux", "utsname_version": "#1 SMP PREEMPT_DYNAMIC Mon Sep 30 13:59:36 EDT 2024" } { "archived": "2024-11-19 10:30:46.056007", "backtrace": [ " File \"/usr/share/ceph/mgr/prometheus/__init__.py\", line 2, in <module>\n from .module import Module, StandbyModule", " File \"/usr/share/ceph/mgr/prometheus/module.py\", line 38, in <module>\n v = Version(cherrypy.__version__)", " File \"/lib/python3.9/site-packages/pkg_resources/_vendor/packaging/version.py\", line 277, in __init__\n raise InvalidVersion(\"Invalid version: '{0}'\".format(version))", "pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: 'unknown'" ], "ceph_version": "18.2.4", "crash_id": "2024-11-19T09:19:42.188291Z_c112de10-cdd5-4ed3-86b5-04dffd660cb8", "entity_name": "mgr.ceph1", "mgr_module": "prometheus", "mgr_module_caller": "PyModule::load_subclass_of", "mgr_python_exception": "InvalidVersion", "os_id": "centos", "os_name": "CentOS Stream", "os_version": "9", "os_version_id": "9", "process_name": "ceph-mgr", "stack_sig": "7fb0c6c17573887e8772c311aac1f3547c8366e50d6f2f8d8bd38deb6e0e9405", "timestamp": "2024-11-19T09:19:42.188291Z", "utsname_hostname": "ceph1.xxxxxx", "utsname_machine": "x86_64", "utsname_release": "6.1.112-1.el9.elrepo.x86_64", "utsname_sysname": "Linux", "utsname_version": "#1 SMP PREEMPT_DYNAMIC Mon Sep 30 13:59:36 EDT 2024" } The MGR can be made active, but attempts to use the dashboard generate more of these. The culprit turns out to be python3-jaraco-text from the epel repository, which upgraded from python3-jaraco-text-3.2.0-6.el9.noarch to python3-jaraco-text-4.0.0-2.el9.noarch. (It also installed python3-jaraco-context-6.0.1-3.el9.noarch as a new dependency). Reverting to python3-jaraco-text-3.2.0 has avoided the problem for now, but that's not a sustainable long-term fix.

2 3

Re: Question about speeding hdd based cluster
by quaglio＠bol.com.br 19 Nov '24

19 Nov '24

1 2

Something like RAID0 with Ceph
by Christoph Pleger 19 Nov '24

19 Nov '24

Hello, Is it possible to have something like RAID0 with Ceph? That is, when the cluster configuration file contains osd pool default size = 4 and I have four hosts with one osd drive (all the same size, let's call it size s) per host, is it somehow possible to add four other hosts with one osd drive (again with size s) per host, so that the resulting Ceph block device is of size 2 * s? Regards Christoph

3 2

What is the Best stable option for production env in Q4/24 Quincy or Reef?
by Özkan Göksu 18 Nov '24

18 Nov '24

Hello! I always track Ceph releases 1-2 versions behind and I used Quincy for all my deployments this year. I’m planning to deploy a new cluster on Ubuntu 22.04 and would like to know if Reef v18.2.4 is stable and ready for production environments in 2025. My use case involves RBD, CephFS, and RGW with multisite S3. Would upgrading to Reef provide significant benefits over Quincy and is it safe & stable enough? Looking forward to any insights and recommendations. Best regards,

1 0