- ceph-users - lists.ceph.io

Can I convert thick mode files to thin mode files in cephfs?
by 苏察哈尔灿 18 Feb '25

18 Feb '25

Does cephfs not support thin provisioning disk formats like vmware virtual machine disks? For example, the same vmware disk file in the rbd block is: root@backup-server:/mnt# du rbd/Win2003-002-198-bak/Win2003-002-198-bak_5-flat.vmdk -h 1.9G    rbd/Win2003-002-198-bak/ win2003-002-198-bak_5-flat-.vmdk After copying to cephfs: root@backup-server:/mnt# du cephfs/Win2003-002-198-bak_5-flat.vmdk -h 30G     cephfs/Win2003-002-198-bak_5-flat.vmdk The capacity was increased to a thick disk format. Is there any way to change this?  Thank you!

2 1

RBD Performance issue
by vignesh varma 17 Feb '25

17 Feb '25

Hi Team, I have set up a 2 Ceph cluster 3-node each cluster with a two-way RBD mirror. In this setup, Ceph 1 is configured two-way mirror to Ceph 2, and vice versa. The RBD pools are integrated with CloudStack. The Ceph cluster uses NVMe drives, but I am experiencing very low IOPS performance. I have provided the relevant details below. Could you please guide me on how to optimize the setup to achieve higher IOPS? fio --ioengine=libaio --direct=1 --randrepeat=1 --refill_buffers --end_fsync=1 --rwmixread=70 --filename=/root/ceph-rbd --name=write --size=1024m --bs=4k --rw=readwrite --iodepth=32 --numjobs=16 --group_reporting read: IOPS=92.5k, BW=361MiB/s (379MB/s)(11.2GiB/31718msec) write: IOPS=39.7k, BW=155MiB/s (163MB/s)(4922MiB/31718msec); 0 zone resets Hardware Specifications: - CPU: Intel(R) Xeon(R) Gold 5416S - RAM: 125 GB - Storage: 8 x 7 TB NVMe disks (Model: UP2A67T6SD004LX) [Drive specifications](https://www.techpowerup.com/ssd-specs/union-memory-uh711a-7… - Network: 4 x 25 Gbps interfaces configured with LACP bonding Each server in the setup is equipped with the above configuration.

3 4

Re: Posix backend for Radosgw
by Varada Kari 17 Feb '25

17 Feb '25

Anyone tried the posix backend for Ceph radosgw? Appreciate any pointers here on configuring and testing. Varada On Fri, Feb 14, 2025 at 12:40 PM Varada Kari <varada.kari(a)gmail.com> wrote: > Hi, > > I am trying to test the posix backend for radosgw in my test machine. It > is running Ubuntu 22.04 with the latest squid release(19.2.0) > I have configured a small single node cluster without any osd. > > $ sudo ceph -v > ceph version 19.2.0 (16063ff2022298c9300e49a547a16ffda59baf13) squid > (stable) > $ sudo ceph -s > cluster: > id: 92d1770c-e83e-11ef-bcc9-5f5839c0a1fa > health: HEALTH_WARN > Failed to place 1 daemon(s) > Reduced data availability: 1 pg inactive > OSD count 0 < osd_pool_default_size 3 > > services: > mon: 1 daemons, quorum testserver (age 3d) > mgr: testserver.iavwwx(active, since 2d) > osd: 0 osds: 0 up, 0 in > > data: > pools: 1 pools, 1 pgs > objects: 0 objects, 0 B > usage: 0 B used, 0 B / 0 B avail > pgs: 100.000% pgs unknown > 1 unknown > > progress: > Global Recovery Event (0s) > [............................] > > > Test machine is running a raided xfs file system. > Using the below config in the radosgw.conf > > [client] > > rgw backend store = dbstore > rgw config store = dbstore > dbstore db dir = /mnt/raidx/rgw/dbstore > dbstore_config_uri = file:///mnt/raidx/dbstore/config.db > > rgw filter = posix > rgw posix base path = /mnt/raidx/posix/root > rgw posix database root = /mnt/raidx/rgw/posix/lmdb > > [client.radosgw.gateway] > rgw init timeout = 1200 > rgw frontends = beast port=80 > > Not able to start the radosgw, fails to initialize the rados. > > 2025-02-11T00:35:13.183-0800 7f76b02eca40 0 ceph version 19.2.0 > (16063ff2022298c9300e49a547a16ffda59baf13) squid (stable), process radosgw, > pid 2673837 > 2025-02-11T00:35:13.183-0800 7f76b02eca40 0 framework: beast > 2025-02-11T00:35:13.183-0800 7f76b02eca40 0 framework conf key: port, > val: 7480 > 2025-02-11T00:35:13.183-0800 7f76b02eca40 1 init_numa not setting numa > affinity > 2025-02-11T00:35:13.183-0800 7f76b02eca40 0 pidfile_write: ignore empty > --pid-file > 2025-02-11T00:35:13.187-0800 7f76b02eca40 -1 rgw main: ERROR: failed to > initialize config store 'dbstore': unable to open database file > 2025-02-11T00:35:13.187-0800 7f76b02eca40 -1 Couldn't init storage > provider (RADOS) > > My intent is to run a standalone radosgw with a filesystem backend, which > works as a s3 gateway. It would be better if i can avoid running a ceph-mon > and other daemons as it is just working as a s3 gateway. > Are there any configuration steps I am missing here? Please point me to > any documentation available on this. Would appreciate any help on this. > > > Best, > Varada > > > > >

2 1

(no subject)
by Vignesh Varma 17 Feb '25

17 Feb '25

Best Regards, Vignesh Varma G Cloud Engineer www.stackbill.com

1 0

Create a back network?
by Nicola Mori 15 Feb '25

15 Feb '25

Dear Ceph users, I have a question about the benefits of creating a back network for my cluster. Currently, my machines are all connected to a gigabit switch, each with 2 bonded interfaces. The cluster just have a front network. Recently I got a new gigabit switch, so now I have 2 and I was wondering if there would be any benefit in setting up a back network on one switch and a front network on the other, using one interface per network. In this way I'll loose the redundancy, but other benefits and drawbacks are not clear to me. Would some expert help me in better understanding this issue, please? Thanks in advance, Nicola

4 3

Automatic OSD activation after host reinstall
by Eugen Block 14 Feb '25

14 Feb '25

Good morning, this week I observed something new, I think. At least I can't recall having seen that yet. Last week I upgraded a customer cluster to 18.2.4 (everything worked fine except RGWs keep crashing [0]), this week I reinstalled the OS on one of the hosts. And after a successful registry login, the OSDs were automatically activated without me having to run 'ceph cephadm osd activate <host>' as documented in [1]. Zac improved the docs just last week, is that obsolete now? Or is that version specific? A few weeks ago we reinstalled our own Ceph servers as well, but we still run Pacific 16.2.15, and there I had to issue the OSD activation manually. Can anyone confirm this? Thanks! Eugen [0] https://tracker.ceph.com/issues/69885 [1] https://docs.ceph.com/en/latest/cephadm/services/osd/#activate-existing-osds

2 2

Does the number of PGs affect the total usable size of a pool?
by Work Ceph 13 Feb '25

13 Feb '25

Hello guys, Let's say I have a cluster with 4 nodes with 24 SSDs each, and a single pool that consumes all OSDs of all nodes. After adding another host, I noticed that no extra space was added. Can this be a result of the number of PGs I am using? I mean, when adding more hosts/OSDs, should I always consider increasing the number of PGs from a pool?

4 11

Ceph Events Survey -- Your Input Wanted!
by Dan van der Ster 13 Feb '25

13 Feb '25

Hi all, The Ceph Foundation and Ambassadors are busy building a full calendar of Ceph events for 2025! As many of you know, Ceph has always thrived on its amazing community. Events like Ceph Days and Cephalocon are key points of interaction where we all can learn, connect, and share experience. Following the success of Cephalocon at CERN and Ceph Days in India, we’ve announced Ceph Days London & Silicon Valley -- check out https://ceph.io/en/community/events/ to get involved. And watch that space -- Ceph Days in Seattle, New York, and Berlin will be announced soon! Looking forward, we need your help to help shape our future events... and to plan our next Cephalocon! If you have a moment, please share your thoughts in our Ceph Events survey: https://forms.gle/Rm41d547Rb59S8xf9 Looking forward to seeing you at an event soon! --- Dan van der Ster Ceph Executive Council

1 0

Radosgw log Custom Headers
by Ansgar Jazdzewski 13 Feb '25

13 Feb '25

Hi folks, i like to make sure that the RadosGW is using the X-Forwarded-For as source-ip for ACL's However i do not find the information in the logs: i have set (using beast): ceph config set global rgw_remote_addr_param http_x_forwarded_for ceph config set global rgw_log_http_headers http_x_forwarded_for https://docs.ceph.com/en/quincy/radosgw/config-ref/#confval-rgw_log_http_he… I hope someone can point me in the right direction! Thanks, Ansgar

3 4

<---- breaks grouping of messages
by Marc 13 Feb '25

13 Feb '25

It is annoying that the subject modification of putting [ceph-users] first is breaking grouping of messages.

2 1