Hi Mikael, This might be a long shot, but I have to ask: have you checked the average file size on the current CephFS filesystem? Apart from extreme cost efficiency or a bad design, the EC choice on HDD could have had a legitimate reason in the past. It's probably not enough to make it the default data pool, but it might help explain the current design. I don't know the exact amount of data or how many users this filesystem has, but the safest long-term approach in your situation is probably to create a new filesystem and then migrate the data over, as you imagined. Best regards, Frédéric. -- Frédéric Nass Ceph Ambassador France | Senior Ceph Engineer @ CLYSO Try our Ceph Analyzer -- https://analyzer.clyso.com/ https://clyso.com | frederic.nass@clyso.com Le mer. 10 sept. 2025 à 16:09, Mikael Öhman <micketeer@gmail.com> a écrit :
The recommendations for cephfs is to make a replicated default data pool, and adding any EC data pools using layouts: https://docs.ceph.com/en/latest/cephfs/createfs/
If erasure-coded pools are planned for file system data, it is best to configure the default as a replicated pool to improve small-object write and read performance when updating backtraces.
I have an cephfs that unfortunately wasn't set up like this: they just made an EC pool on the slow HDDs as the default, which sounds like the worst case scenario to me. I would like to add a NVME data pool to this ceph fs, but recommended gives me pause on if i should instead go through the hassle of creating a new cephfs and migrating all users.
I've tried to run some mdtest with small 1k files to see if i could measure this difference, but speed is about the same in my relatively small tests so far. I'm also not sure what impact I should realistically expect here. I don't even know if creating files counts as "updating backtraces", so my testing might just be pointless.
I guess my core question is; just how important is this suggestion to keep the default data pool on replicated NVME?
Setup: 14 hosts x 42 HDD + 3 NVMEs for db/wal 2*2x25 GbitE bonds 12 hosts x 10 NVME. 2*2x100 GbitE bonds
Old CephFS setup: - metadata: replicated NVME - data-pools: EC 10+2 on HDD (i plan to add a EC NVME pool here via layouts)
New CephFS setup as recommended: - metadata: replicated NVME - data-pools: replicated NVME (default), EC 8+2 on HDD via layout, EC 8+2 on NVME via layout.
Ceph 18.2.7
Best regards, Mikael _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io