).
There is ~5.6PiB stored on /ceph, shown by ceph.dir.rbytes with 132M files and 139M rentries. The pool shows 7PiB stored and 9.7PiB used consistent with 8+3 EC. The layout for most files: ceph.dir.layout="stripe_unit=16777216 stripe_count=1 object_size=16777216 pool=cephfs_data_echdd"
But there is 1.4PiB discrepancy between the pool and the filesystem
Do you have scrubs enabled? Which if any non-default config options do you have set? Any undersized or degraded or backfilling PGs? Which Ceph release? Do you have a sizable fraction of small files? If you’re running Squid or earlier or don’t have EC optimizations enabled, even a tiny file will allocate a multiple of 11*16=176 KB. An 129KB file will consume 352KB, etc. If I understand those layout options correctly. If that’s what’s going on, going to Tentacle with EC optimizations would gain you some efficiency for files newly [re]written. You could also migrate small files to a replicated pool.
which I cannot explain and I suspect there are a lot of orphan objects there. I have run mds scrub on / and ~mdsdir as well. There is some mds damage on some old small files (~400 files), which I do not think it's relevant here.
Hello, We had a similar issue last year with a group of users that created and removed files at a very high rate.
Have you read https://docs.clyso.com/docs/kb/cephfs/#cephfs-pool-data-usage-growth-without... ?
We increased the purge rate parameters (very) aggressively to get back to a comfortable situation (i.e. not a pool w/ near full warnings).
Loïc.
-- _____________________________________________________________ prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic@ijs.si Department of Experimental High Energy Physics - F9 Jozef Stefan Institute, Jamova 39, P.o.Box 3000 SI-1001 Ljubljana, Slovenia Tel.: +386-1-477-3674 Fax: +386-1-477-3166 ------------------------------------------------------------- _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io