Hello Daniel, I recommend that you use the same link to talk to croit: https://www.croit.io/contact Caveat: it goes to sales. As far as I know, croit has a production-ready patch for Ceph-18.2.4 and submitted most (all?) of their patches upstream, but there is no ready-made backport to your Ceph version (19.2.x). Sales guys might view this as an opportunity to sell you developer time necessary for backporting and/or try to sell a support contract. Or, if you don't want to talk to croit, the main patch against taking the wrong directory snapshot when mirroring is this one: https://github.com/ceph/ceph/pull/62281 Note that this does not contain the functionality needed to automatically clean up after bugs in old versions of cephfs-mirror. A patch provided by croit contains this functionality, but I could not find any equivalent in public pull requests (probably didn't search well enough). You also may want to check if your Ceph build contains this: https://github.com/ceph/ceph/pull/61100 On Wed, Apr 16, 2025 at 11:26 PM Daniel Schreiber < daniel.schreiber@hrz.tu-chemnitz.de> wrote:
Hi Alexander,
we are observing similar behaviour on two 19.2.1 clusters that are not in production yet. If you want to debug something, please let me know.
We saw the following bugs:
* missing data on the destination * data from the past on the destination
Regards,
Daniel
Hello Vladimir,
Please contact croit via https://www.croit.io/contact for unofficial (not yet fully reviewed) patches and mention me. They are currently working on a similar problem and have solved at least the following:
* Crashes when two directories are being mirrored in parallel * Data corruption (snapshots on the destination having missing files or files "from the future") due to incorrect cache maintenance in libcephfs * Confusion that results in the failed mirroring when a directory is replaced by a symlink * Useless work done to mirror old snapshots that nobody needs, while the latest snapshot has the highest business value * Useless stat() calls slowing down the mirroring significantly * They also introduced multi-threaded mirroring so that more than one file is copied at a time and more than one subdirectory is scanned at a time - this helps with directories containing many small files
Due to the above, I absolutely cannot recommend using the unpatched cephfs-mirror and would rather (if you don't want to contact croit for patches) ask you to switch to rsync with some wrapper script that runs, in parallel, multiple copies operating on different subdirectories.
On Fri, Mar 21, 2025 at 12:48 AM Vladimir Cvetkovic <vladimir.cvetkovic@hostpapa.com> wrote:
Hi everyone,
We have two remote clusters and are trying to set up snapshot mirroring. Our directories are in a syncing state but never seem to actually sync
Am 21.03.25 um 06:38 schrieb Alexander Patrakov: the
snapshots. Snapshot mirroring seems way to slow for large file systems - does anyone have a working setup?
Thanks! _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Daniel Schreiber Facharbeitsgruppe Systemsoftware Universitaetsrechenzentrum
Technische Universität Chemnitz Straße der Nationen 62 (Raum B303) 09111 Chemnitz Germany
Tel: +49 371 531 35444
-- Alexander Patrakov