Hi again I've managed to simplify this. I think it only affects empty directories. It is still non-deterministic, ceph.dir.rctime will be set correctly between 30% and 80% of the time, the rest of the time it will be the same as the directory's original mtime. #!/bin/bash source="/teraraid4/toby/ceph-test" ### Your NFS file system mkdir -p $source/{old,new} touch -t 201810211300 $source/old cd /ceph/... # somewhere on your ceph file system x=0 while [ $x -lt 10 ]; do top=test-$x rm -rf $top echo "== start $top ==" rsync -a $source $top new_stamp=$(getfattr --only-values -n ceph.dir.rctime $top/ceph-test/new) while [ $(echo "$new_stamp < 1" | bc -l) -eq 1 ]; do new_stamp=$(getfattr --only-values -n ceph.dir.rctime $top/ceph-test/new) sleep 1 done old_stamp=$(getfattr --only-values -n ceph.dir.rctime $top/ceph-test/old) [ $(echo "$new_stamp > $old_stamp" | bc -l) -eq 1 ] && getfattr -n ceph.dir.rctime $top/ceph-test/* ((x++)) echo "== end $top ==" done Cheers Toby On 16/10/2019 13:48, Toby Darling wrote:
Hi
Occasionally, a directory's ceph.dir.rctime isn't updated when it is copied (rsync'd) to ceph, it reads the same as from stat, not the time it was created on ceph.
The cluster is all Scientific Linux release 7.7 with ceph 14.2.4 nautilus (stable), the problem happened with kernel client running on one of the mon nodes. The tree that was rsync'd in was about 20TB, 340k files, 3000 directories. In all, I've found just 5 directories suffering this - possibly of interest, they are all empty (although there were 472 empty dirs in total).
* dir1 is reported correctly:
[root@ceph ~]# ls -ld path/to/dir1 drwxr-xr-x 1 thisuser thisuser 0 Jun 18 2018 path/to/dir1
[root@ceph ~]# getfattr --only-values -n ceph.dir.rctime path/to/dir1 1571035073.09398156844 [root@ceph ~]# stat path/to/dir1 File: ‘path/to/dir1’ Size: 0 Blocks: 0 IO Block: 65536 directory Device: 0h/0d Inode: 1099637142596 Links: 1 Access: (0755/drwxr-xr-x) Uid: (26089/thisuser) Gid: (26089/thisuser) Access: 2019-10-14 07:37:53.397156853 +0100 Modify: 2018-06-18 18:19:12.001114388 +0100 Change: 2019-10-14 07:37:53.398156844 +0100 Birth: -
* But dir2's rctime is wrong:
[root@ceph ~]# ls -ld path/to/dir1/dir2 drwxr-xr-x 1 thisuser thisuser 0 Jun 18 2018 path/to/dir1/dir2 [root@ceph ~]# getfattr --only-values -n ceph.dir.rctime path/to/dir1/dir2 1529342352.091114388 [root@ceph ~]# stat path/to/dir1/dir2 File: ‘path/to/dir1/dir2’ Size: 0 Blocks: 0 IO Block: 65536 directory Device: 0h/0d Inode: 1099637142960 Links: 1 Access: (0755/drwxr-xr-x) Uid: (26089/thisuser) Gid: (26089/thisuser) Access: 2019-10-14 07:37:53.398156844 +0100 Modify: 2018-06-18 18:19:12.001114388 +0100 Change: 2019-10-14 07:37:53.398156844 +0100 Birth: -
Cheers Toby _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Cheers Toby