Dear Cephers, we are currently mounting CephFS with relatime, using the FUSE client (version 13.2.6): ceph-fuse on /cephfs type fuse.ceph-fuse (rw,relatime,user_id=0,group_id=0,allow_other) For the first time, I wanted to use atime to identify old unused data. My expectation with "relatime" was that the access time stamp would be updated less often, for example, only if the last file access was >24 hours ago. However, that does not seem to be the case: ---------------------------------------------- $ stat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root ... Access: 2019-04-10 15:50:04.975959159 +0200 Modify: 2019-04-10 15:50:05.651613843 +0200 Change: 2019-04-10 15:50:06.141006962 +0200 ... $ cat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root > /dev/null $ sync $ stat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root ... Access: 2019-04-10 15:50:04.975959159 +0200 Modify: 2019-04-10 15:50:05.651613843 +0200 Change: 2019-04-10 15:50:06.141006962 +0200 ... ---------------------------------------------- I also tried this via an nfs-ganesha mount, and via a ceph-fuse mount with admin caps, but atime never changes. Is atime really never updated with CephFS, or is this configurable? Something as coarse as "update at maximum once per day only" would be perfectly fine for the use case. Cheers, Oliver
You may be thinking of "lazytime". "relatime" only updates atime when updating mtime, to prevent being inconsistent. On Mon, Dec 2, 2019 at 4:46 AM Oliver Freyermuth <freyermuth@physik.uni-bonn.de> wrote:
Dear Cephers,
we are currently mounting CephFS with relatime, using the FUSE client (version 13.2.6): ceph-fuse on /cephfs type fuse.ceph-fuse (rw,relatime,user_id=0,group_id=0,allow_other)
For the first time, I wanted to use atime to identify old unused data. My expectation with "relatime" was that the access time stamp would be updated less often, for example, only if the last file access was >24 hours ago. However, that does not seem to be the case:
---------------------------------------------- $ stat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root ... Access: 2019-04-10 15:50:04.975959159 +0200 Modify: 2019-04-10 15:50:05.651613843 +0200 Change: 2019-04-10 15:50:06.141006962 +0200 ... $ cat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root > /dev/null $ sync $ stat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root ... Access: 2019-04-10 15:50:04.975959159 +0200 Modify: 2019-04-10 15:50:05.651613843 +0200 Change: 2019-04-10 15:50:06.141006962 +0200 ... ----------------------------------------------
I also tried this via an nfs-ganesha mount, and via a ceph-fuse mount with admin caps, but atime never changes. Is atime really never updated with CephFS, or is this configurable?
Something as coarse as "update at maximum once per day only" would be perfectly fine for the use case.
Cheers, Oliver
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On 2019-12-02 14:22, Nathan Fish wrote:
You may be thinking of "lazytime". "relatime" only updates atime when updating mtime, to prevent being inconsistent.
I was thinking about the behaviour of relatime on kernels since 2.6.30 (quoting mount(8)): -------------------- "Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but it doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.) Since Linux 2.6.30, the kernel defaults to the behavior provided by this option (unless noatime was specified), and the strictatime option is required to obtain traditional semantics. In addition, since Linux 2.6.30, the file's last access time is always updated if it is more than 1 day old." -------------------- This also matches RedHats documentation for RHEL 6 and up: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm... And it also matches what I observe for a local ext4 e.g. on Ubuntu 18.04: -------------------- # mount /dev/sda7 on /var type ext4 (rw,relatime,data=ordered) # stat /var/log/gpu-manager.log ... Access: 2019-09-20 18:26:12.713360857 +0200 Modify: 2019-11-20 01:11:11.594634547 +0100 Change: 2019-11-20 01:11:11.594634547 +0100 # cat /var/log/gpu-manager.log > /dev/null # stat /var/log/gpu-manager.log ... Access: 2019-12-02 14:28:20.643034594 +0100 Modify: 2019-11-20 01:11:11.594634547 +0100 Change: 2019-11-20 01:11:11.594634547 +0100 -------------------- I would also be fine with lazytime, though, but does ceph-fuse support it? Cheers, Oliver
On Mon, Dec 2, 2019 at 4:46 AM Oliver Freyermuth <freyermuth@physik.uni-bonn.de> wrote:
Dear Cephers,
we are currently mounting CephFS with relatime, using the FUSE client (version 13.2.6): ceph-fuse on /cephfs type fuse.ceph-fuse (rw,relatime,user_id=0,group_id=0,allow_other)
For the first time, I wanted to use atime to identify old unused data. My expectation with "relatime" was that the access time stamp would be updated less often, for example, only if the last file access was >24 hours ago. However, that does not seem to be the case:
---------------------------------------------- $ stat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root ... Access: 2019-04-10 15:50:04.975959159 +0200 Modify: 2019-04-10 15:50:05.651613843 +0200 Change: 2019-04-10 15:50:06.141006962 +0200 ... $ cat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root > /dev/null $ sync $ stat /cephfs/grid/atlas/atlaslocalgroupdisk/rucio/group/phys-higgs/ed/cb/group.phys-higgs.17620861._000004.HSM_common.root ... Access: 2019-04-10 15:50:04.975959159 +0200 Modify: 2019-04-10 15:50:05.651613843 +0200 Change: 2019-04-10 15:50:06.141006962 +0200 ... ----------------------------------------------
I also tried this via an nfs-ganesha mount, and via a ceph-fuse mount with admin caps, but atime never changes. Is atime really never updated with CephFS, or is this configurable?
Something as coarse as "update at maximum once per day only" would be perfectly fine for the use case.
Cheers, Oliver
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi together, I had a look at ceph-fuse code and if I read it correctly, it does indeed not seem to have the relatime behaviour since kernels 2.6.30 implemented. Should I open a ticket on this? Cheers, Oliver Am 02.12.19 um 14:31 schrieb Oliver Freyermuth:
I was thinking about the behaviour of relatime on kernels since 2.6.30 (quoting mount(8)): -------------------- "Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but it doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.)
Since Linux 2.6.30, the kernel defaults to the behavior provided by this option (unless noatime was specified), and the strictatime option is required to obtain traditional semantics. In addition, since Linux 2.6.30, the file's last access time is always updated if it is more than 1 day old." --------------------
Yes, CephFS makes no attempt to maintain atime. If that's something you care about you should make a ticket and a case for why it's important. :) On Sat, Dec 14, 2019 at 5:42 AM Oliver Freyermuth <freyermuth@physik.uni-bonn.de> wrote:
Hi together,
I had a look at ceph-fuse code and if I read it correctly, it does indeed not seem to have the relatime behaviour since kernels 2.6.30 implemented. Should I open a ticket on this?
Cheers, Oliver
Am 02.12.19 um 14:31 schrieb Oliver Freyermuth:
I was thinking about the behaviour of relatime on kernels since 2.6.30 (quoting mount(8)): -------------------- "Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but it doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.)
Since Linux 2.6.30, the kernel defaults to the behavior provided by this option (unless noatime was specified), and the strictatime option is required to obtain traditional semantics. In addition, since Linux 2.6.30, the file's last access time is always updated if it is more than 1 day old." --------------------
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Am 16.12.19 um 11:43 schrieb Gregory Farnum:
Yes, CephFS makes no attempt to maintain atime. If that's something you care about you should make a ticket and a case for why it's important. :)
Thanks for confirming :-). For those following along and also interested, I created the ticket here: https://tracker.ceph.com/issues/43337 Our use case (probably applicable to many clusters - also described there) is "cleanup", i.e. find data not read since months which might be a good candidate for deletion (or maybe, moving to a "colder" place). Cheers and thanks, Oliver
On Sat, Dec 14, 2019 at 5:42 AM Oliver Freyermuth <freyermuth@physik.uni-bonn.de> wrote:
Hi together,
I had a look at ceph-fuse code and if I read it correctly, it does indeed not seem to have the relatime behaviour since kernels 2.6.30 implemented. Should I open a ticket on this?
Cheers, Oliver
Am 02.12.19 um 14:31 schrieb Oliver Freyermuth:
I was thinking about the behaviour of relatime on kernels since 2.6.30 (quoting mount(8)): -------------------- "Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but it doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.)
Since Linux 2.6.30, the kernel defaults to the behavior provided by this option (unless noatime was specified), and the strictatime option is required to obtain traditional semantics. In addition, since Linux 2.6.30, the file's last access time is always updated if it is more than 1 day old." --------------------
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi all, We would need the same feature in our HPC cluster. I guess this is not an unfrequent problem, I was wondering if you guys found an alternative solution. Best -- Filippo Stenico Services and Support for Science IT (S3IT) Office Y11 F 52 University of Zürich Winterthurerstrasse 190, CH-8057 Zürich (Switzerland) On Tuesday, 17 December 2019 02:32:35 CET Oliver Freyermuth wrote:
Am 16.12.19 um 11:43 schrieb Gregory Farnum:
Yes, CephFS makes no attempt to maintain atime. If that's something you care about you should make a ticket and a case for why it's important. :)
Thanks for confirming :-). For those following along and also interested, I created the ticket here: https://tracker.ceph.com/issues/43337 Our use case (probably applicable to many clusters - also described there) is "cleanup", i.e. find data not read since months which might be a good candidate for deletion (or maybe, moving to a "colder" place).
Cheers and thanks, Oliver
On Sat, Dec 14, 2019 at 5:42 AM Oliver Freyermuth <freyermuth@physik.uni-bonn.de> wrote:
Hi together,
I had a look at ceph-fuse code and if I read it correctly, it does indeed not seem to have the relatime behaviour since kernels 2.6.30 implemented. Should I open a ticket on this?
Cheers, Oliver
Am 02.12.19 um 14:31 schrieb Oliver Freyermuth:
I was thinking about the behaviour of relatime on kernels since 2.6.30 (quoting mount(8)): -------------------- "Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but it doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.)
Since Linux 2.6.30, the kernel defaults to the behavior provided by this option (unless noatime was specified), and the strictatime option is required to obtain traditional semantics. In addition, since Linux 2.6.30, the file's last access time is always updated if it is more than 1 day old." --------------------
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (4)
-
Filippo Stenico
-
Gregory Farnum
-
Nathan Fish
-
Oliver Freyermuth