Results 1 to 3 of 3
Hello, I need to find modification times for a long list of files (around 25000). Network issues aside, what is the fastest way to do that? If I just try ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
- 01-06-2013 #1
- Join Date
- Jan 2013
what is the fastest way to find modification times for many files?
I need to find modification times for a long list of files (around 25000). Network issues aside, what is the fastest way to do that?
If I just try
>cat list_of_files.txt | xargs stat -c %Y
it takes around 4 minutes on my machine/my filesystem. Is the above the fastest way?
Thank you for any insight...
- 01-06-2013 #2
- Join Date
- Apr 2009
- I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
Your approach is not a bad one. The time mentioned (4 minutes == 240 seconds) is equivalent to 10ms per file to probe. This does not seem bad to me. My guess is that if you were to do the same again right away, it would be a lot faster, given that the directories in question would already be cached in memory - no disc I/O to perform most likely.Sometimes, real fast is almost as good as real time.
Just remember, Semper Gumbi - always be flexible!
- 01-06-2013 #3
Hi and welcome
This should be a bit faster, as there is no need to spawn 25000 stats:
stat -c %Y $(cat list_of_files.txt)
There is a risk to run into an "Argument list too long" error, if the length of arguments in bytes exceeds 1/4 of the stacksize (uname -s).
So if you need to get these modification times via script/cronjob then either use additional logic to keep the argument length below the maximum
or simply add e.g. --max-args=100 to your xargs:
cat list_of_files.txt | xargs --max-args=100 stat -c %Y
Last edited by Irithori; 01-06-2013 at 10:42 PM.You must always face the curtain with a bow.