English Amiga Board

Go Back   English Amiga Board > Other Projects > project.EAB File Server

Thread Tools
Old 27 July 2019, 15:17   #1
Registered User
Join Date: Dec 2008
Location: Ursviken
Posts: 72
Timestamps for files on FTP / Gdrive

Is it possible to add a timestamp to each of the files in filelist.txt on the FTP that gets mirrored over to the gdrive ?

An alternative to not disturb those who parses the filelist.txt (and are not prepared for a simple change), add another filelist with the timestamps in the second column (before size)..

It's painfully slow to run the stat command on the remote files..

To be more clear, I would like the output from stat -c "%Y" to be added (%Y = time of last data modification, seconds since Epoch)

[or even md5 checksums for the files, but that's painfully slow on large files even on the local drive, could be built around, storing the md5s in a database for each file and use the stored value when timestamp has not changed]

Last edited by peo; 27 July 2019 at 15:34.
peo is offline  
Old 28 July 2019, 21:12   #2
Registered User
Join Date: Dec 2008
Location: Ursviken
Posts: 72
More as a proof of concept.. might be faster if coded directly as a shell script (I partially did that first, and that code is included too).

Written in PHP, this creates a filelist of the current directory (recursively) with the same format as EAB FTP filelist.txt but with the added fields for time (seconds since Epoch) and md5 (between date and size)

I ran this for benchmarking on a smaller set of 179 files with the total size of 235MB. Benchmarking done on a DS1517+, with ordinary 3.5" spinning disks.
Initial run (which checksums all the files) took 2.92 seconds. Running every second time took 0.12 seconds (output was redirected to a file).

1. relies on some unix commands as "md5sum" (can be implemented in PHP using md5(), but will be slower)
2. set the "md5path" to a path outside the files to be checksummed, otherwise this would be included in the filelist and the md5-summing on the next run, doubling the files in the list (and so on - doubled every time). Preferrably the md5path should be set to an absolute location.
3. change the current directory to the one you want to create a file list from. Run on a smaller set of files the first times to check that it does what's expected (can be run on any directory)
4. there's no cleanup routine added to the code (could be implemented by moving the md5path folder to another place, and then for each file in the current filelist.txt scan and move back the md5 files into place). Or simply just re-checksum all the 2.5TB and nearly a million of files every year or so.

Now.. only for someone to implement it on the filelist at EAB....
Attached Files
File Type: zip eab-build-filelist.zip (1.1 KB, 13 views)

Last edited by peo; 28 July 2019 at 21:25. Reason: note about cleanup added
peo is offline  

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
How many timestamps does AmigaOS use? Foebane support.Apps 3 18 March 2019 11:48
Password protected files on EAB FTP AeroMaxx project.EAB File Server 1 08 December 2017 08:57
File timestamps lose precision in shared folders mark_k support.WinUAE 9 06 February 2013 17:25
FTP Access to Files Leandro Jardim project.EAB File Server 7 07 December 2010 14:57
New FTP-mirrorsite: ftp.fi.amigascne.org Sallinen Amiga scene 1 07 May 2007 17:29

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 13:01.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.
Page generated in 0.05883 seconds with 16 queries