English Amiga Board


Go Back   English Amiga Board > Support > support.WinUAE

 
 
Thread Tools
Old 03 December 2012, 22:25   #1
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 3,169
File timestamps lose precision in shared folders

It seems WinUAE keeps less-precise timestamps for files which are in shared folders. I noticed this with WinUAE 2.5.0 on native Windows Vista (shared folder on NTFS boot drive) and Wine (shared folder on ext4 partition).

Amiga native filesystem timestamp precision is to the tick (1/50th second). The NTFS native timestamp is apparently precise to 100ns, so NTFS should support accurate-enough timestamps.

To hopefully demonstrate this issue we'll use Cloanto's DirDiff program. That can compare files in two directories, indicating which are different. It compares file comment, protection bits and datestamp as well as the actual file content.

Try this:
  • Configure WinUAE to have at least one shared folder, I'll assume you called it SHARE:
  • Download DirDiff from http://aminet.net/biz/cloan/DirDiff.lha. Unpack it to the shared folder.
  • Boot WinUAE from an emulated hard disk or floppy disk. (You need at least Kickstart 2.0 to run DirDiff.)
  • With a blank/formatted floppy disk in DF1:, open a CLI and type
    Code:
    Echo "test1" >DF1:test1.txt
    Echo "test2" >DF1:test2.txt
    MakeDir RAM:test
    MakeDir SHARE:sharetest
    Copy DF1:#? RAM:test CLONE
    Copy DF1:#? SHARE:sharetest CLONE
    CD SHARE:DirDiff
    DirDiff 2 -b DF1: RAM:test
    DirDiff 2 -b DF1: SHARE:sharetest
The first DirDiff run, comparing with RAM:test, should show no differences. The second run should show one or both the files in SHARE:sharetest having different datestamps from the DF1: copies.

If you're too lazy let me know and I'll try to make a floppy image that does all that from the startup-sequence.
mark_k is offline  
Old 04 December 2012, 20:25   #2
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,806
Filesystem handler internally uses seconds for some reason.. (losing fractions of seconds part of ticks)
Probably some historic reason.

Will be fixed.
Toni Wilen is online now  
Old 05 December 2012, 14:59   #3
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 3,169
Maybe the shared folder handler was first introduced with WinUAE (as opposed to the original UNIX UAE), when Windows 95/98 was current and hence FAT was the most common partition format?

FAT file creation times are accurate to 10ms (since DOS 7.0, else 2 seconds). But file modification times are only accurate to 2 seconds. Perhaps the original developer didn't see a need to maintain greater accuracy in the AmigaDOS handler.

On that subject... are Amiga file datestamps stored along with comments in _UAEFSDB.___? That would stop host filesystem datestamp inaccuracy affecting the emulated Amiga.
mark_k is offline  
Old 05 December 2012, 18:28   #4
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,806
It is Unix based and it seems Unix file date resolution is (was?) 1 second.

EDIT: I replaced time_t with timeval: resolution is now microseconds.

_UAEFSDB.___ can't be changed without breaking compatibility with old versions = not going to happen.

Last edited by Toni Wilen; 05 December 2012 at 18:45.
Toni Wilen is online now  
Old 06 December 2012, 14:09   #5
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,806
Hmm..

There are some unused fields in UAEFSDB (for example status byte that only contains zero or non-zero), it should be possible to fit small time offset (0-99 ticks = max 2 seconds) somewhere without breaking backwards compatibility.
Toni Wilen is online now  
Old 06 December 2012, 16:10   #6
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 3,169
That would be one possibility for retro-fitting more accurate file times. But there are others.

I looked at a _UAEFSDB.___ file in a hex editor. It seems 2 × 256 bytes are allotted for each filename. (Each name is stored twice, first the Amiga native name then the corresponding host name. The host name may be longer than the Amiga one if the Amiga name contains characters such as \ which are "illegal" in host filenames.)

What is the maximum Amiga filename length? struct FileInfoBlock would seem to limit it to 107 characters. So that's about 148 free bytes after every Amiga filename entry. You could easily fit a struct DateStamp in there. [Storing a full date would have the advantage of preserving the Amiga datestamps if the user e.g. copies the contents of the shared folder to another directory on the host using a program which doesn't preserve file dates.]

Assuming that's correct, it should be quite simple to make backward-compatible additions to the _UAEFSDB.___ file. Older WinUAE versions might overwrite any data in the extra region with zeros on exit though?

If you did decide to use that extra space, maybe store the used length in the last byte, and perhaps use the second-last byte to indicate the uaefsdb file version. Then future versions of WinUAE would know to preserve that data if the version is later than they understand.

Last edited by mark_k; 06 December 2012 at 16:23.
mark_k is offline  
Old 06 December 2012, 20:30   #7
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,806
Yeah, there is more free space but saving complete date twice (FSDB and real file metadata) will cause confusion.

Modify data outside of emulation (or just the date) and it is guaranteed weird date difference will be considered a bug.

This is the reason why I prefer tick offset.
Toni Wilen is online now  
Old 06 December 2012, 21:54   #8
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 3,169
This is quite a tricky issue to get right. There probably isn't one right thing to do. But if you were to store an Amiga DateStamp in the UAEFSDB entry you could derive the tick offset from that.

The trouble with only storing a tick offset is, what happens if the user copies files elsewhere? Say they move their shared directory from an NTFS volume to a FAT32 one, e.g. a USB stick or SD card. Assume the program they used to do that preserves host file times (as closely as is possible for FAT32).

When previously accessed on the NTFS volume, all tick offset fields will be zero because NTFS datestamps are accurate enough. On copying to the FAT32 volume all host file times will be rounded to the nearest two seconds (or maybe to the previous two-second point, I don't know). And so will file times from the emulated Amiga's point of view, since all tick offsets are zero.

Whereas, if Amiga DateStamps were stored for every file instead of just a tick offset, you could restore/preserve the original file times. WinUAE could compare the host file date with the DateStamp from UAEFSDB. If the Amiga file time is within 2 seconds of the host time then assume the Amiga time is correct. If the host file time differs by more than that just tell Amiga programs that the Amiga file time is the same as the host file time.

Maybe have an option like "prefer UAEFSDB datestamps". That would allow a user who has accidentally moved/copied a shared folder without preserving the host file times to preserve the file times from the emulated Amiga's point of view.


The down size of this change is that every file written by the emulated Amiga would get its own entry in UAEFSDB. Whereas currently (I think) only files with host-illegal names or comments appear in UAEFSDB. So that would cause the UAEFSDB file to become larger than before.

I guess that could be avoided by maintaining a second file that only contains names and DateStamps for those files which don't have either a host-illegal name or a comment (i.e all files except those already in UAEFSDB). But that's more complexity...
mark_k is offline  
Old 06 February 2013, 14:20   #9
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 3,169
Only 2-second file date precision is supported with WinUAE 2.5.1 for files on FAT partitions. That's a limitation of the FAT last write time. However FAT file creation time supports precision to 10ms which should be enough for tick-accurate datestamps.

Can I suggest a change to support full-precision datestamps for files on FAT partitions:
- When a file is created, no problem since the creation time will be set appropriately.
- When an existing file is written to or the user manually sets its date using the SetDate command, use the SetFileTime function to set its creation time. (Maybe only do that when the file is closed for performance reasons?)
- When returning the Amiga datestamp of an existing file, check whether the creation time and last write time match (as closely as is possible). If they don't, the file may have been modified outside of emulation. So perhaps in that case return the last write time for the Amiga datestamp.
mark_k is offline  
Old 06 February 2013, 16:25   #10
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,806
Good idea, will be done.

EDIT: and done now. Maybe.

Last edited by Toni Wilen; 06 February 2013 at 20:43.
Toni Wilen is online now  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Logitech Precision gamepad hasn't worked since 2.3.3 Sigma support.WinUAE 13 20 March 2013 09:29
Unusual characters in filenames and shared folders mark_k support.WinUAE 3 03 February 2013 19:50
File comments and shared folders mark_k support.WinUAE 2 06 December 2012 10:15
Starting to lose hope :( ElectroBlaster support.Hardware 2 18 November 2012 01:32
How do i Get Miami to mount shared folders Rogueraven support.Hardware 6 13 March 2008 11:57

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 08:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Page generated in 0.07006 seconds with 13 queries