English Amiga Board


Go Back   English Amiga Board > Support > support.FS-UAE

 
 
Thread Tools
Old 13 February 2013, 21:37   #1
FrodeSolheim
FS-UAE Developer
FrodeSolheim's Avatar
 
Join Date: Dec 2011
Location: Førde, Norway
Age: 40
Posts: 3,860
Directory hard drives with full support for file attributes

The implementation of file attribute support with directory hard drives is complete and will be released with the next development version. It just remains to rewrite part of the code because a well-known operating system from Redmond lacks useful functions common on other systems...

I have also written a new documentation page about hard drive support in FS-UAE: http://fengestad.no/fs-uae/using-hard-drives, which includes this information (and also describes support for other hard drive types). Please let me know if you find any inaccuracies.

Here's how it works:

When FS-UAE looks up existing files in a directory hard drives, it initializes metadata about the file either from the host file system, or if it exists, from an accompanying metadata file.

When a file has an accompanying metadata file, date/time, file permissions and comment are read from the metadata file. If a metadata file does not exist, then:
  • The file will get the default permissions: —-rwed.
  • The file date/time will be read from the last modified timestamp of the host file.
  • The file will have no file comment.
When files are created/modified from within the Amiga, the host file mtime (last modified time) is updated based on the Amiga file date/time.

FS-UAE then decides if it needs to store more information about the file in a metadata file:
  • If the file has a file comment, FS-UAE will create a metadata file.
  • If the file has non-default file permissions, FS-UAE will create a metadata file.
  • FS-UAE reads back the last modified time from the host file, and checks that the time was stored with high enough precision (Amiga files are stored with 1/50s precision). If the time could not be stored accurately, FS-UAE will create a metadata file.
  • If the file already has a metadata file, FS-UAE will always update the metadata file.
Additional metadata is stored in files with .uaem extension. If the original file is called Image.jpg, the additional metadata is stored in Image.jpg.uaem in the same directory. The metadata is stored in a text format, and you can even edit this file manually from the host side to alter the information about the file. Here’s an example of the content of a metadata file (where "hello" is a file comment):
Code:
----rwed 2013-02-12 21:20:52.02 hello
I will probably later add support for:
  • Forcing metadata files to always be written (useful if one want to archive / unarchive directories, since archives may not store time with enough precision).
  • Forcing metadata files to never be written (useful if you don't need them, and don't want to clutter the hard drive directories with metadata files).

Last edited by FrodeSolheim; 13 February 2013 at 21:50.
FrodeSolheim is offline  
Old 18 February 2013, 21:04   #2
FrodeSolheim
FS-UAE Developer
FrodeSolheim's Avatar
 
Join Date: Dec 2011
Location: Førde, Norway
Age: 40
Posts: 3,860
This feature is now implemented (and tested) on Linux, Windows and Mac OS X. While testing, I discovered that the default file system on Mac OS X (HFS+) does not support time stamps with higher precision that 1s, so Amiga file date/time cannot be recorded exactly on this file system (without .uaem files). The system as described and implemented will detect this, and the consequence is that .uaem files are always* written on OS X even with default permissions (* always = 49/50 = 98% of the time )

To ensure long-term accuracy of the file attributes, the best way is to always write metadata to the .uaem files. Perhaps the default should be to always store .uaem files for all written/modified files (simply reading an existing file will never cause a metadata file to be written).

Alternative strategies, which can be implemented as options include:
- Write .uaem files when necessary (as described in the initial post). On HFS+ (and FAT) file systems, this means almost always. On NTFS and most Linux file systems, this means when comments/non-default attributes are used.
- Same as above, but relax the requirements for time resolution to 1s.
- Use .uaem files in even fewer cases, for example by ignoring the execute attribute (always enabled).
- Never write .uaem files. Files will always get default permissions, comments are not stored, and date/time is stored as accurately as possible on the host system.

Any thoughts?
FrodeSolheim is offline  
Old 28 February 2013, 12:22   #3
lem79
Registered User
 
Join Date: May 2009
Location: Sunshine Coast, Aus
Posts: 30
About the only thing I can think of is perhaps naming the metadata file .<original filename>.uaem (note the leading period), to perhaps make it hidden by default in regular file browsers? Obviously that's a feature common to Linux, not sure about OSX or Windows..

The other way I could think of is to have a single metadata file like the old __UAEFS.DB__ or whatever it was called back in the E-UAE days, but to use your new format, perhaps just having the first part of the line being the filename, then its attributes, like:

Code:
image.jpg ----rwed 2013-02-12 21:20:52.02 hello
If you're relying on spaces to separate the data, that wont work when filenames contain spaces, so perhaps using one of the common CSV formats would be good? i.e. comma separated, quote enclosed, and escaped (e.g. \" for a literal quote), like this:

Code:
"image with space.jpg","----rwed","2013-02-12","21:20:52.02","hello"
and

Code:
"image with \"quotes\".jpg","----rwed","2013-02-12","21:20:52.02","hello"
Maybe? At least with CSV it's still human readable and editable. It could then also have a header line, and be a bit more robust/extensible. People could also load the file into Calc or other spreadsheet program, if they had a need for that.

edit: If there are changes made to the filesystem by the host, FS-UAE would need to check if the file still exists. Renaming files could also pose a problem. There would be no real way to protect against this, and users would have to be discouraged from making major changes to the host filesystem outside of FS-UAE.

Last edited by lem79; 28 February 2013 at 12:25. Reason: What if people edit the filesystem outside of FS-UAE?
lem79 is offline  
Old 01 March 2013, 19:40   #4
FrodeSolheim
FS-UAE Developer
FrodeSolheim's Avatar
 
Join Date: Dec 2011
Location: Førde, Norway
Age: 40
Posts: 3,860
Hi, yes it would be possible to hide the files (start with a dot, for Linux/Unix, and also use hidden attribute for Windows). Another simple way would be to put the meta files inside a hidden .uaemeta directory.

But it is by design that the files are visible (and put alongside the corresponding files). This is to make the user aware of them. Otherwise, the user would for example move a file to another directory, unaware that the metadata got lost (because he didn't move the meta file, and didn't even know about it) -this would also be a problem with the older __UAEFS.DB__.

So I prefer to let the user know that there are other files to consider when doing file operations from the host side. Making it hidden takes away some of the point!

Another advantage of using one meta file per file (instead of a per-directory file) is that it is possible (and easy) to split and merge directories from the host side. This could not be done without special tools (or manual editing) if all metadata is assembled in one meta file.
FrodeSolheim is offline  
Old 28 April 2013, 04:24   #5
cybernoid
Users Awaiting Email Confirmation

 
Join Date: Jul 2006
Location: Portugal
Posts: 135
Quote:
Forcing metadata files to never be written (useful if you don't need them, and don't want to clutter the hard drive directories with metadata files).
I'm still waiting.
Even better was to make the old E-UAE mechanism work (I only have two ___UAE files, and they are the only needed).

FS-UAE is superb, but I don't agree at all with that paranoic way of registering file changes. It's not Amiga I think Workbench raises 40% with all those crazy-log-files.

I still use FS-UAE 2.0.1 because of that.



Many tks, and keep your excellent work.

Last edited by cybernoid; 28 April 2013 at 04:31. Reason: too much animatedgifs.
cybernoid is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
file attributes amiga2 support.Apps 4 26 January 2012 03:34
HDFs, directory hard drives and CD-ROM ISOs antonvaltaz support.WinUAE 5 25 November 2011 12:17
Hidden, Pure file attributes Bloodwych support.Apps 10 10 December 2007 00:26
Writing the contents of a directory to a text file Shoonay support.Other 8 02 July 2007 17:55
replacing amiga floppy drives with hard drives Gordon support.Hardware 2 06 March 2007 00:44

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 02:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.
Page generated in 0.12033 seconds with 15 queries