English Amiga Board


Go Back   English Amiga Board > News

 
 
Thread Tools
Old 29 August 2017, 16:15   #21
Akira
Registered User

Akira's Avatar
 
Join Date: May 2001
Location: New York
Posts: 18,274
Quote:
Originally Posted by Toni Wilen View Post
But quick answer: for example if you already use FFS and don't bother changing it or just want to use FFS.
Thanks. That's great. It could be useful, for example, when using CD32Load. If it works with it (should!)
Akira is offline  
AdSense AdSense  
Old 29 August 2017, 16:44   #22
Olaf Barthel
Registered User
 
Join Date: Aug 2010
Location: Lehrte, Germany
Posts: 154
Quote:
Originally Posted by Akira View Post
I thought Fast File System was garbage to begin with.
Don't be so harsh on the old file system, and the FFS which followed it.

In 1985 it was a perfectly adequate hierarchical file system for use on microcomputers. It did not suffer from artificial limitations on the number of files per directory or the directory nesting level.

It kept checksums on all metadata and data blocks, and (almost) managed to both detect and recover from damage. File names would use the ISO-8859-1 character set (not just plain 7 bit ASCII), and the maximum file name length (31 characters) wasn't so bad, considering what everybody else had to put up with.

Just imagine Commodore settling for something like GEMDOS and its FAT file system, like Atari did, or Apple's original Macintosh file system, followed by the various HFS incarnations. All of that would have been so much worse. Microsoft never managed to really kick FAT to the curb until NTFS was ready for the mass market in 2001.

The FFS was, technically, quite the impressive implementation which mixed packet and device I/O to great effect. That was almost what you could have called an event-driven architecture. Most of the time, the FFS would just sit there idle, waiting for I/O operations to complete, and if you had a disk controller capable of DMA, then the FFS didn't even have to copy any data around. If the stars aligned, you'd have a very low latency, low CPU load file system layer.

I'd say that the original file system and the FFS did better than average, but the downsides were hard to ignore. Back in 1985 nobody really expected a floppy disk file system to cache data yet to be written to the medium, which made this a rather error prone solution (it really was a file system intended for larger mass storage devices, shoehorned for use with floppy disks). Ejecting the disk at the wrong time would require the validator to repair the damage, and when that failed (when rather than if), the DiskDoctor to finish it off. Good intentions, no doubt, but poor execution.

All the technology which eventually displaced the 1970'ies file system designs, of which the original file system and FFS were typical examples, wasn't ready until the early 1990'ies. That's when OFS and FFS had started to smell really rank.

By this time, also owing to disks with much more than 20 Megabytes of storage (practical limit for OFS on-disk data structures) becoming cheaper and cheaper by the month, the limitations of the OFS/FFS designs really hit home. The file system data structures, both on-disk and in-memory, were all built around lists rather than trees. The scalability just wasn't there, by design. The joke was that if you allowed the file system to use more memory buffers for caching, the slower it became because it had to search the buffer list for the matching block.

Write-ahead journaling could have been retrofitted into the FFS, at least, but it didn't happen because it was not an obvious extension to the existing design. It would have done away with the validation process and made all write operations (including on-disk data structure changes) both faster and resistant to crash damage.

Last edited by Olaf Barthel; 29 August 2017 at 16:51.
Olaf Barthel is offline  
Old 29 August 2017, 16:55   #23
Akira
Registered User

Akira's Avatar
 
Join Date: May 2001
Location: New York
Posts: 18,274
Fair enough, I was being too harsh. But from the get-go I suffered those stupid Validation issues whenever a shitty program crashed while the disk was being accessed somehow, and they were really painful to stomach, and that is when there were no other errors after validation. I really grew to hate it, haha.
Akira is offline  
Old 29 August 2017, 22:43   #24
kolla
Registered User
kolla's Avatar
 
Join Date: Nov 2007
Location: Trondheim, Norway
Posts: 767
There are ways to avoid that problem, if you have some spare RAM
kolla is offline  
Old 30 August 2017, 14:13   #25
Olaf Barthel
Registered User
 
Join Date: Aug 2010
Location: Lehrte, Germany
Posts: 154
Quote:
Originally Posted by Akira View Post
Fair enough, I was being too harsh. But from the get-go I suffered those stupid Validation issues whenever a shitty program crashed while the disk was being accessed somehow, and they were really painful to stomach, and that is when there were no other errors after validation. I really grew to hate it, haha.
The file system turned out to be a poor fit for this operating system.

Both Kickstart and Workbench software suffered greatly from stability problems until the entire code was reviewed and reworked for Kickstart/Workbench 2.0. Some of the big problems resulted from not testing for memory allocation failure, from failing to propagate error conditions through the callchain, and (who knew) operating system code relying on the contents of scratch registers.

Commodore could address these issues, but 3rd party developers often struggled to produce robust software: the Amiga operating system design is very unforgiving when it comes to accidental memory corruption. When it was new, the Amiga operating system was much, much more complex than what came before it, if you were familiar only with the hobbyist and home computers that shipped in the previous 10 years.

Finding the right spot in which the floppy disks were not a nuisance and the file system didn't cause trouble was possible, but it was damn hard to get there

I recall that for me the turning point came when I bought an external disk drive. Finally, I was able to use the external drive to make my first steps into becoming a better programmer, by keeping the project data on it rather than on the boot floppy. Making backups also became much easier. Containing the damage, mitating the risk, was possible at last.

Still, you could not rely upon the file system, and just trying to mitigate the risks made it harder what was already hard to begin with: Amiga software development. You probably know the many anecdotal stories of programmers losing all their work just because one floppy disk failed, no usable backups were available and repairs did not succeed.
Olaf Barthel is offline  
Old 30 August 2017, 18:15   #26
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 2,695
Quote:
Originally Posted by Olaf Barthel View Post
Sorry, the recorded development history (RCS files, etc.) of the FFS, as implemented by Steve Beats, does not go any further back in time than version 34.3 (no version string, and no proper date stamp on the files, so I guess this might have been a 1987 or 1988 release). That version assumes a minimum block size of 512 bytes, there's even the telltale 'moveq #9,d1' for translating byte into block offsets (like trackdisk.device, hddisk.device, etc. prefer it).

It's possible, but I somewhat doubt that a 256 byte option escaped from the lab
It's possible that I mis-remembered of course. But I seem to remember reading a changelog for FFS (would that be on one of the developer CDs?) mentioning removal of support for 256-byte blocks.

If it was ever supported, it was probably first possible with some post-1.3 FFS. Looking at the code here, v36.35 doesn't have the MOVEQ #9,D1 and LSR sequence, whereas
34.85 (WB 1.3) and 36.03 (WB 1.3.3) both do.

I'll try testing various FFS versions in WinUAE to see whether I'm wrong about that. Update: At least FFS 36.104 does support 256-byte filesystem block size! I'll upload WinUAE config and sample HDF in a separate thread.

Last edited by mark_k; 30 August 2017 at 18:37.
mark_k is offline  
Old 03 September 2017, 17:06   #27
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 42
Posts: 19,934
r3 update:

- Force Direct SCSI if device driver is scsi.device, all scsi.device variants support it and at least CDTV scsi.device (and probably older A590/A2091 ROMs too) don't return error when calling unsupported functions, causing TD64 misdetection.
- Fixed bug in SCSI direct code, it never set io_Length in SCSI direct mode. TD64 check sets it to larger than required value (block size) but nothing sets it if TD64 is disabled.
- Always disable TD64 capability test under KS 1.3.

Filesystem is now fully compatible with CDTV SCSI adapter and A590/A2091 pre-7.0 ROMs (if some of them have same bug as CDTV scsi.device), also bypasses 1G drive size limit.
Toni Wilen is offline  
Old 06 September 2017, 19:11   #28
Olaf Barthel
Registered User
 
Join Date: Aug 2010
Location: Lehrte, Germany
Posts: 154
Quote:
Originally Posted by mark_k View Post
It's possible that I mis-remembered of course. But I seem to remember reading a changelog for FFS (would that be on one of the developer CDs?) mentioning removal of support for 256-byte blocks.

If it was ever supported, it was probably first possible with some post-1.3 FFS. Looking at the code here, v36.35 doesn't have the MOVEQ #9,D1 and LSR sequence, whereas
34.85 (WB 1.3) and 36.03 (WB 1.3.3) both do.

I'll try testing various FFS versions in WinUAE to see whether I'm wrong about that. Update: At least FFS 36.104 does support 256-byte filesystem block size! I'll upload WinUAE config and sample HDF in a separate thread.
I just found out the awful truth: the original FFS does not seem to care about the sector size at all. Sector sizes which are not a power of two are fine. So are sector sizes of odd numbers, for example. Just don't use a number smaller than 1, because then either a) memory allocations might fail or b) the log2(sector size) calculation might slip up. Sector sizes smaller than some 240 bytes will be problematic, because then the number of hash table entries in a directory will lead to instant corruption of said data structures.

Seems that the only guard against utter hilarity was in that storage device drivers would not necessary allow for sector sizes smaller than 512 bytes, or for that matter, sector sizes not divisible by 512 bytes. However, one probably shouldn't be so sure about that either, considering how rough-and-tumble many Amiga device drivers used to be

Needless to say, if you want to go ahead and experiment with the software, make sure you have adequate backups of your data at hand, make sure, too, that you can restore them if you need to, and please take pictures of the enfolding desaster and share them with the eager readers of this forum
Olaf Barthel is offline  
Old 08 September 2017, 17:29   #29
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 42
Posts: 19,934
Change of plans. "FFSAIO" will be released soon after all

Current changelog (1.x requesters to do and testing):

Based on v45.13 (OS3.9BB2 version) which is KS2.0+ and NSD only (if partition is outside of first 4G).

Patches:

- Full Kickstart 1.3 compatibility.
- TD64 and DirectSCSI support and autodetection.
- TD_GETGEOMETRY/Direct SCSI/TD64 blacklist from v44.5
- V45.14 validator FreeMem() fix. (I am not sure what v45.16 bug fix does)

Autodetection is different than in PFS3AIO because dostype can't be used to configure detection mode.

- NSD and TD64 detection is skipped if partition is inside first 4G.
- NSD detection is first, NSCMD_DEVICEQUERY must return sane results.
- TD64 detection is next if NSD was not detected and device driver was not in blacklist.

- Direct SCSI is last and detection is only executed if no NSD and no TD64 was detected and device driver was not in blacklist. Partition location (inside 4G or not) is not checked. (To support old A590/A2091 and CDTV SCSI with 1G limit)
- Direct SCSI detection executes HD_SCSICMD READ CAPACITY which must return valid data. Direct SCSI is not selected if returned drive size is less than 1G.
Toni Wilen is offline  
Old 14 September 2017, 02:42   #30
thebajaguy
Registered User

 
Join Date: Mar 2017
Location: Philadelphia / United States
Posts: 26
I recall Bernoulli SCSI removable drives having 256-byte sectors. I remember the original GVP scsidev.device 'Advanced Autoboot v2.2' ROMs being the ones needed to deal with it until later gvpscsi.device 3.x and higher handled things much better. I also recall forcing Quantum disk drives into 2048-block mode for testing. Although not a FFS/OFS issue, I somehow recall there being some CD-ROM devices that in some modes could return 256-byte sectoring. <br />
<br />
That's nearly a 30-yr memory dusted off...
thebajaguy is offline  
Old 14 September 2017, 16:04   #31
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 2,695
SyQuest 44MB disks also have 256-byte sectors, though it may be that some drives' firmware emulated 512 bytes.

If you still have those Advanced Autoboot v2.2 ROMs, please dump them, I'd like to look at the code.
mark_k is offline  
Old 30 September 2017, 22:39   #32
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 42
Posts: 19,934
Quote:
Originally Posted by Toni Wilen View Post
Change of plans. "FFSAIO" will be released soon after all
Here is first test version. Do not upload to Aminet! Do not add to any patch packages!

"FFSAIO"

Based on official v45.13 (OS3.9BB2 version) which is KS 2+ and NSD only (if partition is not located inside first 4G).

Patches:

- Full Kickstart 1.3 compatibility (see below).
- TD64 and DirectSCSI support and autodetection.
- TD_GETGEOMETRY/Direct SCSI/TD64 blacklist from v44.5.
- v45.14 validator FreeMem() fix and v45.16 bitmap size calculation off by one bitmap bit fix.
- ACTION_FH_FROM_LOCK freed the lock if call failed. (Out of memory or lock pointed to a directory)
- Fixed wrong offset (which was supposed to point at the end of block buffer) used in some DirCache-only routine.
- Test read last block of partition, if read fails, partition won't be mounted.

- WARNING! ULTRA-UNSUPPORTED!: Enabled and fixed hidden partially implemented long file name support. If partition dos type is DOS\8, max file/directory name length gets increased to 54 characters (from 30) by using previously unused bytes of file/dir header block. WARNING^2: No recovery tools.

Information:

Autodetection is different than in PFS3AIO because dostype can't be used to configure detection mode:

- NSD and TD64 detection is skipped if partition is fully inside first 4G.
- NSD detection is first, NSCMD_DEVICEQUERY must return sane results (No changes in NSD detection, uses original code).
- TD64 detection is next if NSD was not detected and device driver was not in blacklist.

- Direct SCSI is last and detection is only executed if no NSD and no TD64 was detected and device driver was not in blacklist. Partition location (inside 4G or not) is ignored.
- Direct SCSI detection executes HD_SCSICMD READ CAPACITY which must return valid data. Direct SCSI is not selected if returned drive size is less than 1G.

KS 1.3 patches:

- ACTION_STARTUP dp_Arg1 and dp_Arg3 are not valid under KS 1.3. (Same patch as in PFS3AIO)
- utility.library UMult32 and UDivMod32 calls replaced.
- Requesters (read/write error, you must insert, validation error) replaced with calls to BCPL makesysreq().
- WB 1.3 "ghost" icon fix. ACTION_INFO/ACTION_DISK_INFO must always return DOS\0, if dos type is DOS\1 or higher.

Instructions:

Download patch file http://www.winuae.net/files/b/fastfi...m_45.20_r1.zip
Unpack patch
Find OS39BB2 L:FastFilesystem (45.13)
CD to directory where patch file was unpacked
Copy 45.13 FastFileSystem to current directory
Create 45.20: spatch -ofastfilesystem_45.20 -pfastfilesystem_45.20_r1.pch fastfilesystem
Copy/Rename fastfilesystem_45.20 to L: (or somewhere else)
Use HDToolBox to install it.

NOTE: This thread is not for generic filesystem/hdtoolbox instructions.

FINAL WARNING: You should know what are you doing! Take backups. There is no guarantees. There is no support.
Toni Wilen is offline  
Old 01 October 2017, 17:02   #33
mark_k
Registered User
 
Join Date: Aug 2004
Location:
Posts: 2,695
For anyone brave enough to test the long filename support...

DOS\8 is identical to DOS\3 (international FFS) when there are no files with long names. So you can in-place convert an existing DOS\3 partition to DOS\8 just by changing the DosType in the first longword (and edit your mount file to reflect the new DosType).

Similarly, you could convert back to DOS\3, providing you're certain that there are no files or directories with long names in the partition.

Programs like DiskSalv and ReOrg won't of course work with DOS\8 partitions.

Last edited by mark_k; 01 October 2017 at 18:39.
mark_k is offline  
Old 17 November 2017, 11:55   #34
Aladin
Registered User
 
Join Date: Nov 2016
Location: France
Posts: 136
ffs 45.20 compatible too with os4 classic and morphos classic?
Aladin is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Amiga CD32 FULLY recapped inc CD drive circuit cypher007 MarketPlace 0 13 March 2017 01:15
Large Hard Drive Partition Question Tempest 2084 support.Hardware 15 27 June 2016 19:27
large drive issues with cf trebormills project.ClassicWB 4 02 September 2011 01:06
Large partition support on GVP HD+8 Series II Hewitson support.Hardware 9 19 May 2010 09:16
Hard Drive Partition ironthighs New to Emulation or Amiga scene 2 07 September 2004 23:24

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 05:21.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Page generated in 0.27106 seconds with 12 queries