Fully KS 1.3 compatible large drive/partition FastFileSystem v44.5

[Toni Wilen]

Here is another KS 1.3 filesystem compatibility update, this time for v44.5 FastFileSystem.
This may have very limited use but PFS3AIO-like all-os-compatible FastFileSystem with
big drive/partition support didn't exist previously.

TEST VERSION. Do not upload to Aminet or similar sites!

Patch applies to http://aminet.net/package/disk/misc/ffstd64 (base filesystem comes with WB 3.1)

v44.5 is TD64 and DirectSCSI compatible and supports large partitions. Most 1.3 compatible HD SCSI
controllers are DirectSCSI compatible without size limits even if normal access mode has 1G or 4G limit.

WARNING: Compared to PFS3AIO, this is only a simple patch, it does not introduce any extra functionality,
like validating if complete partition is accessible and because FFS root block is located at the middle
of partition, partition will mount even if data after root block is inaccessible. (PFS3 "superblock"
is located in last block of partition)

WARNING: Some old HD SCSI controllers may not support HD_SCSICMD (DirectSCSI) and in worst case
they may use it for other purposes, possible destructive. Test it by running HDToolBox (don't use
drive that has important data!), if HDToolBox detects the drive: HD_SCSICMD is supported and working.
If not: Don't use it with >4G drives! Any HD_SCSICMD query program is also fine for testing this.
Most old IDE controllers probably have overflow bugs in HD_SCSICMD emulation if drive is too large.

NOTE: Read http://eab.abime.net/showthread.php?t=61666 first! This only changes KS 1.3 compatibility,
(previously it would just hang or crash) no drive size/partition size/size of universe limits change.
Free space shown will be very weird if >2G partition etc, the usual ..

KS 1.3 compatibility patches are applied on the fly, if system is >=v36, original unpatched code runs.

Patches applied:

- ACTION_STARTUP dp_Arg1 and dp_Arg3 are not valid under KS 1.3. (Same patch as in PFS3AIO)
- OpenLibrary("utility.library",36) disabled.
- utility.library UMult32 and UDivMod32 calls replaced.
- Requesters (read/write error, you must insert, validation error) replaced with calls to BCPL makesysreq().

r2:

- WB 1.3 "ghost" icon fix. ACTION_INFO/ACTION_DISK_INFO must always return DOS\0, even if dos type is DOS\1 or higher.

r3:

- Force Direct SCSI if device driver is scsi.device, all scsi.device variants support it and at least
CDTV scsi.device (and probably older A590/A2091 ROMs too) don't return error when calling unsupported
functions, causing TD64 misdetection.
- Fixed bug in SCSI direct code, it never set io_Length in SCSI direct mode. TD64 check sets it to larger than
required value (block size) but nothing sets it if TD64 is disabled.
- Always disable TD64 capability test under KS 1.3.

I also originally made KS 1.3 compatibility patch for v45.13 (v45.16) but it was totally useless because
v45 requires NSD if partition is outside of first 4G which no old controller boot rom supports.

Quick instructions:

Download http://aminet.net/package/disk/misc/ffstd64
Download patch file http://www.winuae.net/files/b/fastfi..._5_ks13_r3.zip
Unpack both to same directory
Find WB 3.1 disk or image (or other source for v40.1 L:FastFileSystem)
CD to directory where you unpacked both archives
Copy v40.1 FastFileSystem to current directory
Copy attached patch file (fastfilesystem_44_5_ks13.pch) to current directory
Create v44.5: spatch -ofastfilesystem_44_5 -pfastfilesystem_40_1.patch fastfilesystem
Create KS1.3 compatible: spatch -ofastfilesystem_44_5_ks13 -pfastfilesystem_44_5_ks13.pch fastfilesystem_44_5
Copy/Rename fastfilesystem_44_6_ks13 to L: (or somewhere else)
Use HDToolBox to install it.

NOTE: This thread is not for generic filesystem/hdtoolbox instructions.

FINAL WARNING: You should know what are you doing! Take backups. There is no guarantees. There is no support.

Tested with my usual SupraDrive 500XP and ACA500plus setups under KS 1.3 with 8G CF containing single 8G FFS partition.
also 10G partition was tested under emulation. (Filled it with 1.5G files, then checked SHA-1 hashes.)

[ShK]

I made one 32GB FFS partition. It's working nicely on A600. Is there any way to get rid of DH0:DOS ghost icon on WB?

[mark_k]

Does the ghost icon still appear if your partition is FFS (not international)?

[ShK]

Yes. I think it comes from FastFileSystem 40.1 when using it with kick 1.3.

[Toni Wilen]

I tried to make ACTION_INFO and ACTION_DISK_INFO to always return DOS\1 (if it is actually DOS\3) but it didn't make any difference.

[ShK]

fastfilesystem_44_5_ks13 + scsi.device4345p_ks13 seems to work nicely so far. This is only a cosmetic issue.

[Toni Wilen]

http://www.winuae.net/files/b/fastfi..._5_ks13_r2.zip fixes "ghost" volume.

WB 1.3 does not like DOS\1 (or higher) in InfoData id_DiskType field. (ACTION_INFO/ACTION_DISK_INFO patched to return DOS\0 even if DOS\1 or higher)

It shouldn't cause any side-effects (original KS 1.3 compatible FFS does this) but I'll wait for reports before replacing original patch.

[ShK]

Nice! I can confirm that fix works.

[Olaf Barthel]

Originally Posted by Toni Wilen

http://www.winuae.net/files/b/fastfi..._5_ks13_r2.zip fixes "ghost" volume.

WB 1.3 does not like DOS\1 (or higher) in InfoData id_DiskType field. (ACTION_INFO/ACTION_DISK_INFO patched to return DOS\0 even if DOS\1 or higher)

That's the way you have to do this

Plenty of legacy software assumes that unless the file system signature of a volume is ID_DOS_DISK (= 'DOS\0'), the contents of the volume are not readable. This includes Workbench, of course. The 1.3 FFS deliberately reports the file system type as ID_DOS_DISK if it finds that it's actually ID_FFS_DISK.

The rules changed with Kickstart 2.0, and the updated workbench.library was the first version which acknowledged that all ID_DOS#? signatures are indicative of a file system being on the job rather than not knowing what to make of the disk contents (ID_NOT_REALLY_DOS = 'NDOS') in the first place. This is why, for example, ram-handler always reports ID_DOS_DISK rather than something more easily associated with this peculiar type of file system.

[Toni Wilen]

Yeah, it was in UAE directory filesystem comments that 1.3 really wants DOS\0.. I just missed it originally and thought DOS\1 was supported but not DOS\3+.

I have one more hack to do, not sure if it is possible: force KS 1.3 to use new filesystem for floppies, automatically.

I assume RootNode->rn_FileHandlerSegment needs to be replaced after dos has been initialized and before floppy handler is started. This can be quite tricky (there is no RTF_AFTERDOS in KS1.3)

Also rn_FileHandlerSegment most likely needs to be BCPL program entry point and assume BCPL (globvec NOT -1) behavior = dos startup packet is already fetched. (I already simulated this in pre-KS 1.2 uae filesystem support where I simply added required BCPL segments and PutMsg()'d the packet back to port..)

It would be nice (and mostly pointless) to be able to boot/read FFS floppies automatically

[Olaf Barthel]

Originally Posted by Toni Wilen

Yeah, it was in UAE directory filesystem comments that 1.3 really wants DOS\0.. I just missed it originally and thought DOS\1 was supported but not DOS\3+.

I have one more hack to do, not sure if it is possible: force KS 1.3 to use new filesystem for floppies, automatically.

I assume RootNode->rn_FileHandlerSegment needs to be replaced after dos has been initialized and before floppy handler is started. This can be quite tricky (there is no RTF_AFTERDOS in KS1.3)

This is likely going to be tricky in more than one way. The file handler segment in the root node refers to the default file system, and not every file system is fit to handle what the default file system is supposed to do.

For example, the original FFS did not support removable media, because it was designed to be used with the dominant type of hard disk at the time (ST506, etc.). SCSI hardware with removable cartridges which reported as "fixed disks" was available by 1988/1989 (SyQuest), but it was rather rare.

Supporting removable media with the patched V44 FFS could get tricky because it must be able to show those "please insert disk" and "read/write error" messages. It uses exec.library/TaggedOpenLibrary, intuition.library/EasyRequest and dos.library/Fault for that. There are no built-in error messages, because the FFS was supposed to into ROM, and space was scarce

Also rn_FileHandlerSegment most likely needs to be BCPL program entry point and assume BCPL (globvec NOT -1) behavior = dos startup packet is already fetched. (I already simulated this in pre-KS 1.2 uae filesystem support where I simply added required BCPL segments and PutMsg()'d the packet back to port..)

If I remember correctly, the V37+ FFS plays by BCPL rules, because it's the default file system, and it has to. The hard disk partitioning software available in the pre-FFS days would set up the partitions to use a GlobVec value of 0, and that had to be supported until the bitter end.

It would be nice (and mostly pointless) to be able to boot/read FFS floppies automatically

I'd say that you don't have to make it work: FFS-formatted floppy disks were something of a curio back in 1987, etc., just like DCFS-formatted floppy disks in 1992, etc.

It sure was useful in the beginning to be able to cram more data onto the disks, and reading the directory contents was faster than for OFS format disks, but there was a downside: you received no warning if the "payload" of the data blocks was damaged. That was the kind of trade-off which was not easily made.

[Toni Wilen]

Originally Posted by Olaf Barthel

TSupporting removable media with the patched V44 FFS could get tricky because it must be able to show those "please insert disk" and "read/write error" messages. It uses exec.library/TaggedOpenLibrary, intuition.library/EasyRequest and dos.library/Fault for that. There are no built-in error messages, because the FFS was supposed to into ROM, and space was scarce

All of that is already included. I don't do partial patches

Fortunately KS 1.3 ROM has all those requesters in ROM, I only needed to wrap 3 function calls to BCPL makesysreq() (which become nearly identical ErrorReport() in 2.0+)

[Toni Wilen]

It looks too painful to add any floppy filesystem hacks. I'll cancel this idea

I got as far as noticing that rn_RestartSeg is called when disk is inserted but as usual, it is very ugly BCPL code that gets dosenvec, device name and unit number as a parameter and makes no sense.

"wb 1.3 ghost icon" update now replaces original patch.

[mark_k]

What about the pre-1.3 Kickstart autoboot hack code? Could that be modified to boot FFS floppies?

[Olaf Barthel]

Originally Posted by Toni Wilen

It looks too painful to add any floppy filesystem hacks. I'll cancel this idea

I got as far as noticing that rn_RestartSeg is called when disk is inserted but as usual, it is very ugly BCPL code that gets dosenvec, device name and unit number as a parameter and makes no sense.

Hang on, where did you stop?

Kickstart 1.3 uses expansion.library (MakeDosNode, AddDosNode), right down to the "strap" module which sets up the mount lists for floppy disk drives (unlike Kickstart 1.2, for which "strap" was very simple and just showed the "please insert disk" hand).

The 1.3 "strap" module sets up the mount list without specifying the segment list of the file system, which means that dos.library will substitute the default file system. Because dos.library/default filesystem/boot shell are one big lump of BCPL code and hand-crafted assembly language, you cannot replace the default file system, but dos.library should honour what's in the mount list provided through expansion.library.

If the restart validator does kick in, it means that the default file system is still in control and has not been shut down through ACTION_INHIBIT. It should never come this far if you want to make the floppy disk drives use the modified FFS.

From what I can tell, your best chance to substitute the file system for the floppy disk drives would be in intercepting expansion.library/MakeDosNode, comparing the names of the devices and DosEnvec contents against the usual suspects ("df0", "df1", etc., "trackdisk.device", units 0-3, lowcyl/highcyl values of 0 and 79) and replacing the segment list.

If this is what you did, and the default file system still kicks in, then there might actually be a bug in the ROM...

[Amiga1992]

Can someone explain to me what the difference between this and PFS3-AIO is? Rather, what is the advantage of using it? I thought Fast File System was garbage to begin with.

[mark_k]

Originally Posted by Olaf Barthel

For example, the original FFS did not support removable media, because it was designed to be used with the dominant type of hard disk at the time (ST506, etc.). SCSI hardware with removable cartridges which reported as "fixed disks" was available by 1988/1989 (SyQuest), but it was rather rare.

This is off-topic, but do you know the first and last versions of FFS which supported 256-byte (filesystem) blocks?

SyQuest 44MB disks have 256-byte sectors. The RDB spec and at least some FFS versions supported that block size. (With others you could use a fake geometry with 512-byte blocks.) Performance with 256-byte blocks must have been pretty bad...

[Olaf Barthel]

Originally Posted by mark_k

This is off-topic, but do you know the first and last versions of FFS which supported 256-byte (filesystem) blocks?

Sorry, the recorded development history (RCS files, etc.) of the FFS, as implemented by Steve Beats, does not go any further back in time than version 34.3 (no version string, and no proper date stamp on the files, so I guess this might have been a 1987 or 1988 release). That version assumes a minimum block size of 512 bytes, there's even the telltale 'moveq #9,d1' for translating byte into block offsets (like trackdisk.device, hddisk.device, etc. prefer it).

SyQuest 44MB disks have 256-byte sectors. The RDB spec and at least some FFS versions supported that block size. (With others you could use a fake geometry with 512-byte blocks.) Performance with 256-byte blocks must have been pretty bad...

It's possible, but I somewhat doubt that a 256 byte option escaped from the lab

The original file system documentation describes the block layout with a minimum block size of 512 bytes. With the exception of the bitmap and OFS file data blocks, all file system blocks contain about 57 words of data, with the remainder constituting tables (e.g. directory hash table slots) or lists (e.g. file data blocks).

For a 256 byte block, this would mean that a directory hash table could contain only 8 entries. As far as hash tables go, you couldn't do much worse than that At 512 bytes per block, you'd have 72 hash table entries, which isn't great, but it might just get the job done.

While the data structure layout of the RDB and FFS could have permitted 256 bytes per block, such a combination would have been so constrained that any advantage the 256 byte block size might have afforded would have had to offset this disadvantage. It probably didn't increase throughput or reduce latency, so all that's left it could have contributed might have been reduced fragmentation. That's not a lot to go on...

Here's another thing which makes me skeptical: Adaptec made an adapter card in 1985 (the ACB-4000 series) which plugged into an ST506 drive and talked SCSI at the other end. This was actually used in development at Commodore, and there are still traces in the HDToolBox/scsi.device code which at a time supported this card. The ACB-4000 supports sector sizes of 256, 512 and 1024 bytes. In the traces of the HDToolBox/scsi.device code which remain you'll find that the controller is always switched hard to use 512 bytes per sector.

[Toni Wilen]

Originally Posted by mark_k

What about the pre-1.3 Kickstart autoboot hack code? Could that be modified to boot FFS floppies?

Part of it would be needed to fake the boot block read to return DOS\0 even if disk is FFS. See below.

Originally Posted by Olaf Barthel

From what I can tell, your best chance to substitute the file system for the floppy disk drives would be in intercepting expansion.library/MakeDosNode, comparing the names of the devices and DosEnvec contents against the usual suspects ("df0", "df1", etc., "trackdisk.device", units 0-3, lowcyl/highcyl values of 0 and 79) and replacing the segment list.

I'd prefer to not do any (temporary) library patches, my patches must be "clean" and "invisible"

Unfortunately strap seems to only accept DOS\0 (It reads bootblock and checks for identifier before it does AddDosNode() stuff) so doing it without patches seems to be impossible.

Originally Posted by Akira

Can someone explain to me what the difference between this and PFS3-AIO is? Rather, what is the advantage of using it? I thought Fast File System was garbage to begin with.

"NOTE: This thread is not for generic filesystem/hdtoolbox instructions."

But quick answer: for example if you already use FFS and don't bother changing it or just want to use FFS.

[Olaf Barthel]

Originally Posted by Toni Wilen

I'd prefer to not do any (temporary) library patches, my patches must be "clean" and "invisible"

Tough call for something as tightly integrated as Kickstart 1.3. There just is not enough wiggling room to cleanly inject a patch. However, if you want the system to boot from a patched FFS V40+, the patch has to be on the job by the time "strap" makes the call.

Unfortunately strap seems to only accept DOS\0 (It reads bootblock and checks for identifier before it does AddDosNode() stuff) so doing it without patches seems to be impossible.

Bummer, the signature is hard-coded Well, you can't have it all, I suppose, but the FFS V40+ is arguably a more mature implementation than both the BCPL default ROM file system and the 1980'ies FFS versions. There could be some good in being able to boot off a floppy even if it has to be DOS\0. There isn't huge practical advantage to the DOS\1, etc. versions when used with floppy disks in the first place. But being able to just read floppy disks through the patched FFS could be useful all by itself.