From the vaults: Why was Workbench 1.1-1.3 so slow when reading disk/drawer contents?

[Olaf Barthel]

If you have been around long enough, you will remember how slow the Amiga floppy disks were in the days of Kickstart 1.1 through 1.3.

Part of that was caused by the floppy disk drive itself, but there is another reason which you may not know about. That reason also explains why Workbench, and in fact any Amiga program reading the list of directory entries, was slow at reading drawer contents on hard disk drives.

The default Amiga file system in Kickstart 1.1-1.3 responds to two operations which perform the directory scanning. First you need to tell it to begin looking at the first entry of a directory, then you ask for the next following entry until the list of entries is eventually exhausted.

Directories may contain both directories and files, of course, and each such entry features a name, a modification date and time, a comment (which may be empty), protection flags and an indication of how many disk blocks the respective entry occupies. For directories that number of blocks tends to be 1. For files the total file size in bytes as well as the number of blocks involved to store the file data is counted.

Now imagine, if you will, you inserted both your "Deluxe Paint II" boot disk and the "Art disk" into your Amiga floppy drives. Workbench boots up, you open the "Deluxe Paint II" boot volume by double-clicking its disk icon and it takes forever to show one big icon. A closer look shows that the disk does not contain a lot of files, just the usual set of directories for fonts, etc. and icon files for art drawers and the program itself. Why does it take so long to process this?

The reason is that it takes longer to examine a larger file when walking through the directory entries than it takes to examine a smaller file. The threshold for floppy disks and hard disk drives which do not make use of the FFS (part of Workbench 1.3) is 35,136 bytes. If a file is shorter than that, it can be examined quickly because only a single block is necessary to learn all about it worth reporting. But if it is longer than that, something peculiar happens.

Once a file becomes longer than 35,136 bytes (equivalent to 72 blocks, with a block payload size of 488 bytes) the file system will need extra storage to keep track of the data blocks which make up the file's contents. That extra storage is known as the "file list" (in the original documentation) or the "extension block list" (as per the Amiga Guru Book), and it is attached to the block which contains the file's name, size, etc.

When the Kickstart 1.1-1.3 default file system examines a file as part of the directory scanning process, it will walk down the "file list" blocks referenced, loading each from disk and picking up the next one to follow it, as indicated by the contents of the previous "file list" block, pointing to the next chain entry. It is doing so in order to count the number of "file list" blocks. That takes its time.

"Deluxe Paint II" is 189,740 bytes in size. 389 blocks data blocks are needed to store it. Of these, 317 have to be managed through the "file list", which consists of 5 blocks. Merely scanning the directory entry thus takes at least 5 times longer than checking out a directory entry of a shorter file. Due to fragmentation, you may have to wait even longer, with the floppy disk's read/write head moving back and forth.

Funny thing about all of that: the file system sort of "validates" the "file list" blocks when you are only interested in the name, size, etc. of the file. The extra time spent is also unnecessary. It is possible for the Kickstart 1.1-1.3 default file system to calculate the number of "file list" blocks from the file size alone. Walking down the list, one block at a time, is not even a requirement. In fact, the FFS (Workbench 1.3) does it exactly like that and does not walk the "file list" blocks just in order to to count them.

[coldacid]

Yet another reason to avoid OFS as much as possible!

[grond]

Doesn't that mean that OFS could be patched to avoid this behaviour? What other changes make FFS fast in comparison to OFS?

[Olaf Barthel]

Originally Posted by coldacid

Yet another reason to avoid OFS as much as possible!

In 1986-1987 the only way to avoid using the default ROM file system was not to switch on your Amiga in the first place ;-)

However, the CDTV finally gave you that option: boot from the CD.

[Olaf Barthel]

Originally Posted by grond

Doesn't that mean that OFS could be patched to avoid this behaviour?

Only if you hacked the ROM and knew exactly what to look for. I suppose back in the day the number of people who knew about this "feature" could be counted on the fingers of one hand, and you would not even need your pinky for that.

What other changes make FFS fast in comparison to OFS?

The data blocks are no longer "protected" by checksums, which means that you can directly read/write consecutive sequences of these without having to fix up or check the checksums. DMA-enabled disk controllers would do this perfectly well.

Storage blocks are allocated in sequence, if possible. You tell the FFS how much data you need to store and it will try to find the largest sequence of consecutive blocks available to store them. This also has a positive effect on block fragmentation. It's not that great, though, but better than nothing at all.

The data structures used for wrangling directory entries are sorted by ascending block order, which means that when reading a directory, the read/write head of your floppy disk drive or hard disk will move only in one direction instead of weaving back and forth.

That's a small list of what went into making the FFS more efficient. I probably forgot a thing or two. But the work which went into the FFS in 1986/1987 was worth it.

[desiv]

Originally Posted by Olaf Barthel

If you have been around long enough, you will remember how slow the Amiga floppy disks were in the days of Kickstart 1.1 through 1.3.

As someone who came from a C64 with a Commodore 1541 drive and no Fastload cart...

No... I hadn't really noticed that...
;-)

[Thomas Richter]

Originally Posted by grond

Doesn't that mean that OFS could be patched to avoid this behaviour?

One should probably clarify that what Olaf describes applies to the original OFS implementation only. Today's OFS implementation is just a variant of the unified ROM file system that implement both FFS and OFS in one code.

Originally Posted by grond

What other changes make FFS fast in comparison to OFS?

Most importantly, the language. The OFS was BCPL, FFS is assembler. If I recall correctly, FFS at some point also implemented the possibility of combining sectors into blocks, but got away with the "interleave" factor OFS still implemented. That is, OFS had the option to allocate sectors not consecutively, but with a particular interleave factor to compensate for slow harddrives.

[tygre]

Thanks Olaf!

"From the Vaults"... reads like a nice book title

Cheers!

[itsmedoofer]

Really interesting read, many thanks Olaf, really appreciated.

A belated thanks for Term as well, I shudder to think how many hours I spent in front of that on the BBSs

[malko]

Originally Posted by tygre

thanks olaf!
"from the vaults"... Reads like a nice book title
cheers!

+1

[Bruce Abbott]

Originally Posted by Olaf Barthel

If you have been around long enough, you will remember how slow the Amiga floppy disks were in the days of Kickstart 1.1 through 1.3.

I remember those days. I also remember that you could dramatically speed up disk operations with some simple optimizations.

Now imagine, if you will, you inserted both your "Deluxe Paint II" boot disk and the "Art disk" into your Amiga floppy drives. Workbench boots up, you open the "Deluxe Paint II" boot volume by double-clicking its disk icon and it takes forever to show one big icon. A closer look shows that the disk does not contain a lot of files, just the usual set of directories for fonts, etc. and icon files for art drawers and the program itself. Why does it take so long to process this?

To be precise, the Deluxe Paint II boot disk contains 113 files and 25 directories. On My WB1.3 A500 it took 6 seconds to open the disk and display the 9 icons. After putting "addbuffers 30" in the startup-sequence it took 4 seconds (which is fast enough that you don't feel like it's dragging). If the disk was optimized for icons it would probably be even faster.

Funny thing about all of that: the file system sort of "validates" the "file list" blocks when you are only interested in the name, size, etc. of the file. The extra time spent is also unnecessary. It is possible for the Kickstart 1.1-1.3 default file system to calculate the number of "file list" blocks from the file size alone. Walking down the list, one block at a time, is not even a requirement. In fact, the FFS (Workbench 1.3) does it exactly like that and does not walk the "file list" blocks just in order to to count them.

'Possible' for a filesystem handling OFS disks, but of course not possible with 1.1 - 1.3 ROMs because they were set in stone - unless you had an A1000.

There were other inefficiencies too, like the redundant block checksums wasting space and using up CPU time, and all those cli commands that had to be loaded from disk every time they were used.

But this just created an incentive for people to find more ways to optimize their disks. Instead of being a chore, it became a game more enjoyable and addictive than many actual games that were pointless exercises in frustration. And Commodore kindly provided one of these 'games' with every Amiga - the Workbench disk!

We should have a competition to see who can make their stock 68000 machine with 1.1-1.3 boot off floppy to Workbench and open it to show the icons in the least time.

[E-Penguin]

I always wondered why 1.3 was so slow to load compared to later versions.

[malko]

Originally Posted by E-Penguin

I always wondered why 1.3 was so slow to load compared to later versions.

Same for formatting a disk...
but again BitD we had no point of comparison, so it was normal

[Tigerskunk]

Interesting stuff.

Thanks Olaf...

[Dunny]

Originally Posted by desiv

As someone who came from a C64 with a Commodore 1541 drive and no Fastload cart...

No... I hadn't really noticed that...
;-)

Same - Spectrum 128k +2 with tapes. AmigaOS 1.2 was lightning fast.

[fryguy]

Interesting stuff!

[shelter]

I only used WB2.x and up, I did have friends with Amiga 500 though but we rarely used Workbench. But it was an interesting read.

I do remember that I switched to AFS from FFS though. And later PFS, which I still use today.

[kremiso]

the optimize option in x-copy have helped me in many cases bitd