Best Amiga performance via storage cache

[AmigaHope]

I wanted to share something here because it's been overlooked a lot by a lot of people. Most Amiga setups have very poor mass storage bandwidth (it seems most people are using a Gayle-style IDE interface) despite having fast processors coupled with fast memory access.

AmigaOS, all the way from 1.0 to 3.1.4/3.9 has had a long of history of poor mass storage caches. The OS implements a read-only cache system, and by default most people never use it to its full potential. This has its roots in the 1980's era of RAM-starved systems, where buffers were kept lean to avoid excessive memory allocation.

Floppy drives, hard drives, flash memory, whatever you use have a read-only buffer. On floppy drives this is configured automatically, and on mass storage this is configured in your mountlist (with the Buffers line) or on the autoconfig mountlist stored in RDB (also Buffers line). You can easily enlarge this using the AddBuffers command. If you have a modern Amiga with a lot of memory, just make this HUGE. e.g. if your system has 128M of memory, give your partitions 10M+ each of buffers and you'll notice the performance improvement.

Even better, use a more modern writeback buffer system like PowerCache or FDA. Your workbench performance will improve by a factor of 10, and any heavily storage-interactive software will improve similarly. You'll also see this performance improvement in games that use the OS routines (whether floppy or hard disk).

The reason for this is that, without exception, the performance on Amiga mass storage controllers is terrible. It's particularly terrible if you're using one on the Amiga's motherboard bus (e.g. A1200 IDE controller, most Gayle emulators, etc.), and even on fast IDE controllers like FastATA the bandwidth is bad and the CPU usage is huge. Even if you have a fast DMA SCSI controller on your CPU card (e.g. Cyberstorm) there's still a bottleneck. None of it can compete with RAM.

The crazy thing is that installing Powercache or FDA can even somewhat improve performance on WinUAE where all the devices are virtualized and executed out of RAM on the host system. The latency improvement helps even then.

If you have an Amiga with some RAM to spare, even sparing a few megabytes on a 32M system, give improving the caches a try. You'll notice the difference immediately. Even if you just give it a few megabytes.

On my A4000 with CyberstormPPC using UltraSCSI/160 drives on the DMA bus, it still made a night and day improvement in general operations.

[Valken]

Can you do some benchmarks with and without READ BUFFER, PowerCache, and FDA.

It would be interesting.

I just checked Aminet and cannot find PowerCache or FDA. Anybody have a spare copy?

[AmigaHope]

Originally Posted by Valken

Can you do some benchmarks with and without READ BUFFER, PowerCache, and FDA.

It would be interesting.

I just checked Aminet and cannot find PowerCache or FDA. Anybody have a spare copy?

Can't with my physical Amigas as they're at my parents' house 1500km away. Could set up with WinUAE but that would take some time. The WinUAE benefit is much smaller (again as it all gets cached by the host OS's RAM) but from experience the writeback-cache enabled setups do provide noticeable improvements even in this situation. It's more like a 1.5X speedup instead of a 10X speedup. The WinUAE speedups are entirely from latency rather than bandwidth gains (since Windows/Linux native disk caching already takes care of bandwidth).

Most of my experience is with PowerCache not FDA.

PowerCache: http://aminet.net/package/disk/cache/PowerCache
FDA: http://aminet.net/package/disk/cache/fda

I don't know where to find the cracks right now (though I used a cracked PowerCache constantly for many years). FDA I didn't use too much back in the day even though it's newer.

I'm sure you can appreciate though that AmigaOS default read cache sizes were tiny (fixable with Addbuffers) , and that it had no writeback cache (hence the need for no "shutdown" command to flush buffers).

Writeback caches help a *LOT* by unblocking software instead of waiting for the disk operation to complete. This is what helps WinUAE -- it saves many cycles of latency between the emulated environment and the host OS's disk caching even though the disk operation is super fast on a modern x86 system.

EDIT: The reason I posted this is that I realized a lot of people are using A1200s with compactflash cards on the Gayle interface. Watching on Youtube I see their workbench performance suffer because of this -- even with the effectively-zero latency of the flash card they're still held back by waiting for I/O operations to complete through the soda straw of Gayle. It's the difference between waiting for a second or two to copy disk images to the process completing in milliseconds (with the operation finishing behind the scenes while you move on). It seems like a small amount of time to wait, but it's huge from a user interface perspective. It's why everything seems so snappy on Windows/Linux.

[Kyle_Human]

Linking Aminet is useless, all files in "cache" directory are missing and have been for an unknown amount of time.

[zipper]

http://grandis.nu/eabsearch/search.p...xclude=&limit=

[Hedeon]

Write caches on a system without memory protection is something I don't take a risk on :-) But to each his own.

[Kyle_Human]

If programs are trashing memory that doesn't belong to them, your computer is going to crash. So you'd have discovered them long before installing a drive cache.

[Locutus]

Originally Posted by Kyle_Human

If programs are trashing memory that doesn't belong to them, your computer is going to crash. So you'd have discovered them long before installing a drive cache.

How optimistic...


And i guess you also don't care about potentially silently writing out invalid data or just completely losing writes from unflushed caches on a system as stable as AmigaOS.


No sorry, i don't think the Amiga was ever something i'd dare to run a write cached filesystem on either.

[Phantasm]

Originally Posted by Locutus

How optimistic...


And i guess you also don't care about potentially silently writing out invalid data or just completely losing writes from unflushed caches on a system as stable as AmigaOS.


No sorry, i don't think the Amiga was ever something i'd dare to run a write cached filesystem on either.

all of the data comes from memory before going to disk anyway, so while it does increase the risk with it being held in memory for longer, if something is randomly corrupting memory it's still very possible to be writing out corrupt data however you look at it.

[Bruce Abbott]

Originally Posted by AmigaHope

The reason for this is that, without exception, the performance on Amiga mass storage controllers is terrible. It's particularly terrible if you're using one on the Amiga's motherboard bus (e.g. A1200 IDE controller, most Gayle emulators, etc.)

The MFM hard drive on my A1000 got about 100kB/s write speed. At the time I thought it was pretty good (1:1 interleave, most PCs needed 3:1 or 5:1). Compared to that, over 1MB/s on my A1200 is excellent.

If you have a modern Amiga with a lot of memory, just make this HUGE. e.g. if your system has 128M of memory, give your partitions 10M+ each of buffers and you'll notice the performance improvement.

I only have 32MB of RAM and 5 partitions, so 10M+ on each is impossible. But if I did have 128MB I wouldn't waste half of it on caching the hard drive (that's what PCs do, and I hate it). I have 300 buffers (150kB) on each partition and it's plenty.

Writeback caches help a *LOT* by unblocking software instead of waiting for the disk operation to complete. This is what helps WinUAE -- it saves many cycles of latency between the emulated environment and the host OS's disk caching

I implement write caching (when necessary) in my applications. I don't have to worry about the host OS's disk caching because I am running real Amiga hardware! No virtual memory sucking up all the free RAM and swapping it out to the hard drive, only to swap it all back again later!

[Daedalus]

Originally Posted by Kyle_Human

Linking Aminet is useless, all files in "cache" directory are missing and have been for an unknown amount of time.

Those links (and the cache directory) appear to be working fine here...

[kriz]

Yes, both links work fine here also.. Btw: at the time of the post the files could not be found, looks like someone fixed that directory as all files works fine now..

[daxb]

Originally Posted by Bruce Abbott

I have 300 buffers (150kB) on each partition and it's plenty.

On other systems it might be not enough. It depends on filesystem and partition size/amount of files what buffer size is a good choice.

[r.cade]

Originally Posted by Bruce Abbott

I only have 32MB of RAM and 5 partitions, so 10M+ on each is impossible. But if I did have 128MB I wouldn't waste half of it on caching the hard drive (that's what PCs do, and I hate it). I have 300 buffers (150kB) on each partition and it's plenty.

"Wasting?". What is it that it's needed for, if not working to make your life faster or easier? You like it all to sit there unused, just "ready to go"? It's not like the AmigaOS uses virtual memory that slows it down when you run low.

I always tell my coworkers- "What are we paying these computers to sit around for?"

[daxb]

I think with >= 64MB fastram you can start waste some megabytes. I was in the same situation with my 32MB fastram system. There isn't much room for spending MBs of ram on such a low ram system. It's easy to get out of memory situations.

[kriz]

Originally Posted by Daedalus

Those links (and the cache directory) appear to be working fine here...

It got fixed, here is a thread about it from Amibay: http://www.amibay.com/showthread.php...rvice-problems

"disk/cache has been restored, wait for a bit for all the mirrors to catch up. Checks are being done to see if other packages are missing from the mirrors as well. "

[Kyle_Human]

The reason why the files on Aminet are now available is because I had them fixed. That's my thread.

I installed a cracked powercache from the FTP server a few days ago. It's such an old bit of software there is a bit of buggy behavior I've noticed. My DH1: is a 2GB PFS partition, and setting the automatic cache to "large" crashes the program when it tries to calculate how big it'd be. My DH0: is 2GB FFS and doesn't provoke that behavior.

Other than that it boots noticeably faster. Haven't noticed much improvement browsing the disks, haven't had any instability or corruption. I should probably try some real world benchmarks archiving big LHAs or something.

This is with FastATA 1200 in PIO5 and an 030 40Mhz.

[AmigaHope]

Originally Posted by r.cade

"Wasting?". What is it that it's needed for, if not working to make your life faster or easier? You like it all to sit there unused, just "ready to go"? It's not like the AmigaOS uses virtual memory that slows it down when you run low.

I always tell my coworkers- "What are we paying these computers to sit around for?"

This. A good disk cache steps out of the way in low memory situations but ensures that memory that is just sitting there gets used for other useful purposes. AmigaOS's flat memory model means you can't go all-in on this method (otherwise you'd get massive memory fragmentation) but the idea is still sound.

Other than that it boots noticeably faster. Haven't noticed much improvement browsing the disks, haven't had any instability or corruption. I should probably try some real world benchmarks archiving big LHAs or something. This is with FastATA 1200 in PIO5 and an 030 40Mhz.

You'll notice the biggest performance improvements in multitasking situations where you have lots of reads and writes going on in rapid succession (which is why it helps during boot). Especially if you have some sort of big batch operation going on in the background while you try to do other stuff.

Note that since you have a FastATA you're a bit less bandwidth-limited than plain Gayle users, and you might notice your CPU performing a bit *slower* because the background I/O will be making more heavy use of the FastATA (instead of software blocking on writes) but overall stuff will complete faster and smoother.

On my A4000 I noticed performance improvements even using the Ultra SCSI controller on my Cyberstorm PPC (best case scenario on classic Amiga), but they were small -- essentially stuff ran slightly smoother in multitasking. Anytime I started to do work on the A4000's onboard IDE controller though, it became like night and day. Stuff stopped blocking on the horrible bottleneck.