Idea for floppy transfer at full HD speed with hardware interface

[Hypex]

Hi guys.


I've been thinking about ways to solve the Amiga floppy problems. This can be broken up into 3 issues:


1. Connecting a common PC floppy.
2. Supporting HD disks.
3. Dealing with faster HD block transfers over slower DD block interface.


My idea would be to solve the last one. Some simple logic can solve the first two. On this, I have thought of three ways to tackle the last problem:
3a. HD floppy slow hack.
3b. HD floppy interface hardware with block cache.
3c. HD floppy interface hardware with turbo mode and driver.


The first one is common but in most cases the drive needs hacking as well.


The next is an idea I have where the drive isn't modded but instead a special interface board will cache the HD block at full speed. Which will then be sent to the Amiga at normal DD speed. This would allow drives to be connected as is. And work like the current solution with no extra drivers needed.


My final idea is the most advanced. This is a special interface that has a turbo mode. DD disks blocks would be transferred at double speed. HD disk blocks transferred at full HD speed. Both of these turbo modes would require a special driver. Technically, the data bits would be transferred at the same speed to the Paula controller, but the data bits would be reduced so they only take half the time to transfer.



You may be thinking of some type of compression being used, which could also be done, but my idea is simpler. It just strips off bits. Redundant data is stripped off. When data is read or written it is sent as a serial stream to or from the drive. And the stream is always in MFM format. However, it is only needed for the drive itself, and as data inside the Amiga MFM is redundant. So, in this case, the MFM clock bits interleaved between the actual data bits would be stripped off.


But, MFM is still needed, so how can that work? That's where the hardware interface comes in. The hardware would do the actual MFM en/coding. Similar I suppose to a PC floppy controller, but able to deal with Amiga disks, and customised for it. When floppy data is written from the Amiga it doesn't need to do MFM block encoding. They might still need some MFM Amiga track headers but the actual data blocks would be sent as direct binary data. The interface would then load the data into a cache and encode the data blocks itself as MFM then send it to the drive. When reading from drive the interface would read the MFM data into a buffer, then strip off the MFM clock bits in the data blocks. It would then send the data to the Amiga, which would have the direct data blocks already to go, to which it could copy directly to the device buffer.


So guys, what do you think? Doable? LOL. I think it would be technically possible, but at this stage in the game probably more work than is worth the end result. Some HD drives can be connected externally. Things like LS120 drives can be connected to IDE internally and transfer HD faster than HD can on a standard HD drive even. Even if Amiga disks aren't supported. Then there are flash floppy kits like Gotek.


I'm not a hardware designer to the level this would need. And others have designed different floppy interfaces. But I could do the driver.

[Bruce Abbott]

I think the first thing you need to consider is - is it worth the effort? I had an A3000 with the FB357A high density floppy, but I never bothered with HD disks. Why not? Several reasons:-

1. 800k was enough.

2. HD floppy disks were less reliable.

3. DD disks were readily available.

4. It wasn't any faster.

5. HD disks were incompatible with my other Amigas.

6. I had Ethernet for transferring files to/from the PC.

Today the situation is not much different. DD disks are not sold anymore, but HD disks are also hard to get. A few days ago I acquired 2 boxes of HD disks from an old Windows 98 Pentium PC (which a workmate brought in to run some ancient software we have to use). The disks all looked pristine and barely used, but over half of them had hard errors when reformatted.

I recently bought 200 second-hand Amiga DD floppy disks from a local computer shop. Around half of those had errors too, but mostly because they had mold growing on them due to improper storage. I have cleaned some of them and they now work fine.

The main reason I still use floppys on the Amiga is to get that true retro experience of running games from floppy (rather than using WHDload or a Gotek) or for backing up source code and transferring files when the network is down. HD disks are not required or wanted for either purpose.

My point is - floppy disks are unreliable and HD disks are even worse, so why go to a lot of trouble to recreate something that was hardly ever used and wasn't much good? People may have desired it for working with PC floppy disks, but today we have much better solutions that would have been used 'back in the day' if they were available.

However that's not to say that it wouldn't be fun trying. So the question is which option would be the best to waste your time and effort on?

3a. HD slow hack - This has the greatest compatibility but... do we really need that? it's been tried before without much success. Perhaps it would work better using a modern MCU to stabilize the motor speed, but will still be specific to each drive model. IMO this is the least interesting option.

3b. HD floppy interface hardware with block cache - may be the simplest to implement. Emulate the drive like a Gotek does, but with floppy storage instead of USB (or perhaps both!).

3c. HD floppy interface hardware with turbo mode and driver - IMO the most interesting because it could show what could have been done to give the Amiga proper HD support. But how to do it via the floppy interface? Can Paula send and receive direct binary at HD rate? If not then you need a separate bus interface.

On Amigas which have a socketed Paula, it could perhaps be put on a daughter board with extra circuitry that intercepts the data stream in HD mode, or you could even replace Paula with an FPGA which has HD support (and extra sound channels, true 16 bit DACs etc.?).

Another option would be to use a different interface such as the clock port or parallel port. In 1998 I developed an interface and driver for parallel port Zip drives. You could do something similar using a small MCU to translate the MFM data stream, similar to how Plipbox works. Drive control signals could be connected directly to parallel port I/O pins for low level control of the drive mechanics. Or you could use a combination of the existing floppy drive port for control signals and power, plus the parallel port for data transfer.

An even more 'retro' method would be to interface a vintage HD floppy controller chip to the parallel port (or clock port, Zorro II bus etc.). I might be interested in this because I have several different floppy controller chips that I want to learn how to operate for other retro projects.

I think using the Amiga's existing external port(s) would be the easiest and most universal way to provide HD floppy drive support. Most users today don't use the parallel port for printing, so it is free for other stuff. A driver is required, but you need one anyway for PC floppy formats, and games that need 100% hardware compatibility don't work with HD format anyway.

[NorthWay]

There was a non-Commodore HD floppy solution sold; does anyone know what they did? (I suspect it might have been a track cache.)

[hooverphonique]

Originally Posted by Hypex

But, MFM is still needed, so how can that work? That's where the hardware interface comes in. The hardware would do the actual MFM en/coding. Similar I suppose to a PC floppy controller, but able to deal with Amiga disks, and customised for it. When floppy data is written from the Amiga it doesn't need to do MFM block encoding. They might still need some MFM Amiga track headers but the actual data blocks would be sent as direct binary data. The interface would then load the data into a cache and encode the data blocks itself as MFM then send it to the drive. When reading from drive the interface would read the MFM data into a buffer, then strip off the MFM clock bits in the data blocks. It would then send the data to the Amiga, which would have the direct data blocks already to go, to which it could copy directly to the device buffer.

Wouldn't your "cache" need the clock bits in order to extract the clock of the data stream when writing (and vice versa for Paula when reading) ?

[nogginthenog]

Originally Posted by NorthWay

There was a non-Commodore HD floppy solution sold; does anyone know what they did? (I suspect it might have been a track cache.)

Power Computing made one. It used a slim IBM (laptop?) drive with a blue eject button. Reading would work fine but writing needed a patch.

As it used an IBM drive I assume it used a cache. I think the A4000 drives were custom?

[ElectroBlaster]

I have an Amitek drive that was built around a Toshiba laptop drive. The techy wizard gubbins was at the plug end, on a small pcb. Proper heath-robinson style secured with hot glue. This allowed the usage of HD disks without any drivers or patches.

The thing is back then it was a novelty! but in reality I never really used it.

I have amiga 4000's and all of them have HD disk drives. Again I have yet to actually format an HD disk I just have not bothered lol.

What I would like to see, is a disk drive that plugs in via usb that is seen by Winuae as a true bonfide amiga disk drive! as in it clicks waiting for a disk, the emulation sees it as an internal drive and you place a disk in it, it loads. Or is seen by things like Xcopy just for the hell of it!

Disks are rotting, some hate the things but for me the nostalgia is the thing.

If I had the brains I would try and create something, but alas I dont lol.

[nogginthenog]

Now you mention it I'm not sure I've used a HD floppy in my A4000 in the last decade. I even removed the CDROM too because I never use it.

On my A500 I would have used the external drive for college work.

[pipper]

Having a simple adapter between the Amiga and a regular HD floppy drive would certainly catch most attention. It would give people an opportunity to replace broken Amiga drives with relatively cheap PC floppy drives without any fuss and on top add the novelty of an HD drive on the Amiga. Not that you need one, but it’s certainly cool to have one :-)

[Thomas Richter]

Originally Posted by Hypex

1. Connecting a common PC floppy.

There is a team working on this as far as I know.

Originally Posted by Hypex

2. Supporting HD disks.

"Supporting" as in "being able to reliably write on them" might be possible, "supporting them" as in "support HD density" is a lot harder.

Originally Posted by Hypex

3. Dealing with faster HD block transfers over slower DD block interface.

That is the tough part.

Originally Posted by Hypex

The next is an idea I have where the drive isn't modded but instead a special interface board will cache the HD block at full speed. Which will then be sent to the Amiga at normal DD speed. This would allow drives to be connected as is. And work like the current solution with no extra drivers needed.

As long as supporting copy protection is not an issue, this may work.

Originally Posted by Hypex

My final idea is the most advanced. This is a special interface that has a turbo mode. DD disks blocks would be transferred at double speed. HD disk blocks transferred at full HD speed. Both of these turbo modes would require a special driver. Technically, the data bits would be transferred at the same speed to the Paula controller, but the data bits would be reduced so they only take half the time to transfer.

Errrr.... MFM is a 1:2 expansion. That is, if you leave out the clock bits, the code space is "full". That means that there is no room left for sync words. Thus, all encoding and decoding would have to happen on the interface, probably including some overhead to indicate the size of a track.

Originally Posted by Hypex

So guys, what do you think? Doable?

Well, doable with "quite a bit of work", but IMHO a bit pointless. Actually, I do not recall when I have used a floppy the last time - probably to load an old game? Well, that is all on DD. Or on a floppy emulator, working with thumb drives or solid state media. Such devices exist, of course.

[Galahad/FLT]

I just can't see how this is in 2020 is relevant.

20 years ago, a great idea, but sourcing decent quality new disks is getting more and more difficult as time goes on.

Chris Wilkins had to scour eBay to get the required 100 disks for Starquake, and took a chance on new old stock.

Stuff like Gotek and similar is the future for "disk" based game playing, physical disks, a collectors item now and not much beyond.

[Hypex]

Originally Posted by Bruce Abbott

I think the first thing you need to consider is - is it worth the effort? I had an A3000 with the FB357A high density floppy, but I never bothered with HD disks. Why not? Several reasons:-

You would have read my final comment about if it would be worth it. I guess it would be a little like AGA. Too little, too late.


One issue that does come up is if you substitute a HD disk for a DD disk it will be unreliable or cause problems and need the "DD hack" with sticker over the hole. That gets annoying after a while. Some disks don't work well in Amiga HD drives and an extra DD drive is needed.


I've got a couple of Verbatim ten packs I've had for years. They work well enough. But, being HD, they only work well as HD or as DD in a DD only drive.


I still use floppies as I have a collection. I recently backed up my personal files. And sometimes some software I have is on floppy.



I also use them to transfer between a PC at times. For this I bought a USB floppy drive. HD disks work better as I can store more data. If I transfer a disk image it can fit on HD disk where compressing an ADF to a 720K disk doesn't always work.



I've got a DVD drive in my A4000 as well but it's slow to write from the Amiga. And writing a small amount of data can take a while. It can be faster to just use a floppy for a small amount of files.

My point is - floppy disks are unreliable and HD disks are even worse, so why go to a lot of trouble to recreate something that was hardly ever used and wasn't much good? People may have desired it for working with PC floppy disks, but today we have much better solutions that would have been used 'back in the day' if they were available.

Mainly for the reasons listed above. And also to right a wrong. Commodore should have kept the floppy interface up with modern standards but they didn't. Amiga HD disks should have become standard. But all they did was hack a HD floppy so it could work. Leaving most Amigas with DD drives. At least it does support DD/HD Amiga format and PC format, I'll give them that.

However that's not to say that it wouldn't be fun trying. So the question is which option would be the best to waste your time and effort on?

Like anything increasing in complexity from one level to another is usually best. Confirming proof of concept. Before going all in. So the block cache would be best in that regard since it's a more simpler idea with best compatibility.

3a. HD slow hack - This has the greatest compatibility but... do we really need that? it's been tried before without much success. Perhaps it would work better using a modern MCU to stabilize the motor speed, but will still be specific to each drive model. IMO this is the least interesting option.

This is the most common solution in an A4000. I use it. But I also had trouble with PC disks for a long time until a break was found in the sync wire of my floppy cable.

3b. HD floppy interface hardware with block cache - may be the simplest to implement. Emulate the drive like a Gotek does, but with floppy storage instead of USB (or perhaps both!).

Yes that's the general idea.

3c. HD floppy interface hardware with turbo mode and driver - IMO the most interesting because it could show what could have been done to give the Amiga proper HD support. But how to do it via the floppy interface? Can Paula send and receive direct binary at HD rate? If not then you need a separate bus interface.

To my knowledge Paula can only transfer at a constant rate. Or some range defined by the CIA chips since it needs to go through a serial to parallel conversion. Before DMA can get hold of it.


That's where my idea of stripping bits comes from. The block data of 512 bytes is doubled with the clock bits. Those clocks bits use a common algorithm to prevent too many runs of one bit. So they would be stripped on the way out by the driver. And on the way in from the floppy the hardware interface would strip them out as well.

On Amigas which have a socketed Paula, it could perhaps be put on a daughter board with extra circuitry that intercepts the data stream in HD mode, or you could even replace Paula with an FPGA which has HD support (and extra sound channels, true 16 bit DACs etc.?).

I thought about that. But there is a lot more involved with all the custom chips. That could be more complicated even though it simplifies the connection of the floppy.

Another option would be to use a different interface such as the clock port or parallel port. In 1998 I developed an interface and driver for parallel port Zip drives. You could do something similar using a small MCU to translate the MFM data stream, similar to how Plipbox works. Drive control signals could be connected directly to parallel port I/O pins for low level control of the drive mechanics. Or you could use a combination of the existing floppy drive port for control signals and power, plus the parallel port for data transfer.

Connecting an external Amiga floppy via parallel instead of the disk port would be interesting. Would still need an interface but could be easier to do. The driver and hardware would be simplified. Though it would need some serial conversion at the drive end.



But it's funny you mention that. As I've been investigating parallel port floppy drives for some time now. A friend sent me an LS120 to investigate if can be connected to an Amiga. However he found a bargain LS120 that lacked any cables, except some stubby micro centronics to parallel port adapter, to which won't plug into a parallel port. I figured out a PSU but I cannot figure out the parallel cable needed, These days parallel cables are very rare to find. Printer cables more easy still. So far I cannot find any info on this cable. Can not find anything on the internet. I sourced a straight through cable and it just makes the drive go dead. I can only conclude the cable is custom and supports daisy chaining to which I also found no info on. I haven't yet got to the Amiga as this all with a PC parallel interface .Well, close enough, the only computer in the house including a desktop PC with an actual parallel port is my AmigaOne XE! :-D



I was recently looking up your PPAZip diagram and cannot figure out anything special in the lines. All I can see on the joystick port is going to printer control lines. But when I looked the Amiga already had these lines. So I just don't get it! :-P



I've read the Amiga parallel port described as "inferior" compared to the PC parallel port. I tend to disagree. Because it is usually is comparing a basic port from 1985 with a port from 1994 with DMA. Almost ten years difference. Well, two different standards introduced in 1991, 1992 and finally standardised in 1994. That is not a level comparison. However, it wasn't until 1987 that SPP ports became common on the PC, with bidirectional lines. They existed but were unpopular and removed from some PCs. The Amiga port has always been bidirectional. Because of this, popular cables like "laplink" are crippled. Laplink only uses 5 bits at a time,a popular PC cable that is "inferior" to the Amiga port. And Amiga parallel link up system can use all 8 bits at a time. The Amiga port roughly follows the SPP standard. Unfortunately, since Commoodore never updated the CIA chips, despite even floppy using DMA, the Amiga parallel port was left crippled.




Most parallel drives would require an ECC/ECP port for operation. Or operation at decent speed. The LS120 uses a Shuttle EPAT interface as listed down here. I've examined the source. It attempts to connect with all common protocols, both DMA and older SPP and below. So that gave me hope an SPP driver could be written. So it can plug straight in. Or that's the hope anyway. ;-)


http://cyberelk.net/tim/parport/paride.html

An even more 'retro' method would be to interface a vintage HD floppy controller chip to the parallel port (or clock port, Zorro II bus etc.). I might be interested in this because I have several different floppy controller chips that I want to learn how to operate for other retro projects.

That could also be a good idea using an "off the shelf" chip. It takes some of the work out. It would be limited to PC only but still useful that way.

I think using the Amiga's existing external port(s) would be the easiest and most universal way to provide HD floppy drive support. Most users today don't use the parallel port for printing, so it is free for other stuff. A driver is required, but you need one anyway for PC floppy formats, and games that need 100% hardware compatibility don't work with HD format anyway.

I was thinking internal because I like neat solutions. It would be good to have. But the average A1200 or A500 user likely wouldn't need it.

[Hypex]

Originally Posted by hooverphonique

Wouldn't your "cache" need the clock bits in order to extract the clock of the data stream when writing (and vice versa for Paula when reading) ?

They would be needed, yes. The interface would create the clock bits on the way out to the drive. The general way is to split a data long word up into odd and even bits, create clock bits by inverting the data, then merging the result with the output being an odd and even long word. Thus, each long word of data, becomes two long words when sent to disk.


The write operation would need to generate this on the fly. So would need to buffer the full MFM data. Like unpacking from one buffer to another.


A read would decode data words so only actual data remains. The 1024 bytes of encoded MFM block data would be decoded to 512 bytes of direct data.


But you make a good point, does Paula need the clock bits? The hardware does detect MFM syncs like 4489 but AFAIK only uses that feature for PC disks.

[Hypex]

Originally Posted by nogginthenog

Power Computing made one. It used a slim IBM (laptop?) drive with a blue eject button. Reading would work fine but writing needed a patch.

As it used an IBM drive I assume it used a cache. I think the A4000 drives were custom?

Yes they were. Mine has one. It modifies the mechanism so it only runs at one speed regardless of density. I suppose that should make HD disk more reliable on the Amiga.

[Hypex]

Originally Posted by ElectroBlaster

I have amiga 4000's and all of them have HD disk drives. Again I have yet to actually format an HD disk I just have not bothered lol.

It's easy to do. Stick in a HD disk. Format on Workbench. Done.



For PC disks I installed Fat95 as I don't like the old fashioned 8.3 format you get with the old Commodore driver.

What I would like to see, is a disk drive that plugs in via usb that is seen by Winuae as a true bonfide amiga disk drive! as in it clicks waiting for a disk, the emulation sees it as an internal drive and you place a disk in it, it loads. Or is seen by things like Xcopy just for the hell of it!

There's been some projects like that but none made it to completion AFAIK. Connecting an Amiga drive to a PC would be more desired than connecting a PC drive to an Amiga.



Another idea I had was a backwards compatible PC compatible Amiga format disk. This would be a disk formatted with 11 PC style sectors per track, or 11 on a HD disk. Since the PC could format disk to other sector sizes. But it wasn't popular to do so. I recall experimenting with the PC FORMAT command, but only under PC-Task, so had no proper results.

[Hypex]

Originally Posted by nogginthenog

Now you mention it I'm not sure I've used a HD floppy in my A4000 in the last decade. I even removed the CDROM too because I never use it.

On my A500 I would have used the external drive for college work.

I may removed a CDROM fro my Amigas. But only because you can't write to it. So I put in a DVDRW drive in its place.

[Hypex]

Originally Posted by pipper

Having a simple adapter between the Amiga and a regular HD floppy drive would certainly catch most attention. It would give people an opportunity to replace broken Amiga drives with relatively cheap PC floppy drives without any fuss and on top add the novelty of an HD drive on the Amiga. Not that you need one, but it’s certainly cool to have one :-)

That's the idea. Of course I went further. Since if you insert a DD formatted HD disk it breaks. You can't use HD disks directly. And are limited to DD.


The other thing is most PC disks are HD. So it helps to use a HD drive as HD when transferring data.

[Hypex]

Originally Posted by Thomas Richter

There is a team working on this as far as I know.

There are adapters in existence already you can buy. I even have an old PCB someone made up. As you would likely be aware it comes down to converting Shugart bus to PC, drive ready and drive ID.

"Supporting" as in "being able to reliably write on them" might be possible, "supporting them" as in "support HD density" is a lot harder.

Yes it is. The common hack makes no provision for density. So all HD disks tend to break and especially Amiga DD formatted HD disks.

That is the tough part.

Which is what I was thinking about.

As long as supporting copy protection is not an issue, this may work.

In this case, it would be best to disable the driver, so the data is passed through. It would really need to be transparent. At reset it would be disabled.

Errrr.... MFM is a 1:2 expansion. That is, if you leave out the clock bits, the code space is "full". That means that there is no room left for sync words. Thus, all encoding and decoding would have to happen on the interface, probably including some overhead to indicate the size of a track.

My idea would be to use the interface for this. But possibly leaving the track header as encoded so checksums could be verified. So the only data stripped would be the 512 byte blocks.

Well, doable with "quite a bit of work", but IMHO a bit pointless. Actually, I do not recall when I have used a floppy the last time - probably to load an old game? Well, that is all on DD. Or on a floppy emulator, working with thumb drives or solid state media. Such devices exist, of course.

It's mostly useful when you need to share data. If you only have HD disks and no sticky tape, which has happened to me, then the cheap HD hack is no good. Then again, an Amiga HD drive is about as good with any HD disk. So an external Amiga DD drive is good as a backup.

[Hypex]

Originally Posted by Galahad/FLT

I just can't see how this is in 2020 is relevant.

Is the Amiga relevant at all in 2020?

[hooverphonique]

Originally Posted by Hypex

But you make a good point, does Paula need the clock bits? The hardware does detect MFM syncs like 4489 but AFAIK only uses that feature for PC disks.

I'm pretty sure it does because it has no other way of getting a clock for sampling the data correctly.
4489 *is* used for amiga format disks, too, yes, so Thomas has a point about "full code space" will make something like sync-matching impossible.

[Galahad/FLT]

Originally Posted by Hypex

Is the Amiga relevant at all in 2020?

Clearly as its still being used and upgraded.

Floppy disks? Not so much