Storing data on a floppy?

[Olaf Barthel]

Originally Posted by ross

The calculations are correct, but the prerequisites should be different.
You should not consider the percentage of increase significant based on the raw data of the sectors, but on the parameters that I specified before: cell duration, drive rotation speed, supported tolerance.

Let's take a practical example: an Amiga with an NTSC crystal clock and a standard 300RPM floppy.
I took the NTSC case because it is worse than PAL, the cell duration is lower and therefore more data under the same conditions can be stored on the track.

A cell is 1/3579545*7=1.95556us
If the floppy disk rotates perfectly at 300RPM the storable data is (60/300)/1.95556~=102273 cells
(you can immediately notice that on Amiga this value is greater than the standard one, where the cells expected are 100000)

The raw MFM data are then ~12784 bytes [cells/8].
Now let's add to this value the maximum we want to tolerate as a speed difference of a drive installed on an Amiga computer.
5%? This is REALLY a super-bad drive...

12784+5%=13423 rounded to 13424 (our buffer, for DMA words handling)

This is now our reference value where we need to 'insert' our raw payload of Amiga sectors (11968 bytes).

So finally the gap: 13424-11968=1456 bytes.

This is a method with foundation for a gap value

The 1660 byte gap was used consistently since the Amiga 1000 shipped, even the modern Kickstart 2.0 trackdisk.device would make use of it. I looked it up: the original trackdisk.device and the Amiga 1000 Kickstart boot ROM are exactly the same vintage with regard to the disk I/O code, this being June 24, 1985 (Neil Katin). Surprisingly, the Amiga 1000 Kickstart boot ROM code is almost entirely written in 'C' with only the power-up diagnostic code (Carl Sassenrath and Siegfried "Dusty" Bleher) written in assembly language. Even the MFM decoding code is written in 'C' and does not make use of the blitter or the index sync interrupt.

So, why 1660 bytes and not something in the low 1500's? The trackdisk.device MFM decoding does not rely upon the index sync interrupt to assist, neither does the Amiga 1000 Kickstart boot ROM. Once you turn on the DMA, the encoded bitstream data will start "pouring in" from whereever the read head is currently hovering over the track. Could the optional use of the index sync be a reason why the gap size is 1660 bytes? That is, could it have side effects which would be mitigated by a larger sector gap?

The Amiga 1000 internal and external 3.5" disk drives (the latter being the A1010 model) were a different make from what was used in the Amiga 2000 and Amiga 500 models. I may be wrong on this, but I seem to recall that the Amiga 1000 3.5" disk drives were manufactured by Toshiba, but the disk drives used in the Amiga 2000 and Amiga 500 were manufactured by Chinon. Could the 1660 bytes be part of "future proofing" the design, for the inevitable mass-produced Amigas which Commodore engineering would have already started working on? It might be that cheaper 3.5" disk drives were expected to have different, worse tolerances than the more expensive models used in the Amiga 1000.

[ross]

Originally Posted by Olaf Barthel

The 1660 byte gap was used consistently since the Amiga 1000 shipped, even the modern Kickstart 2.0 trackdisk.device would make use of it. I looked it up: the original trackdisk.device and the Amiga 1000 Kickstart boot ROM are exactly the same vintage with regard to the disk I/O code, this being June 24, 1985 (Neil Katin). Surprisingly, the Amiga 1000 Kickstart boot ROM code is almost entirely written in 'C' with only the power-up diagnostic code (Carl Sassenrath and Siegfried "Dusty" Bleher) written in assembly language. Even the MFM decoding code is written in 'C' and does not make use of the blitter or the index sync interrupt.

So, why 1660 bytes and not something in the low 1500's? The trackdisk.device MFM decoding does not rely upon the index sync interrupt to assist, neither does the Amiga 1000 Kickstart boot ROM. Once you turn on the DMA, the encoded bitstream data will start "pouring in" from whereever the read head is currently hovering over the track. Could the optional use of the index sync be a reason why the gap size is 1660 bytes? That is, could it have side effects which would be mitigated by a larger sector gap?

The Amiga 1000 internal and external 3.5" disk drives (the latter being the A1010 model) were a different make from what was used in the Amiga 2000 and Amiga 500 models. I may be wrong on this, but I seem to recall that the Amiga 1000 3.5" disk drives were manufactured by Toshiba, but the disk drives used in the Amiga 2000 and Amiga 500 were manufactured by Chinon. Could the 1660 bytes be part of "future proofing" the design, for the inevitable mass-produced Amigas which Commodore engineering would have already started working on? It might be that cheaper 3.5" disk drives were expected to have different, worse tolerances than the more expensive models used in the Amiga 1000.

I think that as usual it's a question of Occam's razor:
- Portia was much simpler and didn't even have a mechanism for WORDSYNC (which in fact isn't present in KS1.x);
- the index sync is a legacy of an old way of managing and synchronizing data on floppy, which in fact is totally ignored and ignorable on Amiga (except to be used by some protection);
- the 'wide gap' is an inheritance for the support of old and no so good drives with bad or imprecise self-correcting motor rotation speed (the Amiga can theoretically support even drives with much more that 5% error on rotation speed.. this allows for very long tracks and to function without problems; or very short actually ), you can also see from the usual KS1.x where the buffer is much larger than the new KSs.

When in a previous message I wrote to you that I put the tolerance at 5% I really exaggerated, the vast majority of drives do not deviate from 1%...
In the Chinon specs is defined a maximum variaton of "less than 1.5%", even long term.

But probably when they wrote the first KSs it was not like that
Or they simply wanted to be sure to support everything, also the first 3.5" drives ever built.

[ross]

Just out of curiosity I checked the sources of RNC diskio, which I would say can be considered a 'modern' professional standard for writing Amiga tracks without using trackdisk.device
It has probably been tested in thousands and thousands of drives.
I thought I remembered it was $1980 words, instead it is even less: $1960+1 (the +1 is the safety word due to Paula's error writing the last bits of the stream).

This means a gap of 12994-11968=1026 bytes

This also seems like a synthetic safety margin, too similar to 1024+2..
So the reasoning could be: "I tried I don't know how many times and drives, the worst one I found gave me $345 as excess, let's round up to $400"

[Olaf Barthel]

Originally Posted by ross

I think that as usual it's a question of Occam's razor:
- Portia was much simpler and didn't even have a mechanism for WORDSYNC (which in fact isn't present in KS1.x);

Ah, yes, I had totally forgotten about the missing word-sync/index-sync feature in the Amiga 1000. This is not covered in the "RKM Devices" trackdisk.device chapter, which, however, defers to the Autodocs which do provide the relevant details. I particularly enjoy this line in the second paragraph of the TD_RAWREAD command: "This interface is intended for sophisticated programmers only." That kind of hint never stopped anybody from doing something foolish, I suppose...

My "raw_disk" recovery tool makes use of the feature when the TD_RAWREAD command is issued, setting both IOTDF_INDEXSYNC and IOTDF_WORDSYNC flags (as per the Autodocs). Regardless of whether any of these features are supported, the program's MFM decoding will deal with an "unaligned" track just fine, with more effort involved in finding the bit pattern which introduces the track data and shifting the data as needed. Hey, this even provides a rare opportunity to make use of the SAS/C __builtin_rol() function

- the index sync is a legacy of an old way of managing and synchronizing data on floppy, which in fact is totally ignored and ignorable on Amiga (except to be used by some protection);

It can be helpful to reduce the effort spent on finding the track start pattern. That said, I darkly remember that the feature is documented as being not completely reliable in quickly delivering a perfectly-aligned track even on the A2000, A500 and beyond. In my "raw_disk" recovery program, I use it as a "bonus feature", if it works.

- the 'wide gap' is an inheritance for the support of old and no so good drives with bad or imprecise self-correcting motor rotation speed (the Amiga can theoretically support even drives with much more that 5% error on rotation speed.. this allows for very long tracks and to function without problems; or very short actually ), you can also see from the usual KS1.x where the buffer is much larger than the new KSs.

The reason for the larger buffer may also be that the Kickstart 1.x trackdisk.device always uses the blitter to decode/encode the track data. If I remember correctly, the Kickstart 2.x trackdisk.device still uses chip RAM for the DMA operations, but performs the encoding/decoding with the CPU.

[Wepl]

Originally Posted by Olaf Barthel

It can be helpful to reduce the effort spent on finding the track start pattern. That said, I darkly remember that the feature is documented as being not completely reliable in quickly delivering a perfectly-aligned track even on the A2000, A500 and beyond. In my "raw_disk" recovery program, I use it as a "bonus feature", if it works.

There are 11/22 sectors and each has double word sync pattern in front. There is no start track pattern. There are 11/22 start sector patterns. So Indexsync helps you to get the mfm data starting with the same sector each time if you do it multiple times. If you don't need this there are only drawbacks using IndexSync. DMA start reading has to wait a half revolution usually.

Originally Posted by Olaf Barthel

The reason for the larger buffer may also be that the Kickstart 1.x trackdisk.device always uses the blitter to decode/encode the track data. If I remember correctly, the Kickstart 2.x trackdisk.device still uses chip RAM for the DMA operations, but performs the encoding/decoding with the CPU.

Kick 1.3 always used blitter for encode/decode, IO_DATA must be in chip memory.
2.0-3.1 used always blitter to encode, if IO_DATA is not chip memory it's copied to a chip memory buffer before. Decoding used CPU only if IO_DATA was in non chip memory.
3.2 uses CPU for encode/decode if 68020+.

[ross]

Originally Posted by Wepl

So Indexsync helps you to get the mfm data starting with the same sector each time if you do it multiple times.

And even this is not guaranteed..
There is no 'hardware support' for index-sync (Paula know nothing about it, only the CIA does) and therefore the latency of the IRQ (or of a polling possibly interrupted) do not guarantee the sequence of the sectors, if used with the operating system active .
Of course you can always read it many times and discard the 'wrong' ones, but I don't know what sense that makes as a reading method...