English Amiga Board


Go Back   English Amiga Board > Coders > Coders. Asm / Hardware

 
 
Thread Tools
Old 28 March 2020, 23:26   #1
DanScott
Lemon. / Core Design

DanScott's Avatar
 
Join Date: Mar 2016
Location: Sunny Bournemouth, UK
Posts: 637
Non-standard disk formats`

Ok, I never got into much regarding Amiga disk formats and my knowledge is fairly limited in this field.... I always used friends standard trackloaders, or trackloader systems available through work etc... so my question is:

What other disk formats other than MFM are usable on Amiga, how much data can one really fit on a standard floppy ???
DanScott is offline  
Old 28 March 2020, 23:49   #2
mcgeezer
Registered User

 
Join Date: Oct 2017
Location: Sunderland, England
Posts: 1,872
Probably one for Galahad this.
mcgeezer is offline  
Old 29 March 2020, 00:53   #3
Galahad/FLT
Going nowhere

Galahad/FLT's Avatar
 
Join Date: Oct 2001
Location: United Kingdom
Age: 46
Posts: 7,644
Quote:
Originally Posted by DanScott View Post
Ok, I never got into much regarding Amiga disk formats and my knowledge is fairly limited in this field.... I always used friends standard trackloaders, or trackloader systems available through work etc... so my question is:

What other disk formats other than MFM are usable on Amiga, how much data can one really fit on a standard floppy ???
If you want a format that everyone can use whether it be real Amiga or WinUAE, you could do a basic MFM system that gives the same capacity as Rob Northens PDOS Protection system but with the ability to be written on a Real Amiga or WinUAE through extended ADF format.

So as you know, a standard Amiga formatted disk using 11 sectors can be 901120 bytes in size which is 160 tracks, 1760 sectors, 1760*512=901120/$dc000

Most typical not very exotic MFM formats can give you 12 sectors which
in total is 983040 bytes which is 160 tracks, 1920 sectors.
1920*512=983040/$f0000

There are bigger capacity formats such as Factor 5's, and the Psygnosis MFM format, but people are unlikely to be able to copy them.

But the good news is some of these MFM formats have been disassembled and are known, Rob Northens PDOS writing code is out in the wild so that can be used, and WinUAE can read from extended ADF files.

Otherwise if you don't want that kind of bollocks, you could stick with usual sector loading AmigaDOS disk but utilise Tracks 81 and 82 which most drives should be fine with.

If you really want to eek every last byte out of a disk, you could utilise something like Gaston / Fairlights loader which allows you to read at any offset into the disk instead of $200 sector offsets.

Pretty sure Ross on this forum has coded his own as well, i've got one but its not very reliable.

But if you want reliable access, quick disk loading, and more importantly no faffing on the user end with MFM formats, Rob Northens standard Sector loader that you used on Heimdall will do exactly what you want, but its 901120 bytes in size and no bigger.
Galahad/FLT is offline  
Old 29 March 2020, 01:26   #4
mr.spiv
Registered User
mr.spiv's Avatar
 
Join Date: Aug 2006
Location: Finland
Age: 48
Posts: 133
I never did anything but MFM with custom sync words.. although, the format I used in "Smarty and the Nasty Gluttons" had $18a0 bytes per track for user data (access offset alignment was 4 bytes) excluding the first track that had to be in normal Amiga format. That gave 1007968 bytes plus then the whole disk was compressed (Stacker-like loading i.e. each track was decrunched while DMA was loading the next one) so I guess we were +1.5MB in a typical case.
mr.spiv is offline  
Old 29 March 2020, 22:38   #5
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
The only possible way to break the ~1000000 bytes limit is to overcome the MFM format (without resorting to long tracks).
I've experimented it in late '80 using four consecutive 0 in raw stream (instead of the max three of standard MFM) and somehow worked (two consecutive 1 bits failed miserably).
So something similar to the M2FM format.
It's long time ago but if I don't remember badly more than 25% more data is possible to stock on track.

Problem here is that is no more possible a fast deMFM routine because no more a (integer) 2:1 ratio and properly 'bit interlacing', but you need to resort to something more similar to a GCR format, requiring LUT and unconfortable bits grouping (shift, mask, reconstruction).
Even the read sync start position can be problematic if you use all the quasi-MFM bit combo.
Moreover you need to take account for drive speed drifting that can lead to read error, indeed a good write drive is needed (no problem anyway if the write drive is the same as the write one, and obviously no problem on 'solid drive' or emulated).

I'm pretty sure some Psygnosis games have used something like this.


PS: and yes, I've used a 'free-position' loader for standard format.
ross is offline  
Old 30 March 2020, 03:10   #6
Retro1234
Boo

Retro1234's Avatar
 
Join Date: Jun 2006
Location: 5150
Posts: 4,662
XFloppy on Aminet can store 1.12mb and DiskSpare 984kb
Retro1234 is offline  
Old 30 March 2020, 09:02   #7
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
Quote:
Originally Posted by Retro1234 View Post
XFloppy on Aminet can store 1.12mb..
Interesting, I'm not aware of this XFloppy.device

Seems to use 14 sectors/track:
Code:
#define XFD_TRACKSECTORS	14
so is fully coherent with my 25% increase and I'm pretty sure is using a similar technique.

This anyway is not encouraging:
Code:
This is a development version, it may have bugs
Particularily the reads and writes are pretty unreliable
and r/w errors are quite often.
I wonder if I can find my old sources and tests somewhere, but without a real machine do not make much sense..
ross is offline  
Old 30 March 2020, 10:22   #8
roondar
Registered User

 
Join Date: Jul 2015
Location: The Netherlands
Posts: 2,068
Quote:
Originally Posted by ross View Post
The only possible way to break the ~1000000 bytes limit is to overcome the MFM format (without resorting to long tracks).
I've experimented it in late '80 using four consecutive 0 in raw stream (instead of the max three of standard MFM) and somehow worked (two consecutive 1 bits failed miserably).
So something similar to the M2FM format.
It's long time ago but if I don't remember badly more than 25% more data is possible to stock on track.

Problem here is that is no more possible a fast deMFM routine because no more a (integer) 2:1 ratio and properly 'bit interlacing', but you need to resort to something more similar to a GCR format, requiring LUT and unconfortable bits grouping (shift, mask, reconstruction).
Even the read sync start position can be problematic if you use all the quasi-MFM bit combo.
Moreover you need to take account for drive speed drifting that can lead to read error, indeed a good write drive is needed (no problem anyway if the write drive is the same as the write one, and obviously no problem on 'solid drive' or emulated).

I'm pretty sure some Psygnosis games have used something like this.

PS: and yes, I've used a 'free-position' loader for standard format.
While I think these kind of things are super cool and interesting, I do wonder whether they are an acceptable compromise or not. As in, does the extra space compensate for the lower reliability?

It's an interesting question. Maybe more so now that most Amiga floppy drives (and most/all 3.5" DD floppies) are so old and probably operating closer to their tolerances than normal.

Personally, for now I'll stick with a basic trackloader and a good cruncher. That'll also give you plenty of "extra space"
roondar is offline  
Old 31 March 2020, 01:58   #9
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 661
I was working on an answer on my old pc, but it died on me (new parts incoming).

Short version: Can you force GCR to work with 2ms? If not then forget that.
Other option: Don't encode and decode the regular way where you blindly throw away half the bits. If you look at the MFM rules and what are "legal" bit patterns you will find that for some fixed number of bits (say 16) the set of possible bit combos are >256 (depending on what your "previous bits" starting condition is).
This isn't a fixed maximum size but will depend on your data. I guess the way to encode it is to look at it as some kind of optimum uuencode algorithm (that was a thing back in the NetNews days - how many valid characters were there, and how could you pack your data into those most efficiently). Possibly it shares traits with arithmethic compression in using fractional bits?
I never got my head around how to do it in real life, it became a thought experiment.
The size of your encoded data can be bigger than 16 bits, but you will eventually run into either a memory issue if you build tables for bit matching, or a speed issue if you loop through all possible combos for N bits until you match the encoded input. The more bits you look at the more savings you gain.

TLDR: MFM en/de-coding is AFAICT wasteful compared to the rules it lays down.
NorthWay is offline  
Old 31 March 2020, 11:15   #10
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
Quote:
Originally Posted by NorthWay View Post
Short version: Can you force GCR to work with 2ms? If not then forget that.
Yes, but data read/written this way is unreliable.
GCR encoding works only if flux transitions are enough spaced, permitting more changes in the defined time.
Much better stick to 2ns MFM.

Quote:
Other option: Don't encode and decode the regular way where you blindly throw away half the bits. If you look at the MFM rules and what are "legal" bit patterns you will find that for some fixed number of bits (say 16) the set of possible bit combos are >256 (depending on what your "previous bits" starting condition is).
This isn't a fixed maximum size but will depend on your data. I guess the way to encode it is to look at it as some kind of optimum uuencode algorithm (that was a thing back in the NetNews days - how many valid characters were there, and how could you pack your data into those most efficiently). Possibly it shares traits with arithmethic compression in using fractional bits?
I never got my head around how to do it in real life, it became a thought experiment.
The size of your encoded data can be bigger than 16 bits, but you will eventually run into either a memory issue if you build tables for bit matching, or a speed issue if you loop through all possible combos for N bits until you match the encoded input. The more bits you look at the more savings you gain.
Yes, this is the basis for pushing MFM limits (you can also add four 0 in a row to the encoding breaking the rules, that usually works also).
ross is offline  
Old 31 March 2020, 16:20   #11
Wepl
Moderator
Wepl's Avatar
 
Join Date: Nov 2001
Location: Germany
Posts: 721
I don't see why other encodings than MFM should cause reliability problems as long as not more that one 1 and three 0 are consecutive encoded.
Important is, that a possible sync word remains available.
Wepl is offline  
Old 31 March 2020, 16:34   #12
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 661
This here was my back-of-an-envelope scribbling about what are the legal encodings for 16 encoded bits of data http://eab.abime.net/attachment.php?...4&d=1539481771
NorthWay is offline  
Old 31 March 2020, 19:19   #13
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
Quote:
Originally Posted by Wepl View Post
I don't see why other encodings than MFM should cause reliability problems as long as not more that one 1 and three 0 are consecutive encoded.
Important is, that a possible sync word remains available.
Sorry Wepl, I don't know if you're referring to my message, but I never said otherwise.
GCR require more than one 1 in a row, so it's practically impossible for it to work with 2ns cells (on Amiga).

Of course if you write an encoding that respect RLL (1,3) codes you do not have any reliability problems.

EDIT:
It's probably only a matter of semantics, when we talk about MFM encoding we usually associate with RLL (1,3), that we know used for encoding on magnetic surfaces, .
Indeed, one should not speak of strict MFM if some change are made (like the SYNC word ); it would be more correct to call it other RLL (1,3) encoding.
The situation is different if the rules are broken, using for example an RLL (1,4) encoding.

Last edited by ross; 31 March 2020 at 19:57.
ross is offline  
Old 31 March 2020, 19:29   #14
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
Quote:
Originally Posted by NorthWay View Post
This here was my back-of-an-envelope scribbling about what are the legal encodings for 16 encoded bits of data http://eab.abime.net/attachment.php?...4&d=1539481771
No, this cannot work, you need to exclude those who break rules when chained.
Or I'm missing something?
ross is offline  
Old 31 March 2020, 22:06   #15
Wepl
Moderator
Wepl's Avatar
 
Join Date: Nov 2001
Location: Germany
Posts: 721
Quote:
Originally Posted by ross View Post
Sorry Wepl, I don't know if you're referring to my message, but I never said otherwise.
Sorry, no. Was meant in general.
Quote:
Originally Posted by ross View Post
It's probably only a matter of semantics, when we talk about MFM encoding we usually associate with RLL (1,3), that we know used for encoding on magnetic surfaces, .
Indeed, one should not speak of strict MFM if some change are made (like the SYNC word ); it would be more correct to call it other RLL (1,3) encoding.
agreed
Wepl is offline  
Old 31 March 2020, 22:32   #16
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
A tentative, first idea that came to my mind.

Use y bits for the pure encoding (14 in my example) and 1 or 2 bits for the chain, this way:
0xxxxxxxxxxxx0, chain bits: 1 if 0X, 10 if 1X
0xxxxxxxxxxxx1, chain bits: 0 if 1X, 01 if 0X
1xxxxxxxxxxxx0, chain bits: 1 if 0X, 10 if 1X
1xxxxxxxxxxxx1, chain bits: 0 if 1X, 01 if 0X

Worst case 16 bits used for A encoding.
Usable value A=$13E versus $100 for strict MFM.

This of course is a fake value because I've to subtract all the values that combined in the raw stream give me a SYNC.
And another problem is how to 'unpack' this A value to something usable

Code:
Code:
    lea rll13(pc),a0
    moveq   #0,d0
    moveq   #0,d5

.st moveq   #-2,d2
    moveq   #-4,d3
    move.l  d5,d1
    moveq   #14-1,d4
.ll add.w   d1,d1
    bcc.b   .cc
    moveq   #-4,d3
    addq.w  #1,d2
    beq.b   .ex
    bra.b   .pp
.cc moveq   #-2,d2
    addq.w  #1,d3
    beq.b   .ex
.pp dbf d4,.ll
    move.w  d5,(a0)+
    addq.w  #1,d0
.ex addq.w  #4,d5
    bne.b   .st
    rts
    
rll13   ds.w $13E
Well, this is just a proof of concept, nothing else and not usable.
Correct me if I made some obvious mistakes

EDIT: well, the obvious mistake is that I cannot decode the data because I've no way to know where the 'grouped' bits start after chained bit(s)
But excluding some A values and selecting a proper chain bit (or two?), making it a constant bit(s) quantity, could solve?
I have to think about it

Last edited by ross; 01 April 2020 at 00:29.
ross is offline  
Old 31 March 2020, 22:56   #17
robinsonb5
Registered User
 
Join Date: Mar 2012
Location: Norfolk, UK
Posts: 804
How about having two or maybe even 4 different encodings for each possible byte value, grouped on where the first 1 falls? After encoding each byte you select an encoding for the next byte depending on the position of the last 1 in the byte just encoded.
The decoder would have to do the same, select a decoding for the next byte based on the last 1 of the byte just decoded.

(Finding a suitable sync value could be tricky, though!)
robinsonb5 is offline  
Old 01 April 2020, 01:25   #18
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 661
Quote:
Originally Posted by ross View Post
No, this cannot work, you need to exclude those who break rules when chained.
Hm?
That was an example of looking at 16 encoded bits when the three previous ones were all '0'.
Three previous 0 bits is a valid prequel to any random point in a bitstream from the floppy. I think I then enumerated all the valid combinations of 16 encoded bits that follow three 0s.
Which by my account totals up to 256+14(==270). This is 5,4% more than the encoding space utilized in standard mfm. The '000' prefix is also IIRC the worst case as the 4 other will give around 300 combinations AFAIR.
I don't know how common each prefix is, but it would be dynamically varying depending on the source data.

As I said, the more bits you look at the more combinations there are compared to just halving it. I started with looking at 8 instead of 16, but that gave a potential for getting _fewer_ combinations than the normal encoding (because of the pessimal '000' prefix).
Decoding 16 bits from table searches would require something like 5*(16/8)*300 bytes, i.e. 3K, and that is a table you can build on the fly. I haven't done the math on how it would grow with longer bit patterns.
NorthWay is offline  
Old 01 April 2020, 01:36   #19
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
Quote:
Originally Posted by NorthWay View Post
Hm?
That was an example of looking at 16 encoded bits when the three previous ones were all '0'.
Three previous 0 bits is a valid prequel to any random point in a bitstream from the floppy. I think I then enumerated all the valid combinations of 16 encoded bits that follow three 0s.
Which by my account totals up to 256+14(==270). This is 5,4% more than the encoding space utilized in standard mfm. The '000' prefix is also IIRC the worst case as the 4 other will give around 300 combinations AFAIR.
I don't know how common each prefix is, but it would be dynamically varying depending on the source data.

As I said, the more bits you look at the more combinations there are compared to just halving it. I started with looking at 8 instead of 16, but that gave a potential for getting _fewer_ combinations than the normal encoding (because of the pessimal '000' prefix).
Decoding 16 bits from table searches would require something like 5*(16/8)*300 bytes, i.e. 3K, and that is a table you can build on the fly. I haven't done the math on how it would grow with longer bit patterns.
Sorry, I fail to understand.
Yes that 000 is a valid prequel, but how you can chain this? (example taken from your data):
000 1000100010101001
10001000101010011000100010101001 is impossible..
ross is offline  
Old 01 April 2020, 01:49   #20
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,716
In the meantime my second try.
Main idea is to use the MSB as the chain bit and use the LSB as a LUT selector (is this an idea similar to that of robinsonb5? I did not understand properly what he meant.., sorry).

Code is practically the same but splitted for MSB:
Code:
    lea rll13_1(pc),a0
    moveq   #0,d0
    moveq   #0,d5

.st moveq   #-2,d2
    moveq   #-4,d3
    move.l  d5,d1
    moveq   #14-1,d4
.ll add.w   d1,d1
    bcc.b   .cc
    moveq   #-4,d3
    addq.w  #1,d2
    beq.b   .ex
    bra.b   .pp
.cc moveq   #-2,d2
    addq.w  #1,d3
    beq.b   .ex
.pp dbf d4,.ll
    move.w  d5,(a0)+
    addq.w  #1,d0
.ex addq.w  #4,d5
    bpl.b   .st
    
    lea rll13_2(pc),a0
    moveq   #0,d0
;    moveq   #0,d5

.s2 moveq   #-2,d2
    moveq   #-4,d3
    move.l  d5,d1
    moveq   #14-1,d4
.l2 add.w   d1,d1
    bcc.b   .c2
    moveq   #-4,d3
    addq.w  #1,d2
    beq.b   .e2
    bra.b   .p2
.c2 moveq   #-2,d2
    addq.w  #1,d3
    beq.b   .e2
.p2 dbf d4,.l2
    move.w  d5,(a0)+
    addq.w  #1,d0
.e2 addq.w  #4,d5
    bne.b   .s2

    rts
    
rll13_1   ds.w $bd
rll13_2   ds.w $81
This guarantee a proper decoding, better than strict MFM but unbalanced, practically unusable..
ross is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Kaffer's disk-analyse - more IPF formats? Jaimie_H project.SPS (was CAPS) 3 08 October 2015 15:08
Anyone adding formats to kaffers Disk-Utilities? BarryB Coders. General 1 09 August 2015 21:11
Custom Disk Formats CorrodedCoder Coders. General 3 08 January 2010 14:54
New Catweasel mk IV drivers & disk formats! stainy Amiga scene 14 14 August 2008 17:20
Disk2FDI & Custom Disk Formats jmmijo support.Apps 9 14 April 2002 22:01

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 06:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Page generated in 0.09808 seconds with 15 queries