English Amiga Board


Go Back   English Amiga Board > Coders > Coders. Asm / Hardware

 
 
Thread Tools
Old 03 April 2020, 22:21   #41
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,707
Ok, I put the second part of today's post, fixed, now it should be okay, but I reserve the right to check it better later

I've changed my code, that one with a single clock/chain_bit and adapted to RLL(1,4) (it's just changing a constant and insert a little filter).
Why do I insist with the single bit? Because it would allow me to extract the valid bits in a easy way and to build the stream by simply setting the chain/sync bit after each 'grouping'.

Well, result is good!

I've used a 11+chain_bit and the result is $86 possible valid combination.
This means having full coverage for 7 bits.
A simple combination of MSB, LSB and chain_bit give me the 'status' for a valid stream in every combination.

Code, the usual, with added the little filter:
Code:
    lea rll(pc),a0
    lea lut(pc),a1
    moveq   #0,d0
    moveq   #0,d5

.st moveq   #-2,d2
    moveq   #-4,d3
    move.l  d5,d1
    moveq   #11-1,d4
.ll add.w   d1,d1
    bcc.b   .cc
    moveq   #-5,d3
    addq.w  #1,d2
    beq.b   .ex
    bra.b   .pp
.cc moveq   #-2,d2
    addq.w  #1,d3
    beq.b   .ex
.pp dbf d4,.ll
    move.l  d5,d6
    and.w   #%0000000111100000,d6
    beq.b   .ex
    move.w  d5,(a0)+
    addq.w  #1,d0
.ex addi.w  #32,d5
    bne.b   .st
    move.w  d0,(a1)
    rts
    
lut ds.w    1
rll ds.w    $86
Here explained why this (should) work.
I force myself to use a maximum of three [EDIT: 0 bits in a row] at the top and bottom of the 'grouping' so as to have the following possible combinations:
previous LSB=0, new MSB=0 -> chain_bit=1
previous LSB=1, new MSB=0 -> chain_bit=0, so start of the successive streamed 'grouping' is four 0 bits in a row
previous LSB=0, new MSB=1 -> chain_bit=0, so end of the previous streamed 'grouping' is four 0 bits in a row
which guarantee me the extra status I need.

With $86 values available I can have also more than a SYNC word.

So I've a constant flux of 12*x bits (11 significant) and I can extract the 7 bits of my original data with a simple indexing in a 2^11 table.

Why is this 12:7 so good? Because the 'channel capacity is now 7/12=0.5833 that is better than the 0.58 that I had in a previous message considered the minimum to be able to have a 25% increase.
My goal is achieved

Ok, ok, everything is to be implemented, but knowing that it can be done it is already a good starting point.

if there are things that don't seem to work, that I haven't considered, or are wrong, let me know

Cheers.

Last edited by ross; 03 April 2020 at 22:41. Reason: []
ross is offline  
Old 04 April 2020, 11:57   #42
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,707
Ok, I go back to the RLL (1,3) to complete the talk using the technique I've used for the single chain bits' RLL(1,4) (I don't know if it is the best but it is certainly effective).
You can edit my code yourself very simply.

10+1 bit $34=lg2(5.700439718) -> channel capacity 0.518221793
11+1 bit $4d=lg2(6.266786541) -> channel capacity 0.522232212
12+1 bit $70=lg2(6.807354922) -> channel capacity 0.523642686
13+1 bit $a5=lg2(7.366322214) -> channel capacity 0.526165872
14+1 bit $f1=lg2(7.912889336) -> channel capacity 0.527525956
15+1 bit $162=lg2(8.467605550) -> channel capacity 0.529225347
16+1 bit $206=lg2(9.016808288) -> channel capacity 0.530400488

You can see that in practice the only usable on 68k (with integer ratio) is the 16+1 bits one (which uses anyway too much memory for the LUT..).
The rest would require rather complex fractional decoding.

practical capacity for 16+1 bit -> 9/17 = 0.529411765
RLL(1,3,9/17)/MFM ratio = 1.058823529
raw usable bytes in a track = ~$1800*1.058823529 = ~6505 -> 12.7 sectors/track

So you can store in a floppy ~$1968*160 = $FE100 = 1040640 bytes
This is the max you can do with this RLL(1,3) and non-long tracks

With $1900 long tracks you can store 6776 -> 13.23 sectors/track
For a grand total of 1084160 bytes

After all these calculations I think answer for Dan is exhaustive .
It remains to be seen whether this modified RLL (1,3) can actually make sense considering the various compression techniques available (well, maybe for some desperate case..).
Different for the RLL (1,4), but then reliability comes into play.
ross is offline  
Old 05 April 2020, 14:23   #43
DanScott
Lemon. / Core Design

DanScott's Avatar
 
Join Date: Mar 2016
Location: Sunny Bournemouth, UK
Posts: 635
Quote:
Originally Posted by ross View Post
After all these calculations I think answer for Dan is exhaustive .
It remains to be seen whether this modified RLL (1,3) can actually make sense considering the various compression techniques available (well, maybe for some desperate case..).
Different for the RLL (1,4), but then reliability comes into play.
Actually, I didn't quite expect the amount of work that was put into this thread

But yes, seems that some things are possible
DanScott is offline  
Old 05 April 2020, 15:29   #44
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,707
I tried to understand what is the technique used in xfloppy.device. So I did a search to find out if anything can be found on the interweb.
Not much, but the result brought me back on this forum to the following message:
http://eab.abime.net/showpost.php?p=1361125&postcount=9
From readme you can see that xfloppy coder is the same as megafloppy .
The difference is that megafloppy allows you to raw read/write data directly on the tracks.

But on megafloppy doc there are some interesting technical indications:
---
To achieve more capacity of floppy I used an enhanced data coding
instead of MFM ( which is normaly used on amiga and other computers )

My method has ratio 1 : 1 + 2/3 ( 1 : 5/3 )
while MFM has 1 : 2

which means that MFM codes 100 bits to 200 bits
and my method codes 100 bits to 166 bits !

---

Apparently the technique used is different from mine since I expect a ratio of
1:12/7
that is
100 bits to 171
.
The 5/3 increase, applied to my technique, seems a 9+1 to 6 bits that's impossible with my encoding and RLL(1,4) (I tried and only $39 values are available that obviously cannot cover the 6 bits required).

The technique I used is relatively simple to apply by a coder perspective, but in this other case there seems to be something more convoluted, probably more state passages or more 'prefix' bits. Probably the bits cannot be extracted with a simple table indexing.

Moreover there is another passage in the document that leaves me quite perplexed:
---
Working ... not working

Recently I discovered that the procedure fails for some kind of input
data ((
for example the browser.lha archive failed at decoding at file offset ~50k
... then it failed right on the beginning (

---

The only explanation is that the flow varies depending on the data introduced and that therefore some state change has not been considered because statistically improbable. Or a bug somewhere..
In my case the problem does not exist since I have a single change of state for the creation of the 'chain' bit.
So I don't depend on statistics at all, all combinations weight the same, otherwise I could not use a LUT.

Well, if I feel like it I try to dissect megafloppy/xfloppy
ross is offline  
Old 06 April 2020, 14:39   #45
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,707
Ok, I have collected other data about the questions still open regarding the definition of "channel capacity", but I don't want to bore you with tables and data, it are available upon request to those who want to deepen.

I quote myself from a previous message:
"Yes, is unspecified how this distribution is.
But as a statistical approach I suppose this data is for a not skewed source, one with sintetically generated equal 50% source bit distribution, otherwise it would make no sense."
and in fact it is exactly as I thought.

There are only 4 game cases for Amiga that use variable RLL(1,3) encoding. And as you can read from the following page:
http://www.softpres.org/wip:2003-04-...cording_format
the system was unsuccessful and no longer used because it was too heavy and terribly slow (Sir Galahad and his battle against Nitro came to my mind ): each bit must be decoded individually, there is a single 'sector' per track with variable final length.., too uncomfortable to build a file system around it.

However statistically the 0 bits present in a generic block of data are higher than the 1 bits, therefore this 'Psygnosis encoding' is able to exceed in most cases the channel capacity.
As I had assumed with a 50% distribution I should have a gain of 0.5515 / 0.5 = 10.3%, and indeed one of the tracks in Nitro has this distribution and this gain. All the others have a higher gain (due to a lower presence of bits at 1), the average gain for Nitro is slightly less than 15%. Very good for a RLL (1,3)!
Unfortunately with too many defects to be usable as generic encoding, you can only construct a custom format around it (and painfully slow on pure 68k).

I would like to comment on what is written on the SPS website:
- "Some time was spent with unsupported Psygnosis titles .. ..The games use MFM recording .."
I don't see any big problems in supporting this format in .IPF files (which is not MFM but a variable RLL(1,3)), it would be enough to put a tag for the import of the raw bits (just like in the eADF files), but I do not know the internal format of .IPF, so I could overlook significant implementation details.
- "..the encoding manages the theoretical minimum storage required using an average of 1.5 bits.."
This is an oxymoron channel capacity define the average for a 50% distribution.
- ".. But storage space is about 30% more than normal."
Well, you need a really unbalanced source data for this gain.. if you start from a base 10.3% even the Nitro 15% is really good.
Different if in this percentage also long-tracks usage is considered, but this is cheating

Cheers.
ross is offline  
Old 06 April 2020, 16:51   #46
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 660
Quote:
Originally Posted by ross View Post
There are only 4 game cases for Amiga that use variable RLL(1,3) encoding. And as you can read from the following page:
http://www.softpres.org/wip:2003-04-...cording_format
Interesting. Two questions comes to mind:
- How do they handle disk sync to find the start of the data?
- Do they load faster on a faster machine?
NorthWay is offline  
Old 06 April 2020, 17:12   #47
ross
Per aspera ad astra

ross's Avatar
 
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 50
Posts: 2,707
Quote:
Originally Posted by NorthWay View Post
Interesting. Two questions comes to mind:
- How do they handle disk sync to find the start of the data?
- Do they load faster on a faster machine?
Of course you are not free to use a SYNC word like usual.
But is not a problem because read and decoding are synced (very demanding..).

All IRQ are disabled, CIAB-ICR is polled for FLG (that is tied to INDEX from floppy).
Then immediately a DMA read is started (any valid MFM SYNC could work because we are on a GAP position for sure).
This requires that the previous writing has been done precisely.

Then the code wait the first DMA word written in memory and start deconding bit by bit (well, a longword is read to speed up a little).
If you are in a fast machine you need to wait next read (long)word but on bare 68k code is much slower than the (simultaneous) DMA transfer.

EDIT: just checked Nitro on 7mhz 68k. Deconding a track require ~440ms. So >2 rotations.
Since it requires anyway a re-synchronization to INDEX, at least 3 rotations (600ms) per track on read are required.
Yes, very slow.

Last edited by ross; 06 April 2020 at 18:37.
ross is offline  
Old 07 April 2020, 08:49   #48
Wepl
Moderator
Wepl's Avatar
 
Join Date: Nov 2001
Location: Germany
Posts: 721
Quote:
Originally Posted by ross View Post
Of course you are not free to use a SYNC word like usual.
When using Indexsync together you can use nearly any sync because it doesn't matter if it is also contained in the data stream.
Wepl is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Kaffer's disk-analyse - more IPF formats? Jaimie_H project.SPS (was CAPS) 3 08 October 2015 15:08
Anyone adding formats to kaffers Disk-Utilities? BarryB Coders. General 1 09 August 2015 21:11
Custom Disk Formats CorrodedCoder Coders. General 3 08 January 2010 14:54
New Catweasel mk IV drivers & disk formats! stainy Amiga scene 14 14 August 2008 17:20
Disk2FDI & Custom Disk Formats jmmijo support.Apps 9 14 April 2002 22:01

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 21:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Page generated in 0.07580 seconds with 15 queries