Creating your own IPFs

[BarryB]

I tested the disk by dumping it with my Kryoflux and it decodes in disk-utilities without any error, so it seems to verify the IPF was written to disk OK?

[acd2001]

keirf disk-utilities 2023-04-22 git f02c928

Cleanup - moved apidya format to chw and renamed and modified to support Mr Nutz, Turrican 3 and Apidya. Removed Eliminator as it is covered by chw. Removed Turrican III from factor 5

[BarryB]

Originally Posted by acd2001

keirf disk-utilities 2023-04-22 git f02c928

Cleanup - moved apidya format to chw and renamed and modified to support Mr Nutz, Turrican 3 and Apidya. Removed Eliminator as it is covered by chw. Removed Turrican III from factor 5

Thanks matey

[Keir]

Originally Posted by BarryB

Thank you

I wrote my copy of Leavin' Teramis back to disk to test it for Abaddon on real hardware but on my A500 & A1200 it just loads constantly on ... LOADING NEXT LEVEL ... so I guess both my drives are out of alignment as the same IPF loads fine in WinUAE!

Would someone else be able to write an IPF back to disk and test it?

I think the disk-analyse generated track for Leavin' Teramis is unnecessarily fragile. I see this in the generator function:

    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x5224);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x44a9);
    tbuf_weak(tbuf, 8);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x4000);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 32, 0x00011484);
    tbuf_weak(tbuf, 16);
    tbuf_weak(tbuf, 8);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x1128);

When I look at track data from the official IPF from the 522444a9 sync I see, for example:

150/950: 522444a9 8642e608 0c85cc11 190b9823 32173047 44a94489 44a944a9 44a92aaa 
150/950: 522444a9 642e608f c85cc11f 90b9823e 2173047c 44a94489 44a944a9 44a92aaa 
150/950: 522444a9 42e608f9 85cc11f2 0b9823e5 173047cb 44a94489 44a944a9 44a92aaa 
150/950: 522444a9 2e608f97 5cc11f2e b9823e5d 73047cbb 44a94489 44a944a9 44a92aaa

Now the submitted dump may not look exactly like the IPF, but it seems clear to me the code should look more like:

    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 32, 0x522444a9);
    tbuf_weak(tbuf, 64);

Shorter code, longer consecutive weak region. Short weak regions (like 8 bits long) are really hard to write!

I will also add that the gap between weak region and 4489 sync, written by disk-analyse, doesn't seem to exist in the official IPF. However, if the game loader accepts it, it's a good idea! It will work on a wider range of drives this way.

Over to Abaddon

[Keir]

Originally Posted by Snoopy1234

With Demolition for example. I've read with default scp settings, format amigados + --raw and also fake index and no clobber. I've tried also adjusting the scp in HxC with set flakey bits, shift track and repair track in differing instances. It hasn't passed protection.
The drive is a Samsung NOS 3.5", GW tools are 1.10 and GW an F1.

SPS IPF passes in WinUAE. My Phalanx scp makes WinUAE crash so I haven't been able to test. I can't recall if aa IPF written to disk passes protection on any of the Kingsoft titles.

Try conversion from SCP to IPF using disk-utilities. There are Windows builds advertised in this very forum thread. SCP will work for direct emulation or writeback sometimes, but not reliably for all protection types. Sometimes analysis and track regeneration is needed, especially for writeback to disk.

[Abaddon]

Originally Posted by Keir

I think the disk-analyse generated track for Leavin' Teramis is unnecessarily fragile. I see this in the generator function:

    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x5224);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x44a9);
    tbuf_weak(tbuf, 8);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x4000);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 32, 0x00011484);
    tbuf_weak(tbuf, 16);
    tbuf_weak(tbuf, 8);
    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, 0x1128);

When I look at track data from the official IPF from the 522444a9 sync I see, for example:

150/950: 522444a9 8642e608 0c85cc11 190b9823 32173047 44a94489 44a944a9 44a92aaa 
150/950: 522444a9 642e608f c85cc11f 90b9823e 2173047c 44a94489 44a944a9 44a92aaa 
150/950: 522444a9 42e608f9 85cc11f2 0b9823e5 173047cb 44a94489 44a944a9 44a92aaa 
150/950: 522444a9 2e608f97 5cc11f2e b9823e5d 73047cbb 44a94489 44a944a9 44a92aaa

Now the submitted dump may not look exactly like the IPF, but it seems clear to me the code should look more like:

    tbuf_bits(tbuf, SPEED_AVG, bc_raw, 32, 0x522444a9);
    tbuf_weak(tbuf, 64);

Shorter code, longer consecutive weak region. Short weak regions (like 8 bits long) are really hard to write!

I will also add that the gap between weak region and 4489 sync, written by disk-analyse, doesn't seem to exist in the official IPF. However, if the game loader accepts it, it's a good idea! It will work on a wider range of drives this way.

Over to Abaddon

The protection reads the protection quite a few times

Excerpt from the protection reads

00002374 4001 4000 0001 1404 4041 0001 4001 1100  @.@.....@A..@...
00002384 1445 4000 0001 1404 1115 5115 4455 1100  .E@.......Q.DU..
00002394 4451 4000 0001 1404 5151 1141 4501 1100  DQ@.....QQ.AE...
000023A4 4115 4000 0001 1404 1155 0455 5155 1100  A.@......U.UQU..

I based the implementation of the protection on this and the code that does the compares.

I will modify the code to what you suggested and run a test.


UPDATE: I have implemented Keir's suggestions and have test the game via winuae from start to finish. I just merged the changes.

[acd2001]

keirf disk-utilities 2023-04-23 git 8713373
Updated the Leavin' Teramis protection with Keir's suggestions

[Snoopy1234]

Originally Posted by Keir

Try conversion from SCP to IPF using disk-utilities. There are Windows builds advertised in this very forum thread. SCP will work for direct emulation or writeback sometimes, but not reliably for all protection types. Sometimes analysis and track regeneration is needed, especially for writeback to disk.

I've converted both of my SCP images, Demolition and Phalanx to IPF with disk-analyse. Running Demolition.ipf in WinUAE loads to title screen and music the blank screen with continuous music. Loading Phalanx.ipf in WinUAE loads boot screen and wipe then black screen, no sound. Both stay at this point.

[Keir]

Originally Posted by Snoopy1234

I've converted both of my SCP images, Demolition and Phalanx to IPF with disk-analyse. Running Demolition.ipf in WinUAE loads to title screen and music the blank screen with continuous music. Loading Phalanx.ipf in WinUAE loads boot screen and wipe then black screen, no sound. Both stay at this point.

Can you please try the official IPFs #2708 and #2712? You can download these from the EAB file server.

This will help indicate if the problem is one of compatibility versus the anco/kingsoft analyser in disk-utilities needing tweaks.

[Snoopy1234]

Originally Posted by Keir

Can you please try the official IPFs #2708 and #2712? You can download these from the EAB file server.

This will help indicate if the problem is one of compatibility versus the anco/kingsoft analyser in disk-utilities needing tweaks.

Demolition #2712 and Phalanx #2708 both run in WinUAE. Writing them to disk with the GW isn't successful. I have to turn off verify as it can't verify protection track 80.0 on either game and both games when run, error at startup with Wait for disk activity to finish and Software Error Finish Task etc.

[Abaddon]

Originally Posted by Snoopy1234

Demolition #2712 and Phalanx #2708 both run in WinUAE. Writing them to disk with the GW isn't successful. I have to turn off verify as it can't verify protection track 80.0 on either game and both games when run, error at startup with Wait for disk activity to finish and Software Error Finish Task etc.

Can you upload the dumps to the zone and I will take a look at why they are hanging in winuae.

[Keir]

Originally Posted by Snoopy1234

Demolition #2712 and Phalanx #2708 both run in WinUAE. Writing them to disk with the GW isn't successful. I have to turn off verify as it can't verify protection track 80.0 on either game and both games when run, error at startup with Wait for disk activity to finish and Software Error Finish Task etc.

Thanks for your testing. Looks like Abaddon is on the case for his anco/kingsoft analyser in disk-utilities.

Failure to write the official IPFs to disk using GW is my domain and I will investigate that one. I aim to support writeback of any official IPF.

[BarryB]

I'm testing the new Leavin Teramis build now on my A500, will let you know in due course what happens!

*UPDATE*

Just used the updated disk-utilities with Keirs/Abaddons changes and using my dump of Leavin' Teramis from the Thalion First Year Compilation the IPF writes back fine to disk with my Greaseweazle and it loads and lets me play now on my A500

Ran out of time on the first level boss though

[Keir]

Originally Posted by Keir

Thanks for your testing. Looks like Abaddon is on the case for his anco/kingsoft analyser in disk-utilities.

Failure to write the official IPFs to disk using GW is my domain and I will investigate that one. I aim to support writeback of any official IPF.

Okay, so I have done some analysis of Demolition (SPS IPF #2012). This may also be relevant for disk-utilities analysis of your dump, so I will give details here.

Mainly for Abaddon: IPF #2012 vs Disk-Utilities analyser:

Firstly, the IPF has Anco/Kingsoft protection on T80.0, and some variant with weak bits on T80.1. I'm not sure whether the latter relates directly to "anco_kingsoft_weak_protection" in disk-utilities because the layout written by that analyser does not correspond to what's in the IPF. To be precise, in the IPF, T80.1 looks rather like normal Anco/Kingsoft protection sector (13 decoded bytes) followed by 5 weak bytes.

The problem could be that more Anco/Kingsoft titles have weak bytes on T80.1 than suggested in the disk-analyse/formats file. And that the weak area would be better represented by replacing:

tbuf_bits(tbuf, SPEED_AVG, bc_raw, 8, dat[1] >> 8);
tbuf_weak(tbuf, 8);

with

tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, dat[1]);
tbuf_weak(tbuf, 5*8);

However this is not certain as I have not analysed any Anco/Kingsoft dumps personally at all!

For you: Regarding writeback of IPF #2012 using GW:

Regarding writeback of SPS IPF #2012 using Gresaeweazle, the problem is that the hidden key value is not valid MFM. Hence, especially on a dense inner track like 80.0, it is easy for the illegal adjacent bitcells to shift apart during write.

The workaround for this is to specify write precompensation during the write, as this will write the illegal bitcells closer together to "precompensate" for their natural tendency to spread apart. See https://github.com/keirf/greaseweazl...recompensation

Note that GW by default applies no precomp. It is hard to set a default value as it does depend on disk and drive to a large extent. However on Amiga, a sensible default would be to always add this to your "gw write" command line:

--precomp=40=125:80=250

When I specified this, I was able to write the protection tracks with verification, and my disk booted as far as the manual protection (Asking for "Secret code").

[Keir]

Originally Posted by BarryB

I'm testing the new Leavin Teramis build now on my A500, will let you know in due course what happens!

I have written the official IPF #3061 back to disk with "--precomp=40=125:80=250" and it loads okay to the title screen (ie. well past the ramdisk screen).

I'm not sure if precomp helps so much on this one: Just as likely is that weak bits are fussy about the drive. Possibly KF writes some different weak pattern, but I do know that the GW methods seems to work okay with any drives I tested.

EDIT: Also worth hard erasing your floppy disk first with a strong magnet.

EDIT2: Please use the '--precomp' line always. It will help! Sometimes even stronger precomp medicine is required (eg. across all tracks)

[BarryB]

The new updated decoder seems ok now for my dump as per my update.

Just tried the official IPF with your precomp settings and it guru'd on the Ram Disk screen? Just tried another disk (the one that my KF wrote and worked) and it loads this time, so the even though the disk wrote back OK my A500 didn't like it but another disk works fine!

[acd2001]

keirf disk-utilities 2023-04-23 git 8710eb6

thalion: Clean up 44A9 handling -> A1 with normal encoding.
No semantic changes are intended.

[BarryB]

Originally Posted by acd2001

keirf disk-utilities 2023-04-23 git 8710eb6

thalion: Clean up 44A9 handling -> A1 with normal encoding.
No semantic changes are intended.

Thanks

[Abaddon]

Originally Posted by Keir

Okay, so I have done some analysis of Demolition (SPS IPF #2012). This may also be relevant for disk-utilities analysis of your dump, so I will give details here.

Mainly for Abaddon: IPF #2012 vs Disk-Utilities analyser:

Firstly, the IPF has Anco/Kingsoft protection on T80.0, and some variant with weak bits on T80.1. I'm not sure whether the latter relates directly to "anco_kingsoft_weak_protection" in disk-utilities because the layout written by that analyser does not correspond to what's in the IPF. To be precise, in the IPF, T80.1 looks rather like normal Anco/Kingsoft protection sector (13 decoded bytes) followed by 5 weak bytes.

The problem could be that more Anco/Kingsoft titles have weak bytes on T80.1 than suggested in the disk-analyse/formats file. And that the weak area would be better represented by replacing:

tbuf_bits(tbuf, SPEED_AVG, bc_raw, 8, dat[1] >> 8);
tbuf_weak(tbuf, 8);

with

tbuf_bits(tbuf, SPEED_AVG, bc_raw, 16, dat[1]);
tbuf_weak(tbuf, 5*8);

However this is not certain as I have not analysed any Anco/Kingsoft dumps personally at all!

For you: Regarding writeback of IPF #2012 using GW:

Regarding writeback of SPS IPF #2012 using Gresaeweazle, the problem is that the hidden key value is not valid MFM. Hence, especially on a dense inner track like 80.0, it is easy for the illegal adjacent bitcells to shift apart during write.

The workaround for this is to specify write precompensation during the write, as this will write the illegal bitcells closer together to "precompensate" for their natural tendency to spread apart. See https://github.com/keirf/greaseweazl...recompensation

Note that GW by default applies no precomp. It is hard to set a default value as it does depend on disk and drive to a large extent. However on Amiga, a sensible default would be to always add this to your "gw write" command line:

--precomp=40=125:80=250

When I specified this, I was able to write the protection tracks with verification, and my disk booted as far as the manual protection (Asking for "Secret code").

The raw dump I have (thanks to BarryB) does not appear to have weak bits

Excerpt from track 80.1

0004A57C 5544 5544 5544 5544 5544 5544 5544 8892  UDUDUDUDUDUDUD..
0004A58C 5544 8895 2514 9115 5555 492A 54A4 94A9  UD..%...UUI*T...

The raw data is the same on every check. First check is done before the entering the code screen appears (Code is ABUSS). Then a check is done at the start of every level.

The first 9 words are checked then if that passes the key is checked. I did not see any further checks of the data after the key 8895.

I then checked the IPF 2712 and confirmed that the raw data for track 80.1 does have weak bits, but the protection check code is identical to the dump I used.

[Keir]

Originally Posted by Abaddon

The raw data is the same on every check. First check is done before the entering the code screen appears (Code is ABUSS). Then a check is done at the start of every level.

The first 9 words are checked then if that passes the key is checked. I did not see any further checks of the data after the key 8895.

I then checked the IPF 2712 and confirmed that the raw data for track 80.1 does have weak bits, but the protection check code is identical to the dump I used.

Fair enough. Different runs of titles do seem to have been mastered differently sometimes, and/or SPS IPFs sometimes deviate from the original (if the game loader doesn't care).