Creating your own IPFs

[Keir]

Hi Folks,

I've been doing some more work on my Amiga disk decoding/encoding utilities. These allow you to read disk dumps from various sources (Kryoflux, IPF, ADF, etc) analyse the contained data according to known encoding schemes (AmigaDOS, Copylock, RNC PDOS, various game publisher custom formats, ...), and write the analysed data in another format. In effect this is an all-in-one Amiga disk archiving/transcoding/reading utility.

The new feature which I just completed is support for writing IPFs. Previously the main useful target was a custom archival format, for which support has to be added to emulators and the like (I have patches to UAE to support reading my disk files, for example).

However, with IPF support you can now create or transcode disk dumps that can be immediately loaded into emulators, read on a real Amiga (latest version of RawDIC/WHDLoad), or written back to a real disk using the latest Kryoflux software!

For example, I was able to back up my own copy of New Zealand Story (a typical Copylocked game disk) as follows. In Linux:

dtc -r6 -fnzs/ -i0 -e79 -i5
disk-analyse --format="New Zealand Story" nzs/ nzs.dsk
disk-analyse --format="New Zealand Story" nzs.dsk nzs.ipf

This dumps the game disk using Kryoflux, into a set of raw Kryoflux STREAM files. It then uses my analyser to archive in my custom format, and then again to transcode to IPF (you could also archive straight to IPF in one step).

Then, since I believe Kryoflux write support is currently only in the Windows tools, I do the writing step from a Windows VM:

 DTC -d0 -fnzs.ipf -s0 -e79 -w -l15

Result, a working copy of my own original game disk -- no cracks or patches!

You can also do things like write out unprotected ADFs through Kryoflux, by transcoding them via disk-analyse first:

 disk-analyse my.adf my.ipf

If you want to try it out, you can grab all my code from Github: http://github.com/keirf/Disk-Utilities

The bad news: It's Linux only (also basic support for Cygwin/MinGW on Windows) and you have to build it yourself, something like:

git clone http://github.com/keirf/Disk-Utilities
cd Disk-Utilities
make
su -c "make install"

Let me know if it's useful to you!

[MrCheese]

I'm going to install Ubuntu in a virtual machine to try this.

Cheers.

[Keir]

Let me know if you get stuck setting it up.

[r.cade]

Any way to get it compiled in cygwin?

[MrCheese]

Just a small update. I've got VMWare player installed and Ubuntu installed.

[Keir]

I didn't think of trying Cygwin! Yes, it was quite easy to do a basic port to Cygwin. You can now compile and run on cygwin as follows:

git clone http://github.com/keirf/Amiga-Disk-Utilities
cd Amiga-Disk-Utilities
SHARED_LIB=n make
cd mfmparse ; ./mfmparse.exe -h

I haven't tried 'make install' on Cygwin. It probably won't do the right thing! Run it out of the build directory, as above.

Originally Posted by MrCheese

Just a small update. I've got VMWare player installed and Ubuntu installed.

Cool, if you have any prior experience with Linux you shouldn't have too much trouble. I think this is the better option than Cygwin overall; neither way is exactly trivial to set up, but the VM gives you a 'proper' Linux environment. Whereas my Cygwin support is quite half hearted.

[MrCheese]

Originally Posted by kaffer

Cool, if you have any prior experience with Linux you shouldn't have too much trouble. I think this is the better option than Cygwin overall; neither way is exactly trivial to set up, but the VM gives you a 'proper' Linux environment. Whereas my Cygwin support is quite half hearted.

Little bit of experience. I'm going to be guided by your forum posts. I'm at work at the mo, so it will be a few hours from now. Will have to dig out some originals and whip em through the Kyroflux. Any suggestions as to a good original to try?

[Keir]

Originally Posted by MrCheese

Little bit of experience. I'm going to be guided by your forum posts. I'm at work at the mo, so it will be a few hours from now. Will have to dig out some originals and whip em through the Kyroflux. Any suggestions as to a good original to try?

If you're talking about copy-protected game disks, then any that are listed in the mfmparse/mfmparse_formats file should work well (although there can be various releases of any given game, with different protections in some cases, but you're somewhat unlikely to experience this).

The point is, you have to give mfmparse some guidance to analyse your disk dump. That is what the --format=blah option is all about -- you specify a name listed in the mfmparse_formats file, which lists all the disks that I've had time to check the format of.

If your disk is not listed, but you're lucky, you may find that the protection is something incredibly common, like Copylock. In which case specifying --format=amigados_copylock will work fine. You'll have to experiment a bit

[MrCheese]

Originally Posted by kaffer

If you're talking about copy-protected game disks, then any that are listed in the mfmparse/mfmparse_formats file should work well (although there can be various releases of any given game, with different protections in some cases, but you're somewhat unlikely to experience this).

The point is, you have to give mfmparse some guidance to analyse your disk dump. That is what the --format=blah option is all about -- you specify a name listed in the mfmparse_formats file, which lists all the disks that I've had time to check the format of.

If your disk is not listed, but you're lucky, you may find that the protection is something incredibly common, like Copylock. In which case specifying --format=amigados_copylock will work fine. You'll have to experiment a bit

I may try with some WorkBench images I made to start with, see if I can fizz up some IPF files with your tools. Start small and work up

[Keir]

Originally Posted by MrCheese

I may try with some WorkBench images I made to start with, see if I can fizz up some IPF files with your tools. Start small and work up

And in that case you don't need to specify --format=amigados, because it is the default. If you can get as far as a working workflow with plain AmigaDOS disks, you're actually most of the way there. Good luck.

[MrCheese]

Right, Ubuntu all working. Followed your instructions, but with 1 change:

git clone http://github.com/keirf/Amiga-Disk-Utilities
cd Amiga-Disk-Utilities
make
sudo su -c "make install"

Then I copied into the Amiga-Disk-Utilities the following files (made via KyroFlux):

Amiga_Extras_3_1.raw
Amiga_Fonts_3_1.raw
Amiga_Install_3_1.raw
Amiga_Locale_3_1.raw
Amiga_Storage_3_1.raw
Amiga_WorkBench_3_1.raw
OS-Install.raw

Then I tried the following:

mfmparse --format=amigados Amiga_WorkBench_3_1.raw Amiga_Workbench_3_1.dsk

and get:

mfmparse: Failed to probe input file: Amiga_WorkBench_3_1.raw

No idea what, probably something dumb I've done. I added a -v to the command and get:

ch_3_1.raw Amiga_Workbench_3_1.dsk
Found format "amigados"
mfmparse: Failed to probe input file: Amiga_WorkBench_3_1.raw

The filename is correct (I used the tab auto fill jobby as well as manually typing it). Maybe the .raw file is in an unexpected format? I created it using the GUI a few weeks ago.

[off topic, if I made a workbench IPF, then created a blank disc and copied a kickstart rom image onto it, then made an IPF of the disc and uploaded them to the zone I could probably break all the rules with 1 .zip file hehe]

[Keir]

Well, you got somewhere! How did you generate these .raw files from Kryoflux? The Kryoflux format I support is generated by the -i0 option to dtc, and actually generates a file per disk track.

A single .raw file is not one I've come across. Perhaps it is a raw sector dump. If you can give me an example command line you used to generate one I may be able to say more.

Ahhhh... hang on, you used the GUI. That has a drop-down menu where you select the output format to dump to. I would need you to be using "Kryoflux stream files". Other formats would probably need support to be added to my library.

[MrCheese]

I do have the directories with all the .raw files in, 1 per track. Would they be better?

[Keir]

Originally Posted by MrCheese

I do have the directories with all the .raw files in, 1 per track. Would they be better?

Yes, they are what you need. Copy the directory across and then specify that, with a trailing /, to mfmparse. For example:

mfmparse Amiga_Workbench_3_1/ Amiga_Workbench_3_1.dsk

By the way, I think the single-file .raw images you were trying are probably "CT raw image" files. I'm planning to add support for those.

[MrCheese]

Tried the command:

mfmparse -v --format amigados Amiga_WorkBench_3_1/ out.ipf

and I get:

Found format "amigados"
mfmparse: Failed to probe input file: Amiga_WorkBench_3_1/

Same if I use out.dsk instead of .ipf.

[Keir]

Originally Posted by MrCheese

Tried the command:

mfmparse -v --format amigados Amiga_WorkBench_3_1/ out.ipf

and I get:

Found format "amigados"
mfmparse: Failed to probe input file: Amiga_WorkBench_3_1/

Same if I use out.dsk instead of .ipf.

@kaffer - check your PM's

This will be because mfmparse could not find a file Amiga_Workbench_3_1/00.0.raw -- this is what mfmparse looks for to 'probe' for the kryoflux stream format.

[MrCheese]

The first file is called track00.0.raw, then track00.1.raw etc

[Keir]

Originally Posted by MrCheese

The first file is called track00.0.raw, then track00.1.raw etc

You specify the full name of the stream files, up to the varying numeric part. So in this case the correct invocation is

mfmparse Amiga_Workbench_3_1/track out.ipf

[MrCheese]

I've got it working. The command I used is:

mfmparse -v --format amigados Amiga_WorkBench_3_1/track out.ipf

WinUAE boots nicely from out.ipf.

Well done!

[Retroplay]

This has probably been asked before elsewhere but are there any plans for CT RAW image support in the future for disk-analyse ?