Acorn Archimedes Software Preservation

[deicidal]

RISC magazine cover disks in the zone

[breach]

[About making a utility to produce APD images on Acorn machines]

Originally Posted by Jon Abbott

That is exactly what I'm saying.

Wow, cool! That would be great! It should make a lot more people able to create APD images, which will make them easier to find, and help us preserve more old software.

Originally Posted by deicidal

RISC magazine cover disks in the zone

Nice! You found a way to image your discs then?

I'll have a look at them now.

[deicidal]

Originally Posted by breach

[About making a utility to produce APD images on Acorn machines]

Nice! You found a way to image your discs then?

I'll have a look at them now.

Yeh I used the program Prowler recomended and nicked an old laptop from work to read them with.

There was also some miscillaneouse disks as well i can shove those in the zone as well if you want to see if there is anything of value on them.

[prowler]

Hi deicidal,

Thanks for your uploads.

It's good to hear that you found the 'Arcimg' program(s) useful (That post I linked to earlier was actually my first post on EAB!).

And yes, please upload those miscellaneous disks too, if you can get valid images of them.

[Jon Abbott]

Originally Posted by breach

[About making a utility to produce APD images on Acorn machines]


Wow, cool! That would be great! It should make a lot more people able to create APD images, which will make them easier to find, and help us preserve more old software.

I need to define a new image format before I can proceed, unfortunately, as it will be reading data from the disc and not raw MFM/FM, APD, IPF etc are not an option. ADF is out as its not sector aware.

Before I dictate yet another image onto the world, I'd like feedback from the community on what they'd like in it. As APD support is almost complete, I'm now starting to look at starting the capture code, my ideas so far for the standard are:

Defined directory structure containing the following:
SOURCE\version - ASCII file containing the game version
SOURCE\desc - ASCII file containing the game description
SOURCE\history - ASCII file containing the game history
SOURCE\instruct - ASCII file containing the game instructions
SOURCE\password - ASCII file containing level passwords (optional)
IMAGE\version - ASCII file containing the disc image version
IMAGE\changelog - ASCII file containing the disc image version changes
IMAGE\desc - ASCII file containing source image related info
ART\COVER\PNG files of cover art. cover1.png being the primary (optional)
ART\MARKET\PNG files of marketing material (optional)
SNAP\PNG files containing screen shots. snap01.png being the primary
ICON\containing a !Sprites RISC OS file and a Windows Icon file
PATCHES\Patches for modern kit/RISC OS - format to be decided (optional)
SECURITY\PNG files containing code wheels or ASCII files containing equivalent (optional)
CHEATS\Cheat files - format to be decided (optional)
DISCS\Multiple disc images for multi-disc games, disc1.? being the primary, containing:
- Track table (a pointer into the sector table)
- Sector table (with size, density, data CRC, sector no, pointer into the data)
- Data
- Delta changes to allow read/write as some games save data back to the disc

These files will be wrapped into one file and deflated with ZLib. The wrapper file could be an IFF style or ZIP for flexibility.

So far I've disappointingly had zero responses.

[breach]

Ah, you want to make a format that contains a lot more than just the disc image?

I don't really understand why APD is not good enough for the disc image portion. I thought it was flexible enough to cover all known protection methods?

A ZIP wrapper, with defined directory structure & filenames sounds best to me. That way it's very easy to update with new material.

[Jon Abbott]

Originally Posted by breach

Ah, you want to make a format that contains a lot more than just the disc image?

I don't really understand why APD is not good enough for the disc image portion. I thought it was flexible enough to cover all known protection methods?

I want to do away with the need for a FDC emulator, this greatly simplifies implementing the standard into emulators. Currently, every emulator would have to implement it's own variation on a 1772 / 82C710 / 82C711.

Discs that are not FDC agnostic (eg Fire & Ice) currently wont work on a 1772 emulator, but the problem goes away if we're not emulating the FDC. I don't know how many other discs this affects.

By doing away with the FDC emulator, you also do away with the need to decode FM/MFM into raw before passing it to the OS, which speeds up disc image loading greatly. ADFFS is taking around 15 secs on a 33MHz CPU.

The final nail in the coffin for storing FM/MFM data is that you can't read it from the floppy on an Arc, so if I'm going to code a disc imager, we don't really want to have try to reconstruct the FM/MFM to only have to decode it back again at load time.

Originally Posted by breach

A ZIP wrapper, with defined directory structure & filenames sounds best to me. That way it's very easy to update with new material.

Agree, directories are a good idea to keep it tidy and makes extending easier. I'll add that to the spec.

[breach]

Ah, I understand! A simpler format for protected discs would be great, as would being able to produce them on Acorn machines.

An APD --> [new simplified format] converter would be useful, to avoid the performance overheads with existing APDs.

[Jon Abbott]

Originally Posted by breach

An APD --> [new simplified format] converter would be useful, to avoid the performance overheads with existing APDs.

Already coded on the Arc. As I have to decode them at runtime, I coded the whole thing as a standalone package, which saves the end result for testing. It just needs modifying to match the structure, once we fix the new standard.

Jon,

As you've had so few replies about the new data format, I thought I should make a post to register my enthusiasm for what you're doing. Its really superb and I do appreciate it, but what has been holding me back from commenting is an underlying lack of confidence that I understand all the technical issues involved, and and the fact that I've not actually had time to try out ADFFS as yet. However, here goes...

1) Your proposed format containing metadata looks similar to the "UEF" format used quite extensively for BBC micro tape images. There is no particular reason that I can think of why you'd need to use the UEF format, but it might give you some ideas.

2) A key issue for me (and I assume everyone else) is how to transfer disk images onto a real machine. I can only use DOS 720k floppies, so the image has to be smaller than this.

3) The alternative approach, presumably, would be to crack the protection on these games and release unprotected images. This was certainly done back in the day although the cracked images don't seem to be in circulation any longer. Would cracking all the APD games be any easier that the approach you are suggesting?

[Jon Abbott]

Originally Posted by jms2

1) Your proposed format containing metadata looks similar to the "UEF" format used quite extensively for BBC micro tape images. There is no particular reason that I can think of why you'd need to use the UEF format, but it might give you some ideas.

I'll look it up for ideas.

Originally Posted by jms2

2) A key issue for me (and I assume everyone else) is how to transfer disk images onto a real machine. I can only use DOS 720k floppies, so the image has to be smaller than this.

I networked my RISC PC specifically for this project, but I can see that's not always going to be possible. How do people do it currently? Floppy isn't going to be a realistic option for much longer, another alternative is via RS232 if networking isn't available. Has anyone written an free RS232 based Filesystem for the Arc? I've seen a commercial package.

As an HD is a requirement for ADFFS, I don't think file size itself is going to be an issue - just transferring it across. Sounds like we need to add a PC<>Arc transfer package to the list of actions if we can't find an open source one.

Originally Posted by jms2

3) The alternative approach, presumably, would be to crack the protection on these games and release unprotected images. This was certainly done back in the day although the cracked images don't seem to be in circulation any longer. Would cracking all the APD games be any easier that the approach you are suggesting?

This isn't an option, the idea is to preserve the software with protection intact and get it working on RISC OS 3.1 minimum, RISC OS 3.7 preferably.

I've just tried ADFFS v1.21 on my A420/1 with Risc OS 3.1. It all works superbly, and I note that in reality the APD files are considerably smaller than 720k so the transfer issue is not that serious.

I did come across some minor problems though which I thought you would want to know about.

1) I tried Drop Ship and, whilst it worked, I immediately ran into the multi disc issue that other posters have referred to.
2) I then tried The Last Ninja. ADFFS seemed to work perfectly but the game did not run. Instead it reported "Cannot move memory". I wonder if there is simply something missing from the image.
3) The command *ADF does not seem to exist, so ADFFS can't be used from the command line.

[Jon Abbott]

Originally Posted by jms2

1) I tried Drop Ship and, whilst it worked, I immediately ran into the multi disc issue that other posters have referred to.

I'm coding the multi-disc support as we speak. Was hoping to have it up today, but having a few issues with the IRQ handler.

Originally Posted by jms2

2) I then tried The Last Ninja. ADFFS seemed to work perfectly but the game did not run. Instead it reported "Cannot move memory". I wonder if there is simply something missing from the image.

Read the "Game specific issues" on http://www.iconbar.com/forums/viewth...&page=3#119907, sounds like the Last Ninja fix needs applying to RO 3.1 as well.

Originally Posted by jms2

3) The command *ADF does not seem to exist, so ADFFS can't be used from the command line.

Use *DIR adfs::0.$ instead. Versions of ADFFS above 1.02 take over :0 when a floppy is mounted.

[antom]

Firstly, a tip for anyone struggling to transfer disc images via 720k DOS floppy - there's an app out there for Risc OS called Chopper that's freeware and available from the APDL website.

Secondly, there's a few different types of copy protection I'm aware of from games I own/have in the past as follows:

4D - usual module based affair that you're probably familiar with, so won't go further than that.

Krysalis - Some games such as Sim City have a WIMP-based copy protection pop-up on loading. From what I remember, the game runs but in the case of Sim City the rate of disaster incidences is multiplied to try and make it as unplayable as possible! Or that was the intention, at least... possibly used on some GamesWare/Software 42 releases (Xenon II I seem to recall).

In fact, Raw Power by S42 had an interesting one where apparently the buyer is asked for their details and this is written to the key disc somewhere, presumably to trace the origin of any copies floating about. Unsure if this would affect loading/gameplay.

Other than that, it's the usual bad sector/track on the disc that need to be read as a "fail" in order for the game to boot.

Finally, some suggestions regarding your proposed image format:

CHEATS\Cheat files - format to be decided (optional)

For this if possible I'd suggest either ASCII for text-based cheats (although that could be covered by the proposed password section - also, wouldn't this need to be unlimited for, as an example, a full walkthrough/guide) or possibly a separate partition or zip/archive containing Risc OS format files for cheating (would be nice to keep them separate from the original where possible so things are kept in an "as-released" format, plus there's plenty of formats to consider - BASIC mods for save games, modules created from The Hacker/Games Wizard, draw files for maps etc)

Also it would be great to have fields for author, distributor, year of release, genre and possibly cover art? Know I'm starting to ask a lot to cram into the file now!

Also, I know said I'd post some files up to the Zone the other week - I'll get around to this ASAP!

[breach]

Originally Posted by Jon Abbott

Already coded on the Arc. As I have to decode them at runtime, I coded the whole thing as a standalone package, which saves the end result for testing. It just needs modifying to match the structure, once we fix the new standard.

Great! That will give the new format a head start in terms of games available for it.

Originally Posted by jms2

2) A key issue for me (and I assume everyone else) is how to transfer disk images onto a real machine. I can only use DOS 720k floppies, so the image has to be smaller than this.

If the new images get too big for transferring to actual hardware via floppy disc, I think that would be a good reason for the new format to be basically a ZIP file. That way, if the software image is too big due to cover art and manual scans, then it should be possible to open up the ZIP file, and take a copy of just the disc image part, for transferring.

Originally Posted by Jon Abbott

I'm coding the multi-disc support as we speak. Was hoping to have it up today, but having a few issues with the IRQ handler.

How will the changing of discs be handled from a user's POV? Some key combination?

Originally Posted by antom

Also, I know said I'd post some files up to the Zone the other week - I'll get around to this ASAP!

Cool. Are they APD format? Or all unprotected software?

[Jon Abbott]

ADFFS 1.22 now available with multi-disc support. See the thread on The Icon Bar for the caveats.

Originally Posted by antom

I'd suggest either ASCII for text-based cheats (although that could be covered by the proposed password section - also, wouldn't this need to be unlimited for, as an example, a full walkthrough/guide) or possibly a separate partition or zip/archive containing Risc OS format files for cheating (would be nice to keep them separate from the original where possible so things are kept in an "as-released" format, plus there's plenty of formats to consider - BASIC mods for save games, modules created from The Hacker/Games Wizard, draw files for maps etc)

That area can be unlimited. For the game cheats it will need to support RISC OS files, I haven't yet decided if they should be in compressed form or another image file. Definitely separate though, as the disk image needs to remain a vanilla snapshot.

Originally Posted by antom

Also it would be great to have fields for author, distributor, year of release, genre and possibly cover art? Know I'm starting to ask a lot to cram into the file now!

I agree, to save clutter the "desc" file should probably have some standard format to contain such information. ie XML or similar.

Originally Posted by breach

If the new images get too big for transferring to actual hardware via floppy disc, I think that would be a good reason for the new format to be basically a ZIP file. That way, if the software image is too big due to cover art and manual scans, then it should be possible to open up the ZIP file, and take a copy of just the disc image part, for transferring.

Sounds sensible to me.

Originally Posted by breach

How will the changing of discs be handled from a user's POV? Some key combination?

You guessed it...currently Ctrl-Shift-F1 to F9

[deicidal]

Other miscellaneouse disks uploaded to the zone, just have a bit of caution i dont know if it was me bug one of them buggered up archulator so i wouldnt want anyone messing up a nice setup. Let me know if you find anything interesting

[prowler]

Originally Posted by deicidal

Other miscellaneouse disks uploaded to the zone...

Thanks, mate. I'll check them when I get a chance.

Edit: A few of them look almost empty from the packed size.

[antom]

Originally Posted by breach

Cool. Are they APD format? Or all unprotected software?

All in ADF format using !DImager - some of the games state that they're copy protected but an image was created fine, so I'm not sure if there'd be any gameplay-affecting routines in there. Ideally the key discs would be in APD/Jon's proposed format, but can always redo these as things develop (don't have the hardware for the other methods at present).

I need to sort stuff out still with them still before uploading (and remember where I've saved them! )

I have tried out v1.22, plus the tweak to The Last Ninja and all is working fine now!

Just out of interest, why does the Setup file need to be commented out? If ADFFS is correctly emulating the protected disc, surely any kind of protection mechanism ought to still work?

Firstly, a tip for anyone struggling to transfer disc images via 720k DOS floppy - there's an app out there for Risc OS called Chopper that's freeware and available from the APDL website.

The Chopper utility (including its DOS-based components for splitting the files) looks excellent (especially for full size 800k ADFs) - thanks!

Serious comment - I reckon ADFFS is the most significant piece of new Risc OS software for years. There hasn't much competition for a long time so you'd have to go back to the 1990s to find anything as innovative and useful. Well done Jon.