Acorn Archimedes Software Preservation

[deicidal]

Originally Posted by breach

Oh! I'd have expected them to have had labels is some kind of info at least.

Nope just the date, will try look into getting them imaged tonight if i can and see if i get anywhere. Thanks for the link.

Any ideas where you can get video cables from so i can display the unit i have on my TV.

Also are there keyboard combinations that you can use instead of the mouse as i dont have one, or for that matter where you could get one of those as well.

[deicidal]

I don't suppose anyone knows if you can image the archimedes disks with an amiga?

[prowler]

No, but if you can't get all the bits you need to do it on your Archimedes, then you can do it on a PC running the DOS utility 'Arcimg'.

http://eab.abime.net/showpost.php?p=458437&postcount=91

[gizmomelb]

Originally Posted by deicidal

I don't suppose anyone knows if you can image the archimedes disks with an amiga?

I used to nibble copy Arc discs for my mates using my A500, seemed to work ok for them with copy protected stuff.

[deicidal]

Well I have ordered a usb floppy drive so the moment that gets here from china i will image them

[MarkYoungIW]

Originally Posted by Jon Abbott

Does anyone have the ability to create APD's? To ensure the density detection works on APD's, I really need an example of SD and QD discs (1.6MB). They don't need to be protected, but an APD and matching ADF would be helpful so I can compare them side by side.

I've found a grand total of 35 ADP's and have successfully decoded and mounted 32 of them. Trouble is, all the APD's are DD discs, so my assumptions about the sync marks may or may not be applicable to SD / QD discs.

Did any software even come out on SD? All the ADF/APD's I've seen so far are DD or QD. Zarch looks like it could be an odd exception as side 0 is DD and side 1 is possibly SD - not sure if that's down to the protection or not though.

Anyhow, now I've reverse engineered the APD structure, I'm looking into taking over ADFS to present mounted images as ADFS:0

Just had a look at some old messages, and Tom had told me that Zarch and Magic Pockets were the only games he was aware of that had single density sectors as well as double density sectors.

I can image some discs if you need, as have an A1200 sitting here ready for the job. Let me know.

Originally Posted by Jon Abbott

Anything hard coded to ADFS:0 will fail, if it just wants <FS>:0 you should be fine.



Multidisc games may be an issue, if you no longer have access to the WIMP to change the disc.



What's the game? Can it be installed to the HD?

What I was sort of thinking is you are making something fairly similar to WHDLoad on the Amiga which is really nice in the fact that you can store everything in the same format (rather than having some ADF's/ADP's and some HD installs etc).

With the multidisc games I have tried none of them give you access back to the wimp to enable you to change the adf. I didn't know if perhaps looking at adding a small loading script for multidisc games was a possibility later on.

[Jon Abbott]

Originally Posted by MarkYoungIW

With the multidisc games I have tried none of them give you access back to the wimp to enable you to change the adf. I didn't know if perhaps looking at adding a small loading script for multidisc games was a possibility later on.

One possible way around this is to look for keyboard combinations and have it automatically load the images associated. ie Ctrl-Alt-Shift-F1 for disc1, Ctrl-Atl-Shift-F2 for disc2. Haven't got that far yet though.

I've analysed all 32 protected APD images in detail now. Only 25 actually use protection, of those there are 18 distinct methods being used, ranging from sector within sector to invalid ID's and alternative sector sizes.

In essence I need to write a 1772 FDC emulator and feed the output into ADFS. That isn't possible though, without patching or rewriting sections of ADFS - which isn't really a viable option due to the vast number of versions out there. The alternative, which I'm trying, is to front ADFS and pass-through calls where appropriate.

I also need to expand the DiscOp functions in ADFFS, as I didn't bother coding DiscOp 3/4 (read/write track) - they weren't required for ADF images. Some of the protection methods however rely on DiscOp 3, to read sector within sector and invalid sector ID's.

Although Zarch and MagicPockets have SD track data in the APD file, it looks like DD tracks that just happen to verify when read as SD. Zarch certainly seems to load with DD track data only, as its protection is to check for alternative sized sectors on the disk. I suspect MagicPockets is the same, as it has a blank track and alternative sized sectors.

If anyone is aware of more protected discs, please let me know so I can try to source the originals to analyse them. One's I'm aware of are:

Cycloids
Darkwood
Drop Ship
The Dungeon
Fire & Ice
FTT
Gods
Gribbly's Day Out
The Last Ninja
Magic Pockets
Nevryon
Pandora's Box
Populous
Powerband
Repton 4
Sensible Soccer
StarFighter 3000
The Wimp Game
Wolfenstein
World Class Leaderboard
Zarch.

Although APD's are available for the following, they do not use disc protection - they use other methods, such as code wheels, manuals etc:

Premier Manager
Repton 3
Stunt Racer 2000

Seems, like an awful lot of coding for so few discs!

[MarkYoungIW]

There can be lots more. I have a number not there. Leave it with me, I'll eMail you some. Tom's APD seems to work for all protected discs, and there are many protected discs for the Arc. I will get on that today.

[breach]

According to this thread http://www.iconbar.com/forums/viewth...d=11868&page=1 Inferno and Quark use copy protection.

I actually have both original games but I don't know how to make APD format files.

APD seems to be a bit too closed a format for useful preservation.

[MarkYoungIW]

I dont think it is intentionally closed and I am sure it will soon be fully documented (also Jon sounds like he has pretty much reversed it now already!!)

With regards to making an apd its fairly simple. It involves Rawread on the Amiga like this and a windows app Tom made called getapd.exe

There were a lot of games that were protected in many ways, I am sure not all of these require apd, but some definately do.

Hope this helps.

[Jon Abbott]

Originally Posted by breach

APD seems to be a bit too closed a format for useful preservation.

It's actually very basic, 160 raw MFM tracks, with FM SD and MFM QD tracks where those densities verify. From what I've seen so far, it's more than sufficient for preservation of all discs.

[breach]

Originally Posted by MarkYoungIW

With regards to making an apd its fairly simple. It involves Rawread on the Amiga like this and a windows app Tom made called getapd.exe

Is there any alternative to the Amiga & Rawread? I don't have an Amiga. Actually the only system I have with a floppy drive is the Acorn.

Originally Posted by Jon Abbott

It's actually very basic, 160 raw MFM tracks, with FM SD and MFM QD tracks where those densities verify. From what I've seen so far, it's more than sufficient for preservation of all discs.

OK, well that's good to know.

[Jon Abbott]

I won't post off topic updates here, but I'm making good progress with ADFFS, APD support is getting there and I now have control of ADFS::0

See the following thread for further updates:
http://riscos.me.uk/forums/viewthrea...threadid=11846

I've heard nothing from SPS after raising a ticket a few weeks ago, so no progress on IPF support.

[MarkYoungIW]

Hi Jon,
Many thanks for the updates. I think most people would agree with me that ADFFS will be as important to Acorn as WHDLoad has been to the Amiga, if not more so!

Really pleased to hear you have got ADFS::0 that could have been a big problem with some images.

Thank you for your work for the Acorn scene, with this for the real hardware and Tom's Arculator for those times we can't be bothered to get the Arc out of the loft, I think its fair to say it makes all the disc images, so much more meaningful!!

[Jon Abbott]

I have a few bugs to iron out, but have over half the APD's now working under ADFFS. Mark - I'll send you an updated beta later, it's still RO 3.5+ only. If that's an issue let me know and I'll spend some time getting backward compatibility fixed.

Anyhow, whilst analysing the various protection methods used, it dawned on me that I could probably automate what I'm doing, to allow floppies to be captured on the Arc. Essentially what I'm doing is recording all the disc access and responses from ADFS and then replaying them in ADFFS, to confirm I'm returning the correct data and errors.

However, there's no reason why this couldn't be automated and possibly with a little user input for the more complex methods, the disc captured into a working image. The bulk of the disc can be read sector by sector, based on the core Disc Rec - the other Disc Recs for the protection areas are all captured by running the game and recording it's disc access. It's then a matter of filling in the gaps in the protection tracks, noting any unformatted tracks / sectors with CRC errors in the data or sector ID - this may not be required, it obviously depends on how the protection is performed. If all the protection checking is done up front, it should be reliable. Sector within sector may need some hand holding, where DiscOp 3 (read track) is used (Fire & Ice for example).

Just a thought, obviously I want to get ADFFS working with APD's reliably first before thinking about extending ADFFS, to add reconstruction.

It will however probably need yet another file format, as IPF / APD are floppy image based, rather than data image based. I suppose it's possible to convert the data back into MFM and reconstruct an IPF / APD - but it won't be a 100% accurate representation as they are currently.

[breach]

Wait, wait, wait! This all sounds spectacularly awesome, even though I'm not completely sure I understand it!

Are you saying you could release a utility that would be able to create APDs on an Acorn? But that it would have to go through an intermediate format first?

[Jon Abbott]

Originally Posted by breach

Are you saying you could release a utility that would be able to create APDs on an Acorn? But that it would have to go through an intermediate format first?

That is exactly what I'm saying.

[deicidal]

Originally Posted by deicidal

Well I have ordered a usb floppy drive so the moment that gets here from china i will image them

Didnt think that one through, the disk imaging software doesn't work with usb drives.

[Jon Abbott]

There's an early beta of ADFFS with APD support available, head over to the Icon Bar thread to see the caveats before downloading though.

http://riscos.me.uk/forums/viewthrea...&page=3#119972

EDIT: 1.21 is available, which is a stable release supporting virtually all APD's: http://adffs.filecore.net

[Jon Abbott]

Now that APD support is nearly complete, I'm looking at expanding ADFFS into a complete retro gaming solution, along the lines of MAME/MAMEUI.

Some of you may be able to help out, with instructions, scans, screenshots etc. Take a look at the "help wanted" section on the Icon Bar thread and let me know your thoughts/ideas:

http://riscos.me.uk/forums/viewthrea...threadid=11846

[deicidal]

RISC magazine cover disks in the zone

[breach]

[About making a utility to produce APD images on Acorn machines]

Originally Posted by Jon Abbott

That is exactly what I'm saying.

Wow, cool! That would be great! It should make a lot more people able to create APD images, which will make them easier to find, and help us preserve more old software.

Originally Posted by deicidal

RISC magazine cover disks in the zone

Nice! You found a way to image your discs then?

I'll have a look at them now.

[deicidal]

Originally Posted by breach

[About making a utility to produce APD images on Acorn machines]

Nice! You found a way to image your discs then?

I'll have a look at them now.

Yeh I used the program Prowler recomended and nicked an old laptop from work to read them with.

There was also some miscillaneouse disks as well i can shove those in the zone as well if you want to see if there is anything of value on them.

[prowler]

Hi deicidal,

Thanks for your uploads.

It's good to hear that you found the 'Arcimg' program(s) useful (That post I linked to earlier was actually my first post on EAB!).

And yes, please upload those miscellaneous disks too, if you can get valid images of them.

[Jon Abbott]

Originally Posted by breach

[About making a utility to produce APD images on Acorn machines]


Wow, cool! That would be great! It should make a lot more people able to create APD images, which will make them easier to find, and help us preserve more old software.

I need to define a new image format before I can proceed, unfortunately, as it will be reading data from the disc and not raw MFM/FM, APD, IPF etc are not an option. ADF is out as its not sector aware.

Before I dictate yet another image onto the world, I'd like feedback from the community on what they'd like in it. As APD support is almost complete, I'm now starting to look at starting the capture code, my ideas so far for the standard are:

Defined directory structure containing the following:
SOURCE\version - ASCII file containing the game version
SOURCE\desc - ASCII file containing the game description
SOURCE\history - ASCII file containing the game history
SOURCE\instruct - ASCII file containing the game instructions
SOURCE\password - ASCII file containing level passwords (optional)
IMAGE\version - ASCII file containing the disc image version
IMAGE\changelog - ASCII file containing the disc image version changes
IMAGE\desc - ASCII file containing source image related info
ART\COVER\PNG files of cover art. cover1.png being the primary (optional)
ART\MARKET\PNG files of marketing material (optional)
SNAP\PNG files containing screen shots. snap01.png being the primary
ICON\containing a !Sprites RISC OS file and a Windows Icon file
PATCHES\Patches for modern kit/RISC OS - format to be decided (optional)
SECURITY\PNG files containing code wheels or ASCII files containing equivalent (optional)
CHEATS\Cheat files - format to be decided (optional)
DISCS\Multiple disc images for multi-disc games, disc1.? being the primary, containing:
- Track table (a pointer into the sector table)
- Sector table (with size, density, data CRC, sector no, pointer into the data)
- Data
- Delta changes to allow read/write as some games save data back to the disc

These files will be wrapped into one file and deflated with ZLib. The wrapper file could be an IFF style or ZIP for flexibility.

So far I've disappointingly had zero responses.

[breach]

Ah, you want to make a format that contains a lot more than just the disc image?

I don't really understand why APD is not good enough for the disc image portion. I thought it was flexible enough to cover all known protection methods?

A ZIP wrapper, with defined directory structure & filenames sounds best to me. That way it's very easy to update with new material.

[Jon Abbott]

Originally Posted by breach

Ah, you want to make a format that contains a lot more than just the disc image?

I don't really understand why APD is not good enough for the disc image portion. I thought it was flexible enough to cover all known protection methods?

I want to do away with the need for a FDC emulator, this greatly simplifies implementing the standard into emulators. Currently, every emulator would have to implement it's own variation on a 1772 / 82C710 / 82C711.

Discs that are not FDC agnostic (eg Fire & Ice) currently wont work on a 1772 emulator, but the problem goes away if we're not emulating the FDC. I don't know how many other discs this affects.

By doing away with the FDC emulator, you also do away with the need to decode FM/MFM into raw before passing it to the OS, which speeds up disc image loading greatly. ADFFS is taking around 15 secs on a 33MHz CPU.

The final nail in the coffin for storing FM/MFM data is that you can't read it from the floppy on an Arc, so if I'm going to code a disc imager, we don't really want to have try to reconstruct the FM/MFM to only have to decode it back again at load time.

Originally Posted by breach

A ZIP wrapper, with defined directory structure & filenames sounds best to me. That way it's very easy to update with new material.

Agree, directories are a good idea to keep it tidy and makes extending easier. I'll add that to the spec.

[breach]

Ah, I understand! A simpler format for protected discs would be great, as would being able to produce them on Acorn machines.

An APD --> [new simplified format] converter would be useful, to avoid the performance overheads with existing APDs.

[Jon Abbott]

Originally Posted by breach

An APD --> [new simplified format] converter would be useful, to avoid the performance overheads with existing APDs.

Already coded on the Arc. As I have to decode them at runtime, I coded the whole thing as a standalone package, which saves the end result for testing. It just needs modifying to match the structure, once we fix the new standard.

[jms2]

Jon,

As you've had so few replies about the new data format, I thought I should make a post to register my enthusiasm for what you're doing. Its really superb and I do appreciate it, but what has been holding me back from commenting is an underlying lack of confidence that I understand all the technical issues involved, and and the fact that I've not actually had time to try out ADFFS as yet. However, here goes...

1) Your proposed format containing metadata looks similar to the "UEF" format used quite extensively for BBC micro tape images. There is no particular reason that I can think of why you'd need to use the UEF format, but it might give you some ideas.

2) A key issue for me (and I assume everyone else) is how to transfer disk images onto a real machine. I can only use DOS 720k floppies, so the image has to be smaller than this.

3) The alternative approach, presumably, would be to crack the protection on these games and release unprotected images. This was certainly done back in the day although the cracked images don't seem to be in circulation any longer. Would cracking all the APD games be any easier that the approach you are suggesting?

[Jon Abbott]

Originally Posted by jms2

1) Your proposed format containing metadata looks similar to the "UEF" format used quite extensively for BBC micro tape images. There is no particular reason that I can think of why you'd need to use the UEF format, but it might give you some ideas.

I'll look it up for ideas.

Originally Posted by jms2

2) A key issue for me (and I assume everyone else) is how to transfer disk images onto a real machine. I can only use DOS 720k floppies, so the image has to be smaller than this.

I networked my RISC PC specifically for this project, but I can see that's not always going to be possible. How do people do it currently? Floppy isn't going to be a realistic option for much longer, another alternative is via RS232 if networking isn't available. Has anyone written an free RS232 based Filesystem for the Arc? I've seen a commercial package.

As an HD is a requirement for ADFFS, I don't think file size itself is going to be an issue - just transferring it across. Sounds like we need to add a PC<>Arc transfer package to the list of actions if we can't find an open source one.

Originally Posted by jms2

3) The alternative approach, presumably, would be to crack the protection on these games and release unprotected images. This was certainly done back in the day although the cracked images don't seem to be in circulation any longer. Would cracking all the APD games be any easier that the approach you are suggesting?

This isn't an option, the idea is to preserve the software with protection intact and get it working on RISC OS 3.1 minimum, RISC OS 3.7 preferably.

[jms2]

I've just tried ADFFS v1.21 on my A420/1 with Risc OS 3.1. It all works superbly, and I note that in reality the APD files are considerably smaller than 720k so the transfer issue is not that serious.

I did come across some minor problems though which I thought you would want to know about.

1) I tried Drop Ship and, whilst it worked, I immediately ran into the multi disc issue that other posters have referred to.
2) I then tried The Last Ninja. ADFFS seemed to work perfectly but the game did not run. Instead it reported "Cannot move memory". I wonder if there is simply something missing from the image.
3) The command *ADF does not seem to exist, so ADFFS can't be used from the command line.

[Jon Abbott]

Originally Posted by jms2

1) I tried Drop Ship and, whilst it worked, I immediately ran into the multi disc issue that other posters have referred to.

I'm coding the multi-disc support as we speak. Was hoping to have it up today, but having a few issues with the IRQ handler.

Originally Posted by jms2

2) I then tried The Last Ninja. ADFFS seemed to work perfectly but the game did not run. Instead it reported "Cannot move memory". I wonder if there is simply something missing from the image.

Read the "Game specific issues" on http://www.iconbar.com/forums/viewth...&page=3#119907, sounds like the Last Ninja fix needs applying to RO 3.1 as well.

Originally Posted by jms2

3) The command *ADF does not seem to exist, so ADFFS can't be used from the command line.

Use *DIR adfs::0.$ instead. Versions of ADFFS above 1.02 take over :0 when a floppy is mounted.

[antom]

Firstly, a tip for anyone struggling to transfer disc images via 720k DOS floppy - there's an app out there for Risc OS called Chopper that's freeware and available from the APDL website.

Secondly, there's a few different types of copy protection I'm aware of from games I own/have in the past as follows:

4D - usual module based affair that you're probably familiar with, so won't go further than that.

Krysalis - Some games such as Sim City have a WIMP-based copy protection pop-up on loading. From what I remember, the game runs but in the case of Sim City the rate of disaster incidences is multiplied to try and make it as unplayable as possible! Or that was the intention, at least... possibly used on some GamesWare/Software 42 releases (Xenon II I seem to recall).

In fact, Raw Power by S42 had an interesting one where apparently the buyer is asked for their details and this is written to the key disc somewhere, presumably to trace the origin of any copies floating about. Unsure if this would affect loading/gameplay.

Other than that, it's the usual bad sector/track on the disc that need to be read as a "fail" in order for the game to boot.

Finally, some suggestions regarding your proposed image format:

CHEATS\Cheat files - format to be decided (optional)

For this if possible I'd suggest either ASCII for text-based cheats (although that could be covered by the proposed password section - also, wouldn't this need to be unlimited for, as an example, a full walkthrough/guide) or possibly a separate partition or zip/archive containing Risc OS format files for cheating (would be nice to keep them separate from the original where possible so things are kept in an "as-released" format, plus there's plenty of formats to consider - BASIC mods for save games, modules created from The Hacker/Games Wizard, draw files for maps etc)

Also it would be great to have fields for author, distributor, year of release, genre and possibly cover art? Know I'm starting to ask a lot to cram into the file now!

Also, I know said I'd post some files up to the Zone the other week - I'll get around to this ASAP!

[breach]

Originally Posted by Jon Abbott

Already coded on the Arc. As I have to decode them at runtime, I coded the whole thing as a standalone package, which saves the end result for testing. It just needs modifying to match the structure, once we fix the new standard.

Great! That will give the new format a head start in terms of games available for it.

Originally Posted by jms2

2) A key issue for me (and I assume everyone else) is how to transfer disk images onto a real machine. I can only use DOS 720k floppies, so the image has to be smaller than this.

If the new images get too big for transferring to actual hardware via floppy disc, I think that would be a good reason for the new format to be basically a ZIP file. That way, if the software image is too big due to cover art and manual scans, then it should be possible to open up the ZIP file, and take a copy of just the disc image part, for transferring.

Originally Posted by Jon Abbott

I'm coding the multi-disc support as we speak. Was hoping to have it up today, but having a few issues with the IRQ handler.

How will the changing of discs be handled from a user's POV? Some key combination?

Originally Posted by antom

Also, I know said I'd post some files up to the Zone the other week - I'll get around to this ASAP!

Cool. Are they APD format? Or all unprotected software?

[Jon Abbott]

ADFFS 1.22 now available with multi-disc support. See the thread on The Icon Bar for the caveats.

Originally Posted by antom

I'd suggest either ASCII for text-based cheats (although that could be covered by the proposed password section - also, wouldn't this need to be unlimited for, as an example, a full walkthrough/guide) or possibly a separate partition or zip/archive containing Risc OS format files for cheating (would be nice to keep them separate from the original where possible so things are kept in an "as-released" format, plus there's plenty of formats to consider - BASIC mods for save games, modules created from The Hacker/Games Wizard, draw files for maps etc)

That area can be unlimited. For the game cheats it will need to support RISC OS files, I haven't yet decided if they should be in compressed form or another image file. Definitely separate though, as the disk image needs to remain a vanilla snapshot.

Originally Posted by antom

Also it would be great to have fields for author, distributor, year of release, genre and possibly cover art? Know I'm starting to ask a lot to cram into the file now!

I agree, to save clutter the "desc" file should probably have some standard format to contain such information. ie XML or similar.

Originally Posted by breach

If the new images get too big for transferring to actual hardware via floppy disc, I think that would be a good reason for the new format to be basically a ZIP file. That way, if the software image is too big due to cover art and manual scans, then it should be possible to open up the ZIP file, and take a copy of just the disc image part, for transferring.

Sounds sensible to me.

Originally Posted by breach

How will the changing of discs be handled from a user's POV? Some key combination?

You guessed it...currently Ctrl-Shift-F1 to F9

[deicidal]

Other miscellaneouse disks uploaded to the zone, just have a bit of caution i dont know if it was me bug one of them buggered up archulator so i wouldnt want anyone messing up a nice setup. Let me know if you find anything interesting

[prowler]

Originally Posted by deicidal

Other miscellaneouse disks uploaded to the zone...

Thanks, mate. I'll check them when I get a chance.

Edit: A few of them look almost empty from the packed size.

[antom]

Originally Posted by breach

Cool. Are they APD format? Or all unprotected software?

All in ADF format using !DImager - some of the games state that they're copy protected but an image was created fine, so I'm not sure if there'd be any gameplay-affecting routines in there. Ideally the key discs would be in APD/Jon's proposed format, but can always redo these as things develop (don't have the hardware for the other methods at present).

I need to sort stuff out still with them still before uploading (and remember where I've saved them! )

[jms2]

I have tried out v1.22, plus the tweak to The Last Ninja and all is working fine now!

Just out of interest, why does the Setup file need to be commented out? If ADFFS is correctly emulating the protected disc, surely any kind of protection mechanism ought to still work?

Firstly, a tip for anyone struggling to transfer disc images via 720k DOS floppy - there's an app out there for Risc OS called Chopper that's freeware and available from the APDL website.

The Chopper utility (including its DOS-based components for splitting the files) looks excellent (especially for full size 800k ADFs) - thanks!

Serious comment - I reckon ADFFS is the most significant piece of new Risc OS software for years. There hasn't much competition for a long time so you'd have to go back to the 1990s to find anything as innovative and useful. Well done Jon.