Caps Hardware project

[whiteb]

Hi folks.

I had a thought (ouch), one of the projects CAPS has, is maybe one day have a hardware hack for a disk drive to allow writing IPF's back to floppy for loading on a REAL amiga.

Now we have "Catweisel" (SP?) the controller board that uses PC drives, could the device driver for the PCI one (new) be updated to allow writing IPF's back ?, maybe the same can be done for the A1200 (clock port) version ?

Or am I once again way off mark ?

[fiath]

It will happen one day. IPF's are specifically designed for mastering back to disk. They actually contain data that you wouldn't use for anything else.

Unfortunately, there is no way to reliably write IPF's back to disk using a Catweasel. We won't do a half-baked solution when we know who people will complain to - us - when it would be entirely out of our control.

We will build hardware to do it. Hopefully a fully fledged mastering/dumping solution similar or better in quality than the commerical mastering machines at the time (which takes some doing). But it will take quite a bit of time, and quite a bit of money - both of which we can't spare any more of right now, and unfortunately not for the near future either.

[whiteb]

Originally posted by fiath
It will happen one day. IPF's are specifically designed for mastering back to disk. They actually contain data that you wouldn't use for anything else.

Unfortunately, there is no way to reliably write IPF's back to disk using a Catweasel. We won't do a half-baked solution when we know who people will complain to - us - when it would be entirely out of our control.

Worth a shot, what with catweasel supporting just about every floppy format there is

[killergorilla]

Are there any plans for mounting of ipf files within the amiga environment? I remember things were mentioned on this ages back I was just wondering if any more thoughts or discussion has taken place regarding it?

PS: Got through all those emails yet Fiath?

[fiath]

Originally posted by whiteb
Worth a shot, what with catweasel supporting just about every floppy format there is

Main system formats. It doesn't support the 100's upon 100's of Amiga formats and their variations. Far too much work, who would be crazy enough to do that!...

Actually, you made an important point. The Catweasel supports disk formats, not copy protection. The two things are quite different.

[fiath]

Originally posted by killergorilla
Are there any plans for mounting of ipf files within the amiga environment? I remember things were mentioned on this ages back I was just wondering if any more thoughts or discussion has taken place regarding it?

Yes, it is planned to go in the next version of the Amiga library. However, the person doing it was in the middle of some exams/qualifications and so it was put off for a while. I'm not sure when they will be able to pick it up again.

PS: Got through all those emails yet Fiath?

Actually, yes. However, yours will take a little home working which I have not been able to get round to yet. Sorry about that.

[killergorilla]

Not a problem Fiath. Not at all.

Answer whenever you can. Actually can you answer the DAT email last as I need to change it a little. I'll re-email an amended copy once you reply to the other one. (The Beast dumping one)

If that makes any sense whatsoever

[Unknown_K]

What percentage of the 1000+ titles that got imaged really need a duplicating type machine to put them back onto disks?


I was just wondering if a device like the old ISA Central Point deluxe option board could be programmed to do 90% of the disk images. this thing could copy just about anything that didnt have physical protection like laser pinholes in the media. I am sure very few people want to buy or make a duplicating machine just to replace an original disk that went bad (out of curiosity what make/model machine is needed to do this kind of duplicating anyway?).

[fiath]

@KG: Thanks. Will do.

@UnknownK:

The kind of machine that did most of the mastering for Amiga disks was something like (models vary) a Trace 7500 ST. However, they were just the writing device. They were controlled by a Unix machine with special software, but that's okay since it would be useless for IPF images anyway.

Actually not even a Trace duplication machine can master some of the games released. One game had to master most of the disk on a Trace, then use another/unknown/custom mastering solution to master one of the tracks. Some other disks were mastered on a Trace but shouldn't have been - i.e. you couldn't do a verify phase because the Trace machine lacked the capabilities to do that, and so the disks had to be mastered "raw". Very very bad. I bet their returns department went crazy - probably why they stopped doing it after a few games. No prizes to anyone for guessing who that was.

We should be able to resolve all these issues. Hence why we want to do it properly and get it right first time.

As for percentage of the games release needing mastering hardware. I really wouldn't know, and I don't fancy checking them all to check - sorry!.Basically a huge ton of them. Anything that uses variable bitcells (long tracks, copylock, etc.), anything that uses flakey bits, loaders that uses timing tricks on track geometry, the list goes on.

ISA Central Point deluxe option board: If you can send a detailed tech spec and manual (ie not just marketing stuff), and it should show what is possible. Are they widely available? Probably not much point anyway if they are hard to get hold of... We were also hoping to provide something a little less "legacy", like a device USB/parallel controlled. I don't know about you, but I personally don't have any ISA slots.

[jmmijo]

@Fiath, these cards have not been manufactured for quite some time. Central Point software made many of the early software based backup utilities for just about all the different machines out there. Remember Copy II 64

Well for the PC they had an Option Board that connected to a free ISA slot on an old XT box and with software similar to Ghost would allow the end use to duplicate most of the copy protection schemes out there at the time, including for other systems

I wish I could remember if there were any of these cards available in a warehouse somewhere but I can't. Perhaps Unknown_K would have some answers on this one, he's much more into the older DOS machines then I ever was

[Unknown_K]

Those cards show up on ebay all the time and go for $40-60 on average.

http://retro.icequake.net/dob/#eob

Lots of information and documentation there including software on the boards.

I braught this board up because it would copy pretty much anything because it would just make a complete copy of the magnetic surface of the disk (doesnt try to create special formats at all).

[fiath]

After some consultation with you know who, it seems that:

1) Altered clock rates are not entirely correct for amiga disks
2) Write precompensation is not possible according to the text file there
3) Some more control would be nice
Otherwise, fairly good.

It is important to note that copying and mastering a disk is slightly different things. I would class copying in the "cross your fingers" territory, whereas mastering is more of a defined entity, focusing on quality assurance, not home backups. Home backups being those that may, or may not work, but you can't really be sure (since they may fail later in game).

The reason I mention this is mainly because this is a device that seems to be designed for copying, not mastering. It certainly seems like a very good (perhaps the best?) copier, but either way, because of the points mentioned above, it won't work with everything.

Mastering requires you to know the disk format / protection exactly rather than blindly writing something - leaving it to chance.

Anyway,

I think we would want to do our mastering device first (not least because of the ISA thing), and if we then decide to support other devices, at least we can say that you should use the official hardware if you expect things to work. Of course, that means being able to get hold of one too.

Thanks for bringing it to our attention though. It looks like an interesting piece of kit. I wonder if it was ever available in Europe? I certainly have never heard of it. But that is no big thing.

[Unknown_K]

Originally posted by fiath
After some consultation with you know who, it seems that:

1) Altered clock rates are not entirely correct for amiga disks
2) Write precompensation is not possible according to the text file there
3) Some more control would be nice
Otherwise, fairly good.

It is important to note that copying and mastering a disk is slightly different things. I would class copying in the "cross your fingers" territory, whereas mastering is more of a defined entity, focusing on quality assurance, not home backups. Home backups being those that may, or may not work, but you can't really be sure (since they may fail later in game).

The reason I mention this is mainly because this is a device that seems to be designed for copying, not mastering. It certainly seems like a very good (perhaps the best?) copier, but either way, because of the points mentioned above, it won't work with everything.

Mastering requires you to know the disk format / protection exactly rather than blindly writing something - leaving it to chance.

Anyway,

I think we would want to do our mastering device first (not least because of the ISA thing), and if we then decide to support other devices, at least we can say that you should use the official hardware if you expect things to work. Of course, that means being able to get hold of one too.

Thanks for bringing it to our attention though. It looks like an interesting piece of kit. I wonder if it was ever available in Europe? I certainly have never heard of it. But that is no big thing.

So the board would be pretty good in copying original amiga disks then?

I have a ton of computers with ISA slots, so I might pick one of the boards up if I see one cheap.

[fiath]

I got a good "resource" for these answers.

The device dumps the bits into the host system first, and writes them back later, ie not a dumb analogue copier like the amiga ones == good.

However, for it to work correctly the clock rate must be known for each track being copied, at least in the write phase. This might be a bit of a problem since you must understand what you are copying.

Also precompensation is required for certain data patterns to be readable after writing... especially towards the inner ring of a disk.
However it won't work with varied density tracks - very common on the amiga - quite simply because the clock rate is constant for a track.

It would be possible to make a program that can write some percentage of the IPF files. Such program would require (1) an NT style device driver, (2) more detailed info on the device - presumably getting it with a logic analyser and/or months of reverse engineering experimenting - and... (3) a current PC with at least an ISA slot.

So making it work is possible, but won't be cheap.

Anyone willing to pay the expenses associated with a project like that for a legacy hardware that is limited in stock and is no longer available or requires ancient PC rigs to use...? If the answer is yes, you know where to write.

Sorry we can't be much more help on this one.

We'd be better off concentrating on our device first as at least we wouldn't have to completely reverse engineer that.

[Unknown_K]

I dont care what is used but sooner or later somebody should write a program and list some kind of affordable hardware needed to be able to dump the images back to a disk.

I have large stacks of original amiga games on disk and i know sooner or later they wont be readable, having a way to make a "backup" would be nice.

To most people its a non issue because all these games are probably cracked and available in adf files for quite a while now.

I mentioned the deluxe option board because it was good enough to make US companies quit bothering with fancy disk protection and turn to codewheels and doc lookups. I am surprised you guys never heard of it before.

[Duke]

Originally posted by Unknown_K
I have large stacks of original amiga games on disk and i know sooner or later they wont be readable, having a way to make a "backup" would be nice.

You can already make a backup with the caps dumping tool, you just can't write it back to disk yet.

[whiteb]

I agree, get Fiath to send you the CAPS dumping disk, 68020+ with wads of memory recommended, the software creates an "Image" that only CAPS can read (MOST copy protections read), they will then turn it into an IPF file which Winuae recognises (with a DLL addon). One day they hope to make a solution to recreate the images back to floppy.

www.caps-project.org

[fiath]

Originally posted by Unknown_K
I dont care what is used but sooner or later somebody should write a program and list some kind of affordable hardware needed to be able to dump the images back to a disk.

Agreed. And it will happen. As I said, this is something that was intended from the beginning.

I have large stacks of original amiga games on disk and i know sooner or later they wont be readable, having a way to make a "backup" would be nice.

Well, as Duke says, you can do the backup now. Just just can't "fix" them yet.

To most people its a non issue because all these games are probably cracked and available in adf files for quite a while now.

Not to us I assure you. We also have very large quanities of original games that we don't like the thought of going bad.

I mentioned the deluxe option board because it was good enough to make US companies quit bothering with fancy disk protection and turn to codewheels and doc lookups. I am surprised you guys never heard of it before.

PC wasn't very big over here for a long time (UK, Hungary, probably Europe in general). In fact, you were considered a ripped off, miserable, pitiful PC gamer by friends. I remember when a friend bought a 1500 UKP ($2500) 286 Amstrad PCW with no hard disk and Windows 1.1/1.2 (it wasn't 1.3 anyway, and looked like crap) and I saw Gremlins on it. I was like - why do you even bother buying games??

This was of course until the good VGA games started to appear in 93-95, where there actually started to be some competition in it.

So I guess pretty much nobody was interested in buying PC + games at that time, let alone copying them. It is certainly an interesting peice of history though! I have often wondered why PC games (by the time I looked at them in any great detail) didn't really use disk-based copy protection. I just assumed it was because they all (by that time) were installed to hard disk.

One way to ensure those games are backed up for good though, is to dump them. As we
verify each and every dump, we can tell you which disks are good, which ones are bad and which ones have been modified after the original mastering. Even if there was working mastering hardware right now, the games still should be dumped first obviously, as there would be nothing to write back otherwise.

For you (if you are willing to dump more than the Halley Project), and anybody else reading. Please only dump games that are described on the wanted page by the introduction text + other platforms, i.e. concentrate mostly on undumped - or dumped but bad etc - disks. The game database is very up to date to what we actually have nowadays.

[fiath]

Originally posted by whiteb
I agree, get Fiath to send you the CAPS dumping disk, 68020+ with wads of memory recommended, the software creates an "Image" that only CAPS can read (MOST copy protections read), they will then turn it into an IPF file which Winuae recognises (with a DLL addon). One day they hope to make a solution to recreate the images back to floppy.

Why do you say "most" copy protections?

The dumping software can read ALL protection - in the case of the Amiga anyway. We don't know for other platforms yet, purely because we have not tried yet. This is from a few thousand dumped disks.

I guess you meant the analyser. MOST protections are supported by the analyser software, just not yet Atari ST/PC tracks, some game specific formats, and the 1.5 bit mfm encoding on a few Psygnosis titles.

Some Atari formats and protections had been popular on the Amiga either as protection or data tracks, sometimes both. Since Atari will be supported properly we don't plan to just quickly put in the ones used on Amiga games only.

The release images contain verified and correct data only, plus disk/track/block geometry information. Data is sorted as it would be by any mastering device, hence the images are relatively compact and can be compressed well, unless the game itself is already compressed. That's why they must be analysed, to ensure these games don't crash on say completing level 5.

In fact most of these things help our cause:

1, to keep the IPF files small considering what they contain
2, make a mastering solution possible (just not yet implemented)
3, make comparing the images in IPF form possible (new in next WIP)
4, ensure these disks work the same as leaving duplication as new
5, ensure the data on the ipf files are as good - or in some cases as bad
- as was meant to be by the people mastering the game.

All these things require very complex analysation, hence the separation of
raw dumps and ipf (release) files.

[Unknown_K]

Have you guys started figuring out a device to dump the files back to disk yet? I found a boxed Central Point Deluxe Option board with software and manual for $10 and it should be here in a week so I can mess with it.

Is there a good book/pdf on floppy formats and copy protection? Do you guys have any documentation of the IPF format?

[IFW]

Site->download->

User/Developer Reference Documentation

Book on copy protections: no unless someone broke NDAs. The same/similar copy protection systems/methods/theories are used on copy protected Cds and DVDs with the ones on floppy disks.

Aren't 3.5" floppies are about to disappear for good? Why develop something for a to be dead medium?

[IFW]

Yes, they are.

[Unknown_K]

Originally Posted by Burseg

Aren't 3.5" floppies are about to disappear for good? Why develop something for a to be dead medium?

Because it is useful since the technology is not obsolete for your needs.

Will a person with 10,000 rare records toss them away if his record player breaks?

He would transfer them to IPF format

[fiath]

Hehe - Actually very possible, but I don't think we want to branch into music any time soon - our scope is big enough!

[wlcina]

Everything what is readable is writable ;-)

what about medium (cf card or USBkey or something like that) with interface (some chips processors or whatever (i dont know)) connected directly to amiga fdd connector. You can write IPF image to card , interface read them and send data to floppy connector like normal floppy drive. maybe this is my dream only :-)

[IFW]

...just you may need specialized equipment to write it
As for the other may not be just a dream.

[jmmijo]

Originally Posted by Unknown_K

Because it is useful since the technology is not obsolete for your needs.

Will a person with 10,000 rare records toss them away if his record player breaks?

I have not yet, just need a new cartridge for my Kenwood Turntable

[Unknown_K]

Originally Posted by jmmijo

I have not yet, just need a new cartridge for my Kenwood Turntable

Kenwood? All the true record fanatics use Technics turntables (except the rich suckers who have turntables with granite bases)!

I never invested a dime into records, spent quite a few on tapes (long term waste of money) and also purchased quite a few cds.

[jmmijo]

Originally Posted by Unknown_K

Kenwood? All the true record fanatics use Technics turntables (except the rich suckers who have turntables with granite bases)!

I never invested a dime into records, spent quite a few on tapes (long term waste of money) and also purchased quite a few cds.

I did have a technics, direct driver instead of belt driven but like a fool I needed some cash and sold it

When I purchased an all-in-one system I went with a mid-range Kenwood system

[Unknown_K]

Originally Posted by jmmijo

I did have a technics, direct driver instead of belt driven but like a fool I needed some cash and sold it

When I purchased an all-in-one system I went with a mid-range Kenwood system

Oh how many friends I had that purchased bling bling one week only to sell it at pennies on the dollar the next

[poohbear]

Originally Posted by IFW

Book on copy protections: no unless someone broke NDAs. The same/similar copy protection systems/methods/theories are used on copy protected Cds and DVDs with the ones on floppy disks.

Kris Kaspersky has written "CD Cracking Uncovered: Protection Against Unsanctioned CD Copying", published by A-LIST.

[jotd]

I hope not being off-topic here:

I think a software solution to write back some IPF images back to disk exists right now on a real amiga, by using Psygore MFM2WWarp tool.

This tool supports capsimage.device and is able to "convert" IPF files to WWarp files (a format that Wepl invented for his warper, and which has been greatly enhanced to support some widely used MFM format).

Stupid example: suppose you dump a standard DOS game. MFM2WWarp converts IPF image to WWP file. To write it back to disk, use the WRITE command of WWarp
(maybe a FORCE DECODE command should be done on the WWP file)

Less stupid example: take Leander, Red Zone (a working dump!), Prime Mover, Alien Breed II AGA, Body Blows Galactic IPF files and convert them to WWarp. WWarp can write those known format (Psygnosis MFM 1, 2, ..., Rob Northen MFM, Rainbow Arts Turrican format, and some others) back to disk and the copied disks boot on a A500! Better: copies are copiable with XCopy unlike the original disks (no more '7' longtrack error).

Ok this is not a 1:1 copy but the game MFM loader cannot see the difference.

I heard that RN protection tracks can be written back to disk using MFMWarp (someone told me he copied RN track from Cool Spot), but not with WWarp, so that's still to be done...

JOTD

[dlfrsilver]

hello JOTD !

So what do we have to write as command line under cli to write back?

thanks for your reply, i think it will greatly help some people here.

Denis

[madenph]

Originally Posted by jotd

Ok this is not a 1:1 copy but the game MFM loader cannot see the difference.

What percentage of games do you suppose could be 'remastered' this way?

Also, what benefit do you get from using disks written this way instead of simply writing back ADFs?

Originally Posted by Unknown_K

Kenwood? All the true record fanatics use Technics turntables (except the rich suckers who have turntables with granite bases)!

That's not true.

Whilst Technic's 1210's are the best record decks for DJ's let me repeat that: The best for DJ's.

For the same money you can get a better sounding record player for listening to music at home. Yes, for losts of money you can get even better ones with granite bases.

With Technics you are paying more for the mixing abilities and workhorse night-after-night reliability rather than the actual sound they produce.

[IFW]

Leander is a bad example each disk holds a Protec track that requires extra hardware to master, that track really must be long as it is the protection track, the rest will be ok.
Most Psygnosis stuff has at least one Protec track that the game checks sooner or later and misbehaves if missing. Exception is afair Walker and Lemmings titles, they used Copylock instead of Protec.

btw...
Red Zone: who has a good disk? Do you know exactly which version does not have the mastering error?
Finding a good Carthage took a few years, maybe it would be easier with Red Zone if we knew what we are looking for exactly...

[alexh]

I provided the NTSC version and that had the manufacturing flaw too didnt it?

[fiath]

Yes, I think so.