Cadaver remake

[gimbal]

Really excellent GUI design there on that tool. I'm loving the tree view of all the elements in a room, I'm going to steal that idea!

[Kroah]

Originally Posted by BoyC

The pc version could be a solution to these problems. During my old research I found out that there were at least two versions of the game released, one with compressed and one with uncompressed data files. The game still supports the uncompressed level format, thats one way of how I messed around with it by editing into the files. It would be a nice goal to try and have the demo levels on the pc version for example.

I already had a look to the PC version, but x86 assembly is soooo ugly to my eyes. I think copy/pasting files is not enough because i saw some differences between both systems, not sure.

I thought to port Atari demo levels to PC, but the PC version contains x86 code in the levels, so we should translate 68k to x86, which is not my cup of tea (unless all levels share the same x86 code, in this case we could use one from the initial levels).
Jotd uses the PC version files, maybe he already checked that.

[BoyC]

Granted I haven't explored the file formats enough to find actual executable code in them and that might complicate things quite a bit, but it's worth a shot. Great job so far, I'd be glad to look at what you guys found out so far, maybe I can even pitch in a bit

[Retro1234]

You could probably have a WHDload install that uses the uncompressed files?

[Kroah]

Originally Posted by Boo Boo

You could probably have a WHDload install that uses the uncompressed files?

Even if i really enjoy using real floppy disks to play my games, this a nice idea.

Concerning loading the demo levels in the viewer, i have a last idea to test before giving up.

[jotd]

the PC data files could be recompressed back but you need a PKWare Data Compression Library software which costs 500 bucks!!!
The unpacker is a reverse engineered C code (dynamite). the compressor is not publicly available.
The idea (at least on PC) maybe would be to "repack" PC file in "store" mode (no compression).
I had tried the uncompressed files on my PC version and it failed.

Good work Kroah you rule!!! But I admit it's a difficult project. My remake is currently on hold too because of all the technical problems (isometric shit, lack of time ...)

bye

[Kroah]

I have good news concerning the Cadaver demos.

Some times ago, i added the possibility to load the payoff levels into my viewer which worked because it uses the same layout as the vanilla Cadaver.
Then i tried the demo levels, but they were stored differently. I looked deeper into the Gate House disassembly and i realized they used an earlier version of the Cadaver engine (confirmed by the Cadaver Secret Shrine http://www.gods-country.de/cadaver/cadframes_eng.htm). That's why i did not go any further to try loading them in the viewer.

Lastly, i came across the release date of the 2 other demo levels (the Temple and the Last Supper) which were around the same time as the release of Cadaver.
So i told myself they might use the last version of the game engine, not a preview like the Gate House. If that was the case, i could load the levels into the viewer without disassembling all the engine. I only needed the data files... but i didn't want to disassemble the unpacker (each version uses a different packer...).

Thanks to Steem Engine, it makes it easy to save memory snapshot during loading time.

Because the game reorganize the loaded data, i couldn't use an in-game memory snapshot:
packed data files on disk -> unpacked data in memory -> reorganization of the data on the fly (which delete the initial unpacked data and is not compatible with the viewer).

So i put some breakpoints, saved memory snapshots during the loading and extracted the data to standard Cadaver data files. Hopefully, the 2 demos uses almost the same data layout as Cadaver, making it compatible to my viewer after only few updates.

Now, I had enough motivation to investigate the case of the first demo/preview (Gate House). I tried the same technique but i only got 2 files instead of 5. Nevertheless, i saw that the second files contained data from the 4 files but mixed up.
With this in mind, i began to disassemble the loading part of the Gate House to understand how they were loaded. As expected, only 2 files were unpacked, the second one being the data from the 4 remaining files mixed up.
Unfortunately some parts of the expected data by Cadaver (not the preview) where stored differently or simply missing. So even if i splitted up the big file to get the 4 missing files, it couldn't be loaded in the viewer as is.
After some disassembling, i found how the data differed between the preview and the game. On top of that, the missing parts wasn't essential.

Now, the first demo can be loaded in the viewer (although i think some scripts could be incompatible, i haven't checked if the script engine evolved between the preview and the game).

To sum up, we have:
- Layout A: Gate House (demo #1) [2 data files] (v0.9 of the engine) (not packed)
- Layout B: Cadaver [data disk format, 5 data files for each level] (packed)
- Layout B: The Payoff (extension) [data disk format, 5 data files for each level] (packed)
- Layout C: The Temple (demo #2) [5 data files] (RNC)
- Layout C: The Last Supper (demo #3) [5 data files] (LZH)

The viewer can read natively .st data disk (layout B) but files have to be extracted for Layout A and C.

I hope this story pleased you. I'll post the viewer with the source soon after some testing.

[Kroah]

Originally Posted by jotd

the PC data files could be recompressed back but you need a PKWare Data Compression Library software which costs 500 bucks!!!
The unpacker is a reverse engineered C code (dynamite). the compressor is not publicly available.
The idea (at least on PC) maybe would be to "repack" PC file in "store" mode (no compression).
I had tried the uncompressed files on my PC version and it failed.

I tried uncompressed files without any success too.
I can't even get my hands on a version of the PKWARE SDK (which can implode/explode DCL).

However, there's a source code available there to the DCL implode: http://shadowflare.samods.org/download/pkcomp.zip.
I don't know if it gives the same DCL file though.

[BoyC]

Wow, nice work on this Kroah! Looking forward to checking out the editor and the source.
In the meanwhile I dug up the uncompressed pc version I worked with if you guys want to have a look at it.

[jotd]

Good work and good find Kroah. I'll have a check on the implode routine to see if they can be deploded by my remake

[Kroah]

Does someone know which packer is used for MJM0 compressed files?
My friend Google does not give me any clues...

It's used by the "ROUTINEn.TAD" files of Cadaver for PC.

Maybe by opening one someone will remember a known pattern, who knows.

I don't mind to disassemble the x86 unpacker of Cadaver...

[BoyC]

MJM0 is the marker for the uncompressed files.
The version I have has all the files in MJM0 that's how I worked with it. Actually now that I recall the original cadaver had the datafiles in that format, the version with the payoff was in MJM1 already.

[jotd]

@BoyC: it works!! I just made the following test: unpack MJM1 files (using dynamite), and add the "mjm0" header at start: game starts, and it is even faster to load (using DOSBox). The files are rather small too so no big deal.

Your version executable is the same as ours, data files are different. Maybe they lost the PKWare license at some point

@Kroah: Means that using the MJM0 header trick it is possible to edit IBM PC files and save them back for new levels.

I reckon that .TAD has a rather generic: handles poison, cure potion, to sum it up it seems to handle objects rather than rooms.

Which means that if you use the objects like in the original level it will work. If you don't use the objects it will work even better

I could've told you that MJM0 was uncompressed. Basically any x86 disassembler will disassemble the .TAD file. You can compare with the 68K version I started to resource some time ago.

[BoyC]

I wonder if we could create pc-playable versions of the demo levels this way?

[jotd]

I think so. Kroah, can you help?

[Kroah]

Originally Posted by BoyC

I wonder if we could create pc-playable versions of the demo levels this way?

I think so too.

Some remarks:
- ST MP1 files have 868 extra bytes used for the copy protection not needed on PC (not present in the ST demos though).
- Cadaver first demo (Gate House) stores data differently and some data are missing. To port it on PC, data must be reordered and completed with existing vanilla levels.
- graphics are stored identically (no need to deinterlace bitplanes).
- no endianness issue in the data files.
- PC files don't contain the length at offset 0 (so should be removed).
- ROUTINEn.TAD files contain executable code (68k for ST, x86 for PC). This is a vector of hard coded functions used in-game.

To reconstruct the ROUTINEx.TAD file for PC:
- Using vanilla Cadaver & extension (not the demos), we could identify each function in ROUTINEx.TAD files. Each function already maps its 'other system' counterpart because the function index to be called is read from the data files. And those data files are the same between ST/PC -> the meaning of the function called is the same -> we have the corresponding x86 code of these 68k functions.
- By comparing vanilla+extension with demos on ST, we could quantify the amount of shared functions. With the hope that 80% of the functions are the same, we would ~only~ have to code the remaining functions on PC. Maybe x86 code should be rebased.

TL;DR:
I see only 1 (big) problem to port the second (Temple) and third (Last Supper) demos. For the first demo (Gate House) more work has to be done to convert the data.

Your thought?

[BoyC]

Sounds worth a look - I'd really like to look into this, but I don't have enough info on the structure of the datafiles (mainly the routine file's indexing) to do so. If we knew the entry points and sizes of the functions in the datafile we could byte-compare the routines in the demos to the routines in the game levels to find matches. Then we could use the same routines from the pc game levels to reconstruct the files.
I looked into the routine file yesterday with a disassembler but without meaningful entry points it's quite hard to know where to start

[Kroah]

Originally Posted by BoyC

Sounds worth a look

Agree, but i rather play on Amiga. So i do it only for fun and those who don't play on ST/Amiga.

Originally Posted by BoyC

I'd really like to look into this, but I don't have enough info on the structure of the datafiles (mainly the routine file's indexing) to do so.

I'm currently writing the structure of the TAD files. I will let you know.

Originally Posted by BoyC

If we knew the entry points and sizes of the functions in the datafile we could byte-compare the routines in the demos to the routines in the game levels to find matches. Then we could use the same routines from the pc game levels to reconstruct the files.

Yes, that's what I'm explaining in my previous post.
- First, find matches between (vanilla & extension) <=> (demos #2 & #3) to get their x86 equivalent on PC.
- Then, code the missing ones directly on x86.

Originally Posted by BoyC

I looked into the routine file yesterday with a disassembler but without meaningful entry points it's quite hard to know where to start

I understand. The file contains several offset tables. Without knowing them, you can't easily bound the code.

I'm working on it this WE.

[Kroah]

Here's the header of the TAD files:

struct TAD_Header { // size: 0x50
  word offsetToFunction_LevelInit;     // +00
  word offsetToVectors_Drink;          // +02 used in IconAction_Drink
  word offsetToVectors_Spell;          // +04 used in CastSpell & ObjectsHeartbeat
  word offsetToVectors_EffectEnd;      // +06 used in HandleTimerAndSpecialZones
  word offsetToUnksWord_LevelEnd;      // +08 used in LevelEnd (items to keep at the end of level)
  word nbRoomsDevelopper;              // +0A nb rooms not counted in complete %
  word floorGfxIndexForDoorOnFloor;    // +0C
  word floorGfxIndexForDoorOnHeight;   // +0E
  word fieldsUnk [12];                 // +10
  word vectors_CreatureThink [10] [2]; // +28 [4][]=WaterDragon, [5][] Worm, [6][] StarTentacle
}

Remarks:
- If the TAD file starts with a LONG containing the file length, remove it (ST/Amiga files).

- All offsets in the header are from the start of the TAD file.

- offsetToFunction_LevelInit:
It points to 2 bytes, then the function.

- offsetToVectors_Drink:
Used when the hero drinks something.
It points to an array of function offsets.
The offset base is offsetToVectors_Drink.
The item has a index for this array.

- offsetToVectors_Spell:
Used when the hero or a creature casts a spell.
It points to an array of function offsets.
The offset base is offsetToVectors_Spell.
The item has a index for this array.

- offsetToVectors_EffectEnd:
Used when an effect ends (an effect has a duration like alcohol, poison).
It points to an array of function offsets.
The offset base is offsetToVectors_EffectEnd.
The item has a index for this array.

There are 52 effects in the game. For example, #32 is Alcohol effect, #39 is invulnerability.
I think that if a level uses an effect, it's included in its TAD file.
We may create a full version of the TAD file containing all effects in the offsetToVectors_EffectEnd by using multiple levels.

- vectors_CreatureThink:
This is an array of function offsets.
The offset base is vectors_CreatureThink.
Each function offset points to 2 bytes, then the function.
The creature has an index for this array.

After fixing some bugs, I'll release the viewer. I won't have time to look deeper to port the TAD files across system and it's not on my priority list (I'm not a fan of the PC version of Cadaver).

Hope it helps,

[BoyC]

Nice work Kroah, I'll have a look at the files when I have the time.
Looking forward to the viewer!