English Amiga Board


Go Back   English Amiga Board > Main > Retrogaming General Discussion

 
 
Thread Tools
Old 12 January 2009, 20:31   #81
gimbal
cheeky scoundrel
gimbal's Avatar
 
Join Date: Nov 2004
Location: Spijkenisse/Netherlands
Age: 36
Posts: 2,943
I wish I had the reverse engineering skills that you guys possess How do you do it? The best I can come up with is writing a small tool to output the first X bytes of a file as separate unsigned bytes, unsigned shorts and if needed unsigned ints to try and see if some of the values make any sense. Often, they don't
gimbal is offline  
AdSense AdSense  
Old 12 January 2009, 21:45   #82
Exl
Registered User
Exl's Avatar
 
Join Date: Jun 2008
Location: Netherlands
Posts: 118
Quote:
Originally Posted by Qube View Post
I just noticed that the PC version of Chaos is slightly different, has a compressed CHAOS.DAT containing all data, but there is no LEVSDAT.BIN in there. Interestingly there is a file called LEVSDAT.GFX, weird.

The PC's CHAOS.DAT file seems to be a custom archive or something, but there is a structure at the start showing the file names, presumably lengths are in there also or start pos etc. The actual exe seems to use PKWare, so who knows..
Then maybe levsdat.bin is just gfx, for things like the map view? I'll have a closer look at the PC version, CHAOS.DAT sounds interesting at least.

Quote:
Originally Posted by gimbal
I wish I had the reverse engineering skills that you guys possess How do you do it? The best I can come up with is writing a small tool to output the first X bytes of a file as separate unsigned bytes, unsigned shorts and if needed unsigned ints to try and see if some of the values make any sense. Often, they don't
That's how I started. Next step is to determine the length of the entries (a table row so to speak). Then you line those up, and try to make sense out of it. Figure out the datatypes, and you can figure out basic things like coordinates as a start. Then you can visualize what you have so far, look at the things at certain coordinates and think, think, think until your head, it go asplode!

Visualization is important for me personally. Also important is being able to poke and prod into a file. Change values, and see what happens in the game. Document that, and move on from there. Some useful tools are usually a HEX editor (or just viewer), some way to vizualise the things you know as I mentioned (write your own, use LvlRip, or w/e), and an emulator to make things easy to test. A really useful tool I found is http://www.sweetscape.com/010editor/
You can write up a file's structure, and make it easy to read and modify:



And, a cliche; occasionally, a little luck is involved. And all of this is still without skills such as 68k assembly knowledge, more advanced use of a debugger, and such.
Exl is offline  
Old 12 January 2009, 21:47   #83
Qube
Registered User
Qube's Avatar
 
Join Date: Aug 2006
Location: England
Posts: 200
PC's CHAOS.DAT:

I think..

B6 0B 00 00 initial header, this equals 2998 (0x00000BB6), which is the size of the TOC.. (offset 0-2997)

TOC:

[00] - Filename length in characters (00-0C = 00 to 12 characters, typically 09-0C).
[FILENAME.EXT] - (00 to 12 characters this can be).
[00 00 00 00] - 4 bytes, this is the position in CHAOS.DAT of the file's data.
[00 00 00 00] - 4 bytes, this is the length of the file's data.
..repeat..

e.g, last entry in archive TOC is..

0A 4C 41 55 4E 43 48 2E 42 49 4E 65 2C 0F 00 1B 05 00 00 =
[10] [LAUNCH.BIN] [65 2C 0F 00] [1B 05 00 00] =
[10] [LAUNCH.BIN] [994405] [1307]

Now to work out the decrunch lol.
Q;

Last edited by Qube; 13 January 2009 at 00:55. Reason: updated for clarity, plus header
Qube is offline  
Old 13 January 2009, 01:12   #84
Qube
Registered User
Qube's Avatar
 
Join Date: Aug 2006
Location: England
Posts: 200
Exl - It could indeed be the level map preview which is stored at the tail end of the file. Here is a quick raw view of the levsdat.bin from the full game (from chaosspy set).. Dunno what it shows or how it is supposed to be viewed, looks messy I know..

Q;
Attached Thumbnails
Click image for larger version

Name:	levsdat_mystery.gif
Views:	383
Size:	4.0 KB
ID:	19753  
Qube is offline  
Old 13 January 2009, 15:12   #85
WayneK
Registered User
 
Join Date: May 2004
Location: Somewhere secret
Age: 43
Posts: 241
I'll backup Exl's advice to use the "010" hexeditor, it has some great features for reversing work...
WayneK is offline  
Old 13 January 2009, 17:36   #86
Exl
Registered User
Exl's Avatar
 
Join Date: Jun 2008
Location: Netherlands
Posts: 118
Quote:
Originally Posted by Qube View Post
Exl - It could indeed be the level map preview which is stored at the tail end of the file. Here is a quick raw view of the levsdat.bin from the full game (from chaosspy set).. Dunno what it shows or how it is supposed to be viewed, looks messy I know..
Looks like map reviews indeed. But at the bottom there's something that looks like a table...
Exl is offline  
Old 15 January 2009, 19:29   #87
dima
 
Posts: n/a
Quote:
Originally Posted by Qube View Post
PC's CHAOS.DAT:

I think..

B6 0B 00 00 initial header, this equals 2998 (0x00000BB6), which is the size of the TOC.. (offset 0-2997)

TOC:

[00] - Filename length in characters (00-0C = 00 to 12 characters, typically 09-0C).
[FILENAME.EXT] - (00 to 12 characters this can be).
[00 00 00 00] - 4 bytes, this is the position in CHAOS.DAT of the file's data.
[00 00 00 00] - 4 bytes, this is the length of the file's data.
I agree with you. But first DWORD (B6 0B 00 00) it is not a count of sections, it is an offset to data start. I thought the same and extracted these files about two weeks ago. But do not understand what they contain. gfx - it is a graphic files, sfx - sound files: I hope. And ever level has two files *.dat and *.pld (for example N1chaos.dat and N1chaos.pld ) but they do not contain information about width, height of map and another stuffs. What do you think about it?

Last edited by dima; 15 January 2009 at 20:24.
 
Old 15 January 2009, 21:34   #88
Qube
Registered User
Qube's Avatar
 
Join Date: Aug 2006
Location: England
Posts: 200
Quote:
Originally Posted by dima View Post
I agree with you. But first DWORD (B6 0B 00 00) it is not a count of sections, it is an offset to data start. I thought the same and extracted these files about two weeks ago. But do not understand what they contain. gfx - it is a graphic files, sfx - sound files: I hope. And ever level has two files *.dat and *.pld (for example N1chaos.dat and N1chaos.pld ) but they do not contain information about width, height of map and another stuffs. What do you think about it?
Heya, I think they are compressed. I don't see any patterns, looks like random data, so I'm guessing custom packer or something, could be a standard zip/deflate type affair with the header torn off, it's unknown as of yet. May require some debugging in dosbox to work out how the data is unpacked, or perhaps dump the uncompressed mem from dosbox to cheat

Q;
Qube is offline  
Old 15 January 2009, 22:20   #89
jotd
Cat freak
jotd's Avatar
 
Join Date: Dec 2004
Location: FRANCE
Age: 46
Posts: 1,973
If you succeed in unpacking the data, my guess it that you'll get the exact copy of the CD32 files. So it's not really worth it since we have them
Ok, with .BIN extension who's been renamed to .GFX. So what?
jotd is offline  
Old 15 January 2009, 22:46   #90
Exl
Registered User
Exl's Avatar
 
Join Date: Jun 2008
Location: Netherlands
Posts: 118
There may be something interesting inside the .pld files, though. Seeing as each map has 2 files in the PC version. levsdat.gfx may be all gfx now, considering that they likely had to rebuild the data files anyway to convert endianness (or they converted that during load time?). So many mysteries to such an old game.
Exl is offline  
Old 15 January 2009, 23:02   #91
jotd
Cat freak
jotd's Avatar
 
Join Date: Dec 2004
Location: FRANCE
Age: 46
Posts: 1,973
all gfx files had the .BIN suffix in the cd32 version, so has levsdat.BIN.

If someones extracts the files from the custom archive, we could try to figure out which packer was used. Maybe it's PKware DCL without the 06 01 header? Gods & Cadaver PC use this packer so why not CE?

please put the files somewhere so I can have a look

BTW I had a first peek at cadaver, and I could figure out some stuff with the floor tiles but not much more
I'll be able to resource game behaviour, which is 68k in the amiga version and possibly Intel in PC version (ROUTINEx.TAD file, unpacked AFAIK, 4K size)
jotd is offline  
Old 15 January 2009, 23:26   #92
Qube
Registered User
Qube's Avatar
 
Join Date: Aug 2006
Location: England
Posts: 200
I just had a poke with DosBox, memdump etc. The first level tilemap was clearly visible in memory, so I guess I was looking at A1CHAOS.PLD. On inspection, looks like endian swap which was expected. Plus there is also some extra data and what appears to be slight differences in the data, so maybe tile index changes. Attached is the PLD from memory. I'm assuming PLD=PC Level Data?

Was there a PC demo for Chaos? I'm gonna go looking.. I can extract some files from CHAOS.DAT if needed, but they won't be decompressed, that still needs to be figured out..

Q;
Attached Files
File Type: zip cepld1.zip (4.6 KB, 115 views)
Qube is offline  
Old 15 January 2009, 23:44   #93
Exl
Registered User
Exl's Avatar
 
Join Date: Jun 2008
Location: Netherlands
Posts: 118
Yeah looks exactly like the level data from the .CAS CD32 files, endian swapped. There's a few bytes too many dumped at the start (skip until the first 0x04L). But, about 300 bytes bigger too, so I'm going to try and read it, see whats different. I'm still wondering whats inside the *1chaos.dat file in the PC version then.

Edit: Looks the same, except that tile pickups are placed on the map itself too, and there's a additional tile flags placed at the origin of some covers, animblocks and pickups. There's also an animblock at coordinate 0,0. Might be an off by 1 error though.

Last edited by Exl; 16 January 2009 at 00:11.
Exl is offline  
Old 16 January 2009, 19:54   #94
dima
 
Posts: n/a
Hi, I have writen program to extract chaos.dat file. (I hope it is work ) Put it near that file and run. *.pld - files are too small for contains map data, I think *.dat files have an information about map.

P.S. Exl I have downloaded "010 editor" it is very simple and useful tool. Thank you
Attached Files
File Type: zip chaos_extract_dat.zip (33.3 KB, 146 views)
 
Old 16 January 2009, 20:07   #95
Exl
Registered User
Exl's Avatar
 
Join Date: Jun 2008
Location: Netherlands
Posts: 118
The files are obviously compressed, and will need to be decompressed for them to be read. The problem is, what compression is used? LZW? Or something custom? The compressed files have no header so it's more difficult to figure out. It extracts all the files nicely though.
Exl is offline  
Old 16 January 2009, 21:06   #96
jotd
Cat freak
jotd's Avatar
 
Join Date: Dec 2004
Location: FRANCE
Age: 46
Posts: 1,973
I had a quick look. It's not ZIP, not GZIP, not PKWare DCL.

First longword (little endian) seems to match the unpacked size (corresponding O1CHAOS.CAS file of the CD32 version has roughly the same size as the first longword found in O1CHAOS.DAT (maybe you figured out this already).

There are not that much packers/unpackers that work on streams/single files (as opposed to archivers).
maybe it's RNC but header is little endian...

Tried Powerpacker, IMP/ATN, no luck... We lack PC culture. Good for us

I figured out some stuff of LEVSDAT.BIN. It seems to confirm that this is a general data file for all game

So far, I have sorted the values just before ENDS marker, and decoded the chunks at the given offsets, size = next offset - current offset (hence the sort without which it would not work)

The offsets make sense. Most of the time I notice a change in the data. I examined the chunk 0x2168 (60 bytes), and it is values starting at 7000, 14000, 21000 ... and ending at 98000. AFAIR you got extra life at 7000 right? then 14000 ... see what I mean?

levsdat_dir.zip

Edit: I suggest that some mod renames this thread to something like "hacking through Bitmap Bros games". I've got some info on Cadaver level format and some level writer too...

Last edited by jotd; 21 January 2009 at 14:28. Reason: some info on LEVSDAT.BIN
jotd is offline  
Old 23 January 2009, 20:12   #97
Exl
Registered User
Exl's Avatar
 
Join Date: Jun 2008
Location: Netherlands
Posts: 118
Has anyone made any more progress on deciphering the level format? I'm stuck, and digging in the levsdat.bin file isn't giving me any more info.
Exl is offline  
Old 23 January 2009, 20:19   #98
jotd
Cat freak
jotd's Avatar
 
Join Date: Dec 2004
Location: FRANCE
Age: 46
Posts: 1,973
nope, but I made good progress on Cadaver
jotd is offline  
Old 16 February 2009, 01:22   #99
Qube
Registered User
Qube's Avatar
 
Join Date: Aug 2006
Location: England
Posts: 200
Hey jotd, I spotted some more info on the development of Gods over on this site if it's of interest..

http://www.guardiansofthepast.co.uk/...?Version_ID=10

It shows some interesting bits n bobs.

Q;
Qube is offline  
Old 16 February 2009, 21:37   #100
jotd
Cat freak
jotd's Avatar
 
Join Date: Dec 2004
Location: FRANCE
Age: 46
Posts: 1,973
@Qube: really f***king interesting!! Too bad I did not see that earlier. I thought that the Bros were very secretive about their code/dev methods. If only we could get hold of those tools!!!
CRIPAL must be the name of the system. I saw that in a PC data file.

Funny thing is if you use my editor you'll find that I've done a very similar system, with associations, etc... My editor is graphically more powerful but some complex stuff is script (Java) based to avoid developping a language that was too complex.
When you check the screens, you see indexes and x,y coords, so you have to input them manually. It was 1989 at the time, so it was great, but they must have had a hard time inputting all the game data!
I think that their system still generated asm code and the game did not embark the engine itself (I saw self modifying code such as NOP becoming RTS when spikes are killed).
Also, I used a smart technique to define what is a platform. I avoided the extra mask layer by tagging each platform tile as "platform", but allowing exceptions. It saved the tedious task of double editing the gamemaps.

The intelligent alien editor part was really very interesting for me. Seems that I missed some stuff such as "avoid shots" for walking monsters (I coded it only for flying monsters).
jotd is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Turrican 2 level viewer Exl Amiga scene 28 22 December 2013 06:32
Chaos Engine level codes Kenan support.Games 3 20 June 2013 18:49
The Chaos Engine Djay request.Modules 7 25 February 2010 00:14
The Chaos Engine, level 3/2: Fortesque Mansion/Confusion fmcpma support.Games 6 12 September 2005 01:30
Chaos Engine vs Chaos Engine 2 NytroX86 Nostalgia & memories 20 31 August 2005 12:25

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 06:47.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Page generated in 0.44913 seconds with 12 queries