Dogs of War remake

[rofl0r]

hi friends, a couple years ago i started a remake of Dogs of War, which is 95% complete. all gameplay, maps and sprites are there, and the original soundtrack is being played via a david whittaker tracker engine specifically made for this purpose (no lame mp3 recordings used here ).

what kept me from finishing it is the huge amount of work required to manually recreate the "spawnmaps" and routes taken by enemy soldiers.

since nobody ever showed up offering help in that task i finally came to the conclusion that the only way forward is to actually use the original gamedata for this case, which basically means attaching a debugger to the game on the amiga and figure out from where it takes the data and how to decode it.

since i'm an utter noob when it comes to m68k disassembly, amiga debugging tools and even executable format i was hoping someone here in this forum might be able to help out so i don't have to dig into this myself, which will probably take quite some time, so this project can finally be completed.


the source is available here: https://github.com/rofl0r/OpenDOW

[jotd]

you don't always have to do that: you can figure the data out by looking at the leveldata itself (possibly changing it to see what happens). I did that for Magic Pockets and Cadaver and that worked great (except for some details where the disassembly can be required).

Then write a python script to decode level data and convert it to whatever format you want.

since nobody ever showed up offering help in that task

yes, that's your project. What is the machine you're targetting and/or the improvements you're planning?

[rofl0r]

What is the machine you're targetting and/or the improvements you're planning?

the current SDL1 version has some improvements like a crosshair that's moved by the mouse while walking quake-style with wasd, so one can can shoot to everywhere on the map (given the weapon has sufficient range).

the plan is to port it to SDL2 at some point and maybe to some retro consoles like NEC PC engine or NeoGeo. it should also work on the PSP with minor changes since an SDL2 port is available. adding an amiga port is also within reach due to existence of modern gcc toolchains (i will leave this task for someone experienced with amiga development though).

[jotd]

adding an amiga port is also within reach due to existence of modern gcc toolchains

no, not with a SDL base, at least not with a decent framerate.

[rofl0r]

Originally Posted by jotd

no, not with a SDL base, at least not with a decent framerate.

right. there would need to be some work done like running a script over the sprite resources to turn them back into bitplane format and changing the scroll and draw code to something more suitable for the amiga.

[jotd]

yeah, rewrite it completely

[LeCaravage]

I wonder and not sure at all, but I'd bet the map datas and ennemies waves could be compressed and decompressed on the fly.

[NicoDE]

Originally Posted by rofl0r

[...] i finally came to the conclusion that the only way forward is to actually use the original gamedata for this case, which basically means attaching a debugger to the game on the amiga and figure out from where it takes the data and how to decode it.

Many research projects play with the game files to see what data changes result in what in-game changes. That's perfectly fine and most of the time results in good progress at the start.

Attaching a debugger is another option to answer specific questions, but requires a strong background in assembly. There is a danger that many information/details are lost if the findings are not written down somewhere (e.g. in a wiki/doc/code of the remake).

Another option would static code analysis with a disassembler. With Ghidra there is an open source project, that can be extended with custom plug-ins, like the ghidra_amiga_ldr, that allows you to import and analyze executable Amiga hunk files. Well, you can also 'analyze' any binary raw file, e.g. data files, by adding structures, constants, and comments. Furthermore, Ghidra can be run as a service on a server, which gives multiple users the possibility to work on a project in parallel. I already setup a Ghidra server for collaborative analysis of Ambermoon. Therefore, if you want to go that route - even if the static analysis most of the time takes longer than the other options, and some open issues still to solve with AmigaOS m68k analysis in Ghidra - I could create a project on my server for you. I'll be away during the Easter holidays, but I can certainly create a new project and new user accounts via my mobile phone. Links:
https://ghidra-sre.org/
https://github.com/lab313ru/ghidra_amiga_ldr
https://ghidra.nicode.net/
https://gitlab.com/ambermoon/research/-/wikis/ghidra

ps: I created an 'openDOW' project on the server and committed an initial analysis of the loader - but I do not have the original disk to load the code/data tracks (go to 0x000080fa in the loader)...

[rofl0r]

Originally Posted by NicoDE

Many research projects play with the game files to see what data changes result in what in-game changes. That's perfectly fine and most of the time results in good progress at the start.

that's a good idea, but i extracted the files on DoW with amitools' xdftool
and it appears the data is saved on raw disk tracks instead.
this makes the approach a whole lot more painful.

Originally Posted by NicoDE

Attaching a debugger is another option to answer specific questions, but requires a strong background in assembly. There is a danger that many information/details are lost if the findings are not written down somewhere (e.g. in a wiki/doc/code of the remake).

looks like the best option to me; for example you can look at what's mapped to memory at the time of playing a map and then lookup the data on disk. this could also make method1 a lot quicker so you dont have to change random bytes.

otoh i suspect the main in-map loop isn't that much code and it'll probably look at a limited set of mapped tables each time you move the character, so someone well versed in e.g. winuae debugger and m68k will probably find the tables and corresponding code in no time.

Originally Posted by NicoDE

Another option would static code analysis with a disassembler. With Ghidra there is an open source project, that can be extended with custom plug-ins, like the ghidra_amiga_ldr, that allows you to import and analyze executable Amiga hunk files.[...]
ps: I created an 'openDOW' project on the server and committed an initial analysis of the loader - but I do not have the original disk to load the code/data tracks (go to 0x000080fa in the loader)...

that's really nice of you, but i think i'll have to pass on that one as i use a custom linux distro that doesn't have java. (in order to get java bootstrapped from source, you need to start with a GCC that has java support (removed in 7.x), then build 20 year old unmaintained gnu classpath and jamvm, and then go through a dozen of OpenJDK's until you finally end up with a contemporary version).

[jotd]

diskimages often contain crunched files. wrip from whdload may be used to find them in the image. Then you unpack them and try to figure out what they are

one thing we agree on: this whole java stuff sucks

[NicoDE]

I spent some hours to analyze and document the loader. Something that does not have to be done a second time, because it is still on the server and could be read/corrected/extended by someone... Seems like the tools I'm using are not cool enough for you.

[rofl0r]

Originally Posted by NicoDE

I spent some hours to analyze and document the loader. Something that does not have to be done a second time, because it is still on the server and could be read/corrected/extended by someone...

oh, that's great. is there any way i can look at that without requiring a ghidra/java install ?

Originally Posted by NicoDE

Seems like the tools I'm using are not cool enough for you.

it's not about cool or not, it's about what i can run on my custom build-from-source distro. to use ghidra i'd have to install a VM with java toolchain in it to build/install ghidra and the custom plugins, which likely requires at least 5 GB of hdd space, which i currently don't have available.

Originally Posted by jotd

diskimages often contain crunched files. wrip from whdload may be used to find them in the image. Then you unpack them and try to figure out what they are

after sending my post i was actually thinking that whdload must have a solution for this kind of thing. let me see whether i can find my whdload license somewhere in my backups...

[jotd]

you don't need a licence. The tools are free, whdload is free now too.

[NicoDE]

Originally Posted by rofl0r

is there any way i can look at that without requiring a ghidra/java install ?

Someone could export the structures and pseudocode to C/text/asm.
I'm on the road and cannot do this right now - maybe you could ask a friend who has Java 11+ on his/her machine or you could boot one of the many Linux live systems from a stick.

Originally Posted by rofl0r

after sending my post i was actually thinking that whdload must have a solution for this kind of thing.

Even better, the latest DOW slave has been released with source code.

[NicoDE]

Originally Posted by rofl0r

[...], but i extracted the files on DoW with amitools' xdftool
and it appears the data is saved on raw disk tracks instead.

Any raw data available or any file format documentation?
I don't want to repeat your analysis to make any progress...

In the meantime I got my hands on an original disk image. Seems the original has a minimal number of Amiga MFM tracks with OFS to boot the loader, and the rest of the tracks are in IBM MFM format with 10 sectors. The cracked versions obviously converted them to AMFM and hacked the loader. So I analyzed the hacked loader Hidden in an unused AMFM track there are leftovers (first 16 OFS sectors) of "loader3.s" - it does not perfectly match the loader code, but you get the idea (attached). If I find some spare time, I'll analyze the original loader (minor differences, just the project setup needs some time).

[rofl0r]

Even better, the latest DOW slave has been released with source code.

you mean whdload source code or DoW source code ? this old thread actually mentions https://eab.abime.net/showthread.php?t=47739 that the original coder found his source code and someone asked him to publish it...

Any raw data available or any file format documentation?
I don't want to repeat your analysis to make any progress...

all my reverse engineering so far was done using tools: gfxrip to extract the sprites from winuae's memory, the soundtrack was already ripped by someone else, and the sound fx were exported via winuae. i didn't look at the original data on disk before opening this thread, so there's no existing analysis.

So I analyzed the hacked loader Hidden in an unused AMFM track there are leftovers (first 16 OFS sectors) of "loader3.s" - it does not perfectly match the loader code, but you get the idea (attached). If I find some spare time, I'll analyze the original loader (minor differences, just the project setup needs some time).

that's great news, downloading now....

[NicoDE]

Originally Posted by rofl0r

[...]i didn't look at the original data on disk before opening this thread, so there's no existing analysis.

I see

As previously mentioned, the disk includes a mix of standard Amiga MFM 11-sector tracks (standard boot block and minimal OFS file system to run the game loader) and standard IBM MFM 10-sector tracks with the game content (the hacked releases obviously converted them to Amiga MFM tracks and modified the track loader code).

I dumped all the track data from the SPS release (attached). The track loader code does not really care about sector headers, numbers, or checksums - it just reads 1A00 aligned words on sync-word 4489, decodes the MFM data in IBM style; (ten times) scans for A1FE, skips some bytes, scans for A1FB, and copies the sector data. The basic disk layout is (the track loader code thinks in single-sided track sequences):

h0c00-00: (Amiga MFM) boot block
h0c01-38: (IBM MFM) game data
h0c39-41: (Amiga MFM) OFS, startup and loader
h0c42-79: (IBM MFM) game data
h1c00-00: (unknown, unformatted?)
h1c01-38: (IBM MFM) game data
h1c39-41: (Amiga MFM) OFS, startup and loader
h1c42-79: (IBM MFM) game data

The game loader takes over the system, relocates itself, and loads the game main code. The game main code again takes over the system, and relocates itself.

[00008000,00008A9C): loader (relocated)
	OFS "d.o.w."	# copy [0024,0FC4) to 00008000, jmp 00008062

[00020000,0002F000): game main code
[00002C00,00011660): game main code (relocated)
	MFM h0c01-12	# jmp 0002001C, copy [001C,EA7C) to 00002C00, jmp 00002C24

Since the game main code takes over the system, it does not need any memory management and all blocks are loaded at fixed addresses. Here is a list of the block references that I found in the game main code:

[00029880,0002AC80):
	MFM	h0c79-79	# highscore
[00036E82,00049A82):
	MFM	h1c01-15
[0004B91E,00051D1E):
	MFM	h1c62-66	# cipher: [001C,5E44).b xor 80
[00051762,00061B62):
	MFM	h0c42-54
[0006154C,0007694C):
	MFM	h0c13-29
	MFM	h0c66-78	# [0006154C,0007194C)
[0006AAF8,0006D2F8):
	MFM	h1c59-59,	# [0006AAF8,0006BEF8)
		h1c60-60,	# [0006AAF8,0006BEF8)
		h1c61-61	# [0006AAF8,0006BEF8)
	MFM	h1c33-34,
		h1c35-36,
		h1c42-42,	# [0006AAF8,0006BEF8)
		h1c37-38,
		h1c43-44,
		h1c45-46,
		h1c47-48,
		h1c49-50,
		h1c51-52,
		h1c53-54,
		h1c55-56,
		h1c57-58
[0006AB5C,0007235C):
	MFM	h1c22-27,
		h1c16-21,
		h1c28-32	# [0006AB5C,00070F5C)

h1c62-66 is somewhat special, as the data is 'encrypted' (with a simple XOR cipher), and the range overlaps with other data (load order is important).

Or to see it from the other side, as list of disk blocks:

h0c00-00:	# Amiga boot block
h0c01-12:	[00020000,0002F000), [00002C00,00011660)
h0c13-29:	[0006154C,0007694C)
h0c30-38:	# ?
h0c39-41:	# Amiga file system
h0c42-54:	[00051762,00061B62)
h0c55-65:	# ?
h0c66-78:	[0006154C,0007194C)
h0c79-79:	[00029880,0002AC80)
h1c00-00:	# unknown, unformatted?
h1c01-15:	[00036E82,00049A82)
h1c16-21:	[0006AB5C,0007235C)
h1c22-27:	[0006AB5C,0007235C)
h1c28-32:	[0006AB5C,00070F5C)
h1c33-34:	[0006AAF8,0006D2F8)
h1c35-36:	[0006AAF8,0006D2F8)
h1c37-38:	[0006AAF8,0006D2F8)
h1c39-41:	# Amiga file system
h1c42-42:	[0006AAF8,0006BEF8)
h1c43-44:	[0006AAF8,0006D2F8)
h1c45-46:	[0006AAF8,0006D2F8)
h1c47-48:	[0006AAF8,0006D2F8)
h1c49-50:	[0006AAF8,0006D2F8)
h1c51-52:	[0006AAF8,0006D2F8)
h1c53-54:	[0006AAF8,0006D2F8)
h1c55-56:	[0006AAF8,0006D2F8)
h1c57-58:	[0006AAF8,0006D2F8)
h1c59-59:	[0006AAF8,0006BEF8)
h1c60-60:	[0006AAF8,0006BEF8)
h1c61-61:	[0006AAF8,0006BEF8)
h1c62-66:	[0004B91E,00051D1E)	# cipher: [0004B93A,00051762).b xor 80
h1c67-79:	# ?

The ? means I found no reference yet (might be in the other code blocks)...

The Ghidra project has been updated with a basic analysis of the game main code block.

[rofl0r]

thanks for looking into this. do you think you might be able to identify the main game loop while playing a mission (e.g. by looking at opcodes in winuae debugger and searching for those bytes in the game "executable"), so we can focus on the subroutines used by it and the data it accesses ?

otherwise this looks as if it would result in a full decompilation project, which is certainly a very nice thing to have by itself, educative etc, though of much greater scale than what's needed for the quite limited set of data i'm after.

[NicoDE]

Originally Posted by rofl0r

do you think you might be able to identify the main game loop [...]

The main loop is not the problem, it is just a lot of code to look at and the 'problem' is to find the functions of interest... Since my spare time is very limited, this will take a while if nobody takes over the analysis

There is a list of 12 track blocks that are loaded at 0006AAF8 (h1c33-34, h1c35-36, h1c42-42, h1c37-38, h1c43-44, h1c45-46, h1c47-48, h1c49-50, h1c51-52, h1c53-54, h1c55-56, h1c57-58). Since there are 12 missions, this is very likely the mission data you are searching for... analysis pending.

[NicoDE]

Originally Posted by rofl0r

(e.g. by looking at opcodes in winuae debugger and searching for those bytes in the game "executable")

There is no need to search for any code patterns, everything is loaded at fixed addresses.

Data loading for the selected mission happens in the function at 00010ABE.
If you want to know which track block are used for what mission:

/* misson map info (map pos, reward, first line) */
[ 1 - 1]	178,72	13000	"SEVERAL MISSILES HAVE BEEN TAKEN"
[ 2 - 1]	159,75	10000	"MY YOUNG SON HAS BEEN KIDNAPPED"
[ 3 - 1]	163,82	13000	"A TERRORIST GROUP HAS THREATENED"
[ 4 - 1]	114,83	7000	"OUR ORGANISATION HAS DISCOVERED"
[ 5 - 1]	169,89	6000	"THE SENIOR MEMBER OF OUR FAMILY"
[ 6 - 1]	89,99	12000	"PLEASE HELP! MY GIRLFRIEND WHO"
[ 7 - 1]	101,100	15000	"A DRUG BARON HAS RECENTLY CAUSED"
[ 8 - 1]	205,101	6000	"A GROUP OF TERRORISTS THREATENS"
[ 9 - 1]	243,110	14000	"CAN YOU HELP ME? MY SON IS A"
[10 - 1]	104,135	5000	"AN INVALUABLE INCA STATUE HAS"
[11 - 1]	246,147	5000	"CRIMINALS HAVE STOLEN A CRATE OF"
[12 - 1]	175,154	9000	"OUR POLITICAL LEADER HAS BEEN"

/* 00010B2A */
struct {
	uint16_t read_1_index,  /* 0-based */
	uint16_t read_2_index,  /* 0-based */
	uint16_t read_3_number, /* 1-based */
	uint16_t color
} const dow_mis_read_info = {
	[ 0] = {1, 2,  5, 0x025},
	[ 1] = {1, 1,  6, 0x233},
	[ 2] = {2, 1,  9, 0x421},
	[ 3] = {2, 0, 12, 0x430},
	[ 4] = {2, 0, 11, 0x034},
	[ 5] = {0, 1,  1, 0x333},
	[ 6] = {1, 1,  7, 0x133},
	[ 7] = {0, 2,  3, 0x131},
	[ 8] = {0, 2,  4, 0x331},
	[ 9] = {1, 0,  8, 0x234},
	[10] = {0, 2,  2, 0x330},
	[11] = {2, 0, 10, 0x410}
};

struct DowDiskRead {
	uint16_t	track;
	uint16_t	side;
	uint16_t	count;
};

/* 00010BE0 */
struct {
	struct DowDiskRead  read;   /* load at 0006AB5C */
	uint32_t            data;
} const dow_mis_read_info_1 = {
	[0] = {{22, 1, 6}, 0x6501},
	[1] = {{16, 1, 6}, 0x6B9F},
	[2] = {{28, 1, 5}, 0x593D}
};

/* 00010C46 */
struct DowDiskRead  /* load at 0006AAF8 */
const dow_mis_read_info_2[3] = {
	[0] = {59, 1, 1},
	[1] = {60, 1, 1},
	[2] = {61, 1, 1}
};

/* 00010CAC */
struct DowDiskRead  /* load at 0006AAF8 */
const dow_mis_read_info_3[12] = {
	[ 1 - 1] = {33, 1, 2},
	[ 2 - 1] = {35, 1, 2},
	[ 3 - 1] = {42, 1, 1},
	[ 4 - 1] = {37, 1, 2},
	[ 5 - 1] = {43, 1, 2},
	[ 6 - 1] = {45, 1, 2},
	[ 7 - 1] = {47, 1, 2},
	[ 8 - 1] = {49, 1, 2},
	[ 9 - 1] = {51, 1, 2},
	[10 - 1] = {53, 1, 2},
	[11 - 1] = {55, 1, 2},
	[12 - 1] = {57, 1, 2}
};