Getting started with WHDLoad development

[paraj]

Prompted by this thread: https://eab.abime.net/showthread.php?t=117542 (that also contains some useful tips)

Part 1: Setting up a development environment under Windows with WinUAE. (below)
Part 2: Slave archetypes
Part 3: Something old
Part 4: Something new
Part 5: Ghidra


First create a directory somewhere (without spaces), e.g. c:\amiga\whdload
In the below it will be referred to as $HDBASE, so $HDBASE/foo is c:\amiga\whdload\foo etc.

Tool setup
====================

Download http://www.whdload.de/whdload/WHDLoad_18.9_dev.lzx and extract to $HDBASE
(Or http://www.whdload.de/whdload/WHDLoad_dev.lha if you don't have a way to uncompress "modern" LZX files.)

Download http://hp.alinea-computer.de/AmigaOS/NDK39.lha and extract to $HDBASE/includes

Download https://whdload.de/whdload/lvo.lha and extract to $HDBASE/includes/NDK_3.9/Include/include_i/

Download http://sun.hasenbraten.de/vasm/bin/r..._mot_Win64.zip (16 Nov 2023 03:57:39), and extract to $HDBASE/wintools

Download https://gnuwin32.sourceforge.net/dow...ke-bin-zip.php (v 3.81) and extract "make.exe" from the bin folder in that archive to $HDBASE/wintools

Download https://gnuwin32.sourceforge.net/dow...ke-dep-zip.php and extract the 2 dll files from the bin folder to $HDBASE/wintools

Unzip the example (attached to this post) to $HDBASE

Create a file called "setup.cmd" in $HDBASE with the following contents (adjust PATH\TO for winuae):

@set HDBASE=%~dp0
@set path=%HDBASE%\wintools;%path%
@set WINUAE=PATH\TO\winuae.exe

Open a command prompt and CD to $HDBASE, run setup.cmd, and CD into example\source

Run make and you should now have an "Example.slave" in the parent directory (i.e. $HDBASE/Example).

Test setup
====================

Create an appropriate WinUAE configuration:
- CPU: 040+FPU+MMU (no JIT) and 64MB of Z3 fast RAM
- Chipset: AGA, Cycle-exact (Full). Disable "Wait for blitter"/"Immediate blitter"
- CD & Hard drives: Add directory and choose $HDBASE
- Other things to your liking
Configurations -> Save As "whdload.uae"

You should now be able to start WinUAE with that configuration by running (after running setup.cmd from the tools section):

%winuae% -config=whdload.uae -G

Install your favorite workbench version to $HDBASE (I use 3.1)

Make sure you have "Installer" in SYS:Tools (http://aminet.net/util/misc/Installer-43_3.lha)

Download and extract http://www.whdload.de/whdload/WHDLoad_18.9_usr.lzx to $HDBASE/T
(Or http://www.whdload.de/whdload/WHDLoad_usr.lha)

In WinUAE install WHDLoad from $HDBASE/T/WHDLoad

(Or install it from the dev package installed in $HDBASE/Whdload you will need an assign called "includes:")

Create S:User-Startup if it doesn't already exists, and add

cd SYS:Example
whdload Example.slave

Make sure the file has Amiga (unix) line endings

====================

In a command prompt (after having run the previously mentioned setup.cmd) go to $HDBASE/Example/source, and run

make test

The Example slave should be built, and automatically start running.


(Maybe to be continued)

[Photon]

Excellent Paraj. This is worth a go, even if just to compile a slave that isn't on WHDownload.

[jotd]

thanks for this paraj. I'm too lazy to write any documentation whatsoever

I would add "IRA" as a must-have tool (windows version is better). I also re-made "wdate" in python (wdate.py). wdate is in whdload package but only runs on amiga. It generates "datetime" file which is included in the slave to timestamp it automatically.

[paraj]

Yes, this just step 1 Wanted to make sure something simple worked.

Probably no existing slaves will build with just this, but I checked that at least my Zener Drive slave build with only minor tweaks (missing datetime file). Maybe part 2 should just explain what needs to be done to get JOTD's large trove of slaves (https://github.com/jotd666/whdload_slaves) running.

IRA is for sure part of the tool chest, but it seems like there's a million things that might be worth covering

[field3d]

@paraj wonderful tutorial. I complimented with the jotd lessons and they are very good. I checked your example.s and it emulates a load of the disk1 and change of colors in screen. Now what would happen if you need to load two disks instead just one. Sometimes there are games that request insert the second floppy, etc. How you load the 2 floppys there as if you have the 2 floppy’s inserted and is there a technique to detect that swap of change the disks by the games to remove and just load from the whdload?

[paraj]

Originally Posted by field3d

@paraj wonderful tutorial. I complimented with the jotd lessons and they are very good. I checked your example.s and it emulates a load of the disk1 and change of colors in screen. Now what would happen if you need to load two disks instead just one. Sometimes there are games that request insert the second floppy, etc. How you load the 2 floppys there as if you have the 2 floppy’s inserted and is there a technique to detect that swap of change the disks by the games to remove and just load from the whdload?

As for loading from the second disk, it's just a matter of having d2=2 when you call resload_DiskLoad and it will just read from "disk.2". That's the "easy" part and what WHDload boils down to. As a whdload auther you need to do the hard part of figuring out where the game/demo reads from floppies in a non-compatible way and patch it. There is much that can be said about this, but my slave for "Batman Rises" might serve as an example. There the demo runs until a point until it expects a disk change (which is faked) and loading continues from disk.2

[field3d]

@paraj Oh I see. Means that you load each disk in different memory position but whdload it loads from the sectors you define? Yes initially I’m trying to make a test to load 2 floppys and that the floppy 1 runs from bootsector, not initial memory but that is loaded replicating as if you plug-in the real Amiga. No patches no nothing. What start memory or jump I need to give to start from the bootsector as if you plug in the real amiga?

[copse]

Some notes. Please do not take these as criticism, I appreciate the write up and just thought I would write these down in case we could streamline the process for others.

• I found the extraction instructions ambiguous and only by examining paths in subsequent steps and reconciling why my files were in the wrong place did I realise it.
• I didn't have any way to extract lzx files. Unfortunately 7zip does not do it. Googling got me http://aminet.net/package/util/arc/W95unlzx which might help others. Note that Brave and possibly other browsers will refuse to download it as it is an http link.
• Is there a reason all the steps before downloading the example can't be removed by just providing a zip of all the downloaded files? I'd throw in useful windows-side tools like unlzx and whatever else is needed to extract and do whatever into the wintools folder if this ever happened.

The example compiled fine once I reconciled all my mistakes. Thanks!

[field3d]

@paraj look From my last message I simulated a code to see if it is correct to load the 2 disks full in memory and load from bootsecotr the disk1. Maybe could be like this?

include	whdload.i
	include	whdmacros.i
	include	kick13.s		; Include the Kickstart 1.3 source

; ====== Header ======
_base		SLAVE_HEADER				; ws_Security + ws_ID
		dc.w	17				; ws_Version
		dc.w	WHDLF_NoError | WHDLF_EmulTrap | WHDLF_EmulPriv | WHDLF_KSNoResInt | WHDLF_SingleNoMem
		dc.l	2048*1024                        ; ws_BaseMemSize (2MB total memory)
		dc.l	0				; ws_ExecInstall
		dc.w	start-_base			; ws_GameLoader
		dc.w	0				; ws_CurrentDir
		dc.w	0				; ws_DontCache
_keydebug	dc.b	0				; ws_keydebug
_keyexit	dc.b	$59				; ws_keyexit = F10
_expmem		dc.l	1*1024*1024			; ws_ExpMem (1MB Slow Fast RAM)
		dc.w	_name-_base			; ws_name
		dc.w	_copy-_base			; ws_copy
		dc.w	_info-_base			; ws_info
		dc.w	0				
		dc.l	0				
		dc.w	0			
		dc.w	_config-_base			; ws_config

_config
		dc.b	1				; 1MB Chip RAM
		dc.b	0

_name	dc.b	"WHDLoad Example",0
_copy	dc.b	"2024 paraj",0
_info	dc.b	"Fixed by you",10
        dc.b    "Version 1.0",10
	dc.b	0

; ====== Slave ======

LOADDR1=$40000
LOADDR2=$100000

start   ; A0 = resident loader
        move.l  a0,a2
	lea	_resload(pc),a0
	move.l	a2,(a0)

        ; Load the entire first disk image into memory
        moveq   #0,d0           ; offset (in bytes)
        move.l  #880*1024,d1    ; size (in bytes)
        moveq   #0,d2           ; disk number (first disk)
        move.l  #LOADDR1,a0     ; destination address
        jsr     resload_DiskLoad(a2)

        ; Load the entire second disk image into memory
        moveq   #0,d0           ; offset (in bytes)
        move.l  #880*1024,d1    ; size (in bytes)
        moveq   #1,d2           ; disk number (second disk)
        move.l  #LOADDR2,a0     ; destination address
        jsr     resload_DiskLoad(a2)

        ; Initialize Kickstart 1.3 environment
        jsr     kick13_Init(a2)

        ; Jump to the boot sector of the first disk
        jmp     LOADDR1

pl_boot PL_START
        PL_NOP  $10,4
        PL_S    $16,12
        ;PL_PS   $44,some_function ; maybe you want to do something different, but remember the patch takes up 6 bytes..
        PL_END

some_function
        rts

_resload:
        ds.l    1

[field3d]

Originally Posted by copse

Some notes. Please do not take these as criticism, I appreciate the write up and just thought I would write these down in case we could streamline the process for others.

• I found the extraction instructions ambiguous and only by examining paths in subsequent steps and reconciling why my files were in the wrong place did I realise it.
• I didn't have any way to extract lzx files. Unfortunately 7zip does not do it. Googling got me http://aminet.net/package/util/arc/W95unlzx which might help others. Note that Brave and possibly other browsers will refuse to download it as it is an http link.
• Is there a reason all the steps before downloading the example can't be removed by just providing a zip of all the downloaded files? I'd throw in useful windows-side tools like unlzx and whatever else is needed to extract and do whatever into the wintools folder if this ever happened.

The example compiled fine once I reconciled all my mistakes. Thanks!

For LZX I just used total commander in PC with the plugin of lzx and runs fine. Also I have AMIKIT and they have all at once and using Directory OPUS is very good.

[copse]

I thought I would try and see if I could get the whdload source for something I was interested in compiling. The source is included in the download and has an lzx suffix. unlzx did not recognise it. Looking at the header it is xpkf, so I installed xfdmaster and used xfddecrunch. Password required.

Is there a standard password that people know about that is not written down? This is a little weird.

[paraj]

Originally Posted by field3d

@paraj Oh I see. Means that you load each disk in different memory position but whdload it loads from the sectors you define? Yes initially I’m trying to make a test to load 2 floppys and that the floppy 1 runs from bootsector, not initial memory but that is loaded replicating as if you plug-in the real Amiga. No patches no nothing. What start memory or jump I need to give to start from the bootsector as if you plug in the real amiga?

I will try to write a part 2 that will hopefully explain it a bit, but it really depends on what game/demo you are after


Generally you would not load both disks fully into memory, but instead figure out where the disk loading function(s) are located and patch them. Very often you will find that there is one place that is essentially "load N sectors from disk M" than you can replace.

Originally Posted by copse

Some notes. Please do not take these as criticism, I appreciate the write up and just thought I would write these down in case we could streamline the process for others.

• I found the extraction instructions ambiguous and only by examining paths in subsequent steps and reconciling why my files were in the wrong place did I realise it.
• I didn't have any way to extract lzx files. Unfortunately 7zip does not do it. Googling got me http://aminet.net/package/util/arc/W95unlzx which might help others. Note that Brave and possibly other browsers will refuse to download it as it is an http link.
• Is there a reason all the steps before downloading the example can't be removed by just providing a zip of all the downloaded files? I'd throw in useful windows-side tools like unlzx and whatever else is needed to extract and do whatever into the wintools folder if this ever happened.

The example compiled fine once I reconciled all my mistakes. Thanks!

Thanks for the feedback. I think I'll just change the links to point to the LHA versions, as that is likely going to be much easier for everyone. I didn't think much of it as I also just used total commander to extract the files on the windows side, and the LZX ones are quite a bit smaller.


The reason I didn't provide a ready made zip file with everything is that I didn't want to deal with distributing it. If I upload to the zone it will go away, and more publicly I would have to think about licenses etc.
I also wanted to explain what parts are needed so it's not just a black box (also in case people want to setup something similar on e.g. Linux).

Originally Posted by copse

I thought I would try and see if I could get the whdload source for something I was interested in compiling. The source is included in the download and has an lzx suffix. unlzx did not recognise it. Looking at the header it is xpkf, so I installed xfdmaster and used xfddecrunch. Password required.

Is there a standard password that people know about that is not written down? This is a little weird.

Unfortunately you have to e-mail the whdload author in this case.. Not everyone is as open about their code as JOTD

[field3d]

@paraj Yes if you can make the 2 disks example but full loaded in memory and boot the boot sector of disk 1. Of course I understand it will change depending the title, but the idea illustrates a lot. And I mention full loaded the 2 disks because the release I guess will need all the disks loaded 100% and switch between the disks.

[paraj]

Part 2: Slave archetypes

When WHDLoad starts you are given a blank slate with a well defined system state (http://whdload.de/docs/autodoc.html#SlaveOverview). From here you can load data from disk(s), patch and run code like in the "Example slave" from part 1.
This is sufficient for "old-school" games/demos than quickly dispense of the system and start using their own track loader.

Maybe they use the system to do a bit of memory / CPU detection, and load a "stage 2" loader, but the defining characteristic is that what they do before properly starting can be easily with standard WHDLoad routines. Very often they will also be using fixed memory addresses.
If you're adapting such software, you want to use "plain" WHDLoad and just patch out system dependencies, and continue from there. An example of that is TBL-Eon (http://www.whdload.de/demos/TBL_Eon.html). You can define the memory layout as you like since the OS is not involved at all.

For anything that doesn't fall in the above category you want/need to use the "kickemu" feature of WHDLoad. With this, you don't start with a blank slate, but rather a normal or less normal booting AmigaOS. Realistically you'll only need to consider 1.3 ("A500") or 3.1 ("A1200") depending on what system was originally targeted.
The sources (that you will need to include into your slave) can be found in $HDBASE/WHDLoad/Src/sources/whdload/ (kick13.s and kick31.s are what you'll want).

When you kickemu the process is different, and you'll want to start with the examples in $HDBASE/WHDLoad/Src/slave-examples/ called kick13.s/kick31.s.

The first major decision point is whether you want/need to use disk images or data will be loaded directly from disk (usually from a directory called "data", but configurable). Sometimes your hand is forced, e.g. an intro with a single executable or an AGA game that needs HD installation (neither of which would work with disk images). This is controlled by the "HDINIT" flag.

Next, you need to consider how the target will be started, and what needs patching. There are multiple ways to hook into the process (see the example for details), e.g. at the boot block or after an executable is laoded. You can also just start an executable directly (maybe after doing a few programmatic "ASSIGN"s). Many options here..

Finally, to make things a bit more concrete, you can grab https://eab.abime.net/attachment.php...0&d=1714519265 via this https://eab.abime.net/showthread.php?t=117500.
This is just to show a "kickemu" example, not (yet) to explain how it was done, but suffice to say it was just the "kick1.3" example cut down.

Make the directories $HDBASE/atz-soil and $HDBASE/atz-soil/Source and extract atz-soil.s to the latter. Then extract the intro exectuable atz-soil.exe from the atz-soil-40k directory of https://xtr1m.com/downloads/atz-soil-40k-final.zip to $HDBASE/atz-soil.

Copy the Makefile from $HDBASE/Example to $HDBASE/atz-soil/Source and change "Example" to "atz-soil" in the second line (for the PROGNAME variable).

Call "setup.cmd" from the first part, and call make in the new source directory. It will fail because "datetime" is missing. Build failures like this are common when compiling other peoples slaves, and there is really no other way other than trying to figure out what is wrong. In this case I'm here to tell you that you just create a file called "datetime" with whatever you like (though it's meant to contain the current time and date), and then try again.

Now you probably want to adjust "S:User-startup" to test the "new" slave rather than the example, and you will also need to make sure you have the required kickstarts in your development environment under DEVS:Kickstarts (http://whdload.de/docs/en/need.html).

[paraj]

Originally Posted by field3d

@paraj Yes if you can make the 2 disks example but full loaded in memory and boot the boot sector of disk 1. Of course I understand it will change depending the title, but the idea illustrates a lot. And I mention full loaded the 2 disks because the release I guess will need all the disks loaded 100% and switch between the disks.

Please check "part 2". I hope it makes some things clearer. It doesn't answer you question, but hopefully it will establish a common vocabulary and we can figure out what you actually need. What you want depends on the software you're adapting. Loading both disks to memory is very unlikely to be what you want.

[field3d]

Originally Posted by paraj

Please check "part 2". I hope it makes some things clearer. It doesn't answer you question, but hopefully it will establish a common vocabulary and we can figure out what you actually need. What you want depends on the software you're adapting. Loading both disks to memory is very unlikely to be what you want.

Yes I see the examples in that cases the demo is about an executable is ok. And about loading specific files is very clear even to define ‘data’ dir as hundred of source codes available. With files I don’t think I have issue as even I could patch some commands in one and ran perfectly.

But what I don’t understand yet is simply. How I load the 2 disks in memory and execute the boot sector as if was inserted the disk 1 in a real amiga just that. I don’t want patches, cracks or things like that. Is possible just to load normally as it is untouched the disk images of the disks? How? I see in your first example was most clear you load a disk1 image and you jump to to $40000 but this was because you loaded a segment of sectors you wanted from the disc only and you jumpe there. Note: the disk images I pretend to use they don’t have files they are converted to the format supported by whdload but if you run that images from winuaue you inserte in df0: disk1 and df1: disk2 and done you but and they load from disk1 the boot sector. The same the real physical disks in amiga you insert and done is what I want in whdload.

Resuming this, you load a segment of a disk1 in your first demo that is better and you jumped to $40000. Suppose I load the full disk image instead of a segment and jumped to $40000 is enough? I think the $40000 is because you have code specific loaded not the full image is why I ask. If I load the whole image of any disk image and I jump to $40000 if I use that memory for example it will be enough? And of course to load the whole disk image I need to change the variable you used for size of in bytes ok? Is what I don’t understand. Most of source codes are complex as they include patches or remove copylocks or manuals, etc. I don’t want in this example that, I have a release I made time ago that has all that included in the same disk image. I just need boot and make load with kick 1.3 compatibility, chip memory 1 mb with whdload the 2 disk images nothing else.

[paraj]

Originally Posted by field3d

Yes I see the examples in that cases the demo is about an executable is ok. And about loading specific files is very clear even to define ‘data’ dir as hundred of source codes available. With files I don’t think I have issue as even I could patch some commands in one and ran perfectly.

But what I don’t understand yet is simply. How I load the 2 disks in memory and execute the boot sector as if was inserted the disk 1 in a real amiga just that. I don’t want patches, cracks or things like that. Is possible just to load normally as it is untouched the disk images of the disks? How? I see in your first example was most clear you load a disk1 image and you jump to to $40000 but this was because you loaded a segment of sectors you wanted from the disc only and you jumpe there. Note: the disk images I pretend to use they don’t have files they are converted to the format supported by whdload but if you run that images from winuaue you inserte in df0: disk1 and df1: disk2 and done you but and they load from disk1 the boot sector. The same the real physical disks in amiga you insert and done is what I want in whdload.

Resuming this, you load a segment of a disk1 in your first demo that is better and you jumped to $40000. Suppose I load the full disk image instead of a segment and jumped to $40000 is enough? I think the $40000 is because you have code specific loaded not the full image is why I ask. If I load the whole image of any disk image and I jump to $40000 if I use that memory for example it will be enough? And of course to load the whole disk image I need to change the variable you used for size of in bytes ok? Is what I don’t understand. Most of source codes are complex as they include patches or remove copylocks or manuals, etc. I don’t want in this example that, I have a release I made time ago that has all that included in the same disk image. I just need boot and make load with kick 1.3 compatibility, chip memory 1 mb with whdload the 2 disk images nothing else.

Start with the kick13.s example slave. Then tweak the options to have 2 drives, boot from disk at start, and it will just work (see attached). But if and only if no patching is required, i.e. all the software is well behaved and only uses trackdisk.device for disk loading.

When you use kickemu there is no need for manually loading the boot block or anything else for that matter. The system will boot normally, startup-sequence will run (if applicable) and so on.

Start with that, and maybe everything will just work as expected. Otherwise you can start to track down what needs to be patched.

[jotd]

all cases are covered in my git repo. But there are 400+ games covered so most of the time I remember previous games that used a similar kikck setup but you won't so reqires asking precisely or digging

[field3d]

Originally Posted by paraj

Start with the kick13.s example slave. Then tweak the options to have 2 drives, boot from disk at start, and it will just work (see attached). But if and only if no patching is required, i.e. all the software is well behaved and only uses trackdisk.device for disk loading.

When you use kickemu there is no need for manually loading the boot block or anything else for that matter. The system will boot normally, startup-sequence will run (if applicable) and so on.

Start with that, and maybe everything will just work as expected. Otherwise you can start to track down what needs to be patched.

Oh wow. This one is very impressive. Is until now the one who could boot the bootblock perfectly. But I have these comments:

1.-I know from time ago the 4 bytes in boot clock 44 4F 53 00 define and are not used to jump. In your source code where you jumped to make run the boot block? I mean after you load the disk image in a memory position and you execute the bootblock after the 44 4F 53 00 where jumps the code or maybe the amiga? How many bytes ahead this 44 4F 53 00 executes the bootblock the amiga or your code?

2.- What means exactly this in the code? blitz is a library for what exactly?

	IFD BOOTEARLY

_bootearly	blitz
		rts

	ENDC

3.-Well the boot block booted fine and I see the interaction I had there to bfe0001 to make a pause to wait a click with the mouse. After make the click it jumps to the location $73040 that is the start of the game. In your code I see the boot block and that waits the mouse button I mention after click the mouse it comes a trap code error. I attach photo of the error.
One note here I make I jump to $73040 as when you load directly the image to winuae or boot what is the same the floppy in Amiga it loads the code starting with the 44 4F 53 00 at $7d70 and from there it populates the famous $73040. This is the routine I use in the boot block to wait a mouse click:

7c9e BTST #6, 00BFE0001
7ca6 BNE 7c98
7ca8 JMP 73040

After that crash your source code.

Another note as I said. This is not a AMIGA WORKBENCH OR CLI release. Is 2 disks and they have a copy lock. Of course the copy lock was removed is just boot the floppy normally and it will run. But crash after mouse click.

@jotd I see you have al to of releases is encyclopedia of slaves. In this case if I understand the one I’m doing I think I will understand rest.

[field3d]

BTW is an impressive code really complex but impressive all the cases are there. This is the our mastership in boots. Honestly, I don’t see manuals that cover all this even some commands.

Adding to my previous comments. I think i detected the issue in this case:

I see what is the problem with his one for disk 1 your code loads in some memory as 5c40 the 44 4F 53 00 and is needed here 7d70. Where you can define the change of memory to 7d70 instead 5c40? 2130 in hex is the difference. I don’t see a real place to change the load of the disk image to run the bootblock. I just want to mention if you boot in a real amiga or winaue the disk image it will load to 7d70 I don’t know why your source code send to 5c40.

BTW is an impressive code really complex but impressive all the cases are there.