Getting started with WHDLoad development

[field3d]

Wonderful a lot of things to learn. But is interesting because I see the famous copylock it was a mess. I remember in that times that release only use the boot sector and check the free loads because the code changed and replaced by other code while was loading. Bit is more clear. I wil try to assemble during these days. Thanks @jotd qnd @paraj@ for the ideas.

One more thing @paraj Yes I understand the python you run, but what I meant is how di you extracted the main source .bin? You attach int eh files but how did you exctracted that bin from the images? With the tool of boot grab of same whdload?

And another minimal thing I’m malign some slaves and they are fine. I tried some with Boot DOS option but I noted in some random releases they run very fast the display of the credits as if you are making fast forward with your remote control. Is there a way to make a cpu delay or make the speed accurate to the old amiga forced in these cases they aren’t to smart their development?

[paraj]

Part 4: Something new.

This is going to be a bit fast, and is just to have an example of something "new-school". We'll look at the demo Jesus Christ Motocross (https://www.pouet.net/prod.php?which=52968).

From the NFO file we can see that it requires FPU+AGA, so we'll start with the generic kick31 slave.

Make a "JesusChr" directory in HDBASE, extract the demo to $HDBASE/JesusChr/data and copy get the generic slave ($HDBASE/WHDLoad/Src/slave-examples/kick31.asm) to $HDBASE/JesusChr/source and rename it to "JesusChr.s".
You'll also want to copy the "generic makefile" from the example in part 1 and edit the Makefile (and s:user-startup) with the new names.

Next, we'll tweak the settings. We fill in the info/name/copyright stuff, and since we'll just need to launch an executable enable BOOTDOS (and change the _program variable to launch "JesusChristMotocross")). Enable INITAGA (since it's 020+ only) and disable NO68020 since that won't work anyway. Point slv_CurrentDir at "data" as that's where we chose to extract the files (you can use whatever you like, but "data" is conventional).

Now is a good time to check if we've got most things right, so try building and launching. You should - if everything else went OK - get an error about it being the wrong version. This is expected because the example slave does a basic CRC check of the executable to see it the expect version is installed. Remember that the data will be provided during installation by the user, so you can't just assume the right version will be there. By checking for the correct version you will help yourself in addition to users.
While you could calculate the CRC yourself, it's much easier to just place move.l d0,d7 \ blitz after call to resload_CRC16 and read out the value in a debugger (it's moved to d7 because blitz clobbers d0). In this case I get $9610. Replace the value checked and remove the debug code.

Testing again (now with the correct CRC value) you should get a complaint about not enough memory being available. I.e. CHIPMEMSIZE/FASTMEMSIZE at the top need to be tweaked. For fast mem, we know from the NFO that it (probably) needs to be 16MB, so start with that. You can iterate a bit here, but 1.5MB of CHIP and 16 MB of fast seems to work.

After this, the demo should start, but will complain about a lack of FPU. At this point you might be tempted to use NEEDFPU, and while that does work for some (most?) setups it doesn't on 060. So enable FPU manually with resload_SetCPU and just fake the necessary bits in AttnFlags (see attached example). This is a bit ugly and maybe I'll elaborate, but just trust me for now

At this point the demo should run to completion, but you'll notice that you can't quit with the quit key (only LMB). That's because the slave modifies the VBR register, but we'll look at that later.

I've removed unused portions from the generic slave in the attached code, but if you're following along, you want to do the above steps yourself to familiarize yourself with the code.

[jotd]

Is there a way to make a cpu delay or make the speed accurate to the old amiga forced in these cases they aren’t to smart their development?

one of the non-trivial things to fix

Track down the infinite loops in the code like itself: DBF D0,itself (51C8FFFE or more generally 51CxFFFE). This doesn't help if there are NOPs. I wrote a m68k_checker.py tool to try to find those infinite loops.

Those can be replaced by "beamdelays" (wait for vertical beam change). Can be found in keyboard handlers and protracker replay routine players.

If the mainloop itself is too fast (game knows it's slow and it expects no more than 1 update per 2 frames but actually because of fast cpu it is updated every frame) you have to count frames and wait 1 frame to simulate 25Hz or the game will be too fast in some sections on fast machines. I fixed that part a lot. For instance in Magic Pockets, Earl Weaver Baseball, LED storm, numerous games.

[paraj]

Part 5: Ghidra

In the last part I mentioned that that the demo modifies VBR, and we need to fix that. Just to show another tool, let's look at Ghidra.
Now I will preface this by saying that I am by no means an expert, and that in this case you don't need it all - plain IRA will do. It's also buggy and often annoying for Amiga use, but it can be a valuable piece in your tool chest.
It works best for things that are written in a high-level programming language or at least doesn't use too many weird calling conventions. I find it mostly useful for getting a quick high level view overview and jumping around in the code.
For complicated stuff I also have an IRA disassembly that I keep a bit in sync with the Ghidra stuff (for labels), since it's easier to get the proper offsets from IRA (search for instruction byte sequences to match them up). IRA doesn't (currently) do FPU stuff and Ghidra is a bit broken, but in combination things can work...

At the time of writing I use Ghidra-10.3 with the Bartman's fork of the amiga plugin (get it from here: https://github.com/BartmanAbyss/ghidra-amiga). You can have multiple versions side by side.
Extract Ghidra somewhere and download (but don't extract) the amiga plugin.
Start Ghidra and use File -> Install Extensions, click the plus sign in the upper right corner of the dialog and select the zip-file (i.e. don't extract it). Then restart Ghidra.
Now choose File -> Import File, and in the dialog Options -> Change Image Base to something nicer (00200000) as that will make making the patch list easier.
Double click the file and do the analysis (click yes etc.)
You should now have a view that looks something like this:

You can now navigate the code by double clicking on functions in either the listing or decompile view, rename functions, start changing data types etc. I will not go into any details here as that would take all year, so we'll just pretend we've done enough analysis.
Select Search -> Program Text from the menu and click the "all fields" radio button in the search for field type "VBR" and click "Search All", and you should see:

Double click the first occurrence and it'll go to "LAB_002006f6" which is the first interesting function. Click on label and type "l" to label it something nicer (and while we're at it do the same for the one below) like "GetVBR" and "SetVBR".

Notice the "XREF" from "FUN_00200622"? Doubleclick on the FUN part to see where this is used. It goes to a code block that looks like this:

        0020063e 2c  78  00  04    movea.l    (PTR_00000004 ).w,A6                             = NaP
        00200642 4b  fa  00  b2    lea        (0xb2 ,PC)=>GetVBR ,A5
        00200646 4e  ae  ff  e2    jsr        (-0x1e ,A6=>exec_library_Supervisor )             BPTR dos_library_Open(CONST_STRP
                                                                                             ULONG exec_library_Supervisor(FU
                             -- Call Destination Override: exec_library_Supervisor (00b400
        0020064a 29  48  00  0c    move.l     A0,(0xc ,A4)=>DAT_0021851c
        0020064e 43  f9  00       lea        (DAT_00218110 ).l,A1
                 21  81  10
        00200654 30  3c  00  ff    move.w     #0xff ,D0w
                             LAB_00200658                                    XREF[1]:     0020065a (j)   
        00200658 22  d8           move.l     (A0)+,(A1)+=>DAT_00218110
        0020065a 51  c8  ff  fc    dbf        D0w ,LAB_00200658
        0020065e 4b  fa  00  9c    lea        (0x9c ,PC)=>SetVBR ,A5
        00200662 41  f9  00       lea        (DAT_00218110 ).l,A0
                 21  81  10
        00200668 4e  ae  ff  e2    jsr        (-0x1e ,A6=>exec_library_Supervisor )

Aha! So it's copying the vector table somewhere (fast ram) and setting VBR to point to it. Label DAT_0021851c as "OldVBR" (Either right click it and select "Edit Label" or use they keyboard to go there and press L).
DAT_00218110 seems to be the new vector table, so let's name that VectorTable. That also seems to be our new point of interest, so let's go there. Either double click it or press enter with your cursor placed on the label.
The vector table is an array of 256 function pointers, so let's type it as such. Right click on the label, choose Data -> Choose Data Type... (or press T). Type "func * [256]" (Choose any of the "func" types presented).
This last step is not necessary but just to show how you'd go about slowly adding proper types to a program.

Now you should see that there are 2 writes to the vector table from "start" in the XREF part:

If you go there, you will see that it sets entries for 27 and 28 (corresponding to interrupt level 3 and 4 respectively).

Now for this demo we only need to fix the VBR access, so it's not really necessary to dive into the details. The above was just an example of how you can use Ghidra to inspect games/demos.

For WHDLoad purposes, we just want to keep the VBR at 0 and not let the demo modify it. One way of doing that is to add/change:

_pl_program    PL_START
                PL_P    $0006f6,novbr   ; GetVBR (In Ghidra 002006f6)
                PL_P    $0006fc,novbr   ; SetVBR
                PL_L    $000054,0       ; Vector table -> 0
                PL_P    $0006a8,quit
        PL_END

novbr
                sub.l   a0,a0
                rte

quit
        pea    TDREASON_OK
        move.l    (_resload,pc),a2
        jmp    (resload_Abort,a2)

See if you can figure out how it works, and thanks for coming to my TED talk. Maybe come up with a better patch

[copse]

Ghidra works pretty well, but there are indeed some pain points with how correctly it does things like correct struct field references. I've listed some in another thread here if anyone wants to see what they might have to watch out for.

[copse]

Do you do anything special to get the `Custom.` references working?

[paraj]

Originally Posted by copse

Do you do anything special to get the `Custom.` references working?

They work out of the box with Bartman's version of the plugin, but not the one from the Ambermoon people (at least not when I tested it before writing the post). OTOH proper LVO detection seems to work better with the latter.. I've contemplated looking into it myself, but I don't really need another timesink.

[field3d]

I found some releases that are very interesting to see what is the issue. All now is clear about the ‘normal’ slaves. But I found other slaves as this one:

“Creatures”

https://www.whdload.de/games/Creatures.html

I’m using vasm with -devpac and also tested without it.

Main code Creatures.s (File is untouched):

https://pastebin.com/9W1kdcBD

File Fire.asm:

https://pastebin.com/ynHPVXr4

I changed the instances “dl” to “dc.l”

That I needed to change because showed errors like: “unknown mnemonic <dl> dl $3ff0009”

File ReadJoypad.s (File is untouched):

https://pastebin.com/CjENi3Sa

Note that ReadJoypad.s in other releases (NOT THIS ONE) show the error in line 60:

"label in operand required > lea potgo(pc),a0”
and is needed to change to override the error to “lea potgo,a0”
In this release I don’t see that error yet when compile then that is the reason I didn’t modified something in the source code.

In this release the errors that show when I try to compile are:

error 23: undefined symbol <_high>

error 23: undefined symbol <_exit>

error 23: undefined symbol <_CD32_Keys>

error 23: undefined symbol <_CD32_Pause>

error 23: undefined symbol <_CD32_Fire>

However, as seen in the main code in the file that declarations are present inside the source code at the bottom. The error is impossible as source code have them. These are the declaration in the code in main Creatures as can be seen in the main source code pastebin:

;======================================================================

_high
		dc.b		"Creatures.high",0
		even

_held_button	dc.l	0
		even

_resload	dc.l	0
_Tags           dc.l    WHDLTAG_CUSTOM1_GET
_Custom1	dc.l    0
         	dc.l    WHDLTAG_CUSTOM2_GET
_Custom2	dc.l    0
         	dc.l    WHDLTAG_CUSTOM3_GET
_Custom3	dc.l    0
         	dc.l    WHDLTAG_CUSTOM4_GET
_Custom4	dc.l    0
		dc.l    WHDLTAG_BUTTONWAIT_GET
_ButtonWait	dc.l    0
                dc.l    TAG_DONE

;======================================================================

_exit     
		pea     	TDREASON_OK
		bra		_end
_debug		pea		TDREASON_DEBUG
		bra		_end
_wrongver	pea		TDREASON_WRONGVER
_end		move.l		(_resload),-(a7)
		add.l		#resload_Abort,(a7)
		rts

 
;======================================================================

Then how can be fixed that errors of undefined symbol? I didn’t touch any order of the code is all intact as it is the original. I just changed what I mention above. I think they have something strange here. Why is not compiling this one? How can be fixed to compile? As I mentioned this case is not particular of this one there are other titles and is very interesting see why is this error and how can be fixed. Any idea?

[Don_Adan]

For potgo, code seems to be buggy for me:

Change this

 lea	potgo(pc),a0
 move.w	#$ffff,a0

to

 lea	_custom,a0
 move.w	#$ffff,potgo(a0)

[jotd]

there may be an "END" directive somewhere, either in the main file or in an included .s file.

this ends assembly and the rest is ignored.

[field3d]

@JOTD I fixed I want to add that the error was Fire.asm the developer entered an end there what happened there hehe

Fire.asm and they are heard now bu. I fixed other errors as

illegal opcode extension
> bset.l d1,(a0)
fix to bset d1,(a0)

a lot of that instances.


@Don Aran. Yes I noted that but that line doesn’t fix the errors of the declarations. After a lot of changes of that illegal operando I placed your fix and worked.

I will test the slave in a real amiga to see if works.

Thank you for the tips

Thanks paraj, jotd and all these lessons are very constructive and make you catch things fast. Last lesson of Ghudra is powerful BTW

[field3d]

Just one thing this declaration says “relocation not allowed”

_end move.l (_resload),-(a7)
add.l #resload_Abort,(a7)
rts

Apparently is the only one with error. Why marks error and now to fix?

[Don_Adan]

Yes, but this is bug and second fire will be doesnt works correctly.

For other problems, this is only question of assembler.
In general Im anti assembler optimisation old guy.

For me if Im using lea base,a0 then IT MUST BE assembled as lea base,a0, not as lea base(pc),a0.
From my memory Barfly assembler has optimisation which convert absolute addressing to PC relative addressing.
Some coders was lazy enough, and used assemblers for slave (PC relative) optimisation.
Perhaps this optimisation is not handled by Vasm.
WHDload slave must be fully PC relative.
Then my first step, if I worked with slave from other coder was removing all optimisations from source code.
Second step was adding missing "(PC)" to code.
After this, you will be received working slave.

[jotd]

yes, Don is right. I hate Barfly optimizer mode. Everything should be PC relative explicitly.

_end move.l (_resload),-(a7)

=> _end move.l (_resload,pc),-(a7)

This is something you'll find a lot in Codetapper slaves. Everything is explicitly PC relative except the resload in the Abort function.

[field3d]

There was other line I fixed following your pattern: move.l (_resload),a2 changed to move.l (_resload,pc),a2

@JOTD yes that is like a prank for for guys as me hehe. But this problem was due Barfly? Exactly what it does Barfly?

Also maybe these releases are compiled in other assembler instead of vasm? In my frustration I used the real ‘devpac’ in amiga thinking it was that but not. It was a beautiful END in Fire.asm all the official releases with this file in whdload have this problem.

I ran the salve and worked 100% even in real amiga. Well another good lesson.

[jotd]

Barfly has in-line opt directives (start of file). It is the historical assembler used by Bert and many others on the amiga. All slaves have a "IFD BARFLY" section with "BOPT" directives, one of them trying to add (pc) to addresses.

But that's bad practice. As you end up doing things like move.l d0,addr and hope that the optimizer adds (pc)...

Glad you could create a slave! You don't have to test everything on a real amiga. Most of the time if it runs on winuae it runs on the real thing. But there are stuff to enable to make sure that you see the defects on winuae:

- no blitter wait or immediate. If you have gfx glitches, set "immediate" and if the glitches disappear, then it's a blitterwait issue!
- chipset hack set to 8: that is to detect wrong/missing dma wait when playing sounds. If sound is trashed in WinUAE, then someone didn't wait enough between turning dma off and starting the next sound.

in winuae config

chipset_hacks=0x8

[Don_Adan]

Originally Posted by field3d

Fire.asm and they are heard now bu. I fixed other errors as

illegal opcode extension
> bset.l d1,(a0)
fix to bset d1,(a0)

a lot of that instances.

This is incorrect fix for buggy joypad code, I think.
I never coded for joypad on Amiga.
But in source code defined are minimum 23 bits.

BITDEF	JP,JOY_R,0
	BITDEF	JP,JOY_L,1
	BITDEF	JP,JOY_U,2
	BITDEF	JP,JOY_D,3
	BITDEF	JP,BTN_PLAY,$11
	BITDEF	JP,BTN_REVERSE,$12
	BITDEF	JP,BTN_FORWARD,$13
	BITDEF	JP,BTN_GRN,$14
	BITDEF	JP,BTN_YEL,$15
	BITDEF	JP,BTN_RED,$16
	BITDEF	JP,BTN_BLU,$17

Then it must be works for longword memory handling, not for byte handling.

Perhaps you must use something like this:

 move.l (A0),Dx
 bset D1,Dx
 move.l Dx,(A0)

[field3d]

@Don_Adan If I hdr that change I receive the error ‘error 23: undefined symbol <Dx>’ Apparently Dx is not recognized.

[Don_Adan]

Dx is any free register, which can be used in code. f.e D3
If not free register, you must put and restore this register on stack using movem commands.

[Don_Adan]

Here is example.

From this

_CD32_Pause	
		movem.l         d0-d1/a0,-(sp)
		move.l		joy1(pc),d0
		lea		_held_button(pc),a0
		move.l		#JPB_BTN_PLAY,d1
		btst.l		d1,d0
		beq		.notPressed
		btst.l		d1,(a0)
		bne		.checkKey
		bset.l		d1,(a0)
 
		cmp.b		d0,d0
		bra		.exit
.notPressed	bclr		d1,(a0)
.checkKey	cmpi.b		#$19,($3539,a5)
.exit		movem.l         (sp)+,D0-D1/A0
		rts

to this

_CD32_Pause	
		movem.l         d0-d2/a0,-(sp)
		move.l		joy1(pc),d0
		lea		_held_button(pc),a0
      move.l (a0),d2
		move.l		#JPB_BTN_PLAY,d1
		btst.l		d1,d0
		beq		.notPressed
;		btst.l		d1,(a0)
    btst d1,d2
		bne		.checkKey
;		bset.l		d1,(a0)
    bset d1,d2
   move.l d2,(a0)
		cmp.b		d0,d0
		bra		.exit
.notPressed
;	bclr		d1,(a0)
        bclr d1,d2
.checkKey
    move.l d2,(a0)
	cmpi.b		#$19,($3539,a5)
.exit		movem.l         (sp)+,D0-D2/A0
		rts