UAE debugger updates (executable debugging, segment tracking, symbols and more)

[Toni Wilen]

Very experimental debugger updates are now available for testing. (Features previously mentioned here: http://eab.abime.net/showthread.php?t=87499)

https://download.abime.net/winuae/files/b/uaedbg.zip (Amiga-side loader)

Updates:

- "zero page" and non-existing address support.
- amiga.lib LVO symbol support.
- stack frame tracking.
- stack frame tracking without debugmem debugging + supervisor stack frame tracking.
- amiga.lib/*.fd parsing for symbols.
- segtracker like functionality
- lvo are resolved if scanned with TL. ("d exec/wait" disassembles exec.library Wait function)
- break to debugger when any unexpected exception happens (tc_TrapCode)

--

New debugger feature: ability to load any Amiga executable to special debugger RAM:

- Static location, custom loader: program is always loaded in same addresses.
- Each hunk is separated with (later configurable) 32k space to detect possible out of bounds accesses.
- Program's memory allocations are also allocated from debugger RAM, with mungwall-like out of bounds access detection.
- Program's stack is also moved to debug RAM.
- Out of bounds detection is real-time and byte-based, detects both reads and writes. Detects also reads from uninitialized memory (read before write).

Note that stack does not have unitialized memory read detection because dos does not always write to all addresses in stack frames it allocates (for example when it creates dos packet structures) which causes too many false positives.

If debug ram is not manually configured and uaedbg is run, 256M of memory is automatically allocated if $70000000 to $f0000000 has at least one free 256M block. It can be manually configured by adding following lines to config file:

debugmem_start=<address>
debugmem_size=<size in megabytes> Must be power of 2.

Amiga-side debugger command (may become built-in later):

uaedbg <program to debug> <program parameters>

uaedbg loads the program, relocates it to debug memory region, moves stack to debug memory region, breaks to debugger with PC pointing to first instruction of program. Hooks also AllocMem() and FreeMem() and if current task is same as uaedbg task, memory allocation is done from debug ram. (This needs to handle programs that create new tasks, later..)

If program includes symbols (HUNK_SYMBOL), they are also loaded and shown in disassembler when current address matches or instruction's effect address matches symbol's address.

Segment 1: 000003e9 70008000-700122ef
__stext:
70008008 [000000] 23c8 7002 3cfc MOVE.L A0,$70023cfc [00000000] ___commandline
7000800E [000006] 23c0 7002 3cf0 MOVE.L D0,$70023cf0 [00000000] ___commandlen

If program includes GCC stab (-g) debugging information in HUNK_DEBUG, it is loaded and parsed, including source files. Disassembly will automatically list matched source files.

70008172 [00016A] 48e7 3f3e MOVEM.L D2-D7/A2-A6,-(A7)
70008176 [00016E] 2478 0004 MOVEA.L $0004 [08000810],A2

240 SysBase = *((struct ExecBase **)4);
241
242 /* init globaldata */

7000817A [000172] 2c4a MOVEA.L A2,A6

243 g = AllocMem(sizeof(struct globaldata), MEMF_CLEAR);

7000817C [000174] 203c 0000 09d0 MOVE.L #$000009d0,D0
70008182 [00017A] 7201 MOVE.L #$01,D1
70008184 [00017C] 4841 SWAP.W D1
70008186 [00017E] 4eae ff3a JSR (A6, -$00c6) == $00f9f4ec

Debugging information also contains variable type data, register and local stack parameter data but it is not yet parsed or used.

Debugger also resolves final address when it notices JSR x(PC) + JMP <final address> instruction combination:

70008252 [00024A] 4eba 05f4 JSR (PC,$05f4) == $70008848 == $70015718 _FixStartupPacket

vs old non-resolved final address:

70008252 [00024A] 4eba 05f4 JSR (PC,$05f4) == $70008848

Chip RAM "zero page" (0 to $400) is now watched (like Enforcer). All writes break to debugger immediately, long read from 4 is allowed, if vbr is 0: long aligned exception vector reads are allowed. All other reads break immediately.

Non-existing RAM/IO read or write also breaks to debugger immediately.

LVO symbol support:

if amiga.lib (from Amiga NDK) is in plugins/debugger, it is scanned for LVO symbols.
if plugins/debugger/fd contains one or more *.fd files, they are parsed and added to LVO symbol list.

When disassembling jsr -$xx(a6) instructions will include name of library and LVO if a6 and -$xx matches.

LVOs are resolved like this (See also TL command):

70008034 [00002C] 4eae feda JSR (A6, -$0126) == $080006ea exec/FindTask == $00f8272a

If LVO jump table points to other known segment (not debugged program's segment), segment information is also shown.

Stack frame tracking:

Tracks debugged program JSR/BSR/RTS instructions.

70008052 -> 700080a0 SP=7003c0f8 [000098] Segment 1: 000003e9 70008000-700122ef
70008052 [00004A] 614c BSR.B #$4c == $700080a0
70008054 [00004C] 2f39 7002 3d00 MOVE.L $70023d00 [00000000] ___env,-(A7) [700080b8]

700080b6 -> 700121ac SP=7003c0f4 [00A1A4] Segment 1: 000003e9 70008000-700122ef ___initlibraries
700080B6 [0000AE] 4e90 JSR (A0) ___initlibraries
700080B8 [0000B0] b584 EOR.L D2,D4

700121d6 -> 080005e8 SP=7003c09c
700121D6 [00A1CE] 4eae fdd8 JSR (A6, -$0228) == $080005e8 exec/OpenLibrary == $00fbf246
700121DA [00A1D2] 2680 MOVE.L D0,(A3) [00000000] _DOSBase

00fbf24e -> 080182b8 SP=7003c078
00FBF24E 4ead 0038 JSR (A5, $0038) == $080182b8
00FBF252 2400 MOVE.L D0,D2

PC before JSR/BSR, PC after JSR/BSR instruction, Stack pointer before JSR/BSR.
Segment offset and segment details new PC points to debugged program.
SR is also listed if supervisor mode stack frame.
Disassembly of branch instruction + following instruction.

Other new debugger features:

- Early boot segtracker-like feature for loadable libraries and others, enable in misc panel.

New debugger commands:

- tr = break when PC points to any allocated debug memory
- tl = break when PC matches next source line, step to next source line.
- seg = list loaded program's segments.
- segs = list all segtracker loaded segments.
- u = inhibit current debugmem break to debugger method. (ua = inhibit all, uc = clear all)
- TL = scan and match library bases (exec library, device and resource lists) with loaded amiga.lib/*.fd symbols. Automatically done when starting debug mem debugging session.
- rs = show tracked stack frame.
- rss = show tracked supervisor stack frame.
- ts = break when tracked stack frame count decreases. (=tracked stack frame matched executed RTS)
- tsp = break when tracked stack frame count decreases or increases. (RTS/BSR/JSR).
- tse/tsd = enable/disable full stack frame tracking. Can be used when no debugmem debugging is active.

ts/tsp stores current supervisor mode and only breaks to debugger if stack frame operation has same supervisor mode.

All debugger commands that take address or register value also support symbol names. (for example "d __stext")
"library/lvo" is also resolved if library was found when scanned with TL. (for example "d exec/wait" disassembles Wait())
Notes:

- Don't mention anything about GUI. Proper source level debugger may require it but I am not yet sure if it really can be done or is worth the trouble..
- Console debugger must be in use. (If GUI mode: enter xx command to switch)
- If executable has chip ram hunks, whole address space is made chip ram compatible and then hunk is loaded to debug ram.
- Very unoptimal data structures, most also have static sizes. Odd errors and crashes can happen..
- JIT must be off.
- 32-bit addressing must be enabled.
- KS 2.0+ only.

TODO:

- Does other popular compilers also generate useful HUNK_DEBUG data?
- Suggestions needed! (But don't mention GUI!)

[alpine9000]

Awesome!

So correct me if I am wrong, but the hunk separation will give improved detection of bad access compared with enforcer?

I.e enforcer would never pick up bad access across hunk boundaries that are still within the programs allocated ram?

[Toni Wilen]

Yes. Usually.

It is impossible to know if program is crossing hunk boundaries because it wants to do it or it is a bug but generally accidental accesses are relative accesses so 32k gap in both sides of hunk will guarantee out of bounds accesses will always hit invalid addresses.

Enforcer is also limited to MMU page size, it can't do byte-perfect checks.

Also because emulator debugger knows exactly what CPU (and whole hardware) is doing, it can do many extra checks that are impossible when using real hardware.

[ross]

Great additions!

If debug ram is not manually configured and uaedbg is run, 256M of memory is automatically allocated if $70000000 to $f0000000 has at least one free 256M block.

Toni, can you elaborate this?
You remap high memory (>$80000000)?

Thanks for everything!

[Toni Wilen]

If $70000000-> is free, it selects $70000000-$7fffffff region. If not, it tries next 256M and so on..

No program should care if highest bit is set in addresses because this memory is not allocated using any normal memory allocation functions. Exec does not even know it exists.

[ross]

Originally Posted by Toni Wilen

If $70000000-> is free, it selects $70000000-$7fffffff region. If not, it tries next 256M and so on..

No program should care if highest bit is set in addresses because this memory is not allocated using any normal memory allocation functions. Exec does not even know it exists.

Ok

Why not use from start $80000000 as base?

[alpine9000]

I tried to run my WIP game with a basic A1200 config but with 8mb fast, and I got the message:

"Failed to automatically allocate debugmem (no space)!"

EDIT: after re-reading your first post I changed to 030 and gave it 512mb and it started!

[alpine9000]

So I gave it a try and not sure if I am using the debugger correctly (I am mainly used to the FS-UAE debugger).

Load my game with uaedbg game.exe

A console windows shows a bunch of Allocation messages, then debugger appears at the entry point to my program - I hit "g" and game starts.

Menu system runs ok, then when I start the game the debugger breaks on my vblank wait routine on the access to $dff004/VPOSR, PC/A7 etc all seems to be in a valid segment range from "seg"

  D0 A3000003   D1 00000000   D2 00000000   D3 00000000
  D4 00000000   D5 00000001   D6 00000000   D7 00000000
  A0 700FA698   A1 00000000   A2 00000040   A3 7005FCE0
  A4 700080EC   A5 700FA658   A6 700C10D4   A7 70180848
USP  70180848 ISP  07802288 SFC  00000000 DFC  00000000
CACR 00002111 VBR  00000000 CAAR 00000000 MSP  00000000
T=00 S=0 M=0 X=0 N=1 Z=0 V=0 C=0 IMASK=0 STP=0
0: FFFF-7FFFFFFF-FFFFF800 +nan FFFF-7FFFFFFF-FFFFF800 +nan
2: FFFF-7FFFFFFF-FFFFF800 +nan FFFF-7FFFFFFF-FFFFF800 +nan
4: FFFF-7FFFFFFF-FFFFF800 +nan FFFF-7FFFFFFF-FFFFF800 +nan
6: FFFF-7FFFFFFF-FFFFF800 +nan FFFF-7FFFFFFF-FFFFF800 +nan
FPSR: 00000000 FPCR: 00000000 FPIAR: 00000000 N=0 Z=0 I=0 NAN=0
700080F0 [0000E8] 2039 00df f004           MOVE.L $00dff004,D0
Segment 1: 000003e9 70008000-70060047
700080F6 [0000EE] c0bc 0001 ff00           AND.L #$0001ff00,D0
Next PC: 700080fc

seg
Segment 1: 000003e9 70008000 - 70060047 (360520)
Segment 2: 000003ea 70068100 - 700c3efb (376316)
Segment 3: 400003ea 700cbf00 - 700cc79b (2204)
Segment 4: 000003eb 700d4800 - 700fb44f (158800)
Segment 5: 400003eb 70103500 - 70176817 (471832)
Segment 6: 0000ffff 7017e900 - 701808ff (8192)

d 700080eC
_hw_waitVerticalBlank:
700080EC [0000E4] 48e7 8000                MOVEM.L D0,-(A7)
700080F0 [0000E8] 2039 00df f004           MOVE.L $00dff004,D0
700080F6 [0000EE] c0bc 0001 ff00           AND.L #$0001ff00,D0
700080FC [0000F4] b0bc 0001 2d00           CMP.L #$00012d00,D0
70008102 [0000FA] 66ec                     BNE.B #$ec == $700080f0 (T)
70008104 [0000FC] 2039 00df f004           MOVE.L $00dff004,D0
7000810A [000102] c0bc 0001 ff00           AND.L #$0001ff00,D0
70008110 [000108] b0bc 0001 2d00           CMP.L #$00012d00,D0
70008116 [00010E] 67ec                     BEQ.B #$ec == $70008104 (F)
70008118 [000110] 4cdf 0001                MOVEM.L (A7)+,D0

[Toni Wilen]

It needs 32-bit address space, amount of RAM is irrelevant.

There should have been some debugger break reason text before register output. Did you miss it?

[alpine9000]

Originally Posted by Toni Wilen

It needs 32-bit address space, amount of RAM is irrelevant.

There should have been some debugger break reason text before register output. Did you miss it?

Nothing above the register output. Is it because I am using the GUI ? Couldn't work out how to use the console debugger ?

[Toni Wilen]

xx = swap modes.

Also update: "zero page" and non-existing address accesses break to debugger, also added command to temporarily disable break to debugger. (see first post for more details)

[alpine9000]

Originally Posted by Toni Wilen

xx = swap modes.

Also update: "zero page" and non-existing address accesses break to debugger, also added command to temporarily disable break to debugger. (see first post for more details)

Cool - console debugger worked much better!

Got the message:

Invalid access. Addr=700cc79c RW=R Size=2: Accessing invalid memory

Which looks like the copper reading beyond the end of the copper list ?

I put a 32bit pad after my 0xFFFF, 0xFFFE in the copper and that break went away.

Tested the rest of the game and no more breaks.

[Toni Wilen]

Originally Posted by alpine9000

Which looks like the copper reading beyond the end of the copper list ?

I put a 32bit pad after my 0xFFFF, 0xFFFE in the copper and that break went away.

Copper does do extra fetch from end of copper list before it jumps to beginning of copper list.

Currently debugger does not know that it was copper or CPU or something else.

[alpine9000]

Originally Posted by Toni Wilen

Copper does do extra fetch from end of copper list before it jumps to beginning of copper list.

Currently debugger does not know that it was copper or CPU or something else.

Yeah it’s no problem for me to put a pad there, as long as I know it’s not a bug in my code ;-)

Other than that, very nice work! Looking forward to testing some older programs and seeing if there are any bad accesses.

[Toni Wilen]

Boring results

Next I am going to parse amiga.lib to get LVO symbols, enabling disassembler to automatically convert jsr -$xxx(a6) to LVO names. If A6 points to valid library or device etc.

This will be generic debugger feature, it won't be restricted to new executable debugger.

[Toni Wilen]

amiga.lib LVO symbol support implemented.

[Toni Wilen]

Stack frame tracking implemented. (Note: currently requires non-prefetch CPU mode)

[alpine9000]

Originally Posted by Toni Wilen

Stack frame tracking implemented. (Note: currently requires non-prefetch CPU mode)

Nice!, What is non-prefetch CPU mode ?

[Toni Wilen]

Originally Posted by alpine9000

Nice!, What is non-prefetch CPU mode ?

Same as "more compatible". (but it also depends on CPU type)

But it is already fixed. Also added new command to exit one level of stack frame (ts).

Because stack frame tracking is very useful, I'll try to make it more generic, to support programs that take over the system. (Makes exiting out of any subroutine very easy and quick, without boring manual disassembly and setting breakpoints)

[alpine9000]

Originally Posted by Toni Wilen

Same as "more compatible". (but it also depends on CPU type)

But it is already fixed. Also added new command to exit one level of stack frame (ts).

Because stack frame tracking is very useful, I'll try to make it more generic, to support programs that take over the system. (Makes exiting out of any subroutine very easy and quick, without boring manual disassembly and setting breakpoints)

Indeed, easily returning from a subroutine would definately be a massive win!

I sometimes accidently go into a subroutine, and it can be very difficult to get back if you were not prepared.