Relinking executables with VLINK

[hop]

I've had fun writing loaders for both standard Amiga Hunk loadfiles, and vlink's rawseg (-q) format. The rawseg loader is considerably more straightforward and efficient - it's a great format which I would prefer to use wherever possible.

When I'm working with my own code, I can easily assemble source and link to rawseg binaries, but obviously most pre-existing (third party) executables are in hunk format.

I thought it might be possible to use vlink to convert a hunk executable into rawseg format, so I threw phx's linker script at it to see what happened.

PHDRS {
    fast PT_LOAD;
    chip PT_LOAD;
}

SECTIONS {
    . = 0x400;
    .chip: {
        *(chipmem)
        . = ALIGN(2);
        _FREECHIP = .;
    } :chip

    . = 0x100000;
    .text: { *(CODE) } :fast
    .data: { *(DATA) }
    .sdata: {
        _LinkerDB = . + 0x7ffe;
        _SDA_BASE_ = . + 0x7ffe;
        *(.sdata __MERGED)
    }
    .bss: { *(BSS) }
}

vasmm68k_mot -Fhunkexe -kick1hunks -no-opt -nosym -wfail -o intermediates\loadfile loadfile.asm
vlink -brawseg -q -Tloadfile.ld -o intermediates\loadfile.rawseg intermediates\loadfile

but it doesn't seem to like it:

Warning 12: "loadfile" is already an executable file.
Warning 64: Section loadfile(S02680a7c) was not recognized by target linker script.
Warning 64: Section loadfile(S02680b14) was not recognized by target linker script.
Warning 64: Section loadfile(S02680b24) was not recognized by target linker script.

This is really just an exercise, but I wondered if this *should* be possible, or if there is not enough information in the hunk exe to allow this process.

Is Warning 12: "loadfile" is already an executable file. really just a warning? Is it saying "processing an executable is not possible" or "just to let you know that this is a loadfile, not an object file, but I can still proceed"?

Would it be possible to modify the linker script to work from section types rather than names to make this work?

Again, I really don't *need* this, just thought it would be nice to use the linker to do the job rather than perhaps considering a custom tool at some point.

[phx]

Originally Posted by hop

I thought it might be possible to use vlink to convert a hunk executable into rawseg format

It is. Although with some restrictions.

This is really just an exercise, but I wondered if this *should* be possible, or if there is not enough information in the hunk exe to allow this process.

You can use vlink to convert from one format into another indeed, similar as linking different object file formats is no problem. But there is not so much information in a hunk-format executable.

Is Warning 12: "loadfile" is already an executable file. really just a warning?

Yes. It just tells you that linking executables is unsual, but it will try.

Would it be possible to modify the linker script to work from section types rather than names to make this work?

The problem is that a linker script works with file-name and section-name matching. A hunk-format executable has no section names anymore, so vlink's amigahunk loader assigns a unique name of the form "S12345678" to every section it finds. This means, you can at most try to match them with

*(S*)

in your script, which merges all sections from the executable into a single output section. Not really what you want, I guess.

Maybe I should improve that section naming, by including the type into the unique name, like "CODE1234", "DATA1234", etc.? I can check that.

[hop]

This sounds very encouraging thanks.

Would if be enough for the generated section name to contain the initial hunk type (CODE, DATA, BSS) and the memory flags (ANY, FAST, CHIP, CUSTOM)

For example: "CODE_ANY_0", "DATA_CHIP_0", "DATA_CHIP_1", "BSS_ANY_0", "DATA_CUSTOM7890ABCD_0"

Then the .ld section name wildcards should be able to match.

[phx]

You're absolutely right! The memory flags should be included into the auto-generated name, because this is the only way a linker script could see them.

The solution, which I just committed (available with tomorrow's daily source snapshot), is to use "code", "data" and "bss" for the type. Followed by an optional memory attribute "chip" or "fast". Which may be followed by an index, starting with "0001", when there are more than one section of a type.

BTW, yesterday I forgot that you can already assign names for different section types with the current version of vlink, when using the

-fixunnamed

option. Disadvantage is that multiple sections of the same type would then be merged, and memory flags are still missing (also fixed with tomorrow's update).

[hop]

That's amazing thanks. I'll look forward to trying this out.

[hooverphonique]

Originally Posted by phx

The solution, which I just committed (available with tomorrow's daily source snapshot), is to use "code", "data" and "bss" for the type. Followed by an optional memory attribute "chip" or "fast". Which may be followed by an index, starting with "0001", when there are more than one section of a type.

Would it not be easier to pattern match these if the memory attribute is not optional, but set to e.g. "ANY" (as suggested by hop) if no attributes are specified in the hunk? This way the format would always be something like HUNKTYPE_MEMATTR_INDEX.

[hop]

Originally Posted by phx

The solution, which I just committed (available with tomorrow's daily source snapshot), is to use "code", "data" and "bss" for the type. Followed by an optional memory attribute "chip" or "fast". Which may be followed by an index, starting with "0001", when there are more than one section of a type.

Many thanks for adding this feature. I've grabbed today's snapshot and it seems to be working great. There may be one issue related to the index suffix, but I'm not sure (see end of post for details).

Updated linker script:

PHDRS {
    fast PT_LOAD;
    chip PT_LOAD;
}

SECTIONS {
    . = 0x400;
    .chip: {
        *(*chip)
        . = ALIGN(2);
        _FREECHIP = .;
    } :chip

    . = 0x100000;
    .text: { *(code*) } :fast
    .data: { *(data*) }
    .sdata: {
        _LinkerDB = . + 0x7ffe;
        _SDA_BASE_ = . + 0x7ffe;
        *(.sdata __MERGED)
    }
    .bss: { *(bss*) }
}

I have a test loadfile, which is contains pretty standard relocs from the code "fast" segment to both "fast" and merged "chip" segments. I added artificial sections of every type for testing (code,data,bss)x(any,fast,chip)

	SECTION code,code  ; name: "code", sec_type: code, mem_type: not specified (either chip or fast will do)

	movea.l #code_red,a0
	movea.l #data_green,a1

	move.w #$00f,bss_blue
	movea.l #bss_blue,a2

	bsr MainLoop

code_red:
	dc.w $f00    ; red

;-------------------------------------------------------------------------------

	SECTION code,code  ; name: "code", sec_type: code, mem_type: not specified (either chip or fast will do)

	; Flash COLOR00 with three colours
	; Please excuse software delay loops - just want to keep the code simple!
MainLoop:
	move.w #$ffff,d1
.loop1	move.w (a0),d0
	move.w	d0,$dff180   ; COLOR00
	dbf d1,.loop1

	move.w #$ffff,d1
.loop2	move.w (a1),d0
	move.w	d0,$dff180   ; COLOR00
	dbf d1,.loop2

	move.w #$ffff,d1
.loop3	move.w (a2),d0
	move.w	d0,$dff180   ; COLOR00
	dbf d1,.loop3

	moveq #0,d0          ; black
	move.w d0,$dff180

	bra MainLoop

;-------------------------------------------------------------------------------
; Dummy code Chip section
	SECTION code_c,code,chip
	rts

; TODO: Try adding another code chip section with custom name e.g. SECTION MyCodeChip,CODE,CHIP

;-------------------------------------------------------------------------------
; Dummy code Fast section
	SECTION code_f,code,fast
	rts

;-------------------------------------------------------------------------------
; Dummy data "any" section
	SECTION data,data
	dc.l 0

;-------------------------------------------------------------------------------
; Data Chip RAM section
;
; Usually there will be no relocs for the Chip-segment, because it doesn't contain any code.

	SECTION data_c,data,chip

	dc.w 0       ; dummy to give next label non-zero offset for testing relocation code
data_green:
	dc.w $0f0    ; green

;-------------------------------------------------------------------------------
; Dummy data Fast section
	SECTION data_f,data,fast
	dc.l 0

;-------------------------------------------------------------------------------
; Dummy bss "any" section
	SECTION bss,bss
	ds.l 1

;-------------------------------------------------------------------------------
; BSS Chip RAM section
; The linker script will merge this with the preceding Data Chip RAM section into a single segment called "chip"

	SECTION bss_c,bss,chip

	; dummy data to give next label non-zero offset for testing relocation code
	ds.l 3
	ds.w 1       
	
bss_blue:
	ds.w 1    ; 1 word in which to store blue

;-------------------------------------------------------------------------------
; Dummy bss fast section
	SECTION bss_f,bss,fast
	ds.l 1

Build script:

SETLOCAL
SET loadfile=hunk_to_rawseg_loadfile
SET loader=hunk_to_rawseg_bootblock_loader

REM Create intermediates folder
IF NOT EXIST intermediates (
	mkdir intermediates
	ECHO Created intermediates folder
)

vasmm68k_mot -Fhunkexe -kick1hunks -no-opt -nosym -wfail -o intermediates\%loadfile% %loadfile%.asm || EXIT /B 1

ira -a -info intermediates\%loadfile% intermediates\%loadfile%_ira.asm

REM Would be great to pass -wfail to vlink here!
vlink -brawseg -q -T%loadfile%.ld -nowarn=12 -o intermediates\%loadfile%.rawseg intermediates\%loadfile% || EXIT /B 1

REM Build the bootblock loader binary which INCBINs the rawbin1 file
vasmm68k_mot -I%NDK%\Include_I -Iintermediates -I%build% -Fbin -no-opt -o %Build%/%loader%.bin -L intermediates/%loader%.list -wfail %loader%.asm || EXIT /B 1

BuildADF %Build%\%loader%.bin %Build%\%loader%.adf 
EXIT /B 0

It is great to be able to move hunk processing from the runtime into an offline build step.

It is far more flexibile not having to stick to a naming convention for sections e.g. "chipmem" from original linker script. Can now just revert to using arbitrary names SECTION bss_c,bss,chip or SECTION data_c,data,chip. Should make large projects easier to manage. (I thought it wasn't working at first, but embarssingly I thought that syntax such as SECTION data_c implied SECTION data_c,data_c).

The relinking command in my build script now uses -nowarn=12. Would it be possible to add a vlink -wfail option, like vasm? I was debugging stale binaries at one point because the build failed and didn't see the linker warnings in the middle of the output.

The possible issue: Why is it that the numbers are only appended to the generated section names for chip or fast mem?. I deliberately broke my linker script [to see the autogenerated names] and this was the output:

Warning 64: Section hunk_to_rawseg_loadfile(code) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(codechip0001) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(codefast0002) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(data) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(datachip0001) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(datafast0002) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(bss) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(bsschip0001) was not recognized by target linker script.
Warning 64: Section hunk_to_rawseg_loadfile(bssfast0002) was not recognized by target linker script.

I still want to:
- Test that merging sections of same type but different name works as expected
- Test with some 3rd party executables.

[hop]

Spotted a bug in the linker script. It needed a * after the "chip" as well in the chip section so it reads:

*(*chip*)

. Without this, it was not pulling in sections with generated names like "codechip0001"

PHDRS {
    fast PT_LOAD;
    chip PT_LOAD;
}

SECTIONS {
    . = 0x400;
    .chip: {
        *(*chip*)
        . = ALIGN(2);
        _FREECHIP = .;
    } :chip

    . = 0x100000;
    .text: { *(code*) } :fast
    .data: { *(data*) }
    .sdata: {
        _LinkerDB = . + 0x7ffe;
        _SDA_BASE_ = . + 0x7ffe;
        *(.sdata __MERGED)
    }
    .bss: { *(bss*) }
}

However this now results in:

Warning 22: Attributes of section .chip were changed from r-x- to rwx- in hunk_to_rawseg_loadfile.

[phx]

Originally Posted by hooverphonique

Would it not be easier to pattern match these if the memory attribute is not optional, but set to e.g. "ANY" (as suggested by hop) if no attributes are specified in the hunk? This way the format would always be something like HUNKTYPE_MEMATTR_INDEX.

Perhaps. My intention was to make the section names appear as natural as possible, when there are executables with only a single code, data or bss section (which is the most common case). So I start appending an index with the first duplicate.
I even had the GNU-tools in mind, which may use section names like .datachip to detect memory attributes. I tried to follow this syntax as well.

Originally Posted by hop

The relinking command in my build script now uses -nowarn=12. Would it be possible to add a vlink -wfail option, like vasm?

Could be done. I check that.

The possible issue: Why is it that the numbers are only appended to the generated section names for chip or fast mem?

Refer to my explanation on top. In your case "codechip0001" is the second code-section found in the executable, so it gets the 0001 index. The first section of a type remains without indexing.

But your and hooverphonique's comments and irritations make me rethink that naming. Maybe I should change it.

Originally Posted by hop

Warning 22: Attributes of section .chip were changed from r-x- to rwx- in hunk_to_rawseg_loadfile.

That's because you merged a code and a data section in the ".chip" output section. Merging code and data may not always be intended and should receive a warning.