VASM/VLINK relocation issues

[pipper]

I'm trying to get Breathless compiled with vasm/vlink contained in Bebbo's toolchain.
https://github.com/mheyer32/Breathless

I'm pretty close to get it linked, but encounter multiple issues of this type during linking:

Error 32: Target amigahunk: Unsupported relocation type R_ABS (offset=0, size=16, mask=ffffffffffffffff) at .text+0x10402.
Error 32: Target amigahunk: Unsupported relocation type R_ABS (offset=0, size=16, mask=ffffffffffffffff) at .text+0x10498.
Error 32: Target amigahunk: Unsupported relocation type R_ABS (offset=0, size=16, mask=ffffffffffffffff) at .text+0x1052e.
Error 32: Target amigahunk: Unsupported relocation type R_ABS (offset=0, size=16, mask=ffffffffffffffff) at .text+0x105cc.

I traced these back to code of this nature:

move.l	Yoffset(a5,d7.w*4),a1  
or
move.l	Yoffset(a5,d7.w*4),a1

and also

tst.w	pause(a1)

The way Breathless structures its data is
CODE (i.e. .text) contains code and constant tables
__MERGED,bss contains runtime variables, base-relative addressed through a5 ("near data")
TABLES,bss contains runtime data too large for __MERGED ("far data")

In the code above, Yoffset and pause are xref'd / xdef'd in separate files. Both are placed into __MERGED.

The lines of nature 'tst.w pause(a1)' that the linker complains about are referencing __MERGED through a1.

I don't really understand what the issue is, I suspect a bug in vasm/vlink maybe?

There's also some other bug I ran into: The linker placed _LinkerDB into the last bss section it
encounters, not always __MERGED. So if TABLES is the last new bss section it runs into, _LinkerDB will be placed into TABLES, which won't work. I had the impression, the linker knows about __MERGED and would treat it accordingly?

And finally, the assembler would complain about illegal relocations when doing this:

lea     ObjectsPunList-4(a5),a1

I think this should be legal to do. I helped myself by placing another symbol right in front of ObjectsPunList and refer to that instead. But It would be nice if that wasn't necessary in first place.

[phx]

Originally Posted by pipper

Error 32: Target amigahunk: Unsupported  relocation type R_ABS (offset=0, size=16, mask=ffffffffffffffff) at  .text+0x10402.

Amiga-Hunk executables only support 32-bit absolute relocations, not 16-bit.

Any 16-bit absolute references should be constants (equates) and will be resolved by the linker. 16-bit address (label) references can only happen with small data, but that would be an R_SD reloc type, not R_ABS.

move.l    Yoffset(a5,d7.w*4),a1

When Yoffset is unknown, my vasm would default to an 8-bit displacement, causing an 8-bit R_ABS xref here. You would have to force 16-bit with "Yoffset.w" to get the error message from above.

__MERGED,bss contains runtime variables, base-relative addressed through a5 ("near data")

Ok. So I assume that you added "near a5" in the source, so vasm can detect base-relative addressing modes?

That doesn't help you with

Yoffset(a5,d7.w)

, though, because only d16(An) addressing modes are recognized for small data.

BTW, I would be interested to know which assembler was originally used for Breathless? Did it really create small-data references for these lines?

The lines of nature 'tst.w pause(a1)' that the linker complains about are referencing __MERGED through a1.

You mean the "unsupported relocation" error from above? Or does it complain about something different?

Theoretically you could fix that, because it is a 16-bit base register displacement addressing mode. When it would be

pause(a2)

you only need

near a2

on top of it to make vasm generate the correct relocation type. But unfortunately only A2 - A6 are allowed for the NEAR directive, because I was under the impression that a base register which is volatile (according to the m68k C-ABI), like A0 and A1, makes no sense.

Now I would agree that such an artificial restriction is stupid. If you want to use A1 as your base register temporarily, why should I stop you? Will change that. It's an easy fix.

I don't really understand what the issue is, I suspect a bug in vasm/vlink maybe?

Debatable. We have three issues:

1. You have to use the NEAR directive to specify the current base register in use. When it changes (e.g. pause(a1)), tell vasm about it. And don't forget to restore the original base register thereafter.
2. Only 16-bit base-register displacement addressing modes are recognized for small data. So Yoffset(a5,d7.w*4) will never work, even with "near a5". Although technically possible, I don't think that small data was intended for any other addressing mode and there is no compiler which would generate such code. If you would allow that, where does it stop? Would you also allow indirect addressing modes?
3. NEAR is not allowed for a0 and a1. You may call that a bug, but it's really only an artificial restriction which can be lifted (will do).

Issue 1 and 2 are the reason that vasm didn't process some lines in your source as small-data and didn't generate the required R_SD reloc type, but R_ABS instead. Which finally caused the errors in vlink.

There's also some other bug I ran into: The linker placed _LinkerDB into the last bss section it
encounters, not always __MERGED.

Maybe there is a small-data reference to that last bss section, which confuses the linker? The base section for _LinkerDB is the first section found, which has small-data references to it.

I had the impression, the linker knows about __MERGED and would treat it accordingly?

Yes, it knows about __MERGED and will automatically merge all sections with this name, even when the type differs (data/bss). But you could still have your small data references on another section.

lea     ObjectsPunList-4(a5),a1

I think this should be legal to do.

Absolutely. That's a bug! Will check that. Thanks.

[pipper]

Hi,
Thanks for your repsonse!

I encourage you to checkout the linked repository and try 'make' in the 'Sorgenti' directory. The project structure is simple and will help you to understand whats going on.

Amiga-Hunk executables only support 32-bit absolute relocations, not 16-bit.

The the lines that the linker complains about, are all reference the small data section __MERGED through the indicated base register a5.
So I think the assembler is not properly recognizing that this is a 16bit relative relocation and wrongly emits a 16bit abslolute relocation.

Ok. So I assume that you added "near a5" in the source, so vasm can detect base-relative addressing modes?

I invoke the assembler with these flags:

AFLAGS = -Fhunk -phxass -sd -sdreg=5 -linedebug

in addition I put 'near a5' into 'System' (which is included by most of the files).

That doesn't help you with Yoffset(a5,d7.w), though, because only d16(An) addressing modes are recognized for small data.

Can you make it support this addressing mode? I don't see why it shouldn't work.

BTW, I would be interested to know which assembler was originally used for Breathless? Did it really create small-data references for these lines?

I don't know. There's a file 'TMap.lnk' in there containing the folowing lines:

FROM TMap.o+TMapMain.o+DrawScreen.o+c2p8.o+3d.o+Interrupt.o+movement.o+sincos.o+Text.o+Animations.o+Objects.o+Scores.o+devices.o+Loader.o+FileAccess.o+Terminal.o+Audio.o+Map.o+SecurityCode1.o+Player/Music.o+Presentation.o+Last.o
LIB Work:Aztec5/lib/libs/amiga.lib
TO TMap
ADDSYM
MAP TMap.map f,h

So my suspicion is that Aztec C (did it come with an assembler?) was used?!?

When it changes (e.g. pause(a1)), tell vasm about it. And don't forget to restore the original base register thereafter.

This happens only for a few Audio related interrupt routines. I suspect the original programmers didn't want to save/restore a5 and move.l a1,a5 just to comply with the a5 register.
What I did instead is to embrace the code in question with

		basereg $7FFE,a1

....
                tst.w	pause(a1)
....
                endb

'near a1' didn't do as it wouldn't allow me to use a1 - as you noted.
But it would still complain about the exact same code.

Maybe there is a small-data reference to that last bss section, which confuses the linker? The base section for _LinkerDB is the first section found, which has small-data references to it.

No, I think there's something afoot with the linker, because I was able to work around this issue by just forcing the linker to encounter section names in a different order.
(see https://github.com/mheyer32/Breathle...76c6b7304R1155 )
This way __MERGED is the last unique section name the linker will see and this way it placed _LinkerDB into __MERGED.

[phx]

Originally Posted by pipper

I encourage you to checkout the linked repository and try 'make' in the 'Sorgenti' directory.

It doesn't easily build on my system, without doing many modifications.

I invoke the assembler with these flags:

AFLAGS = -Fhunk -phxass -sd -sdreg=5 -linedebug

-sdreg=5 is ok. It's the same like having a "near a5" on top the source.

Can you make it support this addressing mode? I don't see why it shouldn't work.

Using extended 020 addressing modes for small data is at least very unusual. I didn't see that a lot, before your example.
Ok. I added support for it through the new command line option

-extsd

. The option is necessary for not breaking compatibility with older sources.

You find that option together with support for "near a1" in tomorrow's daily source snapshot of vasm.

So my suspicion is that Aztec C (did it come with an assembler?) was used?!?

Yes, every compiler comes with an assembler, but AFAIK the Aztec-assembler didn't support the usual Motorola/Devpac directives (at least in my Aztec-C 3.4 from 1988).

What I did instead is to embrace the code in question with

        basereg $7FFE,a1
....
                tst.w    pause(a1)
....
                endb

Yes, that cannot work for several reasons, and will generate a 16-bit absolute relocation type when "pause" is unknown. You need "near a1" for this line. basereg is not for linkable objects and only makes sense for single source files (the directive comes from AsmOne).

No, I think there's something afoot with the linker, because I was able to work around this issue by just forcing the linker to encounter section names in a different order.

I can have a look at this issue, if you want to send me all the object files, and the linker options used, by email.

     lea     ObjectsPunList-4(a5),a1

I think this should be legal to do.

I checked the vasm source this morning and remembered why I cannot do that.

You certainly want to use the full 64K range of the small data segment? Not just 32K. This means that the addend, used in a small-data relocation, has to be unsigned. Sorry, there is nothing I can or will do. When any other assembler supports it, then it either does no range error checks at all or only supports 32K small data segments.

[pipper]

It doesn't easily build on my system, without doing many modifications.

I should have mentioned that the Makefile was setup to run on Linux with Bebbo's Toolchain

Thanks for adding -extsd! I wanted to try it out, but apparently this modification did not end up in the daily source snapshot :-(

'near a1', though, worked.

I can have a look at this issue, if you want to send me all the object files, and the linker options used, by email.

It would be easier if you tried at your end to compile the project so you can toy around with options. But I can also prepare an archive with the .o files.

You certainly want to use the full 64K range of the small data segment? Not just 32K. This means that the addend, used in a small-data relocation, has to be unsigned.

Can you elaborate on that? I don't see how this is an issue if we assume 16bit wraparound.
The code here doesn't really want to address ObjectsPunList at offset -4, but I think it'll apply some sort of preincrement before accessing the data.

There's some other examples where the code attempts to obfuscate accessing the same symbol multiple times:

		xref	SecCodeRow,SecCodeCol

		lea	SecCodeRow-$228(a5),a1
		moveq	#36,d1
		jsr	Rnd
		addq.w	#1,d0
		move.w	d0,a0
		move.l	a0,$228(a1)		;SecCodeRow
		moveq	#10,d1
		jsr	Rnd
		add.w	#65,d0
		move.w	d0,a0
		move.l	a0,$228+4(a1)		;SecCodeCol

But as you can see, it'll eventually offset the original lea SecCodeRow-$228(a5),a1 by 228.
Luckily I can disable all of these places as they are copy protection related anyways

[phx]

Originally Posted by pipper

Thanks for adding -extsd! I wanted to try it out, but apparently this modification did not end up in the daily source snapshot :-(

Snapshots are done around 02:00 UTC, so you have been too early. The "near" fix was already checked in a day before.

It would be easier if you tried at your end to compile the project so you can toy around with options.

Last time I gave up after I did some modifications (fixing the script which expects bin/bash, changing include paths, etc.), when it tried to include from "lvo", which is not part of the standard SDK. But seems that was the last problem, after I got those from Devpac.

Yes, I see the vlink errors. The 8-bit R_ABS references are all due to (Yoffset,a5,d7.w) where vasm defaults to 8-bit, when Yoffset is unknown. You can fix that by adding a .w suffix to Yoffset, so -extsd will work with it. Then only a few errors are left which originate from Audio.asm, but there is no Yoffset in it.

Can you elaborate on that? I don't see how this is an issue if we assume 16bit wraparound.

I admit this is also a home-made problem. vasm and vlink don't differentiate internally between a relocation table and an external reference table, like the AmigaDOS hunk format does. Both are relocs for them (like in the ELF format). But with external references the base-symbol is unknown.

When you have small-data relocs, i.e. the referenced symbol is known, the addend/offset stored with it must be unsigned, because the AmigaDos HUNK_DREL16 relocation table expects an offset to the start of the object's small data section. This is usually not a problem, unless you use a negative displacement on the first symbol in this section.

With small-data external references I agree that a signed 16-bit addend would make sense, but vlink reads the R_SD-addends in both cases as unsigned to put them in the same internal reloc table.

Maybe I should tackle this problem now. I ignored it for too long. Will have a look how much effort it takes to pass a flag from the hunk-format loader module down to the core routines...

[pipper]

I just tested the latest vasm. Only two issues are left, which seem to have every imaginable complication in them:

move.l	Yoffset.w+4(a5,d1.w*4),a1

I guess, the +4 is throwing things off(?). I'll check if I can workaround it.

Thank you for helping out! Very much appreciated!

[phx]

There are problems with your syntax.

Yoffset.w+4

is not correct, because the +4 should be part of the displacement, which is terminated by the .w suffix.

Yoffset+4.w

will work, although I would recommend to use the newstyle syntax when working with 020+ addressing modes. With

((Yoffset+4).w,d1.w*4)

you can also use parentheses, which makes it clearer.

Update: I probably managed to make the addends in external small-data references signed, while keeping the relocs unsigned. To try it you have to update vasm and vlink with the next source snapshot (in 12 hours).

So only the issue with _LinkerDB pointing to a wrong section remains. How do I reproduce that?

[pipper]

So only the issue with _LinkerDB pointing to a wrong section remains. How do I reproduce that?

This is the change that introduced the workaround for it:

https://github.com/mheyer32/Breathle...76c6b7304R1155


Removing

section TABLES,bss

xdef stupid
stupid ds.l 1

Should revert back to the original issue.

[phx]

Yes, got it! And fixed. See next vlink snapshot.

It was just a display problem. The "Linker symbols:" header was missing in the map file, so it looked like _LinkerDB belongs to the last section listed. The generated executable was certainly identical in both cases.

[defor]

I'm sorry if I'm hijacking this thread but I didn't want to create yet another vasm one.

First of all I must thank phx for his work on vasm -- such a great tool!

My problem: Currently I'm working on a quite small project consisting of a few .s files, assembling and linking them together using vasm/vlink (through Amiga-assembly plugin in VSCode). Everything works flawlessly, however when something is wrong it is most probably in a linking phase when linker reports about unsupported relocation (hence why I'm using this thread). Usually I know why it happens yet it is sometimes very hard to figure out the exact line in the code which uses addressing mode which linker has problems with.
For example this error line:
Error 32: Target amigahunk: Unsupported relocation type R_PC (offset=0, size=16, mask=ffffffffffffffff) at Program+0x27de
My guess is that offset exceeds 16bit signed boundary, however it is hard to find which line of code does it since Program section is quite large and offset 0x27de doesn't say much to me.
Is there a way how I could be informed about the line of code which violates restriction or about a symbol which is causing it?
Maybe the answer is obvious?
Thank you.

[pipper]

In my case I was lucky, vlink had already produced an output file. I used objdump to get a disassembly of the whole thing which gave me the offset plus the corresponding assembly. The offending offset will not directly show up as “line number” as the relocation is part of an instruction, but you’ll get close. Use this as reference in the original sources to lookup the actual offending instruction. Maybe there’s a more elegant way. But this worked for me.

[defor]

Originally Posted by pipper

In my case I was lucky, vlink had already produced an output file. I used objdump to get a disassembly of the whole thing which gave me the offset plus the corresponding assembly. The offending offset will not directly show up as “line number” as the relocation is part of an instruction, but you’ll get close. Use this as reference in the original sources to lookup the actual offending instruction. Maybe there’s a more elegant way. But this worked for me.

Thank you for your tip!
I tried to use objdump to disassemble and to find a generated code which can't be relocated, according to your description. However vasm/vlink package comes with vobjdump which can parse VOBJ format only. In my case, vlink produces a hunk executable from .o files which are not in VOBJ format (by taking a look at vobjdump source code, VOBJ file must begin with a series of magic numbers which my .o files do not...). So this is currently the dead end for me.
However your example led me to another way which was successful:
Step #1:

vlink can produce a mapping file ("-M" parameter). There I could see at which offsets all code sections start. By comparing offset from a vlink error message (e.g. ....at Program+0x636), I could find which code section the error is at and compute a relative offset to the instruction byte.

Step #2:

vasm can produce a listing file ("-L" parameter). In there, I could see what instruction is at previously computed offset. And voila -- that's my culprit!


Yes, it is tedious. Most probably I am missing something. Some obvious way how to find which instruction in my source code the linker can't relocate.


P.S.: Can debug info (in .o file) help somehow?

[pipper]

I used objdump from Bebbo’s GCC. It can deal with hunk object files (-Fhunk vasm Option).
But it requires the code section to have the name “.text”. Vasm uses “CODE” by default.

The linker may be taught to use debug info to infer the closest symbol and improve the error message, but that’s up to Phx to decide if that would be reasonable to implement.

[pipper]

Originally Posted by phx

Yes, got it! And fixed. See next vlink snapshot.

It was just a display problem. The "Linker symbols:" header was missing in the map file, so it looked like _LinkerDB belongs to the last section listed. The generated executable was certainly identical in both cases.

Thank you again! Very much appreciated! To be honest, I didn't even check if the executable was affected by this and just trusted the output, thinking that if the symbol indeed ended up in the wrong section, it could possibly not work anyways :-)

When do you think the recent changes will be reflected in a new release, so I can nag Bebbo to update his toolchain

Breathless now compiles and runs...sometimes. I don't know yet what is causing that. I think its an initialization issue. Does this look legitimate to clear the BSS?

GETDBASE	MACRO
		xref	_LinkerDB
		lea	_LinkerDB,a5
		ENDM



		GETDBASE 

	;*** Azzera tutte le variabili della section __MERGED,BSS

		move.l	a5,a0
		move.l	#LastBSSdata,d0
		sub.l	a5,d0
		lsr.l	#2,d0
		subq.w	#1,d0
clrbss		clr.l	(a0)+
		dbra	d0,clrbss

[phx]

Originally Posted by defor

comes with vobjdump which can parse VOBJ format only.

VOBJ is a vasm/vlink internal object file format, which can be used when no other sane object file format exists. It is generally available for all architectures.

For the other object file formats I expect that you have some disassembly tool available. Most reassemblers can also disassemble object files.

vlink can produce a mapping file ("-M" parameter). There I could see at which offsets all code sections start.

Exactly. That's the way to go!

vasm can produce a listing file ("-L" parameter). In there, I could see what instruction is at previously computed offset. And voila -- that's my culprit!

By subtracting the start offset found in the map file you can calculate the offset inside the object file and find the error location. I am usually looking at the test output of vasm (-Ftest or without any -F option), which is similar to a listing file and shows all offsets, relocations and symbols.

Yes, it is tedious. Most probably I am missing something.

No, unfortunately not. The linker just doesn't have the information to tell you anything more than a section offset, or an offset to a preceding global symbol.

P.S.: Can debug info (in .o file) help somehow?

Yes, theoretically. But it is a lot of work, because vlink has to understand all the different source level debugging formats. It already understands line-debug hunks from the Amiga hunk-format, because it has to, to be able to reproduce them. Others, like a.out-stabs and ELF-DWARF are just passed through.

[phx]

Originally Posted by pipper

When do you think the recent changes will be reflected in a new release, so I can nag Bebbo to update his toolchain

Shouldn't be too far away, and there have been lots of changes in vasm and vlink since the last releases. But because these changes it also needs some more testing, so I'm cautious.

I think its an initialization issue. Does this look legitimate to clear the BSS?

This BSS initialization code suggests that the original program expected A5 to point to the start address of the __MERGED section. So it didn't support more than 32K.

With vlink setting _LinkerDB to __MERGED+32766 I fear that your recompiled version does not really clear the BSS-area and overwrites unallocated memory after that.

You are lucky that there are no initialized __MERGED,data sections, so replacing "move.l a5,a0" by "lea -32766(a5),a0" should fix it.

Otherwise vlink, like the SAS/C linkers blink and slink, provides symbols for clearing the BSS-part of a small-data section: __BSSBAS (the start address) and __BSSLEN (length in longwords).

[defor]

Originally Posted by phx

VOBJ is a vasm/vlink internal object file format, which can be used when no other sane object file format exists. It is generally available for all architectures.

For the other object file formats I expect that you have some disassembly tool available. Most reassemblers can also disassemble object files.

Exactly. That's the way to go!

By subtracting the start offset found in the map file you can calculate the offset inside the object file and find the error location. I am usually looking at the test output of vasm (-Ftest or without any -F option), which is similar to a listing file and shows all offsets, relocations and symbols.

No, unfortunately not. The linker just doesn't have the information to tell you anything more than a section offset, or an offset to a preceding global symbol.

Yes, theoretically. But it is a lot of work, because vlink has to understand all the different source level debugging formats. It already understands line-debug hunks from the Amiga hunk-format, because it has to, to be able to reproduce them. Others, like a.out-stabs and ELF-DWARF are just passed through.

Thank you very much for your reply. I was just worried that I was missing some obvious method and doing things utterly wrong.
Fortunately, these relocation errors do not occur often and always are a product of mangling with the code by moving blocks here and there, out of 16bit range.

[redblade]

Good luck with getting this to assemble. I wouldn't mind seeing a NTSC 320x200 version to get that extra 1-2 fps on a 020.

It seems to be hard coded to PAL.

[Sneezeface81]

Sorry for dig this thread up. I manged to compile Breathless under windows using pipper's updated code and phx's latest vasm and vlink. The executable that I get is not working 100% though. It has 3 main issues:

1. no sound effects (title and ingame musics are played correctly)
2. random sky/floor rendering error
3. crashes on exit (error 0x80000006 or 0x8000000B)

if launching the game I get the sky/floor rendered properly, it crashes when returning to WB. the opposite, if sky/floor rendering is glitched no crashes when exit.
have you encountered any of these? looking at the original source code and comments, point 1 seems affected by ObjectsPunList-4, whereas point 2 takes into account Yoffset+4 several times, but it's pure speculation.

No clue how to sort this out.