Correct executable hunk parser for 1.3/3.1?

[Photon]

I don't need the other OS versions, although 1.4 is necessary for A3000 and I want to support 3.1.4. Just too long for title field.

DOS LoadSeg works fine, but I'm looking out for pitfalls from experience.

Some executable files have multiple hunks, decompression etc. The last hunk might have the last 0 longword (IIRC) removed to save 4 bytes, and they still work. In general: all the support needed as if typed from CLI.

[Thomas Richter]

So, what's the question? Actually, most programs have more than one hunk. You find the hunk format defined in the Bantam book.

Things to look out for? Debug information? Overlays?

[Photon]

Originally Posted by Thomas Richter

So, what's the question? Actually, most programs have more than one hunk. You find the hunk format defined in the Bantam book.

Things to look out for? Debug information? Overlays?

I could answer every coding question with a book title and consider myself done

I don't like to do that though, and here, there are compatibility considerations normally not addressed at all in books.

[Thomas Richter]

Originally Posted by Photon

I don't like to do that though, and here, there are compatibility considerations normally not addressed at all in books.

I'm sorry - what don't you want to do? Read a book? Why not? What is the problem you want to solve? The only thing the Bantam manual does not state is that the scatter loader supports both 1015 and 1020 for short relocs, due to a mistake in earlier releases. That is inofficial. If you want to look into the hunk structure, there is for example the Hunk processor in Aminet.

[Photon]

Originally Posted by Thomas Richter

I'm sorry - what don't you want to do? Read a book? Why not? What is the problem you want to solve? The only thing the Bantam manual does not state is that the scatter loader supports both 1015 and 1020 for short relocs, due to a mistake in earlier releases. That is inofficial. If you want to look into the hunk structure, there is for example the Hunk processor in Aminet.

Experienced coders like me will understand exactly what I'm trying to do (which is perfectly summarized in the question title). If you don't, you shouldn't say "read a book".

Imagine if someone answered every hardware coding question with, "just read the Hardware Reference Manual/68000 Instruction Manual".

[Bruce Abbott]

Originally Posted by Photon

Experienced coders like me will understand exactly what I'm trying to do (which is perfectly summarized in the question title).

I consider myself to be an experienced coder, but obviously not like you because I don't understand exactly what you are trying to do.

You say 'DOS LoadSeg works fine' and 'all the support needed as if typed from CLI' but don't explain why you need a 'correct' hunk parser. Exactly what are you trying to do?

[Photon]

Trying to not reinvent the wheel, but apparently no-one has a hunk parser to share. It's fine, I'll set up a bunch of exes to test with and write my own.

[Thomas Richter]

Once again: What exactly are you trying to achieve? You may not know, but LoadSeg() has a customizable cousin, InternalLoadSeg(), so you can make it load from memory, or from an RDSK block, or whatever... It is in general not necessary to write a scatter loader yourself.

For a debugger, you may need to look for the debug hunks and their formats, but that is yet again another business. Nobody will be able to help you if you don't tell what your goal is.

Just to say, I cannot educate you. If you are looking for a documentation of the hunk format, books are much better options than me typing in a book chapter. The best source is Ralph Babel's Guru book, with that not being available, the second best source is the Bantam manual.

[mr.spiv]

Originally Posted by Photon

Trying to not reinvent the wheel, but apparently no-one has a hunk parser to share. It's fine, I'll set up a bunch of exes to test with and write my own.

Maybe http://aminet.net/package/dev/moni/HunkFunc

[Thomas Richter]

Yes, or that: http://aminet.net/dev/misc/Hunk.lha

[jotd]

osemu has a good hunk parser. actually it emulates loadseg pretty well. including overlays. 100% assembly.

[Thomas Richter]

Guess what. The dos.library InternalLoadSeg() / LoadSeg() is also assembler (since v37, before it was BCPL), and it even uses buffering to limit the number of Read() calls. (-;

[jotd]

Thomas can you post something without being patronizing and selling the OS?

It's obvious that Photon doesn't want that solution. Besides do we have the source so we can hack that? yes and no. We have the source but it's not legal to use it.

osemu dos.s source is a free and easy to build & hack into assembler source.

[Thomas Richter]

I'm not selling anything, and I'm not developing anything of it. It seems you seem to advocate another source "because it's assembler". However, leaving language issues all aside, the more important reason for *not* using such a solution is that a) somebody else (not me) already did that, and b) this somebody else's solution is working and the standard and maintained solution for this problem; it defines how this problem is to be solved.

Now, it might well be that LoadSeg() and friends are not adequate, but then it would be necessary to know why not (and there is no information on this). If there is a slim chance that the Os function works for you, then that is the best possibility you have. Somebody else (not me) cares about it, it is always present, and it defines the standard for the problem of "loading hunks".

Maybe the problem isn't "loading hunks", which is also certainly possible, but then it becomes necessary to state what the problem is to provide adequate help.

[Photon]

Let's narrow it down. 1.3 is mentioned specifically because some resources (and much of Aminet) is only available from OS 2.0 and up, so what is best practice to handle 1.3-1.4?

The goal is to load an executable into memory and execute it correctly. There could be thoughts on whether to load certain types of hunks or not, as well as handling hunks seen as malformed by a certain OS version.

In other words, each OS has code that only has to work for that OS version, but my execution command must have code that works on 1.3-3.1 and be self-contained. I will be writing it in Assembler, but pseudocode or a flow chart is fine too.

This is only for executable object files, not scripts.

[jotd]

i will dig osemu loadseg implementation. it cerainly works for 1.3

[Thomas Richter]

Originally Posted by Photon

Let's narrow it down. 1.3 is mentioned specifically because some resources (and much of Aminet) is only available from OS 2.0 and up, so what is best practice to handle 1.3-1.4?

LoadSeg, of course, if your goal is to load a binary into memory.

Originally Posted by Photon

The goal is to load an executable into memory and execute it correctly. There could be thoughts on whether to load certain types of hunks or not, as well as handling hunks seen as malformed by a certain OS version.

There isn't really much that changed. The only hunk that came into v37 is HUNK_RELOC32SHORT, but that's rarely used. The SAS/C linkers such as SLINK do not generate it. Later LoadSeg also clears the bytes allocated in the hunk header, but not accounted in the hunk body. Compilers generate code, however, to clean these bytes.

Originally Posted by Photon

In other words, each OS has code that only has to work for that OS version, but my execution command must have code that works on 1.3-3.1 and be self-contained.

No, that's a completely wrong interpretation of the story. LoadSeg v37 will perfectly load older hunks, and I have yet to see a binary that is intended to work on 1.3, but does not load under 1.3

Originally Posted by Photon

. I will be writing it in Assembler, but pseudocode or a flow chart is fine too.

 move.l DOSBase,a6 jsr _LVOLoadSeg(a6)

That is it. No need to write anything yourself. Chances are that you just create bugs, and the Os scatter loader is stable. Really. Overlays, for example, are typically forgotten, but more important under 1.3 than under 2.x.

[Thomas Richter]

Originally Posted by jotd

i will dig osemu loadseg implementation. it cerainly works for 1.3

Just a quick look at it - I'm sorry, but no. No overlay support, and it does not accept both equivalent hunk types for RELOC32SHORT. Once again - if there is a system function which does what you need, use it. It makes no sense to roll it on your own, chances are better than even that you overlook something.

[phx]

Originally Posted by Thomas Richter

The only hunk that came into v37 is HUNK_RELOC32SHORT, but that's rarely used.

No, there is more. Like HUNK_RELRELOC32 (and HUNK_ABSRELOC16). The last one makes no sense to me and I admit that RELRELOC32 is rarely used - atthough I did and vlink supports it. But RELOC32SHORT is used quite frequently! For example all programs generated by vbcc use it (as long as you don't specify a kick13 config).

Later LoadSeg also clears the bytes allocated in the hunk header, but not accounted in the hunk body.

Yes, that's one of the more important differences since OS2.0.

Compilers generate code, however, to clean these bytes.

Photon certainly wants a general solution, so he cannot rely on that. There are enough programs which don't (because their authors are lacking experience, or don't care about 1.x).

No, that's a completely wrong interpretation of the story. LoadSeg v37 will perfectly load older hunks, and I have yet to see a binary that is intended to work on 1.3, but does not load under 1.3

As far as I understand him, he wants to create a program which can run under any AmigaOS version and is able to load any existing program. LoadSeg doesn't help him under Kickstart 1.x.

Originally Posted by Photon

In other words, each OS has code that only has to work for that OS version, but my execution command must have code that works on 1.3-3.1 and be self-contained.

Unlike object files there are not so many hunk types which can appear in an executable. Implementation is trivial, but a bit of work to support everything. Especially if you want to include Overlays.

What do you need? The documentation of all hunk structures?

[Thomas Richter]

Originally Posted by phx

Photon certainly wants a general solution, so he cannot rely on that.

LoadSeg() *is* the general solution - for loading segments at least (not for other operations on hunks, of course). Name a program that is supposed to run on 1.3 and uses HUNK_RELOC32SHORT. Chances are that this "general solution" is more flawed that the operating system. Look at the proposed solution above - there are problems in it LoadSeg() gets right.

Originally Posted by phx

There are enough programs which don't (because their authors are lacking experience, or don't care about 1.x).

Then if they are not designed to run under 1.x, why load them under 1.x? Chances are better than even that they fail (or crash) because they are compiled for a newer Os.

Originally Posted by phx

As far as I understand him, he wants to create a program which can run under any AmigaOS version and is able to load any existing program. LoadSeg doesn't help him under Kickstart 1.x.

Of course it does. It will load under 1.x any program that will work under 1.x. And it will fail under 1.x for some programs that are not designed to work under 1.x, which does not do any harm.