Include file path priorities for assemblers

[phx]

Let's assume your shell is in the current work directory, called "cwd". When I assemble a source text in "cwd/pathto/source.asm", from where would you expect that an include-file referenced by

        include "incfile.i"

is taken?
From "cwd/incfile.i" or from "cwd/pathto/incfile.i"? Remember, the assembler is called from "cwd" and there is no include-path specified.

There was a bug in vasm's last release and I'm trying to do it correctly now.

[meynaf]

I would like it to be fetched from both positions : if first one is not found, try the other.
The order does not seem important.

[thomas]

Originally Posted by meynaf

The order does not seem important.

It is important if a file with this name is in both locations, with different contents.

[phx]

Yes, order may be important. That was also the bug in the last release. Then I fixed it by reading from the "compile-directory" only, which means "cwd/pathto" in this case. But today I noticed that I have sources which expect include files in "cwd" to be found.

So there are the following possibilities:

1. Read from cwd/pathto only.
2. Read from cwd only.
3. Read first from cwd/pathto, then from cwd when missing.
4. Read first from cwd, then from cwd/pathto when missing.

Of course, there may also be further include paths, specified by -I option or "incdir" directive, which have to be searched before or after the path from above. This is also still undecided.

EDIT: Usually I would check some other Amiga assemblers for comparison, but I'm currently on holiday and my selection of assemblers is quite limited...

[PeterK]

Usually, I use "incdir" directives in PhxAssGUI, and never had that problem yet. But I would prefer to try "pathto/" first and "cwd" only as second.

Btw, is there a rule that "pathto" has to be a subdir or relative to "cwd"? Would an absolute path cause a failure?

[meynaf]

Originally Posted by thomas

It is important if a file with this name is in both locations, with different contents.

What i meant with the order being unimportant, is that this decision in the asm isn't very sensitive. You will always find situations in which the order would have been better the other way...

Anyway, thinking about it i'd prefer current directory first, then path to the source itself. This way it becomes possible to put default files in source dir, then override them with other files from current dir (you can have as many dirs as you want, each one with a different version).

[phx]

Originally Posted by PeterK

Btw, is there a rule that "pathto" has to be a subdir or relative to "cwd"? Would an absolute path cause a failure?

An absolute path is unambigious and references one specific file. This will always work.

Usually you don't want absolute paths in your source, because you might want to assemble it on a different machine at some point.

Originally Posted by meynaf

Anyway, thinking about it i'd prefer current directory first, then path to the source itself. This way it becomes possible to put default files in source dir, then override them with other files from current dir

Strange. I would tend to prefer what PeterK said: first look into cwd/pathto, where my source is, and then look in cwd. For a similar reason as you mentioned: to override the default in cwd with source-specific headers in the source-directory.

Devpac seems to confirm that. I just found Devpac on an E-UAE file system. Devpac checks cwd/pathto first, then cwd.

PhxAss, on the other hand, only looks in cwd.

[PeterK]

Originally Posted by phx

An absolute path is unambigious and references one specific file. This will always work.

Usually you don't want absolute paths in your source, because you might want to assemble it on a different machine at some point.

The Compiler (PhxAssGUI, PhxAss) and its include files are in a directory on my SYS: partition and it seems that this is also the "CWD", but all my source code is in project drawers on another disk and all the OS includes are somewhere else with an Assign "INCLUDE:".

I think that PhxAssGUI loads my "source.asm" from an absolute project path, but without a ChangeDir (couldn't see that in SnoopDos) and so "CWD" stays in the PhxAss program directory.

I must admit, I've never written my own include files, but if I had a file "source.i" at the same location where "source.asm" was loaded from would an "include source.i" try to load it from the project drawer or from the program directory (CWD)?

Tested it: PhxAss looks into its program dir (CWD) and the /include subdir and then into INCLUDE:, but never into the project drawer where "source.asm" was loaded from. That's where I would like to see it searching first.

[phx]

Originally Posted by PeterK

The Compiler (PhxAssGUI, PhxAss) and its include files are in a directory on my SYS: partition and it seems that this is also the "CWD"

Yes. Most likely.

I think that PhxAssGUI loads my "source.asm" with an absolute path, but without a ChangeDir

Correct. The assembler should not change your CWD.

if I had a file "source.i" at the same location where "source.asm" was loaded from would an "include source.i" try to load it from the project drawer or from the program directory (CWD)?

As I mentioned in the last reply PhxAss unfortunately only looks in the CWD when you specified a relative path. So you would need an INCDIR directive or option. Devpac would do what you want.


Regarding vasm: I hope I fixed everything for the next release. Here the search priority is now:

1. Path where the source text is located.
2. Current Work Directory.
3. Paths defined by -I options on the command line.
4. Paths defined by INCDIR.

[meynaf]

Originally Posted by phx

Strange. I would tend to prefer what PeterK said: first look into cwd/pathto, where my source is, and then look in cwd. For a similar reason as you mentioned: to override the default in cwd with source-specific headers in the source-directory.

That means source dir does not contain all the necessary files to assemble the code and there is no incdir of any sort to fetch includes. Looks weird to me.
On the other hand, if you have a mysource.s using myinclude.i and both are in source dir, good luck to try another version of that include without copying files if source dir prevails.

Originally Posted by phx

Devpac seems to confirm that. I just found Devpac on an E-UAE file system. Devpac checks cwd/pathto first, then cwd.

Devpac being an integrated environment, source dir is more meaningful than current dir. Rare are people who will use it as a command-line asm.

Originally Posted by phx

PhxAss, on the other hand, only looks in cwd.

It does not need to do otherwise. PhxAss has PHXOPTIONS config file which can contain

incpath

. This does the job nicely.

[phx]

Originally Posted by meynaf

That means source dir does not contain all the necessary files to assemble the code and there is no incdir of any sort to fetch includes. Looks weird to me.

The base directory may contain some general files, and the sub-directories the specialised ones, which still include files from the base.

As you said yourself you will always find situations where one or another order seems better. So the best I can do is to follow the behaviour of other assemblers. And Devpac is often a good choice.

[PeterK]

As I understand meynaf's comment, he wants to be able to replace some sort of "sourceproperties.i" of a ported source code, which may depend on the original environment, target system or being compiler specific, by his own local preferences without editing "sourceproperties.i".

That could be a reason to look into "CWD" first, but if there is such an include file coming together with the source, will it still compile correctly if you ignore it silently and use your own substitute?

[phx]

Originally Posted by PeterK

That could be a reason to look into "CWD" first

True. But you can find these reasons for both variants.

but if there is such an include file coming together with the source, will it still compile correctly if you ignore it silently and use your own substitute?

Depends on the completeness of your substitute, I guess. I'm not sure what you want to say.

Unfortunately there is still no option to define an include path which has higher priority than the CWD or the source-directory. This can be a problem.

I just checked Devpac again, and an include path specified with INCDIR on the command line has lower priority, just like in vasm. And also in PhxAss an INCPATH option will not take priority over CWD. So... I won't change that.

[PeterK]

I'm sure you will make the right decision, although I'm still not a vasm user.

[meynaf]

Originally Posted by phx

The base directory may contain some general files, and the sub-directories the specialised ones, which still include files from the base.

Usually in this case there is some "include" directory in the tree, and some kind of incdir somewhere as well.
In addition, it is usually a good practice to avoid having files with exact same name in a project.
Anyway, if we have a naming conflict we can't resolve, source in the subdir could include subdir/inc.i instead of just inc.i if there is another inc.i in the root dir.

Originally Posted by phx

As you said yourself you will always find situations where one or another order seems better. So the best I can do is to follow the behaviour of other assemblers. And Devpac is often a good choice.

So if first one makes a mistake, all others must replicate it ?
I'd say that if the two options are valid, the answer is : make it configurable. You already know how to behave in Devpac compatibility mode. Same in Phxass mode, probably.

Originally Posted by PeterK

As I understand meynaf's comment, he wants to be able to replace some sort of "sourceproperties.i" of a ported source code, which may depend on the original environment, target system or being compiler specific, by his own local preferences without editing "sourceproperties.i".

Another example : let's say we have some program using ym2149.i (i do really have such file ). Now we write another ym2149.i using better - but untested - algorithm. We don't want to replace our old lib before this new one works fully. So we want to test it, and thus we go in the dir it is located, and attempt to assemble our program from there. Now if cwd/pathto prevails, no way we can do that without renaming or copying files around.

Originally Posted by phx

I just checked Devpac again, and an include path specified with INCDIR on the command line has lower priority, just like in vasm. And also in PhxAss an INCPATH option will not take priority over CWD. So... I won't change that.

If the option is for global, reusable includes, then it must have lower priority.
But there are probably other cases...

[PeterK]

Originally Posted by meynaf

Another example : let's say we have some program using ym2149.i (i do really have such file ). Now we write another ym2149.i using better - but untested - algorithm. We don't want to replace our old lib before this new one works fully. So we want to test it, and thus we go in the dir it is located, and attempt to assemble our program from there. Now if cwd/pathto prevails, no way we can do that without renaming or copying files around ...

I have no Yamaha sound chip for experiments here, but I prefer to make a complete backup copy of a source before I start editing it in place. Better you have a copy of the sound chip, too.

Yes, an option for the priority of the includes could be a solution.

[meynaf]

Originally Posted by PeterK

I have no Yamaha sound chip for experiments here, but I prefer to make a complete backup copy of a source before I start editing it in place. Better you have a copy of the sound chip, too.

Yes, an option for the priority of the includes could be a solution.

Actually the include file - which contains code for the emulation of said sound chip so you don't need it for experiments - is located in a path every program can access (PhxAss INCPATH in config). So all my ST ports can use same chip emulation. Testing another way just needs putting the new version in current dir.
But Vasm does not have INCPATH so the choice of priorities is perhaps more important for it - having it as an option would address all cases.

[Antiriad_UK]

Originally Posted by phx

Let's assume your shell is in the current work directory, called "cwd". When I assemble a source text in "cwd/pathto/source.asm", from where would you expect that an include-file referenced by

        include "incfile.i"

is taken?
From "cwd/incfile.i" or from "cwd/pathto/incfile.i"? Remember, the assembler is called from "cwd" and there is no include-path specified.

There was a bug in vasm's last release and I'm trying to do it correctly now.

I’d expect it to read from cwd/pathto. I.e everything is relative from the file being assembled rather than what the shell working directory is. If not found then check the folder specified in the -I option.

It gets more confusing when including a file within a file because then the nested included file only looks in the first include local folder. For example I included your pro tracker player from another file in a different folder and it couldn’t find the custom.i include.

I think my assumptions are this way due to working in C in visual studio where it works like this. But of course it could just be that that environment is changing shell folder before compiling each file.

[phx]

Originally Posted by meynaf

Testing another way just needs putting the new version in current dir.

Exactly. The current work directory, or source directory, has priority over all other include paths. That's what you would expect and that's how all assemblers work (also PhxAss and vasm).

The only difference between PhxAss and vasm is PhxAss' inability to search for include files relative to the source directory, when it doesn't match CWD.

But Vasm does not have INCPATH so the choice of priorities is perhaps more important for it

The command line option is called -I and does the same. Or are you refering to INCPATH from the PHXOPTS environment? The only advantage is that you don't have to specify it on the command line. But what are Makefiles for?

Originally Posted by Antiriad_UK

I’d expect it to read from cwd/pathto. I.e everything is relative from the file being assembled rather than what the shell working directory is.

Agreed. That's what vasm implements now.

If not found then check the folder specified in the -I option.

Yes. But only after checking all paths relative to the CWD (in case relative to the source-directory fails).

It gets more confusing when including a file within a file because then the nested included file only looks in the first include local folder.

I wouldn't want to change that behaviour. Might cause some surprises.

[meynaf]

Originally Posted by phx

Exactly. The current work directory, or source directory, has priority over all other include paths. That's what you would expect and that's how all assemblers work (also PhxAss and vasm).

Hopefully global include paths don't take over, it could sometimes have been very annoying

Originally Posted by phx

The only difference between PhxAss and vasm is PhxAss' inability to search for include files relative to the source directory, when it doesn't match CWD.

Well, there is still the incpath option from the command line to cover that case.

In most cases work dir and source dir are the same. I don't have a single project that needs to specify a subdir in the command. I could, however, have some sources included in main one, that are in a subdir.

Originally Posted by phx

The command line option is called -I and does the same. Or are you refering to INCPATH from the PHXOPTS environment?

I was referring to

INCPATH

taken from

ENV:PhxAss/PHXOPTIONS

.

Originally Posted by phx

The only advantage is that you don't have to specify it on the command line.

Which is quite a great advantage for me. I get into the dir, type

phxass source.s

, work done. Or i just double-click the .s and type

phxass

before it...

Originally Posted by phx

But what are Makefiles for?

Makefiles are for C.

I have written many small tools (50+) that are single source over the years (for decode, extract, test, things like that) ; a makefile for each would be overkill.

Originally Posted by Antiriad_UK

It gets more confusing when including a file within a file because then the nested included file only looks in the first include local folder.

Originally Posted by phx

I wouldn't want to change that behaviour. Might cause some surprises.

Wait, are you saying that if i

include somedir/othersource.s

and that this second source includes something, the asm won't look in this second file's dir ?

And now the situation becomes more complicated because you can have a whole tree of directories : a.s includes b/c.s, c.s includes d/e.s, and e.s includes f.s so where could f.s possibly fetch its files ? Could be current dir (command line), dir of a.s, b/ (dir of c.s), b/d/ (dir of e.s), etc.
It looks to me as if considering the source's dir is like opening a can of worms...