Supervisor stack size?

[meynaf]

Originally Posted by grond

The drawback is that if the programmer calculated the need for stack wrong (e.g. when dealing with unexpectedly large objects), stack size can't be increased from the outside without patching the binary.

Nothing will help with that, we don't have automatic (handled by OS) stack increase.
But this gives another shortcoming of the cookie in comparison to normal methods : it's not dynamic. While the program could stack swap on demand with the most appropriate size, the cookie has to contain the maximum possible size. So much for the sparing of memory it's supposed to give.

Originally Posted by grond

The argument with the slow loading of the program doesn't seem very convincing because, again, it is up to the user whether he wants to use an updated loader that needs to search the binary for the cookie or not. For all I know typical 3.9BB users aren't of the A500 with 1MB RAM kind.

Right, so 4KB of wasted memory isn't a big deal for them either...

Originally Posted by Thomas Richter

I'm not *against* stackswap. it's just wasteful.

Then arrange so that your program doesn't need a big stack. Problem solved.

Originally Posted by Thomas Richter

You, as the author, have to use the tools such that the cookie appears at a location that is accessible. Which is not particularly difficult. Hint:

 static const char *stack="$STACK:8192\n";

works nicely for the default configuration. Just place in any source code you like. This is quite "$VER:". As this is a constant string, it ends up in the TEXT segment, like any other constant string.

Declaring static const does not in any manner tell the compiler nor the linker in which hunk the string should end !

Originally Posted by Thomas Richter

Heck, no. Why do you continue to tell such nonsense. The program is fully fine. It is just that under such circumstances, you have to deliver either an icon with the proper stack size set, or tell the user "please use the stack command to increase the stack to .. XXX bytes" if run from the shell.

It is not nonsense. If you realistically believe programmers will always nicely do what you mention above, you're more gullible than i imagined.

Originally Posted by Thomas Richter

Nope. A single read and a table lookup is sufficient. This is not quite as stupid as you seem to imagine.

A table lookup ? Right, it's not quite as stupid as i imagined : it's even stupider !
Now we have a nice bunch of data just for that. We get better every day.

Originally Posted by Thomas Richter

Then that's you. I prefer that programs tell which resources they require.

The stack cookie does not provide this information. Besides, resources a program requires can change depending on what the program has to do.

Originally Posted by Thomas Richter

And for that we have a stack cookie, exactly.

Nope - as it ain't safe in any manner.

Originally Posted by Thomas Richter

You cannot release the stack. You allocate a new one on top.

Not a problem. The shell itself already does this when running a new program, and nobody complained.

Originally Posted by Thomas Richter

See above, no. Give it global lifetime, non-static linkage, constant string, and you will be fine.

Yeah, good ! Give it non-static linkage, which is exactly the opposite of what you did in your above code
Remember ? You wrote "static const char".
Seems it's easily missed.
When will you notice the mistake ? Maybe the end user will, when he sees his program crash ?

Originally Posted by Thomas Richter

Slink does not attempt to remove data with external linkage. Things are really easy.

You don't expect everyone to use Slink, do you ?

Originally Posted by Thomas Richter

In other words: Evolution of AmigaOs is worthless because you don't care. That's not an argument against adding functionality.

You seem to overinterpret what i write...

Originally Posted by Thomas Richter

Then setup properly. It's not that 3.9 is *that* new.

So everyone should be using 3.9, now ? You decided that alone ?

Originally Posted by Thomas Richter

Then $VER is the same hack.

Not exactly. $VER can work in all kinds of files, you often see them in e.g. scripts (which don't have a hunk structure). This is why it is a cookie and not an entry in the executable file format (and also the reason why there is a newline, as it can appear in text files of course not having the null at the end).

Originally Posted by Thomas Richter

Because there is no room in the header. The Tripos HUNK structure does not reserve a hunk for it, quite simple. Of course, scanning ends earlier if you put it near the top of the hunk (which is neither hard to arrange).

It was about how it should have been done from day one (and wasn't), not about what we can do now...

[Don_Adan]

Originally Posted by Thomas Richter

Ok, so you update the programs that work on the Amiga hunk structure, would you? Please start with SLink, and let me know when you are done. Until then, I stick to the stack cookie.

Why? This is very easy to do without breaking compability. For old dos.library versions StackSize hunk will be ignored. Normal Amiga exe files, ended with hunkend $000003F2. Add 8 bytes at end of Amiga exe file, 4 bytes for stack size, 4 bytes for StackSize hunk ID. LoadSeg will be checked for last 4 bytes of exe file, and if StackSize hunk ID at end, then stored stack size will be used.

[Thomas Richter]

Originally Posted by Don_Adan

Why? This is very easy to do without breaking compability. For old dos.library versions StackSize hunk will be ignored.

No, it won't. It would simply abort with "bad hunk".

[Don_Adan]

Originally Posted by Thomas Richter

No, it won't. It would simply abort with "bad hunk".

Of course not, it will be works.I tested this many years ago. Data after last $000003f2 are not readed/checked by LoadSeg.

[Thomas Richter]

Originally Posted by meynaf

Then arrange so that your program doesn't need a big stack. Problem solved.

Are we again at Monty Python's "how to play the flute and save the world"?. It is quite easy: Just blow in here and move the fingers over the holes. Problem solved.

Originally Posted by meynaf

Declaring static const does not in any manner tell the compiler nor the linker in which hunk the string should end !

const char*. And yes, it will work with the default configuration. So don't make things complicated for the matter of complications. "STRSECT=CODE" does the job.

Originally Posted by meynaf

A table lookup ? Right, it's not quite as stupid as i imagined : it's even stupider !

So Knuth is stupid, according to you. Oh well, there you go... This is a well-researched search algorithm, actually. I did not pick this at random. Please, make some background check first.

Originally Posted by meynaf

The stack cookie does not provide this information.

Yes, it does.

Originally Posted by meynaf

Besides, resources a program requires can change depending on what the program has to do.

Oh, really? So STACK is useless as well.

Originally Posted by meynaf

Nope - as it ain't safe in any manner.

In a better manner than depending on the user to arrange the stack.

Originally Posted by meynaf

When will you notice the mistake ? Maybe the end user will, when he sees his program crash ?

Nothing will crash.

Originally Posted by meynaf

You don't expect everyone to use Slink, do you ?

Oh, if you update Blink, that works, too. Or ALink (shudder). Or DevPac, for that matter. All development tools that create the hunk structure and depend on it would require modification. What a great plan... I suppose you get started right away.

Originally Posted by meynaf

So everyone should be using 3.9, now ? You decided that alone ?

Yup, that was only ~18 years ago. Now what?

Originally Posted by meynaf

Not exactly. $VER can work in all kinds of files, you often see them in e.g. scripts (which don't have a hunk structure). This is why it is a cookie and not an entry in the executable file format (and also the reason why there is a newline, as it can appear in text files of course not having the null at the end).

Oh, and this qualifies it also as a good solution for binaries? Why not a hunk for a version there? After all, "Version" also scans for resident structures, and can retrieve the version information from there? And you still need scanning, for example for loading resident libraries (oh boy, what a waste scanning for the resident tag, where is your complaint there?)

Originally Posted by meynaf

It was about how it should have been done from day one (and wasn't), not about what we can do now...

Yes, this is about what we can do now - or what could have been done 18 years ago. I don't care what could have been done 30 years ago, because that is too late already. I care about what happened 18 years ago. After all, $STACK: is not *that* new, really. It is a pretty old concept.

If you want to speculate what should have been done 30 years ago, then I can tell you: Create a robust operating system in first place, one with memory protection and a guard page below the stack. *THAT* would have worked robustly. Unfortunately, that was not really an option for a consumer device. The hunk structure is just a minor inconvenience compared to the major issues the Os has.

[Thomas Richter]

Originally Posted by Don_Adan

Of course not, it will be works.I tested this many years ago. Data after last $000003f2 are not readed/checked by LoadSeg.

That, of course, depends on some details of the binary. And you ignore what happens if you process the binary, or how programs react that attempt to load such a binary. You are opening a compatibility problem that can be quite easily avoided.

[meynaf]

Originally Posted by Thomas Richter

Are we again at Monty Python's "how to play the flute and save the world"?. It is quite easy: Just blow in here and move the fingers over the holes. Problem solved.

What do you try to prove here ?

Originally Posted by Thomas Richter

const char*. And yes, it will work with the default configuration. So don't make things complicated for the matter of complications. "STRSECT=CODE" does the job.

It does the job for your particular development environment.

Originally Posted by Thomas Richter

So Knuth is stupid, according to you. Oh well, there you go... This is a well-researched search algorithm, actually. I did not pick this at random. Please, make some background check first.

You jump to conclusions very fast, don't you ? I did not say the algorithm nor its author is stupid. Just that in this particular case it's not really clever.

But what ? You did not pick this at random ? So you were the developer behind it ?
That explains why you defend it so much

Originally Posted by Thomas Richter

Yes, it does.

No way. It does not say anything but a hint on the stack size. Not what the program will use as resources.

Originally Posted by Thomas Richter

Oh, really? So STACK is useless as well.

In some way, yes. It's just a command to be able to run badly written programs.

Originally Posted by Thomas Richter

In a better manner than depending on the user to arrange the stack.

Being better than the worse, doesn't make it good.

Originally Posted by Thomas Richter

Nothing will crash.

So now stack overflows don't crash ?
Remember, if your cookie gets removed, your stack size won't change...

Originally Posted by Thomas Richter

Oh, if you update Blink, that works, too. Or ALink (shudder). Or DevPac, for that matter. All development tools that create the hunk structure and depend on it would require modification. What a great plan... I suppose you get started right away.

You can list linkers that put the cookie in first hunk if you want, the problem is that it's not all of them.

Originally Posted by Thomas Richter

Yup, that was only ~18 years ago. Now what?

Now nothing has changed. We're not forced to go to 3.9 more than we were 18 years ago.

Originally Posted by Thomas Richter

Oh, and this qualifies it also as a good solution for binaries? Why not a hunk for a version there? After all, "Version" also scans for resident structures, and can retrieve the version information from there?

Resident structures (code in rom) don't contain hunks so adding a hunk for the version would not work for them. This kinda proves my point.

Originally Posted by Thomas Richter

And you still need scanning, for example for loading resident libraries (oh boy, what a waste scanning for the resident tag, where is your complaint there?)

What a very nice example of a straw man fallacy. Resident tags are always there, and rarely placed anywhere but at start of the module - which is only tens of Kb at max.
By comparison, a program can have 1MB as first hunk, in which the stack cookie can just be missing.

[Thomas Richter]

Originally Posted by meynaf

What do you try to prove here ?

That you state something as "easy and obvious", whereas it is, in reality, not simple at all if you look at all the pesky details. Of course the theory of "how to play a flute" is simple. Yet, it requires a bit more to actually play some music on it.

Originally Posted by meynaf

It does the job for your particular development environment.

No, not only for mine. For anyone's, using 3.9.

Originally Posted by meynaf

You jump to conclusions very fast, don't you ? I did not say the algorithm nor its author is stupid. Just that in this particular case it's not really clever.

It is because it minimizes the complexity of the search. You are invited to make it faster...

Originally Posted by meynaf

But what ? You did not pick this at random ? So you were the developer behind it ?

So why should I select a linear search then when there is a *provably* faster variant?

Originally Posted by meynaf

No way. It does not say anything but a hint on the stack size. Not what the program will use as resources.

Nor can anyone set the STACK to a "correct" size. So what does that prove on the functionality as such? That we do not need STACK? That is certainly the false conclusion.

Originally Posted by meynaf

In some way, yes. It's just a command to be able to run badly written programs.

Oh, and how can you write programs that provably work with a limited stack? "How to play a flute". You can turn on the stack check of the compiler, but even that does not guarantee much.

Originally Posted by meynaf

Being better than the worse, doesn't make it good.

You are invited to implement an automatic stack extension with guard pages.

Originally Posted by meynaf

So now stack overflows don't crash ?
Remember, if your cookie gets removed, your stack size won't change...

And why do you want to remove it in first place? And how do you prevent that by other means? Playing with the hunk format is much more likely to break.

Originally Posted by meynaf

You can list linkers that put the cookie in first hunk if you want, the problem is that it's not all of them.

That is not a matter of the linker, really. The compiler will necessarily put the startup code in the TEXT segment, and this must be hunk #0, as this is where shell and workbench will start the shell. The linker will have to put that into #0. This is precisely where the cookie will go. This is not at all hard to arrange. Actually, it is very obvious to arrange. Frankly, I know no compiler which does not provide options to place strings into the text segment. Even then, a "custom hunk structure" is even harder to arrange as you do not have any tools available at this point to create such a structure. So you can discuss as you like: I have a workable solution that works with the existing tool chain. You have a hypthetical construction without existing tools.

Originally Posted by meynaf

Now nothing has changed. We're not forced to go to 3.9 more than we were 18 years ago.

And your point is? That I need to create a time machine to change the past? That I should not arrange new simple mechanisms to help authors?

Originally Posted by meynaf

Resident structures (code in rom) don't contain hunks so adding a hunk for the version would not work for them. This kinda proves my point.

Err, I beg your pardon, but how do you think ramlib initializes disk-based libraries and devices? Of course it scans hunks for a resident structure, what do you think? It just scans all segments.

Originally Posted by meynaf

What a very nice example of a straw man fallacy. Resident tags are always there, and rarely placed anywhere but at start of the module - which is only tens of Kb at max.

There is no need to place them at the start of the segment, and how large a device or library is depends really on the library. Quite the reverse, the stack token is only scanned for in a limited number of places, so scanning is fast, and you can and should place them at the start of the binary. If you can arrange the resident tag at the start of a binary, you can certainly arrange the same for the stack token, can't you?

Originally Posted by meynaf

By comparison, a program can have 1MB as first hunk, in which the stack cookie can just be missing.

It if is missing, you made an error in creating the binary. Same as forgetting the resident tag.

[Don_Adan]

Originally Posted by Thomas Richter

That, of course, depends on some details of the binary. And you ignore what happens if you process the binary, or how programs react that attempt to load such a binary. You are opening a compatibility problem that can be quite easily avoided.

This is 100% safe method. You can check WHDload exe file. Anyway if you dont like this. You can use minimum two other methods.
1. If i remember right not all legal hunks handled by LoadSeg has code (only skip bytes code is used). Choose one as Stack Size.
2. Modify debug hunk code, add extra check for stacksize ID and handle stack size.
Your method is not good.

[Thomas Richter]

Originally Posted by Don_Adan

This is 100% safe method.

Nope. You seem to assume that LoadSeg is the only way how such data is processed, which is not given. There are multiple other tools that operate on such files, and that could fail or misinterpret such data. It is a "protocol violation" because it fails to follow the specs of how a HUNK format should look like.

Originally Posted by Don_Adan

1. If i remember right not all legal hunks handled by LoadSeg has code (only skip bytes code is used). Choose one as Stack Size.

LoadSeg is only the run-time binder of the Os - and by far not the only way how such data is processed. What loadseg does internally is an implementation detail - you violate the specification and silently assume that the Os function has a particular implementation. IOWs, if you append junk at the end, you create an ill-formed hunk-format that violates the specifications.

Originally Posted by Don_Adan

2. Modify debug hunk code, add extra check for stacksize ID and handle stack size.

Another protocol violation. The debug hunk is there to contain debug information, thus any tool on working on hunk data can assume that it is safe to strip such hunks off. Brilliant idea. Besides, even if the data would be there, LoadSeg() would still ignore it, thus the data would not be easily accessible with the given loading mechanism. It would neither be available for binaries that have been made resident as any trace to its representation as hunks is lost.

Originally Posted by Don_Adan

Your method is not good.

And yet you failed to explain why. You do not understand that this is an Os, it has to work according to a specification applications want to depend upon. The specification is public, and taken for granted. You want to change that, which is not possible without risking compatibility issues. The cookie does not require any specification to change. It works with all existing tools transparently, does not require any modification to tools or the run-time binder / LoadSeg(), it also works no matter where a segment comes from (LoadSeg() or else)

[meynaf]

Originally Posted by Thomas Richter

That you state something as "easy and obvious", whereas it is, in reality, not simple at all if you look at all the pesky details. Of course the theory of "how to play a flute" is simple. Yet, it requires a bit more to actually play some music on it.

Frankly limiting the stack usage of a program is far from being rocket science.

Originally Posted by Thomas Richter

No, not only for mine. For anyone's, using 3.9.

Yeah, and surely you can ensure not even GCC with max optimization will not remove your cookie...

Originally Posted by Thomas Richter

It is because it minimizes the complexity of the search. You are invited to make it faster...

Sure, but at the price of a table taking space in every case, even when it's not needed because the cookie won't be there at the end. Not my view of a sparing memory method.
But the problem here isn't even the search algorithm. It is the mere fact of having to perform a search. If you don't do this, believe me, it'll be a magnitude faster.

Originally Posted by Thomas Richter

So why should I select a linear search then when there is a *provably* faster variant?

Why should you use a search when you don't need to search anything at all ?

Originally Posted by Thomas Richter

Nor can anyone set the STACK to a "correct" size. So what does that prove on the functionality as such? That we do not need STACK? That is certainly the false conclusion.

It does not prove we do not need STACK. It only proves the cookie does not say much about the needed resources of the program.

Originally Posted by Thomas Richter

Oh, and how can you write programs that provably work with a limited stack? "How to play a flute". You can turn on the stack check of the compiler, but even that does not guarantee much.

Sounds like an argument against using compilers.

Originally Posted by Thomas Richter

You are invited to implement an automatic stack extension with guard pages.

No thanks. Simple stack swap is enough for my needs.

Originally Posted by Thomas Richter

And why do you want to remove it in first place? And how do you prevent that by other means? Playing with the hunk format is much more likely to break.

It can be unintentionnally removed, for example by writing "static" by mistake in the declaration like you did before...
You can prevent the removal if it's simply code that gets executed as part of the startup.
And right, playing with the hunk format may be likely to break, but why not just using overlay hunks.

Originally Posted by Thomas Richter

That is not a matter of the linker, really. The compiler will necessarily put the startup code in the TEXT segment, and this must be hunk #0, as this is where shell and workbench will start the shell. The linker will have to put that into #0. This is precisely where the cookie will go. This is not at all hard to arrange. Actually, it is very obvious to arrange. Frankly, I know no compiler which does not provide options to place strings into the text segment. Even then, a "custom hunk structure" is even harder to arrange as you do not have any tools available at this point to create such a structure. So you can discuss as you like: I have a workable solution that works with the existing tool chain. You have a hypthetical construction without existing tools.

"The" text segment ? But, good boy, on the Amiga there can be many text segments (code hunks, actually), not only one !
Only startup code is guaranteed to be in the first one, you cannot do that for any data.

Originally Posted by Thomas Richter

And your point is? That I need to create a time machine to change the past? That I should not arrange new simple mechanisms to help authors?

You can arrange new simple mechanisms to help authors if you want, but this one doesn't help anyone.

Originally Posted by Thomas Richter

Err, I beg your pardon, but how do you think ramlib initializes disk-based libraries and devices? Of course it scans hunks for a resident structure, what do you think? It just scans all segments.

Can't you read ? The point was about ROM based residents, which do NOT contain hunks - but still support version.

Originally Posted by Thomas Richter

There is no need to place them at the start of the segment, and how large a device or library is depends really on the library. Quite the reverse, the stack token is only scanned for in a limited number of places, so scanning is fast, and you can and should place them at the start of the binary. If you can arrange the resident tag at the start of a binary, you can certainly arrange the same for the stack token, can't you?

Ahem. Current tools of course support doing libraries and devices. But alas they know nothing about the stack cookie.
Resident tags are mandatory for libraries and devices so the search will hopefully end with a success - and needs not be fast otherwise. On the other hand, not even 1% of executables that'll run actually contain the stack cookie and it'll be searched in vain by scanning all of their first hunk entirely. Definitely not the same situation.

Originally Posted by Thomas Richter

It if is missing, you made an error in creating the binary. Same as forgetting the resident tag.

By "missing" i meant "not there because not used", not "should be there but isn't".

[Don_Adan]

Originally Posted by Thomas Richter

Nope. You seem to assume that LoadSeg is the only way how such data is processed, which is not given. There are multiple other tools that operate on such files, and that could fail or misinterpret such data. It is a "protocol violation" because it fails to follow the specs of how a HUNK format should look like. LoadSeg is only the run-time binder of the Os - and by far not the only way how such data is processed. What loadseg does internally is an implementation detail - you violate the specification and silently assume that the Os function has a particular implementation. IOWs, if you append junk at the end, you create an ill-formed hunk-format that violates the specifications. Another protocol violation. The debug hunk is there to contain debug information, thus any tool on working on hunk data can assume that it is safe to strip such hunks off. Brilliant idea. Besides, even if the data would be there, LoadSeg() would still ignore it, thus the data would not be easily accessible with the given loading mechanism. It would neither be available for binaries that have been made resident as any trace to its representation as hunks is lost. And yet you failed to explain why. You do not understand that this is an Os, it has to work according to a specification applications want to depend upon. The specification is public, and taken for granted. You want to change that, which is not possible without risking compatibility issues. The cookie does not require any specification to change. It works with all existing tools transparently, does not require any modification to tools or the run-time binder / LoadSeg(), it also works no matter where a segment comes from (LoadSeg() or else)

Sorry, but your way is bad. It must works for all possibilities. I resourced many megabytes of Amiga code. And nothing will be break, f.e ReSource cant handle word hunk relocation. Very often I see wrong debug offsets in debug hunk. Adding "dc. l "STAK","SIZE", stacksize" at begining of hunk debug, nothing can break. New resource/dissasembler can add support for this without problem. Ok. and tell me which other Amiga system tools handle Amiga exe files? I dont know others. Using word hunks relocation in Amiga exe files, breaks all violation up to kickstart 2.05 or maybe 3.0 (I dont checked this exactly). This is real problem. Not adding 8 bytes at end of Amiga exe file, not adding 12 bytes to debug hunk, or choose unused/legal/supported hunk as StackSize.
I know, that you think that your version is good, but this is useless, only wasting some bytes or kilobytes in Amiga ROM space.
If my ideas are really bad, then we can try to find other clear solution for stacksize support. For me your way is dirty hack only, not clean way.

[Thomas Richter]

Originally Posted by Don_Adan

Sorry, but your way is bad. It must works for all possibilities.

Which is precisely what your proposal doesn't. Let's boil it down to one sentence: You modify a well-documented file format without being able to update the tools that depend on it. That is really it. Nothing else to say.

[Thomas Richter]

Originally Posted by meynaf

Frankly limiting the stack usage of a program is far from being rocket science.

Write a recursive algorithm and you have the problem at hand.

Originally Posted by meynaf

Yeah, and surely you can ensure not even GCC with max optimization will not remove your cookie...

Absolutely, I can. Here is how many system programs ensure that the version cookie cannot get lost, just to give you an idea: DOSBase = OpenLibary("dos.library\0$VER:list 47.1",47); Besides this little trick, "const char *" also works well as I have yet to see a compiler that strips an object with external linkage away. So please, I have used cookies in many programs, don't tell me what works and what does not. You have very limited experience, apparently.

Originally Posted by meynaf

Sure, but at the price of a table taking space in every case, even when it's not needed because the cookie won't be there at the end.

I have no idea what you are talking about. If the cookie is there and is honored, the stack will have at least the requested size.

Originally Posted by meynaf

But the problem here isn't even the search algorithm. It is the mere fact of having to perform a search. If you don't do this, believe me, it'll be a magnitude faster.

Have you measured? Apparently - not. The shell is an I/O bound algorithm, not a memory-bound algorithm. This is not even noticable - as you have not noticed until now. Of course the shell would be a magnitude faster if it would not have to load binaries.... but that is beyond the point.

Originally Posted by meynaf

Why should you use a search when you don't need to search anything at all ?

You *DO* need the search for the cookie. It is a pretty quick search as well.

Originally Posted by meynaf

It does not prove we do not need STACK. It only proves the cookie does not say much about the needed resources of the program.

It says that the programmer assumes that the requested stack size is sufficient. Which is much easier and less error prone than telling that the user, manually.

Originally Posted by meynaf

No thanks. Simple stack swap is enough for my needs.

No problem for me, go ahead. It's a bit wasteful, but that's your decision.

Originally Posted by meynaf

It can be unintentionnally removed, for example by writing "static" by mistake in the declaration like you did before...

And then you will notice during testing the program. You do test, don't you? So what is your point? By using the standard development toolkit, the cookie is stuck to the code. However, with a modified file format, all bets are off as whether programs preseve it or not. Even worse, any meta information that is only in the format and not in memory is lost as soon as you make a program "resident". The cookie sticks. The design did not come from nowhere.

Originally Posted by meynaf

You can prevent the removal if it's simply code that gets executed as part of the startup.

Works for me, go ahead. Still not an argument against the stack cookie. Which is easier to put in, but what the heck, your choice.

Originally Posted by meynaf

And right, playing with the hunk format may be likely to break, but why not just using overlay hunks.

Which has absolutely no relation to cookies or stacks whatsoever.

Originally Posted by meynaf

"The" text segment ? But, good boy, on the Amiga there can be many text segments (code hunks, actually), not only one !
Only startup code is guaranteed to be in the first one, you cannot do that for any data.

And your point is? Then put it into the first hunk, would you? Actually, it is very hard *not* to put it there, by any means. I already told you how to do that with your average compiler. This is trivial. Don't create problems where there are none.

Originally Posted by meynaf

You can arrange new simple mechanisms to help authors if you want, but this one doesn't help anyone.

Declares who? Do you speak for all authors? How can you know anybody else's opinion.

Originally Posted by meynaf

Can't you read ? The point was about ROM based residents, which do NOT contain hunks - but still support version.

Can't you read? The point was about disk-based libraries, which DO contain residents that need to be scanned for. Besides, even ROM-based segments could have stack-cookies. But no other stack indicators.

Originally Posted by meynaf

Ahem. Current tools of course support doing libraries and devices. But alas they know nothing about the stack cookie.

Because they don't need to. That is *exactly* the point. You *do not* update your tools to get support for the stack cookie. Just put it in. I already gave an example how to arrange that. It is elementary to "convince" the compiler to put a string into the first hunk. You do not need to update the compiler or linker to enable the feature. That was one requirement for the design.

Originally Posted by meynaf

Resident tags are mandatory for libraries and devices so the search will hopefully end with a success - and needs not be fast otherwise. On the other hand, not even 1% of executables that'll run actually contain the stack cookie and it'll be searched in vain by scanning all of their first hunk entirely. Definitely not the same situation.

Then change that and put more stack cookies into programs. They can be put to good use to avoid telling the user the stack requirements of a program. Anyhow. $STACK: will stick - it is in Os 4, Os 3.9, and 3.1.4, and in any continued version from there on.

[meynaf]

Originally Posted by Thomas Richter

Write a recursive algorithm and you have the problem at hand.

How so ? I have written several of them with no problem at all. Just put a reasonable limit to the number of recursions - it rarely, if ever, needs to be big.

Originally Posted by Thomas Richter

Absolutely, I can. Here is how many system programs ensure that the version cookie cannot get lost, just to give you an idea: DOSBase = OpenLibary("dos.library\0$VER:list 47.1",47); Besides this little trick, "const char *" also works well as I have yet to see a compiler that strips an object with external linkage away. So please, I have used cookies in many programs, don't tell me what works and what does not. You have very limited experience, apparently.

Yeah of course now it's me who have very limited experience
What you did here is a typical behavior of people out of argument. I didn't know you were the type, though. This is deceiving. Aren't you better than that ?

Anyhow, my "very limited experience" tells me that none of your tricks will guarantee you get the cookie in the right hunk. It works for you purely by luck.
Ok, it will not be stripped (at the price of something quite ugly). So it will be in your executable and still possibly in the wrong section. The fact it never happens in your setup isn't a proof it can't happen at all. IOW, "works for me" isn't a correct reply.

Originally Posted by Thomas Richter

I have no idea what you are talking about. If the cookie is there and is honored, the stack will have at least the requested size.

I'm not talking about when the cookie is there and honored.
The point here was that you have some table (256 bytes ?) that is there permanently eating memory, just to spare a few temporary kb in some rare cases.
Clear now ?

Originally Posted by Thomas Richter

Have you measured? Apparently - not. The shell is an I/O bound algorithm, not a memory-bound algorithm. This is not even noticable - as you have not noticed until now. Of course the shell would be a magnitude faster if it would not have to load binaries.... but that is beyond the point.

How could I have measured or noticed anything, knowing that this thing has never been, and never will be, present in any of my configs ?
But indeed it's probably not noticeable. Neither is the second stack when doing a swap.

Originally Posted by Thomas Richter

You *DO* need the search for the cookie. It is a pretty quick search as well.

No i don't need to search for it. My system does not search for it. Should I find it in a program i need to alter, i'd delete it.

Originally Posted by Thomas Richter

It says that the programmer assumes that the requested stack size is sufficient. Which is much easier and less error prone than telling that the user, manually.

Yes, and ? Any way of setting the stack without requesting it from the user will do the same.

Originally Posted by Thomas Richter

No problem for me, go ahead. It's a bit wasteful, but that's your decision.

Wasteful ? Who is wasteful ?
Gains i get by rewriting parts of compiled code into asm easily spare a lot more bytes than that little "waste" uses.

Originally Posted by Thomas Richter

And then you will notice during testing the program. You do test, don't you? So what is your point? By using the standard development toolkit, the cookie is stuck to the code. However, with a modified file format, all bets are off as whether programs preseve it or not. Even worse, any meta information that is only in the format and not in memory is lost as soon as you make a program "resident". The cookie sticks. The design did not come from nowhere.

Not sure a stack that is just slightly too small will be noticed easily during testing... You have to test in conditions so that the cookie contains the largest value of them all, otherwise you won't see the error.

The fact the cookie is still there for resident programs may seem a good argument for it...
... Except for the fact resident made programs are quite rare in current setups, and they are for small, often used commands that of course don't need big stacks.

Originally Posted by Thomas Richter

Works for me, go ahead. Still not an argument against the stack cookie. Which is easier to put in, but what the heck, your choice.

The stack cookie is a hack. Accept it or not, it does not change what it is.
And i don't put hacks in programs when i can do otherwise.

Originally Posted by Thomas Richter

Which has absolutely no relation to cookies or stacks whatsoever.

Yes it has. I was just suggesting using overlay hunks to store the wanted stack size, instead of changing executable format. These already exist. They don't break anything. (Even if i'd still prefer having a proper API for setting up the stack.)

Originally Posted by Thomas Richter

And your point is? Then put it into the first hunk, would you? Actually, it is very hard *not* to put it there, by any means. I already told you how to do that with your average compiler. This is trivial. Don't create problems where there are none.

It's actually very easy to put it in another hunk. You need, however, a significantly sized project so that the output will contain several code hunks (and a linker that does not merge them).

Originally Posted by Thomas Richter

Declares who? Do you speak for all authors? How can you know anybody else's opinion.

How i can know anybody else's opinion ? Several of them seen in this thread, maybe ? With all of them saying the cookie isn't especially useful, maybe ? Frankly, aside of yourself - which is easy to understand as you designed it - who defends the stack cookie ?

Originally Posted by Thomas Richter

Can't you read? The point was about disk-based libraries, which DO contain residents that need to be scanned for. Besides, even ROM-based segments could have stack-cookies. But no other stack indicators.

No, the point wasn't about disk-based libraries. They have nothing to do here, as they can be handled exactly like any other executable file.
And yes, ROM-based segments can have a stack cookie -- but neither libraries, devices, resources, nor small resident commands need to use something like that.

Originally Posted by Thomas Richter

Because they don't need to. That is *exactly* the point. You *do not* update your tools to get support for the stack cookie. Just put it in. I already gave an example how to arrange that. It is elementary to "convince" the compiler to put a string into the first hunk. You do not need to update the compiler or linker to enable the feature. That was one requirement for the design.

But you still need to update the operating system to use the feature. On others it will fail silently.
And again you can not tell the compiler in which hunk to put things.

Originally Posted by Thomas Richter

Then change that and put more stack cookies into programs. They can be put to good use to avoid telling the user the stack requirements of a program. Anyhow. $STACK: will stick - it is in Os 4, Os 3.9, and 3.1.4, and in any continued version from there on.

No, i will never use that.
But i don't really care if it sticks or not, even if i would certainly delete it if i had to rewrite the OS. At least it's relative harmless.

[phx]

Originally Posted by meynaf

Anyhow, my "very limited experience" tells me that none of your tricks will guarantee you get the cookie in the right hunk.

Of course you can guarantee that. Even if you think that a compiler messes things up (which is not true - as Thomas correctly said "const" works most of the time, otherwise there are often special attributes), you could just link with an object module (for example written in assembler, if you like) which places the cookie into the first section.

Frankly, aside of yourself - which is easy to understand as you designed it - who defends the stack cookie ?

Most people won't read these stupid discussions, which lead to nothing.

I think the STACK cookie is a good idea. There is nothing bad about it (I don't care about a few cycles scanning for it at program start) and it is implemented in a flexible, backwards-compatible way. Other operating systems, like Atari MiNT, have a similar cookie.

My problem is that it is much too late. In my software I would never expect that most users are running 3.9 or 3.1.4, or even requiring it. So I will still use StackSwap() to be safe. Now I need to know: which additional advantage would I get by including the STACK cookie?

When it means that the STACK cookie launches my program with a large enough stack, so that StackSwap() is no longer needed under 3.9/3.1.4, then I should probably add it. I will check the current stack size anyway, so I can avoid StackSwap() when the stack was already large enough.

[meynaf]

Originally Posted by phx

Of course you can guarantee that. Even if you think that a compiler messes things up (which is not true - as Thomas correctly said "const" works most of the time, otherwise there are often special attributes), you could just link with an object module (for example written in assembler, if you like) which places the cookie into the first section.

And how do you tell the linker to put that object module in the same section as the startup code ?

Originally Posted by phx

Most people won't read these stupid discussions, which lead to nothing.

Eh, sorry. Stupid ideas always lead to stupid discussions. This is life.

Originally Posted by phx

I think the STACK cookie is a good idea. There is nothing bad about it (I don't care about a few cycles scanning for it at program start) and it is implemented in a flexible, backwards-compatible way. Other operating systems, like Atari MiNT, have a similar cookie.

My problem is that it is much too late. In my software I would never expect that most users are running 3.9 or 3.1.4, or even requiring it. So I will still use StackSwap() to be safe. Now I need to know: which additional advantage would I get by including the STACK cookie?

When it means that the STACK cookie launches my program with a large enough stack, so that StackSwap() is no longer needed under 3.9/3.1.4, then I should probably add it. I will check the current stack size anyway, so I can avoid StackSwap() when the stack was already large enough.

As you have to do StackSwap anyway, there is next to zero advantage in using the stack cookie.
But as it's taking its toll regardless if wanted or not, indeed why not using it.

[phx]

Originally Posted by meynaf

And how do you tell the linker to put that object module in the same section as the startup code ?

Use the right section name. Any AmigaDOS linker automatically merges sections which have the same name and type.

[meynaf]

Originally Posted by phx

Use the right section name. Any AmigaDOS linker automatically merges sections which have the same name and type.

And the right section name is ?
Are programmers supposed to know the section name of the startup code generated by their compiler ?

[meynaf]

Another problem i'm just thinking about.
Even if the stack cookie is there and honored, things can still go haywire.
I think many compilers will setup the stack as part of their startup code. Or at least they should.
Now the programmer, not knowing about this fact, will put the cookie in his code.
A cookie at the right place. Perfectly detected, with stack perfecly allocated.
And then the startup code performs its regular stack swap...