AmigaOS 3.2 performance vs 3.1

[meynaf]

Originally Posted by Thomas Richter

If you don't care about new releases, then why all the rumble?

I don't care that intuition can be flushed or not in v47 (which is "that v47 detail"). Why would it mean that i don't care about whole new versions overall ?

[Don_Adan]

Originally Posted by Thomas Richter

Because just because you update a library does not mean that you update its users. Once again: FindConfigDev() of expansion *passes out* a pointer to an expansion library database structure (a ConfigDev). There is no interface to *release this pointer again*. Even if a future release would provide such a function, the library cannot assume that every user calls this function, and thus, the design is stuck. Or, to put it differently:


"It's as in real life: Once you make a mistake, you have to support it for your lifetime."



If expansion is ever supposed to be updated, then rather into the direction of detecting autoconfiguring hardware on other bus systems and provide them to the user, but transparently with existing interfaces. That is a potential and useful development direction, and not to worry about its minimal RAM footprint (and it is really minimal compared to other system libraries).




That is exactly the same question and has exactly the same reasons. It does not help to update a library if you do not update its users. See RKRM AmigaDOS. How to represent a file size >2GB in a FileInfoBlock where the file size is indicated as LONG? What happens if the file size in the FileInfoBlock does not correspond to its true file size? What does a Seek() with a OFFSET_BEGINNING and a negative argument mean? Can we depend that this is actually a seek beyond 2GB, or is it actually an error by attempting to seek backwards?



At this point, my usual canon starts: The problem is that the *interface* of the dos.library is not properly defined to handle such cases, and it is too late to change an interface if we have implementations in the wild that instead depend on the actual implementation details since the documentation of this interface was so lousy (and in case of the dos.library, that was unfortunately really the case).

This is easy, if this LONG is equal (-1) $ffffffff, then this is extended FileInfoBlock structure.
And file size is stored as 2 longs or 3 words (this is enough for file size) in different place.

For OFFSET_BEGINING 2 longs or 3 words are necessary, again easy using combination of D0 and D1.

And for expansion.library, only 2 changes must be done.
If I understand correctly all infos.
1st. Expansion library must be available to close like every standard Amiga library.
2nd. Allocaction memory must be changed from chip mem (D1=2) to any mem (D1=1).

Then first opening will be for chip mem, and after closing this lib. Expansion.library will be opened in fast mem (if available).

[Thomas Richter]

Originally Posted by Don_Adan

This is easy, if this LONG is equal (-1) $ffffffff, then this is extended FileInfoBlock structure.

That doesn't make applications work that expect (correctly so) a LONG there. They will understand this as -1, with whatever consequences this may have. The interface is "this is a LONG", like it or not, and existing programs have been implemented under the assumption that it is a LONG, with whatever consequences this has - for example, using signed comparisons.



All these ideas would work if you would have a living software ecosystem where you could tell authors "please recompile your software", but we don't. This design is *stuck*. You can create a new "Examine64()/ExNext64()" with a new fib structure, but that does not say what legacy applications see with Examine()/ExNext() on a large file. There are only bad options - either fail, or report a wrong result, again with very little gains for practical Amiga applications. A 4GB file on such an old system? What for?

Originally Posted by Don_Adan

1st. Expansion library must be available to close like every standard Amiga library.

Expansion library *can* be closed. It cannot be flushed, and that is quite like *every other ROM-based library* (but unlike intuition). Neither graphics, dos, exec can be flushed. What for?

Originally Posted by Don_Adan

2nd. Allocaction memory must be changed from chip mem (D1=2) to any mem (D1=1).

Expansion is allocated with MEMF_ANY. Of course it is. The only reason why it pops up in chip memory *on some machines* is because there is no other memory available at the time expansion is initialized, and expansion is responsible for configuring additional memory. The reason why the Blizzard 1230 has *also* exec in chip memory is because it does not have autoconfig memory at all. Its fast RAM is some kind of hacky "P5" construction which is likely added through a resident module in the debug ROM, and that is initialized after expansion and exec. If things had been done properly, at least exec would be in fast ram and we would not have this thread.

Originally Posted by Don_Adan

Then first opening will be for chip mem, and after closing this lib. Expansion.library will be opened in fast mem (if available).

Which does not work, because it still needs access to the scanned expansions, which it won't.


Anyhow, this is again one of these pointless discussions. a) this is not the AmigaOs development forum, b) I'm not an AmigaOs developer, c) the ratio between "investment of time" is no useful ratio to the result (saving less than 1K of chip RAM on some systems). expansion, in particular, is very hard to debug because it is so early in bootstrapping the system, and the chance for breaking anything for saving so little RAM is quite high. If there is any advice to give then that: "By all means, do not touch expansion without really good reason", and this is not even a mediocre reason. It is close to "no reason at all".

[Don_Adan]

Sorry, but this long can be -1. It will break nothing.
For old programs it can show wrong size and nothing else.
Even for now exist many old Amiga programs which can not be installed on newest HD .
Because partition is greater than 2GB and for old programs this is NO FREE SPACE on disk.
And simple You forget that for handling files greater than 4GB, new file system is necessary too.
Then new version of Amiga OS and new kick .
Everything can be done.

[Don_Adan]

And last thing.
You know how many old Amiga programs crashes (or doesnt works correctly), if filename is longer than 31 signs?
And what? Amiga users very often break this limit, and many programs is crashed.
Amiga users learned what can be done with old programs and what can not be done.
Same for file sizes.

[meynaf]

Originally Posted by Thomas Richter

A 4GB file on such an old system? What for?

Hard drive images maybe ?
Anyhow, the case already exists. Emulators can map a host directory, which may well contain anything.

[Thomas Richter]

Originally Posted by Don_Adan

Sorry, but this long can be -1. It will break nothing.

How do you know? I don't. It is a negative file size. Whatever that will mean to some programs.

Originally Posted by Don_Adan

For old programs it can show wrong size and nothing else.

You seem to assume that this size is just used for display purposes, but that might be wrong. It could be used to estimate the amount of buffering required to copy a file, just to give one example - and that could be outright wrong. It could be outright confusing - -1 divided by the block size, well, probably rounded up - zero buffers required - boom.

Originally Posted by Don_Adan

Everything can be done.

But not everything can be done without breaking something. That is a design "based on assumptions", but whether that may be true or not ... who knows. I do not claim that I would know.

[Thomas Richter]

Originally Posted by Don_Adan

And last thing.
You know how many old Amiga programs crashes (or doesnt works correctly), if filename is longer than 31 signs?

The limit is actually 30 characters, and personally never run into a particular problem with 30 characters. Volume, assign and device names are still limited to 30 characters. The limit is actually per component, the maximum path lenth is actually 255 characters.

[meynaf]

Originally Posted by Thomas Richter

How do you know? I don't. It is a negative file size. Whatever that will mean to some programs.

It is negative if you think it is negative. Else it is 4294967295.

Originally Posted by Thomas Richter

You seem to assume that this size is just used for display purposes, but that might be wrong. It could be used to estimate the amount of buffering required to copy a file, just to give one example - and that could be outright wrong. It could be outright confusing - -1 divided by the block size, well, probably rounded up - zero buffers required - boom.

Normally a coder should know a file size is an unsigned number...

Originally Posted by Thomas Richter

But not everything can be done without breaking something. That is a design "based on assumptions", but whether that may be true or not ... who knows. I do not claim that I would know.

Nothing will break, as long as big files are not used with such old software.

Originally Posted by Thomas Richter

The limit is actually 30 characters, and personally never run into a particular problem with 30 characters. Volume, assign and device names are still limited to 30 characters. The limit is actually per component, the maximum path lenth is actually 255 characters.

Some file systems support names much larger than 30. This is when the problem happens : software reserves buffer for 30, and...

[Thomas Richter]

Originally Posted by meynaf

It is negative if you think it is negative. Else it is 4294967295.

It is negative because dos/dos.h defines it to be a LONG. That's the interface. Yes, it's a silly interface, but that is the situation. It is documented to be a LONG and not a ULONG.

Originally Posted by meynaf

Normally a coder should know a file size is an unsigned number...

Normally a coder should use the system provided header files and not make assumptions.

Originally Posted by meynaf

Nothing will break, as long as big files are not used with such old software.

Except that all software on Amiga is "such old software". Again, if this would be a living vivid system, the situation would be somewhat different.


There are many possible ideas how to resolve such problems. One option is to encode the maximum LONG integer (0x7fffffff) and use that as indicator. Or use -1 as indicator. Or skip over such files when iterating over a directory. Or throwing an error in case such a file is examined with the old API (which is probably the most conservative approach avoiding too much breakage).


There are many possible approaches, each with drawbacks and advantages. I'm not happy with "shoot first" solutions.

Originally Posted by meynaf

Some file systems support names much larger than 30. This is when the problem happens : software reserves buffer for 30, and...

Sure.

[meynaf]

Originally Posted by Thomas Richter

It is negative because dos/dos.h defines it to be a LONG. That's the interface. Yes, it's a silly interface, but that is the situation. It is documented to be a LONG and not a ULONG.

If the interface is silly, better fix it.

Originally Posted by Thomas Richter

Normally a coder should use the system provided header files and not make assumptions.

So we should blindly follow interface mistakes. Cool.

Originally Posted by Thomas Richter

Except that all software on Amiga is "such old software". Again, if this would be a living vivid system, the situation would be somewhat different.


There are many possible ideas how to resolve such problems. One option is to encode the maximum LONG integer (0x7fffffff) and use that as indicator. Or use -1 as indicator. Or skip over such files when iterating over a directory. Or throwing an error in case such a file is examined with the old API (which is probably the most conservative approach avoiding too much breakage).


There are many possible approaches, each with drawbacks and advantages. I'm not happy with "shoot first" solutions.

This is being overprudent. Very large files are rare. Old software that would choke on them is rare. I prefer that a few things break in comparison to weird compatibility tricks.

[Thomas Richter]

Originally Posted by meynaf

If the interface is silly, better fix it.

It is too late to fix it. Software was designed/compiled with the header in place. If you just replace LONG by ULONG, you do not "unbreak" such software magically. Again, if this would be a vivid platform, you would hunt down authors and tell "fix this!" but we don't have that luxury.

[meynaf]

Originally Posted by Thomas Richter

It is too late to fix it. Software was designed/compiled with the header in place. If you just replace LONG by ULONG, you do not "unbreak" such software magically. Again, if this would be a vivid platform, you would hunt down authors and tell "fix this!" but we don't have that luxury.

I think we're "vivid" enough here to be able to fix a few software, especially due some of us don't even need the source code to be able to do this. IOW even if authors can't be contacted or don't want to budge, we're not locked.

[SpeedGeek]

Originally Posted by ShK

How to move AmigaOS 3.1 "expansion.library" into fast memory without MMU?

Remapping the ROM with various tools (e.g. Blizkick, GvpCpuCtrl) already does this. But having the expansion.library in Fast memory is not the same as having the Expansion base in Fast RAM.

Originally Posted by Thomas Richter

Why? The expansion.library is only used to identify and configure the hardware, and not any time further.

To answer your question, you cannot. The purpose of the library is to find autoconfiguring RAM, and for that reason, it cannot be in the RAM it is supposed to configure. Exec is (re-)constructed after expansion has run, and thus, if autoconfig RAM is present, is moved to fast RAM.

I generally agree with this point, but there were a few options that were not considered for the Expansion base. $C00000 memory (AKA Slow-Fast memory) and MEMF_LOCAL memory. Even though these memory types are not available on every system, they could be used to reduce the Chip memory used by Expansion base.

BTW, if you have a system with a lot of Expansions installed it could use more than 1 KB of RAM.

[Thomas Richter]

Originally Posted by SpeedGeek

I generally agree with this point, but there were a few options that were not considered for the Expansion base. $C00000 memory (AKA Slow-Fast memory) and MEMF_LOCAL memory. Even though these memory types are not available on every system, they could be used to reduce the Chip memory used by Expansion base.

Err... if MEMF_LOCAL or ranger-mem is available, expansion *will* be in that type of RAM. As said, expansion will be allocated with MEMF_PUBLIC, and thus ends up in whatever public memory is available. That is only MEMF_CHIP in worst case.

[SpeedGeek]

Originally Posted by Thomas Richter

Err... if MEMF_LOCAL or ranger-mem is available, expansion *will* be in that type of RAM. As said, expansion will be allocated with MEMF_PUBLIC, and thus ends up in whatever public memory is available. That is only MEMF_CHIP in worst case.

MEMF_LOCAL is typically configured by expansion.library. So, not available for Expansion base as MEMF_PUBLIC. IIRC Ranger-mem is configed by exec.library so it would available, but I don't remember if it was assigned a higher priority then MEMF_CHIP?

[Thomas Richter]

Sure, it is.

[ShK]

Originally Posted by SpeedGeek

Remapping the ROM with various tools (e.g. Blizkick, GvpCpuCtrl) already does this. But having the expansion.library in Fast memory is not the same as having the Expansion base in Fast RAM.

Yes, indeed e.g., SKick can move "expansion.library" to Fast memory when kickstart ROM is soft-kicked there, but LibBase is what meant to have in Fast RAM. What I've learned now from the thread, it's not even possible.

[Don_Adan]

Originally Posted by Thomas Richter

The limit is actually 30 characters, and personally never run into a particular problem with 30 characters. Volume, assign and device names are still limited to 30 characters. The limit is actually per component, the maximum path lenth is actually 255 characters.

Seems You never try to use any CD which used Joliet standard on Amiga.
It has over 60 signs limit, if I remember right.
Like mentioned meynaf it can crash some old programs, because only 30 bytes for names was reserved for file name.
Other old programs (not crashed) only readed max 30 signs.
And You can see something like this "file not found" or "cant load file".
MP3 files very often has very long file names, over 30 signs.
Then really -1 value as file length, can be only very small problem. If it can be problem.
Files greater than 2GB can be useful mostly for images (DVD, HD etc) or for movie (mostly uncompressed, light compressed etc).

[Firestone]

Originally Posted by patrik

For the Blizzard1230, given that it is a 030, the highest performing system would be achieved by:
1. BlizKick ROM
2. FastExec
3. Disable mmu.library

FastExex does the same as MuMove4K and MuFastZero - move the library base of exec.library to fastmem and optionally move SSP too - this is what gives the snappy feel, but without using the MMU. Not enabling the MMU on the 030 (with a fairly complicated config) will give you anywhere from 10-20% extra performance when doing things that uses some memory bandwidth, like networking, extracting archives etc.

This is interesting, patrik.
In this case, I guess "FastExec" is this one? : https://aminet.net/package/util/boot/FastExec30

Can you please elaborate about "Not enabling the MMU on the 030 (with a fairly complicated config) ?

Or is this just a matter of removing mmu.library and the 68k-libs from LIBS: ? And of course any MuTools from startup.

EDIT: Removed mmu.library and I tried the Blizkick approach again:

C:Blizkick devs:kick32 QUIET
C:FastExec FASTSSP FASTEXP

System kicks 32 and then reboots as normal.
Second boot, it will boot to workbench, but residentspeed tells me that exec.library and expansion.library is still in chipram.
On third reboot, everything is in place.

I could add a 'REBOOT' to the FastExec command, but did you ever achieve to do this with only one reboot?