AmigaOS 3.2 released!

[Thomas Richter]

Originally Posted by Don_Adan

Im sure that some exec LVO's set bit 31 as error, not -1.

Yes, AllocEntry() does. Doesn't change the fact that existing programs check on bit 31 for an error because this is how the interface looks like.

Originally Posted by Don_Adan

And yes, Read64, Write64 etc is cleaner solution than adding ID.

Cough. How can one write more than 4GB requiring a 64 bit length in a system with at most 2GB?

[OlafSch]

Originally Posted by mschulz

I know this is a chicken and egg problem. And even if dos and filesystems get necessary improvements the question remains - will there ever be a software written to use this new features. Regarding structures - yes, I'm aware of that. But, considering all this, maybe it would be worth looking at OS4/MOS/AROS how they did that?

As I understand it the amigaos updates are for preserving the platform. Lowering compatiblity means breaking software without new software developed (or at best few) that makes use of new features

[Don_Adan]

Originally Posted by Thomas Richter

Yes, AllocEntry() does. Doesn't change the fact that existing programs check on bit 31 for an error because this is how the interface looks like.

Yes, but for error handling some code not use only:

 
 btst #31,D0
 bne.b error

but

 
 tst.l D0
 bmi.b error

When the best is

 
 cmp.l #-1,D0
 beq.b error

because all other values (except -1) can be correct, not only values from 0 to $7fffffff are correct.

Cough. How can one write more than 4GB requiring a 64 bit length in a system with at most 2GB?

Some Amiga filesystems perhaps can work with files up to 4GB, not only with 2GB. Perhaps 3GB file can be visible, but works fully as video file for 2GB length. I can try to find this thread on PPA, if You want.

[Thomas Richter]

Originally Posted by Don_Adan

When the best is

 
 cmp.l #-1,D0
  beq.b error

because all other values (except -1) can be correct, not only values from 0 to $7fffffff are correct.

Actually, the best would be to simply use a test for zero, but that doesn't change the fact that the interface is broken to begin with and we cannot just change it for something else without breaking compatibility. Note again that there are programs out in the wild that check for a negative result (and not -1), and thus would fail if AllocEntry() would return a struct MemEntry in "upper memory". AllocEntry itself could work around that by placing the struct MemEntry forcefully in "lower memory".

Originally Posted by Don_Adan

Some Amiga filesystems perhaps can work with files up to 4GB, not only with 2GB. Perhaps 3GB file can be visible, but works fully as video file for 2GB length. I can try to find this thread on PPA, if You want.

The problem is to a lesser degree the file system end of it. The problem are user programs that read the "struct FileInfoBlock" and take the file size exactly as it is indicated in this structure, a "LONG". The interface is broken, and it is too late to fix that.


Consider

printf("Size of file %s is %ld\n",fib->fib_FileName,fib->fib_Size);

Such code very likely exists, and it is fitting perfectly to the interface we have right now. Still, it would print nonsense for files larger than 2GB.


Certainly, fib_Size should have been defined as ULONG, but BCPL is a typeless language, and it only knows (signed) integers (which are also pointers... or rather indices into its memory model of an array of LONGs).

[Don_Adan]

Originally Posted by Thomas Richter

Actually, the best would be to simply use a test for zero, but that doesn't change the fact that the interface is broken to begin with and we cannot just change it for something else without breaking compatibility. Note again that there are programs out in the wild that check for a negative result (and not -1), and thus would fail if AllocEntry() would return a struct MemEntry in "upper memory". AllocEntry itself could work around that by placing the struct MemEntry forcefully in "lower memory".



The problem is to a lesser degree the file system end of it. The problem are user programs that read the "struct FileInfoBlock" and take the file size exactly as it is indicated in this structure, a "LONG". The interface is broken, and it is too late to fix that.


Consider

printf("Size of file %s is %ld\n",fib->fib_FileName,fib->fib_Size);

Such code very likely exists, and it is fitting perfectly to the interface we have right now. Still, it would print nonsense for files larger than 2GB.


Certainly, fib_Size should have been defined as ULONG, but BCPL is a typeless language, and it only knows (signed) integers (which are also pointers... or rather indices into its memory model of an array of LONGs).

I dont know if it can be changed/fixed without breaking compatibility in clean or smart way.
But it seems that files up to 4GB works under OS 3.9.

I find this thread about files over 2GB.

https://www.ppa.pl/forum/emulacja-na...rtycja#m766064

Post 6. and 20. mostly.

[Thomas Richter]

Originally Posted by Don_Adan

But it seems that files up to 4GB works under OS 3.9.

It depends on what you consider "working". Consider the following code segment on such a file:

ok = Seek(file,0x8000ffff,OFFSET_CURRENT);

Now, what exactly is this supposed to do on a large file? Seek to the current file pointer forwards by 0x8000ffff, or seek backwards by 0x7fff0001 bytes? What happens if we replace OFFSET_CURRENT by OFFSET_BEGINNING or by OFFSET_END?


I do not know what the FFS attempts to do if such requests come in. I doubt it is doing "the right right" (whatever this may be).

[nogginthenog]

Hey guys, can you create a new thread for this please?

[Thomas Richter]

Originally Posted by nogginthenog

Hey guys, can you create a new thread for this please?

Why? I think it is pretty central to future AmigaOs development and fits quite well.

[A4000Bear]

Originally Posted by Thomas Richter

That does not prove anything, actually. The ROM neither explains why the board crashes when then machine is already up as at this time, expansion has already done its job. Thus, all bets are open. I still believe some timing is off as the components are not suitable for 50Mhz and this adds together with an older "chinese recycled" EPROM that may have some timing issues by itself.

Again, to make this an issue, report on a *real* 50Mhz board with components qualified to 50Mhz.

I'm using an MX27C4100, one of a batch I purchased about 15 years ago, long before Chinese fakes became a problem. Indeed, I have used them for Kickstarts in many machines with complete reliability.

Not sure if I already mentioned it, but even with a genuine 3.1 ROM installed, the kickstart modules will load from disk OK, but will crash shortly after the reboot, so it can't be the ROM.

The board has a real 50MHz 030 installed, and all PAL chips are identical, including their speed ratings to the various original 50 MHz boards I have seen. I have even (temporarily) tested this board on other versions of Amiga OS, at 54MHz, with complete reliability. Having said that, I would love to hear from anyone who has confirmed operation of any version of 3.2 on an originally purchased 50MHz board, which is the reason why I posted in the first place.

[Rotzloeffel]

Originally Posted by A4000Bear

Having said that, I would love to hear from anyone who has confirmed operation of any version of 3.2 on an originally purchased 50MHz board, which is the reason why I posted in the first place.

My Betatest machine is an a500 with a 50 MHz derringer 030 and it works perfekt with a lot of 3.2 Versions since year

[Olaf Barthel]

Originally Posted by Thomas Richter

It depends on what you consider "working". Consider the following code segment on such a file:

ok = Seek(file,0x8000ffff,OFFSET_CURRENT);

Now, what exactly is this supposed to do on a large file? Seek to the current file pointer forwards by 0x8000ffff, or seek backwards by 0x7fff0001 bytes? What happens if we replace OFFSET_CURRENT by OFFSET_BEGINNING or by OFFSET_END?


I do not know what the FFS attempts to do if such requests come in. I doubt it is doing "the right right" (whatever this may be).

Hm... I see the utility, but the FFS is likely not the best foundation for rework right now.

The future was already there in AmigaOS4 It features dos.library APIs which offered additional functionality for use with 64-bit offset random access in the form of GetFilePosition() and SetFilePosition(), both of which are modeled on the ANSI 'C' fgetpos() and fsetpos() functions, respectively. It also features a GetFileSize() function which augments what the Examine/ExNext/ExAll APIs for Kickstart 2.04 and beyond accomplish.

Before such APIs can become useful, the file systems which could benefit from them would have to be reworked. The FFS is, because of its Tripos legacy, making use of only the simplest data structures and algorithms known in 1978 (if the copyright information for the original ROM file system is correct). This means: tables, singly-linked lists, bitmaps and (the peak of algorithmic sophistication) hashing with separate chaining.

The FFS has no dedicated directory data structure and merely reuses the metadata blocks of the directory entries. Managing the storage involves the use of singly-linked lists which for files contain a list of all the data blocks which are part of the file. This bites for large files because in order to perform random access, you first need to travel the entire list of blocks which constitute the file until you find the block the file system should use for read/write access.

While, in theory, the file system data structures should scale like crazy, they being signed 32-bit integers, you can't work around the fact that there can be only up to 2^31 blocks (really: blocks, not sectors) in use by a volume. That means 2,147,483,648 blocks, and if you're using 512 bytes per block, you get to use up to 1 TByte per partition, ain't that nice for such an old dog to know such tricks What's the catch? For 512 bytes per block the block allocation bitmap would have to be some 268 MBytes in size on disk and, it's a bad habit for sure, the FFS likes to keep the entire block allocation bitmap in memory at all times. While you could use larger block sizes, to reach even greater partition size limits, the larger granularity has a price, too, but at this scale you probably don't get into too much trouble so quickly.

If this sounds like fun, consider this: many file system operations involve changing 2-3 blocks at a time in order to complete a task such as extending the size of a file you are writing to. Such as: allocate space for the data to go into, allocate space for another file list block, add that file list block to the file, update the file list block to reference the data block just allocated, write the data to the new block, update the file header block, write back the block allocation bitmap. Bonus round: update the dircache list for the directory which the file is stored in. The FFS performs these steps in a certain order and does not flush (!) the write cache at crucial points. That makes it fast, but if the operating system or the file system crashes at an inopportune moment, the file system data structures will be damaged and inconsistent.

The cherry on top is that the FFS does not have a well-defined strategy for allocating storage space and may have to walk the entire block allocation bitmap to find space for something to fit, somehow in the proximity of the last data block. This is why the FFS prefers to keep the bitmap in memory, since the alternative tends to be too slow.

This the story so far: if you want to be able to store more than 4 GBytes of data on an FFS volume, then the FFS better be good at managing data and metadata at the scale this invariably requires. This is not possible yet. Directory scanning is still dog slow and will become slower the more directory entries need to be squeezed into the hash tables. Finding available storage in close proximity is very slow. Managing the data blocks associated with a file is very slow and wastes a lot of memory (at 512 bytes per block a 2 GByte file needs 30 MBytes worth of file list blocks to account for every single data block). Making changes to the file system structure can be derailed quickly and restoring the consistency can take a very long time indeed.

All of this is solvable by either replacing data structures and algorithms or by augmenting the existing data structures (and can be kept backwards-compatible if you want it to, but it will cost you). But you probably won't catch me doing that because the FFS in its 68k assembly language form is beyond my means. I prefer the 'C' version for which it could happen, though. I wrote a 'C' FFS between 2001-2006 which is used by both MorphOS and AmigaOS4.

[Don_Adan]

Originally Posted by Thomas Richter

It depends on what you consider "working". Consider the following code segment on such a file:

ok = Seek(file,0x8000ffff,OFFSET_CURRENT);

Now, what exactly is this supposed to do on a large file? Seek to the current file pointer forwards by 0x8000ffff, or seek backwards by 0x7fff0001 bytes? What happens if we replace OFFSET_CURRENT by OFFSET_BEGINNING or by OFFSET_END?


I do not know what the FFS attempts to do if such requests come in. I doubt it is doing "the right right" (whatever this may be).

I think that new AmigaOS can has 32 and 48 (or 64 if this better) bit modes.
All new compiled/assembled programs can call 48 bit mode too.
Old programs will be using original 32 bit mode.
For 48(64) bit modes 2 registers are necessary.
Where is necessary similar code can be added to Amiga LVO's:

 cmp.w #"48 b",Dx ; check for ID
 bne.b 32bitMode
 swap Dx
 ext.l Dx ; ready to use first 32 bits
....
32bitMode
 moveq #0,Dx
....

[SpeedGeek]

Originally Posted by A4000Bear

I'm using an MX27C4100, one of a batch I purchased about 15 years ago, long before Chinese fakes became a problem. Indeed, I have used them for Kickstarts in many machines with complete reliability.

Not sure if I already mentioned it, but even with a genuine 3.1 ROM installed, the kickstart modules will load from disk OK, but will crash shortly after the reboot, so it can't be the ROM.

The board has a real 50MHz 030 installed, and all PAL chips are identical, including their speed ratings to the various original 50 MHz boards I have seen. I have even (temporarily) tested this board on other versions of Amiga OS, at 54MHz, with complete reliability. Having said that, I would love to hear from anyone who has confirmed operation of any version of 3.2 on an originally purchased 50MHz board, which is the reason why I posted in the first place.

I don't think he was referring to the Kickstart ROM. Rather it was the GVP driver ROM. He thinks all the components on the G-Force 030 must be "Certified" for 50 MHz operation.

He doesn't understand that there are only 3 components on this Board which actually have a MHz rating - the CPU, FPU and oscillator. The other components have either access time (ROM and RAM) or switching speed (PAL and TTL devices) ratings. The DPRC was designed for 7 MHz operation but it's capable of running @ 14MHz on certain GVP products (G-Force 040 and the A1291).

He doesn't understand that your Board is now 50 MHz "Qualified" just as much as any factory produced Board ever was or ever will be.

NOTE: The PAL and TTL devices do have maximum clock speed ratings in registered output mode which is >= 100 MHz for these particular components.

[SpeedGeek]

Originally Posted by Rotzloeffel

My Betatest machine is an a500 with a 50 MHz derringer 030 and it works perfekt with a lot of 3.2 Versions since year

According to the Derringer manual it has no on-board Auto-Config devices. The optional Fast RAM is configured by a Software tool.

This points the finger right at the OS3.2 expansion.library which I already suggested replacing.

[Thomas Richter]

Originally Posted by SpeedGeek

He doesn't understand that there are only 3 components on this Board which actually have a MHz rating - the CPU, FPU and oscillator.

And that is exactly not correct. All the components between the CPU and the bus interface and the memory also need to keep up with the CPU and have to satisfy the tighter timing requirements of a 50Mhz clock. The PALs and GALs and the MACH chips on the board in particular. I do not know how GVP selected components and qualified boards, but it is not untypical to first build the boards from all the same components, and then run a burn-in test to select those boards that run stable at 50Mhz for selling them at 50Mhz, and sell the others at lower clock rates. This is at least how CPU vendors act.

[SpeedGeek]

Originally Posted by Thomas Richter

And that is exactly not correct. All the components between the CPU and the bus interface and the memory also need to keep up with the CPU and have to satisfy the tighter timing requirements of a 50Mhz clock. The PALs and GALs and the MACH chips on the board in particular. I do not know how GVP selected components and qualified boards, but it is not untypical to first build the boards from all the same components, and then run a burn-in test to select those boards that run stable at 50Mhz for selling them at 50Mhz, and sell the others at lower clock rates. This is at least how CPU vendors act.

Yes, it is quite correct. I just explained that the PALs and TTL components on 40 and 50 MHz Boards are the same and more than fast enough to keep up. The other components (including the MACH130) don't need to keep up since they don't run from the CPU clock.

[fgh]

IIRC Robert Miranda (aka thebajaguy, ex GVP employee) said that at least some GVP 030 boards for A2000 were not designed to be upgraded to 50mHz, even if 50mHz versions of the same board exist.
(Due to RAM timing, PAL ratings, ++)

[SpeedGeek]

Originally Posted by fgh

IIRC Robert Miranda (aka thebajaguy, ex GVP employee) said that at least some GVP 030 boards for A2000 were not designed to be upgraded to 50mHz, even if 50mHz versions of the same board exist.
(Due to RAM timing, PAL ratings, ++)

Thebajaguy was probably referring to the 25 MHz variant which in fact has a different PAL set. A4000Bear was using the 40 MHz variant which had exactly same PAL set as the 50 MHz variant. Please read his other posts reporting that his Board is now running stable @ 50 MHz. Thank you.

[Thomas Richter]

I only read posts that the board is not running stable in certain combinations. The components used on the board might be the same types, but the board did not qualify for 50MHz at GVP, as it was not sold as 50Mhz. This is not about the chip types put on the board, this is about the chip tolerances and variations that were already at the edge at 40Mhz, and are now kicking in.

In case you do not understand, the worst thing that can happen if expansion does not do its job is that expansions are not recognized. It does not explain instabilities as soon as the Os is running and expansion has done its job (or not). At that point, expansion is "out". This is not a software issue you are seeing here.

[Rotzloeffel]

Originally Posted by SpeedGeek

According to the Derringer manual it has no on-board Auto-Config devices. The optional Fast RAM is configured by a Software tool.

This points the finger right at the OS3.2 expansion.library which I already suggested replacing.

You are right, but you an not make the Ram resident anymore, system Crashes, that worked with 3.1