Creating a file in RAM: sets IoErr

[Krashan]

AmigaOS 3.0. When I open a file for writing in RamDisk with

Open("RAM:foo", MODE_NEWFILE);

and the file does not exist before, following

IoErr()

returns 205 (ERROR_OBJECT_NOT_FOUND) in spite of the file being created successfully.

It does not happen when I do the same anywhere on harddisk (FFS, if it matters).

IoErr()

is still 0 then.

Is it a bug?

[thomas]

Could be a bug. OS 3.0 is very old. But does it really matter? If Open() returns a non-zero value, it succeeded. You need to check IoErr only if Open returns null.

[hooverphonique]

As thomas said, you can't rely on IoErr() if there is no error (unless explicitly specified so by the operation in question).

[Thomas Richter]

Well, actually, you should, and actually, IoErr() should be 0 on success (unless documented otherwise). This is a defect of the RAM-handler.

[Bruce Abbott]

Autodocs 3.1: dos.doc

dos.library/IoErr

NAME

IoErr -- Return extra information from the system

SYNOPSIS

error = IoErr()
D0

LONG IoErr(void)

FUNCTION

Most I/O routines return zero to indicate an error. When this
happens (or whatever the defined error return for the routine)
this routine may be called to determine more information. It is
also used in some routines to pass back a secondary result.

Note: there is no guarantee as to the value returned from IoErr()
after a successful operation, unless to specified by the routine.

RESULTS

error - integer

[Thomas Richter]

Except that ACTION_FINDOUTPUT aka Open() shall create a 0 IoErr() on success. If it does not, it is a bug. There are functions in the dos.library which do not set IoErr(), or which set it to other values on success, but Open() is none of them.

[Olaf Barthel]

Originally Posted by Krashan

AmigaOS 3.0. When I open a file for writing in RamDisk with

Open("RAM:foo", MODE_NEWFILE);

and the file does not exist before, following

IoErr()

returns 205 (ERROR_OBJECT_NOT_FOUND) in spite of the file being created successfully.

It does not happen when I do the same anywhere on harddisk (FFS, if it matters).

IoErr()

is still 0 then.

Is it a bug?

No, as far as I can tell. It is a by-product of how the ram-handler implementation is designed, which focuses on code reuse in order to keep its size "manageable" (for some values of manageable, anyway).

When you call

Open("RAM:foo", MODE_NEWFILE)

, ram-handler will check if there already is an object named "foo". If it's a file, then checks will follow to determine whether it can be overwritten (it could be write-protected, a Lock or FileHandle may be associated with it). If it's a directory, then it can never be overwritten by a file. That sort of thing...

But it all begins by checking if there is a directory entry of that name. ram-handler reuses the code which underpins the

Lock()

operation, among other things, and this function will put an error code of ERROR_OBJECT_NOT_FOUND into a global variable if there is no directory or file of that name.

This global variable value will persist when the

Open("RAM:foo", MODE_NEWFILE)

call returns, even if it succeeds, and its value will go into the secondary result member of the DosPacket which underpins the file system interaction. That value will become available to you when you call the

IoErr()

function.

That said, you should be aware that the value returned by the

IoErr()

function is meaningless without context. Hence, if

Open("RAM:foo", MODE_NEWFILE)

successfully returns a pointer to a FileHandle, then the

IoErr()

function value is meaningless. It only becomes meaningful if

Open("RAM:foo", MODE_NEWFILE)

returns NULL.

There is some ambiguity here, I grant you. But for functions such as

Lock()

,

Delete()

,

Rename()

, etc. an indication of failure means that an error code will be available through

IoErr()

. This is a protocol or contract, if you will.

[Thomas Richter]

Originally Posted by Olaf Barthel

No, as far as I can tell. It is a by-product of how the ram-handler implementation is designed, which focuses on code reuse in order to keep its size "manageable" (for some values of manageable, anyway).

No, Olaf, I beg you. This is an implementation detail that leaks outside, and that should not leak outside. If an operation via a packet succeeds, the Res2 of the packet shall be zero, as simple as that. There are unfortunately a couple of dos.library functions that do not set IoErr consistently in some cases, but that's also something I would call a defect that should better be fixed.

[Olaf Barthel]

Originally Posted by Thomas Richter

No, Olaf, I beg you. This is an implementation detail that leaks outside, and that should not leak outside. If an operation via a packet succeeds, the Res2 of the packet shall be zero, as simple as that. There are unfortunately a couple of dos.library functions that do not set IoErr consistently in some cases, but that's also something I would call a defect that should better be fixed.

I already checked what it takes to make Result2 consistently come up with 0 if an operation succeeds. It makes ram-handler significantly larger (this has to go into every single packet handling function), and we're already tight in terms of ROM space.

If I should stumble down this rabbit hole, I need to save space elsewhere. Right now, the only thing which I believe might be easy to axe is the setdosdate function in dos.library. What do you think? Is booting off a BCPL file system even possible in Kickstart 2.x and beyond?

[hooverphonique]

Originally Posted by Thomas Richter

Except that ACTION_FINDOUTPUT aka Open() shall create a 0 IoErr() on success. If it does not, it is a bug. There are functions in the dos.library which do not set IoErr(), or which set it to other values on success, but Open() is none of them.

That doesn't really matter much since the dos documentation explicitly specifies not to rely on IoErr when an operation returns success.

[Olaf Barthel]

Originally Posted by hooverphonique

All that doesn't matter since the dos documentation explicitly specifies not to rely on IoErr when an operation returns success.

The real problem is that "nobody" seems to read the documentation, and with good reason. It tends to be unhelpful and incomplete. Even the exceptionally well-researched "The Amiga Guru book" has its limitations, with errata unlikely to be published in the future

[Thomas Richter]

Originally Posted by Olaf Barthel

I already checked what it takes to make Result2 consistently come up with 0 if an operation succeeds. It makes ram-handler significantly larger (this has to go into every single packet handling function), and we're already tight in terms of ROM space.

Frankly, I doubt that this makes much of a difference. If the handler provides 32 packets (estimated), we need in worst case (actually, it is not that bad) 32x "if (result == ok) result2 = 0", which accounts to 32 branches (4 bytes) and resetting result2 32 times (~6 bytes), tihs accounts to 320 bytes estimated. Let's make this even 400. I doubt that it is *that* tight*.

Originally Posted by Olaf Barthel

If I should stumble down this rabbit hole, I need to save space elsewhere. Right now, the only thing which I believe might be easy to axe is the setdosdate function in dos.library. What do you think? Is booting off a BCPL file system even possible in Kickstart 2.x and beyond?

My original 1.2 boot disk booted fine under 3.2. But if you want to axe off something, I would look in the blib.c and all the BCPL emulation nonsense. This can run right into a software alert instead. The only thing remains are BCPL startup functions for the port-handler and "CLI" in createtask, and this should be moderately easy to get rid of either. At least, it would be more than 400 bytes you can save.

[Thomas Richter]

Originally Posted by Olaf Barthel

The real problem is that "nobody" seems to read the documentation, and with good reason. It tends to be unhelpful and incomplete. Even the exceptionally well-researched "The Amiga Guru book" has its limitations, with errata unlikely to be published in the future

That is exactly why I try to create something that is a bit more helpful and a bit more complete. Concerning IoErr(), the issue is that the original documentation does not document much, and there are plenty functions in the dos.library that do not set IoErr consistently, but I also tried to document these cases, though I would rather prefer to mark these as defects rather than as features.



But it is a long story, and it takes a while to review. I'm now in the middle of the second iteration, checking every function and every description again, finding many defects, but nobody is perfect and certainly there are still errata left. Right in the middle of chapter 7 btw.

As always, suggestions for improvement are welcome.

[Olaf Barthel]

Originally Posted by Thomas Richter

Frankly, I doubt that this makes much of a difference. If the handler provides 32 packets (estimated), we need in worst case (actually, it is not that bad) 32x "if (result == ok) result2 = 0", which accounts to 32 branches (4 bytes) and resetting result2 32 times (~6 bytes), tihs accounts to 320 bytes estimated. Let's make this even 400. I doubt that it is *that* tight*.

I've been there before and the simple changes required to work around or replace existing ram-handler code rarely stay simple. The number of bugs/side effects which survive the previous attempt to squash them remains surprisingly constant. That said, the 'C' version of ram-handler had fewer bugs than the original version, but the new bugs were more complex

My original 1.2 boot disk booted fine under 3.2.

Yes, but it did not use a BCPL file system As far as I can tell, the only client of the setdosdate function is the Kickstart 1.x default file system. If the system time is 0 (unset), it will use the "Volume last altered date and time" DateStamp of the root directory of the boot volume to set the system time through setdosdate.

Fun fact: the "Volume last altered date and time" DateStamp does not contain the date and time of the last change made to the volume. It is more specific: the default file system updates it only when changes made to the bitmap are written back to disk. Basically, whenever storage space is released or allocated, the DateStamp will be updated, eventually.

But if you want to axe off something, I would look in the blib.c and all the BCPL emulation nonsense. This can run right into a software alert instead. The only thing remains are BCPL startup functions for the port-handler and "CLI" in createtask, and this should be moderately easy to get rid of either. At least, it would be more than 400 bytes you can save.

Some of the BCPL emulation nonsense is still being used and will impact the boot process for Workbench 1.x era disks, if broken. I discovered this the hard way when I introduced a memset() into the Examine/ExNext code of ram-handler which cleared the entire FileInfoBlock contents, including the reserved portion. Bad idea: the BCPL commands "Dir", "List", etc. use a much shorter form of the FileInfoBlock than is defined in even the Kickstart 1.0 "libraries/dos.h" header file, which results in corrupting the BCPL stack.

This is an undocumented trap for file system developers. Basically, the FileInfoBlock structure as defined by the 3.0 header files only works for 'C' and assembly language code. If the client happens to be a legacy BCPL program, you can expect only the fields fib_DiskKey..fib_Comment to be accessible. If you write beyond the end of the fib_Comment field in your file system code, you may trash the BCPL stack.

[Krashan]

Thank you all for extensive explanations. I have modified my code, so it only checks

IoErr()

when

Open()

fails.

[tygre]

PS.

Originally Posted by Olaf Barthel

The number of bugs/side effects which survive the previous attempt to squash them remains surprisingly constant.

Isn't that one of Lehman’s Laws?

[Olaf Barthel]

Originally Posted by tygre

PS.

Isn't that one of Lehman’s Laws?

Definitely sounds a bit like it. The recent rework (ahem) by yours truly was performed with the goals to render the code more readable, more maintainable and more robust, with no new features added and code size growing only in the low 20-100 bytes, if possible.

ram-handler is hard to maintain because the 'C' implementation is beholden to the design principles of the Kickstart 1.x BCPL version. Simple algorithms (not necessarily a bad thing), short functions which do the absolute minimum of what is required of them, terse variable and function names and code documentation which tends towards the decorative rather than the illuminating

You'd hope that this foundation would make it easier to spot something amiss, but your hopes would be dashed so very quickly. ram-handler is simple, if not primitive, because it has to deliver, with very little code spent on making it work.

The 'C' version uses function and variable naming a little better than the BCPL precursor, but it's often unhelpful because some data structure naming makes it hard to distinguish between the different types which, curiously, have very similar names. Part of my rewrite was to try and make the naming of things clearer so that you can tell from the name what you are dealing with. Without that foundation, maintaining the code becomes so much harder

ram-handler is big on code reuse for the sake of keeping the code size under control. This has its perks, but they come with a Spaghettic Code tax. Yes, it is shorter and possibly faster, but you may wonder at some point if it's too much of a distraction from finding and fixing the bugs which hide in loops of the Spaghetti.

And there were whole families of bugs hiding in plain sight, such as in the code which manages the memory allocation/free/resize operations. For example, due to a combination of bugs, closing a file just opened would almost always end up reallocating the memory spent on the last data chunk of the file, even if the file was unchanged. The code allocated new memory, copied the old data from the block to be replaced, then removed the old one, added the new one and freed the old block. Completely unnecessary, and not even a bit artful.

Figuring out how much memory was being used and how that would relate to the total amount of memory installed on the Amiga was mostly guesswork, with some of the maintenance operations squirreled away into subroutines where nobody would ever look for them.

As with all the file systems which came from the Tripos legacy, the ram-handler code rarely knew its own limitations, if at all. Not just file sizes, also name length for volumes and paths.

Code reuse for the benefit of keeping ram-handler small had curious side effects. A file lock and a file handle were basically the same data structure type and you could in some dark corners of ram-handler use both interchangeably. How's that for fun? Looks like a magic trick, except when it goes wrong.

I have some small hopes that the next person to walk into the current ram-handler code will no longer feel an uncanny urge to get the hell out of here before something slinky emerges from a ventilation shaft and grins without smiling...

[koobo]

Originally Posted by Olaf Barthel

ram-handler is hard to maintain because
I have some small hopes that the next person to walk into the current ram-handler code will no longer feel an uncanny urge to get the hell out of here before something slinky emerges from a ventilation shaft and grins without smiling...

That was a delightful read, thank you Olaf!