AmigaOS 3.3 Feature Request [Shutdown]

[ljmarent]

This may not be the best place to post a new feature request, but the Hyperion site has broken the user account file and denied many users posting privileges.
So, I'm going to throw the idea up here and ... maybe someone will see it.

The idea is simple .. we need a workbench [shutdown] option.

The reason why is NAND flash... specifically the cheap NAND flash storage without asynchronous power fail protection that 90%+ of Amiga users choose to buy ... oh .. and USB thumb sticks...

I know that "The Amiga Way" was to wait for the HDD LED to go out and hit the KA-CHUNK switch to power off ... and in the day of spinning rust media .. that was perfectly valid.

Today .. we have a ton of new accelerators which are getting populated with cheap NAND flash storage solutions, not spinning rust, and users are not opting to pay for enterprise storage, with last-gasp holdup caps..

What can and does happen is.... You wait for the HDD LED to go out.. and you flip the power switch... the SSD was a cheap pSLC cache TLC SSD and it had just begun background operations or BKOPS and was copying the pSLC cache to the TLC array and you killed the power... Now on power-up... the SSD will notice the FTL is corrupt and refuse to respond to the host while it's rebuilding the FTL from a trail of bread crumbs.. This takes time.. but Kickstart is not injured SSD aware, so it skips the non-responding SSD and tries to boot elsewhere... the user thinks .. hey this is weird and keeps power cycling the computer.. Eventually the SSD gets rebooted so many times trying to repair itself that it becomes corrupt.

There are other such NAND flash storage corruption mechanisms..
Many involve BKOPS just doing its thing and BLAMO! someone kills the power.
I've only given one such common example above.

There are people in the community that might say 'shutdown' is a driver issue or a storage vendor problem and shutdown is not an OS specific function..
To them I would say:
- Drivers have a strict policy of refusing to work during power failure.
- The ATA storage vendors have provided the [Standby Immediate] command, we just have to take advantage of it.
- If the OS has no responsibility for Shutdown... then we should remove the Startup scripts as well..

What I think would help is a [Shutdown] menu item that sends the [Standby Immediate] command to the ATA devices on the computer and then pops up a window that says "You're computer is safe to power off"

It would also be nice.. someday.. to flush and unmount any USB storage devices, but one step at a time.

I realize the Amiga is different from other computer systems.. but I feel like many of us are modernizing it (look we're running OS 3.2.2.) and using present day hardware .. maybe it's time to implement [shutdown] or at least a shutdown script to assist where the new hardware really needs it.

[Steady]

Thanks for your suggestion @ljmarent.

I will add it to our list for consideration to see how this might be done. The example is helpful and not something I was personally aware of (though the other devs might be).

[Steady]

BTW, a good place to ask such questions is https://developer.amigaos3.net/forum

[derSammler]

Sorry, but I don't see the problem. First, how does a shutdown function protect from a power failure?! Second, the Amiga OS, or rather the file system, does not use any sort of caching. There is nothing to flush to make the file system "clean".

Also, what you describe does not even happen on a PC when power fails. What you describe is a very, very theoretical issue that is less likely to happen than winning the lottery.

Finally, it should not at all the OS' task to make sure that some dirt-cheap flash storage does not get corrupted.

[Jope]

Originally Posted by derSammler

Sorry, but I don't see the problem.

It was explained in the opening post.

First, how does a shutdown function protect from a power failure?!

It doesn't, but that was not implied. The opening post also explains what happens during power failures.

Second, the Amiga OS, or rather the file system, does not use any sort of caching. There is nothing to flush to make the file system "clean".

If you had read the opening post, you would realise that this proposed os functionality has nothing to do with file system caches.

Also, what you describe does not even happen on a PC when power fails.

It can happen with flash storage on any system when power is cut suddenly.

What you describe is a very, very theoretical issue that is less likely to happen than winning the lottery.

On an Amiga a power failure happens every time you turn it off because the flash device isn't notified. Which was explained in the opening post.

Finally, it should not at all the OS' task to make sure that some dirt-cheap flash storage does not get corrupted.

You would be surprised at how expensive some of these dirt-cheap flash storages are. It does not only cover china junk, but consumer grade SSDs in general.

Most consumer SSDs have a SMART attribute for this. Here's mine from my creaky home server:
192 Unsafe_Shutdown_Count 0x0012 100 100 000 Old_age Always - 74

My power has been quite stable over the years, thankfully.

[drHirudo]

Yes, shutdown, even if not real shutdown, like shutdown from the power switch is still useful.
On my Macintosh Performa 450, when I shutdown MacOS 7.X, it says at the end that it's safe to turn off, or I can restart with click of a button.
If I don't do the procedure of shutdown from the menu item, or the button on the keyboard, I get the annoying message with the next boot of MacOS that I need to shutdown. And some validation probably happens in the back.

On AmigaOS 4.1 I am very happy to have the shutdown menu which turns off the machine as well. One very advanced feature that AmigaOS got it only in late 2010-ies.

[ljmarent]

Originally Posted by derSammler

Sorry, but I don't see the problem. First, how does a shutdown function protect from a power failure?!

It helps by telling the NAND flash storage that a shutdown is coming:
- write the FTL down
- stop writing data
By sending [Standby Immediate], we are just placing the NAND flash controller in the crash position.

Originally Posted by derSammler

Second, the Amiga OS, or rather the file system, does not use any sort of caching. There is nothing to flush to make the file system "clean".

You are talking about data in flight.
I'm talking about data at rest from the hosts perspective.
SSDs are quite busy internally when you are not actively accessing them.
BKOPS are happening inside the NAND flash storage device that you cannot observe.
Giving the controller some warning that you are about to kill the power makes a huge difference.
By halting BKOPS, you are helping the NAND flash storage controller to preserve your data at rest integrity.

Originally Posted by derSammler

Also, what you describe does not even happen on a PC when power fails. What you describe is a very, very theoretical issue that is less likely to happen than winning the lottery.

I'm going to 'suggest' that you've not worked at a company that manufactures and sells NAND flash storage devices, their field service department or their RMA department.
Nor have you been involved in third party client NAND flash storage testing.

Client grade NANDflash storage corruption due to power failure is extremely common, it is far from a "lottery".
A lot of things have been added to your PC OS \ BIOS to help mitigate this.
Corruption is not as common on PC's today 'exactly because' we have a shutdown warning and BIOS's that wait for an injured SSD to recover on boot.
So, we don't get into doom loop scenarios as easily on PC's, but even with those improvements, power fail corruption is still an issue, just a mitigated one.
Which the Amiga.. today... does nothing to mitigate.

Originally Posted by derSammler

Finally, it should not at all the OS' task to make sure that some dirt-cheap flash storage does not get corrupted.

Is it the degree of dirtiness or the degree of cheapness that you are most concerned with here?
The reality is that 'common' client grade NAND flash storage devices from reputable vendors can be corrupted by repeat power failure, which is the same thing as switching off the power switch on an Amiga.
The OS is the BEST place for us to broadcast to all IO subsystems to prepare for power loss and prevent data corruption.
Making it safe to throw the power switch.

My final notes on all of this are:
Why should the Amiga OS not be a leader in taking full advantage of NAND flash storage technology ?
Why choose to voice an objection to adding a common thing like:
- ATA Power Fail Warning [Standby Immediate]
- USB flush and auto-unmount
- Heaven forbid I ask for TRIM support
These sorts of things are OS level and help to reduce failure and user frustration.
Which is another way of saying.. these things increase user happiness... how often do we get a chance to do that ?

[daxb]

Originally Posted by ljmarent

The OS is the BEST place for us to broadcast to all IO subsystems to prepare for power loss and prevent data corruption.

I would think that the best place is on the device itself or the controller. I.e. on hardware level. So a power off will never harm the storage device.

- ATA Power Fail Warning [Standby Immediate]
- USB flush and auto-unmount
- Heaven forbid I ask for TRIM support
These sorts of things are OS level and help to reduce failure and user frustration.

How does it work? Is there a standard API or similar that would work on Amiga systems? Is it hardware dependent (I would guess yes)?

[ljmarent]

Originally Posted by daxb

I would think that the best place is on the device itself or the controller. I.e. on hardware level. So a power off will never harm the storage device.

1) There are no enterprise grade Amiga storage controllers.

2) Unless you choose to pay for enterprise grade NAND flash storage solution, you're not getting this functionality on the device itself..

Amiga users see $20 256GB client SSDs and maybe out of the corner of their eye they spot the $150 256GB enterprise SSDs and guess what runs through their head ?
"They are trying to rip me off"
Almost no one is up-to-speed on the differences between client and enterprise NAND flash storage products.
We're not going to educate our way out of this, because the cost disparity is too great.

On other computer systems, to a large extent, users don't need to be concerned.
When the BIOS and OS are NAND flash storage aware, client grade devices can get along well enough.

On the Amiga .. no...

I ask you in return, why should the AmigaOS, in active development in 2024 remain a BIOS / OS which is completely unaware of, and lacking any support for NAND flash storage ?
There are active devs.. and this is a feature that will reduce frustration and unhappiness, without breaking in any way what it is to be Amiga..

If you use spinning rust or enterprise NAND, go ahead and throw the power switch like you always did.
Just now .. there would be a Shutdown option in the Workbench menu that can prepare the computer for power off..

And then we can talk to the AmigaOS devs about 'maybe' supporting shutdown triggered flush and unmount for SD cards and USB thumb sticks and heaven forbid ... trim support..

We could actually start supporting NAND flash storage, like other actively developed OS's do and maybe .. we could even do it better ?

If only ... I don't know ... maybe someone who is an insider at a major NAND flash and storage manufacturer,
could guide interested devs to tech-notes covering known issues and work arounds and answer their questions.
Though .. surely no one like that would ever appear ...
And I tell you.. if someone like that ever did appear..
She\he would likely be argued 'to death', by people that don't work in the NAND flash storage industry,
and 'choose' to ignore how other systems have dealt with the NAND flash storage challenges .. at the BIOS \ OS level.
But that's just a hunch..

Originally Posted by daxb

How does it work? Is there a standard API or similar that would work on Amiga systems? Is it hardware dependent (I would guess yes)?

ATA has a ready [Standby Immediate] command that you send and the storage devices will make preparation for power-loss.

The Ideal Solution:
"STANDBY IMMEDIATE (E0), and wait for the output.
If that outputs an ABRT, wait a second, then retry.
Then, I would issue SLEEP (E6).
If that outputs an ABRT, the drive does not support SLEEP, and you should wait 3 seconds, then power off.
If the output is successful, wait for the INTRQ line to be asserted.
Then, read the ATA status register (to clear the INTRQ).
When the INTRQ clears, it is safe to power off."

The Good Enough Solution:
Just issuing STANDBY IMMEDIATE, stopping all IO, and wait 3 seconds
Would do most of the heavy lifting.

It's the ATA standard and ATA devices will comply, so it's not something I'd call storage device specifc, no.

There could be storage controllers that don't pass the [Standby Immediate] command to the drive or response to the driver.
Once we are making the requests at the OS level, we'd find out which controllers do not support this, if any, and could ask those hardware devs to help, if they are still around.
If a few ATA host controllers cannot be updated to support this .. that's unfortunate but it does not make the situation worse, in the way that doing nothing does over time.

[meynaf]

Should AmigaOS destroy one of its advantages just to support stupidly designed hardware from the PC world ?
After all, mechanical hard drives have been able to park their heads at power off without any special command, since very long. And one should be able to unplug an USB stick without unmounting it first.
A separate program doing what is asked here could be fine. But by pity, not in the OS itself.

[cloverskull]

I think this seems like a great idea with zero downsides. I don't understand all the pushback. For many people, this is a home run. For other people, it would be a feature that can easily be ignored forever. *shrug*

[meynaf]

Yeah, sure. Make it look like win95.

[Dunny]

Originally Posted by meynaf

Should AmigaOS destroy one of its advantages just to support stupidly designed hardware from the PC world ?

I guess if it's implemented you have two options. You can use the menu item (or however it's implemented) to send the shutdown signal to anything attached that needs it and gracefully shutdown.

Or you could just pull the plug anyway. I mean, there's literally no way anything bad could happen, is there?

[meynaf]

Originally Posted by Dunny

I guess if it's implemented you have two options. You can use the menu item (or however it's implemented) to send the shutdown signal to anything attached that needs it and gracefully shutdown.

If it's in a separate app, this is all right. Put it in wbstartup and it adds an option in tools menu. No need to touch the OS.

Originally Posted by Dunny

Or you could just pull the plug anyway. I mean, there's literally no way anything bad could happen, is there?

Up to recently, nothing could. Why should this change ?

[DisasterIncarna]

sounds like the procedure for the a590, needing to park the heads before turning off the machine.

[Aulapatchuc]

[QUOTE=meynaf;1678461
Up to recently, nothing could. Why should this change ?[/QUOTE]

Up to recently nobody used SSD or SD cards on their Amigas, now they use. So why not to change? This is a particularly useful and sensible request. If implemented, you could simply ignore and continue to shutdown your Amiga on the power button, just like in 1988. Nothing changes for you.

[meynaf]

Originally Posted by Aulapatchuc

Up to recently nobody used SSD or SD cards on their Amigas, now they use. So why not to change? This is a particularly useful and sensible request. If implemented, you could simply ignore and continue to shutdown your Amiga on the power button, just like in 1988. Nothing changes for you.

And one day SSD will do this in hardware just like mechanical HD did before, leaving you with a useless option. If this is a separate program, you can just delete it and forget about this hiatus.
I'm not against this feature. I'm against putting it at the wrong place.

[vulture]

Well, it can be optional if that's the problem. But it does make sense right now to have it as a choice.

[daxb]

I just asked because in theory it sounds good and easy but I fear that in praxis it isn't. Then we have so many different systems (hardware and software configurations) that I also would suggest to have a separate tool the cover all or the most of it. Maybe a small hardware device between IDE port and storage device would be better if possible.

For filesystems you have this: http://aminet.net/package/disk/salv/DiskSafe

[Dunny]

Originally Posted by daxb

I just asked because in theory it sounds good and easy but I fear that in praxis it isn't. Then we have so many different systems (hardware and software configurations) that I also would suggest to have a separate tool the cover all or the most of it.

I guess if there's a modern standard signal that modern flash drives will respond to, then making use of that would be fine?