Any way to flush HDD buffers to disk?

jarp · 02 January 2017, 10:00

Hi! Is there any way I can flush all unwritten changes / buffers / whatever to the HDD before I shutdown or reboot OS3.x? I tried to google around and found pretty much nothing except "just wait few seconds after HDD led has stopped blinking" and on top of that Aminet seems to be down so I cannot search.

I am asking because after I got Vampire 600 it seems waiting few seconds no longer is enough. I need to wait few minutes(!!!) to make sure that everything gets written to the HDD. Not sure if this is a bug in Apollo Core or if it's just because of bigger memory available thus bigger buffers?

As end result I've been installing, re-installing, re-installing, configuring, re-configruring, re-configuring the same software again and again because NOTHING goes to the HDD if I do not wait for minutes. Quite bizarre experience, really. I also just experienced my first total HDD corruption, I bet it's because I rebooted on very wrong moment. Luckily I've got backups and it seems pfsdoctor was able to fix the problem also...

Thanks in advance!

Zeph · 02 January 2017, 10:40

Afaik there is nothing so complicated in AmigaOS?

Toni Wilen · 02 January 2017, 10:44

PFS3 should automatically flush all write cached buffers in few seconds after last write. It also adds reset handler (Keyboards send "reset warning" keycode and delay reset for max 10 seconds if it is handshaked properly, most other keyboards except A500 support it) that flushes all buffers immediately.

Zeph · 02 January 2017, 10:51

Ah so it is there, just transparent (as it should be)

Toni

Toni Wilen · 02 January 2017, 10:57

Does it use mainboard IDE or some 3rd party driver and/or some driver level cache utility? It sounds like driver (or something between driver and filesystem) does not write or flush all the data.

jarp · 02 January 2017, 11:05

Hmm interesting so it's not AmigaOS then, it's something else.

I am using Amiga 600's internal IDE port with SD-card adapter and 16GB SDHC card. I'm using PFS3 partitions (used WinUAE and PFS3AIO to prepare the card) and that's pretty much it. There's probably no buffer in the SD-card adapter?

I had no such troubles before I installed Vampire 600 (I think, but I am not sure I could've just been slower to reboot back then). Before the Vampire I used Cloanto's Kickstart 3.x and scsi.device provided within it. After Vampire I have no idea what's there as Vampire comes with it's own patched Kickstart (I actually removed Cloanto's chip). I took a quick look and version of scsi.device and it is the same than with 3.x Kickstart so probably they have based their modified Kickstart on that. But who knows what has been patched.

I'll have a closer look at evening and try find out how long it really takes to flush buffers and how HDD led behaves.

Thanks Toni, appreciate it!

demolition · 02 January 2017, 11:12

I have had similar issues where I would try to modify SS on my Vampire-A600, but after a keyboard-reboot those changes had not been written to the HDD even though I did wait for a few seconds after seeing the HDD LED flash. Since PFS3AIO should flush the buffers on a soft reset, this shouldn't even be necessary with a keyboard reset on the A600. If I wait at least 5-10 secs then it seems to work.

paul1981 · 09 January 2017, 17:22

Hmm...perhaps reports of this PFS3 issue are true after all. There's currently a necro-thread about it right now on amiga.org. Obviously way before Vampire days though.

Toni Wilen · 09 January 2017, 17:34

I have used PFS3 very long time (and still do on real hardware) and it has always flushed all data in 2-3 seconds.

jarp · 09 January 2017, 18:13

I've been still unable to replicate this at will... Seems whenever I try something trivial like creating empty test file and reboot, it works perfectly. No need to even wait for N seconds; after save HDD led blinks once and after one second it blinks second time and that second blink seems to indicate that buffers have been flushed. Immediate reboot and test file survives.

So the problem occurs on real life scenarios only. Since I got Vampire I've been doing a lot's of CPU and I/O intensive stuff like playing MP3's using AmigaAMP, extracting lot's of archives to the HDD at the same time, using web browser etc. All at the same time because Vampire allows me to, before it was of course impossible.

I'll try to replicate the problem by simulating heavy load scenario when I find some time...

demolition · 09 January 2017, 18:17

Quote:

Originally Posted by jarp

I've been still unable to replicate this at will... Seems whenever I try something trivial like creating empty test file and reboot, it works perfectly. No need to even wait for N seconds; after save HDD led blinks once and after one second it blinks second time and that second blink seems to indicate that buffers have been flushed. Immediate reboot and test file survives.

Perhaps it depends on the number of buffers used for that particular partition? Do you use the standard 150 or more?

AMIGASYSTEM · 09 January 2017, 18:24

I used many years PFS on A4000/060, is a great Filesystem, only flaw is exactly the one you mentioned although much less. Care must be taken when copying files, if you do not wait for the right time, or stop copying, the files are copied incomplete so corrupt

.

talybont · 09 January 2017, 19:07

I haven't tried this, but maybe it helps:

http://aminet.net/package/disk/salv/DiskSafe

Daedalus · 10 January 2017, 12:55

Perhaps you could issue a Diskchange command to force it to flush buffers and remount? It's made for floppies with no automatic insertion detection, but appears to work for any DOS device.

Diskchange DH0:

Or a script with a Diskchange command for each of your partitions. Provided something doesn't immediately start using them again of course...

This was something I was asking for for OS4 as well - an ability to flush and unmount a volume, specifically for USB drives but it might be useful for any such situation.

02 January 2017, 10:00	#1
jarp Registered User Join Date: Apr 2016 Location: Helsinki / Finland Posts: 176	Any way to flush HDD buffers to disk? Hi! Is there any way I can flush all unwritten changes / buffers / whatever to the HDD before I shutdown or reboot OS3.x? I tried to google around and found pretty much nothing except "just wait few seconds after HDD led has stopped blinking" and on top of that Aminet seems to be down so I cannot search. I am asking because after I got Vampire 600 it seems waiting few seconds no longer is enough. I need to wait few minutes(!!!) to make sure that everything gets written to the HDD. Not sure if this is a bug in Apollo Core or if it's just because of bigger memory available thus bigger buffers? As end result I've been installing, re-installing, re-installing, configuring, re-configruring, re-configuring the same software again and again because NOTHING goes to the HDD if I do not wait for minutes. Quite bizarre experience, really. I also just experienced my first total HDD corruption, I bet it's because I rebooted on very wrong moment. Luckily I've got backups and it seems pfsdoctor was able to fix the problem also... Thanks in advance!

09 January 2017, 18:24	#12
AMIGASYSTEM Registered User Join Date: Aug 2014 Location: Brindisi (Italy) Age: 70 Posts: 8,248	I used many years PFS on A4000/060, is a great Filesystem, only flaw is exactly the one you mentioned although much less. Care must be taken when copying files, if you do not wait for the right time, or stop copying, the files are copied incomplete so corrupt. Last edited by AMIGASYSTEM; 09 January 2017 at 18:33.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
"Out of buffers" error with PFS3AIO	Amiga1992	support.Apps	10	06 December 2017 22:53
Amiga 4000 FDD / HDD and HDD brackests	Seblington	MarketPlace	1	24 January 2012 15:10
How not to flush caches.	Toni Wilen	Coders. General	18	28 October 2011 10:05
REQ:ASM How to use buffers	jman	Coders. Tutorials	7	01 December 2010 01:41
'clone' virtual HDD in winuae to real Amiga HDD?	gizmomelb	support.Other	1	11 December 2009 09:13

02 January 2017, 10:40	#2
Zeph Registered User Join Date: Nov 2015 Location: Poole, Dorset Posts: 79	Afaik there is nothing so complicated in AmigaOS?

02 January 2017, 10:44	#3
Toni Wilen WinUAE developer Join Date: Aug 2001 Location: Hämeenlinna/Finland Age: 49 Posts: 26,506	PFS3 should automatically flush all write cached buffers in few seconds after last write. It also adds reset handler (Keyboards send "reset warning" keycode and delay reset for max 10 seconds if it is handshaked properly, most other keyboards except A500 support it) that flushes all buffers immediately.

02 January 2017, 10:51	#4
Zeph Registered User Join Date: Nov 2015 Location: Poole, Dorset Posts: 79	Ah so it is there, just transparent (as it should be) Toni

02 January 2017, 10:57	#5
Toni Wilen WinUAE developer Join Date: Aug 2001 Location: Hämeenlinna/Finland Age: 49 Posts: 26,506	Does it use mainboard IDE or some 3rd party driver and/or some driver level cache utility? It sounds like driver (or something between driver and filesystem) does not write or flush all the data.

02 January 2017, 11:05	#6
jarp Registered User Join Date: Apr 2016 Location: Helsinki / Finland Posts: 176	Hmm interesting so it's not AmigaOS then, it's something else. I am using Amiga 600's internal IDE port with SD-card adapter and 16GB SDHC card. I'm using PFS3 partitions (used WinUAE and PFS3AIO to prepare the card) and that's pretty much it. There's probably no buffer in the SD-card adapter? I had no such troubles before I installed Vampire 600 (I think, but I am not sure I could've just been slower to reboot back then). Before the Vampire I used Cloanto's Kickstart 3.x and scsi.device provided within it. After Vampire I have no idea what's there as Vampire comes with it's own patched Kickstart (I actually removed Cloanto's chip). I took a quick look and version of scsi.device and it is the same than with 3.x Kickstart so probably they have based their modified Kickstart on that. But who knows what has been patched. I'll have a closer look at evening and try find out how long it really takes to flush buffers and how HDD led behaves. Thanks Toni, appreciate it!

02 January 2017, 11:12	#7
demolition Unregistered User Join Date: Sep 2012 Location: Copenhagen / DK Age: 43 Posts: 4,190	I have had similar issues where I would try to modify SS on my Vampire-A600, but after a keyboard-reboot those changes had not been written to the HDD even though I did wait for a few seconds after seeing the HDD LED flash. Since PFS3AIO should flush the buffers on a soft reset, this shouldn't even be necessary with a keyboard reset on the A600. If I wait at least 5-10 secs then it seems to work.

09 January 2017, 17:22	#8
paul1981 Registered User Join Date: Oct 2013 Location: England Posts: 419	Hmm...perhaps reports of this PFS3 issue are true after all. There's currently a necro-thread about it right now on amiga.org. Obviously way before Vampire days though.

09 January 2017, 17:34	#9
Toni Wilen WinUAE developer Join Date: Aug 2001 Location: Hämeenlinna/Finland Age: 49 Posts: 26,506	I have used PFS3 very long time (and still do on real hardware) and it has always flushed all data in 2-3 seconds.

09 January 2017, 18:13	#10
jarp Registered User Join Date: Apr 2016 Location: Helsinki / Finland Posts: 176	I've been still unable to replicate this at will... Seems whenever I try something trivial like creating empty test file and reboot, it works perfectly. No need to even wait for N seconds; after save HDD led blinks once and after one second it blinks second time and that second blink seems to indicate that buffers have been flushed. Immediate reboot and test file survives. So the problem occurs on real life scenarios only. Since I got Vampire I've been doing a lot's of CPU and I/O intensive stuff like playing MP3's using AmigaAMP, extracting lot's of archives to the HDD at the same time, using web browser etc. All at the same time because Vampire allows me to, before it was of course impossible. I'll try to replicate the problem by simulating heavy load scenario when I find some time...

09 January 2017, 19:07	#13
talybont Registered User Join Date: Oct 2016 Location: Norway Posts: 170	I haven't tried this, but maybe it helps: http://aminet.net/package/disk/salv/DiskSafe

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

10 January 2017, 12:55	#14
Daedalus Registered User Join Date: Jun 2009 Location: Dublin, then Glasgow Posts: 6,335	Perhaps you could issue a Diskchange command to force it to flush buffers and remount? It's made for floppies with no automatic insertion detection, but appears to work for any DOS device. Diskchange DH0: Or a script with a Diskchange command for each of your partitions. Provided something doesn't immediately start using them again of course... This was something I was asking for for OS4 as well - an ability to flush and unmount a volume, specifically for USB drives but it might be useful for any such situation.