HDToolbox using all available memory...

-Acid- · 23 October 2016, 01:38

This is a weird one and it has only just started happening.

I have a large 160Gb drive partitioned up (2+2+52+52+52Gb partitions). They all work perfectly fine in a real A1200 and in WinUAE. I have recently been messing around with different workbench installs and have had both the 2Gb partitions bootable. I tried to go back and make only one of them bootable in HDToolbox but it will not load up, something is sucking all available memory to the point the system cannot do anything as no ram is available.

This happens on my A1200 with 64mb ram total, and also happens in WinUAE with 130mb available. As soon as I load it the mem counter at the top of the screen starts to decrease at a rate of megabytes per second until there is only a few kilobytes left. I can quit HDTB but only 9k of chip and roughly 225kb of fast is restored. The file system in use is SFS, besides this the drive works normally it boots all partitions and data are accessible and games and tools load fine.

What can be causing this memory leak when I load HDTB?

idrougge · 23 October 2016, 06:29

What version?

thomas · 23 October 2016, 10:15

Quote:

Originally Posted by -Acid-

What can be causing this memory leak when I load HDTB?

Most likely a loop in the block pointers of the RDB area. Can you dump the first 2 - 3 MB of the drive and attach it here for examination.

-Acid- · 23 October 2016, 11:54

Quote:

Originally Posted by thomas

Most likely a loop in the block pointers of the RDB area. Can you dump the first 2 - 3 MB of the drive and attach it here for examination.

If you can tell me how to do that sure. This is using WB 3.0 idrougge.

thomas · 23 October 2016, 23:23

There are several methods. Unix dd comes to mind.

You can also use TSGUI. http://thomas-rapp.homepage.t-online...oads/tsgui.lha
Select a partition on the HDD from the list, then enable the "Full HDD image" checkbox, select a destination file, then start reading the disk. Let it run a few seconds, then cancel it but keep the unfinished image file. Check that the file is bigger than 2 MB, zip it and upload the the zip archive here as attachment.

Before you do the above you might try to select "Save RDB to file" from the menu. If TSGUI eats up all memory, too, it's almost proven that there is a problem with the RDB. If not, please upload the resulting file, too.

There is another program you might want to try: http://thomas-rapp.homepage.t-online.../hddreport.lha
If it succeeds, please attach the report, too. If there is a problem with the RDB, it might touch the memory limit, too.

-Acid- · 24 October 2016, 15:16

Running hdreport sucks up the memory and eventually crashes the system. TSGUI managed to complete both tasks though, find the results attached.

thomas · 24 October 2016, 20:23

Ok, even without looking into the files I have a guess what the problem could be. Is the drive connected to IDE on your A1200? The .rdb file is 129 KB. As you might know, scsi.device has a problem with chunks bigger than 128 KB. This is usually referred to as MaxTransfer problem, but MaxTransfer is a DOS thing and HDToolbox acts at a lower level. You cannot change MaxTransfer for HDToolbox.

Give me a few days and I'll make a repaired RDB for you. (Currently I am quite busy at work and don't like sitting at the computer at home, too.)

You might try to write your RDB back to the disk with TSGUI. I don't remember, maybe it repairs it automatically.

emufan · 24 October 2016, 20:48

not sure it will help, but there is RDBSalv v1.1. may happen this can fix your problems.
it suggests it can backup and restore RDBs, so it can be of some sort of help.

-Acid- · 24 October 2016, 20:59

Quote:

Originally Posted by thomas

things

Yes it's on IDE (same behavior on A4000 IDE as well). If I write the RDB back to the drive is there any danger of losing the contents of the drive or is it relatively safe?

-Acid- · 24 October 2016, 21:00

Quote:

Originally Posted by emufan

not sure it will help, but there is RDBSalv v1.1. may happen this can fix your problems.
it suggests it can backup and restore RDBs, so it can be of some sort of help.

TSGUI can also do that, do you know if there is any benefit to using RDBSalv over that or is it all the same thing?

emufan · 24 October 2016, 21:10

RDBsalv got a "salvage" button - compared to tsgui, thats all i can say.
it was also used in the past to do some repairs.
but I for myself was not in need to use it, so i cannot say for sure.
First use the backup function before you try the salvage thing

btw. when u use the installer change the path to installer in icon-information.
but it will also run without installing, just unarchive the lha somewhere.
at least worth a look.

thomas · 24 October 2016, 23:16

Well, RDB-Salv is old, no longer supported and the above is an illegal pirated copy.

From its own description it cannot handle anything but FFS and as far as I know it is not able to access beyond the 4GB limit. Not to mention that reading a 160 GB drive over the internal IDE bus would take hours if not days. That's what salvage means.

Quote:

Originally Posted by -Acid-

Yes it's on IDE (same behavior on A4000 IDE as well). If I write the RDB back to the drive is there any danger of losing the contents of the drive or is it relatively safe?

Well, the RDB area is outside all partitions, so writing to that area is safe for all the files you have.

But writing to the partition table has the risk of losing the partition table which means losing access to all partitions until the original table has been restored. So a certain danger is present.

When fiddling with the partition table you should always have another boot medium at hand in case your main drive becomes no longer bootable. This other medium should hold everything you need to recover the main drive, including a means to access all support files you get sent by me or other people here.

thomas · 29 October 2016, 10:47

Ok, verified the loop. Block 257 points to itself. This is exactly what the MaxTransfer problem would cause.

TSGUI will not help. It just writes the RDB as it was, including the looping pointer.

You can use this program to correct it: http://thomas-rapp.homepage.t-online...ads/rdbman.lha

Open a shell window and run it like this:

Code:

rdbman scsi.device 0 file ram:rdb backup restore

replace scsi.device 0 by the driver and unit of your HDD.

Note that this does only remove the loop. The corrupted file system is still there. It's FastFileSystem version 39 and it's now cut off after 50 blocks (25 KB). You should remove it from the RDB, it's in Kickstart ROM anyway. By this the RDB would also shrink below 128 KB and avoid the MaxTransfer problem.

-Acid- · 29 October 2016, 12:06

Quote:

Originally Posted by thomas

Note that this does only remove the loop. The corrupted file system is still there. It's FastFileSystem version 39 and it's now cut off after 50 blocks (25 KB). You should remove it from the RDB, it's in Kickstart ROM anyway. By this the RDB would also shrink below 128 KB and avoid the MaxTransfer problem.

I'm not too technical so I just want to clarify this, you mention FFS there and just want to make sure you are aware this drive is using SFS. But at any rate I have managed to backup everything I need from the drive and my intention is to repartition it and start over.

However I still need this fix so I can do that in HDTB, I will give it a try this afternoon as I have some Amiga time today

Thanks for all your help Thomas.

thomas · 29 October 2016, 12:41

The partitions are using SFS, but the code for both SFS and FFS is stored in the RDB. You don't need FFS there, it only bloats the size of the RDB.

-Acid- · 30 October 2016, 13:27

Excellent Thomas that has fixed it I can access it in HDTB again

I don't know where Amiga users would be without people like you and Toni Wilen, you guys really know the Amiga inside out!

kolla · 30 October 2016, 16:06

Not to hijack this thread, but... Thomas, you have lots of filesystem related tools on your site, do you have a tool to read and write creation time of a FFS filesystem? As you know, the Amiga, if lacking RTC, will use creation time of the filesystem it boots from to set time at boot, so it would be great to be able to adjust it without reformatting

thomas · 30 October 2016, 17:24

Not yet, but sounds interesting. I'll have a look into it.

23 October 2016, 01:38	#1
-Acid- Registered User Join Date: Oct 2012 Location: South Shields Posts: 812	HDToolbox using all available memory... This is a weird one and it has only just started happening. I have a large 160Gb drive partitioned up (2+2+52+52+52Gb partitions). They all work perfectly fine in a real A1200 and in WinUAE. I have recently been messing around with different workbench installs and have had both the 2Gb partitions bootable. I tried to go back and make only one of them bootable in HDToolbox but it will not load up, something is sucking all available memory to the point the system cannot do anything as no ram is available. This happens on my A1200 with 64mb ram total, and also happens in WinUAE with 130mb available. As soon as I load it the mem counter at the top of the screen starts to decrease at a rate of megabytes per second until there is only a few kilobytes left. I can quit HDTB but only 9k of chip and roughly 225kb of fast is restored. The file system in use is SFS, besides this the drive works normally it boots all partitions and data are accessible and games and tools load fine. What can be causing this memory leak when I load HDTB?

24 October 2016, 20:48	#8
emufan Registered User Join Date: Feb 2012 Location: #DrainTheSwamp Posts: 4,545	not sure it will help, but there is RDBSalv v1.1. may happen this can fix your problems. it suggests it can backup and restore RDBs, so it can be of some sort of help. Last edited by emufan; 24 October 2016 at 20:54.

29 October 2016, 10:47	#13
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	Ok, verified the loop. Block 257 points to itself. This is exactly what the MaxTransfer problem would cause. TSGUI will not help. It just writes the RDB as it was, including the looping pointer. You can use this program to correct it: http://thomas-rapp.homepage.t-online...ads/rdbman.lha Open a shell window and run it like this: Code: rdbman scsi.device 0 file ram:rdb backup restore replace scsi.device 0 by the driver and unit of your HDD. Note that this does only remove the loop. The corrupted file system is still there. It's FastFileSystem version 39 and it's now cut off after 50 blocks (25 KB). You should remove it from the RDB, it's in Kickstart ROM anyway. By this the RDB would also shrink below 128 KB and avoid the MaxTransfer problem.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
WinUAE 3.2.1. memory dump for fake fast memory areas broken	StingRay	support.WinUAE	1	14 December 2015 20:39
CF Card and HDtoolbox	source	Hardware mods	7	04 November 2010 01:13
Blizzard Turbo Memory - SMD memory chips doesn't work	sanjyuubi	Hardware mods	5	26 May 2010 15:40
HDtoolbox	leongt	request.Apps	3	14 May 2003 10:17
HDToolBox	Mick_AKA	request.Apps	3	15 July 2002 00:49

23 October 2016, 06:29	#2
idrougge Registered User Join Date: Sep 2007 Location: Stockholm Posts: 4,332	What version?

23 October 2016, 23:23	#5
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	There are several methods. Unix dd comes to mind. You can also use TSGUI. http://thomas-rapp.homepage.t-online...oads/tsgui.lha Select a partition on the HDD from the list, then enable the "Full HDD image" checkbox, select a destination file, then start reading the disk. Let it run a few seconds, then cancel it but keep the unfinished image file. Check that the file is bigger than 2 MB, zip it and upload the the zip archive here as attachment. Before you do the above you might try to select "Save RDB to file" from the menu. If TSGUI eats up all memory, too, it's almost proven that there is a problem with the RDB. If not, please upload the resulting file, too. There is another program you might want to try: http://thomas-rapp.homepage.t-online.../hddreport.lha If it succeeds, please attach the report, too. If there is a problem with the RDB, it might touch the memory limit, too.

24 October 2016, 20:23	#7
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	Ok, even without looking into the files I have a guess what the problem could be. Is the drive connected to IDE on your A1200? The .rdb file is 129 KB. As you might know, scsi.device has a problem with chunks bigger than 128 KB. This is usually referred to as MaxTransfer problem, but MaxTransfer is a DOS thing and HDToolbox acts at a lower level. You cannot change MaxTransfer for HDToolbox. Give me a few days and I'll make a repaired RDB for you. (Currently I am quite busy at work and don't like sitting at the computer at home, too.) You might try to write your RDB back to the disk with TSGUI. I don't remember, maybe it repairs it automatically.

24 October 2016, 21:10	#11
emufan Registered User Join Date: Feb 2012 Location: #DrainTheSwamp Posts: 4,545	RDBsalv got a "salvage" button - compared to tsgui, thats all i can say. it was also used in the past to do some repairs. but I for myself was not in need to use it, so i cannot say for sure. First use the backup function before you try the salvage thing btw. when u use the installer change the path to installer in icon-information. but it will also run without installing, just unarchive the lha somewhere. at least worth a look.

29 October 2016, 12:41	#15
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	The partitions are using SFS, but the code for both SFS and FFS is stored in the RDB. You don't need FFS there, it only bloats the size of the RDB.

30 October 2016, 13:27	#16
-Acid- Registered User Join Date: Oct 2012 Location: South Shields Posts: 812	Excellent Thomas that has fixed it I can access it in HDTB again I don't know where Amiga users would be without people like you and Toni Wilen, you guys really know the Amiga inside out!

30 October 2016, 16:06	#17
kolla Banned Join Date: Nov 2007 Location: Trondheim, Norway Posts: 1,893	Not to hijack this thread, but... Thomas, you have lots of filesystem related tools on your site, do you have a tool to read and write creation time of a FFS filesystem? As you know, the Amiga, if lacking RTC, will use creation time of the filesystem it boots from to set time at boot, so it would be great to be able to adjust it without reformatting

30 October 2016, 17:24	#18
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	Not yet, but sounds interesting. I'll have a look into it.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)