CF / SD and large drives FAQ - Page 10

mark_k · 26 August 2014, 15:50

Sure, though that applies to any >2TB hard drive, not only advanced format ones. Non-advanced format drives larger than 2TB do exist.

The same issue/limit applies to older PC operating systems too. To get around the 32-bit sector number limit, some >2TB external USB hard drives actually tell the host computer they have 4KB sectors! That allows up to 16TB drives to be used with MBR partitioning. With an external USB drive there isn't the same concern for compatibility since the PC BIOS doesn't need to boot from them.

fgh · 22 September 2014, 01:42

I've tried one of the new SDXC-IDE adapters..

The good news: It reports as a fixed device, so no IDEfix trouble or Type 7 error messages in HDToolbox.

The bad news: It doesn't provide 'illegal' CHS values larger than 7,87GiB, like the older SDHC-IDE adapter does, so you cannot skip the scsi.device patch with cards above 8GB.

Otherwise it works fine, but I'd still prefer the SDHC one in setups where it could save you a reboot.

fgh · 04 November 2014, 14:35

Update: I've added information about the CHS drive size issue that was brought up by ShK earlier in this thread. Thanks to Toni Wilen and ShK

thomas · 04 November 2014, 16:26

IMHO your description of the CHS issue is not entirely correct.

With CHS you are limited to 16383 cylinders, 16 heads and 63 sectors, a.k.a. 16,514,064 blocks, a.k.a. 8,455,200,768 bytes, a.k.a. 8,063 MiB, a.k.a. 7.87 GiB.

You say that the issue applies mainly to cards which are smaller than this limit.

IMHO this is not correct. Of course a mismatch between LBA and CHS can happen with all devices. But naturally it applies to all devices which are *larger* than the CHS limit. Only devices with poorly implemented firmware have that mismatch on smaller capacities.

I am sure you'll find a lot of cards with 8 GB capacity which are a bit larger than 8,063 MB. Therefore these are the most common victims. With bigger cards it's obvious that they have to mismatch. With smaller cards it should not happen.

There is another issue which might happen with poorly implemented firmware, too: the command to inquire the LBA capacity shall return the highest LBA. Because the LBA starts at 0, the last block has an LBA which is by one smaller than the number of blocks on the device. Some CF cards however return the number of blocks which is one too high.

If you partition such a card, the last partition might not appear because (what it thinks is) the last block is not accessible.

fgh · 04 November 2014, 16:49

Quote:

Originally Posted by thomas

IMHO your description of the CHS issue is not entirely correct.

With CHS you are limited to 16383 cylinders, 16 heads and 63 sectors, a.k.a. 16,514,064 blocks, a.k.a. 8,455,200,768 bytes, a.k.a. 8,063 MiB, a.k.a. 7.87 GiB.

You say that the issue applies mainly to cards which are smaller than this limit.

IMHO this is not correct. Of course a mismatch between LBA and CHS can happen with all devices. But naturally it applies to all devices which are *larger* than the CHS limit. Only devices with poorly implemented firmware have that mismatch on smaller capacities.

I am sure you'll find a lot of cards with 8 GB capacity which are a bit larger than 8,063 MB. Therefore these are the most common victims. With bigger cards it's obvious that they have to mismatch. With smaller cards it should not happen.

There is another issue which might happen with poorly implemented firmware, too: the command to inquire the LBA capacity shall return the highest LBA. Because the LBA starts at 0, the last block has an LBA which is by one smaller than the number of blocks on the device. Some CF cards however return the number of blocks which is one too high.

If you partition such a card, the last partition might not appear because (what it thinks is) the last block is not accessible.

Above 7,87GiB you have to patch scsi.device anyway, so this particular issue is solved.

There is obviously a larger discrepancy for >7,87GiB drives, due to ATA-6 CHS maximums you mention, hence the already well established need for scsi.device patches.

However it is quite common that devices report a marginally larger size via LBA than CHS. (ATA identify device word 60-61 for LBA, words 1,3 and 6 for CHS).
This is the source of the problem for devices below 7,87GiB, and the reason for this entry to the FAQ.

Regarding '8GB' card size: I tested between 15 and 20 8GB CF/SD cards of all sorts earlier, and the largest one I found was 7787 MiB , but indeed, if there are cards larger than 8063 MiB they will be hit by this issue. I will add this to the FAQ, as well as the LBA 0/1 firmware issue.

Edit: I've updated the FAQ with the 'stupid firmware' issue and described the possibility for 8GB cards being larger than 8,45GB.

Arnie · 04 November 2014, 17:52

Quote:

Originally Posted by thomas

There is another issue which might happen with poorly implemented firmware, too: the command to inquire the LBA capacity shall return the highest LBA. Because the LBA starts at 0, the last block has an LBA which is by one smaller than the number of blocks on the device. Some CF cards however return the number of blocks which is one too high.

If you partition such a card, the last partition might not appear because (what it thinks is) the last block is not accessible.

I have a couple of Kingston 4GB CF cards that wont let the last 800mb be used in a real Amiga but work fine in WinUAE.

Could the reason in your statement be the cause of this? If so, would I be able to solve it by reducing the endcyl or blocks (track or cylinder ? ) by one?

fgh · 04 November 2014, 18:00

That LBA issue should only affect one block, so not the same issue.
How does it fail on the amiga?

Edit: If you mean that there is an 800MB partition at the end of the card that does not mount on the amiga without patching scsi.device, you could be experiencing one of the issues discussed today.

ShK · 12 December 2014, 10:40

I have two clean WB3.1 installations on memory cards. The only difference is the file system. With FFS Workbench's clock works as expected. But with the PFS3 file system WB's clock is updated by flashing.

I have tested it in WinUAE and Amiga 600. The results are the same.

Tested file systems:

PFS3:
PFS\03 = 0x50465303 standard

PFS3 AIO:
PFS\03 = 0x50465303 standard
PDS\3 = 50445303 PFS3ds

https://www.dropbox.com/sh/510plbc3h...id0u90BEa?dl=0

Can anyone confirm this?

ShK · 14 December 2014, 06:14

Original Professional File System 3 has a issue, I wonder why this has not been noticed. Please, test it.

thomas · 14 December 2014, 10:09

Your second video has some CPU-intensive loop running in the background or otherwise reduced the CPU speed or CPU->chip RAM access speed. Maybe CPU caches disabled.

ShK · 14 December 2014, 10:45

This does not happen to you? Thank you for testing. For me it also occurs in WinUAE (A600 + KS3.1 and 8MB Z2 fast)

Toni Wilen · 14 December 2014, 11:57

Date has not been set and it seems to confuse WB Clock program. I guess it thinks date has changed all the time, forcing full refresh to also update date info. 1-JAN-78 = bug, 2-JAN-78 or higher: works fine.

I guess it is indirectly PFS "problem" because AFAIK FFS sets date to boot disk creation date if clock is not set or something like that. (Don't remember details)

ShK · 14 December 2014, 12:10

Confirmed. Date 02-JAN-78 Fix the problem. A small surprise.

Thanks for your investigation, Toni.

Devlin · 16 December 2014, 17:22

Is there a downside to doing a quick format to the CF partitions in WinUAE ?

fgh · 16 December 2014, 17:24

No

Devlin · 16 December 2014, 22:28

Quote:

Originally Posted by fgh

No

That's good to hear. How can I make it load stuff faster?
I set it up with two partitions both on FFS (200Mb/3.78Gb) and it is slower than the old platter drive i had in before it which is a similar setup(but the drive is 2gb) and don't want a terrible amount of fuss - I don't mind reformatting the CF card but i'd like to minimize the time my amiga is in bits to do it.

fgh · 16 December 2014, 22:44

Use a good file system like pfs3 aio and large enough disk buffers for each partition.
(Each buffer is 512bytes, so adding 300 buffers to a partition will use 150k of RAM.)
I use 500 buffers on my two active partitions, but if yours are set to 30 I guess even 100 will make a noticable difference.

Amiga IDE is always terribly slow though. It's limited by the hardware (PIO 0 + low CPU speed), not by your CF card.

Devlin · 16 December 2014, 23:01

Can I just set up PFS3AIO in HDToolbox and then keep going or does it require further setup?

I want to be able to keep my WinUAE and real amiga as close together as possible in terms of files so if I wreck one, I can restore it from the other.

fgh · 16 December 2014, 23:21

Yes, just add it in hdtoolbox and select it for each partition.
The file system is then stored in the RDB of the card, and that's it.

Create an image file of the card with WinUAE when all is set up. (add hard drive -> Create hdd image file)
Then you have a backup to use as hdf in WinUAE..

Devlin · 17 December 2014, 01:53

Quote:

Originally Posted by fgh

Yes, just add it in hdtoolbox and select it for each partition.
The file system is then stored in the RDB of the card, and that's it.

Create an image file of the card with WinUAE when all is set up. (add hard drive -> Create hdd image file)
Then you have a backup to use as hdf in WinUAE..

I got it all figured out now, feels like it's shooting along now it has 300 buffers and is set up properly

22 September 2014, 01:42	#182
fgh Registered User Join Date: Dec 2010 Location: Norway Posts: 832	I've tried one of the new SDXC-IDE adapters.. The good news: It reports as a fixed device, so no IDEfix trouble or Type 7 error messages in HDToolbox. The bad news: It doesn't provide 'illegal' CHS values larger than 7,87GiB, like the older SDHC-IDE adapter does, so you cannot skip the scsi.device patch with cards above 8GB. Otherwise it works fine, but I'd still prefer the SDHC one in setups where it could save you a reboot. Attached Thumbnails

04 November 2014, 18:00	#187
fgh Registered User Join Date: Dec 2010 Location: Norway Posts: 832	That LBA issue should only affect one block, so not the same issue. How does it fail on the amiga? Edit: If you mean that there is an 800MB partition at the end of the card that does not mount on the amiga without patching scsi.device, you could be experiencing one of the issues discussed today. Last edited by fgh; 04 November 2014 at 18:30.

14 December 2014, 10:45	#191
ShK Registered User Join Date: Mar 2013 Location: Lahti / Finland Age: 53 Posts: 452	This does not happen to you? Thank you for testing. For me it also occurs in WinUAE (A600 + KS3.1 and 8MB Z2 fast) Last edited by ShK; 14 December 2014 at 11:30. Reason: e. config

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Large Hard-Drives (over 4gb)	keitha1200	support.Hardware	4	20 April 2012 08:09
GVP 4.15 Roms & Large Hard drives...	Info-Seeker	support.Hardware	21	09 August 2010 12:06
What sort of Filemaster to use with large drives?	Ebster	support.Apps	4	08 February 2009 17:53
replacing amiga floppy drives with hard drives	Gordon	support.Hardware	2	06 March 2007 00:44
Large hard drives and WB3.0...	darkwave	support.Hardware	3	05 July 2004 03:19

26 August 2014, 15:50	#181
mark_k Registered User Join Date: Aug 2004 Location: Posts: 3,349	Sure, though that applies to any >2TB hard drive, not only advanced format ones. Non-advanced format drives larger than 2TB do exist. The same issue/limit applies to older PC operating systems too. To get around the 32-bit sector number limit, some >2TB external USB hard drives actually tell the host computer they have 4KB sectors! That allows up to 16TB drives to be used with MBR partitioning. With an external USB drive there isn't the same concern for compatibility since the PC BIOS doesn't need to boot from them.

04 November 2014, 14:35	#183
fgh Registered User Join Date: Dec 2010 Location: Norway Posts: 832	Update: I've added information about the CHS drive size issue that was brought up by ShK earlier in this thread. Thanks to Toni Wilen and ShK

04 November 2014, 16:26	#184
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 7,026	IMHO your description of the CHS issue is not entirely correct. With CHS you are limited to 16383 cylinders, 16 heads and 63 sectors, a.k.a. 16,514,064 blocks, a.k.a. 8,455,200,768 bytes, a.k.a. 8,063 MiB, a.k.a. 7.87 GiB. You say that the issue applies mainly to cards which are smaller than this limit. IMHO this is not correct. Of course a mismatch between LBA and CHS can happen with all devices. But naturally it applies to all devices which are larger than the CHS limit. Only devices with poorly implemented firmware have that mismatch on smaller capacities. I am sure you'll find a lot of cards with 8 GB capacity which are a bit larger than 8,063 MB. Therefore these are the most common victims. With bigger cards it's obvious that they have to mismatch. With smaller cards it should not happen. There is another issue which might happen with poorly implemented firmware, too: the command to inquire the LBA capacity shall return the highest LBA. Because the LBA starts at 0, the last block has an LBA which is by one smaller than the number of blocks on the device. Some CF cards however return the number of blocks which is one too high. If you partition such a card, the last partition might not appear because (what it thinks is) the last block is not accessible.

12 December 2014, 10:40	#188
ShK Registered User Join Date: Mar 2013 Location: Lahti / Finland Age: 53 Posts: 452	I have two clean WB3.1 installations on memory cards. The only difference is the file system. With FFS Workbench's clock works as expected. But with the PFS3 file system WB's clock is updated by flashing. I have tested it in WinUAE and Amiga 600. The results are the same. Tested file systems: PFS3: PFS\03 = 0x50465303 standard PFS3 AIO: PFS\03 = 0x50465303 standard PDS\3 = 50445303 PFS3ds https://www.dropbox.com/sh/510plbc3h...id0u90BEa?dl=0 Can anyone confirm this?

14 December 2014, 06:14	#189
ShK Registered User Join Date: Mar 2013 Location: Lahti / Finland Age: 53 Posts: 452	Original Professional File System 3 has a issue, I wonder why this has not been noticed. Please, test it.

14 December 2014, 10:09	#190
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 7,026	Your second video has some CPU-intensive loop running in the background or otherwise reduced the CPU speed or CPU->chip RAM access speed. Maybe CPU caches disabled.

14 December 2014, 11:57	#192
Toni Wilen WinUAE developer Join Date: Aug 2001 Location: Hämeenlinna/Finland Age: 49 Posts: 26,570	Date has not been set and it seems to confuse WB Clock program. I guess it thinks date has changed all the time, forcing full refresh to also update date info. 1-JAN-78 = bug, 2-JAN-78 or higher: works fine. I guess it is indirectly PFS "problem" because AFAIK FFS sets date to boot disk creation date if clock is not set or something like that. (Don't remember details)

14 December 2014, 12:10	#193
ShK Registered User Join Date: Mar 2013 Location: Lahti / Finland Age: 53 Posts: 452	Confirmed. Date 02-JAN-78 Fix the problem. A small surprise. Thanks for your investigation, Toni.

16 December 2014, 17:22	#194
Devlin Bane of Magic Join Date: Nov 2005 Location: Bradford, UK Age: 38 Posts: 335	Is there a downside to doing a quick format to the CF partitions in WinUAE ?

16 December 2014, 17:24	#195
fgh Registered User Join Date: Dec 2010 Location: Norway Posts: 832	No

16 December 2014, 22:44	#197
fgh Registered User Join Date: Dec 2010 Location: Norway Posts: 832	Use a good file system like pfs3 aio and large enough disk buffers for each partition. (Each buffer is 512bytes, so adding 300 buffers to a partition will use 150k of RAM.) I use 500 buffers on my two active partitions, but if yours are set to 30 I guess even 100 will make a noticable difference. Amiga IDE is always terribly slow though. It's limited by the hardware (PIO 0 + low CPU speed), not by your CF card.

16 December 2014, 23:01	#198
Devlin Bane of Magic Join Date: Nov 2005 Location: Bradford, UK Age: 38 Posts: 335	Can I just set up PFS3AIO in HDToolbox and then keep going or does it require further setup? I want to be able to keep my WinUAE and real amiga as close together as possible in terms of files so if I wreck one, I can restore it from the other.

16 December 2014, 23:21	#199
fgh Registered User Join Date: Dec 2010 Location: Norway Posts: 832	Yes, just add it in hdtoolbox and select it for each partition. The file system is then stored in the RDB of the card, and that's it. Create an image file of the card with WinUAE when all is set up. (add hard drive -> Create hdd image file) Then you have a backup to use as hdf in WinUAE..

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)