[fgh]

.
Introduction

This is not meant to be a step-by-step guide to set up a large SD or CF card as an amiga system drive, but it will help you identify many of the common issues.
Parts of this are based on info collected from other EAB threads, often with research by Toni Wilen, Thomas Rapp, or others.
I've recently moved the 'max drive capacity' to the top. You'll find everything else is still there, just a bit further down...

Originally this FAQ was mainly written for A600/1200/4000 IDE. I'm (very slowly) updating it to also cover SCSI controllers, but there are still parts that are not updated.

Maximum drive capacity

The 4GiB limit

Description: Max HDD or card capacity 4GiB (total drive size).
Applies to: Any IDE or SCSI drive or CF/SD card used without a 64 bit file system
Reason: Maximum with 32bit addressing
Sources: Commodore's FFS file system, plus some parts of Workbench.
Consequence: Writes above the 4GiB threshold will 'wrap around' and destroy data and the RDB on your device.

The 7,87GiB limit

Description: Max HDD or card capacity 7,87GiB (total drive size).
Applies to: A600/1200/4000 IDE with modern filesystem but original IDE driver (scsi.device)
Reason: CHS addressing is artificially limited to 7,87GiB (16383/16/63) by the ATA-specification, therefore larger drives still report 7.87GiB size when accessed with CHS. (CHS was made obsolete in ATA-6 in 2002)
Source: Scsi.device, as it only supports CHS addressing. (But when given larger than ATA-standard CHS values, it works correctly up to 128GiB.)

The 128GiB limit

Description: Max HDD capacity 128GiB (total drive size).
Applies to: A600/1200/4000 IDE with modern filesystem and many 'modern' versions of scsi.device (os 3.5/3.9 versions, etc..)
Reason: Maximum with 28bit LBA addressing. 48bit LBA (ATA-6, 2002) is required for drives larger than 128GiB.
Source: Scsi.device, as only recent versions of scsi.device support 48bit LBA (v43.45 and v44.2, see table below)

Recommendations for IDE

4GB cards: You can use original scsi.device and FFS if you want, but I recommend using a modern file system (SFS/PFS3), as it's faster, safer and more elegant (no endless validation, etc).

8GB cards: These are smaller than 7,87GiB, so the easiest is to use original scsi.device with a modern file system supporting direct-SCSI mode. (I recommend PFS3 All-in-One, but PFS3 (DS) or SFS v1.84 also work). If you choose to patch scsi.device, file systems using TD64 or NSD also work, but check the table below to make sure the file system is compatible with the scsi.device version. (PFS3 AIO is compatible with all)
Note: It is possible that '8GB' cards exist that are larger than 7,87GiB (= 8.45 GB = 8063 MB = 8455200768 bytes). I've tested ~15 so far, and they were a lot smaller though. If you have such a card, please let me know, and restrict the drive geometry below 8455200768 bytes or use a scsi.device patch)

Above 8GiB: You generally need to both use a modern file system and patch the IDE driver, scsi.device to a newer one supporting LBA addressing.
(There is one exception: the IDE-SD adapter described below.)

From the table below you will see that the only versions of scsi.device to support 48bit LBA is v43.45 and 44.2.
As some people claim they have issues with 44.2, I recommend version 43.45 for now. (Although v44.2 is faster, and worked 100% for me)
You load the updated version using 'LoadModule' in your startup-sequence.

IDEfix supports drives up to 128GiB, but doesn't like removable type CF cards, as described under 'Removable vs fixed flash media', below.

Recommendations for SCSI

The 4GiB limit applies to SCSI as well. (The 7,87 and 128GiB limits do not)
However almost all SCSI controllers work with directSCSI, so with a compatible file system like PFS3AiO the limit will be 2TiB.
SCSI drivers supporting TD64, like those for Cyberstorm SCSI controllers, should theoretically take drives up to 16 EiB (Exabytes!).
Some old controllers (firmware) like GVP ROM v1.x and 2.x and Commodore A590/2091 ROM v1-5 are limited to 1GiB, as they do not support directSCSI.
In some cases (Some old GVP firmware versions, or when using IDE-SCSI adapters) the 'read configuration' feature of HDToolbox etc might not work properly, but the full drive could still be accessible with directSCSI as long as you enter the drive geometry manually.
(You just need the total number of blocks, the rest can be faked)

Overview, IDE drivers and file systems

This table displays some limitations and compatibilities between amiga IDE drivers and file systems.
As you will see, not all combinations will work. I recommend PFS3 AiO edition, because PFS3 is great, and the AIO version works with all CPU types, scsi.device versions and has a workaround for the maxtransfer issue.

Direct SCSI, NSD and TD64?

All amiga .device drivers use the commands CMD_READ and CMD_WRITE that are limited to 32bit addresses and therefore a 4GiB drive size.
The command HD_SCSICMD (aka Direct SCSI) uses block addresses, and can therefore address 2TB (512 times more).
At one point TD64 (Trackdisk64) was created, with TD_READ64 and TD_WRITE64 commands, supporting 64bit addresses and therefore (theoretically) 16 exabytes.
Later the functionally identical but incompatible NSD was introduced, and NSCMD_TD_READ64 and NSCMD_TD_WRITE64 was supported in AmigaOS 3.5/3.9.
To use either NSD and TD64 requires both a device driver and a file system that supports it. (See the table above.)

NSDpatch is a standalone patch available for 3.1 and included as part of setpatch in OS 3.5/3.9.
It patches NSD support in to various drivers, (by using using HD_SCSICMD, or even TD64!), but since it's a patch it's not available after a cold boot.
And note that a600/1200/4000 scsi.device from 3.1 and before is limited to 7,87GiB due to CHS addressing, even if you use direct SCSI or NSDpatch/OS3.5/3.9.
Newer versions of scsi.device not supporting 48bit LBA are also limited to 128GiB (28bit LBA).

Other problems > 4GiB

As mentioned above, some parts of Workbench also have issues with devices above 4GiB..

HDToolbox only ever writes to the RDB of your drive, and it accesses your drive through the device driver (scsi.device).
With correct drive geometry, all versions of HDToolbox can set up drives up to 2TB (two terabyte).
The 'read configuration' function will be limited by the device driver (scsi.device) though.
This means that geometry will only be read correctly on devices up to 7,87GiB without patching scsi.device. (8GB cards are slightly smaller than this)
(With 'illegal' CHS values as with the SD-IDE adapters mentioned below, it detects up to 128GiB correctly)
Oterwise, if you write in your own values, or the drive has an RDB with correct geometry, (set up in WinUAE or otherwise), any version of HDToolbox is able to partition cards above 7,87GiB without patching scsi.device first. (But you still have to patch it later, to be able to access those partitions)

HDToolbox from older than OS3.5 also has some none-critical issues you need to be aware of:
It will report the total drive size incorrectly for drives above 2GiB. (showing some negative value).
When using 'read configuration' with a removable type device, it will say "Error:Unit is not a disk (Type 7)".
You do need to do some math when creating large partitons, as the partition size counter resets at 4096MB.
These issues are simply annoyances, and can be ignored.

The AmigaOS format function accesses the device directly, and is limited to 32bit.
Therefore it will also 'wrap around' and corrupt data and partitions when formatting a partition above the 4GiB limit. This is fixed in OS3.5
So always use quick format on any partition above the first 4GiB of any drive. (Actually, always use quick format anyway..)

Other programs accessing the harddrive directly will also destroy your data and partitions when accessing >4GiB, even if a new file system and scsi.device is setup correctly. (ReOrg, DiskSalv, AmibackTools, QuarterbackTools, DynamiCache, ShapeShifter, PC-Task, PCx and others) So make sure these operate on partitions below 4GiB..
This doesn't apply to Elbox drivers (for 4xEIDE'99 and FastATA adapters) if used in SPLIT mode, as this mode accesses large drives as multiples of 4GiB, so normal format and such programs are again safe to use)

Size of boot partition

A bootable partition can be of any size as long as the entire partition is accessible before any scsi.device patches.
(= First 4 GiB w/o scsi.device patch or directSCSI filesystem, first 7,87GiB with dSCSI, 128GiB with dSCSI and clever IDE-SD, or entire drive with a custom burned 3.9 ROM)

But if you have more than 2 or 4GiB free space on a partition, lots of old installation programs will fail when calculating free partition space, stating 'not enough free space' or similar. (32bit overflow again)
There are workarounds such as installing to RAM: and moving the installation afterwards, or even filling up the partition with stuff before and remove after (!), but this quickly gets tedious.
So I suggest you keep system partition size less than 2GiB. (Not nessecarily the first 2GiB, as long as the end is accessible before scsi.device patch)

OS 3.9 Boing Bag 2 issue

Scsi.device v43.43 from OS 3.9 Boing Bag 2 attempts to support 48bit LBA addressing required for IDE/ATA drives/cards larger than 128GiB. (No problem for SCSI drives)
Unfortunately it was not implemented correctly in v43.43, and only later fixed by Toni Wilen in v43.45.
OS3.9 BB2 uses its buggy LBA48 code with any IDE/ATA device above 4GiB that reports LBA48 capability, and will guru immediately after BB2 setpatch has rebooted the system.

If you have such a drive/card and want to use OS3.9 BB2, the solution is to skip the scsi.device rom update from BB2. There are at least two ways to achieve this:
1 Loading a newer version of scsi.device (43.45) with LoadModule >NIL: DEVS:scsi.device NOREBOOT and Setpatch SKIPROMUPDATES "scsi.device" QUIET
2 Using maprom feature or an actual EPROM to load a custom 3.9 ROM with a newer scsi.device and SetPatch NOROMUPDATE QUIET

Smart ATA devices with correct (but 'illegal') CHS values

The 7,87GB limit is due to a limitation given by the ATA specification that ATA devices should never report more than 7,87GB capability when asked for the 'CHS' geometry - size as defined by cylinders, heads and sectors.
Patching scsi.device adds LBA capability so this limitation disappears.
However if the device instead reports a higher than 7,87GB capacity, the original scsi.device drivers work just fine up to 128GB, as long as the file system supports >4GiB.
Some CF cards by Sandisk do this. The 16 and 32GB silver colored Ultra models were confirmed in 2017.
Perhaps oher models do the same, but the problem is they can change the firmware at any time, so you can never be sure until you test it.
Testing for this can be done by reading its CHS geometry if you can (Larger than 16383/16/63), or you can create a pfs3aio partition extending beyond 7,87GiB/8.45GB and see if it mounts. (it will only mount if the partition is safe to use)

There used to be a common and cheap SD-IDE adapter that did this as well, so any 16/32G card worked without a scsi.device patch. But it's not produced anymore.
It was the SDHC only version that looked like this.
(It reported cards as 'removable' though, so IDEFix and other drivers by Elaborate Byte fails with it, like some CF cards)
The newer SDXC version looking like this, unfortunately follows the specification. But at least it reports as a fixed device, so it works with IDEfix etc.

The CHS drive size issue

Problem: If, on a system limited to CHS addressing, you access a drive that was initialized and partitioned on an LBA capable system, you will get in trouble if the last partition of the device is extended to the very end of the drive.

Applies to: Devices of any size up to 7,87GiB (including 8GB CF/SD cards), initialized in any way in WinUAE, or in OS3.5/3.9 HDToolbox on real HW, and then used somewhere else without a scsi.device patch.
Also applies to 16/32GB SD cards used without scsi.device patch on clever ide-sd adapters, and to IDE devices initialized on a SCSI-IDE adapter with OS3.5/9, and moved to IDE without scsi.device patch.

Reason: Total drive size by LBA is exact, but by CHS it is limited to multiples of Heads and Sectors-per-track.
It is common for devices to report the full size with LBA, and a slightly smaller size with CHS. Hence, with CHS you are not able to address the last blocks of the drive.
WinUAE does not retrieve the size information directly from your device, but uses what's provided by Windows, so even when using WinUAE's IDE HD-controller you get the full size.

Consequence: With FFS or regular PFS you will get read/write errors when you attempt to access the last area. (IE when the last partition is ~half full, as partitions fill up from the middle)
With SFS or PFS AiO the last partition will not mount.
(It will not destroy data, you'll simply be unable to read from or write to this area.)

Solution: Patch scsi.device, or reduce the RDB size definition by 2 MiB or more, or leave 2 MiB or more unused at the end of the drive.
Or you can initialize the drive on a real amiga with WB3.1, or with OS3.5/3.9 using HDToolbox from WB3.1 or older.
It's just the drive definitions that need to be done in the old version, the partitioning can be done in WinUAE or OS3.5/3.9 HDToolbox.
(This 'old hdtoolbox in OS3.5/3.9' scenario is not thorougly tested, but as long as HDToolbox reports maximum 16 Heads and 63 Bytes per track you're fine)

Why 2MiB? 2MiB is largest discrepancy between CHS and LBA size. For ATA compliant devices it's only 504KB (16 H x 63 SPT x 512 B), but up to 2MiB within CHS theoretically (with 256 Heads).

If you want to check if your existing drive is affected, run a program that can read the RDB and check if the stored CHS values have more heads than 16 or more Blocks/Sectors per track than 63.

Update: A related issue submitted by Thomas is that some SD/CF cards with very poor firmware forget that the first LBA block is number 0, and consequently tries to access one block too far.
This would also fail with the same consequences when used in LBA capable system

Capacity limits other than a600/1200/4000 IDE

• No amiga setup can safely access > 4GiB without a 64bit file system.
• This FAQ details a600/1200/4000 IDE, but some third-party IDE drivers and older SCSI controllers have other, lower limits.
• Using PCMCIA with FAT95, PFS3 or SFS, there are no capacity limits, as you're using compactflash.device and TD64 compatible file systems.

General information

WinUAE considerations

If you want to access a physical amiga drive/card in WinUAE, you need to run the program as administrator (except in WinXP).
Second, Windows does not allow applications (WinUAE) to overwrite the MBR of active windows partitions.
So in order to write amiga partitions to the card, you need to remove all windows (MBR) partitions from the card first.

• In Windows Vista/7/8, the diskpart commandline utility supports removable media (use 'clean' command).
• For Win XP you need a tool like Partition Wizard or EASUS Partition Master.
• On Linux you can use the dd command
• On Mac OS use disk utility (Partition -> 1 Mac OS partition -> Free space)

Finally, when prepping a drive smaller than 8GiB in WinUAE, leave at least 2MiB of unused space after the last partition, as the last two megabytes of the drive might not be accessible on real HW. (More info under 'CHS drive size issue', above)

Some WinUAE options explained
For accessing amiga flash media in WinUAE you can select either IDE or UAE hd-controller.
If you select IDE, WinUAE will use scsi.device IDE driver from the kickstart file.
This way you won't need to change any HDToolbox tooltypes, and you might spot some errors already in WinUAE.
If you are done prepping the card and want faster file transfers, you can use 'UAE'.
(Using 'UAE', HDToolbox can still access the card but you need to change hdtoolbox's SCSI_DEVICE_NAME tooltype to uaehf.device.)

Later versions (>3.0.0) also have CF/HD choice for hardfiles. A 'CF' type hardfile will indentify itself as typical removable CF card, and will trigger associated errors (idefix failure, 'Unit is not a disk (Type 7)' in hdtoolbox, etc)
Leave it at 'HD' unless you have a specific reason not to.

Hardfiles also have an new ATA level option, deciding to which standard the drive will adhere. ATA2+ is default and will give you LBA48 support (>128GiB) and no max transfer errors.
The other options are ATA1 (no LBA48, no max transfer problem) and ATA2+ Strict (LBA48 support, max transfer problem will occur)
As with the CF/HD option, these features can trigger errors that weren't reproducable in WinUAE before, and can be useful, but I suggest you use them with caution.

Max transfer and mask

• First: The max transfer setting limits the max size of one file transfer. It has no effect on speed.
• When using onboard IDE and scsi.device, a maxtransfer setting of 0x1fe00 (or lower) is required on all partitions with any file system.
  Be aware that wrong (too high) values are mentioned lots of times on EAB!
• The reason it must be limited is that higher values will lead to read- and write errors, and data will be corrupted. (Changing to the correct value will not get corrupted data back..)
• This is due to scsi.device's outdated ATA1 behavior. (incompatible with newer ATA standards). The problem is described here.
• Specific max transfer settings are not required if you use IDEfix97 on internal IDE, when using PFS3 All-in-One or when using PCMCIA
• I do not know which drivers for third party IDE controllers are affected. It is likely that drivers made before 1994 (before EIDE, and later ATA-2 in 1996) are affected.
• Mask settings are for DMA capable controllers and not required for Amiga IDE or PCMCIA use.

FAQ continues in next post!

After hitting the maximum post length, the remainder has been moved to the next post below..

.