CF / SD and large drives FAQ

[fgh]

.
Introduction

This is not meant to be a step-by-step guide to set up a large SD or CF card as an amiga system drive, but it will help you identify many of the common issues.
Parts of this are based on info collected from other EAB threads, often with research by Toni Wilen, Thomas Rapp, or others.
I've recently moved the 'max drive capacity' to the top. You'll find everything else is still there, just a bit further down...

Originally this FAQ was mainly written for A600/1200/4000 IDE. I'm (very slowly) updating it to also cover SCSI controllers, but there are still parts that are not updated.

Maximum drive capacity

The 4GiB limit

Description: Max HDD or card capacity 4GiB (total drive size).
Applies to: Any IDE or SCSI drive or CF/SD card used without a 64 bit file system
Reason: Maximum with 32bit addressing
Sources: Commodore's FFS file system, plus some parts of Workbench.
Consequence: Writes above the 4GiB threshold will 'wrap around' and destroy data and the RDB on your device.

The 7,87GiB limit

Description: Max HDD or card capacity 7,87GiB (total drive size).
Applies to: A600/1200/4000 IDE with modern filesystem but original IDE driver (scsi.device)
Reason: CHS addressing is artificially limited to 7,87GiB (16383/16/63) by the ATA-specification, therefore larger drives still report 7.87GiB size when accessed with CHS. (CHS was made obsolete in ATA-6 in 2002)
Source: Scsi.device, as it only supports CHS addressing. (But when given larger than ATA-standard CHS values, it works correctly up to 128GiB.)

The 128GiB limit

Description: Max HDD capacity 128GiB (total drive size).
Applies to: A600/1200/4000 IDE with modern filesystem and many 'modern' versions of scsi.device (os 3.5/3.9 versions, etc..)
Reason: Maximum with 28bit LBA addressing. 48bit LBA (ATA-6, 2002) is required for drives larger than 128GiB.
Source: Scsi.device, as only recent versions of scsi.device support 48bit LBA (v43.45 and v44.2, see table below)

Recommendations for IDE

4GB cards: You can use original scsi.device and FFS if you want, but I recommend using a modern file system (SFS/PFS3), as it's faster, safer and more elegant (no endless validation, etc).

8GB cards: These are smaller than 7,87GiB, so the easiest is to use original scsi.device with a modern file system supporting direct-SCSI mode. (I recommend PFS3 All-in-One, but PFS3 (DS) or SFS v1.84 also work). If you choose to patch scsi.device, file systems using TD64 or NSD also work, but check the table below to make sure the file system is compatible with the scsi.device version. (PFS3 AIO is compatible with all)
Note: It is possible that '8GB' cards exist that are larger than 7,87GiB (= 8.45 GB = 8063 MB = 8455200768 bytes). I've tested ~15 so far, and they were a lot smaller though. If you have such a card, please let me know, and restrict the drive geometry below 8455200768 bytes or use a scsi.device patch)

Above 8GiB: You generally need to both use a modern file system and patch the IDE driver, scsi.device to a newer one supporting LBA addressing.
(There is one exception: the IDE-SD adapter described below.)

From the table below you will see that the only versions of scsi.device to support 48bit LBA is v43.45 and 44.2.
As some people claim they have issues with 44.2, I recommend version 43.45 for now. (Although v44.2 is faster, and worked 100% for me)
You load the updated version using 'LoadModule' in your startup-sequence.

IDEfix supports drives up to 128GiB, but doesn't like removable type CF cards, as described under 'Removable vs fixed flash media', below.

Recommendations for SCSI

The 4GiB limit applies to SCSI as well. (The 7,87 and 128GiB limits do not)
However almost all SCSI controllers work with directSCSI, so with a compatible file system like PFS3AiO the limit will be 2TiB.
SCSI drivers supporting TD64, like those for Cyberstorm SCSI controllers, should theoretically take drives up to 16 EiB (Exabytes!).
Some old controllers (firmware) like GVP ROM v1.x and 2.x and Commodore A590/2091 ROM v1-5 are limited to 1GiB, as they do not support directSCSI.
In some cases (Some old GVP firmware versions, or when using IDE-SCSI adapters) the 'read configuration' feature of HDToolbox etc might not work properly, but the full drive could still be accessible with directSCSI as long as you enter the drive geometry manually.
(You just need the total number of blocks, the rest can be faked)

Overview, IDE drivers and file systems

This table displays some limitations and compatibilities between amiga IDE drivers and file systems.
As you will see, not all combinations will work. I recommend PFS3 AiO edition, because PFS3 is great, and the AIO version works with all CPU types, scsi.device versions and has a workaround for the maxtransfer issue.

Direct SCSI, NSD and TD64?

All amiga .device drivers use the commands CMD_READ and CMD_WRITE that are limited to 32bit addresses and therefore a 4GiB drive size.
The command HD_SCSICMD (aka Direct SCSI) uses block addresses, and can therefore address 2TB (512 times more).
At one point TD64 (Trackdisk64) was created, with TD_READ64 and TD_WRITE64 commands, supporting 64bit addresses and therefore (theoretically) 16 exabytes.
Later the functionally identical but incompatible NSD was introduced, and NSCMD_TD_READ64 and NSCMD_TD_WRITE64 was supported in AmigaOS 3.5/3.9.
To use either NSD and TD64 requires both a device driver and a file system that supports it. (See the table above.)

NSDpatch is a standalone patch available for 3.1 and included as part of setpatch in OS 3.5/3.9.
It patches NSD support in to various drivers, (by using using HD_SCSICMD, or even TD64!), but since it's a patch it's not available after a cold boot.
And note that a600/1200/4000 scsi.device from 3.1 and before is limited to 7,87GiB due to CHS addressing, even if you use direct SCSI or NSDpatch/OS3.5/3.9.
Newer versions of scsi.device not supporting 48bit LBA are also limited to 128GiB (28bit LBA).

Other problems > 4GiB

As mentioned above, some parts of Workbench also have issues with devices above 4GiB..

HDToolbox only ever writes to the RDB of your drive, and it accesses your drive through the device driver (scsi.device).
With correct drive geometry, all versions of HDToolbox can set up drives up to 2TB (two terabyte).
The 'read configuration' function will be limited by the device driver (scsi.device) though.
This means that geometry will only be read correctly on devices up to 7,87GiB without patching scsi.device. (8GB cards are slightly smaller than this)
(With 'illegal' CHS values as with the SD-IDE adapters mentioned below, it detects up to 128GiB correctly)
Oterwise, if you write in your own values, or the drive has an RDB with correct geometry, (set up in WinUAE or otherwise), any version of HDToolbox is able to partition cards above 7,87GiB without patching scsi.device first. (But you still have to patch it later, to be able to access those partitions)

HDToolbox from older than OS3.5 also has some none-critical issues you need to be aware of:
It will report the total drive size incorrectly for drives above 2GiB. (showing some negative value).
When using 'read configuration' with a removable type device, it will say "Error:Unit is not a disk (Type 7)".
You do need to do some math when creating large partitons, as the partition size counter resets at 4096MB.
These issues are simply annoyances, and can be ignored.

The AmigaOS format function accesses the device directly, and is limited to 32bit.
Therefore it will also 'wrap around' and corrupt data and partitions when formatting a partition above the 4GiB limit. This is fixed in OS3.5
So always use quick format on any partition above the first 4GiB of any drive. (Actually, always use quick format anyway..)

Other programs accessing the harddrive directly will also destroy your data and partitions when accessing >4GiB, even if a new file system and scsi.device is setup correctly. (ReOrg, DiskSalv, AmibackTools, QuarterbackTools, DynamiCache, ShapeShifter, PC-Task, PCx and others) So make sure these operate on partitions below 4GiB..
This doesn't apply to Elbox drivers (for 4xEIDE'99 and FastATA adapters) if used in SPLIT mode, as this mode accesses large drives as multiples of 4GiB, so normal format and such programs are again safe to use)

Size of boot partition

A bootable partition can be of any size as long as the entire partition is accessible before any scsi.device patches.
(= First 4 GiB w/o scsi.device patch or directSCSI filesystem, first 7,87GiB with dSCSI, 128GiB with dSCSI and clever IDE-SD, or entire drive with a custom burned 3.9 ROM)

But if you have more than 2 or 4GiB free space on a partition, lots of old installation programs will fail when calculating free partition space, stating 'not enough free space' or similar. (32bit overflow again)
There are workarounds such as installing to RAM: and moving the installation afterwards, or even filling up the partition with stuff before and remove after (!), but this quickly gets tedious.
So I suggest you keep system partition size less than 2GiB. (Not nessecarily the first 2GiB, as long as the end is accessible before scsi.device patch)

OS 3.9 Boing Bag 2 issue

Scsi.device v43.43 from OS 3.9 Boing Bag 2 attempts to support 48bit LBA addressing required for IDE/ATA drives/cards larger than 128GiB. (No problem for SCSI drives)
Unfortunately it was not implemented correctly in v43.43, and only later fixed by Toni Wilen in v43.45.
OS3.9 BB2 uses its buggy LBA48 code with any IDE/ATA device above 4GiB that reports LBA48 capability, and will guru immediately after BB2 setpatch has rebooted the system.

If you have such a drive/card and want to use OS3.9 BB2, the solution is to skip the scsi.device rom update from BB2. There are at least two ways to achieve this:
1 Loading a newer version of scsi.device (43.45) with LoadModule >NIL: DEVS:scsi.device NOREBOOT and Setpatch SKIPROMUPDATES "scsi.device" QUIET
2 Using maprom feature or an actual EPROM to load a custom 3.9 ROM with a newer scsi.device and SetPatch NOROMUPDATE QUIET

Smart ATA devices with correct (but 'illegal') CHS values

The 7,87GB limit is due to a limitation given by the ATA specification that ATA devices should never report more than 7,87GB capability when asked for the 'CHS' geometry - size as defined by cylinders, heads and sectors.
Patching scsi.device adds LBA capability so this limitation disappears.
However if the device instead reports a higher than 7,87GB capacity, the original scsi.device drivers work just fine up to 128GB, as long as the file system supports >4GiB.
Some CF cards by Sandisk do this. The 16 and 32GB silver colored Ultra models were confirmed in 2017.
Perhaps oher models do the same, but the problem is they can change the firmware at any time, so you can never be sure until you test it.
Testing for this can be done by reading its CHS geometry if you can (Larger than 16383/16/63), or you can create a pfs3aio partition extending beyond 7,87GiB/8.45GB and see if it mounts. (it will only mount if the partition is safe to use)

There used to be a common and cheap SD-IDE adapter that did this as well, so any 16/32G card worked without a scsi.device patch. But it's not produced anymore.
It was the SDHC only version that looked like this.
(It reported cards as 'removable' though, so IDEFix and other drivers by Elaborate Byte fails with it, like some CF cards)
The newer SDXC version looking like this, unfortunately follows the specification. But at least it reports as a fixed device, so it works with IDEfix etc.

The CHS drive size issue

Problem: If, on a system limited to CHS addressing, you access a drive that was initialized and partitioned on an LBA capable system, you will get in trouble if the last partition of the device is extended to the very end of the drive.

Applies to: Devices of any size up to 7,87GiB (including 8GB CF/SD cards), initialized in any way in WinUAE, or in OS3.5/3.9 HDToolbox on real HW, and then used somewhere else without a scsi.device patch.
Also applies to 16/32GB SD cards used without scsi.device patch on clever ide-sd adapters, and to IDE devices initialized on a SCSI-IDE adapter with OS3.5/9, and moved to IDE without scsi.device patch.

Reason: Total drive size by LBA is exact, but by CHS it is limited to multiples of Heads and Sectors-per-track.
It is common for devices to report the full size with LBA, and a slightly smaller size with CHS. Hence, with CHS you are not able to address the last blocks of the drive.
WinUAE does not retrieve the size information directly from your device, but uses what's provided by Windows, so even when using WinUAE's IDE HD-controller you get the full size.

Consequence: With FFS or regular PFS you will get read/write errors when you attempt to access the last area. (IE when the last partition is ~half full, as partitions fill up from the middle)
With SFS or PFS AiO the last partition will not mount.
(It will not destroy data, you'll simply be unable to read from or write to this area.)

Solution: Patch scsi.device, or reduce the RDB size definition by 2 MiB or more, or leave 2 MiB or more unused at the end of the drive.
Or you can initialize the drive on a real amiga with WB3.1, or with OS3.5/3.9 using HDToolbox from WB3.1 or older.
It's just the drive definitions that need to be done in the old version, the partitioning can be done in WinUAE or OS3.5/3.9 HDToolbox.
(This 'old hdtoolbox in OS3.5/3.9' scenario is not thorougly tested, but as long as HDToolbox reports maximum 16 Heads and 63 Bytes per track you're fine)

Why 2MiB? 2MiB is largest discrepancy between CHS and LBA size. For ATA compliant devices it's only 504KB (16 H x 63 SPT x 512 B), but up to 2MiB within CHS theoretically (with 256 Heads).

If you want to check if your existing drive is affected, run a program that can read the RDB and check if the stored CHS values have more heads than 16 or more Blocks/Sectors per track than 63.

Update: A related issue submitted by Thomas is that some SD/CF cards with very poor firmware forget that the first LBA block is number 0, and consequently tries to access one block too far.
This would also fail with the same consequences when used in LBA capable system

Capacity limits other than a600/1200/4000 IDE

• No amiga setup can safely access > 4GiB without a 64bit file system.
• This FAQ details a600/1200/4000 IDE, but some third-party IDE drivers and older SCSI controllers have other, lower limits.
• Using PCMCIA with FAT95, PFS3 or SFS, there are no capacity limits, as you're using compactflash.device and TD64 compatible file systems.

General information

WinUAE considerations

If you want to access a physical amiga drive/card in WinUAE, you need to run the program as administrator (except in WinXP).
Second, Windows does not allow applications (WinUAE) to overwrite the MBR of active windows partitions.
So in order to write amiga partitions to the card, you need to remove all windows (MBR) partitions from the card first.

• In Windows Vista/7/8, the diskpart commandline utility supports removable media (use 'clean' command).
• For Win XP you need a tool like Partition Wizard or EASUS Partition Master.
• On Linux you can use the dd command
• On Mac OS use disk utility (Partition -> 1 Mac OS partition -> Free space)

Finally, when prepping a drive smaller than 8GiB in WinUAE, leave at least 2MiB of unused space after the last partition, as the last two megabytes of the drive might not be accessible on real HW. (More info under 'CHS drive size issue', above)

Some WinUAE options explained
For accessing amiga flash media in WinUAE you can select either IDE or UAE hd-controller.
If you select IDE, WinUAE will use scsi.device IDE driver from the kickstart file.
This way you won't need to change any HDToolbox tooltypes, and you might spot some errors already in WinUAE.
If you are done prepping the card and want faster file transfers, you can use 'UAE'.
(Using 'UAE', HDToolbox can still access the card but you need to change hdtoolbox's SCSI_DEVICE_NAME tooltype to uaehf.device.)

Later versions (>3.0.0) also have CF/HD choice for hardfiles. A 'CF' type hardfile will indentify itself as typical removable CF card, and will trigger associated errors (idefix failure, 'Unit is not a disk (Type 7)' in hdtoolbox, etc)
Leave it at 'HD' unless you have a specific reason not to.

Hardfiles also have an new ATA level option, deciding to which standard the drive will adhere. ATA2+ is default and will give you LBA48 support (>128GiB) and no max transfer errors.
The other options are ATA1 (no LBA48, no max transfer problem) and ATA2+ Strict (LBA48 support, max transfer problem will occur)
As with the CF/HD option, these features can trigger errors that weren't reproducable in WinUAE before, and can be useful, but I suggest you use them with caution.

Max transfer and mask

• First: The max transfer setting limits the max size of one file transfer. It has no effect on speed.
• When using onboard IDE and scsi.device, a maxtransfer setting of 0x1fe00 (or lower) is required on all partitions with any file system.
  Be aware that wrong (too high) values are mentioned lots of times on EAB!
• The reason it must be limited is that higher values will lead to read- and write errors, and data will be corrupted. (Changing to the correct value will not get corrupted data back..)
• This is due to scsi.device's outdated ATA1 behavior. (incompatible with newer ATA standards). The problem is described here.
• Specific max transfer settings are not required if you use IDEfix97 on internal IDE, when using PFS3 All-in-One or when using PCMCIA
• I do not know which drivers for third party IDE controllers are affected. It is likely that drivers made before 1994 (before EIDE, and later ATA-2 in 1996) are affected.
• Mask settings are for DMA capable controllers and not required for Amiga IDE or PCMCIA use.

FAQ continues in next post!

After hitting the maximum post length, the remainder has been moved to the next post below..

.

[Photon]

for effort. Crikey!

I mostly stay out of trouble by buying known brand makes that are max 2GB and split them into 2 x <1 GB partitions, the second starting before 1GB. All of them have worked interchangably on A500/kick 1.3 (via misc. interfaces) and up to A1200-060 so far.

But it's good to have more info. BBOAH or similar should really link to a web page like this (maybe they do already, haven't looked!)

[Toni Wilen]

Really nice FAQ. Finally there is Amiga IDE related information that isn't filled with weird myths

Mounting with 'IDE0' will use the scsi.device driver from the kickstart file, and you will spot errors (maxtransfer or others) already in WinUAE

Max transfer errors only appear in the log, latest version now emulates ATA-1 behavior (previous versions didn't emulate neither but generally it did cause similar problem)

ATA IDENTIFY DEVICE data is 99% generated under emulation. Only size comes from real drive.

direct-scsi mode (SFS or PFS3 file systems).

Only old versions of SFS have direct-scsi. It was removed long time ago for some reason.


No one still hasn't explained why some CF cards and IDE drives don't work at all when connected to Amiga IDE port. (No drive gets detected) I had ~2G 2.5" drive that didn't work, but it worked fine on a PC. I assume it is some hardware timing issue that can't be fixed in software.

[fgh]

Thanks guys. And thanks to whoever made it sticky
Tony: I have changed the text based on your comments.


Edit: After hitting the maximum post length, I've moved the second half into this post:



General info, continued...

Removable vs fixed flash media (short version)

Original scsi.device IDE drivers works well with both fixed and removable type CF cards.
The Amiga PCMCIA driver compactflash.device also works with both types of cards.
Some CF cards do not work with compactflash.device, but this is for other reasons.

IDE drivers confirmed NOT working with removable type CF cards:

• IDEfix (Elaborate Bytes)
• Buddha (Elaborate Bytes/Individual Computers)
• XSurf (Elaborate Bytes/Individual Computers)
• Catweasel (Elaborate Bytes/Individual Computers)

When loading these drivers (usually during boot) with a removable type card, the amiga will hang/freeze.

Doobrey has made patches for IDEfix and the other Elaborate Bytes IDE drivers, that correctly identify CF cards, and allow removable type cards to work correctly.

SD cards are neither removable or fixed in themselves, as they are not ATA devices.
It's therefore up to the IDE-SD adapter which response is given. (Examples under 'Clever SD-IDE adapters' above.)

See 'reasons for incompatibility' below for more information..

HDinstTool vs HDToolbox

Never use HDinstTool with a drive that was partitioned using HDToolbox if you want to keep any data on it.
HDinstTool has a nasty bug that will destroy partitions that are not defined and saved by HDinstTool originally.

About wear levelling

Due to the concept of 'wear levelling' on flash media, it is smart to avoid block-by-block writing, like a normal format, or writing back an image file over the entire card.
On a hard drive, if you write to a certain block, this exact physical block will be written to every time.
On flash media, the firmware will spread out the writes (and keep track of them) to even the strain on the fragile flash cells.
When a cell is damaged (worn out or faulty), the firmware will notice, and move the data to an unused cell, and avoid using the damaged cell again.

The problem with a full format or writing an image over the entire card is that it writes to ALL the blocks and makes the card's firmware consider them all used (It doesn't know your file system).
Therefore none of those cells can be used as replacement for damaged cells any more, and data will be corrupted faster.
Most cards have some over provisioning, meaning some extra cells available to the firmware for such tasks. But you never know if older/cheaper cards do.
So always use quick format, and if you'd like to maximise the lifespan of your card, copy files to it rather than writing an image to it.
(And don't even think about low level format, there is -physically- no way of low level formatting flash media.)
You could also leave some space unpartioned at the end of the drive, to avoid using all the cells. (= Your own over provisioning)

You might have heard about TRIM. This is used to keep the firmware informed when blocks become unused again, to optimise wear levelling to improve life span and performance.
But TRIM is only available on SSDs and requires software support that does not exist for Amiga.

Removable media troubleshooting

My CF or SD works in WinUAE but not in my real Amiga

This is often caused by a WinUAE configuration that does not match the real hardware. (amiga model, kickstart versions, ram, etc), so certain software would work in WinUAE but not on real hw.
If the filesystem that you have installed requires a certain CPU or kickstart that your real hardware does not have, the machine would guru before it starts booting the hdd.
(SFS requires 68020 and KS2.04, PFS3 works with 68000 but requires KS2.04, PFS3AiO works on 68000 and KS1.3)

Another filesystem scenario is when using a drive larger than 4GB with original PFS3 version in non-DS mode.
Unless you are using IDEfix, original PFS3 needs to be in DS mode. (See compatibility table above, PFS3 supports TD64, not NSD)
The reason it would work in WinUAE is that WinUAE gets the IDE drive size from windows, and is not limited by lack of correct TD64/NSD support. (Applies to both UAE and IDE hdd-controller selections).
Solution: Use PFS3DS or PFS3 AiO version by Toni Wilen instead.

Otherwise you might have other software installed that is incompatible with your real amiga hardware. In this case you should still have access to the drive if booting without startup-sequence:
Hold both mouse buttons down during boot, check that all expected partitions show up, select boot without startup-sequence and check that you have access to the boot partition.
(Access to partitions above 4GB will not be available until a scsi.device patch has been run)
Then you can troubleshoot your startup-sequence etc to figure out what's causing the problem.

Another scenario that came up a couple of times after I started this FAQ is that the area that WinUAE accesses is not starting entirely at the beginning of the drive.
A couple of guys ended up with a drive without windows partitons, where WinUAE still didn´t have access from the very beginning. In both cases there was 8MB of unused space before the first partition, indicating that the drive might have been GPT formatted, and not entirely recovered.

If you don't think these are the reasons:
Try hdtoolbox on the real hardware (boot with workbench install disk or any other wb installation).
Make sure that hdtoolbox is using the original scsi.device (SCSI_DEVICE_NAME tooltype) and that you are not running unpatched IDEfix, or other IDE drivers.

If the drive shows up, you might have to check your favourite hd prepping tutorial. If your drive is 4GB or smaller, or you know the geometry so you can enter the size values manually, you could quick format it on the amiga before filling it up in WinUAE.

If it does not show up in hdtoolbox, or does not work reliably, It might be one of those cards/drives that are simply not compatible with the old drivers, and I have no suggestions at this point. (Suggestions please!)

Hdtoolbox says 'Unit is not a disk (Type 7)'

Don't worry, nothing is wrong. Your removable type card is mistaken for an optical removable media (CDR, etc) by hdtoolbox.
This occurs with all cards that report as removable media (848A), when you go to 'change drive type' and select 'read configuration'.
Just ignore the message, fill out your own manufacturer and model info and proceed to partition the card.
(HDToolbox from wb 2.x/3.0/3.1 will show negative capacity for large drives (>2gb?). Ignore it, don't worry unless the partition page shows the wrong size.)
Also, if you use WinUAE to prepare the cards you should not have this issue (even if you mount the CF using IDE0).

Rare issue: Drive shows up as 'no disk in drive' after partitioning?

You have partitioned the drive normally, then after a reboot, instead of showing up as NDOS / not a dos disk, some or all FFS partitions don't appear on workbench at all, and C:Info lists all missing partitions with "no disk in drive" state. (HDToolbox still shows all partitions correctly.)

This is a rare DOS/FFS bug described by Toni Wilen.
If the first 4 bytes of the first block of the partition contains bytes $FFFFFFFF, the FFS file system gets confused and thinks there is no disk.. (It might confuse dos disk type $FFFFFFFF = -1 = ID_NO_DISK_PRESENT with real dos type)
This bug can trigger even if drive is new, for example some DOM's or solid state memory cards may contain random looking non-zero data from factory. (Test patterns?) Fix is to wipe the drive completely, or better: Use this little program by Thomas.
OS3.9BB2 works (reports correct "not a dos disk" state), I haven't tested OS3.5-OS3.9BB1. 3.1 and probably older versions have the bug.

Booting with Boing Bag 2 freezes system

If your OS3.9 Boing Bag 2 system manages to get to the setpatch reboot, but freeze immediately after the reboot, you may have fallen victim to scsi.device 43.43's LBA48 bug.
See 'OS 3.9 Boing Bag 2 issue' paragraph above.

CF or SD does not work in PCMCIA adapter

First - you need to install compactflash.device (pcmcia CF driver) and Fat95 (file system).
For automount, move CF0 from Storage/Dosdrivers to Devs/Dosdrivers.
And make sure to use the latest version as compatibility increased over the years.
Specifically, if your machine freezes when inserting the card/adapter, the first thing you should do is update to the latest version of compactflash.device. (v1.32 at time of writing)

Your amiga can access cards in FAT (windows) format or in amiga formats through the PCMCIA.
It is most common to use FAT, so your PC can access the card as well.
The CF0 mountfile specifies the file system. If CF0 is set up with Fat95 (default), you can read a FAT16/32 card.
If you want to read a card with amiga filesystem, you need to change filesystem in the CF0 mountfile PLUS define the size etc of the card.
(FAT95 does this automagically.) You may use Giggledisk on aminet to create the mountfiles.

FAT-formatted card (accessible on pc) shows up as CF0:NDOS in PCMCIA adapter
New cards come formatted either as superfloppy (without MBR partition table) or with a MBR partition table with one partition.
It seems that Fat95 accepts both types but not all variants.
Specifically FAT12 format seems to show up as NDOS, but there may be others as well. Let me know if you find out
Anyway, quick-formatting the card on the amiga should solve it.

Other possible problems

If your amiga has kickstart 3.1 and a memory expansion or accelerator that can take maximum 8MB RAM, adding any fast RAM above 4MB will conflict with PCMCIA address space and the PCMCIA port will be disabled.
Many cards of this type have a jumper to limit RAM to 4MB to avoid this conflict. Use it.
Kickstart 3.0 does not disable the PCMCIA port, so storage and I/O cards will still work with 8MB, but any PCMCIA SRAM will conflict with other fast RAM >4MB.

A1200 also has an PCMCIA issue that requires reinserting the card after a reboot. (Hardware (Gayle) changed, but kickstart ROM did not..)
CFD v1.21 (2003) and newer fixes this. (As does the cardreset software). A600 does not have this issue.

People often suggest installing Cardpatch when troubleshooting CF over PCMCIA.
I don’t know if it is required for storage media, but it will not harm. (Applies to both a600 and a1200)

SDHC card does not work in PCMCIA SD- or multi-adapter

So far I have not found a single PCMCIA adapter that supports SDHC cards (4-32GB).
All the SDHC compatible adapters I have found are 32bit Cardbus, not 16bit PCMCIA.
Look specifically for a 16bit PCMCIA adapter supporting SDHC (SD High Capacity), and let me know when you find one!
For SD cards (2GB and below) there are still cheap PCMCIA adapters around, but expect them to become increasingly rare.

Only my first FAT partition partition shows up

FAT95 does not automatically mount multiple partitions.
You need to create seperate mountfiles for these. (See giggledisk on aminet)

Reasons for incompatibility

Background

The ATA standard supports lots of different types of media, both fixed and removable types.
IDE drivers communicate with devices using the ATA protocol, and require devices to identify as removable media or not.
For the CF card itself this type info does not change anything, but it has to give an answer.
Many CF cards are set to report as removable media, but not all, and it's hard to know in advance. ('Industrial' level cards are usually fixed type)
The information is stored on CF card's CIS (Central information structure) and communicated by the ATA controller inside the card when replying to the ATA IDENTIFY DEVICE command.

The problem with certain amiga IDE drivers and 'removable' media

As mentioned above, original scsi.device IDE drivers have no problem with removable type CF cards or SD cards.
However, drivers by Elaborate Bytes - idefix97 and individual computers product drivers - do not work with removable type devices.

The drivers do not comply with the ATA-4 (or newer) specification.
IDE drivers use the ATA command IDENTIFY DEVICE to get all kinds of information from the device.

The first word of the device reply (word 0, 'General configuration') is 848Ah for removable type CF cards, and 044Ah or 0040h for fixed type CF cards.



First, bit 15 set in the reply (cards reporting 848A) can cause drivers to fail.
Drivers not written in accordance to ATA-4 (or newer) do not expect bit 15 set for ATA devices.

• ATA-4 standard ( 1998 ) says that bit 15 in word 0 shall be 0 for ATA devices (page 95), but can be 1 (848Ah) for CF compatible cards (page 52).
  It also says that ATAPI devices shall not respond to IDENTIFY DEVICE, but reply to IDENTIFY PACKET DEVICE instead (page 41)
• ATA-3 standard ( 1997 ) says it must be 0 for ATA devices and 1 for ATAPI devices (page 49)
• ATA-2 standard ( 1996 ) says it is 'reserved for non-magnetic devices' (page 37)
• ATA-1 standard ( 1994 ) says it is 'reserved for non-magnetic devices' (page 44)
• CF specification v3.0 ( 2004 ) says word 0 standard is 848Ah, but can alternatively be 044Ah or 0040h to "turn off removable media". ( page 130 )

Second, bit 7 set indicates that a device is removable media. This did not change since the original ATA-1 specification.
(Additionally bit 6 could be set to specifically mark a disk as fixed, but this was removed in ATA-6 standard (2002) so I guess modern drivers use bit 7.)

See attachments
Attached are ATA specifications 1-5, CF spec 3, and the entire IDENTIFY DEVICE replies from my four CF cards and one SD card/adapter.

Amiga IDE drivers that fail with removable type CF cards probably expect bit 15 to be 0 for ATA devices, and therefore either give up on the device, or regard it as an ATAPI device and fail.
It's confirmed by Doobrey that Elaborate Bytes drivers simply stop processing the device if bit 15 is set in the reply.
It's possible that other drivers might accept bit 15 set but still fail with removable type CF cards because bit 7 is set.

A CF card's reply can not be changed manually except on a few old cards where the manufacturer (Sandisk, Lexar) provided tools to ‘flip the removable bit’.
My Transcend 4GB 133x, Kingston 4GB (flower img) and Kingston 16GB 266x cards report as fixed (word 0 = 044A)

The ATA Identify Device replies can be read with the BusTrace program.

J.Schönfeld made the TrueIDE dual CF-IDE adapter that works around the problem with his amiga drivers. (And also lets windows users partition and boot from removable type CF cards)
This adapter modifies the CF cards IDENTIFY DEVICE reply to allow removable type cards on drivers that only work with fixed CF cards.

Regarding PCMCIA operation and incompatibility

Compact Flash storage devices operate in three modes:

• PC Card Memory Mode
• PC Card I/O Mode
• True IDE Mode

Compact Flash devices must support operation in all three modes, but operate only in a single mode at any given time.
The CF card senses (using pin #9) if it's connected to IDE or PCMCIA, and switches modes accordingly.
Connected to IDE, it uses True IDE mode, and becomes a hardware-level PATA IDE device.
Connected to PCMCIA, things are getting more complex, as it takes the role of both IDE controller and IDE device, using PC Card I/O mode.
CF cards needs to support several different PCMCIA access modes, different register locations, etc.

Some CF cards (Kingston 4Gb etc) work with IDE/scsi.device but not with pcmcia/compactflash.device.
This can theoretically be due to the card, the driver or the amiga pcmcia implementation.
It is confirmed (by Toni Wilen) that some cards do not support all mandatory register locations, causing problems with amiga pcmcia operation.

Let me know if you have more information

Regarding SD cards

SD cards are not ATA devices and have less complex controllers than CF cards.
Therefore there are far less differences between SD cards than between CF cards.
For the same reason SD-IDE adapters need to be active adapters, not just passive like most CF-IDE or CF-PCMCIA adapters.

As I wanted to experiment with SD cards, I have successfully tested 12 different 8GB SDHC cards on my a600 (PFS3 direct-scsi and ROM resident scsi.device) with a £6 SD-IDE adapter.

When connected to a PC (over IDE), the adapter reported as a removable type CF card (word 0 = 848Ah).
Therefore, drivers having problems with removable type CF cards will also have problems with such adapters.

I have later tested the newer SDXC capable version of this SD adapter, and this one reports as a fixed device.

Relevance of PCMCIA / CF card voltage ratings?

The PCMCIA specification allows 3,3 or 5 volt devices, and the amiga pcmcia implementation only works with 5v devices.
The ‘low voltage key’ (physical incompatibility) on 3.3v pcmcia devices prevent using a 3.3v device in an amiga.
But if a 3.3v only CF card existed, and was used in a generic PCMCIA adapter, this could be a problem…
However: According to this document on the Compact Flash Association, all CF cards are able to operate on both 3.3v and 5v operation.
So voltage rating should not be relevant.

That’s it for now, well done if you are still reading!
There certainly are incompatible cards, but I hope some will find their card is compatible after all with this little guide.
Additions and suggestions are welcome! Thanks

[paulo_becas]

Now that's what a call a really good explanation...

I was going to start a thread regarding my transcend 16Gb CF Card that a can't get it to boot on my 1200.
I prep the card in winuae and copied over my backup OS to it and it works great under WinUAE but when i try to boot my amiga with it it just goes black screen.
I formated it with SFS. Made on 1 partition with 512Mb and the other with the rest.


This is the card i have.
As soon i get home i'm gonna try to fix this using your faq. Let's see what happens.

[fgh]

Hi Paulo, I'm glad the post can be of use to you.

Are you using the standard a1200 IDE or another IDE controller?
With a >8GB drive I guess you use a modern scsi.device or idefix driver?
As you will see above, idefix driver does not support removable type CF cards, and the symptoms are (iirc) a system hang with black screen like you describe.
So if you use idefix, this is the first thing I would check.

Good luck!

[paulo_becas]

Originally Posted by fgh

Hi Paulo, I'm glad the post can be of use to you.

Are you using the standard a1200 IDE or another IDE controller?
With a >8GB drive I guess you use a modern scsi.device or idefix driver?
As you will see above, idefix driver does not support removable type CF cards, and the symptoms are (iirc) a system hang with black screen like you describe.
So if you use idefix, this is the first thing I would check.

Good luck!

I'm using idefix, i've downloaded the patch that you talk above but i don't know how to aply it, the readme file isn't very explicit.
My other card is an 8gb CF card that works flawlessly til this date.
Do you think if i tried it without idefix and connecting it directly to the ide port it would solve the problem?
Sorry if this seems dumb questions but i'm in a learning experience.

[fgh]

I understand that you're using IDEFix Express or some other buffered IDE interface, and idefix software..

I cannot guarantee that your problem is related to this, but unless you are certain that the card reports as a fixed ATA device, it's worth a try.
If you're unable to apply Doobrey's patches from the instructions, you can still check whether IDEfix is causing the problem by connecting the card directly on the IDE port.
However If your CF card is removable (which is tricky to find out) you still need to disable the IDEfix driver from your startup-sequence. (add ; before the relevant lines)

And since this is a 16G card, once you disable IDEfix, you need to apply another scsi.device patch for the system to see the whole card.

There are plenty of guides for patching scsi.device, but here is a short version:
If your system is based on classicWB, just drop a modern scsi.device in devs: and it will run when you reboot.
If not on classicWB you need to download and manually patch the newer scsi.device with the loadmodule command in the beginning of your startup-sequence. ('c:loadmodule devs:scsi.device')

So, to summarize..
1: In WinUAE, disable IDEfix from startup-sequence and setup a modern scsi.device patch.
2: Plug card directly onto a1200 IDE and try again.

Edit:
Your questions are not dumb in any way, this stuff is very confusing, far far away from 'plug and play' ...
You'll find some more info on amiga large HDD support here and here.

[Amiga1992]

Originally Posted by Toni Wilen

Max transfer errors only appear in the log, latest version now emulates ATA-1 behavior (previous versions didn't emulate neither but generally it did cause similar problem)

Could my problems come from having prepared the card using uae device instead of ide0?

[fgh]

I don't know your problems, but in any case, using 'UAE' IDE driver in WinUAE should not cause any trouble. It's just easier to use IDE0, so people do not have to change their hdtoolbox tooltypes.

[Amiga1992]

I am having those "line 1111 Emulator"issues with WHDLoad, exceptions, and other things, but mostly Line 1111 errors.

I had teh wrong MAxTransfer and I changed it, and now I copied everything fresh, but I am afraid I need to format again, maybe? Because those errors are just way annoying.

No otehr tool crashes but that could be because I don't save big data. Then again, I have some 500KB MODs in Protracker that work just fine. Only WHDLoad is giving me problems.

[fgh]

Ok, I don't know WHDload extremely well, other than using it successfully for my games.
A couple of searches gives plenty of topics to review though.

At least you were correct to reinstall everything if the maxtransfer setting was wrong in prolonged use, as files would be corrupted.

Off the top of my head, if you don't know if it is hardware, your wb setup, or your whdload setup that is to blame, you could always run checks with a plain 3.1 install, other whdload versions, the same setup in winuae, on other amiga hardware, etc..

[Amiga1992]

Yeah I gotta check the card in WinUAE, it's just that I hate opening up the machine to do this sort of check. At this moment I am working with it so I cant but I will try in a couple of weeks.

I thank you for your in-depth research on this matter.

[paulo_becas]

Originally Posted by Akira

it's just that I hate opening up the machine to do this sort of check.

Well, i was feeling the same until i did this and all becomes much more easier.
I'll advice you to do the same if you don't mind messing around with the amiga case a little bit.

[Amiga1992]

No, I don't like messing around with the case, however I don't feel safe having a non-hot-swappable thing like my internal CF to be exposed outside. It's just trouble for me.

I just had a great idea to check if it is a problem with teh drive: I will copy the games that are giving me the problem to the PCMCIA SD card and then run them from there. If they run fine, it should be a problem with the IDE CF, otehrwise it's something else.

[lordofchaos]

Hi,

I`m trying to get a 4 gig compact flash (sandisk ultra) card to work in WINUAE. I have a cheap "all in one mini card reader" that outputs usb to the pc. I plug the flash card into this adpater and then connect it via usb to the pc.

Windows only reads the device as "Removeable disk E:" and there doesnt appear to any drive attributes showing up and if i try to open it, windows says "please insert a disk into removable disk e: ". Anyways when I startup WinUAE and attach it using add hard drives, WinUAE lists it as " [NO MEDIA] [N/A,RW] Usb mass storage". When booting to my workbench in WinUAE the device does not show up. What am I missing, any help?

Also enclosed is a picture of the adpater. Also would like to add that I can access the Flash card on my real amiga when its plugged into PCMCIA slot.

[fgh]

My similar adapter assigns a different windows drive (E: F: G: etc) to each media type/slot.
I think this is common, to allow access to several cards simultaneously.
If yours only assigns E:, perhaps you need to install the drivers?

[lordofchaos]

Mmmm? As far as I know there are no drivers for this adapter..Also I know it works as I've plugged a SDHC Card into and was able to access it.

[fgh]

If drivers do not exist for it and windows cannot see CF cards in it, you either have a faulty unit, some conflicting software or unsupported windows version.

WinUAE can not see the card if Windows does not.

[Toni Wilen]

Check Windows Disk Management. It is hardware problem if it does not appear in disk list (with correct size information).