CF / SD and large drives FAQ

[Toni Wilen]

I remembered (was reminded by seeing something somewhere..) one important CF related feature that can make CF appear slower than normal hard drives. I noticed this when I was working with aca500plus scsi.device optimizations.

For some reason CF cards either don't support PIO multiple transfer mode at all or only up to max 4. (At least I haven't seen larger values)

Multiple transfer mode: number of 512 byte blocks that can be transferred in single pass, without generating interrupt and/or need to wait for data request ready.

Normal A600/A1200/A4000 scsi.device uses multiple transfer up to 16 if drive supports it. (16*512=max 8k can be transferred in single pass without waiting or interrupts)

Low multiple transfer value (or zero) means device driver needs to Wait()'s for interrupt handler to Signal() the task after every 512 bytes (if zero multiple transfer, or 2048 bytes if 4). This can cause very high overhead, especially with CPUs that have no caches or very small instruction cache.

I handled this in aca500plus by disabling IDE interrupts during transfer and when driver was going to wait for interrupt, I chose to simply poll for drive status register until BSY is inactive and DRQ is active. This increased transfer rate (if I remember correctly) almost 1MB/s more in fast 68000 mode.

Interrupt waiting probably also causes slow down even when using non-ssd modern drives because they have huge buffers (vs normal Amiga file size), are internally much faster than Amiga and device waits for first interrupt anyway (send read command, wait for interrupt, transfer all data)

NOTE: this has nothing to do with max transfer.

[dJOS]

I've found the Seagate ST1 microdrive is faster in my Amiga's and had lower CPU overheads, your post might just explain why?

[patrik]

Originally Posted by Toni Wilen

Low multiple transfer value (or zero) means device driver needs to Wait()'s for interrupt handler to Signal() the task after every 512 bytes (if zero multiple transfer, or 2048 bytes if 4). This can cause very high overhead, especially with CPUs that have no caches or very small instruction cache.

Is this simply because the amount of code to execute during the interrupt takes a significant time compared to the 512 Bytes transfer itself on a slow cpu?

Can it be checked what number of multiple transfer blocks a card/drive supports software wise?

Originally Posted by Toni Wilen

Interrupt waiting probably also causes slow down even when using non-ssd modern drives because they have huge buffers (vs normal Amiga file size), are internally much faster than Amiga and device waits for first interrupt anyway (send read command, wait for interrupt, transfer all data)

Is what you mean that with an old drive, things would not go any faster disregarding method, because the drive will actually take a while to finish a read which is longer than the interrupt processing time, but with a modern drive which often is ready instantly, it you cannot utilize that benefit because scsi.device wastes that time in interrupt processing? And using the above method of polling the status register instead, would read the drive as ready more or less directly with minimal overhead, being able to start the transfer instantly?

[Toni Wilen]

Originally Posted by patrik

Is this simply because the amount of code to execute during the interrupt takes a significant time compared to the 512 Bytes transfer itself on a slow cpu?

I'd say it is something from 10% to 30% of time it takes to transfer single block if 68000. But note that ACA500plus scsi.device uses MOVEM loop if 68000 because its data port is designed to be large enough for MOVEM so relative interrupt overhead is larger than standard A600/A1200/A4000 IDE. MOVEM is much more optimal when CPU does not have instruction cache. Normal A600/A1200/A4000 data port width is 2 words only. (Can use MOVE.L to transfer two words with single instruction)

Can it be checked what number of multiple transfer blocks a card/drive supports software wise?

Yes but scsi.device does not have any API to query it or execute ATA commands directly.

I have very ugly direct hardware access Amiga executable that returns IDE identity data but it only works with A600/A1200.

Is what you mean that with an old drive, things would not go any faster disregarding method, because the drive will actually take a while to finish a read which is longer than the interrupt processing time, but with a modern drive which often is ready instantly, it you cannot utilize that benefit because scsi.device wastes that time in interrupt processing? And using the above method of polling the status register instead, would read the drive as ready more or less directly with minimal overhead, being able to start the transfer instantly?

Yes but time between executing read command and data becoming available afaik is still relatively long even with solid state drives (few hundred microseconds vs milliseconds with platter drives) so initial interrupt waiting probably still makes sense.

[Methanoid]

I have both CF and SD adapters and my SD adapter is the latter type that allows SDXC too.

Can I just safely install the SD now
1) in A600 with no SCSI patch
2) using a 3.5+ HDToolbox to format/partition
3) Workbench partition under 2GB and Rest up to 128GB partition sizes
4) using PFS AIO

Nothing else funky to do?

[pgovotsos]

Originally Posted by Toni Wilen

I remembered (was reminded by seeing something somewhere..) one important CF related feature that can make CF appear slower than normal hard drives. I noticed this when I was working with aca500plus scsi.device optimizations.

For some reason CF cards either don't support PIO multiple transfer mode at all or only up to max 4. (At least I haven't seen larger values)

That's why I've started using DOMs instead of CF cards. I get much faster transfers for not much more cost and not having to deal with adapters that are sometimes flakey.

[fgh]

Only the older SDHC SD-IDE adapter provides 'illegal' CHS values, the new one requires that you take the same precautions as with a CF or HDD. (scsi.device patch and proper filesystem)

Originally Posted by Methanoid

I have both CF and SD adapters and my SD adapter is the latter type that allows SDXC too.

Can I just safely install the SD now
1) in A600 with no SCSI patch
2) using a 3.5+ HDToolbox to format/partition
3) Workbench partition under 2GB and Rest up to 128GB partition sizes
4) using PFS AIO

Nothing else funky to do?

[Toni Wilen]

Not mentioned in this FAQ yet and not related related to CF/SD or large drives but it probably is good idea to mention it:

Old and small (max few hundred megabyte) IDE drives that appear as zero size drives in Windows Disk Manager and all normal read attempts return errors. It is now confirmed that the reason is very simple: most (all? Lazy firmware developers..) USB adapters assume LBA support but very old IDE drives are CHS only. LBA was optional in first ATA specs.

Adapter reads drive's identity data and gets max LBA field. CHS only drive's LBA field equals zero..

I managed to find a way to fully image them, confirmed with one LBA-only adapter: http://eab.abime.net/showthread.php?t=87269 post 10+.

[patrik]

Originally Posted by Toni Wilen

Yes but scsi.device does not have any API to query it or execute ATA commands directly.

I have very ugly direct hardware access Amiga executable that returns IDE identity data but it only works with A600/A1200.

Anything you would be willing to share?

[dJOS]

Originally Posted by dJOS

I've found the Seagate ST1 microdrive is faster in my Amiga's and had lower CPU overheads, your post might just explain why?

Btw, these drives (well the 4GB units I have) have a 4MB Cache which seems to perform exceptionally well.

[Toni Wilen]

Originally Posted by Toni Wilen

I have very ugly direct hardware access Amiga executable that returns IDE identity data but it only works with A600/A1200.

http://www.winuae.net/files/b/idequery.zip

Very minimal. A600/A1200 compatible only. Unit 0 only. CLI only. Dumps IDE identity data and shows some extra info.

[patrik]

Originally Posted by Toni Wilen

http://www.winuae.net/files/b/idequery.zip

Very minimal. A600/A1200 compatible only. Unit 0 only. CLI only. Dumps IDE identity data and shows some extra info.

Hi Tony,

Sorry for taking so long time to get back to you regarding this.

Did some testing today and intended to test a few devices, so created a disk with a script to run showconfig, version scsi.device, idequery and scsispeed automatically and log the output to a file.

However, for some reason if I redirect the output of idequery to a file, the result is not the same as what is output in a shell:

9.Ram Disk:> version full `which idequery`
Could not find version information for 'System3.1:C/idequery'
9.Ram Disk:> sha1sum `which idequery`
8615C6FD AF309793 7D0BDA29 6CD751F3 1B977217
9.Ram Disk:> idequery
044A3FFFC837000F00000000003F01D8 .J?..7.......?..
00000000415844323233323136473230 ....AXD223216G20
31363036323030380002000200004158 16062008......AX
4430393239415041444F4D3434565330 D0929APADOM44VS0
314D2D30313620202020202020202020 1M-016          
20202020202020202020202020208001               ..
00000F0040000200000000073FFF000F ....@.......?...
003F3C4F00EC0101000001D800000407 .?<O............
00030078007800780078000000000000 ...x.x.x.x......
00000000000000000000000000000000 ................
03FC0000742975004000740804004000 ....t)u.@.t...@.
007F0000000000000000604F00000000 ..........`O....
0000000000000000000001D800000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................

LBA: 0030932992
Cylinders: 0000016383 Heads: 0000000015 Sectors: 0000000063
Cylinders: 0000016383 Heads: 0000000015 Sectors: 0000000063
Total sectors: 0015481935
Total sectors: 0030932992
Multiple transfer: 0000032769

9.Ram Disk:> idequery >T:idequery.log
9.Ram Disk:> type T:idequery.log 
044A3FFFC837000F00000000003F01D8 .J?..7.......?..
00000000415844323233323136473230 ....AXD223216G20
31363036323030380002000200004158 16062008......AX
4430393239415041444F4D3434565330 D0929APADOM44VS0
314D2D30313620202020202020202020 1M-016          
20202020202020202020202020208001               ..
00000F0040000200000000073FFF000F ....@.......?...
003F3C4F00EC0101000001D800000407 .?<O............
00030078007800780078000000000000 ...x.x.x.x......
00000000000000000000000000000000 ................
03FC0000742975004000740804004000 ....t)u.@.t...@.
007F0000000000000000604F00000000 ..........`O....
0000000000000000000001D800000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................

LBA:Cylinders:Cylinders:Total sectors:Total sectors:Multiple transfer:
9.Ram Disk:>

Not relevant to the issue, but perhaps interesting as it reported 32769 for multiple transfer: the tested device is a "Zheino PATA IDE DOM 44PIN 16GB Disk On Module Vertical+Socket MLC " which is built on a Silicon Motion EN2232T chipset (seems to be a compact flash controller) and one Toshiba TH58TEG7DDKTA20 flash chip. There is room for one more flash chip on the board.

[patrik]

Output from a Toshiba 24GB mSATA SSD on an SATA->IDE-adapter with JMicron JM20330 chipset:

9.Ram Disk:> idequery
0C5A3FFFC837001000000000003F0000 .Z?..7.......?..
00000000383253533130435754363559 ....82SS10CWT65Y
20202020202020200000000000045441         ......TA
374142424630544F5348494241205448 7ABBF0TOSHIBA TH
4E534E55303234474D43502020202020 NSNU024GMCP     
20202020202020202020202020208010               ..
40000F00400002000200000700000010 @...@...........
003F000000000110B7B002CB00000007 .?..............
00030078007800780078400000000000 ...x.x.x.x@.....
000000000000001F0006000000000000 ................
01FC0110746B746961637429B4496163 ....tktiact).Iac
207F0001000100FEFFFD000000000000  ...............
0000000000000000B7B002CB00000000 ................
00000008400000000000000000000000 ....@...........
0000000000000000000000000000401A ..............@.
40180000000000000000000000000000 @...............

LBA: 0046905264
Cylinders: 0000016383 Heads: 0000000016 Sectors: 0000000063
Cylinders: 0000000000 Heads: 0000000016 Sectors: 0000000063
Total sectors: 0000000000
Total sectors: 0046905264
Multiple transfer: 0000032784

[Toni Wilen]

Multiple transfer field is simply dumped as is (in decimal!) which makes it quite confusing.. It is probably a bit too basic..

If value > 32768 (bit 15 set): multiple mode supported.
Subtract 32768.
Remaining value = max number of blocks supported in multiple mode. (for example 32784 - 32768 = 16)

[patrik]

Originally Posted by Toni Wilen

Multiple transfer field is simply dumped as is (in decimal!) which makes it quite confusing.. It is probably a bit too basic..

If value > 32768 (bit 15 set): multiple mode supported.
Subtract 32768.
Remaining value = max number of blocks supported in multiple mode. (for example 32784 - 32768 = 16)

Thanks for the explanation, I thought it looked a bit odd that it supported _that_ many multiple blocks.

So the IDE DOM (CF controller) in the earlier post supports 32769 - 32768 = 1 blocks in multiple mode. Can that even be called multiple when it just supports 1?

Btw, do you have any idea why the LBA, Cylinders, Total sectors and Multiple transfer lines are output without number and newline when the output of idequery is redirected to a file (as shown in second idequery run in the post with the IDE DOM).

[patrik]

Ran idequery on the two different types of SD-card adapters I am aware of.

This was tested with idequery on an A1200 with kickstart 3.0, but I did a re-initialization of the SD-card for each adapter temporarily using scsi.device 45.7 from OS 3.1.4. The reason being that scsi.device from kickstart 3.0 could not read the drive type when attempting to read configuration from the drive in HDToolBox.

The reason for this was that I wanted the manufacturer name, drive name and drive revision for the correct adapter to show in the hex-dump output of the beginning of the disk by idequery.


First the older adapter type which is based on a KTC FC1306T GP chip. As I understand, this one is supposed to be able to support drives larger than 4GB when accessed with CHS. Connected to it is a "32GB" Transcend SDHC card with speed class 10:

2> idequery
848AF5320000001008000200003F03C5 ...2.........?..
78000000202020202020202020202020 x...
20202020202030200001000200043636       0 ......66
3239313330364D656D6F727920436172 291306Memory Car
64204164617074657220202020202020 d Adapter
20202020202020202020202020200001               ..
000003000000020000000007F5320010 .............2..
003F74E003C50101780003C500000007 .?t.....x.......
00030078007800F00078000000000000 ...x.x...x......
00000000000000000000000000000000 ................
007E0600000000000000000000000000 .~..............
001F0000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000001 ................

LBA: 0063272960
Cylinders: 0000062770 Heads: 0000000016 Sectors: 0000000063
Cylinders: 0000062770 Heads: 0000000016 Sectors: 0000000063
Total sectors: 0063272160
Total sectors: 0063272960
Multiple transfer: 0000000001

One odd thing is that idequery reports is "Multiple transfer: 0000000001". Is it not supposed to report 32768+X if it supports it?

If calculated correctly, the real size is 63272960*512/(1024^3) ~ 30.17GB and of that 62270*16*63*512/1024^3 ~ 29.93GB is accesible using CHS addressing?


The second adapter tested is the newer type which is based on a Sintechi 1309 AK chip. These adapters cause an always lit HD-led on the A600/A1200 unless patched and does not give access to more than the full card when accessed using CHS. Connected to it is a "32GB" Transcend SDHC card with speed class 10:

2> idequery
045A3FFF0000001000000240003F00FB .Z?........@.?..
FC100000373144323433453120202020 ....71D243E1
20202020202020200001001000045265         ......Re
7620322E302046432D31333037205344 v 2.0 FC-1307 SD
20746F20434620416461707465722042  to CF Adapter B
4F4F54204241434B5550205632208001 OOT BACKUP V2 ..
00000F0000000200000000073FFF0010 ............?...
003FFC1000FB010177FF03C500000107 .?......w.......
00030078007800F00078000000000000 ...x.x...x......
00000000000000000000000000000000 ................
00FE0600880150004000880050004000 ......P.@...P.@.
003F0000000000000000000000000000 .?..............
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................
00000000000000000000000000000000 ................

LBA: 0063272959
Cylinders: 0000016383 Heads: 0000000016 Sectors: 0000000063
Cylinders: 0000016383 Heads: 0000000016 Sectors: 0000000063
Total sectors: 0016514064
Total sectors: 0063272959
Multiple transfer: 0000032769

So for this one idequery reports "Multiple transfer: 0000032769" which according to earlier explanation would mean 32769-32768=1 multiple sectors.

Also, I it looks like 16384*16*63*512/1024^3 = 7.875GB is accessible using CHS addressing.

[rsn8887]

Quick question: Why are you listing FFS partition size limit as 2 GB, when it seems to be 4 GB? E.g. I can create a 4Gb .hdf file and format it with FFS and use all 4 GB just fine.

Am I missing something?

Hi All,

Just curious, is this thread still correct after the release of 3.1.4? Or, what has changed?

Thanks!

[Jope]

I guess it could do with an update. With 3.1.4 the scsi.device handles large drives and large maxtransfers without patches, but for interoperability reasons you ought to still set the maxtransfer to 0x1fe00 when using the IDE bus. PFS3AIO still recommended.

[AC/DC HACKER!]

The only thing I would correct about the initial post is GiB to GB. Otherwise, very, very cool post and thread.