IDE & Max Transfer

alexh · 16 June 2009, 14:49

I was replying to someone over at Amiga.org about MAX TRANSFER settings for IDE controllers (such as A1200/A600 Gayle IDE) giving them the standard "try 0x1FE00 and if that doesnt work try 0xFE00".

While writing I accidentally worked out the logical reason behind these "Magic numbers".

IDE commands (ATA5 or lower) have "sector count" field which is 8-bits.
SCSI commands (SCSI10) have a "transfer size" field which is 16-bits.

For those who do not know, a sector is the smallest individual amount you can access on a Hard drive and has always been (until recently) 512 bytes. And "sector count / transfer size" is the amount to be transferred in a single command.

I noticed that "0x1FE00" is 255 * 512, and that 255 is the largest value that can be represented by 8-bits. So by using "0x1FE00" it prevents the driver (scsi.device) creating malformed IDE commands with sector counts larger than 8-bit.

I am positive this is the logical answer as to why MAX TRANSFER should be "0x1FE00" for IDE devices.

[EDIT]

I was close but not correct. Toni Wilen discovered the true answer :

http://eab.abime.net/showpost.php?p=759918&postcount=25

[/EDIT]

------------

Now while the IDE (ATA5 and lower) command set supports an 8-bit sector count, very few operating systems use the maximum value. Windows (and most other operating systems) tend to use 64 or 128 sector commands. I know this because I discovered at work several years ago that PATA hard drives have their cache optimised for 64 sectors while SATA were optimised for 128 sector commands.

Back to the plot:

The second magic number touted for MAX transfer "0xFE00" is 127 * 512. I am not sure exactly why this value would be more compatible than 0x1FE00, but I imagine it has to do with the fact poor quality IDE devices do not expect sector counts above 128 sectors.

You pretty much guarantee "sector count" compatibility with any IDE device with 0xFE00 and I cannot believe that it is slower.

Anyhow, hope that has explained things, rather than confused things.

prowler · 16 June 2009, 21:30

That's a useful display of detective work, alexh!

Thanks for sharing your thoughts on this with us.

alexh · 16 June 2009, 22:19

I dunno if my explanation is dumb enough.

You still have to be reasonably computer literate (at a low level) to get it.

Eclipse · 17 June 2009, 01:59

That's very clear and actually explains a lot.

Zetr0 · 17 June 2009, 05:09

I think it should be added to the CF-IDE for the Amiga Wiki...... if we have one....

Leandro Jardim · 04 December 2010, 03:07

Thanks alexh !

I put the magic number in the RDB of my hardfile, and now it works much better!

mech · 04 December 2010, 05:19

Nice one alex,
another good reason to use SCSI on amiga also

Jope · 04 December 2010, 10:18

Added Alexh's post to Amiga Problems.

Rixa · 06 December 2010, 11:46

Nicely reasoned, and makes sense.

However, when I was setting up my CF card, I went with what looked like the most reliable-looking advice I could find at the time and set the Max Transfers value as high as 0x1FE0000. At 256 times more than reasoned above, it seems quite absurd.

The magic is it has worked just fine, and apparently also in some cases where 0x1FE00 hasn't. I would like to know why it works, whether it is just waiting to cause trouble, or if there is any other reason to change it. I was thinking perhaps it is not taking the entire 32-bit value into account, but the less significant word is all zeroes so wouldn't it likely result in a MaxTransfer of 0 then?

Loedown · 06 December 2010, 13:01

Quote:

Originally Posted by Rixa

Nicely reasoned, and makes sense.

However, when I was setting up my CF card, I went with what looked like the most reliable-looking advice I could find at the time and set the Max Transfers value as high as 0x1FE0000. At 256 times more than reasoned above, it seems quite absurd.

The magic is it has worked just fine, and apparently also in some cases where 0x1FE00 hasn't. I would like to know why it works, whether it is just waiting to cause trouble, or if there is any other reason to change it. I was thinking perhaps it is not taking the entire 32-bit value into account, but the less significant word is all zeroes so wouldn't it likely result in a MaxTransfer of 0 then?

If I understand correctly CF cards have a block transfer mode and as long as you stay under the maximum limit for their block transfer and your Amiga has enough ram to accomodate the block before writing then all should be good. 256 bytes and 512 bytes are older block sizes, C64 disks use a 256 byte sector for example, keep in mind when the hardware came out 512 byte sector size was fine and now we have legacy keeping it alive.

thomas · 06 December 2010, 13:53

Actually there shouldn't be a problem with MaxTransfer. The device driver should know the hardware limitations and automatically split large packets into smaller ones. Just like ping and MTU in TCP/IP world.

But scsi.device does not do this. If you specify a MaxTransfer value larger than 0x1fe00, it will send a mangled command to the device, resulting in corrupted data. So it is important to set MaxTransfer to a reasonable value when using scsi.device.

However, it also depends on the version of scsi.device which is used. For example when using IDEfix, a new scsi.device is used which does automatically split large packets. So with IDEfix MaxTransfer should not be an issue.

Finally MaxTransfer issues only arise when using files larger than the critical limit. Files smaller than 127.5K are not affected at all. Similarly large files might not be affected if a copy program is used which does not attempt to write more than 127.5K at a time.

Gordon · 02 June 2011, 20:01

Where can I set the max transfers properly? I have tried HD tool box but it doesnt save my changes? im trying to put 0x1fe00 but it kees changeing. Do i need to change the mask as well?

Retro-Nerd · 02 June 2011, 20:09

You have to press the enter button after typing 1fe00, then save the settings.

Jope · 02 June 2011, 20:15

No need to adjust the mask.

Gordon · 02 June 2011, 20:20

Thank you Retro-Nerd for the very quick response, that worked. I cant believe thats what I was missing!

Ok Jope thanks.

03 June 2011, 15:16

Quote:

Originally Posted by alexh

I was replying to someone over at Amiga.org about MAX TRANSFER settings for IDE controllers (such as A1200/A600 Gayle IDE) giving them the standard "try 0x1FE00 and if that doesnt work try 0xFE00".

While writing I accidentally worked out the logical reason behind these "Magic numbers".

IDE commands (ATA5 or lower) have "sector count" field which is 8-bits.
SCSI commands (SCSI10) have a "transfer size" field which is 16-bits.

For those who do not know, a sector is the smallest individual amount you can access on a Hard drive and has always been (until recently) 512 bytes. And "sector count / transfer size" is the amount to be transferred in a single command.

I noticed that "0x1FE00" is 255 * 512, and that 255 is the largest value that can be represented by 8-bits. So by using "0x1FE00" it prevents the driver (scsi.device) creating malformed IDE commands with sector counts larger than 8-bit.

I am positive this is the logical answer as to why MAX TRANSFER should be "0x1FE00" for IDE devices.------------

Actually, max sector count by IDE (ATA) is 256 sectors, because 0 in sector count registers means not 0 (nonsense) but 256 .

With 48-bit LBA (what is necessary with drives over 128GB) max count is 65536 as sector count is given as 16-bit value.

alexh · 03 June 2011, 16:05

Quote:

Originally Posted by AmigTart

Actually, max sector count by IDE (ATA) is 256 sectors, because 0 in sector count registers means not 0 (nonsense) but 256 .

Agreed, but I didn't want to confuse people. The largest number of sectors possible in a non zero 8-bit number is still 255 which corresponds to MAX TRANSFER of 0x1FE00 bytes.

Quote:

Originally Posted by AmigTart

With 48-bit LBA (what is necessary with drives over 128GB) max count is 65536 as sector count is given as 16-bit value.

The recent patches to the Amiga IDE scsi.device which adds ATA-6 command set (48-bit LBA) may have increased the sector count to 16-bit but I don't know. Can anyone confirm?

You would need this version of scsi.device in your Kickstart ROM or eFlash/Kickflash/Deneb to be able to boot from a device with a the MAX TRANSFER set to larger than 0x1FE00. This is going to be a VERY small set of people.

I thought that I had used the scsi.device (45.45) with 48-bit LBA support using a MAX TRANSFER larger than 0x1FE00 with a SanDisk CF card and had data corruption. I will be able to confirm soon as I'm making my own KS3.9 EPROMs for my A4000

rkauer · 05 June 2011, 00:12

The real problem is not the sector count, but the fact compact flash & microdrives doesn't have a built-in buffer .

Notice that real HD doesn't suffer from this issue, from the old IDE to SCSI drives.

So anything over 255 will choke in the Amiga IDE bus (read "Gayle"), who will still try to put more data than the slow IDE channel can really cope.

alexh · 05 June 2011, 00:41

???? How did you work that out? I was sure the problem is about malformed commands. The Amiga tries to transfer more data on the bus than the drive is expecting.

AmigaOS tries to send a command for say 131584 bytes (257 sectors). It made sense that scsi.device can only send ATA commands with an 8-bit sector count (and it doesn't check the incoming request size) which gets converted into an ATA command of 1 sector.

Quote:

Originally Posted by rkauer

Notice that real HD doesn't suffer from this issue

With a MAX TRANSFER above 0x1FE00 I get data corruption on Gayle IDE using 2.5" PATA IDE disks just as I do with CF cards.

I imagined it was not a problem for SCSI drives because SCSI commands have a bigger (32-bit?) sector count field.

Retro-Nerd · 05 June 2011, 00:44

Really? I never had any issues with standard MaxTransfer settings and PC hard drives. Pure luck?

16 June 2009, 14:49	#1
alexh Thalion Webshrine Join Date: Jan 2004 Location: Oxford Posts: 14,336	IDE & Max Transfer I was replying to someone over at Amiga.org about MAX TRANSFER settings for IDE controllers (such as A1200/A600 Gayle IDE) giving them the standard "try 0x1FE00 and if that doesnt work try 0xFE00". While writing I accidentally worked out the logical reason behind these "Magic numbers". IDE commands (ATA5 or lower) have "sector count" field which is 8-bits. SCSI commands (SCSI10) have a "transfer size" field which is 16-bits. For those who do not know, a sector is the smallest individual amount you can access on a Hard drive and has always been (until recently) 512 bytes. And "sector count / transfer size" is the amount to be transferred in a single command. I noticed that "0x1FE00" is 255 * 512, and that 255 is the largest value that can be represented by 8-bits. So by using "0x1FE00" it prevents the driver (scsi.device) creating malformed IDE commands with sector counts larger than 8-bit. I am positive this is the logical answer as to why MAX TRANSFER should be "0x1FE00" for IDE devices. [EDIT] I was close but not correct. Toni Wilen discovered the true answer : http://eab.abime.net/showpost.php?p=759918&postcount=25 [/EDIT] ------------ Now while the IDE (ATA5 and lower) command set supports an 8-bit sector count, very few operating systems use the maximum value. Windows (and most other operating systems) tend to use 64 or 128 sector commands. I know this because I discovered at work several years ago that PATA hard drives have their cache optimised for 64 sectors while SATA were optimised for 128 sector commands. Back to the plot: The second magic number touted for MAX transfer "0xFE00" is 127 * 512. I am not sure exactly why this value would be more compatible than 0x1FE00, but I imagine it has to do with the fact poor quality IDE devices do not expect sector counts above 128 sectors. You pretty much guarantee "sector count" compatibility with any IDE device with 0xFE00 and I cannot believe that it is slower. Anyhow, hope that has explained things, rather than confused things. Last edited by alexh; 30 August 2017 at 09:40.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
CF Hd Max Transfer Problem	flushentity	support.Hardware	2	12 May 2012 22:15
IDE max transfer (CF, Hard disk, FAT95)	xraynorm	support.Hardware	2	27 March 2011 20:58
Max transfer & mask for CF card?	PowerPie5000	support.Hardware	7	29 November 2010 12:35
IDE & Max Transfer-temp	prowler	support.Hardware	0	16 June 2009 21:30
CF to IDE adapter - Max size card?	crispywotsits	support.Other	11	19 February 2009 16:38

16 June 2009, 21:30	#2
prowler Global Moderator Join Date: Aug 2008 Location: Sidcup, England Posts: 10,300	That's a useful display of detective work, alexh! Thanks for sharing your thoughts on this with us.

16 June 2009, 22:19	#3
alexh Thalion Webshrine Join Date: Jan 2004 Location: Oxford Posts: 14,336	I dunno if my explanation is dumb enough. You still have to be reasonably computer literate (at a low level) to get it.

17 June 2009, 01:59	#4
Eclipse Turpentine Join Date: Oct 2007 Location: Kent, United Kingdom Posts: 744	That's very clear and actually explains a lot.

17 June 2009, 05:09	#5
Zetr0 Ya' like it Retr0? Join Date: Jul 2005 Location: United Kingdom Age: 49 Posts: 9,768	I think it should be added to the CF-IDE for the Amiga Wiki...... if we have one....

04 December 2010, 03:07	#6
Leandro Jardim Registered User Join Date: Nov 2009 Location: Legoland Age: 45 Posts: 1,461	Thanks alexh ! I put the magic number in the RDB of my hardfile, and now it works much better!

04 December 2010, 05:19	#7
mech Registered User Join Date: Mar 2009 Location: texas,usa Posts: 235	Nice one alex, another good reason to use SCSI on amiga also

04 December 2010, 10:18	#8
Jope - Join Date: Jul 2003 Location: Helsinki / Finland Age: 43 Posts: 9,861	Added Alexh's post to Amiga Problems.

06 December 2010, 11:46	#9
Rixa Registered User Join Date: Sep 2007 Location: Tampere / Finland Posts: 127	Nicely reasoned, and makes sense. However, when I was setting up my CF card, I went with what looked like the most reliable-looking advice I could find at the time and set the Max Transfers value as high as 0x1FE0000. At 256 times more than reasoned above, it seems quite absurd. The magic is it has worked just fine, and apparently also in some cases where 0x1FE00 hasn't. I would like to know why it works, whether it is just waiting to cause trouble, or if there is any other reason to change it. I was thinking perhaps it is not taking the entire 32-bit value into account, but the less significant word is all zeroes so wouldn't it likely result in a MaxTransfer of 0 then?

06 December 2010, 13:53	#11
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	Actually there shouldn't be a problem with MaxTransfer. The device driver should know the hardware limitations and automatically split large packets into smaller ones. Just like ping and MTU in TCP/IP world. But scsi.device does not do this. If you specify a MaxTransfer value larger than 0x1fe00, it will send a mangled command to the device, resulting in corrupted data. So it is important to set MaxTransfer to a reasonable value when using scsi.device. However, it also depends on the version of scsi.device which is used. For example when using IDEfix, a new scsi.device is used which does automatically split large packets. So with IDEfix MaxTransfer should not be an issue. Finally MaxTransfer issues only arise when using files larger than the critical limit. Files smaller than 127.5K are not affected at all. Similarly large files might not be affected if a copy program is used which does not attempt to write more than 127.5K at a time.

02 June 2011, 20:01	#12
Gordon Settler Join Date: Feb 2007 Location: Serf City Posts: 1,760	Where can I set the max transfers properly? I have tried HD tool box but it doesnt save my changes? im trying to put 0x1fe00 but it kees changeing. Do i need to change the mask as well?

02 June 2011, 20:09	#13
Retro-Nerd Missile Command Champion Join Date: Aug 2005 Location: Germany Age: 52 Posts: 12,435	You have to press the enter button after typing 1fe00, then save the settings.

02 June 2011, 20:15	#14
Jope - Join Date: Jul 2003 Location: Helsinki / Finland Age: 43 Posts: 9,861	No need to adjust the mask.

02 June 2011, 20:20	#15
Gordon Settler Join Date: Feb 2007 Location: Serf City Posts: 1,760	Thank you Retro-Nerd for the very quick response, that worked. I cant believe thats what I was missing! Ok Jope thanks.

05 June 2011, 00:12	#18
rkauer I hate potatos and shirts Join Date: Oct 2007 Location: Sao Leopoldo / Brazil Age: 58 Posts: 3,482	The real problem is not the sector count, but the fact compact flash & microdrives doesn't have a built-in buffer . Notice that real HD doesn't suffer from this issue, from the old IDE to SCSI drives. So anything over 255 will choke in the Amiga IDE bus (read "Gayle"), who will still try to put more data than the slow IDE channel can really cope.

05 June 2011, 00:44	#20
Retro-Nerd Missile Command Champion Join Date: Aug 2005 Location: Germany Age: 52 Posts: 12,435	Really? I never had any issues with standard MaxTransfer settings and PC hard drives. Pure luck?

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)