Clearing up the best files for Large Hard Drive support
 

I've mentioned the next ClassicWB packs are going to have all the necessary files for Large Hard Drive support. That means files to allow up to 8GB without a patched "scsi.device" and >8GB with a patched "scsi.device".

Problem is, there are a lot of different solutions and standards to achieve this, so I want to cut down to the best files to use as I'm confused. :crazy

Partitioning and software that support large hard drives:

• A replacement for HDToolbox http://aminet.net/package/disk/misc/hdinst
• Utility to check partitions are safe http://aminet.net/package/disk/misc/check4gb

Replacement file system for up to 8GB support:

• SFS v1.84 http://www.xs4all.nl/~hjohn/SFS184.lha
• Standard FFS v40.1/40.4 patched with TD64 http://aminet.net/search?query=td64 to become FFS v44.5
• FFS v43.20 from this page http://os.amigaworld.de/index.php?lang=en&page=37 patched to remove the beta yellow alert on boot.

Replacement "scsi.device" for greater than 8GB support:

• IDEfix (commercial)
• doobrey's patched scsi.device v44.2 http://www.doobreynet.co.uk/amiga.html
• patching the scsi.device v43.24 from this page http://os.amigaworld.de/index.php?lang=en&page=37 with these patches http://aminet.net/package/util/sys/patchstrip

The latest SFS or PFS can be used with the above "scsi.device" replacements for large hard drive support.

Have I missed any?

What does the NSD patch do? Is it necessary? http://aminet.net/search?query=nsdpatch Where does it fit in to all of this?

I'm confused as to which are the best files to include in the next packs? There are a few options and I want to keep things as simple as possible.

Please help! All the talk of NSD64, TD64, scsidirect and several different versions of these files is a little overwhelming! :)

Please do not recommend HDInstTool. It does not work any better than HDToolbox (except that it displays the right numbers), but it has one major bug which makes it incompatible to any other partitioning tool: it does not use the values for cylinders/heads/sectors which the user enters or which are already stored in the RDB, but it always uses its own values internally. This makes it completely unusable for editing HDDs which were already partitioned with another program. And it also makes it unusable for big harddrives if the values it uses are not good.

Actually it is 44.2 and it needs 43.43 (OS 3.9 BB2) as base, so it is effectively not freely available as opposed to all the other files.


It patches existing device drivers to support NSD commands.

Not in any situation.


NSD, TD64 and direct-SCSI a.k.a. HD_SCSICMD are three different command sets which can be used to access large harddrives. It is not necessary to understand which command set does what. It is only needed to find a combination of file system and device driver which both "speak" the same command language.

See these two lists (from memory, might be incomplete or wrong):

scsi.device V40: NSD -> no, TD64 -> no, SCSI -> yes, but only up to 7.8 GiB.
scsi.device V43+: NSD -> yes, TD64 -> no, SCSI -> yes
scsi.device V116+ (IDEfix): NSD -> yes, TD64 -> yes, SCSI -> yes
cybppc.device (and most other Phase5 controllers): NSD -> no, TD64 -> yes, SCSI -> yes

FFS V40 and below: NSD -> no, TD64 -> no, SCSI -> no
FFS V43: NSD -> yes, TD64 -> no, SCSI -> no
FFS V44 (FFSTD64): NSD -> no, TD64 -> yes, SCSI -> yes
FFS V45 (OS 3.5 or 3.9): like V43
PFS: NSD -> no, TD64 -> yes, SCSI -> no
PFSds: NSD -> no, TD64 -> no, SCSI -> yes
SFS V1.84: NSD -> yes, TD64 -> yes, SCSI -> yes
SFS V1.279: NSD -> yes, TD64 -> yes, SCSI -> no

Now take a device and a file system which both have a "yes" in the same column, then this combination can be used for large hard drives. If you don't find a combination, large hard drives are not possible. Basically that's all check4gb does, nothing else.

Depending on whether the combination is already active before the boot process starts or not, the boot partition has to be inside the first 4GB or not.

For example cybppc.device + PFS3 are active early enough so that you can create boot partitions of any size and anwhere on the disk. But scsi.device V43 or higher is usually loaded only after the boot partition is already mounted, therefore the boot partition has to be inside the first 4GB (or inside the first 7.8 GB if you use a file system which "speaks" SCSI).

Is this enough confusion for now ?

Oh man, Thomas you are awesome as usual! That makes it all really clear - really appreciate your help. :)

I don't have the time I used to for working on the packs, so being able to rely on knowledgeable Amiga users when doing the updates saves loads of time. :great

Ok, partitioning. I'll take your advice and won't include the "HDInstall" replacement. I take it HDtoolbox can see up to 8GB yes? But it can be used for larger than 8GB drives, but doesn't calculate the correct values. Is that correct?

So do you think it's best just to stick to the standard HDtoolbox or do you know a better utility that handles greater than 8GB calculations?

Ok, on to the "scsi.device". doobrey's patch uses a OS3.9 file - since it isn't free AND not well tested, I guess I'm better sticking to the patched v43.24. I also noticed there is a patch on Aminet:

http://aminet.net/package/driver/media/SCSI4345p

but again, this uses the OS3.9 scsi.device so I can't include it except for the ClassicWB OS39 pack, which could do the patching during the install.

In your opinion (or anyone else that may know/tested these), does the patched v43.24 work adequately and without problems? Or are their definite advantages to using the v43.45 or v44.2?

Finally, the command sets. So NSD, TD64 and direct-SCSI a.k.a. HD_SCSICMD are all ways for the device driver and filesystem to communicate and support larger hard drives.

As long as I have a scsi.device and file system that can communicate using one of those, the others are not required. Is that right?

You should include both. I just wish that you don't prefer HDInstTool over HDToolbox. HDInstTool is the best we have regarding usability, if it only hadn't this damn bug. HDToolbox fails to display numbers bigger than 4096MB and it even fails to draw the diagram correctly if partitions are becoming huge. But HDToolbox does everything correctly if you enter numbers into text fields, e.g. cylinders/heads/sectors on the Install Drive page as well as start and end cylinders for partitions.


HDToolbox calculates correctly, it just displays wrong values. The cause is just the same as the 4GB problem: numbers which do not fit into 32 bits are cut off.

It "sees" everything the driver can see. It is scsi.device which sees only 8GB. As soon as scsi.device has been patched to support larger drives, HDToolbox allows to partition the larger space.


I don't dare to give a recommendation. I for myself use HDToolbox, but I also know how to handle it and how to identify wrong values (it's obvious that if you make a partition which covers one half of a 80 GB drive and HDtoolbox displays it as 2000 MB, that the display is wrong. But what about a 6 GB partition which is also displayed as 2000 MB. That's not so obvious.)

A beginner probably runs better with HDInstTool, because he does not care about internal values anyway. But HDInstTool cannot be used to edit a partition table which was created by another program or to recover an RDB which was created by another program. I.e. HDInstTool cannot be used for "advanced" actions.


The only other program I know is SCSIConfig from Phase5's SCSI disk. http://powerup.amigaworld.de/index.php?lang=en&page=13
I can't remember if I ever used it.


43.24 is perfectly stable IMHO, but it cannot handle harddrives bigger than 128 GB (the drives can be used, but only 128 GB can be seen. Just like the 8GB of version 40).

43.43 (the unpatched BB2 version) crashes when using harddrives bigger than 128 GB.

43.45 (and probably Doobrey's too) is the only version which can use LBA48 (> 128 GB drives). No idea if it's safe, though. I never had anything bigger than 20 GB in one of my Amigas.

The scsi.device V40 may not support NSD commands and 64bit addressing right out of the box .. but that's what NSDPatch is for.
Call NSDPatch in ur startup-sequence when using Kick 3.1 + WB 3.1 and u don't need to load a patched scsi.device or have to burn an updated Kickstart-EPROM.

And NSDPatch does much more than that.

cybppc.device, cybscsi.device and many other devices are covered by NSDPatch too.

The only reason why cybppc.device is supporting TD64 right out of the box is that TD64 is a brainchild of Ralph Schmidt (and Ralph Babel and others of course).


Ciao, Rick ...

btw ... NSD was the official solution of AT and is part of AmigaOS 3.5 and 3.9 .. so it was the way to go for me.

TD64 was a dead end in my eyes.

But u can still use TD64, especially on older Kickstarts.


Ciao, Rick ...

Thanks Thomas for getting back to me and to you Skylight for adding to the info. :great

I'm now crystal clear on how everything works. I'll probably use a lot of this info in a sticky when I get around to updating the >4GB thread.

It would be useful in the abime wiki too I'd imagine.

btw .. i forgot to mention the NSDPatch 43.20 works with Kickstart 2.04 and up.
So u don't need Kick 3.1 to use it.


Ciao, Rick ...

Bullshit. NSDPatch translates NSD commands into HD_SCSICMD commands. So you only gain additional 3.8 GB for the price that the boot partition must be in the first 4GB and you cannot use the upper part if you boot from floppy for example.

The same can be achieved by FFSTD64 or SFS V1.84 or PFS3ds in a much more comfortable way.


Which is completely useless because there is not a single program which supports it (except OS 3.9's HDToolbox perhaps).


IDEfix and scsi.device V43 do work on Kick 2.0, either.


You still need a <4GB partition to run NSDPatch while a file system supporting TD64 gives you >4GB support at boot time.


NSD was dictated by one person at AT against the majority of third-party manufacturers.

Regarding 64bit support NSD and TD64 work just the same way (put the upper 32 bits into io_Actual), they just use different command codes (24 vs. 0xc000). So IMHO one should prefer the command set which supplies more compatibility. And for Phase5 and Elaborate Bytes controllers this is TD64 because their firmware does not support NSD without a patch.

Ah, a nice and juicy thread! I appreciate your efforts at trying to get info down on all these obstacles and custom drivers/tools/file systems :great

I've visited Thomas' webpage and read the stuff here, but I'm having a hard time getting "into" all the stuff. I must say I've not quite been "un-confused" by this thread either, hehe.

If I take an example: I have an A600 with kick 3.1 and WB 3.1 (and also an A1200-060 with the same kick/WB, but the question is about the A600). I'd like to keep WB 3.1 on both. On both I have a Hong Kong IDE-CF adapter and a SanDisk Extreme IV CF-card.

If I were to buy a bigger CF-card, what is the largest I can go, while keeping some FFS-compatible filesystem (ie. sees standard FFS partition and WinUAE "harddisk folders" for when I backup stuff), how big a card could I buy, and how small do the partitions need to be? And what files would I need?

If I get some tips on what to go with, I could write up a tutorial when I've made it work.

As always: "it depends".

For drives up to 8.45 GB (1 GB = 10 to the power of 9 bytes) or 7.87 GiB (1 GiB = 2 to the power of 30 bytes) it is sufficient (and most efficient) to install a file system which supports HD_SCSICMD. You don't need any further patch and can create partitions anywhere and as big as you like. Such file systems are SFS 1.84, FFSTD64 or PFS2/3ds. (Links see in Bloodwych's first post).

For bigger drives you also need a patch for scsi.device. Please note that each patch needs some memory as well as the buffers for partitions need memory. The more partitions you make and the bigger the partitions are, the more buffers you need. For a 40 GB HDD with let's say 10 partitions (historically grown) with buffers for decent performance you might loose 1 or 2 MB of memory. For an A600 this is clearly too much. So for computers with few RAM I would stick with small hard drives.

Nevertheless, you can put scsi.device (any version, preferably 43.45 or 44.2) into the Devs directory and add a line like

loadmodule devs:scsi.device

to the top of your startup-sequence and there you go with big harddrives up to 2 TiB / 2.2 TB. (You see, the difference between binary and decimal measurement units grows significantly in these dimensions. The difference between 2 TB and 2 TiB is 200 GB).

With big harddrives also the limits of the file systems become relevant. With PFS3 you can use partitions up to 107 GB, with SFS up to 128 GB. I don't know the limit of FFS and how it calculates but I am sure that with a block size of 1024 bytes you can safely create partitions of 20 or 40 GB. With SFS2 there is no limit AFAIK (SFS2 = SFS with an identifier of 0x53465302).

For the decision which file system to use you should look into the two tables I mentioned above. Take the version of scsi.device you've got, look up which command sets it supports and choose a file system which also supports at least one of these command sets.

Whether you go the SFS, the PFS or the FFS route is up to your personal taste. SFS and PFS both are much faster than FFS but less redundant and therefore more difficult to recover in case of an error.

Edit: SFS needs a 68020 and PFS maybe too (I am not sure atm). So they are not usable on a plain A600.

Standard FFS partitions are always seen, because FFS is built into the Kickstart ROM (at least in 2.0 and above).

Harddisk folders are completely independant from all of the above because they are completely handled by WinUAE internally.

And of course you can use as many file systems as fit into your RAM. Each partition can have a different file system. So which file system you use for one HDD is completely unrelated to other HDDs. Only if you try to run all this with a very small amount of RAM, you'll run into troubles.

I know some people don't like NSD.
And there were even religious wars between TD64 and NSD believers.
But "Bullshit" is something else ...

Do u wanna tell me u can only use 7.8 GiB when using NSDPatch on Kick 3.1 with FFS V43 and SFS???
Are u sure?
I'm pretty sure i used 36 GB IBM SCSI hard drives with Kick 3.1 and NSDPatch before i moved on to AmigaOS 3.9.
And here is the NSDPatch already included in setpatch.

I'm in Bangkok now, but when i'm back home i will check what NSDPatch will support on Kick 3.1 and even on Kick 2.04.

btw ... u can load setpatch and NSDPatch even from floppy. ;)

And with a patched scsi.device u would have the same "Floppy-Problem", if u don't burn an updated Kickstart-EPROM.

What's the problem with that?
U must keep ur first partition below the 4GB boundary anyway if u don't use an Phase5 or Elaborate Bytes controller or a patched scsi.device in an updated Kickstart-EPROM with TD64 support right out of the box.

I always had a System partition with max. 2GB and FFS and every other partition could use the remaining disk space.
For large partitions i prefer SFS anyway.
And i never had any problems.

We didn't talk about the best solution for Phase5 and Elaborate Bytes controllers, but about general solutions!

And NSD would cover more devices than TD64.

NSD is part is part of AmigaOS 3.5, 3.9 and 4.x anyway.

Like it or not, but NSDPatch is an easy way to use large hard drives, especially with scsi.device and other devices without TD64 support right out of the box.


Ciao, Rick ...

I have 7.5 MB ram right now, to make it not overlap PCMCIA memory addresses. I can add a 4MB PCMCIA ram card (if I could somehow get hold of one! :P)

The reason I mentioned WinUAE is because it seemed I had succeeded in partitioning and formatting a CF card as PFS. But when booting from it, it couldn't see the harddisk folders that FFS could see, and if I booted from a harddisk folder, it couldn't see the PFS volume. So I couldn't really do anything. It's likely that I simply messed up the install.

PFS should work fine, but some parts of SFS do require 68020. Buzz will maybe have a look at recompiling the last version that has a source on SourceForge when he has time, I think.

CF cards are pretty expensive at sizes over 8GB, and SSD is of course the best you can get (apart from battery-backed ram drives), but much faster than the CPU/DMA can reach. So CF cards are perfect.

So if I would do it right away, I'd probably go with PFS2/3ds. Redundancy/error-fixing is not really an issue since I'm a backup-freak and it's so quick to copy even 8GB back.

If partition size is not limited to < 8GB, I'll go with one partition.


On the other hand, I'm using two 980MB partitions on a 2GB card with FFS, and really, it's all I need. Using a single card for all my Amigas saves tons of copying "the same old files to each card and keep them all up to date", so maybe I'll go with that and spend some time on an "adaptive startup-sequence" to cope with the differences in setups (see signature) instead. :)

With PFS buffer size calculation is easy. It only suports a block size of 512 bytes which means two blocks are one kilobyte. I don't remember exactly but if I am not wrong the manual recommends at least 300 buffers (150 kb) for each partition but not more than 600 buffers (300 kb) even for the largest partition.

FFS is different. Firstly it supports block sizes up to 32768 bytes and secondly it runs faster the more buffers it has. 1000 or 2000 buffers can make sense when working with large files.

It's rather difficult to hide directories mounted in WinUAE from Workbench. I wonder how you made this.

Edit: perhaps you gave them the same names as the partitions ? Duplicate names can cause all kinds of strange things.

Another question.

Whilst the v40 scsi.device from 3.1 supports up to 8GB through directSCSI, what about the v37 scsi.device in the 3.0 ROM?

And in the 2.0 ROM?

Can they support 8GB in the same way using SFS v1.84 and FFS v44 FFSTD64 without further patches?

Not sure, but it should work the same. HD_SCSICMD was available from the very beginning of scsi.device and the 8 GB limit arises from the way the hardware is addressed (CHS mode with a maximum of 16383 cylinders, 16 heads and 63 sectors).

37 is the version of the 2.0 ROM, so if 3.0 still uses the same version, it seems they didn't make many changes.

Forgive my stupidity but is SFS 1.84 newer than 1.279 ? 1.8 is greater than 1.2
BUT if the revision is to be read as v1 revision 84 VS v1 revision 279.... I've done this before and got it wrong so could someone enlighten please ? ;)

v1.84 can be read as v1.084, so it's older than v1.279. ;)

Version numbers are usually two (or more) separate numbers that just happen to be separated by dots. For example, 2.13 is newer than 2.7, but 1.10.3 is newer than 1.10

Sorry, I missed your post, so I anwser only now.


I am not about liking or not, I am about usabiliy.

Yes, I am absolutely sure. Read the docs about what NSDPatch does and check what scsi.device can do. scsi.device can access only 7.8 GiB and there is no patch which could change this except a complete replacement.


You cannot connect a SCSI drive to the internal IDE controller. That's what we are talking about, the internal IDE controller of some Amiga models. There are other controllers which also use the name scsi.device, but their limits are different.

Such a problem simply does not exist when using SFS 1.84 or FFS V44.


How many publicly used controllers do exist which are not Phase5 or Elaborate Bytes in one or the other way ?


But there are other people who have problems, otherwise this thread would not exist.


Really ? How many more ? How big is the difference between "general" and "Phase5" ?


But they all do not work on an unexpanded A600.


No, it's not easy. That's what it's all about. It's not easy. With NSDPatch you can use up to 7.8 GiB on the internal IDE bus. And you need a new file system, too. This can be done with a new file system only, NSDPatch is not needed. And to use more than 7.8GiB on the internal IDE bus, you need a replacement for scsi.device as well as a new file system. All existing replacements for scsi.device do support NSD already. NSDPatch is not needed.

I do not complain about NSD in general. I only say that NSDPatch is not needed in any cases. And it does not add large harddrive support to devices which do not already have it through HD_SCSICMD.

And again, if a device already supports larger harddrives by HD_SCSICMD, then it's much easier and much more comfortable to use a HD_SCSICMD file system rather than NSDPatch.

NSDPatch is not needed. Nice to have, but not needed. It does not give any functionality to the average user. From a programmer's point of view it might be different, but a user can live better without it.