Emulation error

[jdow]

A long time ago in a world far far away (from Microsoft) I used to do software development for a little known and vastly under appreciated computer known as the Amiga.

So much for the introduction. I'll note that I used to develop SCSI software for Microbotics. (I also did the HD configuration tool for 3.9.) This means I tended to leave divots on various and sundry filesystems on the disks I had. One of the most helpful tools I used was something from CATS called DiskEd.

It appears it does not work on WinUAE.

There is another handy dandy tool from CATS called "DevInf". It prints out gobs of obscure data of deep interest to device developers. The disk I want to edit is rather clearly mounted as "DH0:" on a cloned AmigaForever AmiKit. I notice something rather singularly interesting about it. The partitions are labeled to be using uaehf.device. This seems to "work" in a qualified sense. My older disk prep tool requires a SCSI interface and does not work on the drives. HDWrench/HDWTest works. So I know uaehf.device is the device that needs to be accessed. DevInf reports DH0: has an fssm record that points to a non-existent "uae.device".

I am making a giant logical jump here that "uae.device" is an error somewhere in the bowels of the so far remarkably excellent WinUAE/UAE code. It might be worth fixing it. That would save me writing my own version of DiskEd. I've more exciting things to do.

{^_-} Joanne

[Toni Wilen]

I guess you are trying to use directory filesystem drive, it does not have block device (uae.device is just a fake name because something needs to be in that field). It is more like a network drive or ram disk.

Only RDB hardfiles (or real harddrives) have real block device have SCSI support.

Standard single partition hardfiles reject SCSI commands to protect it from accidental data loss if someone attempts to use hdtoolbox with it.

[jdow]

Erm, I am using an image of an 18 GHz disk with (cough) custom RDBs on it. (There are atrocities you can commit with RDBs. I committed most of them on this disk. {^_-} Hey, I helped WRITE the RDB specification with Randal Jesup. These days with ubiquitous Internet connections I'd resort to using signing certificates for the filesystems embedded within RDBs. Also may I suggest that uaehf.device disable interpreting the driverinit LSEGs? Simply drop them on the floor. To my knowledge only one company implemented them and by the Deathbed Vigil I believe they had abandoned them.)

DiskEd edits the raw disk. You feed it the command "DiskEd DH0:" or "DiskEd DH0", it's pleasantly agnostic about that it appears. That goes off and searches for the DH0 device information and pulls out "uae.device". And it promptly comes up empty. So you do need to patch that error if DiskEd and some other related tools are going to work right. You probably need a copy of that and of DevInf. They were in some old devcon packages. They are assuredly on the CATS CDROM, if you have it.

Speaking of SCSI - your implementation appears to be faulty. The old HardFrame RDB tool, RDPrepX, uses direct SCSI commands to probe for possible multiple SCSI buses, multiple LUNs, as well as multiple SCSI IDs. It bombs out the first time it tries to probe a second LUN. It also uses mode sense and other support commands. I'd have to dig out the source code to double check what all it uses. Personally I prefer RDPrepX to HDFoolBox given a choice. I also use the hdwrench.library and the tester tool I built for it. I get finer grade control that way. {^_-}

{^_^} Joanne "RDBs R Us"

[Toni Wilen]

Originally Posted by jdow

Also may I suggest that uaehf.device disable interpreting the driverinit LSEGs?

driveinit is not implemented, never found any documentation (it only says "driveinit is not yet supported." in log if it is set in RDB). And I want to emulate everything just because you can, you are supposed to be able to run some ancient and stupid software on emulation. Even if it is most stupid thing to do

Normal LSEG blocks of course needs to be handled.

If you want "official" support, mount the HDF as A600/A1200/A4000 IDE or A590/A2091 SCSI drive (You also need A590/A2091 boot rom image). This bypasses uaehf.device and uses official binary driver. Good idea for comparison purposes too.

DiskEd edits the raw disk. You feed it the command "DiskEd DH0:" or "DiskEd DH0", it's pleasantly agnostic about that it appears. That goes off and searches for the DH0 device information and pulls out "uae.device". And it promptly comes up empty.

RDB hardfiles shouldn't have anything to do with uae.device, it is directory filesystem only. Are you sure you don't have any directory filesystems active at the same time?

Speaking of SCSI - your implementation appears to be faulty. The old HardFrame RDB tool, RDPrepX, uses direct SCSI commands to probe for possible multiple SCSI buses, multiple LUNs, as well as multiple SCSI IDs. It bombs out the first time it tries to probe a second LUN. It also uses mode sense and other support commands. I'd have to dig out the source code to double check what all it uses. Personally I prefer RDPrepX to HDFoolBox given a choice. I also use the hdwrench.library and the tester tool I built for it. I get finer grade control that way. {^_-}

I am quite sure one has used those tools (until now). hdtoolbox and other popular tools that use scsi commands work fine.

It should return normal CHECK CONDITION + ILLEGAL REQUEST/INVALID LUN if lun is > 0. (Not much point in supporting LUNs with HD emulation anyway)

Even Linux/NetBSD or even Amix works with low level SCSI emulation so the problem has to be somewhere, probably uaehf.device HD_SCSICMD does not update all structure fields 100% correctly.

Attach all tools with problems and those problems will disappear. No need for sources but I'd would of course help.

btw, does those tools support SCSI-II or only SCSI-I? Because very ancient versions of hdtoolbox for example don't like what INQUIRY returns if drive is SCSI-II.

[thomas]

uae.device is a fake name which is only used by directory file systems. You are probably mixing up your DH0 partition with a DH0 directory mounted by WinUAE. Check for DH0.1 which is probably your HDD partition.

[mark_k]

Nice to see another Amiga Wizard(ess) from the old days on this board.

I downloaded RDPrep from Aminet. After adding a DEVICE=uaehf.device tooltype to its icon and running it, I get a software failure when it runs rdprepx to scan for disks. (I'm booting Workbench 3.1 with an RDB-type hardfile.)

[jdow]

Originally Posted by Toni Wilen

driveinit is not implemented, never found any documentation (it only says "driveinit is not yet supported." in log if it is set in RDB). And I want to emulate everything just because you can, you are supposed to be able to run some ancient and stupid software on emulation. Even if it is most stupid thing to do

Normal LSEG blocks of course needs to be handled.

You have very private mail regarding the above. There is a reason this was VERY seldom implemented.

If you want "official" support, mount the HDF as A600/A1200/A4000 IDE or A590/A2091 SCSI drive (You also need A590/A2091 boot rom image). This bypasses uaehf.device and uses official binary driver. Good idea for comparison purposes too.

Hm, don't have an A2091 image. Is there a clean one floating around? (I am pretty sure that will not clean up the DiskEd issue.)

RDB hardfiles shouldn't have anything to do with uae.device, it is directory filesystem only. Are you sure you don't have any directory filesystems active at the same time?

Quite a few - that 18G disk has a system partition, whole lot of modest size partitions copied from smaller disks using FFS and one huge SFS partition for half the disk and in some spare space at the end a backup system partition. The spare space partition has block zero contaminated. I can tell that from the way it doesn't mount and diagnostics I can get from 3.9's hdfoolbox replacement using my low level test too.

I am quite sure one has used those tools (until now). hdtoolbox and other popular tools that use scsi commands work fine.

It should return normal CHECK CONDITION + ILLEGAL REQUEST/INVALID LUN if lun is > 0. (Not much point in supporting LUNs with HD emulation anyway)

Even Linux/NetBSD or even Amix works with low level SCSI emulation so the problem has to be somewhere, probably uaehf.device HD_SCSICMD does not update all structure fields 100% correctly.

That might be. RDPrepX went off and used a bunch of the direct access SCSI commands to help it figure out an optimum formatting mode. It fails on its scan for disks as soon as it tries to read LUN 1 on ID 0. I'll have to dig it out of the code tomorrow if my age enfeebled mind can still read that code. {^_-}

The DiskEd tool is the one I need working. Is the CATS CDROM still floating around? If you don't have that CDROM I'll have to drag it off my image to the Windows machine and email it to you. That CDROM is a must have for developers.

Attach all tools with problems and those problems will disappear. No need for sources but I'd would of course help.

btw, does those tools support SCSI-II or only SCSI-I? Because very ancient versions of hdtoolbox for example don't like what INQUIRY returns if drive is SCSI-II.

Not using HDToolBox. It always made me nervous the way it worked. When you have large amounts of source code on disks on your system you don't want to risk it getting messed up. And I had my own very nice tool that supported Inquiry correctly and all that. It was ugly in that it was a command tool. Dan Barrans did the GUI for it. (I wonder how he is these days.)

Heh, did you know that you CAN embed the RDBs within your first partition? I simply reserved 256 blocks. And because of behaviors of other OSs I started allocating RDBs at block 3. That way a Windows machine was not likely to cream my disks if I hooked them up to one. And the RDBs coexisted nicely using crossdos. So the disk's first block starts out DOS\1 on that giant disk - because I can. That block is still OK. Something else messed up the last partition on the disk's first block. Other diagnostics indicate the partition is still there with its data.

{^_^}

Originally Posted by thomas

uae.device is a fake name which is only used by directory file systems. You are probably mixing up your DH0 partition with a DH0 directory mounted by WinUAE. Check for DH0.1 which is probably your HDD partition.

Erm, DiskEd accesses raw disk. And it knows the device name about the only good way to find it - from the fssm block which has the bad name in it. That SHOULD be an easy fix.

Momma knows what she's doing. My first Amiga was delivered unto me Nov 2 1985. And I both developed driver code and was chief moderator for the Amiga conferences on BIX through its demise - when I was the BIX sysmgr as well.

{^_-}

Originally Posted by mark_k

Nice to see another Amiga Wizard(ess) from the old days on this board.

I downloaded RDPrep from Aminet. After adding a DEVICE=uaehf.device tooltype to its icon and running it, I get a software failure when it runs rdprepx to scan for disks. (I'm booting Workbench 3.1 with an RDB-type hardfile.)

That IS a commandline program in its "rdprepx" incarnation, you know. It will work without the GUI.

Yup - exactly. I don't remember if it ever worked with IDE drives. I think I used it on my A1200. I still have that one somewhere, too. (Also have two exotic A4000s that were once part of a theme park, an A3000T that's ill (bad ram chip on the board), a couple A3000s, and a seriously hypertrophied A2000. Oh, and both a CDTV and a CD32. I believe the CDTV has 2.04 era ROMs in it, too. This girl's been around.

{^_^}

[thomas]

Not sure if we understand each other correctly. A so-called "directory file system" is one which is added to WinUAE when you use the "add directory or archive" button. It works like a network file system. There is no real device behind it. "uae.device" is put into the fssm just because something has to be put there. You won't ever be able to access this .device even if it was named differently.

The thing is that directory file systems are added by WinUAE as DH0, DH1, DH2 and so on. So if your 18gig HDD has a partition called DH0, too, there is a conflict. Such conflicts usually are resolved by the drivers by adding .1 to the partition name. My guess is that UAE's directory file systems take precedence over real harddrives and such your real harddrive DH0 partition is renamed to DH0.1 while the directory DH0 keeps its name.

Therefore I suggest you check the fssm of DH0.1 which should then correctly contain uaehf.device.

[Toni Wilen]

Originally Posted by jdow

Erm, DiskEd accesses raw disk. And it knows the device name about the only good way to find it - from the fssm block which has the bad name in it. That SHOULD be an easy fix.

It has to be caused by something else because FSSM device name is uaehf.device, even Scout Amiga system utility shows uaehf.device if hardfile and uae.device if directory filesystem.

[Toni Wilen]

Originally Posted by mark_k

Nice to see another Amiga Wizard(ess) from the old days on this board.

I downloaded RDPrep from Aminet. After adding a DEVICE=uaehf.device tooltype to its icon and running it, I get a software failure when it runs rdprepx to scan for disks. (I'm booting Workbench 3.1 with an RDB-type hardfile.)

Worked for me with 2G RDB hardfile with single partition. Guru is division by zero which usually points to some unsupported geometry value or something.

[jdow]

Originally Posted by Toni Wilen

It has to be caused by something else because FSSM device name is uaehf.device, even Scout Amiga system utility shows uaehf.device if hardfile and uae.device if directory filesystem.

OK, I looked closer at the tools output. The ordering of the output confused me last night. I'm sorry about that. I'm attaching some utilities that are interesting and the output of some of them. Good luck making DiskEd work. It says it cannot find the device. WinUAE is not doing what I thought it was doing. Let's see what happens when I try other ways to access the disk. I'll spend more time diagnosing it.

RDPrepX issue: (I did find the problem - noted below.)
Meanwhile here's rdprepx to play with. Suppose you simply want a list of drives rdprepx can work with the command is "rdprepx -d uaehf.device". That simply stops after the first SCSI id is accessed as it moves on to the second LUN. That's what the single dot means. It should show a dot for every board, LUN, and ID. (note: unit 123 would be board 1 (the second board), LUN 2, and SCSI ID 3.)

For reading out the RDBs on unit 2, SCSI ID 2, you'd use this command: "rdprepx -d uaehf.device -u 2 -t -s 002.list" The "002.list" name is simply a convention I have here to keep mountlist copies nicely named. "ml" would work just as well. The "-t" keeps rdprepx from searching for all disks before grabbing the mountlist data out of the rdbs and saving it. If you take -t out rdprepx quietly fails again after printing out one dot.

The testing loop prints a dot. Then it goes off to a testing routine. It opens the device. If that fails it exits. Otherwise it performs up to three TestUnitReady commands. (Some drives responded slowly.) Then it performs a SCSI Inquiry. Again it tries twice due to some funkity drives. It then performs a raw read. If the -n option was entered it perfoms another SCSI Inquiry and prints the returned name for each drive found. It tests with an outer loop of board, middle loop of LUN, and inner loop of SCSI ID. So it appears I was wrong on the LUN issue. The failure comes within the test for a SCSI ID that's not present.

And with some comparison with the hdwrench.library output I note that your device driver has the same problem as the ApolloSCSI - it reported HFERR_NoBoard rather than TDERR_BadUnitNum when testing unit 0. That kills the loop. I've fixed rdprepx to include the ApolloSCSI exception I placed in hdwrench.library. So that's sort of fixed for me. You might want to be nice and change the error report to BadUnitNum if there are other valid units on the selected board ID. (I suspect two ApolloSCSI boards would fail out the second board if the first SCSI ID was not zero. But who, other than me, had more than one board in a box?)

DISKED item:
Note that disked apparently has some other form of problem. "DiskEd WB_2.x" and "DiskEd SFS.1" both fail.

NOTE: I am adding a fixed rdprepx, called rdprepz for convenience. It has considerable debug output to play with, particularly with a "-q 0" option.

{^_^}

Originally Posted by Toni Wilen

Worked for me with 2G RDB hardfile with single partition. Guru is division by zero which usually points to some unsupported geometry value or something.

The error he saw was probably because "rdprep" did not know to use uaehf.device and fed rdprepx bad values. I always use rdprepx as a CLI tool and avoid one potential set of problems.

{^_-}

[Toni Wilen]

Originally Posted by jdow

And with some comparison with the hdwrench.library output I note that your device driver has the same problem as the ApolloSCSI - it reported HFERR_NoBoard rather than TDERR_BadUnitNum when testing unit 0. That kills the loop.

This is by design. Due to internal design of emulation, uaehf.device unit numbers are also reserved for directory filesystems (if first filesystem in Harddrives list is directory, it gets unit 0 and so on.. these all return HFERR_NoBoard)

This was made for HDToolBox (which is the most common HD tool for "normal" users) compatibility, if it gets NoBoard it will keep enumerating later units. If it gets other error, it stops. Other tools seem to do just the opposite (for example hdinsttools). Annoying.

If you are going to use low level HD utilities, it is good idea to move all directory harddrives after HDFs in HardDrives panel. This way first HDF (the one you want to work with) will get unit 0 and so on..

I'll test other problems later.

[jdow]

Originally Posted by Toni Wilen

This is by design. Due to internal design of emulation, uaehf.device unit numbers are also reserved for directory filesystems (if first filesystem in Harddrives list is directory, it gets unit 0 and so on.. these all return HFERR_NoBoard)

This was made for HDToolBox (which is the most common HD tool for "normal" users) compatibility, if it gets NoBoard it will keep enumerating later units. If it gets other error, it stops. Other tools seem to do just the opposite (for example hdinsttools). Annoying.

If you are going to use low level HD utilities, it is good idea to move all directory harddrives after HDFs in HardDrives panel. This way first HDF (the one you want to work with) will get unit 0 and so on..

I'll test other problems later.

Technically, though, the error results are dead wrong. HDToolBox will properly run if you return TDERR_BadUnitNum. That is the expected return. Allowing HFERR_NoBoard to work was a concession for the ApolloSCSI.

HDToolBox works on the Microbotics HardFrame, which returned proper values. And RDPrepX works on the Commodore scsi.device in A3000s and GVP's scsi boards. That shows BadUnitNum is the expected return. It would be nice to get it right in memorial of Steve Beats and Randall Jesup if not me. Steve did the original work on the Commodore drivers. Randall did lots of cleanup and developed the RDBs in conjunction with me and GVP. Those were fun days. (Amigas read disks faster than any other OS around. It was used for testing disk speeds for several years in the early days. That's how I got some cheap drives. {^_-})

It's probably no big deal except to perfectionists. However, getting DiskEd to work would be nice.

(Low level disk work is a "joy". All manner of interesting bugs turn up. I just discovered Windows will happily truncate off the end of a disk to report the size as a multiple of 822580 bytes - 512 bytes/sector, 63 cylinders, 255 sectors/cylinder. Yick. I gotta boot to CloneZilla and take a new copy of that 18G SCSI disk to get the last blocks. SIGH! It's not that important. It's just annoying.)

{^_^} Joanne

[Toni Wilen]

Originally Posted by jdow

Technically, though, the error results are dead wrong. HDToolBox will properly run if you return TDERR_BadUnitNum. That is the expected return. Allowing HFERR_NoBoard to work was a concession for the ApolloSCSI.

HDToolBox works on the Microbotics HardFrame, which returned proper values. And RDPrepX works on the Commodore scsi.device in A3000s and GVP's scsi boards. That shows BadUnitNum is the expected return. It would be nice to get it right in memorial of Steve Beats and Randall Jesup if not me. Steve did the original work on the Commodore drivers. Randall did lots of cleanup and developed the RDBs in conjunction with me and GVP. Those were fun days. (Amigas read disks faster than any other OS around. It was used for testing disk speeds for several years in the early days. That's how I got some cheap drives. {^_-})

I coudn't get hdtoolbox to keep enumerating without using NoBoard. Perhaps it was changed in later hdtoolbox versions (OS3.5+?). I'll need to to retest all versions someday..

(Low level disk work is a "joy". All manner of interesting bugs turn up. I just discovered Windows will happily truncate off the end of a disk to report the size as a multiple of 822580 bytes - 512 bytes/sector, 63 cylinders, 255 sectors/cylinder. Yick. I gotta boot to CloneZilla and take a new copy of that 18G SCSI disk to get the last blocks. SIGH! It's not that important. It's just annoying.)

AFAIK this limit only existed in Windows 2000 and older. XP and later can access all blocks, even if "geometry" does not match but you need to use IOCTL_DISK_GET_LENGTH_INFO to get the real size of drive. Other standard methods return "geometry compatible" size. WinUAE HD to HDF converter will use this method too.

[jdow]

I'll have to give IOCTL_DISK_GET_LENGTH_INFO a try. I used IOCTL_DISK_GET_DRIVE_GEOMETRY. So, yes, your version is better. For Disk to Disk copying that's the way to go. Thanks for the tip.

{^_^}

Now, could you give DiskEd a try and make it work, please.

{^_-}

[Toni Wilen]

Originally Posted by jdow

Now, could you give DiskEd a try and make it work, please.

It didn't work because uaehf.device validates ioreq->io.Message.mn_Length field and only accepted invalid (too small) values if KS ROM was old enough. I guess I'll just remove the test completely.

[jdow]

Originally Posted by Toni Wilen

It didn't work because uaehf.device validates ioreq->io.Message.mn_Length field and only accepted invalid (too small) values if KS ROM was old enough. I guess I'll just remove the test completely.

Hm, I'd make that an emulator flag. I rather prefer validating variable length fields like that. I'm surprised that Commododo never fixed that bug. I guess they did it for historical purposes.

{^_^}

[emufan]

winuae_2710b7.zip

- Accept uaehf.device open call even if ioreq->io.Message.mn_Length is invalid or too small.

not that I understand it's impact or things, but seems added/fixed

[Toni Wilen]

I didn't bother with any extra options, there are too many "bad" programs, for example KS 1.x has it zero if boot drive and other non-boot automount drives also use same mysterious 12 that also disked used.

I simply removed all previous magic number checks.

[SpeedGeek]

Hi Joanne,

I have an hdwrench.library question for you. (It's OT so if the EAB moderators want to move it to another thread that's OK).

OS3.9 HDtoolbox no longer supports the SCSI_MAX_LUN tooltype. This means some 3rd party scsi.device users (e.g. gvpscsi.device, omniscsi.device) have to wait 1 minute to scan all LUNs on systems with no SCSI LUNs installed!

Does hdwrench.library have any secret options to fix this?

Thanks,
Kevin