Simulated Hardfile-Error?

[sigma63]

Hi There,

I'm working on a Disk-Monitor/Editor, so I would like to have some Testcase.

Does Anybody knows how to prepare a Hardfile that it behaves like a faulty Disk (have some Readerrors or such).

Or does Anybody have a Hardfile with some detected BADB-Blocks (the ones that are chained in the Rigid-Disk-Block) ?

TIA

Sigma63

[thomas]

A hardfile cannot have read errors. Just like an ADF it only stores user data, nothing below that level. The only way to get an error when reading from a HDF is to make partitions bigger than the HDF so that it tries to read outside of the disk.

[Toni Wilen]

Also returned error can be device driver specific or even semi-random if error reason is real bad block.

[sigma63]

Originally Posted by thomas

A hardfile cannot have read errors.

Thank you for clarifying this.

My other Question still remains: Does anybody have a harddisk with some BADB-Blocks (created by HD-Toolbox or so) and made a hardfile from this and is willing to share it?

[prowler]

Originally Posted by sigma63

My other Question still remains: Does anybody have a harddisk with some BADB-Blocks (created by HD-Toolbox or so) and made a hardfile from this and is willing to share it?

Is it actually possible to create a hardfile with bad blocks flagged as such?

[thomas]

You can flag any block as bad, but you cannot have true bad blocks in a HDF.

[prowler]

I guess what I'm really asking is would HDTools, for example, actually create a hardfile with sectors flagged as bad from a hard disk with bad sectors? Would it not quit when it encounters a bad sector? (I haven't had any Amiga hard drives with bad sectors, so I really don't know.)

If so, then what the OP wants could be conjured up from a good RDB image and knowledge of how the sectors are mapped as bad in the RDB.

In this way I imagine it would only be possible to mark the block as physically unreadable, but might that not be sufficient to test the OP's proposed Disk-Monitor/Editor error-trapping routines?

[prowler]

Well, it seems that I've opened a can of worms here!

Paul Overaa only briefly mentions hard disks in his otherwise excellent book Amiga Disks and Drives:

Hard Disks

Many of the principles of hard disk operation are similar to those already discussed [in relation to floppy disks]. The larger capacities involved do mean that much more organization is required. Blocks are tagged with a unique identifier, checksummed and linked together and the root of these blocks is contained in somethging called the RigidDiskBlock description.

The disk layout recommended by Commodore is to use the first cylinder to store all the drive data specified by these blocks and this includes partition descriptions, file system load images, bad block maps and spare blocks. At the moment only 512 byte blocks are currently supported by the file system. The RigidDiskBlock structure contains a lot of info and a full description can be obtained from the Addison Wesley ROM Kernel Reference Manuals. Here however ia a brief C structure based description of the data fields present:

I have found the RigidDiskBlock description in the AMIGA ROM Kernel Reference Manual - DEVICES volume, Third Edition, Page 253:

"RigidDiskBlock- Fields and Implementation

The RigidDiskBlock (RDB) standard was borne out of the same development effort as HD_SCSICMD and as a result has a heavy bias towards SCSI. However, there is nothing in the RDB specification that makes it unusable for devices using other bus protocols. The XT style disks used in the A590 also support the RDB standard.

The RDB scheme was designed to allow the automatic mounting of all partitions on a hard drive and subsequent booting from the highest priority partition even if it has a soft loaded filing system. Disks can be removed from one controller and plugged into another (supporting the RDB scheme) and will carry with it all the necessary information for mounting and booting with them.

The preferred method of creating RigidDiskBlocks is with the HDToolBox program supplied by Commodore. Most controllers include an RDB editor or utility.

When a driver is initialized, it uses the information contained in the RDB to mount the required partitions and mark them as bootable if needed. The driver is also responsible for loading any filing systems that are required if they are not already available on the filesystem.resource list. Filesystems are added to the resource according to DosType and version number.

The following is a listing of devices/hardblocks.h that describes all the fields in the RDB specification:

/*--------------------------------------------------------------------
 *
 *	This file describes blocks of data that exist on a hard disk
 *	to describe that disk. They are not generically accessable to
 *	the user as they do not appear on any DOS drive. The blocks
 *	are tagged with a unique identifier, checksummed, and linked
 *	together. The root of these blocks is the RigidDiskBlock.
 *
 *	The RigidDiskBlock must exist on the disk within the first
 *	RDB LOCATION LIMIT blocks. This inhibits the use of the zero
 *	cylinder in an AmigaDOS partition: although it is strictly
 *	possible to store the RigidDiskBlock data in the reserved
 *	area of a partition, this practice is discouraged since the
 *	reserved blocks of a partition are overwritten by "Format",
 *	"Install", "DiskCopy", etc. The recommended disk layout,
 *	then, is to use the first cylinder(s) to store all the drive
 *	data specified by these blocks: i.e. partition descriptions,
 *	file system load images, drive bad block maps, spare blocks,
 *	etc.
 *
 *	Though only 512 byte blocks are currently supported by the
 *	file system, this proposal tries to be forward-looking by
 *	making the block size explicit, and by using only the first
 *	256 bytes for all blocks but the LoadSeg data.
 *
 *------------------------------------------------------------------*/

/*
 *  NOTE
 *	optional block addresses below contain $ffffffff to indicate
 *	a NULL address, as zero is a valid address
*/
struct RigidDiskBlock {
    ULONG   rdb_ID;		/* 4 character identifier */
    ULONG   rdb_SummedLongs;	/* size of this checksummed structure */
    LONG    rdb_ChkSum;		/* block checksum (longword sum to zero) */
    ULONG   rdb_HostID;		/* SCSI Target ID of host */
    ULONG   rdb_BlockBytes;	/* size of disk blocks */
    ULONG   rdb_Flags;		/* see below for defines */
    /* block list heads */
    ULONG   rdb_BadBlockList;	/* optional bad block list */
    ULONG   rdb_PartitionList;	/* optional first partition block */
    ULONG   rdb_FileSysHeaderList; /* optional file system header block */
    ULONG   rdb_Driveinit;	/* optional drive-specific init code */
				/* Driveinit(lun,rdb,ior): "C" stk & dO/aO/al */
    ULONG   rdb_Reservedl[6];	/* set to $ffffffff */
    /* physical-drive characteristics */
    ULONG   rdb_Cylinders;	/* number of drive cylinders */
    ULONG   rdb_Sectors;	/* sectors per track */
    ULONG   rdb_Heads;		/* number of drive heads */
    ULONG   rdb_Interleave;	/* interleave */
    ULONG   rdb_Park;		/* landing zone cylinder */
    ULONG   rdb_Reserved2[3];
    ULONG   rdb_WritePreComp;	/* starting cylinder: write precompensation */
    ULONG   rdb_ReducedWrite;	/* starting cylinder: reduced write current */
    ULONG   rdb_StepRate;	/* drive step rate */
    ULONG   rdb_Reserved3[5];
    /* logical drive characteristics */
    ULONG   rdb_RDBBlocksLo;	/* low block of range reserved for hardblocks */
    ULONG   rdb_RDBBlocksHi;	/* high block of range for these hardblocks */
    ULONG   rdb_LoCylinder;	/* low cylinder of partitionable disk area */
    ULONG   rdb_HiCylinder;	/* high cylinder of partitionable data area */
    ULONG   rdb_CylBlocks;	/* number of blocks available per cylinder */
    ULONG   rdb_AutoParkSeconds; /* zero for no auto park */
    ULONG   rdb_Reserved4[2];
/* drive identification */
    char    rdb_DiskVendor[8];
    char    rdb_DiskProduct[16];
    char    rdb_DiskRevision[4];
    char    rdb_ControllerVendor[8];
    char    rdb_ControllerProduct[16];
    char    rdb_ControllerRevision[4];
    ULONG   rdb_Reserved5[10];
};

#define IDNAME_RIGIDDISK	Ox5244534B	/* 'RDSK' */

#define RDB_LOCATION LIMIT	16

#define RDBFB_LAST	0	/* no disks exist to be configured after */
#define RDBFF_LAST	Ox01L   /*   this one on this controller */
#define RDBFB_LASTLUN	1	/* no LUNs exist to be configured greater */
#define RDBFF_LASTLUN	Ox02L	/*   than this one at this SCSI Target ID */
#define RDBFB_LASTTID	2	/* no Target IDs exist to be configured */
#define RDBFF_LASTTID	Ox04L	/*   greater than this one on this SCSI bus */
#define RDBFB_NORESELECT 3	/* don't bother trying to perform reselection */
#define RDBFF_NORESELECT Ox08L	/*   when talking to this drive */
#define RDBFB_DISKID	4	/* rdb Disk ... identification valid*/
#define RDBFF_DISKID	Ox10L
#define RDBFB_CTRLRID	5	/* rdb Controller ... identification valid */
#define RDBFF_CTRLRID	Ox20L

/*------------------------------------------------------------------*/
struct BadBlockEntry {
    ULONG   bbe_BadBlock;	/* block number of bad block */
    ULONG   bbe_GoodBlock;	/* block number of replacement block */
);

struct BadBlockBlock {
    ULONG   bbb_ID;		/* 4 character identifier */
    ULONG   bbb_SummedLongs;	/* size of this checksummed structure */
    LONG    bbb_ChkSum;		/* block checksum (longword sum to zero) */
    ULONG   bbb_HostID;		/* SCSI Target ID of host */
    ULONG   bbb_Next;		/* block number of the next BadBlockBlock */
    ULONG   bbb_Reserved;
    struct BadBlockEntry bbb_BlockPairs[61]; /* bad block entry pairs */
    /* note [61] assumes 512-byte blocks */
};

#define IDNAME_BADBLOCK 	Ox42414442	/* 'BADB' */

/*------------------------------------------------------------------*/
struct PartitionBlock {
    ULONG   pb_ID;		/* 4 character identifier */
    ULONG   pb_SummedLongs;	/* size of this checksummed structure */
    LONG    pb_ChkSum;		/* block checksum (longword sum to zero) */
    ULONG   pb_HostID;		/* SCSI Target ID of host */
    ULONG   pb_Next;		/* block number of the next PartitionBlock */
    ULONG   pb_Flags;		/* see below for defines */
    ULONG   pb_Reserved1[2];
    ULONG   pb_DevFlags;	/* preferred flags for OpenDevice */
    UBYTE   pb_DriveName[32];	/* preferred DOS device name: BSTR form */
				/* (not used if this name is in use) */
    ULONG   pb_Reserved2[15];	/* filler to 32 longwords */
    ULONG   pb_Environment[17]; /* environment vector for this partition */
    ULONG   pb_EReserved[15];	/* reserved for future environment vector */
};

#define IDNAME_PARTITION	Ox50415254	/* 'PART' */

#define PBFB_BOOTABLE	0	/* this partition is intended to be boatable */
#define PBFF_BOOTABLE	1L	/* (expected directories and files exist) */
#define PBFB_NOMOUNT	1	/* do not mount this partition (e.g. manually */
#define PBFF_NOMOUNT	2L	/* mounted, but space reserved here) */

/*------------------------------------------------------------------*/
struct FileSysHeaderBlock {
    ULONG   fhb_ID;		/* 4 character identifier */
    ULONG   fhb_SummedLongs;	/* size of this checksummed structure */
    LONG    fhb_ChkSum;		/* block checksum (longword sum to zero) */
    ULONG   fhb_HostID;		/* SCSI Target ID of host */
    ULONG   fhb_Next;		/* block number of next FileSysHeaderBlock */
    ULONG   fhb_Flags;		/* see below for defines */
    ULONG   fhb_Reserved1[2];
    ULONG   fhb_DosType;	/* file system description: match this with */
				/* partition environment's DE DOSTYPE entry */
    ULONG   fhb_Version;	/* release version of this code */
    ULONG   fhb_PatchFlags;	/* bits set for those of the following that */
				/* need to be substituted into a standard */
				/* device node for this file system: e.g. */
				/* Ox180 to substitute SegList & GlobalVec */
    ULONG   fhb_Type;		/* device node type: zero */
    ULONG   fhb_Task;		/* standard dos "task" field: zero */
    ULONG   fhb_Lock;		/* not used for devices: zero */
    ULONG   fhb_Handler;	/* filename to loadseg: zero placeholder */
    ULONG   fhb_StackSize;	/* stacksize to use when starting task */
    LONG    fhb_Priority;	/* task priority when starting task */
    LONG    fhb_Startup;	/* startup msg: zero placeholder */
    LONG    fhb_SegListBlocks;	/* first of linked list of LoadSegBlocks: */
				/* note that this entry requires some */
				/* processing before substitution */
    LONG    fhb_Global Vec;	/* BCPL global vector when starting task */
    ULONG   fhb_Reserved2[23];	/* (those reserved by PatchFlags) */
    ULONG   fhb_Reserved3[21];
};

#define IDNAME_FILESYSHEADER	Ox46534844	/* 'FSHD' */

/*------------------------------------------------------------------*/
struct LoadSegBlock {
    ULONG   lsb_ID;		/* 4 character identifier */
    ULONG   lsb_SummedLongs;	/* size of this checksummed structure */
    LONG    lsb_ChkSum;		/* block checksum (longword sum to zero) */
    ULONG   lsb_HostID;		/* SCSI Target ID of host */
    ULONG   lsb_Next;		/* block number of the next LoadSegBlock */
    ULONG   lsb_LoadData[123];	/* data for "loadseg" */
    /* note [123] assumes 512 byte blocks */
};

#define IDNAME_LOADSEG		Ox4C534547	/* 'LSEG' */

[prowler]

Originally Posted by thomas

You can flag any block as bad, but you cannot have true bad blocks in a HDF.

I agree, but I think the the OP is asking for an RDF.

From what I have seen in the RigidDiskBlock explanation, I imagine that if anyone had an RDF image created by HDTools from a hard drive with some BADB-Blocks, then these blocks would have bad block entries in the RDB and been substituted with replacement blocks from the reserved pool of spare disk blocks.

Therefore, at the filesystem level, there would indeed be no bad blocks and in that case such a hard file would be of no use as a testcase, unless the OP's proposed Disk Monitor/Editor is also intended for use at the low level.

Really, I wish to know whether, in this instance, I'm wasting my time trying to find a way of artificially creating bad sectors on an RDB image where there are none, as far as I yet know.

[mark_k]

Originally Posted by prowler

Note that the listing above may not be 100% accurate. Some of the non-alphanumeric characters were corrupted during the transfer, though I've corrected them where possible.

Online version of that (hardblocks.h include file) is here.

I'm not sure exactly which device drivers (scsi.device etc.) support bad block lists. I know at least one does because I remember looking at its code. The driver has to handle remapping accesses to blocks which are marked as bad. Many/most probably don't because it would significantly complicate the code, and most(?) SCSI drives can be told to remap bad sectors so from then on access is transparent to the host computer.

[prowler]

Originally Posted by mark_k

Online version of that (hardblocks.h include file) is here.

Thanks for the link!

I have corrected a typo in my listing. It appears to be an earlier version than the one given in your link. However I have left mine as is to match the one in the manual.

Originally Posted by mark_k

I'm not sure exactly which device drivers (scsi.device etc.) support bad block lists. I know at least one does because I remember looking at its code. The driver has to handle remapping accesses to blocks which are marked as bad. Many/most probably don't because it would significantly complicate the code, and most(?) SCSI drives can be told to remap bad sectors so from then on access is transparent to the host computer.

scsi.device certainly supports bad blocks. The RigidDiskBlock explanation quoted above appears in the scsi.device section (but no other) in the DEVICES volume of the Amiga ROM Kernel Reference Manual.

[thomas]

Originally Posted by prowler

I agree, but I think the the OP is asking for an RDF.

An RDF is a HDF with an RDB. You cannot have real bad blocks in any type of image format. These images only store user data, 512 bytes per block. There is no way to make such a block bad (unreadable). A real harddrive stores error detection data on the magnetic disk so that it can determine whether the user data is still intact or not. The data in an image file is always intact, there is no way to store a bad block in an image file.

[prowler]

Originally Posted by thomas

An RDF is a HDF with an RDB. You cannot have real bad blocks in any type of image format. These images only store user data, 512 bytes per block. There is no way to make such a block bad (unreadable). A real harddrive stores error detection data on the magnetic disk so that it can determine whether the user data is still intact or not. The data in an image file is always intact, there is no way to store a bad block in an image file.

Then the RDB part of an RDF cannot strictly conform to the RigidDiskBlock specification given in the Amiga ROM Kernel Reference Manual, and my thoughts above about how the nature of the bad block mapping and replacement would render an RDF quite useless as a testcase for bad block handling of a proposed Disk Monitor application are therefore invalid.

Nevertheless, what you have written above confirms that an RDF won't, in any event, provide the testcase that the OP is seeking.

[Oxygene]

This may be stupid idea, but since WinUAE can use real hard drives, why not just use one that is known to have bad blocks?That way you don't have to bother with image files, and can have a real test case.

[prowler]

Originally Posted by Oxygene

This may be stupid idea, but since WinUAE can use real hard drives, why not just use one that is known to have bad blocks?That way you don't have to bother with image files, and can have a real test case.

That's not a stupid idea at all. At this stage it appears to be the only option left.

[thomas]

Originally Posted by prowler

Then the RDB part of an RDF cannot strictly conform to the RigidDiskBlock specification given in the Amiga ROM Kernel Reference Manual,

Why not? The RDB is stored in disk blocks, 512 bytes per block, just like anything else on the disk.

and my thoughts above about how the nature of the bad block mapping and replacement would render an RDF quite useless as a testcase for bad block handling of a proposed Disk Monitor application are therefore invalid.

A BADB list on the RDF would let the driver avoid some blocks and replace it by other blocks. That's what the BADB list is for. The avoided blocks would not be bad, though. You could test BADB handling with it, but if it does not work, you would not get read errors like on a real disk but just read the "bad" blocks which aren't bad. You would have to store some data in the blocks which lets you identify the original blocks against the replacement blocks.

[prowler]

Originally Posted by thomas

Why not? The RDB is stored in disk blocks, 512 bytes per block, just like anything else on the disk.

Well, given what you went on to say below, indeed there's no reason at all why not.

Originally Posted by thomas

A BADB list on the RDF would let the driver avoid some blocks and replace it by other blocks. That's what the BADB list is for. The avoided blocks would not be bad, though. You could test BADB handling with it, but if it does not work, you would not get read errors like on a real disk but just read the "bad" blocks which aren't bad. You would have to store some data in the blocks which lets you identify the original blocks against the replacement blocks.

If you had offered this information at the start instead of just "A hardfile cannot have read errors.", all this detective work could have been avoided. Then again, we wouldn't have such a popular forum here if all our questions were answered with a single post.

[Toni Wilen]

HDF is exactly same as normal ADF. It can only contain "user" data, neither can have read errors.

UAE has external "geometry" file support (*.geom) that could also include list (or ranges) of blocks that have virtual read error.

I'll think about it..

[sigma63]

OK, It's time for me to come in again

First i would like to thank you all for your input and apologize for all who "wasted time" or got confused.

To clarify some Things: Surely the Idea from Oxygene to use a real drive is great, but unfortunately i (think i) have no such drive, which is known to be weak or have Errors.

So there was the idea of having a hardfile. I didn't know the term rdf to be a hdf with embedded RDB, so of cource i'm talking about an rdf!

And yes i understand that for now a hardfile could not have "errors".
What i want is a rdf, were the RDB contains some linked BADB-Block, regardless if they refer (in the rdf) to readable blocks or not.

If anybody knows for sure he has a real drive with such BADB-Blocks, it would be nice to know how they were created (by HDToolbox or other?) and is willing to share a dump of the RDBs only (no need to get the actual data-blocks)!

Otherwise i think i create simply a BADB-block by hand and see what will happen.

@Toni: i would like to see such "external geometry file support" for hardfiles also, but that has no high priority

[sigma63]

Originally Posted by thomas

A BADB list on the RDF would let the driver avoid some blocks and replace it by other blocks. That's what the BADB list is for. The avoided blocks would not be bad, though. You could test BADB handling with it, but if it does not work, you would not get read errors like on a real disk but just read the "bad" blocks which aren't bad. You would have to store some data in the blocks which lets you identify the original blocks against the replacement blocks.

Thank you Thomas, this is exactly what i meant and what i want to test/play with