HstWB Imager - New disk imaging tool

[cyberhead97]

I'm making very good progress on the project. I have integration tests passing that creates a RDB from scratch with PFS3 filesystem, 2 partitions, writes it to a .hdf file and reads it's RDB again with a compare that is equal 1:1.

As this is a .Net Core project, I can now write fluent write C# as following:

await RigidDiskBlock
    .Create(4.GB().ToUniversalSize())
    .AddFileSystem("PDS3", await File.ReadAllBytesAsync("pfs3aio"))
    .AddPartition("DH0", 300.MB(), bootable: true)
    .AddPartition("DH1")
    .WriteToFile("amiga.hdf");

This creates a new RDB of size 4GB. Universal size means it trimmes down size by 0.05% to make it fit on various SF/CF-cards, SSD. Then it adds PFS3 filesystem, it also parses version string from "pfs3aio" file, so it automatically sets file system version to 19.2. Then it adds a 300MB bootable partition with device name DH0 and a DH1 partition without size defined, so it will fit remaining space left approx 3.7GB. As last step it writes RDB to file "amiga.hdf".

To get development going and latest changes testable, I will focus on making an automated release process with GitHub workflow, so I can publish prerelease versions easily and frequently.

This will help feedback on ongoing development.

[jbenam]

Originally Posted by cyberhead97

Even idea's of importing and exporting partitions to and from other images is possible .

Great stuff, finally I won't need to re-create my WHDLoad partitions

[RoC]

Cool, many thanks.

Ready to test and use it ;-)

[enigma776]

This is going to be amazing when coupled up with HstWB, would make updating real easy if you can just do partitions.

[cyberhead97]

Time for an update on the project with good news.

HstWB Imager is now a functional GUI and working well and able to do the following:
- Read physical disk to image.
- Write physical disk to image.
- Convert image from .hdf to .vhd and vice versa.
- Verify image file or physical disk against image file.
- Display information about physical disk to image with Rigid Disk Block overview and details.

So the most advanced parts are working and tested, primarily Windows build. macOS and Linux builds will be tested as well.

Creating a new blank image and optimize an image is implemented in code, but are missing GUI.

Automated build and release pipeline is almost complete as well, so I can soon publish a preview/prerelease and keep publishing updates/tweaks with short intervals.

The partition editing has not started yet, but I'm considering to use a split pane like this https://codesandbox.io/s/wr0gf to make it possible to drag partitions in size.

I have following ideas for partition editing functionality ordered by priority:

1. Initialize disk with new Rigid Disk Block (RDB).
2. Create, edit, delete partitions.
3. Create, edit, delete file systems.
4. Ability to create Hybrid disks with both RDB and MBR, so Amiga has access to both Amiga file systems and a FAT32 partition for file transfer.
5. Format FFS and PFS3 without using an Amiga emulator.
6. Import and export partitions as single .hdf files.

Formatting FFS seems simple enough, but PFS3 is more challeging. Luckily source code for PFS3 formatting is available at aminet and I will re-implement this in HstWB Imager to support PFS3 formatting.

I have actually also done some tiny research into PFS3 file system and can read/parse it's root block to get volume name. I would really like to re-implement PFS3 file system as well, so I could read and maybe also write files to and from PFS3 partitions. For me this would be a killer feature and an idea to extend HstWB Installer with a commander or Directory Opus like view to copy files to and from any image file (.hdf, .vhd, etc.), physical drive and .adf floppy images. Understanding PFS3 file system is quite a task and therefore this idea has currently low priority and will be prioritized higher at a later time. Maybe some of it becomes more clear when implementing PFS3 formatting.

Here's some screenshots of latest unreleased HstWB Imager being used .

[cyberhead97]

Finally I have a working beta version of HstWB Imager ready for testing and can be downloaded at https://github.com/henrikstengaard/h...aller/releases.

Getting elevated rights in Windows and Mac OS required lot of refactoring to get raw disk access working properly.

HstWB Imager is released as a Windows, MacOS and Linux 64-bit app. It's beta, but very functional. There is a bug in the 3rd party plugin used to read and write .vhd files,
that results in MacOS and Linux not being able to read .vhd files. Since it's open source, I have going to make a fix for it in a few days.

Downloading Windows release will cause Chrome (and maybe other browsers) to complain about the file as it not signed. You have to select keep or continue download. Also first HstWB Imager is started you have to click "More Info" and click "Run anyway".

The same goes for MacOS when trying to launch the app, MacOS security will also complain about the app has downloaded from web and not signed/trusted.
You need to open security & privacy in settings and select open anyway. On M1 macs you need to install Rosetta, which shows automatically during installing.

The Linux version has not been tested much and might not work with reading and writing image files.

Next I'm going to work on the partition functionality. I have code to handle it, just need to make GUI for it.
I'm also going to examine PFS3 source code and figure out how to implement format PFS3 partitions. Same goes for FFS.

Try it out and let me know how it works, if it works

[TommyTomorrow]

Well.. the link you've pasted is for HstWB Installer, not Imager

[cyberhead97]

Originally Posted by TommyTomorrow

Well.. the link you've pasted is for HstWB Installer, not Imager

I had created some incorrect tags that caused some issues with the prerelease.

They are fixed now and HstWB Imager is listed same location as HstWB Installer and is available at top of releases https://github.com/henrikstengaard/h...aller/releases.

[cyberhead97]

Since the release of the first beta, I have been examining FastFileSystem structure and after a few days work and thanks to ADF format FAQ at http://lclevy.free.fr/adflib/adf_info.html#p65 I have implemented formatting of partitions with DOS\3 / FastFileSystem.

This will be part of the partitioning, but I can't include Fast File System handler as it's part of Amiga OS and commercially sold. So I need to add an import function, that can import FastFileSystem from an existing .hdf/.img or Amiga OS .adf install disk.

With C# I can now format FastFileSystem with following code:

var rigidDiskBlock = await RigidDiskBlock
    .Create(300.MB())
    .AddFileSystem("DOS3", await File.ReadAllBytesAsync("FastFileSystem"))
    .AddPartition("DH0", bootable: true)
    .WriteToFile("hstwb.hdf");

await using var stream = File.OpenWrite("hstwb.hdf");
await FastFileSystemHelper.FormatPartition(stream, rigidDiskBlock.PartitionBlocks.First(), "WorkbenchHstWB");

Here's a screenshot of WinUAE showing the .hdf file the code created and it worked right away copying files to and from it.

I'm really looking forward to making the partitioning with formatting

[RoC]

Great. Good to see it is moving well.

I tried the x64 portable version, which works fine. Like for many other good Amiga tools, Defender is keep on bothering but it is enough to accept the risk to move on.

Tried to open the images and it returns the infos. Created a blank HDF image.

One (maybe stupid) question: What the convert to/from IMG/HDF image does? I mean, I assume there are differences in the two formats, but there what is the advantage of one format over the other?


I am one of those, looking forward to the formatting part too. I know it is not easy and maybe not feasible, but PFS3 support would be juicy :-)

Many thanks for your work

[cyberhead97]

Originally Posted by RoC

Great. Good to see it is moving well.

I tried the x64 portable version, which works fine. Like for many other good Amiga tools, Defender is keep on bothering but it is enough to accept the risk to move on.

Tried to open the images and it returns the infos. Created a blank HDF image.

One (maybe stupid) question: What the convert to/from IMG/HDF image does? I mean, I assume there are differences in the two formats, but there what is the advantage of one format over the other?


I am one of those, looking forward to the formatting part too. I know it is not easy and maybe not feasible, but PFS3 support would be juicy :-)

Many thanks for your work

The convert option will convert from one format to another and is intended for converting between img/hdf and vhd and vice versa. Maybe this is not clear enough and I need more description text to help.

Try selecting a .hdf as source file and .vhd as destination file and convert it, e.g. try doing this with your blank image.

Create blank image can also create .vhd directly.

The important note here is that destination file ends with .vhd, which will trigger creating the virtual hard disks.

If you try converting a blank 16gb .hdf to .vhd, this should result in a .vhd file of approx 50 - 100 KB.

PFS3 formatting will be an option and I'm investigating PFS3 format source code from aminet.

[cyberhead97]

I started porting PFS3AIO from Tony Wilen's github repository from C to C# in order to format PFS3 partitions. The code will be a 1:1 more or less except for IO operations reading, writing and converting between structs and binary data.

Compared to FFS, PFS3 is a lot more complex.

It's going well and I'm nearing a point where all code covering formatting is compiling. Since formatting involves so much of PFS3 code in total, I'm not far from being able to read and write files directly to and from .hdf/.img or disk attached via USB.

If I can get to such a state with PFS3 filesystem read and write, I'm considering to make a simple Norton Commander or Directory Opus style app (cross platform of course) to simplify file transfer without having to use an emulator for such tasks.

But first focus is on making partition UI and partitioning work. Then formatting.

[cyberhead97]

All PFS3 code for formatting a disk is fully implemented in C# and I can format a partition. The disk is recognised by Amiga OS, so this is a big step towards formatting a partition without using an Amiga emulator.

But when trying to write to it I get "cache inconsistency", so I need to examine on what's causing it.

I have attached two few screenshots to see PFS3 partition shown in Workbench and the error I get when trying to write to it.

[cyberhead97]

It works now... after the last couple of checks between sectors of a partition formatted with PFS3AIO on an Amiga (WinUAE) and a partition formatted with HstWB Imager.

I'm very happy with the result and did also run pfsdoctor to check the partition to confirm it works and it reports 0 errors.

Now to the partition editor with FFS and PFS3 formatting.

Here's a couple of screenshots showing pfsdoctor results and Workbench 3.1 floppy being copied to the partition to verify it works .

[RoC]

Congrats! You made it.

Awesome work, many many thanks

[keito]

This is a really useful application, thanks for making and sharing this.

[cyberhead97]

Originally Posted by keito

This is a really useful application, thanks for making and sharing this.

Nice, what part of the application are you using?

[kriz]

AHh looks very nice, got it download and tested running fine.. Will use next time i need to prepare something for an Amiga!!

[cyberhead97]

Two weeks ago I saw a post on Facebook with a user having issues mounting 20 years old .HDF backups of his partitions. I decided to help the guy as I was already way into Fast File System structure with recent success of formatting FFS partitions.

So I began porting ADFLib to C# and after a week of coding, I was able to read and extract files from the .HDF files. But it was not that easy. There was a reason why this guy was struggling. With all the attempts i tried to mount the .HDF files changing different harddrive geometry settings, all I got was "not valid dos disk". It basically means the root block is not located where is it supposed to be. Then i coded a scanner to examine every block and check if it could be a valid root block and this worked. A valid root block was found, but approx. 13000 bytes off from where it was supposed to be.

Then I added a salvage like function to try and read root block from expected location, otherwise scan and use first valid root block. Next it recursively extracts any files in same directory structure as in the .HDF. And it worked well, I even got it to boot into workbench and send it back to the guy and he was very happy.

This is great as I can now read and write .ADF and .HDF. It makes me want to create that DirOpus or commander like app to enable transfer files on any OS. I really miss such tool

[enigma776]

Any more progress on this?