English Amiga Board


Go Back   English Amiga Board > Support > support.Other

 
 
Thread Tools
Old 09 February 2022, 15:10   #21
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
I'm making very good progress on the project. I have integration tests passing that creates a RDB from scratch with PFS3 filesystem, 2 partitions, writes it to a .hdf file and reads it's RDB again with a compare that is equal 1:1.

As this is a .Net Core project, I can now write fluent write C# as following:
Code:
await RigidDiskBlock
    .Create(4.GB().ToUniversalSize())
    .AddFileSystem("PDS3", await File.ReadAllBytesAsync("pfs3aio"))
    .AddPartition("DH0", 300.MB(), bootable: true)
    .AddPartition("DH1")
    .WriteToFile("amiga.hdf");
This creates a new RDB of size 4GB. Universal size means it trimmes down size by 0.05% to make it fit on various SF/CF-cards, SSD. Then it adds PFS3 filesystem, it also parses version string from "pfs3aio" file, so it automatically sets file system version to 19.2. Then it adds a 300MB bootable partition with device name DH0 and a DH1 partition without size defined, so it will fit remaining space left approx 3.7GB. As last step it writes RDB to file "amiga.hdf".

To get development going and latest changes testable, I will focus on making an automated release process with GitHub workflow, so I can publish prerelease versions easily and frequently.

This will help feedback on ongoing development.

Last edited by cyberhead97; 09 February 2022 at 15:11. Reason: spelling
cyberhead97 is offline  
Old 09 February 2022, 15:16   #22
jbenam
Italian Amiga Zealot

 
Join Date: Jan 2009
Location: Italy
Age: 34
Posts: 1,671
Quote:
Originally Posted by cyberhead97 View Post
Even idea's of importing and exporting partitions to and from other images is possible .
Great stuff, finally I won't need to re-create my WHDLoad partitions
jbenam is offline  
Old 10 February 2022, 10:17   #23
RoC
Registered User

 
Join Date: May 2011
Location: Italy
Posts: 187
Cool, many thanks.

Ready to test and use it ;-)
RoC is offline  
Old 10 February 2022, 19:00   #24
enigma776
Bringer of Death

enigma776's Avatar
 
Join Date: Apr 2017
Location: Peterborough, UK
Age: 38
Posts: 364
This is going to be amazing when coupled up with HstWB, would make updating real easy if you can just do partitions.
enigma776 is offline  
Old 24 February 2022, 22:53   #25
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Time for an update on the project with good news.

HstWB Imager is now a functional GUI and working well and able to do the following:
- Read physical disk to image.
- Write physical disk to image.
- Convert image from .hdf to .vhd and vice versa.
- Verify image file or physical disk against image file.
- Display information about physical disk to image with Rigid Disk Block overview and details.

So the most advanced parts are working and tested, primarily Windows build. macOS and Linux builds will be tested as well.

Creating a new blank image and optimize an image is implemented in code, but are missing GUI.

Automated build and release pipeline is almost complete as well, so I can soon publish a preview/prerelease and keep publishing updates/tweaks with short intervals.

The partition editing has not started yet, but I'm considering to use a split pane like this https://codesandbox.io/s/wr0gf to make it possible to drag partitions in size.

I have following ideas for partition editing functionality ordered by priority:
  1. Initialize disk with new Rigid Disk Block (RDB).
  2. Create, edit, delete partitions.
  3. Create, edit, delete file systems.
  4. Ability to create Hybrid disks with both RDB and MBR, so Amiga has access to both Amiga file systems and a FAT32 partition for file transfer.
  5. Format FFS and PFS3 without using an Amiga emulator.
  6. Import and export partitions as single .hdf files.

Formatting FFS seems simple enough, but PFS3 is more challeging. Luckily source code for PFS3 formatting is available at aminet and I will re-implement this in HstWB Imager to support PFS3 formatting.

I have actually also done some tiny research into PFS3 file system and can read/parse it's root block to get volume name. I would really like to re-implement PFS3 file system as well, so I could read and maybe also write files to and from PFS3 partitions. For me this would be a killer feature and an idea to extend HstWB Installer with a commander or Directory Opus like view to copy files to and from any image file (.hdf, .vhd, etc.), physical drive and .adf floppy images. Understanding PFS3 file system is quite a task and therefore this idea has currently low priority and will be prioritized higher at a later time. Maybe some of it becomes more clear when implementing PFS3 formatting.

Here's some screenshots of latest unreleased HstWB Imager being used .
Attached Thumbnails
Click image for larger version

Name:	hstwb_imager_2022-02-24_start.PNG
Views:	71
Size:	39.5 KB
ID:	74811   Click image for larger version

Name:	hstwb_imager_2022-02-24_convert1.PNG
Views:	66
Size:	43.0 KB
ID:	74812   Click image for larger version

Name:	hstwb_imager_2022-02-24_convert2.PNG
Views:	63
Size:	39.1 KB
ID:	74813   Click image for larger version

Name:	hstwb_imager_2022-02-24_info1.PNG
Views:	65
Size:	49.5 KB
ID:	74814   Click image for larger version

Name:	hstwb_imager_2022-02-24_info2.PNG
Views:	56
Size:	48.4 KB
ID:	74815  

Click image for larger version

Name:	hstwb_imager_2022-02-24_info3.PNG
Views:	61
Size:	37.1 KB
ID:	74816   Click image for larger version

Name:	hstwb_imager_2022-02-24_read.PNG
Views:	54
Size:	41.0 KB
ID:	74817   Click image for larger version

Name:	hstwb_imager_2022-02-24_write.PNG
Views:	56
Size:	41.3 KB
ID:	74818  

Last edited by cyberhead97; 24 February 2022 at 23:52.
cyberhead97 is offline  
Old 12 March 2022, 22:49   #26
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Finally I have a working beta version of HstWB Imager ready for testing and can be downloaded at https://github.com/henrikstengaard/h...aller/releases.

Getting elevated rights in Windows and Mac OS required lot of refactoring to get raw disk access working properly.

HstWB Imager is released as a Windows, MacOS and Linux 64-bit app. It's beta, but very functional. There is a bug in the 3rd party plugin used to read and write .vhd files,
that results in MacOS and Linux not being able to read .vhd files. Since it's open source, I have going to make a fix for it in a few days.

Downloading Windows release will cause Chrome (and maybe other browsers) to complain about the file as it not signed. You have to select keep or continue download. Also first HstWB Imager is started you have to click "More Info" and click "Run anyway".

The same goes for MacOS when trying to launch the app, MacOS security will also complain about the app has downloaded from web and not signed/trusted.
You need to open security & privacy in settings and select open anyway. On M1 macs you need to install Rosetta, which shows automatically during installing.

The Linux version has not been tested much and might not work with reading and writing image files.

Next I'm going to work on the partition functionality. I have code to handle it, just need to make GUI for it.
I'm also going to examine PFS3 source code and figure out how to implement format PFS3 partitions. Same goes for FFS.

Try it out and let me know how it works, if it works
Attached Thumbnails
Click image for larger version

Name:	imager_1.PNG
Views:	51
Size:	62.6 KB
ID:	75068   Click image for larger version

Name:	imager_2.PNG
Views:	48
Size:	61.4 KB
ID:	75069   Click image for larger version

Name:	imager_3.PNG
Views:	46
Size:	55.8 KB
ID:	75070   Click image for larger version

Name:	imager_4.PNG
Views:	44
Size:	45.4 KB
ID:	75071   Click image for larger version

Name:	imager_5.PNG
Views:	41
Size:	42.9 KB
ID:	75072  

Click image for larger version

Name:	imager_6.PNG
Views:	40
Size:	39.7 KB
ID:	75073  

Last edited by cyberhead97; 12 March 2022 at 23:29.
cyberhead97 is offline  
Old 17 March 2022, 21:11   #27
TommyTomorrow
Registered User

 
Join Date: Jul 2020
Location: PL
Posts: 22
Well.. the link you've pasted is for HstWB Installer, not Imager
TommyTomorrow is offline  
Old 22 March 2022, 13:41   #28
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Quote:
Originally Posted by TommyTomorrow View Post
Well.. the link you've pasted is for HstWB Installer, not Imager
I had created some incorrect tags that caused some issues with the prerelease.

They are fixed now and HstWB Imager is listed same location as HstWB Installer and is available at top of releases https://github.com/henrikstengaard/h...aller/releases.
cyberhead97 is offline  
Old 24 March 2022, 20:57   #29
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Since the release of the first beta, I have been examining FastFileSystem structure and after a few days work and thanks to ADF format FAQ at http://lclevy.free.fr/adflib/adf_info.html#p65 I have implemented formatting of partitions with DOS\3 / FastFileSystem.

This will be part of the partitioning, but I can't include Fast File System handler as it's part of Amiga OS and commercially sold. So I need to add an import function, that can import FastFileSystem from an existing .hdf/.img or Amiga OS .adf install disk.

With C# I can now format FastFileSystem with following code:
Code:
var rigidDiskBlock = await RigidDiskBlock
    .Create(300.MB())
    .AddFileSystem("DOS3", await File.ReadAllBytesAsync("FastFileSystem"))
    .AddPartition("DH0", bootable: true)
    .WriteToFile("hstwb.hdf");

await using var stream = File.OpenWrite("hstwb.hdf");
await FastFileSystemHelper.FormatPartition(stream, rigidDiskBlock.PartitionBlocks.First(), "WorkbenchHstWB");
Here's a screenshot of WinUAE showing the .hdf file the code created and it worked right away copying files to and from it.

I'm really looking forward to making the partitioning with formatting
Attached Thumbnails
Click image for larger version

Name:	hstwb-imager_ffs_format_working.png
Views:	50
Size:	8.6 KB
ID:	75135  
cyberhead97 is offline  
Old 27 March 2022, 11:07   #30
RoC
Registered User

 
Join Date: May 2011
Location: Italy
Posts: 187
Great. Good to see it is moving well.

I tried the x64 portable version, which works fine. Like for many other good Amiga tools, Defender is keep on bothering but it is enough to accept the risk to move on.

Tried to open the images and it returns the infos. Created a blank HDF image.

One (maybe stupid) question: What the convert to/from IMG/HDF image does? I mean, I assume there are differences in the two formats, but there what is the advantage of one format over the other?


I am one of those, looking forward to the formatting part too. I know it is not easy and maybe not feasible, but PFS3 support would be juicy :-)

Many thanks for your work
RoC is offline  
Old 28 March 2022, 13:16   #31
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Quote:
Originally Posted by RoC View Post
Great. Good to see it is moving well.

I tried the x64 portable version, which works fine. Like for many other good Amiga tools, Defender is keep on bothering but it is enough to accept the risk to move on.

Tried to open the images and it returns the infos. Created a blank HDF image.

One (maybe stupid) question: What the convert to/from IMG/HDF image does? I mean, I assume there are differences in the two formats, but there what is the advantage of one format over the other?


I am one of those, looking forward to the formatting part too. I know it is not easy and maybe not feasible, but PFS3 support would be juicy :-)

Many thanks for your work
The convert option will convert from one format to another and is intended for converting between img/hdf and vhd and vice versa. Maybe this is not clear enough and I need more description text to help.

Try selecting a .hdf as source file and .vhd as destination file and convert it, e.g. try doing this with your blank image.

Create blank image can also create .vhd directly.

The important note here is that destination file ends with .vhd, which will trigger creating the virtual hard disks.

If you try converting a blank 16gb .hdf to .vhd, this should result in a .vhd file of approx 50 - 100 KB.

PFS3 formatting will be an option and I'm investigating PFS3 format source code from aminet.
cyberhead97 is offline  
Old 07 April 2022, 22:04   #32
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
I started porting PFS3AIO from Tony Wilen's github repository from C to C# in order to format PFS3 partitions. The code will be a 1:1 more or less except for IO operations reading, writing and converting between structs and binary data.

Compared to FFS, PFS3 is a lot more complex.

It's going well and I'm nearing a point where all code covering formatting is compiling. Since formatting involves so much of PFS3 code in total, I'm not far from being able to read and write files directly to and from .hdf/.img or disk attached via USB.

If I can get to such a state with PFS3 filesystem read and write, I'm considering to make a simple Norton Commander or Directory Opus style app (cross platform of course) to simplify file transfer without having to use an emulator for such tasks.

But first focus is on making partition UI and partitioning work. Then formatting.
cyberhead97 is offline  
Old 19 April 2022, 01:20   #33
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
All PFS3 code for formatting a disk is fully implemented in C# and I can format a partition. The disk is recognised by Amiga OS, so this is a big step towards formatting a partition without using an Amiga emulator.

But when trying to write to it I get "cache inconsistency", so I need to examine on what's causing it.

I have attached two few screenshots to see PFS3 partition shown in Workbench and the error I get when trying to write to it.
Attached Thumbnails
Click image for larger version

Name:	pfs3_format_almost_working1.png
Views:	29
Size:	14.8 KB
ID:	75343   Click image for larger version

Name:	pfs3_format_almost_working2.png
Views:	30
Size:	15.2 KB
ID:	75344  
cyberhead97 is offline  
Old 19 April 2022, 22:57   #34
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
It works now... after the last couple of checks between sectors of a partition formatted with PFS3AIO on an Amiga (WinUAE) and a partition formatted with HstWB Imager.

I'm very happy with the result and did also run pfsdoctor to check the partition to confirm it works and it reports 0 errors.

Now to the partition editor with FFS and PFS3 formatting.

Here's a couple of screenshots showing pfsdoctor results and Workbench 3.1 floppy being copied to the partition to verify it works .
Attached Thumbnails
Click image for larger version

Name:	pfs3_hstwb_format_pfsdoctor_success.png
Views:	37
Size:	37.7 KB
ID:	75349   Click image for larger version

Name:	pfs3_hstwb_format_workbench.png
Views:	35
Size:	19.9 KB
ID:	75350   Click image for larger version

Name:	pfs3_hstwb_format_workbench_copied_files.png
Views:	32
Size:	21.8 KB
ID:	75351   Click image for larger version

Name:	pfs3_hstwb_format_pfsdoctor_experimental_success.png
Views:	31
Size:	36.5 KB
ID:	75352  
cyberhead97 is offline  
Old 19 April 2022, 23:06   #35
RoC
Registered User

 
Join Date: May 2011
Location: Italy
Posts: 187
Congrats! You made it.

Awesome work, many many thanks
RoC is offline  
Old 26 April 2022, 10:05   #36
keito
Registered User

 
Join Date: Dec 2013
Location: UK
Posts: 75
This is a really useful application, thanks for making and sharing this.
keito is offline  
Old 26 April 2022, 21:52   #37
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Quote:
Originally Posted by keito View Post
This is a really useful application, thanks for making and sharing this.
Nice, what part of the application are you using?
cyberhead97 is offline  
Old 26 April 2022, 22:25   #38
kriz
Junior Member

kriz's Avatar
 
Join Date: Sep 2001
Location: No(R)Way
Age: 40
Posts: 2,998
AHh looks very nice, got it download and tested running fine.. Will use next time i need to prepare something for an Amiga!!
kriz is offline  
Old 13 May 2022, 21:30   #39
cyberhead97
Registered User

 
Join Date: Feb 2016
Location: Denmark
Posts: 324
Two weeks ago I saw a post on Facebook with a user having issues mounting 20 years old .HDF backups of his partitions. I decided to help the guy as I was already way into Fast File System structure with recent success of formatting FFS partitions.

So I began porting ADFLib to C# and after a week of coding, I was able to read and extract files from the .HDF files. But it was not that easy. There was a reason why this guy was struggling. With all the attempts i tried to mount the .HDF files changing different harddrive geometry settings, all I got was "not valid dos disk". It basically means the root block is not located where is it supposed to be. Then i coded a scanner to examine every block and check if it could be a valid root block and this worked. A valid root block was found, but approx. 13000 bytes off from where it was supposed to be.

Then I added a salvage like function to try and read root block from expected location, otherwise scan and use first valid root block. Next it recursively extracts any files in same directory structure as in the .HDF. And it worked well, I even got it to boot into workbench and send it back to the guy and he was very happy.

This is great as I can now read and write .ADF and .HDF. It makes me want to create that DirOpus or commander like app to enable transfer files on any OS. I really miss such tool
cyberhead97 is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Disk Collection Merging Tool? Fuzzball Collections 1 30 June 2021 11:11
Best disk imaging software? lloyd request.Apps 6 05 May 2012 23:24
BAWarp disk imager BlueAchenar Coders. General 415 24 May 2011 01:17
Disk verification tool? dansalvato support.Apps 1 25 December 2010 04:50
Maverick disk copying tool boing_1000 request.Apps 6 03 October 2006 18:43

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 04:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Page generated in 0.09130 seconds with 15 queries