HstWB Installer - Making preinstalled HDF legal for sharing

[liveevil]

I'm liking the sound of this :-

The support for user packages comes with a preconfigured installation of EAB WHDLoad Packs 3.0 demos, games, games aga and games that must be unpack on amiga. With a self install image, it is very easy to install these packs. It only required that users download, extract EAB WHDLoad Packs them selves and copies the extracted folders to a user packages directory with same naming of directories. The self install process will then detect these user packages and install them as part of the installation process.

Because I use Retroplay's Up-to-date / Repacked WHDLoad LHA's. Turran's are slightly out of date and a few errors (one of which you reported in Retroplay's thread) and the Unpack-on-Amiga section is missing some that get errors on Windows filesystem and can't be deleted, so I think it would be better to use Retroplay's.

Looking forward to your release.

[cyberhead97]

Oh dear, it's just been too long since my last post. Christmas came and some critical bugs was discovered causing delay in releases.

I did manage to release v1.2.0-BETA1 for download here https://github.com/henrikstengaard/h...ag/1.2.0-BETA1. This is only HstWB Installer and not any self install images.

Here's an overview of the critical issues I have discovered and being fixed in upcoming v1.2.0-BETA2:

1. FirstLine from aminet.net requires minimum 68020 and that doesn't work as I want it to be 68000 compatible. It has been fixed and replaced by a script that uses simple sed operations to do the same.
2. ReqList seems to require fast mem and it must be possible to run it on plain unexpanded Amiga's. It will be fixed by replacing ReqList with RequestList.

The images included may become corrupt either by misconfiguration or bugs that still exist in PFS3. All images has been prepared, partitioned and formatted using Workbench 3.1 HDToolBox, PFS3-AIO v2.3, pfsformat from pfs53_3.lha. Partitions use dostype 0x53465303 (PFS\3), where they should use 0x50445303 (PDS\3) when hard drive/partitions are larger than 4GB. There more details about this here https://github.com/henrikstengaard/h...ller/issues/15 currently being investigated by Tomasz Janiszewski and Toni Wilen. I created new hdf images with Amiga OS 3.9 HDToolBox and PDS\3 dostype and they seems to work better. But they still get corrupt especially when extracting/deleting EmeraldMinesCD.lha from EAB WHDLoad Packs v3.0 Games_UnpackOnAmiga.

Some new things has also been developed since last post and is a new demos and games launcher called HST-Launcher. It's main purpose is for low memory use and targeted for unexpanded Amiga's with only chip mem. The EAB WHDLoad Games menu packages have been updated with new hst-games menus using the new HST-Launcher. The hst-games menus comes in a hst-games1mb version, which is filtered for games that should work with only only 1mb chip mem. From what I can see this coveres ~900 games, which is pretty impressive thanks to the WHDLoad team. It's off course with blinking when loading, since preload in fast ram is not possible.

To support a setup with only chip ram, I also created a new HstWB-Mini package. It's a bare minimum Workbench 3.1 setup, which should be used FFS filesystem to maximise available chip ram. This is because filesystems like PFS3 and SFS are loaded into ram during boot. I have noticed if PFS3 is used without fast ram, it seems to go crasy and eat 200-300kb chip ram per partition. A special image has also been created for chip ram only setup, which has 2 partitions with FFS v45.16. It's second partition is not automounted to save/free 20-25 kb chip ram. It should contain a Workbench 3.1 installation and is mounted when needed e.g. from HstWB-Mini's boot menu.

I'm now focusing on getting v1.2.0-BETA2 released and record videos to show how it works.

[cyberhead97]

I have just released HstWB Installer v1.2.0-BETA2 and it's available for download here https://github.com/henrikstengaard/h...ag/1.2.0-BETA2.

It includes following changes:

• Updated setup and run scripts to show warnings when images uses harddrives larger than 4GB. In some situations PFS3AIO v2.3 (used for all images) can become corrupt and result in requester popups "Wrong block id". This is currently being investigated with new experimental versions of PFS3AIO from Toni Wilen.
• Replaced firstline with sed script for 68000 compatibility.
• Replaced reqlist with requestline for 68000 compatibility.
• Changed package installation and user package installation menu's options for better layout using requestlist.
• Updated hstwb-installer setup, run scripts and self install mode to autodetect Cloanto Amiga Forever 7 Workbench adf and Kickstart rom files.
• Updated self install images to remove packages installation files after they are installed.
• Added back option to change installer mode menu.
• Added a few new images installed and partitioned with dostype PDS3.

Next I'm going to build an 8GB self install image based on HstWB Installer v1.2.0-BETA2 and use it to record a demo video.

[cyberhead97]

Building new self install images has been postponed until the PFS3AIO corruption and errors are solved or at a more stable state. Currently I'm testing a new experimental version from Toni Wilen with a fix from Michiel Pelt, which seems to solve corruption and errors when copying or extracting large amount of files very fast, e.g. when extracting EmeraldMinesCD.lha from EAB WHDLoad Packs v3.0 Games_UnpackOnAmiga.

The good news is that tests with latest experimental version of PFS3AIO has been successful and I haven't been able to get any corruption or errors so far. I still need run additional tests with different sized hdf files to get better overview of it's current state.

[cyberhead97]

Here a quick status update for HstWB-Installer and progress with v1.2.0-BETA3 and finalizing v1.2.0.

All hdf images have been re-created from scratch using AmigaOS 3.9 and calculated drive geometry. They all have 5 unused cylinders at end of the last partition. Those that use PFS3 have the latest PFS3AIO v3.0 installed, which was released recently by Toni Wilen.

The hdf images are double zipped to save space. Usually it doesn't make sense to zip a zip file, but in this case, it does making all images take only 3MB instead of ~150MB before double zipping was introduced.

The EAB WHDLoad Games and Demos Menu package has gotten a few tweaks added. This is to minimize amount requesters being shown during installation, so there now just one configure menu allowing more control of what will be installed: AGS2 Menu, Hst Launcher Menu and iGame screenshots. I have also introduced iGame screenshots in resolutions 320x128 and 320x256 for both native, interlaced and RTG resolutions. I have also added additional assigns used for installation to give more flexibility of where menus and screenshots are installed.

I plan to do the same for the ClassicWB packages, converting requester popups to one configure menu during installation.

This result of this will be v1.2.0-BETA3 and if all goes well, it will be promoted to final v1.2.0. Last steps going from BETA to final will be adding the missing documentation and updating tutorials, which are currently lacking for e.g. AmigaOS 3.9 and user packages installation.

At this point HstWB-Installer has grown from a simple script to a big and slight complex application. There’s a lot of code I would like to rewrite and at the same time I also want to have a GUI version with cross platform support. Python and TK seems a good candidate for this as I can have one code base with support for Windows, macOS and Linux. I have already done a few experiments making Python run as if it was a native Windows or macOS app and even written scripts to produce a .dmg for macOS users to easily install and use HstWB Installer.

But first it’s v1.2.0-BETA3, which will be ready within a few days together with a few self-install images. This time there will also be an Amibian .img self installer for Raspberry Pi users.

[cyberhead97]

I got the latest amibian and making a self install tailored for it. It automatically detects and installs a1200 kickstart 3.1 during startup before launching enulator. I have also created both amiberry and chips uae4arm configs running dh0 using latest pfs3aio v3.0. So far it works really well, but needs a few bash script tweaks and a readme describing the installation.

[kriz]

Excellent work cyberhead97 !! Looking forward to try that amibian image

[cyberhead97]

HstWB Installer v1.2.0-BETA3 has been released at http://hstwb.firstrealize.com.

Following downloads are available:

• 4GB HstWB self install HDF image v1.2.0-BETA3.
• 8GB HstWB self install HDF image v1.2.0-BETA3.
• Amibian v1.4.1001 img preconfigured with HstWB self install HDF image v1.2.0-BETA3.
• HstWB Installer v1.2.0-BETA3 MSI Installer.
• HstWB Installer v1.2.0-BETA3 Portable.

v1.2.0-BETA3 is very close to being final. All functionality is there, but readme and documentation has not been updated much yet. This is what's left from being final and potential bug fixing.

As you can see I have created updated version Amibian with HstWB Installer preconfigured and ready to use for both Chips UAE4ARM and Amiberry. I recommend using Chips UAE4ARM as it more stable with JIT enabled, which makes installation process very fast.

The Amibian version is made as automated as possible for users, so they only need to do the following:

1. Write img to sd-card.
2. Extract hstwb-installer_install.zip to root of FAT32 formatted usb stick.
3. Copy Cloanto Amiga Forever Workbench 3.1 and Kickstart roms to usb stick directories. Own dumps of original Workbench 3.1 floppy disks and Kickstart roms are also supported.
4. Insert sd-card and usb stick in RaspBerry Pi and power on.
5. First time boot shows dialogs asking to expand filesystem yes/no, automatically installs kick31.rom for emulators and selection of emulator to use.
6. Emulator starts HstWB Installer self install.

Amibian version includes a readme describing step by step how to use it.

Please give it a try while I update readme and documentation

EDIT - Forgot to mention all images have been updated and use PFS3AIO v3.0.

[cyberhead97]

Running latest version of HstWB Installer in install, build self install or test modes installs a hstwb-installer theme for FS-UAE, which adds transparent floppy, cd-rom and harddrive activity led overlays.

I needed this for testing installation of EAB WHDLoad Packs done my the included user packages and they are so large, so the theme made the activity visible as shown in lower right corner of the attached screenshot.

[cyberhead97]

I noticed the EAB WHDLoad Packs has changed recently and therefore the user packages will not work as they expect them to be extracted on Windows, macOS or Linux since it copies it's directories.

It's a great change from extracted directories to single compressed lha/lzx to improve issues with Amiga only filenames. This also improves usage in general as compressed lha/lzx can easily be extracted through added harddrive directories in emulators, which is how HstWB Installer will use them.

Updating the user packages install scripts for HstWB Installer doesn't take long time it self, but I have to download all from Retroplay directory in EAB file server first to get an overview of demos and games included for updating scripts and test with. This takes some time with a 50kb download limit.

[DamienD]

Originally Posted by cyberhead97

Updating the user packages install scripts for HstWB Installer doesn't take long time it self, but I have to download all from Retroplay directory in EAB file server first to get an overview of demos and games included for updating scripts and test with. This takes some time with a 50kb download limit.

...as you have more than 50 posts, register an FTP account here: https://www.grandis.nu:444/eabadduser/

[cyberhead97]

Thanks, I will add user to speed up my downloads

[cyberhead97]

For the Raspberry Pi and Amibian users I have recorded a video showing setup of Amibian with HstWB Installer available here [ Show youtube player ]. There's chapters in the description to quickly jump to interesting parts.

A EAB ftp user account did help my download speed of Retroplay's WHDLoad demos and games. Thanks for the tip, DamienD.

I will update the user packages installing Retroplay's WHDLoad demos and games by making an AmigaDOS script that automatically detects an uncompress demos and games users have downloaded from EAB file server. For a fancy version it could ask users which versions they want installed: All, AGA, CD32, CDTV etc.

... and at the same time finalize documentation.

[cyberhead97]

Support for Retroplay's WHDLoad demos and games is taking shape.

I have created a powershell script to scan Commodore_Amiga_-_WHDLoad_-_Games and Commodore_Amiga_-_WHDLoad_-_Demos for *.lha and *.lzx and builds a menu for easy selection and filtering of entries to install. It's able to detect hardware and language from filenames and lists these in a menu. Click on hardware or language switches between install, skip and updates number of entries available and to install.

This is currently in a proof-of-concept state and when it's becoming more stable I will also write a python version to for cross platform support.

Here's some screenshots of menu the powershell script can build.

[Sinphaltimus]

I have been following this since the OP and I have to say bravo. It's been interesting following you up to this point and it really looks like you're extraordinary progress. Great job. Can't wait to give the final a spin. I've been very busy of late or I'd take your beta3 for a spin. Just wanted to say thanks and congrats.

[RoC]

@Cyberhead

GREAT! This is a work of love. I installed a new CF based on HstWB and works like a champ. I appreciated very much the "read-to-go" images, which allows to quick start as a matter of minutes. Moreover, the script to create the ags entries makes it ready to go with WHDLoad

I made a small donation to contribute to your hard work

[cyberhead97]

@Cyberhead

GREAT! This is a work of love. I installed a new CF based on HstWB and works like a champ. I appreciated very much the "read-to-go" images, which allows to quick start as a matter of minutes. Moreover, the script to create the ags entries makes it ready to go with WHDLoad

I made a small donation to contribute to your hard work

Thanks roc, thats much appreciated and glad to hear you find it useful.

I have been following this since the OP and I have to say bravo. It's been interesting following you up to this point and it really looks like you're extraordinary progress. Great job. Can't wait to give the final a spin. I've been very busy of late or I'd take your beta3 for a spin. Just wanted to say thanks and congrats.

Thanks Sinphaltimus.

It has been quite a journey building HstWB Installer so far and I keep getting idea about improving it and make it a good tool for Amiga users to easily setup their Workbench and additional installations.

I have managed to complete the Build EAB WHDLoad Install script in both Powershell and Python with extra files to run them directly from Windows Explorer, macOS Finder and Ubuntu Files.

I have created Build EAB WHDLoad Install versions for anyone to test:

1. Standalone: Can be run from Workbench by double-clicking UserPackage-Installation icon. Download standalone version from https://s3-eu-west-1.amazonaws.com/h...standalone.zip.
2. User Packages: Can be used in HstWB Installer user packages for self install. Make sure to remove existing files from user packages directory build by HstWB Installer. Download user packages version from https://s3-eu-west-1.amazonaws.com/h...erpackages.zip.

I have also written a preparing self install tutorial at https://github.com/henrikstengaard/h...#user-packages, which describes how to use the Build EAB WHDLoad Install scripts with screenshots and details for Windows, macOS and Linux.

This tutorial is part of the documentation being updated and placed in HstWB Installer github wiki instead of the readme.

[cyberhead97]

Getting python 2 to handle utf-8 properly and convert it to iso-8859-1 charset for AmigaDOS gave a lot of scratching in the head.

Im sure other python programmers have been down that road.

[RoC]

Originally Posted by cyberhead97

Thanks roc, thats much appreciated and glad to hear you find it useful.

My pleasure

Originally Posted by cyberhead97

It has been quite a journey building HstWB Installer so far and I keep getting idea about improving it and make it a good tool for Amiga users to easily setup their Workbench and additional installations.

I have managed to complete the Build EAB WHDLoad Install script in both Powershell and Python with extra files to run them directly from Windows Explorer, macOS Finder and Ubuntu Files.

I have created Build EAB WHDLoad Install versions for anyone to test:

1. Standalone: Can be run from Workbench by double-clicking UserPackage-Installation icon. Download standalone version from https://s3-eu-west-1.amazonaws.com/h...standalone.zip.
2. User Packages: Can be used in HstWB Installer user packages for self install. Make sure to remove existing files from user packages directory build by HstWB Installer. Download user packages version from https://s3-eu-west-1.amazonaws.com/h...erpackages.zip.

I have also written a preparing self install tutorial at https://github.com/henrikstengaard/h...#user-packages, which describes how to use the Build EAB WHDLoad Install scripts with screenshots and details for Windows, macOS and Linux.

This tutorial is part of the documentation being updated and placed in HstWB Installer github wiki instead of the readme.

Good you are keeping up the good work. As a user, a few (minor) comments/suggestions in case you wish to consider them for the future:

- I am not sure whether this comes from the BetterWB, however the HDToolbox comes out of the box with scsi.device instead of uaehf.device. I believe many of us install the CF on WinUAE and some even use it exclusively on emulation, so this may help. Not a big deal to change it manually, though

- The HstLauncher looks very promising for supporting low-end classics. While I use AGS2 on the A1200, I plan using HstLaunch for the rest of the WHDload collection such the magazines/intros/betas, although I should start the work and don't know if this is going to be feasible. I am reporting if you like the idea for the "official" release.

- I found extremely convenient using the images and started with the release v11 with 16 Gigs. Any luck to get a v12 with 16 GB for the BetterWB 31?

[cyberhead97]

All comments and suggestions are very welcome.

- I am not sure whether this comes from the BetterWB, however the HDToolbox comes out of the box with scsi.device instead of uaehf.device. I believe many of us install the CF on WinUAE and some even use it exclusively on emulation, so this may help. Not a big deal to change it manually, though

I don't know if it's wise to change the default tooltype of BetterWB's HDToolbox from scsi.device to uaehf.device. For HDToolbox uses I personally use Amiga OS 3.9, which shown a prompt asking which device (lists both scsi.device and uaehf.device for selection) to use before starting to initialize and prepare a harddrive.

The HstLauncher looks very promising for supporting low-end classics. While I use AGS2 on the A1200, I plan using HstLaunch for the rest of the WHDload collection such the magazines/intros/betas, although I should start the work and don't know if this is going to be feasible. I am reporting if you like the idea for the "official" release.

The mail goal with Hst Lanucher is supporting the low-end Amigas with chip mem only. The current version needs an updated to support the new Retroplay WHDLoad entries incl. beta and demos. AGS2 needs the same update.

For low-end Amigas PFS3/PDS3 partitions are not optimal as these eats about 200-300kb of chip mem per partition, when there is no fast mem available. I have created FFS images suitable for low-end Amigas and created a HstWB Mini, which is a absolute minimal setup to get maximum available chip mem. It needs some work to fully run as I want it to, which is planned for HstWB Installer v1.2.0-BETA4.

- I found extremely convenient using the images and started with the release v11 with 16 Gigs. Any luck to get a v12 with 16 GB for the BetterWB 31?

I will build a v1.2.0-BETA3 16gb self install image tonight and make it available for download. 16gb seems to be required to install all WHDLoad demos, games incl. beta and demos.