English Amiga Board - View Single Post

mrv2k · 15 December 2021, 19:45

So, I've been writing a tool to help manage and download Retroplay's WHDLoad archives. It is very much the same as Turran's version except it is a lot more flexible and can do some really useful stuff.

Main Features

Uses Retroplay's dat files for it's database, so is always up to date.

Flexible on where files are saved to.

Supports the beta WHDLoad set

Over 20 filters available including all supported languages.

Create your own curated lists that can be shared.

Scan your existing set and remove unneeded files.

Only downloads what you are missing.

***Warning***

IMPORTANT! Before you do anything to your existing WHDLoad sets, MAKE A BACKUP!. If you delete everything, TOUGH! Post what happened and I can try to fix the problem.

To leave feedback, bugs and suggestions, only use this thread. Please try to make sure that any feedback is reasonable and constructive.

Posts on the Commodore Amiga Facebook group and the r/Amiga Reddit will be ignored as they are a PITA to keep on top of.

Version 0.2a is available

Fixes / Updates

Updated Turran FTP info in docs.
Changed donate link to correct one.
Fixed path gadgets actions when loading prefs files.

Version 0.3a is available

The connection errors have been fixed! Thanks to Conrad Fenech and Brian Lloyd for the help debugging and testing!

Fixes / Updates

Load / Save Prefs now only show prefs files in the file requester.
Changed FTP connections to PBEx_FTP DLL which has fixed the connection errors.
Fixed a typo in the demo download paths. Demos are now downloaded to the right subfolder if the option is enabled.
Fixed bug in list creator/editor that ignored highlighted files when updating main list.
Changed the download window to a console window and added escape key to abort the download at any time.
Blocked close console button

Version v0.4a is available.

Fixes / Updates

Added support for WHDLoad based disk magazines.
Added a check to FTP download to catch if a file is created but no data is downloaded. The error is logged in the ftp.log file and the empty file is deleted.
Fixed a crash if the required DLL files are missing by preventing the program loading.
Added minimize to main window.
Switched dat file download to a console and added ftp log support.
The edit window now work with the filters. You can now filter and add/remove files to the same list.
Added Chip and Fast memory files to the filter.
Preference files have been updated for new filters.
Re-worked the filter panel to make room for the new filters by moving game, demo and magazine filters next to their path boxes.
All FTP settings and FTP paths are now stored in the preferences files and can be edited.
Updated the help window to reflect all the new changes.

Version v0.5a is available.

Fixes / Updates

Added magazines to the 'Clean Files' procedure.
Added 0kb file check to 'Clean Files' procedure.
Added temporary fix to FTP download for 0kb files via DOS FTP command.
Added a few more donation notifications to the download procedure.

Version 0.6a is available

Fixes / Updates

Fixed escaping FTP download crashing the download window.
Added separate download folder for Beta games.
Added preview window to show download file list.
Fixed Port gadget overlap on main window.
Fixed disabled gadgets when loading a prefs file.
Added switch to change between default PB or alternate FTP procedures.
Updated help to reflect changes.
Adapted filter to show beta files separate to games

Version 0.7a is available

This release should hopefully fix the remaining FTP download and connection issues. Make sure that you allow the program through your firewall when Windows asks.

You may get a false AV notification. This is due to how Purebasic wraps Window API commands. Just whitelist the executable if you want to use the program. I've included the source code in the zip file.

Fixes / Updates

Moved FTP procedures to Windows API functions.
Fixed FTP fallback related console and UI crashes.
Fixed non unicode characters in download name saving as empty files.
Fixed open beta folder button.
Removed requirement for PB_FTPEx libraries
Clean files now only lists lha & lzx files.
Added experimental support for Linux Ubuntu 20+ & MacOS (Source available on GitHub. Executables will be released in 0.8a).
Added check for lists folder. Creates if not found.
Added donate button to main window.
Added version information to executables

Version 0.8a is available

There's two versions in this release. A stable release based on the v0.7a FTP code and an unstable version based on the v0.6a FTPEx library download code that crashes the GUI sometimes. The reason is, there were still a few users who had connection issues with the new code. Having both versions available, should hopefully mean that everyone can use the tool.

Only one major addition. There is now a 'Make Folder' button which is there to create new folders out of your existing downloads. For instance, if you have a full set but only want to copy ECS/OCS games to an A500/600/2000/3000, just untick AGA and CD32 in the filter, press the 'Make Folder' button, select a new folder, decide whether to keep the folder structure and press go.

Fixes / Updates

Fixed old dat files not being deleted in Scan FTP procedure.
Cleaned up unneeded variables and lists from the FTP download procedures.
Fixed console actions on Download FTP procedure.
Tweaked FTP procedures to use real FTP variables rather than 1 & 0.
Minor speed up when processing strings.
Added button that makes a new folder of archives based on your filter selections.
Make Folder opens the output folder window when complete.

Version 0.8b is available

This is a quick update that fixes the Turran FTP address as it has changed. You can either delete any prefs files and re-create them or change the ftp address line in any existing prefs file to ftp.grandis.nu with a text editor.

Version 0.9a (Test Version) is available

This is a test release which has HTTP support for downloading from the Turran server. Hopefully this will eliminate the connection issues that have been reported in previous versions. FTP is still supported but is slower due to waiting for connections and folder changes.

Fixes / Updates

Fixed a bug in subfolder creation.
Added ability to download to folder structures either by category or alphabetically.
Changed 'Save Prefs' and 'Load Prefs' to be disabled until the database is loaded.
Centered console on main window.
Added HTTP based downloads.
HTTP & FTP downloading now selectable fron the main window. HTTP is now the default, but the setting is saved in the prefs files if the user
wants to change it.
Added HTTP path to main window and prefs.
Removed 'Use Subfolders' gadget and replaced with selectable combo gadgets for sorting downloads.
Improved the download preview window procedure.
Added a selectable 255 file limit to the download folders for the A500 mini home computer.
All FTP and download gadgets are enabled by default.
The FTP procedure has been updated to support the 0-Z and category downloads.
Removed 'Make Folder' until I can make a better implementation of it.
Fixed a small bug in scrape data procedure.
Updated help file to reflect this versions changes.
Improved the filter speed.
Clear data now keeps the server and filter settings.
Fixed a couple of bugs in the save/load prefs procedure. All gadgets now update properly.
The clean files process is a bit dumber in the sense that it just looks for the right filename and deletes any lha or lzx file that doesn't match. It does still look for zero kilobyte files.
Added a 'refresh list' keyboard shortcut (F5) to update the available files if the user deletes any files manually.

Just a quick update to the full release. I've added several suggested filters and squished a couple more bugs. There is also an x86 version in the archive again.

Version 1.1 is available

Fixes / Updates

Added 28 new filters.
Fixed bug in scrape procedure that forgot to add beta games to Amiga system.
Added fix for x86 compilation
Fixed download number in download screen

Version 1.11 is available

Fixes / Updates

Fixed beta path gadget updating wrong path
Fixed compilation version info
Added paypal.me link to the about window if you think I deserve a coffee

Version v1.12 Released

Fixes / Updates

Fixed a bug where the default.prefs file was loaded at startup but the gadgets didn't update.

This version adds a couple of feature requests. Namely downloading by category with 0-Z sorting and also a facility to make a new folder from an existing downloaded set of files with different sorting and filtering. There's also a couple of squished bugs as well.

Version v1.2 Released

Fixes / Updates

Added alphabetical sorting to category download.
Fixed the sort gadgets not updating when the default.prefs was loaded at startup.
Added an improved 'Make Folder' procedure back. It doesn't support A500 mini 255 file limit though.
Fixed sort gadget logic.
Updated the help window to reflect the new changes.
Increased the window height to fit new button in and space out the filter a bit better.

Version v1.21 Released

Fixes / Updates

Fixed a bug when the FTP and HTTP settings weren't applied to the relevant gadgets when loading the default prefs.

Version v1.3 Released

This version adds CRC32 checking and support for the new Beta - Demos folder in retrogame's folder on the Turran server.

Fixes / Updates

Added CRC32 checking to the update procedure.
Added support for the new Beta Demo folder on the FTP. All beta files are still saved to the selected beta folder.
Users are now prompted to download the dat files when a prefs file is loaded and no data has been downloaded.
All download files are size checked against the dat files and removed it they are wrong.
HTTP download errors are logged in a file called error_log.txt.
You can now toggle the file tree in the download preview.
Downloads are a tiny bit slower as the files are size and crc checked once received.
You can now save the download list as a text file.
Fixed status bar spacing.
Changing prefs files will prompt for clearing the database and will rescan the WHDLoad folders.
Save Prefs button is now active all the time.
The file list now shows any errors in your files by highlighting them with red for missing and blue for bad CRC32.
Moved all donate links to a new window.

Version v1.3 Released

Fixes / Updates

Fixed list filter not updating for games and beta files
Fixed Make Folder not finding any games.
Clear data now resets all settings and filters including sort and download paths.
You can now save the download list as a text file from the download window.

Here's the link...

https://github.com/MrV2K/WHDLoad-Download-Tool

15 December 2021, 19:45	#1
mrv2k Registered User Join Date: Apr 2020 Location: England Age: 53 Posts: 431	WHDLoad Download Tool So, I've been writing a tool to help manage and download Retroplay's WHDLoad archives. It is very much the same as Turran's version except it is a lot more flexible and can do some really useful stuff. Main Features Uses Retroplay's dat files for it's database, so is always up to date. Flexible on where files are saved to. Supports the beta WHDLoad set Over 20 filters available including all supported languages. Create your own curated lists that can be shared. Scan your existing set and remove unneeded files. Only downloads what you are missing. *Warning* IMPORTANT! Before you do anything to your existing WHDLoad sets, MAKE A BACKUP!. If you delete everything, TOUGH! Post what happened and I can try to fix the problem. To leave feedback, bugs and suggestions, only use this thread. Please try to make sure that any feedback is reasonable and constructive. Posts on the Commodore Amiga Facebook group and the r/Amiga Reddit will be ignored as they are a PITA to keep on top of. Version 0.2a is available Fixes / Updates Updated Turran FTP info in docs. Changed donate link to correct one. Fixed path gadgets actions when loading prefs files. Version 0.3a is available The connection errors have been fixed! Thanks to Conrad Fenech and Brian Lloyd for the help debugging and testing! Fixes / Updates Load / Save Prefs now only show prefs files in the file requester. Changed FTP connections to PBEx_FTP DLL which has fixed the connection errors. Fixed a typo in the demo download paths. Demos are now downloaded to the right subfolder if the option is enabled. Fixed bug in list creator/editor that ignored highlighted files when updating main list. Changed the download window to a console window and added escape key to abort the download at any time. Blocked close console button Version v0.4a is available. Fixes / Updates Added support for WHDLoad based disk magazines. Added a check to FTP download to catch if a file is created but no data is downloaded. The error is logged in the ftp.log file and the empty file is deleted. Fixed a crash if the required DLL files are missing by preventing the program loading. Added minimize to main window. Switched dat file download to a console and added ftp log support. The edit window now work with the filters. You can now filter and add/remove files to the same list. Added Chip and Fast memory files to the filter. Preference files have been updated for new filters. Re-worked the filter panel to make room for the new filters by moving game, demo and magazine filters next to their path boxes. All FTP settings and FTP paths are now stored in the preferences files and can be edited. Updated the help window to reflect all the new changes. Version v0.5a is available. Fixes / Updates Added magazines to the 'Clean Files' procedure. Added 0kb file check to 'Clean Files' procedure. Added temporary fix to FTP download for 0kb files via DOS FTP command. Added a few more donation notifications to the download procedure. Version 0.6a is available Fixes / Updates Fixed escaping FTP download crashing the download window. Added separate download folder for Beta games. Added preview window to show download file list. Fixed Port gadget overlap on main window. Fixed disabled gadgets when loading a prefs file. Added switch to change between default PB or alternate FTP procedures. Updated help to reflect changes. Adapted filter to show beta files separate to games Version 0.7a is available This release should hopefully fix the remaining FTP download and connection issues. Make sure that you allow the program through your firewall when Windows asks. You may get a false AV notification. This is due to how Purebasic wraps Window API commands. Just whitelist the executable if you want to use the program. I've included the source code in the zip file. Fixes / Updates Moved FTP procedures to Windows API functions. Fixed FTP fallback related console and UI crashes. Fixed non unicode characters in download name saving as empty files. Fixed open beta folder button. Removed requirement for PB_FTPEx libraries Clean files now only lists lha & lzx files. Added experimental support for Linux Ubuntu 20+ & MacOS (Source available on GitHub. Executables will be released in 0.8a). Added check for lists folder. Creates if not found. Added donate button to main window. Added version information to executables Version 0.8a is available There's two versions in this release. A stable release based on the v0.7a FTP code and an unstable version based on the v0.6a FTPEx library download code that crashes the GUI sometimes. The reason is, there were still a few users who had connection issues with the new code. Having both versions available, should hopefully mean that everyone can use the tool. Only one major addition. There is now a 'Make Folder' button which is there to create new folders out of your existing downloads. For instance, if you have a full set but only want to copy ECS/OCS games to an A500/600/2000/3000, just untick AGA and CD32 in the filter, press the 'Make Folder' button, select a new folder, decide whether to keep the folder structure and press go. Fixes / Updates Fixed old dat files not being deleted in Scan FTP procedure. Cleaned up unneeded variables and lists from the FTP download procedures. Fixed console actions on Download FTP procedure. Tweaked FTP procedures to use real FTP variables rather than 1 & 0. Minor speed up when processing strings. Added button that makes a new folder of archives based on your filter selections. Make Folder opens the output folder window when complete. Version 0.8b is available This is a quick update that fixes the Turran FTP address as it has changed. You can either delete any prefs files and re-create them or change the ftp address line in any existing prefs file to ftp.grandis.nu with a text editor. Version 0.9a (Test Version) is available This is a test release which has HTTP support for downloading from the Turran server. Hopefully this will eliminate the connection issues that have been reported in previous versions. FTP is still supported but is slower due to waiting for connections and folder changes. Fixes / Updates Fixed a bug in subfolder creation. Added ability to download to folder structures either by category or alphabetically. Changed 'Save Prefs' and 'Load Prefs' to be disabled until the database is loaded. Centered console on main window. Added HTTP based downloads. HTTP & FTP downloading now selectable fron the main window. HTTP is now the default, but the setting is saved in the prefs files if the user wants to change it. Added HTTP path to main window and prefs. Removed 'Use Subfolders' gadget and replaced with selectable combo gadgets for sorting downloads. Improved the download preview window procedure. Added a selectable 255 file limit to the download folders for the A500 mini home computer. All FTP and download gadgets are enabled by default. The FTP procedure has been updated to support the 0-Z and category downloads. Removed 'Make Folder' until I can make a better implementation of it. Fixed a small bug in scrape data procedure. Updated help file to reflect this versions changes. Improved the filter speed. Clear data now keeps the server and filter settings. Fixed a couple of bugs in the save/load prefs procedure. All gadgets now update properly. The clean files process is a bit dumber in the sense that it just looks for the right filename and deletes any lha or lzx file that doesn't match. It does still look for zero kilobyte files. Added a 'refresh list' keyboard shortcut (F5) to update the available files if the user deletes any files manually. Just a quick update to the full release. I've added several suggested filters and squished a couple more bugs. There is also an x86 version in the archive again. Version 1.1 is available Fixes / Updates Added 28 new filters. Fixed bug in scrape procedure that forgot to add beta games to Amiga system. Added fix for x86 compilation Fixed download number in download screen Version 1.11 is available Fixes / Updates Fixed beta path gadget updating wrong path Fixed compilation version info Added paypal.me link to the about window if you think I deserve a coffee Version v1.12 Released Fixes / Updates Fixed a bug where the default.prefs file was loaded at startup but the gadgets didn't update. This version adds a couple of feature requests. Namely downloading by category with 0-Z sorting and also a facility to make a new folder from an existing downloaded set of files with different sorting and filtering. There's also a couple of squished bugs as well. Version v1.2 Released Fixes / Updates Added alphabetical sorting to category download. Fixed the sort gadgets not updating when the default.prefs was loaded at startup. Added an improved 'Make Folder' procedure back. It doesn't support A500 mini 255 file limit though. Fixed sort gadget logic. Updated the help window to reflect the new changes. Increased the window height to fit new button in and space out the filter a bit better. Version v1.21 Released Fixes / Updates Fixed a bug when the FTP and HTTP settings weren't applied to the relevant gadgets when loading the default prefs. Version v1.3 Released This version adds CRC32 checking and support for the new Beta - Demos folder in retrogame's folder on the Turran server. Fixes / Updates Added CRC32 checking to the update procedure. Added support for the new Beta Demo folder on the FTP. All beta files are still saved to the selected beta folder. Users are now prompted to download the dat files when a prefs file is loaded and no data has been downloaded. All download files are size checked against the dat files and removed it they are wrong. HTTP download errors are logged in a file called error_log.txt. You can now toggle the file tree in the download preview. Downloads are a tiny bit slower as the files are size and crc checked once received. You can now save the download list as a text file. Fixed status bar spacing. Changing prefs files will prompt for clearing the database and will rescan the WHDLoad folders. Save Prefs button is now active all the time. The file list now shows any errors in your files by highlighting them with red for missing and blue for bad CRC32. Moved all donate links to a new window. Version v1.3 Released Fixes / Updates Fixed list filter not updating for games and beta files Fixed Make Folder not finding any games. Clear data now resets all settings and filters including sort and download paths. You can now save the download list as a text file from the download window. Here's the link... https://github.com/MrV2K/WHDLoad-Download-Tool Last edited by mrv2k; 29 February 2024 at 15:10. Reason: Update