Retroplay's WHD uploads

[Enverex]

Hey Retroplay, I noticed that one of the games is LZX rather than LHA, any particular reason why?

Calephar_v1.0_Files_2661.lzx

[Retroplay]

Lha fails to extract the archive properly.

That goes for Zine 11 too btw.
http://eab.abime.net/showpost.php?p=...&postcount=579

The way i update on linux is installing realpath, megasync and megatools, copy the 'new' folder i want to update to my mega cloud drive and use this script. The script removes all the local files whose names don't match in the cloud folder (meaning this is of use for downloaders, not the uploader that *wants* to delete and upload files from the local dir to mirror on the cloud). Then i use megasync to update the 'wrong checksum' and new files. Watch out for the megasync bug mentioned though.

As explained in the link, mega is rather terrible to keep public links downloads synced, but it's possible, in spite of the annoying bugs and missing features in megasync. Maybe megatools will fill the hole eventually. Unfortunately, mega is still the best upload service for syncing in spite of this. The main issue is megasync not working on public folders, which is just stupid imo; but this problem of outdated local files being uploaded to your 'copy' is annoying enough to merit a script too.

Anyway, i think i might end using the other script mentioned here, seems like it might be a better solution, because of less steps (taking advantage that every different whdload archive on the mega folder has a different name, instead of a general solution that has to care about checking checksums). This will fail if a upload of a different file with the same name happens, as i'm sure everyone is aware.

It's possible megatools will end up providing a 'safe' (with checksum check) public sync eventually, or at least a more safe download method with megadl that deletes extra files and files which look outdated (comparing file timestamps/creation dates). Megadl already works on public folders, and it already doesn't redownload files which already exist, it just needs to delete extra files to be nearly equivalent to a sync.

BTW, the RageV1 and RageV2 files mentioned 2 pages back appear to not get scanned correctly by fs-uae-launcher so they never appear on the autoconfig. It's probably to be expected from some illegal character or incompatibility from whatever library fs-uae-launcher is using borking on the archives. I don't know if turning them to lzx would help because these errors tend to be on the host side.
edit: nope. The reason for this appears to be the openretro db not having the correct entry yet. Shutting up now.

[zzarko]

For those interested, new version of PyRetroplay script is on the Bitbucket:
https://bitbucket.org/zzarko/pyretroplay/overview. Unpacking of weird lha/lzx archives on Linux should now be solved (so far, I haven't found one that unpacks incorrectly), if using Linux filesystem, of course.

Changelog:
* Switched from 7z to The Unarchiver (unar package on Linux) for unpacking on PC side
* Added support for LZX archives (if same version of file are available both as LHA and LZX, LZX will be used)
* Added Debug mode of execution (doesn't change local files)
* Added current FTP list to local database
* Script now checks for archives that aren't on FTP server, but exist in local directories
* Backup of local database is made before writing a new one
* Many small improvements to code
* New bugs?

[zzarko]

@Retroplay: Any reason for EmeraldMines_v1.0_CD to exist in both lha and lzx versions?

[Foul]

Excellent news ! thx

edit : something wrong happened !

foul@Foulnux:/mnt/Amiga/WHDLoad$ ./PyRetroplay.py 
Starting PyRetroplay ...

Connecting to ftp://grandis.nu ...
Checking Commodore_Amiga_-_WHDLoad_-_Demos/Z ...               

Scanning local archives and directories ...
Scanning Retroplay/Commodore_Amiga_-_WHDLoad_-_Demos/Z ...               
Found newer version (1.2) for:  Games_WHDLoad/A/
Deleting directory: Games_WHDLoad/A/
Traceback (most recent call last):
  File "./PyRetroplay.py", line 426, in <module>
    if not Debug: os.remove(unpackpath + ".info")
OSError: [Errno 2] No such file or directory: 'Games_WHDLoad/A/.info'
foul@Foulnux:/mnt/Amiga/WHDLoad$

Deleted my whole Games_WHDLoad/A/ directory !!! :\

[Retroplay]

Originally Posted by zzarko

@Retroplay: Any reason for EmeraldMines_v1.0_CD to exist in both lha and lzx versions?

http://eab.abime.net/showpost.php?p=...&postcount=579

[zzarko]

Originally Posted by Foul

edit : something wrong happened !

I sent you a PM, I hope to solve this. No wonder your post number was 666...

[Foul]

foul@Foulnux:/mnt/Amiga/WHDLoad$ ./PyRetroplay.py 
Starting PyRetroplay ...

Connecting to ftp://grandis.nu ...
Checking Commodore_Amiga_-_WHDLoad_-_Demos/Z ...               

Scanning local archives and directories ...
Scanning Retroplay/Commodore_Amiga_-_WHDLoad_-_Demos/Z ...               

These archives no longer exist on Retroplay FTP:
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/C/Calephar_v1.0_Files.lha 
   Unpacked at: Games_WHDLoad/C/CalepharFiles
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/G/GunbeeF99_v1.1_AGA.lha 
   Unpacked at: Games_WHDLoad_AGA/G/GunbeeF99AGA
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/K/K240_v1.0_1565.lha 
   Unpacked at: Games_WHDLoad/K/K240
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_De_Kick13_0075.lha 
   Unpacked at: Games_WHDLoad/L/LoomDeKick13
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_De_Kick31_0075.lha 
   Unpacked at: Games_WHDLoad/L/LoomDeKick31
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_Es_Kick13.lha 
   Unpacked at: Games_WHDLoad/L/LoomEsKick13
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_Es_Kick31.lha 
   Unpacked at: Games_WHDLoad/L/LoomEsKick31
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_Fr_Kick13.lha 
   Unpacked at: Games_WHDLoad/L/LoomFrKick13
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_Fr_Kick31.lha 
   Unpacked at: Games_WHDLoad/L/LoomFrKick31
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_Kick13_0618.lha 
   Unpacked at: Games_WHDLoad/L/LoomKick13
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/L/Loom_v1.2_Kick31_0618.lha 
   Unpacked at: Games_WHDLoad/L/LoomKick31
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/M/Marblelous_v1.1.lha 
   Unpacked at: Games_WHDLoad/M/Marblelous
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/M/MegaBlast_v1.0.lha 
   Unpacked at: Games_WHDLoad/M/MegaBlast
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/P/PinballBrainDamage_v1.0_AGA.lha 
   Unpacked at: Games_WHDLoad_AGA/P/PinballBrainDamageAGA
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/P/PulsatorDemo_v1.1_AGA.lha 
   Unpacked at: Games_WHDLoad_AGA/P/PulsatorDemoAGA
  Retroplay/Commodore_Amiga_-_WHDLoad_-_Games/W/WayOfTheLittleDragon_v1.1_NTSC.lha 
   Unpacked at: Games_WHDLoad/W/WayOfTheLittleDragonNTSC
If they are not needed, they should be removed manually.

Saving current database...

Done.
foul@Foulnux:/mnt/Amiga/WHDLoad$

Any way to remove them automatically ?

[zzarko]

Originally Posted by Foul

Any way to remove them automatically ?

I thought about it, but decided not to do it, as I'm not sure if such archives should be erased from local disk automatically. For example, I was playing a game, had saved position, and after the update, it is gone (ok, sort of like your games/A/ directory, but that was an error). I'm thinking about moving them to separate directory where non-Retroplay games could reside, that would be administered by user, or I could leave them like this and let the user decide what to do.

I know the reason for removing some of those games were legal issues, but I guess there are people that have a legal copies, it would be wrong to delete them if they keep those copies with all their other games. As for Loom/K240, I couldn't devise a logic that could reliably conclude that there is a newer version of the game with different naming and that others should be erased and that this situation is different than previous (mostly because there are other games that have multiple copies in Retroplay archive).

So, for now, it stays like this. If someone has a good solution for this, I'll gladly implement it. One easy solution is to add an option to erase them that would be False by default, and if you want those games erased, just set the option to True? Or script could ask the user if they should be deleted, but that would be annoying for those who do not want those games erased (I know it would be for me). If you have a suggestion, please tell...

[Foul]

no prob i understand and let them like that for the moment

[Foul]

AmigaDOS update script genrated as: update2017-10-11-1
Copy the script and its files to real Amiga.

Just a typo on "generated"

[liveevil]

[Enverex]

They won't always throw errors when extracting, but chances are they have broken one or more characters in one or more files due to how Windows handles the filenames. It'll work fine on Amigas and most Unix systems, but Windows will break things.

Hired Guns for example isn't in your list, but if you extract it on Windows, it'll screw up a load of the filenames silently.

[liveevil]

Originally Posted by Enverex

Hired Guns for example isn't in your list, but if you extract it on Windows, it'll screw up a load of the filenames silently.

Yeah, I did mention that "the rest of the games listed on the server in 'Games_WHDLoad_UnpackOnAmiga' have some files that contain special characters but they extract fine on Windows".

Hired Guns has files with special characters :-

Espa±ol
Franþais

But they extract on Windows fine and the game works including running the files with the special characters. The ones I listed have definite problems when extracted to Windows and you'll need a special program or skills to delete/move the said files.

[Enverex]

Odd, they didn't work for me when I tried to play (complained about those files missing). I wonder, if you're using WinUAE, that the requests for the files are also being mangled, so in a weird round-about way it still works.

[cyberhead97]

I don't know if this is the right place to post fixes for EAB WHDLoad Pack 3.0, but here goes.

Starting AmigaKarate whdload slave results in an error about " AMIGA KARATE" doesn't exist. Renaming the file from "AMIGA KARATE" to " AMIGA KARATE" in data seems to fix it

[Enverex]

My version direct from RetroPlay's FTP seems fine so that issue may be specific to the pack.

[Retroplay]

Yup, no problems here either.
I always test if the game/demos run before uploading anything.

I guess you should post the issue here:
http://eab.abime.net/showthread.php?t=68704

Hi, I think you missed the games 'Read & Rhyme' and 'Decimal Dungeon' from WHDLoad's latest updates.