UHC Tools

[DisasterIncarna]

Been using AGET a lot for my Work in Progress GUI for UHC Tools, adding support to download from sites UHC doesnt do via its CLI commands, i can search and download music now from Remix.KWED.org and AmigaRemix, however i have come accross a bit of a situation, not sure if its an issue with AGET or with "the interwebs" doing its thing.

Try the following:

UHC:C/AGET "https://remix.kwed.org/download.php/6609/Dr%20Future%20-%20Space%20Harrier%20%28NAPAPBWPMIF%20Mix%29.mp3" to ram:

results in

Buffer size: 128kB
Connected to remix.kwed.org:443
TLS/SSL connection using 'TLS_CHACHA20_POLY1305_SHA256'
Sent GET request!
Server response: 302 Found
Redirecting to: https://remix.kwed.org/files/RKOfiles/Dr Future - Space Harrier (NAPAPBWPMIF Mix).mp3
Connected to remix.kwed.org:443
TLS/SSL connection using 'TLS_CHACHA20_POLY1305_SHA256'
Sent GET request!
https://remix.kwed.org/files/RKOfiles/Dr Future - Space Harrier (NAPAPBWPMIF Mix).mp3: 400 Bad Request

It has failed after a URL redirect because all the percentage codes have been stripped when redirecting, causing the download to fail, i havent encountered this often, but it does happen.

if you manually add the missing % codes back into the redirected URL.

uhc:c/aget "https://remix.kwed.org/files/RKOfiles/Dr%20Future%20-%20Space%20Harrier%20%28NAPAPBWPMIF%20Mix%28.mp3" to ram:

then you get success.

Buffer size: 128kB
Connected to remix.kwed.org:443
TLS/SSL connection using 'TLS_CHACHA20_POLY1305_SHA256'
Sent GET request!
Server response: 200 OK
Receiving file: [o]
103.82kB received in 266ms - 390.01kB/s

Ofc adding them manually after a fail isnt ideal, having the redirected URL get the % codes stuffed back in would be the better solution.....if possible.

are the %20's, %28's etc being erased by AGET itself or is it getting the cleansed/broken link back as part of the redirect?

[patrik]

It is getting a cleaned redirect string from the server, see the Location header field returned:

$ curl -v "https://remix.kwed.org/download.php/6609/Dr%20Future%20-%20Space%20Harrier%20%28NAPAPBWPMIF%20Mix%29.mp3"
*   Trying 185.39.146.118...
* Connected to remix.kwed.org (185.39.146.118) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: remix.kwed.org
* Server certificate: R3
* Server certificate: ISRG Root X1
> GET /download.php/6609/Dr%20Future%20-%20Space%20Harrier%20%28NAPAPBWPMIF%20Mix%29.mp3 HTTP/1.1
> Host: remix.kwed.org
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Server: nginx/1.18.0 (Ubuntu)
< Date: Sat, 16 Jul 2022 08:11:22 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Permissions-Policy: interest-cohort=()
< Location: /files/RKOfiles/Dr Future - Space Harrier (NAPAPBWPMIF Mix).mp3
< 
* Connection #0 to host remix.kwed.org left intact

I need to make aget escape redirect urls to work around that server behavior.

[patrik]

Originally Posted by DisasterIncarna

Ofc adding them manually after a fail isnt ideal, having the redirected URL get the % codes stuffed back in would be the better solution.....if possible.

The problem is actually only missing percent-encoding for the space character. The reason is that the space character is used to separate parts of the HTTP request, so if spaces are used in the path part of the request without %-encoding them, this will generate an invalid request.

Please give this updated version a try:
http://megaburken.net/~patrik/pt/aget

As a bonus, the following will now also work:

aget "https://remix.kwed.org/download.php/6609/Dr Future - Space Harrier (NAPAPBWPMIF Mix).mp3"
Buffer size: 128kB
Connected to remix.kwed.org:443
TLS/SSL connection using 'TLS_CHACHA20_POLY1305_SHA256'
Sent GET request!
Server response: 302 Found
Redirecting to: https://remix.kwed.org/files/RKOfiles/Dr Future - Space Harrier (NAPAPBWPMIF Mix).mp3
Connected to remix.kwed.org:443
TLS/SSL connection using 'TLS_CHACHA20_POLY1305_SHA256'
Sent GET request!
Server response: 200 OK
Receiving file: 0% - [oooooooooooooooooooooooooooooooo] - 100% (9.84MB)
9.84MB received in 249.72s - 40.35kB/s

[DisasterIncarna]

works perfectly, cheers \o/ So glad you can update UHC Tools and its components so damn quickly, now I can get back to making my GUI work better, with less bugs and with more website support.

[ Show youtube player ]

[patrik]

, I have updated uhc tools with the new aget version now.

[DisasterIncarna]

legend. \o/

[EctoOne]

Hi, would it be possible to extend the installation script so that it can create some sort of portable installation?

We were talking about creating some sort of minimal essentials installer over at the UHC GUI thread and I came up with the idea to also put UHC and the GUI on a single disk which also does not necessarily needs a hard drive installation.

My personal issue is that I want to provide as less (unpacked) software as possible on that disk. And when I was putting some thoughts into it, I figured that it would be enough to just provide lha.run and the UHC installer packages.

Now the problem is that the UHC installer has a fixed path which is SYS:UHC. For my idea, I obviously can't just use that in case someone is using just a plain Workbench disk for example. So it would be great to have a way to extract it in the current dir or maybe set the path as argument, and also put the ENV files (in a subfolder) into the extracted UHC directory. And then instead of adding the lines into the user-startup, it puts it in its own file and obviously adds a line to copy the ENV files into ENV: . Which then can be started from the shell (script) or by using IconX or something similar.

Then I could simply write a script to extract lha, either directly to a hard drive or disk, use lha to extract the UHC installer, and then execute the portable installation script. This would ensure that the files I need to provide are unaltered. And from there I can use AGET to download the UHC GUI and maybe some other stuff

I probably could rewrite the installation script myself, but since it requires to fetch some files on its own, that seems to much work for me.

Hmm, now that I think about it. How about the installer checks if there is already a UHC: assign? I assume that is how the uhcupdate script works (haven't looked at it).

[jayminer]

The path to UHC isn't exactly hardcoded, when you run the installation script we add a check to S:User-Startup that checks for the ENV-variable UHCBIN and if that path exists we assign UHC: there and add paths to UHC:C and UHC:S

However the default path during installation is SYS:UHC and can't be changed until after installation. I guess we could make the installer take an argument for where it all should be installed instead of just defaulting to SYS:UHC since how it's currently working is a bit stupid.

Almost all ENV-variables will be re-created if they are missing, the only important one IIRC is the UHCBIN-variable and that is easy to circumvent to just assign UHC: manually and add the paths manually and then everything would be working.

But all in all, I think you could just manually fetch the uhctools_os3.lha-file from our servers, extract it where you want to, assign UHC: there, add paths to UHC:C and UHC:S and then everything should be up and running.

[EctoOne]

Thanks for the info, I will look into it.
Any recommendations on which server I should use to fetch the uhctools_os3.lha file? I really don't want to add some method of getting the mirrors first.

[patrik]

Originally Posted by EctoOne

Thanks for the info, I will look into it.
Any recommendations on which server I should use to fetch the uhctools_os3.lha file? I really don't want to add some method of getting the mirrors first.

If you base your script on http://uhc.driar.se/uhcinstall and change the destination directory, it would already do the mirror part for you.

[EctoOne]

Originally Posted by patrik

If you base your script on http://uhc.driar.se/uhcinstall and change the destination directory, it would already do the mirror part for you.

Ahh, I actually was trying to base my script on the Installation script that comes with UHCTools.lha but I quickly noticed that it only downloaded the uhcinstall script. And for some reason I didn't look at it.

I just finished a script that points to one of your servers to fetch the uhctools_os3.lha file. I will see if I can insert the mirror part into it.

[DisasterIncarna]

only just noticed on my cleaner/more basic OS3.1 setup that using the morphos readme script is requiring the use of AMISSL, is that normal/unavoidable?

[patrik]

Originally Posted by DisasterIncarna

only just noticed on my cleaner/more basic OS3.1 setup that using the morphos readme script is requiring the use of AMISSL, is that normal/unavoidable?

It is because www.morphos-storage.net force-redirects http to https. I have asked them if they can allow http, but they did not answer.

I used to run a http->https proxy on one of my servers for the access to www.morphos-storage.net, but I removed it recently when I was doing some cleaning of UHC Tools. Nowadays aget can do https:// and all MorphOS computers can run AmiSSL, so it made complete sense I think.

However, please ask the people at www.morphos-storage.net to allow http, I see no point in the force-redirect on this kind of site.

[DisasterIncarna]

Originally Posted by patrik

It is because www.morphos-storage.net force-redirects http to https. I have asked them if they can allow http, but they did not answer.

I used to run a http->https proxy on one of my servers for the access to www.morphos-storage.net, but I removed it recently when I was doing some cleaning of UHC Tools. Nowadays aget can do https:// and all MorphOS computers can run AmiSSL, so it made complete sense I think.

However, please ask the people at www.morphos-storage.net to allow http, I see no point in the force-redirect on this kind of site.

noted, in the meantime i just updated my readme to state that due to redirects https might be an issue and that AMISSL is the cure.

[patrik]

Originally Posted by DisasterIncarna

noted, in the meantime i just updated my readme to state that due to redirects https might be an issue and that AMISSL is the cure.

I have updated the error output of aget to state that what the reason is for attempting to open amisslmaster.library. This instead of just saying that it couldn’t open it.

[DisasterIncarna]

[AMIGASYSTEM]

Hi patrik, UHC Tools installer on AROS x86 does not work nothing is downloaded !

[patrik]

Originally Posted by AMIGASYSTEM

Hi patrik, UHC Tools installer on AROS x86 does not work nothing is downloaded !

Is it UHCTools.lha from aminet, or from one of the mirrors?

Or is it the uhcinstall script from one of the mirrors?

[patrik]

Originally Posted by patrik

Is it UHCTools.lha from aminet, or from one of the mirrors?

Or is it the uhcinstall script from one of the mirrors?

Tested installing using all of them on my AROS installation without issues. The only thing that concerns me is that my AROS installation is very old, so there could be some difference in behaviour.

But you will have to be more specific. For example, the output of the install script would be very helpful.

[AMIGASYSTEM]

Thank you patrik for responding, I attach a comprehensive video peak showing the problem.

https://eab.abime.net/zone/UHC-Tools.zip