Amiga File Server

[enigma776]

Originally Posted by Hache

Thanks for your quick response!! I'm using Filezilla. Regarding to "allow 1 connection", I think this is you are you talking about?

Attachment 75552

Correct, but also in site manager if you have it saved there also as it has its own setting under Transfer Settings

[Hache]

SOLVED!!!

Thank you so much!!!

[Turran]

Experimenting with the web download from https://grandis.nu/turran/FTP.

The download limit of 30 GiB and 500 Files currently does not count towards the following file extensions: jpg gif png css htm html cgi txt

I've now added so that the limit does not count towards the following folders as well.
Pimiga, Retroplay WHDLoad Packs, DamienD's WinUAE Collection

Meaning that if you use https://grandis.nu/turran/FTP, you can grab anything from those folders without the limit header counting up any files or GiB.

This should help but I'm not sure yet if it'll be permanent. If its permanent, it should help to write "WHDLoad games downloaders" programs as they no longer have to deal with FTP.

[reno]

Originally Posted by Turran

Experimenting with the web download from https://grandis.nu/turran/FTP.

This should help but I'm not sure yet if it'll be permanent. If its permanent, it should help to write "WHDLoad games downloaders" programs as they no longer have to deal with FTP.

Hi,

Firstly : thanks for all the hard work, it's an invaluable resource for Amigans !

Have you considered making it available over rsync in addition to ftp/http ?

I think this could make everybody happy :
- Allow people to maintain their local mirrors (which has perfectly legitimate use cases, and is the right thing to do for preservation of such an important resource in the community anyway)
- Keep your bandwidth reasonable low by giving people a way to keep up to date without a full download.

In fact, I was doing just that with the Google Drive mirror : I downloaded the full Retroplay WHDLoad Packs just once a long time ago, then I wrote a script using "rclone" (=rsync for cloud storage) to keep my NAS (and my Amigas, and now A500 mini) in sync with the latest releases.

If you don't want to set rsync up on grandis.nu and are looking for another cloud provider to replace GDrive, it would be really nice if it was one that's supported by rclone (https://rclone.org/#providers)

Cheers

[Turran]

@reno

I did use rsync when we had an official mirror a while back. I'd be happy to do so again if someone wants to set a public mirror up. Just opening up rsync to everyone is not on the table as we have no use for another private copy on someones USB disk in a closet somewhere.

As for rclone, I did switch the backup to JottaCloud (see link in first post). It supports rclone just fine. In fact, its what I use to sync to JottaCloud every night

Not sure what would be needed for you to sync from it as well. I can look that up but probably not this week as its going to get busy for me right now at work.. I have a feeling you already know more about it than I do.

I have no problems if you write a guide for how to download the whole thing from JottaCloud using rclone. I'm sure all the anonymous ftp users would love it so they are not tempbanned from the ftp and website over and over for trying to leech the whole thing, heh.

[reno]

Originally Posted by Turran

@reno

I did use rsync when we had an official mirror a while back. I'd be happy to do so again if someone wants to set a public mirror up. Just opening up rsync to everyone is not on the table as we have no use for another private copy on someones USB disk in a closet somewhere.

I suppose one thing you'd get from that, is to make it much more likely that multiple people will have full, always up to date offsite backups in case disaster strikes on your end, and can come to the rescue. As Linus Torvalds put it : "Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it."

Bandwidth costs aside, is there another reason you wouldn't want to make it *too* easy to download the full packs ?

Originally Posted by Turran

As for rclone, I did switch the backup to JottaCloud (see link in first post). It supports rclone just fine. In fact, its what I use to sync to JottaCloud every night

Yes I saw that, but also that you were looking for another solution because of the 1000 files limitation ?

Besides, it looks like rclone doesn't support anonymous syncing from Jottacloud. It was the same with GDrive, but nearly everyone already has a Google account. I don't know if people would be able to create a Jottacloud account for free then access a shared folder like you could do with GDrive ?

I have no problems if you write a guide for how to download the whole thing from JottaCloud using rclone. I'm sure all the anonymous ftp users would love it so they are not tempbanned from the ftp and website over and over for trying to leech the whole thing, heh.

If you're going to settle on JottaCloud and we find a way to solve the accounts/shared folder issue, sure !

Right now, I think the http plugin for rclone is actually a better solution. I did manage to update my Retroplay WHDLoad packs from your website using it. Definitely slower than GDrive but it was just 30 files or so since I last synced, so no big deal. I only had an issue with this demo : the "å" in the filename didn't get transcribed properly after downloading (it turned into "?E5"), and that crashed the HstWB "build_install_entries" Python script that I run in the end. I think the root cause is that your Apache server uses charset=ISO-8859-1 for text/html, instead of UTF-8.

[Turran]

Originally Posted by reno

Bandwidth costs aside, is there another reason you wouldn't want to make it *too* easy to download the full packs ?

Well, yes, I have a reason.

EDIT: Just to be clear, I have no problem with anyone wanting to download a whole "pack" like the WHDLoad games or DamienDs pack. That is why I made it free on the website. I'm talking about the whole FTP here.

There are simply to many people that wants to download the whole thing without having any intention of sharing it. In case of a disaster, I have double backups here and also the JottaCloud. We don't need any more really. If someone said they want to host a backup, I'd be happy to oblige.


But, as it stands right now, the bandwidth is increasing by a lot day by day. More and more anonymous users try to download the whole FTP for no other reason than "they want it", I guess. Sure, there might be people who are serious about downloading it all for preservation purposes, but those guys can do it the "right" way. Show you have something to give to community by making 50 useful posts on the EAB forums and create your own account. Then you can download the whole thing.


To prove the point, since 2018-05-17, when I enabled logging, we have 218TiB downloaded. That is the entire FTP downloaded over 54 times. How many of those "backups" can you access today? None I bet. How many would come to the rescue with "I have it all backed up and will make it public" if I crash? No idea. =)
For preservation purposes, as you say, should everything crash here, we should already have 50 full backups out there, but nobody knows where.


There are simply to many people thinking the same way and I would have 10-20 anonymous users online 24/7 leeching, had it not been for the temporary bans I throw out for trying exactly this.


As for the downloads increasing by a lot:
In 2021, the total download was 52.11 TiB. Thats the entire FTP 13 times.
In 2022, SO FAR, we have 80 TiB downloaded. Entire FTP 20 times and we're not even halfway through 2022.



I am not hosting the FTP for people to download the whole thing. There are official ways to do that if you feel the need to do so (contact me for a public mirror or register an upload account once you pass the requirements).


I want the files to be available to everyone. I don't want the "Server is full" message when someone wants a WHDLoad game or similar. That is what will happen if I open the flood gates and was actually happening a lot before I implemented the limits on the anonymous account a while back.

Originally Posted by reno

Yes I saw that, but also that you were looking for another solution because of the 1000 files limitation ?

Yeah, but I've yet to find a better one. JottaCloud have been great in responding to my requests and I got one bug fixed in a couple of days after reporting it. Its in their development pipeline to have a look at the 1000 files limit, so I'm trusting that will be fixed some day.

[XPD]

Can I get my password reset please ?

Thanks!

[XPD]

If I had the storage space, I'd offer to host a mirror - but at the moment, I'm almost outta space.... need to sell another kidney to get another drive to add to the server

[Turran]

Reset and PM sent. Sorry it took a while. This week have been a very busy one so no time to even check messages =(

Originally Posted by XPD

Can I get my password reset please ?

Thanks!

[XPD]

No prob, appreciated!

[andy2004]

on the FTP: under the upload folder of scize. there are 4 zip archives. amiga 1 2 3 and 4. all large.. theres no txt file as to whats inside them..
any one know?..

[Crashdisk]

Originally Posted by andy2004

on the FTP: under the upload folder of scize. there are 4 zip archives. amiga 1 2 3 and 4. all large.. theres no txt file as to whats inside them..
any one know?..

Try here...

[chip]

Using the turran search make me appear the following message:

"Visiting an untrustworthy website has been prevented"

I can't reach the search logically

This is due to antivirus Kaspersky i installed on my system

Anybody know why this message comes up ?

[Turran]

Originally Posted by chip

Using the turran search make me appear the following message:

"Visiting an untrustworthy website has been prevented"

I can't reach the search logically

This is due to antivirus Kaspersky i installed on my system

Anybody know why this message comes up ?

Seems my certificate expired. I'll fix it.

Update: Done. Try now.

[chip]

Yep, that's the reason then ..... thanks

[chip]

And now it's fixed ..... nice

[Turran]

Originally Posted by chip

And now it's fixed ..... nice

Great, thanks for the report


And I must say its brave of you to install a Russian antivirus at these troubling times. =)

[zx80]

Hello,

pretty new to EAB and still slightly confused in whats the best place/way to get the latest WHDLoad package with Amiga games, lets say for AGS/OGS or specific languages only? Have seen a tool, which allows to filter/select games for certain criteria here a while back but can’t find it at the moment. I am also confused about the background processes and/or maintainer of all those game packs available in the wild and if and how someone may contribute to.

Maybe someone can enlight me on that topic or share some links to read for clarification.

Cheers

[enigma776]

Originally Posted by zx80

Hello,

pretty new to EAB and still slightly confused in whats the best place/way to get the latest WHDLoad package with Amiga games, lets say for AGS/OGS or specific languages only? Have seen a tool, which allows to filter/select games for certain criteria here a while back but can’t find it at the moment. I am also confused about the background processes and/or maintainer of all those game packs available in the wild and if and how someone may contribute to.

Maybe someone can enlight me on that topic or share some links to read for clarification.

Cheers

Probably the wrong place for it but the tool is https://github.com/MrV2K/WHDLoad-Download-Tool

The maintainer is Retroplay I believe. Thread http://eab.abime.net/showthread.php?t=61028