Amiga File Server

[DamienD]

Originally Posted by Turran

We should remove it all together and tell people to grab Filezilla or something instead.

And we should change it to reflect that all members (with 15 posts, last one no older than 5 days) are encouraged to register their own accounts instead.
Right now it only says to create an "upload account".

The post linked in the first post to create an upload account was updated yesterday, but I can't update the first post =)
http://eab.abime.net/showpost.php?p=...&postcount=780

...I'm out at the moment, will update the first post later

[DamienD]

Ok first post has now been updated... I may have forgotten stuff so...

@Turran, please have a read and let me know if there is any additional information required

[Phantom]

Thanks for the additional info, really helped.

[Turran]

Originally Posted by DamienD

Ok first post has now been updated... I may have forgotten stuff so...

@Turran, please have a read and let me know if there is any additional information required

Looks good! One clarification though
"Once your account is created, you need to login / use this within 7 days otherwise it will be disabled."

Once your account is created, you need to login / use it within 7 days. This is to verify your account and you only have to login once, not every 7 days.

Thanks!

[DannyBoy]

And since everybody just loves grammar police:

To be eligable for..
To be eligible for..

[DamienD]

Originally Posted by Turran

Looks good! One clarification though
"Once your account is created, you need to login / use this within 7 days otherwise it will be disabled."

Once your account is created, you need to login / use it within 7 days. This is to verify your account and you only have to login once, not every 7 days.

Thanks!

Cheers, done

[Turran]

Originally Posted by DannyBoy

And since everybody just loves grammar police:

To be eligable for..
To be eligible for..

Hihiih. Changed on 4 occasions on the webpage.

[Turran]

Originally Posted by DamienD

Hmmm, interesting Turran

Someone seems to be downloading the "SNES_DVD.ISO" that I created and uploaded under "~Uploads/DamienD/SNES DVD":

Still going strong... 2 people downloading it currently, and not the same. Heh

[Turran]

Added two more buttons to eabsearch. "Show top downloaders" and "Show top uploaders".

This is only for registered users and does not include the "ftp" account.

https://grandis.nu:444/eabsearch/

[DamienD]

That can be right Turran?

For example, if I look at "Show Top Uploaders" I'm not even in the list:

                        TOP UPLOADERS FOR ALL TIME
.-=-------------------------------------------------------------------=-.
| ## User         Location/Tagline         Files  Megabytes   Avg k/s   |
|-=-------------------------------------------------------------------=-|
     <snip>
[22] Aladin       No Tagline Set           1,545    3,912MB   1739K/s
[23] Codetapper   No Tagline Set              10    3,150MB    131K/s
[24] dirkies      No Tagline Set              67    2,926MB   2717K/s
[25] leeuw        No Tagline Set               4    2,629MB    188K/s
[26] Crashdisk    No Tagline Set              60    2,397MB    110K/s
[27] jarre        No Tagline Set               5    2,072MB    608K/s
[28] Sea7         No Tagline Set              17    2,003MB     47K/s
[29] Peter        No Tagline Set               3    1,868MB     58K/s
[30] CodyJarrett  No Tagline Set               2    1,568MB   2920K/s
`-=-------------------------------------------------------------------=-'

Under my ~Uploads directory, even if you just look at the LHAs and HDFs folders, these are each over 2GB...

So I should at least be above [22]? My whole directory must be over 8GB...

Do I not show as I'm an "EAB File Server" admin?

Anyway, it's only stats, so not a big deal really...

[Turran]

Originally Posted by DamienD

That can be right Turran?

It only showed members from the eabuploader and you are eabadmin =)
It also only shows stats since I took over hosting the FTP. If you had an upload folder before that time, the stats are not counted.

I've added you to eabuploader as well (should have no ill effects) so you'll show up now as well.

Oh, and I'll explain the tagline thing. "No Tagline Set" means its an account that was created before the check towards the EAB Forums userbase. If you have a tagline, that is your actual username on the EAB Forums (verified).

[DamienD]

Originally Posted by Turran

It only showed members from the eabuploader and you are eabadmin =)
It also only shows stats since I took over hosting the FTP. If you had an upload folder before that time, the stats are not counted.

I've added you to eabuploader as well (should have no ill effects) so you'll show up now as well.

That's fixed it, cheers

[Cylon]

Originally Posted by Turran

I honestly thought people would be happy to register an account and use that instead to get rid of the 500k/sec limit, but I guess I was wrong. Either you are to lazy to do so or you do not see the benefits from it. I'm simply trying to find a way to save the bandwidth for the EAB members instead of all the people who found the link to the FTP somewhere else and have no interest in the EAB community.

If I am wrong in all this, please let me know. I'm happy to put everything back the way it was, slowing the whole thing down when a lot of people is downloading.

I honestly don't know what your real motivation is behind this effort. You're granting 50GB/week to anyone registered and at the same time reducing the dl speed for every anonymous users (which are not anonymous to you, anyway) to 50kB/sec. Well, downloading 50 gigs is almost the definition of leeching to me, an anon login would need aproximatly 12 continuous days of uninterupted download to get this amount of data (roughly 1 day with 500kB/s). Why not limit the amount per IP address instead of the speed? you would get rid of leechers ('anon' leechers, alas) pretty fast.
Or, is it that you just want to know who is downloading which files? And then publish this information for some reason?
I'm not too lazy to create an account for a useful service, but this FTP seems to be at the home of EAB, or at least it is closely connected to it. Therefore i donot see the need to ask for another login.
If you don't want non-members to download why allowing these in the first place?
What amount of data is an 'anonymous' login usually downloading per average? More than these 50gigs?

[DamienD]

Wow, nice post there Cylon!!! Talk about gratitude towards Turran for:

a) Hosting at his own cost.
b) Providing a solution to maximise speed for EAB members.
c) Still providing alternatives for non EAB members.

Originally Posted by Cylon

I'm not too lazy to create an account for a useful service, but this FTP seems to be at the home of EAB, or at least it is closely connected to it. Therefore i donot see the need to ask for another login.

You have sooooo many other options if you cannot be bothered to create an account:

Originally Posted by Magix

Non-EAB members:

If you are not a registered EAB member then there are a few options available to you; and of course, access is still given:

a) You can continue to use the main "EAB File Server" but will be subject to 50kb/sec maximum download speeds + only 20 logins are allowed at any one time.
b) You can use the newly set up GoogleDrive instead.
c) You can use the mirror FTP instead as detailed above (but please note that this is only synced once daily with the main FTP).

...it's not like it's even difficult to register, is virtually automated and takes less than 1 minute

Originally Posted by Magix

Turran and modrobert have made this very simple... Just click here and fill in the relevant details.

[Cylon]

@DamienD
This is all you end up finding in my post?:

Edit
Please stop editing your post. Cannot keepup if QFT would be needed.

[DamienD]

Tell me, what is so important that I missed??? I got the gist of it

All is see is you being ungrateful / complaining and asking Turran multiple questions.

Ok, let's wait for Turran's view on your post then...

Originally Posted by Cylon

Edit
Please stop editing your post. Cannot keepup if QFT would be needed.

...which you've done after my response to your original post

Anyway, so now you're going to tell a GM what he can / cannot do. Not very smart, keep going...

[Cylon]

I wasn't complaining. Your are bitchin', in my opinion.

[DamienD]

Originally Posted by Cylon

Your are bitchin', in my opinion.

Who's bitchin'??? Please re-read your post directed at Turran.

...someone's going on vacation, guess who?

[DH]

Originally Posted by Cylon

I wasn't complaining. Your are bitchin', in my opinion.

And you're ungrateful for the service that's being offered Cylon, if you don't like it or think there's room to moan and complain about this completely free service offered, maybe we should implement restrictions on ip addresses too so we can eliminate those who like to stir the pot

Sent from my PRA-LX1 using Tapatalk

[Turran]

Well, I see the banhammer was struck, but I'll respond anyway since I do have reasons for all you've posted.

Originally Posted by Cylon

I honestly don't know what your real motivation is behind this effort. You're granting 50GB/week to anyone registered and at the same time reducing the dl speed for every anonymous users (which are not anonymous to you, anyway) to 50kB/sec. Well, downloading 50 gigs is almost the definition of leeching to me

The motivation for all this started a few weeks ago when complains started coming in that upload and download speeds were horrific when there was 15-20 users online. The suspicion was that a lot of the anonymous users that was leeching almost 24/7 was not members of the EAB Forum/community since the link to the FTP was widespread on other sites as well.

To make a long story short, I found 2 issued that caused the slow downs (A RAID array and an old firewall rule).
This gave everyone full speeds as much as the link and hardware could supply.

Another problem then crept up. It was now using almost all bandwidth, or at least a large chunk of it as we could have as many as 40 anonymous FTP users and 60 registered accounts online at the same time (in reality, its only about 2-3 registered accounts online at a time, but 40 anonymous users were not unheard of.) The problem for you with slowdowns was, in reality, not fixed because it was STILL slow for you, only now it became a problem for me, using to many resources than I was willing to spare.

In short, fixing the bottleneck led the FTP to use too much bandwidth. Sure, I have 1 GB up and down, but this is a consumer ISP I'm using and that much bandwidth 24/7 would have raised suspicions and I did not accept that.

Since the original poll raised was about a lot of outsiders using the FTP that had nothing to with the EAB community, we essentially was too generous with the material on the FTP to people that had no "right" to it, or did not care about the community. And yes, everyone has a "right" to it. Don't get me wrong. But I'm hosting this FTP for the EAB members. Its EAB Members that contribute it to. That is what I mean by "right".

I then changed the anonymous FTP account to 500k/sec download speed limit.

The problem was also that the anonymous users often stayed on for days, leeching the whole 1.2TB for no reason. I wanted to stop that from happening.

So how do you stop people from leeching the whole FTP and at the same time providing a better service for the EAB community?

I did not want to start a shit storm, so I registered (and pay for) an unlimited google drive account and synced the whole 1.2TB to it (took days since the software is crap and I had to redo it multiple times). That, I thought, would allow the anonymous users who has no interest in the EAB community to grab the whole thing still and everyone would be happy.

After doing that, I had the option to either delete the anonymous FTP account and force everyone to register their own account. You see, with glFTPd, the FTP server I use, the anonymous FTP user was flagged as "anonymous", giving it some restrictions. There are a very few things I can do. For example, I can not restrict how much they can download per week as I can with a registered account, so I opted to limit the account to 20Kb/sec at first.

I thought I would keep it like that for a few days so that people would understand that it was worth it to register an account instead, but then opted to raise it to 50k/Sec instead. I also lowered the max logins for the anonymous FTP account from 40 to 20 and also changed num logins per IP from 2 to 1, so you could only log in once per IP, if anonymous.

I had now stopped the "leech the whole site with an anonymous account" from the FTP (I thought). You still can from the google drive though.

Originally Posted by Cylon

Well, downloading 50 gigs is almost the definition of leeching to me

I have no problem with EAB forum members leeching the while site really. Well, not as much as anonymous users anyway. But I still would like to discourage it for registered accounts as well. Limiting it to 50GB/week still allows you to leech the whole site, but it would take you 24 weeks to do so (1200 GB / 50 GB = 24 weeks) so that is rather discouraged.

Originally Posted by Cylon

Why not limit the amount per IP address instead of the speed? you would get rid of leechers ('anon' leechers, alas) pretty fast.

There is no way to do that with glFTPd when the FTP user is flagged as anonymous, which it has to be for multiple users to be able to use it without abusing it. For example, with a "normal" account, you can log in with a ! infront of your name to kick out any other sessions you have logged in. If we do not flag the anonymous FTP account as anonymous, anyone could login with "!ftp" and kick everyone else out. That is just one way to mess it up for others.

Originally Posted by Cylon

Or, is it that you just want to know who is downloading which files? And then publish this information for some reason?

No, but its nice to gets stats, which I do not get from the anonymous FTP account since its flagged as anonymous. Stats are not saved then in the same way so.. It allows for fun lists of top uploaders / downloaders, which is always fun to look at.
Example:
https://grandis.nu:444/eabsearch/trafficdn.php
https://grandis.nu:444/eabsearch/trafficup.php

That is not a name and shame list. Its simply facts and quite fun to look at.

Originally Posted by Cylon

I'm not too lazy to create an account for a useful service, but this FTP seems to be at the home of EAB, or at least it is closely connected to it. Therefore i donot see the need to ask for another login.

Exactly. Its the home of EAB, so why allow anyone to download from it that is not connected or have any interest in the EAB community. It was not a problem allowing anyone to grab it all before I fixed the speed issues. Once I fixed that, it became a problem.
Before fixing the speed issues, it was an issue for the users that it got slow as hell. After fixing the speed issues, it became an issue for ME.

Originally Posted by Cylon

If you don't want non-members to download why allowing these in the first place?

It had always been that way, that anyone could doiwnload from it, when I took over hosting the FTP. It just followed along. At the same time, I do not want to cut everyone outside EAB off entirely. Thus the google drive and leaving the anonymous FTP account with a 50kb/sec limit. You can still grab a few files here and there without problems, but trying to leech the whole site is very inconvenient now.

Not that this has stopped them from trying...

.----------------------------------------------------------------------------.
| Username | PID | OnLine | Action |
|----------+-------+----------+----------------------------------------------|
| ftp | 13381 | 04:24 | "RETR All-American Basketball (1992)(Zeppeli |
| ftp | 2153 | 01:09:04 | Dn: Magazyn Amiga CD0 at 45.5K/sec |
| NLS | 7632 | 05:04 | "RETR New Intro Stuff (1992-09-12)(Hardline) |
| ftp | 3361 | 08:21:45 | Dn: Amiga_Museum_DVD_ at 50.0K/sec |
| ftp | 26625 | 32:06:26 | Dn: Magazyn Amiga CD0 at 50.0K/sec |
| ftp | 5148 | 04:41:02 | Dn: SNES_DVD.ISO at 49.9K/sec |
| ftp | 13405 | 00:03 | Dn: /Commodore_Amiga/ at 57.0K/sec |
| ftp | 19452 | 27:34:02 | Dn: Magazyn Amiga CD0 at 50.0K/sec |
| ftp | 9625 | 02:10:36 | Dn: BlakeStoneAliens- at 50.0K/sec |
| ftp | 15151 | 22:13:12 | Dn: Gateway! v3 (Scha at 50.0K/sec |
| ftp | 428 | 01:50 | Idle: 00:46 |
| ftp | 2945 | 01:31 | Dn: /Commodore_Amiga/ at 50.1K/sec |
| ftp | 20211 | 06:47:41 | Dn: Aminet CDROM (199 at 50.0K/sec |
| ftp | 5440 | 20:51 | Dn: /Commodore_Amiga/ at 50.0K/sec |
| ftp | 20764 | 01:45:39 | Dn: Protracker V2.3d at 50.0K/sec |
| ftp | 14462 | 13:35:39 | Dn: Magazyn Amiga CD0 at 50.0K/sec |
| ftp | 24725 | 02:47 | "PASV" |
| ftp | 23783 | 02:22:33 | Dn: Flash Rom Vol. 1 at 49.9K/sec |
| ftp | 20374 | 47:46 | Dn: /Commodore_Amiga/ at 47.8K/sec |
| DamienD | 32057 | 16:25 | Dn: TOGA 2009-09 (200 at 4636.4K/sec |
`----------------------------------------------------------------------------'

Sure, I can still disable the ftp account and only allow registered users on. If you're anonymous, use the google drive or mirror FTP. I chose not to since it still works fine if you just want a file or two every now and again.

Originally Posted by Cylon

What amount of data is an 'anonymous' login usually downloading per average? More than these 50gigs?

Hard to say. I disabled the traffic logs in the firewall and erased the history when troubleshooting the speed issues that started all this. I can not say "on average", but I can say that on average, 3-4 ftp accounts are online 24/7 from the same IPs, leeching the whole 1.2 TB. Just look above from the online users. That they are limited to 50k/sec has not stopped them from trying and it did look the same before when there was no limits on them so.. I'd say a LOT more that 50gigs on average with certainty though.

I can give more specific traffic reports in the future for those interested, as traffic logging is now on again. Its now 17:03 and we are at 227.4G out so far today.

Anyway, this turned out to be a novel.