EAB Mirror

Hi Everyone,

I realise you probably don't really need it, but I thought I'd make a mirror of the EAB file server.

This should be relatively easy to do it, but as some of the files have invalid and/or incomplete multibyte or wide characters in the filenames some of the transfers are failing.

Transferring file `PoWeR/PoWeR_15/Bonus/PÚlda.src.lzx'
mirror: /Commodore_Amiga/Collection/Coverdisk/PoWeR/PoWeR_15/Bonus/PÚlda.src.lzx: Invalid or incomplete multibyte or wide character

I was hoping to use lftp on debian to easily mirror the file server once every 3-4 days, I don't think it changes that often.

Thanks,

[DannyBoy]

I dunno if this is such a hot idea. EAB files are available at three different places now. It's getting silly

Originally Posted by DannyBoy

I dunno if this is such a hot idea. EAB files are available at three different places now. It's getting silly

Sure I realise that hence why I said you probably don't really need it, but if I did still want to make a mirror do you have any recommendations as to how to keep it up to date?

How do you keep your mirror up to date? I noticed you use vsftp, whilst the official one uses glFTPd.

[DannyBoy]

Originally Posted by AeroMaxx

Sure I realise that hence why I said you probably don't really need it, but if I did still want to make a mirror do you have any recommendations as to how to keep it up to date?

How do you keep your mirror up to date? I noticed you use vsftp, whilst the official one uses glFTPd.

Main FTP syncs daily to mirror with rsync. You can do this with lftp too, but I don't think it's the tool for the job.

I think you should reconsider this. Have a look at the main thread (last few pages) as there is some discussion about the problems that leeching the whole filetree is causing.

Originally Posted by DannyBoy

I think you should reconsider this. Have a look at the main thread (last few pages) as there is some discussion about the problems that leeching the whole filetree is causing.

Thanks for the information, but I will go and have a read of that thread to see what you mean.

I actually already have about 80% of the FTP, but the FTP isn't as tidy as i'd like it to be personally.

Originally Posted by DannyBoy

Main FTP syncs daily to mirror with rsync. You can do this with lftp too, but I don't think it's the tool for the job.

I didn't think rsync worked with ftp, but I checked and read the man pages and it seems I am correct, so unless your using sftp instead somehow I can't see how rsync is working for you.

as previously mentioned lftp doesn't work fully due to the multibyte characters in some filenames.

[DannyBoy]

Originally Posted by AeroMaxx

I didn't think rsync worked with ftp, but I checked and read the man pages and it seems I am correct, so unless your using sftp instead somehow I can't see how rsync is working for you.

as previously mentioned lftp doesn't work fully due to the multibyte characters in some filenames.

Ugh.. They are two different things. rsync has been working very well for us like four years. Feel free to grab files you actually want, but I recommend dropping the mirroring plans.

[ptyerman]

Have you even asked for permission to mirror it? You can't just jump in and leech the whole ftp, people have done this and it's caused untold problems, this is the reason for the new restrictions!

Originally Posted by DannyBoy

Ugh.. They are two different things. rsync has been working very well for us like four years. Feel free to grab files you actually want, but I recommend dropping the mirroring plans.

What are two different things?

Anyway I tried rsync with a folder that I know has multibyte and/or wide characters and as such fails to transfer using lftp, but the connection just times out.

rsync --recursive --times --compress --verbose --progress ftp://ftp@grandis.nu:/Commodore_Amig..._1_REPULSE.ori "/home/ftp/Commodore_Amiga/App/Archive/SFX_4_1_REPULSE.ori/";

[Turran]

You can not use rsync. Me and DannyBoy have a rsync relationship set up so I can run rsync to the mirror from the main FTP. That is done every night and is not something you can do yourself. Both parties needs to be involved and it has nothing to do with FTP.

As DannyBoy said, we do not really need a 3rd mirror right now. We already have 2 and a google drive backup open to the public (also synced from me every night). Appreciate the thought though. Opening one anyway will only get you alienated from EAB. Especially if you plan to resort it (although I agree that it needs a big cleanup).

I see you registered today, both to the forums and the FTP. I have no problems with the 15GB you have leeched already today, but be advised that you are limited to 50GB/Week so you won't get it all in one go.

Regarding the files that you fail to download, I'm guessing the filesystem you are downloading to can not handle it, while ours can. Or its your client that can not handle it. Not much we can do there for you I'm afraid.

[DannyBoy]

Originally Posted by AeroMaxx

What are two different things?

Turran explained this throghly, but I'll have a word on this yet. FTP is for downloading stuff (and sometimes uploading as with the primary server). Rsync is for synchronizing both servers. Rsync does not speak FTP, you need a service for it. Read the manpage again

I agree with you on some things though. Directory hierarchy could be more tidy, and filenames ought to be plain ascii for sure

[DamienD]

@AeroMaxx; how many times do you need to be told from the 2 people hosting both the main FTP and the mirror (for many years now without issue) that we don't need another backup (if that's really the main reason why you're grabbing everything anyway, which I doubt)???

Seriously; please stop your leeching of our resource otherwise action will be taken; I guarantee it!!!

[nzlpoa]

Originally Posted by DannyBoy

I dunno if this is such a hot idea. EAB files are available at three different places now. It's getting silly

and two places isn't ?

Three (if u count Google drive)

Just saying

Originally Posted by DamienD

@AeroMaxx; how many times do you need to be told from the 2 people hosting both the main FTP and the mirror (for many years now without issue) that we don't need another backup (if that's really the main reason why you're grabbing everything anyway, which I doubt)???

Seriously; please stop your leeching of our resource otherwise action will be taken; I guarantee it!!!

Google drive is there. so really the main problem i see is no-one likes Google own limitations as the main reason for this.

[DamienD]

Originally Posted by nzlpoa

Google drive is there. so really the main problem i see is no-one likes Google own limitations as the main reason for this.

Well then, do the following:

... Join up to EAB (which you have already done).
... Be part of the EAB community and at least create 15 posts (yeah I know, very taxing indeed, I do double that amount in one day).
... Sign up here afterwards to gain proper access to the "EAB File Server".

If the above is all too much for you to handle then alternatively:

a) Use the mirror.
b) Use the GoogleDrive.
c) Use main FTP (but with limitations).

...I mean; seriously though... you've only just joined EAB today + made 4 posts; yet you feel you have the right to complain???

F$%k me, what's this world coming to???

[DamienD]

@Turran; been thinking after this new member nzlpoa's crazy post above my two in this thread...

15 posts is way too easy for any old leechers to achieve and then have full access / speed at the cost of "real" EAB members

Could you please increase the limit on registering new accounts to 100 posts (which is really still peanuts for anyone seriously wanting to join our community and contribute in a meaningful manner)?

Many thanks in advance my friend

Once done; I'll update all notifications in the relevant threads

[Turran]

You da boss =)

Done

[DamienD]

Cheers Turran; have now updated the first post in the EAB File Server thread