English Amiga Board


Go Back   English Amiga Board > Main > Amiga scene

 
 
Thread Tools
Old 13 May 2009, 22:42   #141
hippie2000
Registered User

 
Join Date: Oct 2007
Location: Germany
Posts: 44
Exclamation omg omg omg

hello,

bad news - our main server crashed (likely by some weather based power peak) in a very evil way - worst case scenario - raid 5 array has one disk completely destroyed and a second out of sync - so it may be a complete data loss of the main site and ftp

until this issue is resolved the main site will redirect to the team server at
http://team.back2roots.org (please be kind to it, it's not that strong)

and the ftp server is gone for now (2mio++ files, ouch)

all hosted sites including http://winuae.net are not affected any more and should work as expected.

you may experience occasional down times as we work on this.

theres a backup i made of most stuff (~500GB) over winter so worst case would be a half year upload and a complete reinstall of the main server...

now the good part - any crisis we survived improved things horribly - and there have been many

regarding news - well sorry i havent posted news but just uploaded and uploaded (about 50.000 new files over winter and about 100.000++ virus scan protocols - i heated my flat with uae)

i promise to post news again after that issue is addressed

and of course the archive will remain if at all possible (including recrowding the ftp) - it was too much work to just disappear in time...

and last but by far not least - big thanks to warlock from nvg.org - without his local hands such issues would not be addressable at all. a pleasureful non stressy work with him, even in crisis times - hippie approved (tm)...

Last edited by hippie2000; 14 May 2009 at 03:55. Reason: fixed all http accesses - and spelchacking ;-)
hippie2000 is offline  
AdSense AdSense  
Old 13 May 2009, 23:56   #142
DH
Global Moderator

DH's Avatar
 
Join Date: Sep 2008
Location: In a Pub
Age: 50
Posts: 3,975
OMG, I'm really sorry to here that

I'm glad not all is lost, except time, maybe you should think about backing up the backup server too
DH is offline  
Old 14 May 2009, 02:10   #143
MethodGit
Junior Member
MethodGit's Avatar
 
Join Date: Dec 2002
Location: The Streets
Age: 33
Posts: 2,723
Damn, that's horrible! =[ Is the FTP server down for good? I'm just wondering where else the TOGA, Fred Fish etc collections could be made available for the time being. Possibly Magix's server? Of course, someone would have had to have backed up all the collections before they could upload anything....
MethodGit is offline  
Old 14 May 2009, 02:26   #144
prowler
Global Moderator

prowler's Avatar
 
Join Date: Aug 2008
Location: Sidcup, England
Posts: 10,278
Quote:
Originally Posted by MethodGit View Post
Damn, that's horrible! =[ Is the FTP server down for good? I'm just wondering where else the TOGA, Fred Fish etc collections could be made available for the time being.
http://fish.back2roots.dyndns.org
prowler is offline  
Old 14 May 2009, 03:10   #145
hippie2000
Registered User

 
Join Date: Oct 2007
Location: Germany
Posts: 44
hehe,

DemonHellraiser - guess what i do since today morning (90.000 files - 18GB so far - fetched from team server

MethodGit - ive backed up about everything (glad i did in winter) - so if i can help you (or anyone else) with urgent stuff - abime irc is the fastest way to reach me. and no - it will return if i' stay healthy to do so. (the unique license for all of fred's commercial work is not a ego luxus but a public duty imho)

prowler - nice you remember the fallback - but i just reactivated http://fish.back2roots.org - however, the http and ftp links are dead - but at least the iso links work for now.

magix ftp - nono - that would be useless redundancy - ive not seen much overlapping (tnx god - the space is needed for the remainder stuff

Last edited by hippie2000; 14 May 2009 at 03:23.
hippie2000 is offline  
Old 14 May 2009, 03:14   #146
MethodGit
Junior Member
MethodGit's Avatar
 
Join Date: Dec 2002
Location: The Streets
Age: 33
Posts: 2,723
Quote:
Originally Posted by prowler View Post
The actual disks are still hosted on the now-down FTP server, dude.

And thanks hippie - if I have a disk in my mind I'm interested in I'll let you know.
MethodGit is offline  
Old 14 May 2009, 03:18   #147
prowler
Global Moderator

prowler's Avatar
 
Join Date: Aug 2008
Location: Sidcup, England
Posts: 10,278
Quote:
Originally Posted by MethodGit View Post
The actual disks are still hosted on the now-down FTP server, dude.
Quote:
Originally Posted by hippie2000 View Post
hehe, prowler - nice you remember the fallback - but i just reactivated http://fish.back2roots.org - but te ftp links are dead - but at least the iso links work.
prowler is offline  
Old 14 May 2009, 13:40   #148
cosmicfrog
The 1 who ribbits
cosmicfrog's Avatar
 
Join Date: Apr 2006
Location: leek, Staffs, UK
Age: 50
Posts: 3,555
Send a message via MSN to cosmicfrog
tar for the update hippie I`m sure you will get it sorted in time

I`m also glad to hear your comments on fred, may his fishy name never be forgotten
cosmicfrog is offline  
Old 14 May 2009, 15:28   #149
MethodGit
Junior Member
MethodGit's Avatar
 
Join Date: Dec 2002
Location: The Streets
Age: 33
Posts: 2,723
Just remembered - this place still works for fishy goodness.
MethodGit is offline  
Old 14 May 2009, 17:36   #150
DH
Global Moderator

DH's Avatar
 
Join Date: Sep 2008
Location: In a Pub
Age: 50
Posts: 3,975
Quote:
Originally Posted by hippie2000 View Post
hehe,

DemonHellraiser - guess what i do since today morning (90.000 files - 18GB so far - fetched from team server
Wow! Not much left to do then

Seriously, Good work, and I hope everything runs very, very smoothly for yourself and anyone else who's involved
DH is offline  
Old 14 May 2009, 18:20   #151
BuZz
Registered User
 
Join Date: May 2002
Location: Oxford / UK
Age: 41
Posts: 582
Send a message via ICQ to BuZz
Sorry to hear about your crash. I hope you get everything restored. Running an important online archive, occasional backups is not enough however. Running an offsite rsync backup is relatively easy to setup, and is a must have for such things. good luck with the restoration.
BuZz is offline  
Old 15 May 2009, 04:10   #152
hippie2000
Registered User

 
Join Date: Oct 2007
Location: Germany
Posts: 44
DemonHellraiser - thanks for the nice words

Buzz - of course i know rsync and backups and im doing what i can. that would be easy for a 1GB site - but it took two years to at least get the storage space and hardware to run the archive at all (with 250 people's paypal contribution). I run hourly crons which sync the main site to a redundant server (team, from where its synced to me so triple redundancy), but for the ftp i plain dont have the resources - so only occasional backups.

Remember back2roots was coded without a computer or a line! I used a demo terminal of the german telekom and to make this possible the engine of the main site (code and data) still fits on a single floppy disk. The files and screenshots of the main site were uploaded by bobic and others of the team (and later synced to me via 56k modem). Hundreds of people have uploaded the rest. The huge ftp was created by me - all own content came from a local copy and took ages to upload.

I'll have to repeat a paypal collection for new disks - and if it yields enough response both servers may receive same (double) space for full redundancy. If you know some terabytes of safe backup space with a good line let me know. rsync is a oneliner, np - but bandwidth and storage space is not included in its command...

Well let's see - we got some recreation tool from the controller manufacturer - the disks are smashed for sure (one is said to sound like marbles in it and is physically void) but maybe a part could be restored. Of course we could not buy industry grade equipment and with increasing bandwidth and "multithreading hammering tools" a 1mio downloads per week archive is very stressy for the disks.

Thanks god i did a full virusscan this winter (which required a resync of my copy)....
hippie2000 is offline  
Old 15 May 2009, 04:45   #153
BuZz
Registered User
 
Join Date: May 2002
Location: Oxford / UK
Age: 41
Posts: 582
Send a message via ICQ to BuZz
well i just hope you get the data restored. disks are cheap these days, and having an offsite backup is really needed with such large projects. having only ftp access isnt great. It wasnt possible to have some shell access to to arrange some automated backups ?

I recommend rdiff-backup. rsync + diffs. you can have a up to date rsync backup, plus so many days of diffs for rolling back. I use this for most of my servers, that backup to a local raid 5 array (offsite).
BuZz is offline  
Old 15 May 2009, 18:52   #154
MethodGit
Junior Member
MethodGit's Avatar
 
Join Date: Dec 2002
Location: The Streets
Age: 33
Posts: 2,723
Ooooh another thing...... the 17bit CDs are still available. (Although they're missing the last two sets.)
MethodGit is offline  
Old 16 May 2009, 07:10   #155
hippie2000
Registered User

 
Join Date: Oct 2007
Location: Germany
Posts: 44
Smile

hello buzz - thanks for the hint but of course i know and use rsync, and agree its a pretty good solution, but it doesnt finance storage space. and of course i do not only have shell access, these are 2 dedicated back2roots servers and without being root there these wouldnt serve much. im not a newbee just the costs with time are huge.

redundancy is one thing - but all content of the ftp labeled back2roots comes from here so the backup existed BEFORE the online presence. no point in repeating that.

prob is backup without fat line in such domensions (as i have) is worth nothing and a pain in the *** to keep in sync with payable fast (16/1mbit) home lines. upping that back is predictable monthes of time...

backup to the secondary server (full redundancy as space allows) is fast - restoring too as you soon can see - the missing remainder comes from cheap terabyte disks as you recommend with cheap private line so expect it in winter (WITH rsync .

today the recovery finished and showed that content is 25% lost (striped through any file and directory, directory context lost - so all stuff is absolutely void) - with a fat storage space AND a fat line that wouldnt be an issue. so its now sure that i must start from scratch (and slow line hosted backups)

the team server has a dedicated gigabit line on a secondary nic directly connected to the main server - it just lacks the storage space for the ftp (everything else is redundant backed up) as the money which came plain wasnt enough for that part (i dont think that many people who donated had some insight on double storage needs - i of course had - but we were VERY happy with what we got - thanks to all people who made that possible at all

having redundancy with most stuff (www and files, the stuff known as bttr before) was far beyond what we expected - the ftp was a further luxus powered by donors - and i see it positive - even if it takes time - next time its better even as it always was after such problems. and again a 1GB site is easy to maintain, this is another dimension as its not only space needs but also bandwidth needs (latest uploads all were half GB per file)...

MethodGit: all 139 CDs are still online - just only as an image (so often fat and not for everybody's line) tnx to redundant space.
http://back2roots.org/CDs/

Last edited by hippie2000; 16 May 2009 at 07:37. Reason: typos
hippie2000 is offline  
Old 17 May 2009, 23:55   #156
MethodGit
Junior Member
MethodGit's Avatar
 
Join Date: Dec 2002
Location: The Streets
Age: 33
Posts: 2,723
I knew all the CDs were still online. It's just I was saying that if anyone wanted, say, disks from the 17-Bit catalogue, then the CDs would suffice for now.


BTW, is anyone getting a CRC error when attempting to extract Cutthroats? WinRAR is complaining at least.
MethodGit is offline  
Old 17 May 2009, 23:59   #157
BuZz
Registered User
 
Join Date: May 2002
Location: Oxford / UK
Age: 41
Posts: 582
Send a message via ICQ to BuZz
Quote:
Originally Posted by hippie2000 View Post
hello buzz - thanks for the hint but of course i know and use rsync, and agree its a pretty good solution, but it doesnt finance storage space. and of course i do not only have shell access, these are 2 dedicated back2roots servers and without being root there these wouldnt serve much. im not a newbee just the costs with time are huge.
The good thing is that you have the data, even if it will take a while to get it back online.

Actually I realised you knew rsync, hence I was talking about rdiff-backup which is "rsync + diffs". ;-) It is a great system for having the functionality of an rsync backup, whilst also keeping an archive of extended attributes and binary diffs. I use it on my systems, so I can not only get the "last" backup but roll back a number of days also.

http://www.gnu.org/savannah-checkout.../rdiff-backup/

Cheers!
BuZz is offline  
Old 19 May 2009, 04:03   #158
MethodGit
Junior Member
MethodGit's Avatar
 
Join Date: Dec 2002
Location: The Streets
Age: 33
Posts: 2,723
At this time of writing, the whole of BTTR seems inaccessible. Hopefully it's just no more than a little maintenance going on atm.....
MethodGit is offline  
Old 08 June 2009, 09:51   #159
SunChild
EAB Noobie

SunChild's Avatar
 
Join Date: Jun 2009
Location: Natori-shi, JAPAN
Age: 42
Posts: 117
Bumping the thread just to ask what's going on with BTTR's FTP section.
Did anyone here noticed it's all down (all the FTP directories are marked "404/offline" since 30+ days when checking the site's FTP index page).

I'd just like to be reassured that it's only gonna be temporary!...
Man, how could I survive without such a big amiga resource? (;T_T)
SunChild is offline  
Old 08 June 2009, 09:54   #160
TCD
Registered User

TCD's Avatar
 
Join Date: Sep 2006
Location: Germany
Age: 39
Posts: 24,032
Have a look here in this thread SunChild (thread merges are fun )
The name hippie2000 should ring a bell Welcome to EAB btw

Last edited by TCD; 08 June 2009 at 10:44.
TCD is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Back to the roots - online again viddi Amiga scene 40 22 October 2007 22:10
Back To The Roots back from the dead! MethodGit News 11 22 October 2007 20:27
Back to the roots Arnie support.WinUAE 1 16 April 2006 16:28
Back 2 the Roots jjsmith Amiga websites reviews 8 23 July 2005 21:30
Back to the Roots Update Paul Amiga scene 2 05 July 2003 14:55

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 03:09.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Page generated in 0.41674 seconds with 11 queries