English Amiga Board


Go Back   English Amiga Board > Other Projects > project.EAB File Server

 
 
Thread Tools
Old 30 October 2017, 08:37   #1561
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
Turran,

I threw together a Python script 'eab_post_count.py' to get post count of a given EAB forum user.

This is kind of hairy in the sense that it depends on counting html tags and such on a dynamic webpage, pretty far from ideal, but it works (at the moment).

Code:
#!/usr/bin/env python 

import sys, re, urllib2

version="1.0"

if len(sys.argv) < 2:
    print(sys.argv[0] + ' v' + version +' by modrobert in 2017')
    print('Function: Returns the number of posts for a given EAB forum user.')
    print('Syntax: eab_user_count.py <username> [-i]')
    print('Options: -i ignore case sensivity in user name')
    print('Result: 0 = user found, 1 = user not found, 2 = invalid option.')
    sys.exit(1)
else:
    username = sys.argv[1]
    usernameurl = re.sub('[ ]', '%20', username)

eaburl = "http://eab.abime.net/memberlist.php?do=getall&pp=1&ausername=" + usernameurl

def parseurl(url):
    if not url:
        print("Empty URL!")
        return
    headers = { 'User-Agent' : 'Mozilla/5.0' }
    request = urllib2.Request(url, None, headers)
    response = urllib2.urlopen(request)
    content = response.read()
    return content

def pagesearch(content, trigger, start, end):
    sane = 0
    needlestack = []
    while sane == 0:
        curpos = content.find(trigger)
        if curpos >= 0:
            testlen = len(content)
            content = content[curpos:testlen]
            curpos = content.find('"')
            testlen = len(content)
            content = content[curpos+1:testlen]
            curpos = content.find(end)
            needle = content[0:curpos]
            result = content[len(start):curpos]
            if needle.startswith(start):
                needlestack.append(result)
        else:
            sane = 1
    return needlestack

def unescape(s):
    s = s.replace("&lt;", "<")
    s = s.replace("&gt;", ">")
    # this has to be last:
    s = s.replace("&amp;", "&")
    return s

eabcontent = parseurl(eaburl)

countlist = pagesearch(eabcontent, 'td class', 'alt2">', '</td>')
userlist = pagesearch(eabcontent, 'member.php?', '>', '</a>')

try:
    if sys.argv[2] == '-i':
        if unescape(str.lower(''.join(userlist))) != str.lower(username):
            print("User not found: " + username)
            sys.exit(1)
    else:
        print("Invalid option. ")
        sys.exit(2)
except Exception:
    if unescape(str(''.join(userlist))) != username:
        print("User not found: " + username)
        sys.exit(1)

for item in countlist:
    stripitem = re.sub('[,]', '', item)
    try:
        print(int(stripitem))
        sys.exit(0)
    except Exception:
        continue
Attached Files
File Type: zip eab_post_count_v1_0.zip (1.1 KB, 17 views)

Last edited by modrobert; 30 October 2017 at 14:30. Reason: Fixed bug, apparently a user with > 1000 posts get a ',' in their counter. Added handling of space in user name. Fixed "more than one hit" bug (thanks Turran).
modrobert is offline  
Old 30 October 2017, 08:49   #1562
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Quote:
Originally Posted by modrobert View Post
Turran,

I threw together a Python script 'eab_post_count.py' to get post count of a given EAB forum user.

This is kind of hairy in the sense that it depends on counting html tags and such on a dynamic webpage, pretty far from ideal, but it works (at the moment).
Neat, thanks I did something similar in bash but decided against it. Its far too easy to just take one of the 5000 registered member names, see who has more then <number> of posts and register an FTP account with that name, if all you want to do is download.

Thats why I wanted to move the register account page here, behind a "member only" page. Kinda like The Zone in a way. Then we ensure that the username registered on the FTP is the same as the forum username.
Turran is offline  
Old 30 October 2017, 09:36   #1563
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
Quote:
Originally Posted by Turran View Post
Neat, thanks I did something similar in bash but decided against it. Its far too easy to just take one of the 5000 registered member names, see who has more then <number> of posts and register an FTP account with that name, if all you want to do is download.

Thats why I wanted to move the register account page here, behind a "member only" page. Kinda like The Zone in a way. Then we ensure that the username registered on the FTP is the same as the forum username.
Yes, that makes sense.

PS: Found a bug in the script, edited previous post. Apparently a user with > 1000 posts get a ',' in their counter.
modrobert is offline  
Old 30 October 2017, 10:10   #1564
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Quote:
Originally Posted by modrobert View Post
Yes, that makes sense.

PS: Found a bug in the script, edited previous post. Apparently a user with > 1000 posts get a ',' in their counter.
Thanks. I tried implementing it for fun but the "more then one user found" page messes it up. Not your fault but we need a function to get the exact username from the forums, not just "containing" like it is today.

For example, if I registered an EAB user called "Turbo", I can not check its post count or even existence from your script since we get a lot of users with "Turbo" in their name..

This can of course be extended in the script to check all hits found, if one is an exact match, but perhaps more work than its worth right now.

Last edited by Turran; 30 October 2017 at 10:16.
Turran is offline  
Old 30 October 2017, 10:19   #1565
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
Quote:
Originally Posted by Turran View Post
Thanks. I tried implementing it for fun but the "more then one user found" page messes it up. Not your fault but we need a function to get the exact username from the forums, not just "containing" like it is today.

For example, if I registered a EAB user called "Turbo", I can not check its post count or even existence from your script since we get a lot of users with "Turbo" in their name..
If there is more than one user name in the result (from the actual search at EAB), then the user name given is incomplete/invalid.

The exit codes of my Python script are:

0 = user found, 1 = user not found, 2 = more than one user found.

So, for example, if you want to use this from a Bourne shell script.

Code:
#!/bin/sh
USER="Turran"
OUTPUT=`eab_post_count.py "$USER"`
RTN=$?
if test $RTN -eq 0 ; then
    # user found, do your stuff here
    echo "Post count: $OUTPUT"
fi
Note that the user name match is case-insensitive due to the forum search function.

PS: Updated the Python script post with another bug fix, the forum allows spaces in user names, handling that now.

Last edited by modrobert; 30 October 2017 at 11:54. Reason: Improved the shell example.
modrobert is offline  
Old 30 October 2017, 11:59   #1566
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
So try getting the postcount for "Turbo". You cant since the result is only "More then one user found".

We need to go through the results if its more then one, and only when it finds an exact match, report the postcount. If no exact match is found then return "user not found".

Otherwise, the user "Turbo" can never register an EAB FTP account since we can not check the postcount =)
Turran is offline  
Old 30 October 2017, 12:01   #1567
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
Quote:
Originally Posted by Turran View Post
So try getting the postcount for "Turbo". You cant since the result is only "More then one user found".

We need to go through the results if its more then one, and only when it finds an exact match, report the postcount. If no exact match is found then return "user not found".

Otherwise, the user "Turbo" can never register an EAB FTP account since we can not check the postcount =)
Look at my shell script example again.

EDIT:

OK, here's another example, checking of exit code is "not equal" to zero.

Code:
#!/bin/sh
USER="Turran"
OUTPUT=`eab_post_count.py "$USER"`
RTN=$?
if test $RTN -ne 0 ; then
    # user name not found or incomplete (more than one)
    echo "User not found"
    exit 1
fi
# we have a valid user name
# do your stuff here
echo "Post count: $OUTPUT"

Last edited by modrobert; 30 October 2017 at 12:22. Reason: Added another shell script example.
modrobert is offline  
Old 30 October 2017, 12:22   #1568
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
I dont think we understand each other. If the user "Turbo" tries to add an EAB FTP account, he is going to be denied because eab_post_count.py is going to report
print("* Found more than one user: " + username), when he indeed has a valid EAB Forums account.
This is because there are three other usernames also containing "Turbo" (RobTurbo, Turbo2Xs & TurboCrash).


We have no way to get the postcount from the user "Turbo" using the existing script, therefor he can not create an FTP account, even though he is a valid user (Well, he does not exist right now. Its just an example).

So my suggestion would here, here. Instead of the sys.exit(2):
print("* Found more than one user: " + username)
sys.exit(2)

go through all the results we got back, check if the username is an exact match to any of the result. If it is, grab the post count and return that.
If no user in all the results are a match, return
print("* User not found: " + username)
sys.exit(1)

But then again, do not spend time on this because its easy to exploit anyway by just browsing the eab member list and creating an FTP user with the same name as an existing member.
Turran is offline  
Old 30 October 2017, 12:29   #1569
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
Quote:
Originally Posted by Turran View Post
I dont think we understand each other. If the user "Turbo" tries to add an EAB FTP account, he is going to be denied because eab_post_count.py is going to report
print("* Found more than one user: " + username), when he indeed has a valid EAB Forums account.
This is because there are three other usernames also containing "Turbo" (RobTurbo, Turbo2Xs & TurboCrash).
No, it will report "User not found" and "exit 1", same as if the name was incorrect, in my previous edited post.

Quote:
We have no way to get the postcount from the user "Turbo" using the existing script, therefor he can not create an FTP account, even though he is a valid user (Well, he does not exist right now. Its just an example).
If user "Turbo" gives more than one hit he is not a valid EAB forum user, or he entered the user name incomplete, my previous Bourne shell example handles this correctly using the exit/result code (-ne 0 / != 0).


Perhaps you posted your reply while I was editing my previous post to add another example?

Last edited by modrobert; 30 October 2017 at 12:51.
modrobert is offline  
Old 30 October 2017, 12:52   #1570
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Quote:
Originally Posted by modrobert View Post
No, it will report "User not found" and "exit 1", same as if the name was incorrect, in my previous edited post.
Wrong.
python eabpostcount.py Turbo
* Found more than one user: Turbo

That user does not exist as its just an example. Lets give a real life example:

Quote:
Originally Posted by modrobert View Post
If user "Turbo" gives more than one hit he is not a valid EAB forum user, or he entered the user name incomplete
Not correct. Check user AGN. Fully valid user.
If he tries to create an FTP account:
# python eabpostcount.py AGN
* Found more than one user: AGN

Why? There are other users that has the lettering agn in their name. All of them are valid

http://eab.abime.net/memberlist.php?...&ausername=AGN

That is because memberlist.php uses "like" instead of "=". Its not an exact match were getting but simply a "contains" search.

Last edited by Turran; 30 October 2017 at 12:58.
Turran is offline  
Old 30 October 2017, 12:55   #1571
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
Quote:
Originally Posted by Turran View Post
Not correct. Check user AGN. Fully valid user.
If he tries to create an FTP account:
# python eabpostcount.py AGN
* Found more than one user: AGN

Why? There are other users that has the lettering agn in their name. All of them are valid

http://eab.abime.net/memberlist.php?...&ausername=AGN

That is because memberlist.php uses "like" instead of "=". Its not an exact match were getting.
Yes, but my previous Bourne shell example adds the functionality you want by testing the result/exit code (-ne 0 / != 0) of eab_post_count.py. Here's the example again.

Code:
#!/bin/sh
USER="Turran"
OUTPUT=`eab_post_count.py "$USER"`
RTN=$?
if test $RTN -ne 0 ; then
    # user name not found or incomplete (more than one)
    echo "User not found"
    exit 1
fi
# we have a valid user name
# do your stuff here
echo "Post count: $OUTPUT"
Both incorrect and incomplete (more than one hit) user names will be rejected.

EDIT: I see what you mean now, finally, sorry for the misunderstanding. I will fix it.

Last edited by modrobert; 30 October 2017 at 13:03.
modrobert is offline  
Old 30 October 2017, 13:12   #1572
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Sweet =)

Is it possible to grab the "Last visit" as well as postcount? Then we check that as well. If the user has not been active/logged on to EAB for a while, we can deny the creation as well.

Then we might block (some of) the users just grabbing someone elses EAB Forums username to create an FTP account with..
Turran is offline  
Old 30 October 2017, 13:13   #1573
DannyBoy
Registered User
 
Join Date: Jul 2013
Location: Helsinki / Finland
Age: 38
Posts: 90
Damn, this was painful to read Hope you guys get them codes working
DannyBoy is offline  
Old 30 October 2017, 13:39   #1574
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Perhaps we should move to PM =)
Turran is offline  
Old 30 October 2017, 13:45   #1575
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
No thanks, considering some of the crap posted in this thread so far, a bit of code can't hurt. Almost done.

Last edited by modrobert; 30 October 2017 at 16:14.
modrobert is offline  
Old 30 October 2017, 14:40   #1576
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 52
Posts: 520
OK, edited my previous "release post" with new eab_post_count.py v1.0.

Changelog:

* Fixed bug with "more than one user", now compares given user name with output from EAB forum (thanks Turran).
* Added -i option to ignore case in user name, this has to be the last option on command line, default (no option) is now case-sensitive user name.
* User names with '<', '>' and '&' characters are now handled correctly by converting html escape codes.
* Result/exit codes are now as follows: 0 = user found, 1 = user not found, 2 = invalid option.

I skip the "Last Visit" option for now, vBulletin have these "Today", "Yesterday", "A week ago" and then "DD Month YYYY" strings, not something you can directly do math operations on, need a big table.

Last edited by modrobert; 30 October 2017 at 15:10.
modrobert is offline  
Old 30 October 2017, 17:32   #1577
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Thanks. Testing.
Turran is offline  
Old 30 October 2017, 18:09   #1578
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
OK done. Thanks modrobert, it works fine. https://grandis.nu:444/eabadduser/ updated =)
Turran is offline  
Old 30 October 2017, 18:13   #1579
Dunny
Registered User

Dunny's Avatar
 
Join Date: Aug 2006
Location: Scunthorpe/United Kingdom
Posts: 1,312
Worked perfectly, many thanks
Dunny is offline  
Old 30 October 2017, 18:20   #1580
Turran
Moderator

Turran's Avatar
 
Join Date: May 2012
Location: Stockholm / Sweden
Age: 45
Posts: 1,106
Quote:
Originally Posted by Dunny View Post
Worked perfectly, many thanks :)
Postcount confirmed =)

Quote:
17-10-30 17:11:24 - CensoredIP - OK Adding: User:Dunny Question:amiga PostCount:817
17-10-30 17:11:24 - CensoredIP - Result: 200 User (Dunny) successfully added.
Turran is offline  
 


Currently Active Users Viewing This Thread: 3 (0 members and 3 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
While EAB file server is down... Retroplay project.EAB File Server 23 31 December 2013 00:48
Setting up an EAB file server account. fatman2021 project.EAB 1 26 August 2013 08:24
Is EAB file server deleted ? Ribdevil project.EAB File Server 13 03 February 2012 19:28
EAB File Server structure update TCD project.EAB File Server 38 18 November 2010 06:49
EAB File Server team reinforcement TCD project.EAB File Server 9 22 August 2010 20:39

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 16:00.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.
Page generated in 0.15984 seconds with 15 queries