Problem making a login function for a scraper
#1
I'm trying to create a login function for http://www.webtelek.com/members/account.php. I've followed this guide but I can't quite get it to work. I've sniffed the HTTP traffic and I know that when I log in it looks like this

Code:
email_address=me%40example.com&password=MyPassword&utcoffset=-4

If you click on the remember me button it also adds persistent=y.

No matter what I try, my code below always returns Login: False. I need some guidance on what I'm doing wrong.

Code:
# -*- coding: UTF-8 -*-

import httplib
import os
import re
import urllib,urllib2
import cookielib

myusername = '[email protected]'
mypassword = 'MyPassword'
#note, the cookie will be saved to the same directory as weblogin.py when testing


def check_login(source,username):

    #the string you will use to check if the login is successful.
    #you may want to set it to:    username     (no quotes)
    logged_in_string = 'regBoxHello' #this is the name of an element once I'm logged into webtelek.com

    #search for the string in the html, without caring about upper or lower case
    if re.search(logged_in_string,source,re.IGNORECASE):
        return True
    else:
        return False


def doLogin(cookiepath, username, password):

    #check if user has supplied only a folder path, or a full path
    if not os.path.isfile(cookiepath):
        #if the user supplied only a folder path, append on to the end of the path a filename.
        cookiepath = os.path.join(cookiepath,'cookies.lwp')

    #delete any old version of the cookie file
    try:
        os.remove(cookiepath)
    except:
        pass

    if username and password:

        #the url you will request to.
        login_url = 'https://www.webtelek.com/members/account.php'
        #the header used to pretend you are a browser
        header_string = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3'

        #build the form data necessary for the login
        login_data = urllib.urlencode({'persistent': 'y', 'email_address': username, 'password': password, 'utcoffset': '-4'})
        print login_data
        #build the request we will make
        req = urllib2.Request(login_url, login_data)
        req.add_header('User-Agent', header_string)

        #initiate the cookielib class
        cj = cookielib.LWPCookieJar()

        #install cookielib into the url opener, so that cookies are handled
        opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))

        #do the login and get the response
        response = opener.open(req)
        #print str(response)
        source = response.read()
        print source
        response.close()

        #check the received html for a string that will tell us if the user is logged in
        #pass the username, which can be used to do this.
        login = check_login(source,username)

        #if login suceeded, save the cookiejar to disk
        if login == True:
            cj.save(cookiepath)

        #return whether we are logged in or not
        return login

    else:
        return False

#code to enable running the .py independent of addon for testing
if __name__ == "__main__":
    if myusername is '' or mypassword is '':
        print 'YOU HAVE NOT SET THE USERNAME OR PASSWORD!'
    else:
        logged_in = doLogin(os.getcwd(),myusername,mypassword)
        print 'LOGGED IN:',logged_in
Reply
#2
From a quick glance, it appears you are POSTing to the wrong url.

I just checked, and it appears to be submitting the POST to
Code:
https://www.webtelek.com/members/index.php?action=process
In your code you are submitting the request to
Code:
https://www.webtelek.com/members/account.php
Reply

Logout Mark Read Team Forum Stats Members Help
Problem making a login function for a scraper0