urllib.urlopen doesn't follow HTTP redirects
#1
I want my addon to be able to download a file from Sourceforge, but this requires following *two* HTTP redirects. With Python 2.7 urllib.urlopen() automatically follows the redirects and downloads the file. When I use the same code in XBMC it doesn't follow the redirects and just downloads the "HTTP/1.1 302" page.

Is there a difference in the version of Python used in XBMC (v2.4?) that causes the different behaviour and if so is there some setting I can tweak? Alternatively can anyone think of a way to get around the problem that doesn't involve parsing the HTML response and doing the redirects manually?

Thanks,

JR
Reply
#2
You don't need to parse the HTML the new location should be part of the header, this should help you do it http://diveintopython.org/http_web_servi...rects.html
Reply
#3
Hello,

there is a Python Lib called "Mechanize". It's a browser implementation which can handle redirects, forms etc.

Look in Facebook Pictures Plugin for example!


cu NordishByNature
Reply
#4
Thanks both. The thing is that urllib.urlopen does follow redirects in Python 2.7 without any messing around and I'd much prefer this to work with the version of Python used in XBMC. If I have to code up my own version of urlopen I will, but I'd prefer not to.

Later: I installed Python 2.4.4 on my PC with that version urllib.urlopen also follows redirects. It seems to be just the Python interpreter in XBMC that's the problem. The offending code is:

Code:
_KEYMAPEDITURL = "http://sourceforge.net/projects/xbmcmce/files/KeyMapEdit.exe/download"

def GetKeyMapEdit():

    import urllib

    dialog = xbmcgui.Dialog()

    dstpath = xbmc.translatePath("special://home/") + "addons\\" + _thisPluginName + "\\resources\\data\\KeyMapEdit.exe"

    # Attempt to open KeymapEdit.exe from Sourceforge
    try:
        webFile = urllib.urlopen(_KEYMAPEDITURL)

        # Attempt to open the local file
        try:
            localFile = open(dstpath, "wb")
            localFile.write(webFile.read())
            webFile.close()
            localFile.close()

        # Attempt to open KeyMapEdit.exe failed
        except IOError, e:
            dialog.ok("MCERemote", "Failed to open KeyMapEdit.exe:", str(e.args[1]))
            return False
        except:
            dialog.ok("MCERemote", "Unidentified error downloading KeyMapEdit.exe")
            return False

    # Attempt to open Sourceforge failed
    except IOError, e:
        dialog.ok("MCERemote", "Failed to connect to Sourceforge:", str(e.args[1]))
        return False
    except:
        dialog.ok("MCERemote", "Unidentified error connecting to Sourceforge")
        return False

    # Return indicating the file was successfully downloaded
    return True

This works when run on my PC with Python 2.4.4 (after changing the "dialog.ok" calls to "print" but not when run in XBMC.

JR
Reply
#5
Did you ever find a solution?

I'm seeing similar results, works fine in Python 2.7 but returns wrong html in XBMC
Reply
#6
i try to handle the same problem described http://forum.xbmc.org/showpost.php?p=767870&postcount=1
i think mechanize solve our problem but we have to add neccesary file to our zip file...

now time to read and learn new stuff Smile
Reply
#7
I've given mechanize a quick try but still getting the same results - fine in Python 2.7, wrong html returned in XBMC

The addon you have listed in your sig, is that the one where you used it?

http://code.google.com/p/turkishxbmcscraper/
Reply
#8
no just in my computer.i release that when i solve it.
Reply
#9
i find the solution from
Code:
http://stackoverflow.com/questions/2353689/how-do-i-get-the-url-of-an-http-redirects-target

urllib2.urlopen(my_url).geturl() give me my answer

Code:
[b]urllib2.urlopen('http://molasaati.com/molasaati.php?git=http://video.l3.fbcdn.net/cfs-l3-ash4/213405/495/202851533071086_58429.mp4').geturl()[/b]
'[i]http://video.ak.fbcdn.net/cfs-ak-ash4/214368/495/202851533071086_53471.mp4?oh=0b19cf9926d16570552415350c522769&oe=4DA87C00&__gda__=1302887424_4c08d2b3183dce216e176e602c7c7f59[/i]'

but i dont know how to use new url any help to continue code ?
Code:
url='http://www.sineman.net/jackass-3-d-hd-film-izle.html'
req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
response = urllib2.urlopen(req)
link=response.read()
response.close()
match=re.compile('<embed src="https://www.rolution.com/flash/player.swf\?file=(.+?)&').findall(link)
video = match[0]
urllib2.urlopen(video).geturl()
Reply
#10
Ok i find solution to continue my code. i hope it helps to someone..

Code:
url='http://www.sineman.net/jackass-3-d-hd-film-izle.html'
req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
response = urllib2.urlopen(req)
link=response.read()
response.close()
match=re.compile('<embed src="https://www.rolution.com/flash/player.swf\?file=(.+?)&').findall(link)
video = match[0]
resp = urllib2.urlopen(video)
url2 = resp.geturl()
print url2
Reply



Logout Mark Read Team Forum Stats Members Help
urllib.urlopen doesn't follow HTTP redirects0
This forum uses Lukasz Tkacz MyBB addons.