urllib2 POST and Header python
#1
I am working on an adultswim site script and I am having a 403 access denied issue that im trying to solve.
here is the code
Code:
def getplaylist(pid):
    url = getVideoPlaylist + '?id=' + pid + '&r=' + getservertime()
    token = getservertime() + '-' + md5.new(getservertime() + pid + '-22rE=w@k4raNA').hexdigest()
    req = urllib2.Request(url,'')
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5')
    req.add_header('Referer', 'http://www.adultswim.com/video/ASFix.swf')
    req.add_header('Cookie', 's_cc=true; s_sq=%5B%5BB%5D%5D; s_vi=[CS]v1|2FB4496CC86F-A3A08150007496[CE]; adDEmas=R00&hi&comcast.net&73&usa&659&37066&43&37&T2&Y1&1529&; adDEon=true; TEGid=G44340d2e-16094-1231877458-0')
    req.add_header('x-prefect-token', token)
    response = urllib2.urlopen(req)
    link=response.read()
    response.close()
my sniffed header of the the original transaction
Code:
POST /asfix-svc/episodeservices/getVideoPlaylist?id=8a2505951ea6dcca011ea7f37295001c&r=1231880055745 HTTP/1.1
Host: asfix.adultswim.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: s_cc=true; s_sq=%5B%5BB%5D%5D; s_vi=[CS]v1|2FB4496CC86F-A3A08150007496[CE]; adDEmas=R00&hi&comcast.net&73&usa&659&37066&43&37&T2&Y1&1529&; adDEon=true; TEGid=G44340d2e-16094-1231877458-0
Referer: http://www.adultswim.com/video/ASFix.swf
Content-type: application/x-www-form-urlencoded
x-prefect-token: 1231880057614-86c50877032420909b0c6d0cff48aa04
Content-length: 8
my sniffed header sniffs of the scripts transaction
Code:
POST /asfix-svc/episodeservices/getVideoPlaylist?id=8a2505951ea6dcca011ea7f37295001c&r=1231881827429 HTTP/1.1
Accept-Encoding: identity
Content-Length: 0
Host: asfix.adultswim.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5
Connection: close
Referer: http://www.adultswim.com/video/ASFix.swf
X-Prefect-Token: 1231881827559-e1aaf1e805129ac4b88caaa662700b0d
Cookie: s_cc=true; s_sq=%5B%5BB%5D%5D; s_vi=[CS]v1|2FB4496CC86F-A3A08150007496[CE]; adDEmas=R00&hi&comcast.net&73&usa&659&37066&43&37&T2&Y1&1529&; adDEon=true; TEGid=G44340d2e-16094-1231877458-0
Content-Type: application/x-www-form-urlencoded

I just don't think i know how to properly use urllib2 and was looking for help with the correct coding. i am going to continue reading but i was hoping someone might try to help me out. If anyone needs my cap files or anything i can post them somewhere.

The script in is current form isn't really useful because i am just in the research stage trying to get all the functions working in IDLE. This is my stumbling block because this playlist page provides the http link and auth code for the flv file.

thanks in advance for any help. I have been tinkering for a while and i don't think i am going anywhere.
Reply
#2
don't worry about replying. someone releases a plugin for boxee that seems to work pretty well with xbmc. so this isn't needed.

post about boxee plugin
http://forum.xbmc.org/showthread.php?tid=43744
Reply
#3
I was also having problems with the 403 error from adultswim while building a scraper (in Perl undef Linux). I did get it working, though - it turns out that it doesn't need the cookie, referer or content-type to be set. It doesn't even care if it's GET or POST method. Apparently, all that's required is that the x-prefect-token be calculated with a timestamp that's close to their server time. Note: the timestamp must be millisecond-based (13 digits), not second-based (10 digits).

I was initially trying to use the value for "r" they had in the query URL as the timestamp, but it turns out that "r" is set 15 hours in the future. Calculating a new current timestamp made it all come together. I then started removing the other stuff I had specified one element at a time (referer, cookie, content-type, post method, etc) and found that it works with only the "id" parameter in the URL and the "x-prefect-token" header added.

The scraper exists in the form of a web-based Navi-X (XBMC script) playlist at http://navix.turner3d.net/scrape/adultswim/ if anyone would like to see it in action. (trailing slash required)

BTW, I was only able to get the script working after finding the formula for calculating the token in this post. Was that snagged by using a swf decompiler or similar?
Reply

Logout Mark Read Team Forum Stats Members Help
urllib2 POST and Header python0