(2017-09-28, 00:37)Ebs_1 Wrote: no still nogo 404 error
import urllib,urllib2
print urllib2.urlopen("https://www.myabandonware.com/download/1d4-commander-keen-keen-dreams").geturl()
As this is a learning forum, here's a tip:
try add 'Http Header Referer', like this:
Code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import urllib, urllib2, re, HTMLParser, os
import html5lib
from bs4 import BeautifulSoup
def msg(text):
print(text)
def log(text):
print(text)
def open_url(url, H=''):
pg_source = ''
req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36')
if H != '': req.add_header('Referer', H)
try:
response = urllib2.urlopen(req)
pg_source = response.read().decode('utf-8' , 'ignore')
#pg_source = response.read()
response.close()
except urllib2.URLError, e:
if hasattr(e, 'code'):
try:
msg('HTTP error nº: ' + str(e.code) + ' \n' + str(e.reason) + '\n' + e.url)
log('HTTP error nº: ' + str(e.code) + ' \n' + str(e.reason) + '\n' + e.url)
except:
msg('HTTP error nº: ' + str(e.code) + '\n' + e.url)
log('HTTP error nº: ' + str(e.code) + '\n' + e.url)
elif hasattr(e, 'reason'):
msg(str(e.reason))
log(str(e.reason))
return pg_source
url = 'https://www.myabandonware.com/game/commander-keen-keen-dreams-1d7'
print open_url(url, H=url)
In my example, I set the header equal to url
But by looking at the urls in question, you can see that there is a pattern in the zip files:
Code:
--------- example 01
Url: https://www.myabandonware.com/game/commander-keen-keen-dreams-1d7
DOS Version
Button link: https://www.myabandonware.com/download/1d4-commander-keen-keen-dreams
Zip link: https://www.myabandonware.com/media/files/games/c/commander-keen-keen-dreams.zip
--------- example 02
Url: https://www.myabandonware.com/game/get-dexter-703
Atari ST ROM
Button link: https://www.myabandonware.com/download/i7q-get-dexter
Zip link: https://www.myabandonware.com/media/files/games/g/get-dexter_atari-st_0i7q.zip
--------- example 03
Url: https://www.myabandonware.com/game/007-licence-to-kill-kq
DOS Version
Button link: https://www.myabandonware.com/download/kp-007-licence-to-kill
Zip link: https://www.myabandonware.com/media/files/games/0-9/007-licence-to-kill.zip
Atari ST ROM
Button link: https://www.myabandonware.com/download/ikv-007-licence-to-kill
Zip link: https://www.myabandonware.com/media/files/games/0-9/007-licence-to-kill_atari-st_0ikv.zip
pattern:
DOS Version
Button link:
https://www.myabandonware.com/download/kp-
007-licence-to-kill
Zip link:
https://www.myabandonware.com/media/files/games/0-9/007-licence-to-kill.zip
Atari ST ROM
Button link:
https://www.myabandonware.com/download/ikv-
007-licence-to-kill
Zip link:
https://www.myabandonware.com/media/files/games/0-9/007-licence-to-kill_atari-st_0ikv.zip