2013-05-30, 03:48
(2013-05-30, 02:56)Angelscry Wrote:(2013-05-30, 02:34)o9r1sh Wrote: Hello everyone, And thanks you Angelscry for this great addon. Just thought i would let you all know that Gamefaqs seems to dislike having there website scraped excessively as I've received an indefinate ban on my ip address after importing a few sets with that scraper. To be fair i do have a ton of roms so its understandable that I would trigger an automated ip block. I'm considering contacting the site to see if I can get unbanned and was wondering if anyone else has had similar experiences with gamefaqs or if anyone has some advise on how I should proceed.Here are some suggestions :
- Avoid to base all your scrapers (data, thumbs and fanarts) on the same website. Scraping stuff need 2 website connections (1 for searching the game and 1 for downloading game stuff). So, if you use the same website for all scrapers, it will represent a total of 6 website connections per game. Try to mix, for example : data -> Allgame, thumbs -> MobyGames and fanart -> GameFaq. It will then represent only 2 connections par website and per game.
- Use a dynamic IP... so you will never be permanently blocked.
- You can try to pass by a proxy or a VPN server (but connection speed will probably be slower).
A VPN sounds good, I mostly like Gamefaqs for the Jap covers, Emumovies does a pretty good job in most cases but theres times when Gamefaqs wins the race. Thanks for the suggestions.