[WIP] AniDB.net Anime Video Scraper

  Thread Rating:
  • 3 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Xeijin Offline
Member
Posts: 62
Joined: Jul 2007
Reputation: 0
Post: #121
spiff Wrote:you're not attempting to scan the problematic show in that log.

Well I'm now planning to rename my entire library with the AniDB.net to solve that problem, however here is a full scan of my Anime anyway:

xbmc.log

It picks up the names of shows but there's nothing in my library (i.e. the shows themselves, even without episodes do not appear).
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #122
lines 736-737 indicates a scraper bug.
find quote
Xeijin Offline
Member
Posts: 62
Joined: Jul 2007
Reputation: 0
Post: #123
spiff Wrote:lines 736-737 indicates a scraper bug.

Ah I see, upon clicking the URL in line 736

(http://api.anidb.net:9001/httpapi?reques...ver=1&aid=)

You get:

Code:
<error>Banned</error>
Does this mean XBMC has been banned? Or is it just a result of me clicking the link?
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #124
problem is we have no id. it's probably just the url without an id that yields that. either that or it's checking the user agent.
find quote
Xeijin Offline
Member
Posts: 62
Joined: Jul 2007
Reputation: 0
Post: #125
spiff Wrote:problem is we have no id. it's probably just the url without an id that yields that. either that or it's checking the user agent.

Well thanks for all your help spiff. Going to try the ANN scraper and see if I have any luck with that.
find quote
Kirov Offline
Junior Member
Posts: 4
Joined: Sep 2009
Reputation: 0
Post: #126
Xeijin Wrote:Ah I see, upon clicking the URL in line 736

(http://api.anidb.net:9001/httpapi?reques...ver=1&aid=)

You get:

Code:
<error>Banned</error>
Does this mean XBMC has been banned? Or is it just a result of me clicking the link?

I've run into this issue. Most likely the scraper still doesn't play by AniDB's HTTP API Flooding Rules (their scraping/api policy is strict to say the least) in all use cases. I don't know if a ban is permanent or time based but it is IP based.
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #127
might be related to the fact that the cache was ineffective for quite some time. vdrfan fixed it yesterday
find quote
Xeijin Offline
Member
Posts: 62
Joined: Jul 2007
Reputation: 0
Post: #128
Yes it seems this way as the ANN scraper has worked fine for me, whereas the <results></results> for AniDB always seems to be empty Confused problem is some ANN info is not accurate (episode titles).

Also, a bit off topic, and I don't think it's a problem anymore but they seem to have their own facility for title searching now (as opposed to using google?) http://anisearch.outrance.pl/ (Documentation - http://anisearch.outrance.pl/doc.html)
(This post was last modified: 2010-08-28 02:28 by Xeijin.)
find quote
salival Offline
Member
Posts: 65
Joined: May 2010
Reputation: 0
Post: #129
Xeijin Wrote:Also, a bit off topic, and I don't think it's a problem anymore but they seem to have their own facility for title searching now (as opposed to using google?) http://anisearch.outrance.pl/ (Documentation - http://anisearch.outrance.pl/doc.html)
I believe that's a third party service that searches the same animetitles.xml that this scraper does.
find quote
jfgi Offline
Junior Member
Posts: 24
Joined: Jan 2010
Reputation: 0
Post: #130
Would it be possible to use the english title in the library instead? So Xam'd: Lost Memories instead of Bounen no Xamdou.
find quote
CourierSS Offline
Junior Member
Posts: 8
Joined: Jul 2010
Reputation: 0
Post: #131
jfgi Wrote:Would it be possible to use the english title in the library instead? So Xam'd: Lost Memories instead of Bounen no Xamdou.

I'm not sure if there's a way to retrieve titles by language, but you can rename the entry in library mode by right clicking on the title and clicking "Edit title."
find quote
jfgi Offline
Junior Member
Posts: 24
Joined: Jan 2010
Reputation: 0
Post: #132
It is getting them. I can see it in the cache.

Code:
<titles>
<title xml:lang="x-jat" type="main">Bounen no Xamdou</title>
<title xml:lang="ru" type="synonym">Ксам`д: Потерянные Воспоминания</title>
<title xml:lang="ar" type="synonym">ذكريات زامد الضائعة</title>
<title xml:lang="tr" type="synonym">Xam`d: Kayıp Hatıralar</title>
<title xml:lang="zh-Hans" type="synonym">亡念的扎穆德</title>
<title xml:lang="x-unk" type="short">xamd</title>
<title xml:lang="x-jat" type="short">BnX</title>
<title xml:lang="ja" type="official">亡念のザムド</title>
<title xml:lang="en" type="official">Xam`d: Lost Memories</title>
</titles>

I know i can edit them manually for each one, but with the size of my library i'd prefer not to.
find quote
Xeijin Offline
Member
Posts: 62
Joined: Jul 2007
Reputation: 0
Post: #133
It still doesn't even scan for me No is it worth going on the AniDB forums and posting to ask if I've been banned? I can't think of any other reason it wouldn't work.
find quote
jfgi Offline
Junior Member
Posts: 24
Joined: Jan 2010
Reputation: 0
Post: #134
Xeijin Wrote:It still doesn't even scan for me No is it worth going on the AniDB forums and posting to ask if I've been banned? I can't think of any other reason it wouldn't work.


http://forum.xbmc.org/showpost.php?p=584...stcount=93 Try that advancedsettings.xml.
find quote
snyft Offline
XBMC Fan
Posts: 177
Joined: Jul 2010
Reputation: 0
Location: Sweden
Post: #135
CourierSS Wrote:I'm not sure if there's a way to retrieve titles by language, but you can rename the entry in library mode by right clicking on the title and clicking "Edit title."

This is the only thing the scraper need and then its complete imo.
Awesome work anyway!
find quote
Post Reply