• 1
  • 28
  • 29
  • 30(current)
  • 31
  • 32
  • 37
[WIP] AniDB.net Anime Video Scraper
Hi,
I'm using xbmc 11 with Anidb scraper 2.0...the problem is : I get banned all the time!!!!
Before all was ok but now I get banned after use for a minute or two...I waited a week to used again but same result , so I went to anidb's irc an this is the log :

rebelyouth>: hi
<rebelyouth>: I have one question: I'm using xbmc with the anidb scraper and when I tried to update it I always get a ban, this is happening for couples of weeks...any idea why?
<DerIdiot>: then talk to the developers of xbmc and tell them that
<DerIdiot>: the api bans naughty clients
<rebelyouth>: But I received a ban form anidb ..you think is a bug of xbmc?
<DerIdiot>: ...
<rebelyouth>: k
<DerIdiot>: whatever code they have running ther eis THEIR code
<rebelyouth>: but the scraper is the same I always used in the past
<DerIdiot>: our api allows for data being pulled, but there are limits you have to respect otherwise you get a kick iun the balls
<rebelyouth>: they didn't change
<DerIdiot>: still their problem and not ours
<rebelyouth>: got it...tnx
<DerIdiot>: clearly they are doing somethign stupid
<DerIdiot>: probalby not respecting the server is busy response and keep on hammering instead
<rebelyouth>: ok I will contact the XBMC's people an let them know..tnx for ur help
<DerIdiot>: anidb is not imdb
<DerIdiot>: you can't keep on hammering
<DerIdiot>: we do not have the ressources for shit like that


I'm using the default setting an also tried to roll back...any idea why i have this problem?
Tnx
Reply
Hello All,

I'm having a recent issue today getting data from anidb, is this just an outage or do I need to change something on my end? I've already cleared my sources but no luck.

Code:
:36:47 T:3344  NOTICE: VideoInfoScanner: Starting scan ..
23:36:59 T:3344 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 28
23:36:59 T:3344   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
23:36:59 T:3344   ERROR: ADDON::CScraper::Run: Unable to parse web site
23:36:59 T:3344   ERROR: VideoInfoScanner: Asked to lookup episode W:\Anime\Berserk\Berserk s01e01.mkv online, but we have no episode guide. Check your tvshow.nfo and make sure the <episodeguide> tag is in place.
23:36:59 T:3344   ERROR: VideoInfoScanner: Asked to lookup episode W:\Anime\Berserk\Berserk s01e02.mkv online, but we have no episode guide. Check your tvshow.nfo and make sure the <episodeguide> tag is in place.
23:36:59 T:3344   ERROR: VideoInfoScanner: Asked to lookup episode W:\Anime\Berserk\Berserk s01e25.mkv online, but we have no episode guide. Check your tvshow.nfo and make sure the <episodeguide> tag is in place.
23:36:59 T:3344   ERROR: CPicture::CacheImage Unable to create new image special://masterprofile/Thumbnails/Video/e/e0929d50.tbn from image
23:36:59 T:3344  NOTICE: VideoInfoScanner: Finished scan. Scanning for video info took 00:12
23:37:18 T:2136 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 28
23:37:18 T:2136   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
23:37:18 T:2136   ERROR: ADDON::CScraper::Run: Unable to parse web site
23:37:18 T:2136   ERROR: CVideoInfoDownloader::Process: Error getting details from http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=98
Reply
@Rebelyouth The scraper only needs to download one xml file from anidb per series, and generally the time it takes XBMC to add even one episode is delay enough between requests. At least, in my experience - I haven't been banned in ages, and the last time was very much my fault.

So, this is my guess at what is happening:
You have several videos in your anime collection which can not be enumerated by XBMC (i.e any file whose name doesn't match a tvshowmatching regexp), and you also have cleanonupdate set to true in your advancedsettings. With that combination every time you scan, the scraper will pull the xml file for any series with those files in, but finding no enumerable episodes, XBMC will quickly move on to the next series. Too quickly, perhaps. Cleanonupdate will then make XBMC "forget" that it's already checked those non-library files, ensuring that the same thing will happen next time.

Obviously, that's just speculation on my part, there are probably other things that could cause the same effect (you may just have manually cleaned the library between updates rather than using cleanonupdate, for example.) A debug log (wiki) of you scanning and getting banned would probably clarify things.
Reply
(2012-06-05, 08:55)Kupo Wrote: Hello All,

I'm having a recent issue today getting data from anidb, is this just an outage or do I need to change something on my end? I've already cleared my sources but no luck.

Code:
:36:47 T:3344  NOTICE: VideoInfoScanner: Starting scan ..
23:36:59 T:3344 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 28
23:36:59 T:3344   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
23:36:59 T:3344   ERROR: ADDON::CScraper::Run: Unable to parse web site
23:36:59 T:3344   ERROR: VideoInfoScanner: Asked to lookup episode W:\Anime\Berserk\Berserk s01e01.mkv online, but we have no episode guide. Check your tvshow.nfo and make sure the <episodeguide> tag is in place.
23:36:59 T:3344   ERROR: VideoInfoScanner: Asked to lookup episode W:\Anime\Berserk\Berserk s01e02.mkv online, but we have no episode guide. Check your tvshow.nfo and make sure the <episodeguide> tag is in place.
23:36:59 T:3344   ERROR: VideoInfoScanner: Asked to lookup episode W:\Anime\Berserk\Berserk s01e25.mkv online, but we have no episode guide. Check your tvshow.nfo and make sure the <episodeguide> tag is in place.
23:36:59 T:3344   ERROR: CPicture::CacheImage Unable to create new image special://masterprofile/Thumbnails/Video/e/e0929d50.tbn from image
23:36:59 T:3344  NOTICE: VideoInfoScanner: Finished scan. Scanning for video info took 00:12
23:37:18 T:2136 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 28
23:37:18 T:2136   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
23:37:18 T:2136   ERROR: ADDON::CScraper::Run: Unable to parse web site
23:37:18 T:2136   ERROR: CVideoInfoDownloader::Process: Error getting details from http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=98
Hi,

It seems to be an outage, it's not working for me either. But as someone mentioned on the page before, this happens from time to time. Guess we just have to wait a day or two.
Reply
~DerIdiot | the udp api will be down for some time

until its up im pretty sure we're SOL
Reply
@Kupo, Sodden, vask: AniDB.net scraper uses Http API, but I guess it apply for both. You can check if API is up by opening

http://api.anidb.net:9001/httpapi?reques...r=1&aid=98

in your browser (aid is anime id). And looks like it's up now.
Reply
(2012-06-07, 09:30)bambi73 Wrote: @Kupo, Sodden, vask: AniDB.net scraper uses Http API, but I guess it apply for both. You can check if API is up by opening

http://api.anidb.net:9001/httpapi?reques...r=1&aid=98

in your browser (aid is anime id). And looks like it's up now.

So... if it says "<error>Banned</error>" there I reckon it's not working? I'm not sure why I would have been banned...
Reply
(2012-06-09, 11:07)Pawn Wrote:
(2012-06-07, 09:30)bambi73 Wrote: @Kupo, Sodden, vask: AniDB.net scraper uses Http API, but I guess it apply for both. You can check if API is up by opening

http://api.anidb.net:9001/httpapi?reques...r=1&aid=98

in your browser (aid is anime id). And looks like it's up now.

So... if it says "<error>Banned</error>" there I reckon it's not working? I'm not sure why I would have been banned...

Maybe you scraped too much at once. Wait 24h and try again.
Reply
Since two days I'm getting this error: "Unable to connect to remote server. Would you like to continue scanning?"


Somehow it won't scrape anymore :S
Does anyone knows the problem, because I can still view the XML page.
So I'm not banned or something for overflooding the servers.

Here's the debug
Code:
15:01:34 T:3064  NOTICE: VideoInfoScanner: Starting scan ..
15:01:34 T:3064   DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'D:\Animes\Fullmetal Alchemist Brotherhood\'
15:01:34 T:3064   DEBUG: ADDON::CScraper::FindMovie: Searching for 'Fullmetal Alchemist Brotherhood' using AniDB.net scraper (path: 'C:\Users\#####\AppData\Roaming\XBMC\addons\metadata.anidb.net', content: 'tvshows', version: '2.0.0')
15:01:34 T:3064   DEBUG: scraper: CreateSearchUrl returned <url gzip="yes" cache="anidb.xml">http://sites.google.com/site/anidblist/anidb.xml</url>
15:01:34 T:3064   DEBUG: Using proxy http://:8080
15:01:34 T:3064   DEBUG: FileCurl::Open(0678ECB4) http://sites.google.com/site/anidblist/anidb.xml
15:01:34 T:3064    INFO: XCURL::DllLibCurlGlobal::easy_aquire - Created session to http://sites.google.com
15:01:35 T:3064 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 56
15:01:35 T:3064   ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
15:01:35 T:3064   ERROR: ADDON::CScraper::Run: Unable to parse web site
Reply
hi yall.

Gintama (the original) > aniDB:3468
Gintama' (the squeal) > aniDB:8126

adding a <qoute> is a stupid way to name a sequal... but that's what we have here
what does one need to do so both are *not* picked up as the original
rPi 2&3 | android phones | fireHD8 | linux | win10 + NFS NAS w/ mySQL + props to...
libreElecyatse, titan, AELflexGet, context.manageTags (a zosky original)
Reply
(2012-07-20, 19:52)zosky Wrote: hi yall.

Gintama (the original) > aniDB:3468
Gintama' (the squeal) > aniDB:8126

adding a <qoute> is a stupid way to name a sequal... but that's what we have here
what does one need to do so both are *not* picked up as the original
Two options:
a) Simply manually refresh the Gintama' folder and select the right series. (Navigate to the folder in Files view, press 'i' and select Refresh.)

2. Create a tvshow.nfo file containing "aid=8126" in the Gintama' folder (technically, you should put in the full anidb url as linked in your post, but that part is all you really need). You'll still need to manually update as above to fix it, but if you ever need to automatically scan in again, it should work straight away without needing to be manually fixed.
Reply
Hey, I've been having some issues getting anime to work on XBMC.

I've tried it on several different computers, with several different naming schemes and even using a complex as hell advancedsettings file, and yet it hasn't worked so far Tongue

Right now my setup is like this:

S:\shares\anime\Cowboy Bebop
and under that
Cowboy Bebop - 21 - Boogie-Woogie Feng-Shui
(...)

S:\shares\anime\Eureka Seven Ao
Eureka Seven Ao - 01 - Born Slippy (Deep Blue)
(...)

This is the same naming convention i use for my TV Shows, using FileBot. (Retro Game Master - 2x02 - Challenger; Black Mirror - 1x01 - The National Anthem...).

I've tried several others, and yet XBMC refuses to let me watch anime with the fancy features! It picks up banners and whatnot, but then just explodes. Last time, it even started recognizing episodes wrong... Help?

https://dl.dropbox.com/u/14752045/xbmc.log - my most recent log file. Opened XBMC up, it refreshed library, nothing. De-set content, set it back up and had it refresh files, still nothing.

edit: Just switched to using some regexps posted in the earlier thread and the anidb renamer... That'll have to do.
Reply
Still having "issues" with this. A lot of shows are not recognized even though they are reasonably named, with a directory that ONLY contains the title of the show and filenames that have been carefully formatted.

For example, the anime "Aika"...

Anidb URL: http://anidb.net/perl-bin/animedb.pl?show=anime&aid=280
Not recognized at all with a folder named "Aika" or "Agent Aika", so I created a tvshow.nfo file with the anidb URL in it as instructed above.

Filenames are as follows:
Aika - 1 - Beautiful Agent - [1997-1999][anime-takeover][394A72BA].ogm
Aika - 2 - Naked Mission - [1997-1999][anime-takeover][F63F2BB9].ogm
..etc etc

This is a standard format applied via Anidb'O'Matic that works for about 85% of my shows, but about 15% it fails.

With tvshow.nfo, it gets the series information -- but not episode details.
Obviously the name is there, there's only ONE show by that name, so why in blue blazes is the scraper not picking up episode details? What's it take to get this thing to work?

Logfile below (cleaned up a bit and hopefully all the relevant bits)

Code:
19:37:07 T:2725125008   DEBUG: CFileSMB::Open - opened Anime C/Aika/tvshow.nfo, fd=10000
19:37:07 T:2725125008   DEBUG: CFileSMB::Close closing fd 10000
19:37:07 T:2725125008   DEBUG: scraper: NfoUrl returned <url gzip="yes" cache="280.xml">http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280</url>
19:37:07 T:2725125008   DEBUG: VideoInfoScanner: Found matching URL NFO file: smb://EDI/Anime C/Aika/tvshow.nfo
19:37:07 T:2725125008   DEBUG: VideoInfoScanner: Fetching url 'http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280' using AniDB.net scraper (content: 'tvshows')
19:37:07 T:2725125008   DEBUG: GetVideoDetails: Reading movie 'http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280' using AniDB.net scraper (file: '/root/.xbmc/addons/metadata.anidb.net', content: 'tvshows', version: '2.0.0')
19:37:07 T:2725125008   DEBUG: FileCurl::Open(0xa26e0a04) http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280
19:37:07 T:2725125008    INFO: easy_aquire - Created session to http://api.anidb.net
19:37:07 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:07 T:2725125008   DEBUG: scraper: GetDetails returned <details><url function="GetDetailsExt" cache="anime-list.xml">http://sites.google.com/site/anidblist/anime-list.xml</url></details>
19:37:07 T:2725125008   DEBUG: FileCurl::Open(0xa26e0a04) http://sites.google.com/site/anidblist/anime-list.xml
19:37:09 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:10 T:2725125008   DEBUG: scraper: GetDetailsExt returned <details><id>280</id><title>Aika</title><year>1997</year><premiered>1997-04-25</premiered><votes>1530</votes><rating>5.98</rating><genre>Ecchi</genre><genre>Pantsu</genre><plot>A catastrophic earthquake has left Tokyo, and most of Earth for that matter, under the sea level. Aika is a salvager, who retrieves various remains from the watery ruins. When Aika accepts the dangerous mission to locate and retrieve the mysterious Lagu, she discovers that other dangerous parties are also interested.</plot><actor><name>Sakuma Rei</name><role>Sumeragi Aika</role><thumb>http://img7.anidb.net/pics/anime/23759.jpg</thumb></actor><actor><name>Konishi Hiroko</name><role>Aida Rion</role><thumb>http://img7.anidb.net/pics/anime/25984.jpg</thumb></actor><actor><name>Takayama Minami</name><role>Bianca</role><thumb>http://img7.anidb.net/pics/anime/46104.jpg</thumb></actor><credits>Studio Fantasia</credits><credits>Yamauchi Noriyasu</credits><director>Nishijima Katsuhiko</director><studio>Studio Fantasia</studio><episodeguide><url gzip="yes" cache="280.xml">http://api.anidb.net:9001/httpapi?request=anime&client=xbmcscrap&clientver=1&protover=1&aid=280</url></episodeguide><url function="GetFanartDataAPI" cache="tvdb-280.xml">http://www.thetvdb.com/api/1D62F2F90030C444/series/80900/all/en.zip</url></details>
19:37:10 T:2725125008   DEBUG: FileCurl::Open(0xa26e0a04) http://www.thetvdb.com/api/1D62F2F90030C444/series/80900/all/en.zip
19:37:10 T:2725125008   DEBUG: scraper: GetFanartDataAPI returned <details><url function="GetFanartDataReturnAPI" cache="280.xml">&</url></details>
19:37:10 T:2725125008   DEBUG: scraper: GetFanartDataReturnAPI returned <details><thumb>http://img7.anidb.net/pics/anime/5721.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-4.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-1.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-2.jpg</thumb><thumb>http://thetvdb.com/banners/posters/80900-3.jpg</thumb><fanart url="http://thetvdb.com/banners/"><thumb dim="1280x720" colors="|25,35,37|25,35,37|25,35,37|" preview="_cache/fanart/original/80900-2.jpg">fanart/original/80900-2.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-1.jpg">fanart/original/80900-1.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-3.jpg">fanart/original/80900-3.jpg</thumb><thumb dim="1920x1080" colors="" preview="_cache/fanart/original/80900-6.jpg">fanart/original/80900-6.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-8.jpg">fanart/original/80900-8.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-7.jpg">fanart/original/80900-7.jpg</thumb><thumb dim="1280x720" colors="" preview="_cache/fanart/original/80900-9.jpg">fanart/original/80900-9.jpg</thumb><thumb dim="1920x1080" colors="" preview="_cache/fanart/original/80900-4.jpg">fanart/original/80900-4.jpg</thumb><thumb dim="1920x1080" colors="" preview="_cache/fanart/original/80900-5.jpg">fanart/original/80900-5.jpg</thumb></fanart></details>
19:37:10 T:2725125008   DEBUG: VideoInfoScanner: Adding new item to tvshows:smb://EDI/Anime C/Aika/
19:37:11 T:2725125008   DEBUG: OpenDir - Using authentication url xxxxxxxxxxxxxxxxxxxxxxx erased
19:37:11 T:2725125008    INFO: Caching image from: http://thetvdb.com/banners/fanart/original/80900-2.jpg to special://masterprofile/Thumbnails/Video/Fanart/4e1c3bb1.tbn
19:37:11 T:2725125008   DEBUG: FileCurl::Open(0xb10c4168) http://thetvdb.com/banners/fanart/original/80900-2.jpg
19:37:11 T:2725125008    INFO: easy_aquire - Created session to http://thetvdb.com
19:37:11 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:13 T:2725125008    INFO: Caching image from: http://img7.anidb.net/pics/anime/5721.jpg to special://masterprofile/Thumbnails/Video/4/4e1c3bb1.tbn with width 512 and height 512
19:37:13 T:2725125008   DEBUG: SECTION:LoadDLL(special://xbmcbin/system/ImageLib-i486-linux.so)
19:37:13 T:2725125008   DEBUG: Loading: /usr/local/lib/xbmc/system/ImageLib-i486-linux.so
19:37:13 T:2725125008   DEBUG: FileCurl::Open(0xa26e09ac) http://img7.anidb.net/pics/anime/5721.jpg
19:37:13 T:2725125008    INFO: easy_aquire - Created session to http://img7.anidb.net
19:37:13 T:2939726736   DEBUG: ### [Qlock] - Delaying 3 secs
19:37:14 T:2725125008   DEBUG: CAnnouncementManager - Announcement: OnUpdate from xbmc
19:37:14 T:2725125008   DEBUG: GOT ANNOUNCEMENT, type: 8, from xbmc, message OnUpdate
19:37:14 T:2725125008   DEBUG: OpenDir - Using authentication url xxxxxxxxxxxxxxxxxxxxxxx erased
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 1 - Beautiful Agent - [1997-1999][anime-takeover][394A72BA].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 2 - Naked Mission - [1997-1999][anime-takeover][F63F2BB9].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 3 - Takeoff Position - [1997-1999][anime-takeover][6082355A].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 4 - A Flower Blooming in Space - [1997-1999][anime-takeover][88CD34D0].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 5 - The Golden Delmo Operation - [1997-1999][anime-takeover][D71BEC74].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 6 - Delmo Operation  White Silver - [1997-1999][anime-takeover][79D6FCA6].ogm
19:37:14 T:2725125008   DEBUG: VideoInfoScanner: Could not enumerate file smb://EDI/Anime C/Aika/Aika - 7 - Break the Decisive Battle! Delmo Base! - [1997-1999][anime-takeover][BB3EB663].ogm
19:37:14 T:2725125008  NOTICE: CleanDatabase: Starting videodatabase cleanup ..

Looks like it (as I thought) actually finds the URL I passed it... but it REFUSES to recognize any of the episodes! Is it bouncing back to thetvdb instead of anidb for episode details? It should NOT be doing that, but it looks like what it's doing.
Reply
Hi
im using xbmc-12.0-Frodo_alpha4, everything works fine until i try to scrape some anime from my disk. For example: let's take "Accel World", anidb ID is 8691 ... and I get message saying that im banned.
After doing some digging i got to information if i can get XML from AniDB api via my browser then i should get info.
So i can see that XML - but XBMC can't.
Any workarounds ?

File tree can be found here.
Log is here.

Regards
pigers
Reply
Hi,
New user here and have been reading all day now but still feel a little lost.

I have an extensive anime collection and have created my own filing/naming scheme that is incompatible with xbmc. I'm searching for the best strategy to attack renaming everything with a high probability of success. I don't mind hand editing some stuff to get it accurate, but I would like to start on the right track, so I'm looking for some success stories or a pointer to an updated guide that walks you through and what to expect. Maybe just a good naming scheme that catches most series, movies, and OVAs.

Is Filebot a good option for renaming and is there a successful script for it?

Lastly I did read that there will be no support for OPs & EDs (specials?) which is fine, but I was curious how you deal with them as far as naming and filing as to not disrupt matching. (sub-folder?)

Any help, strategies, or anecdotes would be much appreciated.
Reply
  • 1
  • 28
  • 29
  • 30(current)
  • 31
  • 32
  • 37

Logout Mark Read Team Forum Stats Members Help
[WIP] AniDB.net Anime Video Scraper3