Kodi Community Forum

Full Version: [MOD] AniDB.net scrapers for TV shows and Movies
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Scudlee, is there a trick I can get rid of cache? I mean the usual trick (appending a random number to url for example)
.. well i can't do that i guess... haven't looked into scraper souce yet.
And removing cache folder itself doesnt rly work as well (I manually edited the cache .xml and added episode 1 myself)

of: http://anidb.net/perl-bin/animedb.pl?sho...e&aid=9418
Which aired not even 4hours ago.
The caching that's affecting you is happening at the AniDB end of things.

This is the URL the scraper uses:
http://api.anidb.net:9001/httpapi?reques...1&aid=9418
...No episodes are currently listed in the XML despite clearly being on the website.

Nothing the scraper can do to fix that.
I don't know of anyway to invalidate the cache at their end (and I doubt they'd be very happy if I did and implemented it).
(2013-04-09, 20:37)scudlee Wrote: [ -> ]The caching that's affecting you is happening at the AniDB end of things.

This is the URL the scraper uses:
http://api.anidb.net:9001/httpapi?reques...1&aid=9418
...No episodes are currently listed in the XML despite clearly being on the website.

Nothing the scraper can do to fix that.
I don't know of anyway to invalidate the cache at their end (and I doubt they'd be very happy if I did and implemented it).
I never said you should do it, I just asked for a way some user can do it Tongue..
I'd say Aiura is an edge case anyway, the first episode must have only just been added to the database today on it's airdate, whereas usually they get added well in advance (see how it now list up to episode 4).

Just put it down as one the inevitable bumps that happen at the start of a new season until things settle down.

Last season I had issues with Courtesy of Zettai Karen Children: The Unlimited - Hyoubu Kyousuke, as it started off not having the "Courtesy of" part in the title on AniDB. So episode 1 was in one folder, episode 2 in another, and the scraper wouldn't match the new title until I updated the animetitles.xml.
I don't know what I did differently today, but I loaded a few new animes into XBMC and went to update the library, only to find a message saying I'd been banned from AniDB and "to wait some time until you are unbanned"..

### YOU ARE BANNED FROM ANIDB.NET ###
WAIT SOME TIME UNTIL YOU ARE UNBANNED

So.. what now? How long until Anidb unban me? And why is this happening now??
I ran into the same banning problem when I tried to scrape my entire anime folder with the mod. I waited 24 hours like a good little boy, but was still getting the ban message when trying to scrape.
I then deleted the cache thinking that it had somehow cached the ban, and that turned out to be the solution.

I imagine the reason I got banned when trying to scrape the entire folder is because my system is ridiculously fast, and I'm on a gigabit fiber connection. Now I'm scraping one series at a time rather than the whole folder at once.
Yeah, I don't think the bans last that long (it may only be in the order of a few hours), but since everything is cached for 48 hours, it can feel like a lot longer.

Like otakuhjp said, it usually happens because things are going too fast. If say, you add a single new episode to a large number of existing series at the same, then there might too many requests to AniDB without enough delay between them.

The movie scraper is particularly vulnerable to this (since it doesn't bother with episodes at all), which is why I added the Delay parameter to the settings to try and slow it down. In the 2.0.0 release (...whenever that happens) I've also added it to the TV scraper, so you'll be able to crank up the value if you keep getting banned.
Back again. Things are working really well for the most part, but I got a new "strange" hit. The first Patlabor movie.

According to the AniDb main title, it is named "Kidou Keisatsu Patlabor". -> But if I use this I get an automatic hit on the OAV series from 1990.
So I try to add the year, "Kidou Keisatsu Patlabor (1989)". -> Still get the automatic hit on the 1990 OAV series.
Then I try to add the extra title as seen in the official Japanese name on AniDb, "Kidou Keisatsu Patlabor - The Mobile Police". -> No automatic hit.
Then I try to add the extra title as seen in the official english name on AniDb, "Kidou Keisatsu Patlabor - The Movie". -> Now I get an automatic hit on "Kidou Keisatsu Patlabor 2: The Movie".
Turns out I needed to name it "Kidou Keisatsu Patlabor Gekijouban", which is found in the synonym list at the bottom of the second tab on the info page at AniDb.

I'm not saying it's wrong, as this is one of the known titles even though it's not listed as official at AniDb. Just wanted to point it out.

EDIT:
I'm using a folder tagged as "movies", not "TV series", with the AniDb [Mod] scraper. As it is a movie...
I'm totally not seeing that. I had a really good theory as well as to why it might be happening, but when I tested to see... it just worked.

Folder name "Kidou Keisatsu Patlabor" comes up with the movie (A777) as the first hit, the second OVA series (A1518) next (using a synonym), the third movie (A537) and then the first OVA (A1195).

As you're using the movie scraper, do you have the "Movies are in separate folders that match the movie title" setting on? If not, what is the exact file name you're using? If the setting is on, the folder name is used, otherwise the file name is used and perhaps something in the file name is making the OVA match better.

(Adding the year is a bad idea in this case, as XBMC will absolutely prioritize the "one-year-out" match of the OVA over the "no-year-supplied" movie.)
Appreciate the help, guys! I deleted the cache and refreshed each series individually and it's working fine now Smile
Each movie is in a separate folder, and I'm using the Movie version of the scraper with that setting on.

It have the year on the folder for most movies (anime, live action & cartoons), and any tv series that allow it (for first aired season), as I find it makes it easier to get an overview on the file server. But I did test to remove it earlier without success in this case. But when I tested it again a moment ago it hit the movie directly. Strange... Not sure why it didn't earlier today.
I hope we will have one more scraper sone Big Grin

I have PM scudlee on a new idée.
(2013-04-10, 02:47)PopeTackler Wrote: [ -> ]I don't know what I did differently today, but I loaded a few new animes into XBMC and went to update the library, only to find a message saying I'd been banned from AniDB and "to wait some time until you are unbanned"..

### YOU ARE BANNED FROM ANIDB.NET ###
WAIT SOME TIME UNTIL YOU ARE UNBANNED

So.. what now? How long until Anidb unban me? And why is this happening now??

Do they ban by IP, given there's no login required to scrape? I hope I don't run into these issues, having just scraped my entire collection twice now.

Many thanks to the creator of the AniDB scraper! Very nice indeed! Big Grin And yes, would of course be heaps nicer if it would use ed2k hashes, but it's a definite improvement from what I had previously.
Could someone please remind me what the settings are for downloading fanart. My setup used to work but after a reinstall I'm not able to download any fanart with the anidb[mod] and it used to work. Thanks
will this also work if you keep all your (unseen) anime in one folder?
Want to install this scraper for my wife who is currently using MAL and I can't get my harmony to work with that