[RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies

  Thread Rating:
  • 6 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
scudlee Offline
Team-Kodi Member
Posts: 839
Joined: Jul 2011
Reputation: 51
Post: #31
No idea, off hand.

Start with the dumb questions:
Do other scrapers work? (i.e. Your XBMC device can definitely connect to the internet.)
Can you reach AniDB.net through a web browser? (For example, this link.)
How about Github.com? (Where the anime-list.xml is retrieved from.)

Assuming the answer is "yes" to all three, can you post a debug log (to xbmclogs.com or pastebin.com) of you running a scan, or preferably of you just pressing 'i' whilst highlighting an anime series folder that hasn't been added to the library (and thus initiating a scan for just that series). That may or may not shed light on what's tripping you up.
find quote
olympia Online
Team-Kodi Member
Posts: 2,526
Joined: May 2008
Reputation: 32
Post: #32
@scudlee, sent you a pm.
find quote
sabin357 Offline
Junior Member
Posts: 3
Joined: Jan 2013
Reputation: 0
Post: #33
Could someone tell me how I would install this? I am running a nightly on Win 8. Any help would be appreciated, as my searching has failed me. I really would like to try this as the current scraper gets me banned from Anidb frequently, I assume from scanning too quickly.
find quote
scudlee Offline
Team-Kodi Member
Posts: 839
Joined: Jul 2011
Reputation: 51
Post: #34
To install, just download the repo linked in the first post, and then in XBMC go to System/Settings -> Add-ons and select "Install from zip file" and pick the zip file you downloaded, if you now go to "Get Add-ons" you should see my repository listed and then in there you should see the movie and TV scrapers.

As for AniDB banning you, I don't think this scraper will make any difference, but I do have a theory as to what's causing it... Do you happen to have cleanonupdate set to true in your advancedsettings.xml (or just clean the library frequently)?
find quote
sabin357 Offline
Junior Member
Posts: 3
Joined: Jan 2013
Reputation: 0
Post: #35
Thank you very much. I was looking in the wrong part of XBMC for that option.

I do clean the library about once every week or two, but this seems to happen from just scanning my anime folder for new content. I can scan each show individually/manually without problem, but scanning the whole anime folder for new content seems to want to fetch each show all over again. I'm painstakingly setting each show to skip updating.
find quote
scudlee Offline
Team-Kodi Member
Posts: 839
Joined: Jul 2011
Reputation: 51
Post: #36
This is the gist of my theory: If you have files which are not enumerated correctly by XBMC (either not at all, or, for example, being wrongly counted as season 2), then during a scan XBMC will see those files as not being in the library and fetch the episode guide (from anidb) for that show - it doesn't need to perform a search, since the URL is stored in the database. However since the files can't be enumerated (or just don't match any of the episodes) then XBMC will quickly move to the next show and repeat, with quickly being the operative word. The problem would be compounded once you actually are banned, since then the episode guide will be empty and not even correctly enumerated files will find a match. However, the problem with that theory is that XBMC should just ignore folders that haven't changed since the last scan, so those unenumerated files shouldn't get checked again... Unless, that is, you've cleaned the library...

If the theory is right, then it should just be case of finding those files that aren't enumerating correctly and either removing them or renaming them so they are.

The only other thing I can think of is that there's something causing XBMC to rescan unchanged folders despite all the files in them already being in the library, but I don't know what that could be, or how to fix it.

A debug log of you scanning (and getting banned) would be very helpful.
find quote
sabin357 Offline
Junior Member
Posts: 3
Joined: Jan 2013
Reputation: 0
Post: #37
Fortunately for me (unfortunately for you trying to figure it out), the latest nightly paired with manually telling it to not update each series/movie seems to have resolved things. It's also set to only update the Anime folder when I manually tell it to scan for new content. Seems to be working fine so far.

I was using Filebot to rename everything and doublechecked each file before letting them be named, so I don't think naming was the issue. I frequently compared with the ANIDB to ensure proper numbering/naming.

Since I just got this resolved, I will likely try your mod this week. Thank you for your hard work.
find quote
14junk Offline
Junior Member
Posts: 1
Joined: Jan 2013
Reputation: 0
Post: #38
Made an account just to say... This is completely awesome!

The instructions are pretty clear, and worked perfectly. I've been searching for something that "just works" for ages, and this is pretty much it. I've finally got my RPi pulling content off of my NAS with all of the metadata, etc. I couldn't be happier! In your face, Plex!
find quote
ricochet Offline
Junior Member
Posts: 2
Joined: Jan 2013
Reputation: 0
Post: #39
I'm having issues with getting results back from the scraper. I'm using absolute ordering for my file names. Does anyone know what I might be doing wrong? Thanks in advance




Code:
19:12:04 T:5204    INFO: Loading skin file: DialogVideoScan.xml
19:12:04 T:6380   DEBUG: Thread VIDEO::CVideoInfoScanner start, auto delete: 0
19:12:04 T:6380  NOTICE: VideoInfoScanner: Starting scan ..
19:12:04 T:6380   DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'E:\Bleach\Bleach - 193.avi'
19:12:04 T:6380   DEBUG: ADDON::CScraper::FindMovie: Searching for 'Bleach - 193' using AniDB.net [MOD] scraper (path: 'C:\Users\Ricky\AppData\Roaming\XBMC\addons\metadata.tvshows.anidb.net.mod', content: 'tvshows', version: '1.1.0')
19:12:04 T:6380   DEBUG: scraper: CreateSearchUrl returned <url gzip="yes" cache="animetitles.xml">https://raw.github.com/ScudLee/anime-lists/master/animetitles.xml</url>
19:12:04 T:6380   DEBUG: FileCurl::Open(0A40EF80) https://raw.github.com/ScudLee/anime-lists/master/animetitles.xml
19:12:04 T:6380    INFO: XCURL::DllLibCurlGlobal::easy_aquire - Created session to https://raw.github.com
19:12:04 T:5204   DEBUG: ------ Window Deinit (DialogProgress.xml) ------
19:12:05 T:6380   DEBUG: scraper: GetSearchResults returned <results>'</results>
19:12:05 T:6380   DEBUG: ADDON::CScraper::FindMovie: Searching for 'Bleach   193' using AniDB.net [MOD] scraper (path: 'C:\Users\Ricky\AppData\Roaming\XBMC\addons\metadata.tvshows.anidb.net.mod', content: 'tvshows', version: '1.1.0')
19:12:05 T:6380   DEBUG: scraper: CreateSearchUrl returned <url gzip="yes" cache="animetitles.xml">https://raw.github.com/ScudLee/anime-lists/master/animetitles.xml</url>
19:12:06 T:6380   DEBUG: scraper: GetSearchResults returned <results>'</results>
19:12:06 T:6380   DEBUG: VideoInfoScanner: No (new) information was found in dir E:\Bleach\
19:12:06 T:6380  NOTICE: VideoInfoScanner: Finished scan. Scanning for video info took 00:01
19:12:06 T:6380    INFO: Video scan was stopped or finished ... restoring FindRemoteThumbs
19:12:06 T:6380   DEBUG: Thread VIDEO::CVideoInfoScanner 6380 terminating
19:12:06 T:5204   DEBUG: CGUIMediaWindow::GetDirectory (sources://video/)
19:12:06 T:5204   DEBUG:   ParentPath = [sources://video/]
[/php]
find quote
scudlee Offline
Team-Kodi Member
Posts: 839
Joined: Jul 2011
Reputation: 51
Post: #40
Did you set the scraper just for your Bleach folder?

If so, make sure you have "Selected folder contains a single TV show" on in the Set content dialog (highlight the folder in file view, bring up the context menu - 'c' or right-mouse-click, and select Change content).
find quote
eurofag Offline
Junior Member
Posts: 10
Joined: Dec 2012
Reputation: 0
Post: #41
Maybe a little off-topic, but can we somehow save the tagging setting in the AniDB applet? It's annoying to copy-paste it 2-3 times a day.
find quote
scudlee Offline
Team-Kodi Member
Posts: 839
Joined: Jul 2011
Reputation: 51
Post: #42
Yeah, sorry about that, a little slip up on my part. I prettified the code to (hopefully) make it easier to understand and edit, but it managed to push it over some internal character limit.

Here's a simplified version, that should save:
http://pastebin.com/raw.php?i=ga5Rmcmd

I'll update the first post to mention this.
find quote
ricochet Offline
Junior Member
Posts: 2
Joined: Jan 2013
Reputation: 0
Post: #43
(2013-01-22 09:58)scudlee Wrote:  Did you set the scraper just for your Bleach folder?

If so, make sure you have "Selected folder contains a single TV show" on in the Set content dialog (highlight the folder in file view, bring up the context menu - 'c' or right-mouse-click, and select Change content).

That seemed to be the problem. I can't believe I missed that. Thanks for you help!
find quote
hentai23 Offline
Fan
Posts: 688
Joined: Sep 2010
Reputation: 0
Location: another universe ..it would seem xD
Post: #44
how to rename all my anime folders and files to the appropriate episodic format for xbmc tv show compatibility , as most here already mentioned having specials and endings , openings , ovas , movies , in one folder, help please


filebot misses out like 35% since some are not even released on anidb , and then theres raws etc , that filebot doesnt even bother to properly rename since it just greys those out and leaves them be


this is the naming scheme ive set in filebot

O:/Animes/{n}/{episode.special ? 'Special' : 'Season '+s}/{n} - {episode.special ? 'S00E'+special.pad(2) : s00e00} - {t}


heres a couple of examples that get greyed out >




"O:\Animes\Sora no Otoshimono\Sora no Otoshimono + OVA (BD 1280x720 x264 FLAC)\[ReinForce-Nekomimi] Sora no Otoshimono - 01 (BD 1280x720 x264 FLAC).mkv"
"O:\Animes\Crayon Shin Chan TV\11~15\[Best]クレヨンしんちゃん DVD TV版傑作選 15\011.avi"
"O:\Animes\すきで THE ANIMATION Pretty.2\(18禁アニメ) [120928] [ピンクパイナップル] 好きで好きで、すきで THE ANIMATION Pretty.2「浮空」&「ふたごころ」.mp4"
"O:\Animes\Maji de Watashi ni Koi Shinasai!!\[Final8]Maji de Watashi ni Koi Shinasai!! - 12 (BD 10-bit 1920x1080 x264 FLAC)[B3C12071].mkv"
(This post was last modified: 2013-01-24 12:56 by hentai23.)
find quote
saitoh183 Offline
Posting Freak
Posts: 959
Joined: Jul 2011
Reputation: 15
Location: Canada
Post: #45
If they arent on Anidb , you cant use the scraper

[Image: watched-clearlogo.jpg]

If my replies help you, please click on my reputation [Image: rep_xbmc.JPG] below :) thanks :)
find quote
Post Reply