Kodi Community Forum
[WIP] AniDB.net Anime Video Scraper - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Metadata scrapers (https://forum.kodi.tv/forumdisplay.php?fid=147)
+---- Thread: [WIP] AniDB.net Anime Video Scraper (/showthread.php?tid=64587)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37


- ZERO <ibis> - 2011-12-15

I try to go with the 100% correct library rather than have my friends over and go hey lets watch.. oh wait it looks like a series was mislabeled. Then it turns into a 1 hour argument over watching something we already have or finding that series that was mislabeled b/c now that someone saw it in there we all got to watch it even though no one has it. Then 5 hours later after we get it we want to do something else anyways...

This is why I make sure my library is right, it is just too dangerous otherwise. lol


- fems - 2011-12-15

I'm having some weird behavior lately with some series. If the series name is selected manually new episodes aren't added. For example, "Mirai Nikki" in the name folder gets scrapped as Mirai Nikki OVA instead of TV, fixed doing a refresh and selecting Mirai Nikki (2011). Same thing happens with Gundam AGE or Toaru Majutsu no Index. Looking at logs, anidb.net response is an empty <episodeguide></episodeguide> when scrapping the episode details, which makes the episode not listed in the library. Opening the url in a browser gets a normal response with all the details. The episodes are named "xxx Ep01.mkv" which gets correctly resolved to s1e1 in the logs.

Updated to latest build, removed all plugin files, reinstalled the scrapper, removed library files, creating a tvshow.nfo... nothing works, only creating .nfo files manually for each episode. Selecting the episodes in Videos and pressing Episode information gets a "No info" error. Any help, please?

EDIT: deleting scrapper cache fixed the issue


- hentai23 - 2011-12-16

ZERO <ibis> Wrote:I try to go with the 100% correct library rather than have my friends over and go hey lets watch.. oh wait it looks like a series was mislabeled. Then it turns into a 1 hour argument over watching something we already have or finding that series that was mislabeled b/c now that someone saw it in there we all got to watch it even though no one has it. Then 5 hours later after we get it we want to do something else anyways...

This is why I make sure my library is right, it is just too dangerous otherwise. lol


i see Shocked....Rolleyes


- Garandor - 2011-12-27

Quick question: Why no scraper for Movies?

I have some anime movies/OVAs not registered on TVDB but on aniDB, and having a method to get them scraped and listed under movies would be great.

Anyone else had that problem, or do you all just list your anime movies as "TV shows"?


- ZERO &lt;ibis&gt; - 2011-12-28

Garandor Wrote:Quick question: Why no scraper for Movies?

I have some anime movies/OVAs not registered on TVDB but on aniDB, and having a method to get them scraped and listed under movies would be great.

Anyone else had that problem, or do you all just list your anime movies as "TV shows"?

Correct all anime moves are listed under the tv show section. This is why I wish I could just relabel the section anime instead b/c that is the only thing in there. Wink


- Garandor - 2011-12-28

ZERO <ibis> Wrote:Correct all anime moves are listed under the tv show section. This is why I wish I could just relabel the section anime instead b/c that is the only thing in there. Wink

Did you find a way to distinguish between Movie/OVA and normal TV Series inside the TV Series listing? With my 100+ Series/Movies the library overview would become pretty cluttered...


- Garandor - 2011-12-31

Hi, I've run into a problem using the anidb scraper. I have a renaming structure, which is basically this:
Hellsing Ultimate (Hellsing (2006))\Hellsing Ultimate - [02]II (720x480-DVD)[COR].mkv
F:\Anime\OVAs\Pale Cocoon (2005)\Pale Cocoon - [1]Complete Movie (720x416-DVD)[ViDoM].avi

aka. in the case where the japanese anime title is different from the english one, it's appended in parentheses. (+year)
This seems to give the scraper some problems
http://pastebin.com/acBT6nnx (relevant sections of the debug log)

Basically, it thinks the title is "Hellsing Ultimate (Hellsing" (the parser only seems to strip the first set of parentheses, which obviously returns no result.

Therefore the scraper only finds the correct series and gets fanart etc. for series consisting only of the english name, but oddly enough, episodes don't get scraped for any of my series, even though i doublechecked that the regex (found in above debug log pastebin) matches the "official" return-values...

Anyone ran into similar problems and could help me on that one? :confused2:


- TREX6662k5 - 2011-12-31

http://regexpal.com/
http://www.jslab.dk/tools.regex.php

Handy for creating\learning\testing regex.

1. [Anime Name] ([YEAR])\24v2 - Fist [ACX][640x480][DVD].mkv
2. [Anime Name] ([YEAR])\01 - Sign Up [GotWoot][1280x720][HDTV].mkv
3. [Anime Name] ([YEAR])\1 - OVA [NoS][704x336][DVD]
4. [Anime Name] ([YEAR])\C1 - Opening 1 [TSR][720x480][DVD]
5. [Anime Name] ([YEAR])\101 - SLAM DUNK! [ANF & A.A][DVD][640x480]

After hours of failure I finally came up with [\\/]()([0-9]+) which works on those.


- Garandor - 2012-01-03

TREX6662k5 Wrote:http://regexpal.com/
http://www.jslab.dk/tools.regex.php

Handy for creating\learning\testing regex.

1. [Anime Name] ([YEAR])\24v2 - Fist [ACX][640x480][DVD].mkv
2. [Anime Name] ([YEAR])\01 - Sign Up [GotWoot][1280x720][HDTV].mkv
3. [Anime Name] ([YEAR])\1 - OVA [NoS][704x336][DVD]
4. [Anime Name] ([YEAR])\C1 - Opening 1 [TSR][720x480][DVD]
5. [Anime Name] ([YEAR])\101 - SLAM DUNK! [ANF & A.A][DVD][640x480]

After hours of failure I finally came up with [\\/]()([0-9]+) which works on those.

As already stated, the episode files get cleaned correctly by my regexes (found in above pastebin snippet)
The problem lies in parsing of foldernames not being influenced by regexes in advancedsettings. It seems to me, that the anidb scraper parses the foldername by starting from the right, stripping all characters until a word character is encountered and then looking that up. This behaviour seems hardcoded and is not influenced at all by advancedsettings.xml.
Only if a match is found, the episodelist is scraped and matched to episode files inside that folder, where the custom regexes come in.

I "solved" my problem by renaming folders to "English Anime Name" "Year" instead of "English ANime Name" ("Japanese Anime Name") ("Year"), but this is just a workaround, would be nice if folder/series-lookup could be customized the same way as episode lookup

Also: I don't really understand why you are giving you regex the backslash and slash characters to strip from the beginning, since the regex only works on the filename, not the full datapath...
Are you sure your simplistic regex catches
4. [Anime Name] ([YEAR])\C1 - Opening 1 [TSR][720x480][DVD]
correctly?
Taking a guess, I'd say it will not catch this file at all because of the "C" not provided for in the regex.


- TREX6662k5 - 2012-01-03

Ah my post wasn't meant as a reply to you. Sorry if it seemed that way. I just posted it in case someone found it useful.

Not picking up C1 is intentional as I don't want opening/ending sequences to be picked up as they are usually part of segmented chapters.
Slight problem I have is specials not being picked up (S1, S2 etc) but that's another issue.

Lastly if I didn't include the backslashes XBMC will crash. I assumed it needed it in order to fetch the anime name from the folder.


- Garandor - 2012-01-03

TREX6662k5 Wrote:Slight problem I have is specials not being picked up (S1, S2 etc) but that's another issue.

In case you name your specials to the standard used on anidb with a client like WebAOM (where specials are always numbered with a scheme "SXX" where X stands for a digit), you can use/modify my regex for specials:
<regexp>[\._ \-]+\[[S](0)([0-9])(-[0-9]+)?\]</regexp> (important part: [S](0)([0-9])(-[0-9]+)? )
This avoids manually renaming the files to S00EXX, but only works for specials 0-9 of a show, since the first digit of the Episode number like in
Bakemonogatari - [S02]Tsubasa Cat - Part 3 (640x360-www)[gg].mkv
gets passed to xbmc as season, and the second digit as episode number.

Also this regex leaves normal episodes unaffected, as it specifically checks for the "S" directly preceding the episode number.
If you mix anime/normal TV shows in your database with the same naming convention, this might fuck up your normal shows though,
I'm avoiding this by using a completely different naming scheme for normal shows.

Since I have only few shows with more than 9 specials, this solution works pretty well for me.

And about the backslash, my xbmc (eden beta) works fine and scrapes nicely without it.


- TREX6662k5 - 2012-01-04

Thanks for your suggestions.
At the moment I'd rather wait to see if any other solutions surface. Forcing seasons in advancedsettings.xml for certain regex is a good idea and hope it will be included in XBMC.

I probably may have done something else to cause it to crash. It was only after hours of messing around with regex I found something that actually worked.

PS Awesome pic.


- Garandor - 2012-01-04

Another problem I just found with the current behaviour of foldername parsing:

F:\Anime\TV\X (2001) <- The TV Series (26eps)
F:\Anime\Movies\X <- The Movie(5eps)

Both are the official names according to anidb. The same problem would occur with the recent "Mirai Nikki (2011)" series which has a 1Ep OVA called "Mirai Nikki"

Since both folders get their years stripped before checking and the older entry comes up first in the lookup, the TV Series gets categorised as the Movie, and as the movie only has 5eps, only the first 5eps of the TV Series get (incorrectly) scraped at all.

I would really appreciate if one of the devs of this plugin could comment on this


- scudlee - 2012-01-04

I'm not a dev, but I do know how to fix that, it's very simple.
This is only advisable if you use the anidb titles, and only have the year when it's needed.
Go to the addon folder, and open anidb.xml in a text editor (for Win7 it'll be C:\Users\<Username>\AppData\Roaming\XBMC\addons\metadata.anidb.net\anidb.xml)
Change line 18 from:
Code:
<RegExp input="$$1" output="\1" dest="18">
to:
Code:
<RegExp input="$$1%20$$2" output="\1" dest="18">
(If you use the Google search, change line 12 similarly.)

That's it. This way, the year ($$2) is no longer stripped! If you always add the year, though, the search will fail when the year isn't part of the official name (so don't do this if you do that).

A more complete fix, so it works for people who always add the year also, would be to pass $$2 to a separate destination and then run the checks in GetSearchResults twice, first with the year attached, and then without. Something like that, anyway.


- Garandor - 2012-01-06

scudlee Wrote:That's it. This way, the year ($$2) is no longer stripped! If you always add the year, though, the search will fail when the year isn't part of the official name (so don't do this if you do that).

Thanks for your solution, after rehashing/renaming all my episodes again (20hours hashing for the fourth time now >_>) to use foldernames consisting only of %ann, it now works with your workaround.

After 2 weeks I finally have a completely hashed/libraried anime collection in xbmc yay Confused (Excluding all Specials with Epnum>9,Trailer,OPs,EDs)