[RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies

  Thread Rating:
  • 8 Vote(s) - 4.5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
scudlee Offline
Team-Kodi Member
Posts: 911
Joined: Jul 2011
Reputation: 58
Post: #721
Should all be updated now. Thanks!

As always, clear your cache for immediate results, otherwise it could take up to 48 hours.
find quote
TheFireKahuna Offline
Junior Member
Posts: 8
Joined: Oct 2015
Reputation: 0
Post: #722
New split-cour anime season, Gate 2016

https://anidb.net/perl-bin/animedb.pl?sh...&aid=11602

has no link atm, needs to link to 13+ from here:

http://thetvdb.com/?tab=season&seriesid=...5720&lid=7


Also, Tokyo Ghoul OVA's that came out here:

https://anidb.net/perl-bin/animedb.pl?sh...&aid=11268

needs to link to here (specials 3/4)

http://thetvdb.com/?tab=season&seriesid=...4423&lid=7

Not sure why it needed to be split off into its own anidb entry, doesn't even have a description.
find quote
scudlee Offline
Team-Kodi Member
Posts: 911
Joined: Jul 2011
Reputation: 58
Post: #723
Updated (along with a bunch of other stuff).
find quote
scudlee Offline
Team-Kodi Member
Posts: 911
Joined: Jul 2011
Reputation: 58
Post: #724
New Features in 2.4.0 (TV) and 2.2.0 (Movies)

Genres are back! Rejoice, that one person who noticed they were missing!

This was actually quite a tricky fix, a lot of complex sorting and filtering was needed to get it into a workable state, but I'm quite pleased with the results.
I've also reset the two related settings (Minimum Genre weight and Maximum number of genres) in order to give a healthy batch of genres for each title. If you'd previously changed those settings, you will need to reapply your changes - but do test the new values first, you may like them.

I've also added a new setting regarding, ahem, "Adult" genres, namely the child tags of the pornography tag (NSFW text). Although they're primarily for hentai titles, they do occasionally crop up for shows that are on the more ecchi side of things (you know the ones I mean, I'm sure).
The new setting lets you choose when to allow these genres - "Never", "Only for 18+ Restricted Titles", or "Always". The default is "Only for 18+ Restricted Titles".
Note that "allow" is not the same as "include" - the genres are still subject to the same weight limits and cut-offs as all the rest. They might not appear even if present.

Do let me know if you see anything that is not remotely a genre (adult or otherwise). If I agree, I'll add it to the filters. Similarly, if you see one genre that is redundant to another also present, that can be filtered. However I won't accept filtering classic genres like "Comedy" for more specific sub-genres like "Parody" or "Slapstick".


There were also a couple of minor fixes to plot handling (at the show/movie level, not episode plots). An old bug, where a show with without a description would occasionally use the description of one of its tags instead, had reappeared - if you have the second season of Is the Order a Rabbit??, you can see the bug in action. This has been permanently fixed.
I've also expanded the filtering of those "Source: Foo" lines, previously only specific sources were filtered, now there's a blanket removal for any name. Let me know if this negatively affects any show.
(I'm currently on the fence about removing those "Note: the first episode aired blah blah" lines also.)


The pre-Gotham warning is also in full effect. Genres and plots will now no longer appear for anyone using a version lower than 13.0, because of these changes.
find quote
Vaneska Offline
Member
Posts: 79
Joined: Dec 2011
Reputation: 3
Location: Switzerland
Post: #725
Great News! Thank you for fixing/adding new Features to the Scraper.

I have a question or maybe a suggestion. There are more and more of those Split-Season-Animes (Durarara!!x2, Aldnoah.Zero, Seraph of the End, etc. ), who are split on AniDB and together as one Season on TVDB.

For the sake of simplicity and folders Smile i would really like to use the TVDB order, i.e. get them together in one Season. Would it be possible to add such a feature? Sorry if this has already been discussed.

If i understand this correctly, the "Store TVDB-ID" is working like this, just the other way around?
find quote
scudlee Offline
Team-Kodi Member
Posts: 911
Joined: Jul 2011
Reputation: 58
Post: #726
My stock answer to wanting to use tvdb episode order... is to use the tvdb scraper, I'm afraid.

Even within a source set to use the anidb scraper, if you make a tvshow.nfo with the tvdb URL (and it has to be the full URL, you can't cheat like you can for anidb) then the tvdb scraper will be used instead, just for that folder. That's the best you can do.

There's just too many technical issues to achieve what you want, when in the end it amounts to just returning almost all data from the tvdb anyway and ignoring anidb.
find quote
Vaneska Offline
Member
Posts: 79
Joined: Dec 2011
Reputation: 3
Location: Switzerland
Post: #727
(2016-02-07 16:38)scudlee Wrote:  Even within a source set to use the anidb scraper, if you make a tvshow.nfo with the tvdb URL (and it has to be the full URL, you can't cheat like you can for anidb) then the tvdb scraper will be used instead, just for that folder. That's the best you can do.

If this works im already happy Smile Unfortunately my try didn't work. The Episodes from 13 didn't get scraped.

Folder: Seraph of the End Vampire Reign
Files from S02: Seraph of the End Vampire Reign - 13 - Human World - [HorribleSubs][900C7738] etc.

tvshow.nfo: http://www.thetvdb.com/?tab=series&id=289906

I tried to clear the Cache. Here is a Log: http://xbmclogs.com/pb4fscjhr
find quote
gates Offline
Senior Member
Posts: 148
Joined: Jun 2010
Reputation: 2
Location: France
Post: #728
Just a quick word : thank you scudlee ! Smile

for keeping alive this scraper.

Cheers
find quote
scudlee Offline
Team-Kodi Member
Posts: 911
Joined: Jul 2011
Reputation: 58
Post: #729
@Vaneska: I just tested it and it worked for me.

I'm wondering what the difference could be - If you refresh the show do you get a "Locally stored information..." dialog? You should. If so, select "no", as you very much want to use the local info.
find quote
Vaneska Offline
Member
Posts: 79
Joined: Dec 2011
Reputation: 3
Location: Switzerland
Post: #730
Hm, you are right, i should get the dialog, but i don't :/

When i "refresh" the Folder, the "Anidb Search: A10813 - Seraph...." dialog pops up.

Same happens on my Nuc installation with OpenELEC. Hm..
find quote
crazygambit Offline
Junior Member
Posts: 14
Joined: Jan 2015
Reputation: 0
Post: #731
How does Anidb deal with specials? For the life of me I can't figure it out. They seem to be individual entries on that site, so how do you go about adding them along the rest of the episodes of your show. I understand of the TVDB does it, even though I don't particularly like having to rename something like Movie 01 to Movie 04 since there are some other specials before.

I came up with some pretty nifty regex for matching anime using absolute ordering. So far they work on 100% of my non renamed files (even multi episode ones), except for having to re number the specials to match TVDB. I'd like to give this a shot, but I can't figure out how to deal with specials and tv movies, so any help would be appreciated.

Edit:

(2015-10-17 23:20)Sp4rh4wk Wrote:  
(2015-10-17 10:37)scudlee Wrote:  @Sp4rh4wk: http://forum.kodi.tv/showthread.php?tid=...pid1531110

deja vu?


well this time its different.. Im using

Code:
<tvshowmatching action="prepend">
  <regexp>(?i)().*(?:\D|^)(\d+)(?=(\[[^\]]+]|\([^)]+\)|\{[^}]+\}|[^\]\[(){}]+)*$)</regexp>
</tvshowmatching>

it works great for almost all formats.. but then tv series (not anime) scraper is fucked up..
I tried this one for tv series http://pastebin.com/UPPrk7VU but I wasnt able to implement anime one to it..
Is there a way to lock that regexp thing only for anime scraper?

for example...

if I dont have advancedsettings.xml at all, tv series are recognized fine..
but if I create advancedsettings.xml and add that regexp there, tv series arent recognized right Sad
I think thats cause that regexp is used by all scrapers and not only animdb one Sad
any way to fix this? thanks

I realize I'm very late on this and you may not need it anymore, but there's an easy fix to that. If you keep your anime separate from your tv shows, it's trivial to implement. Since the regex looks at the whole path you can just have it match the name of the parent folder your anime is at. I recently posted a set of regexes that do just that for absolute ordering, just look through my post history.
(This post was last modified: 2016-02-12 04:12 by crazygambit.)
find quote
gates Offline
Senior Member
Posts: 148
Joined: Jun 2010
Reputation: 2
Location: France
Post: #732
Hi crazygambit,

I think you can find the answer on the very first post of this thread.

Special EP are mapped with Season 0.
find quote
crazygambit Offline
Junior Member
Posts: 14
Joined: Jan 2015
Reputation: 0
Post: #733
(2016-02-12 12:02)gates Wrote:  Hi crazygambit,

I think you can find the answer on the very first post of this thread.

Special EP are mapped with Season 0.

Right, I understood that. What I didn't understand is what the episode number should be, like what is it matched against and how it works on a show with more than 1 "season". I'm thinking it's picking those up from the TVDB? It's very confusing on those shows.

Take Saint Seiya for example. I have Saint Seiya, then the Hades Chapter Sanctuary, Hades Chapter Inferno, Hades Chapter Elysium and the Lost Canvas each as their own separate show. Why? Because that's how there were released (the episodes on each of those shows started as 01) and I loath to rename stuff. To get that to work in the original Saint Seiya would be to rename episode 1 from Hades Chapter Sanctuary as episode 115 and so on, which I'm not prepared to do. So for that show I'm using this scraper.

Now about the specials. What I did here was place them all in the original Saint Seiya folder and named them according to the TVDB. It's easy in this case since there are only 5 movies. It's annoying when you have something like Special 01 and Movie 01 and have to rename them to fit the TVDB ordering, though to be honest I can't really think of a better solution. Special 6 is The Hades Chapter - Sanctuary: Myth of the resurrected Gold Saints (Part 1), which I want to group with the Hades Chapter Sanctuary show. With the TVDB scraper it's impossible, with this there may be a chance, but like I said at the beginning I'm not sure how it treats specials.

Like what should I name special 6 so it's picked up as special 1 from Hades Chapter Sanctuary? In my limited testing specials seem to appear as individual shows. Like each movie would be it's own show, which is not what I want. Unless you can do something akin to movie sets on tv shows on kodi? I'm thinking not. Therefore I'd like to understand how you guys are dealing with this issue because so far I don't get it.

The set of regexes I made are already pretty good at assigning Season 0 to any file placed in a "Specials" folder, so I've got that part covered. What the actual episode number should be and in which show it shows up is where it gets tricky.
find quote
gates Offline
Senior Member
Posts: 148
Joined: Jun 2010
Reputation: 2
Location: France
Post: #734
TV Show Sets is the key Smile
one day maybe ... http://forum.kodi.tv/showthread.php?tid=...pid1951478

personaly, i dont have a lot of specials,
and for anime like fairy tail, i use thetvdb scraper (with absolute number).

maybe you can scrap some OAV / movie / (special?) with anidb movie scraper, a and use the "link movie to tv-show" kodi function.
find quote
scudlee Offline
Team-Kodi Member
Posts: 911
Joined: Jul 2011
Reputation: 58
Post: #735
(2016-02-12 15:09)crazygambit Wrote:  In my limited testing specials seem to appear as individual shows. Like each movie would be it's own show, which is not what I want. Unless you can do something akin to movie sets on tv shows on kodi? I'm thinking not. Therefore I'd like to understand how you guys are dealing with this issue because so far I don't get it.

This is pretty much the truth of it, the special episode on anidb are the just the ones listed as S1, S2, etc. on the shows page. If it has it's own page, it's treated a separate show in it's own right.

The best thing you can do is to keep the movies in a separate source folder and use a movie scraper (e.g. the AniDB.net [MOD] movie scraper) to add them as movies to the library. You can then link each movie to the relevant TV show. The movies will also then be grouped together in a movie set in the movie library.
find quote
Post Reply