Release [MOD] AniDB.net scrapers for TV shows and Movies
(2020-02-01, 19:37)scudlee Wrote: So...  Having woken somewhat from my slumber, and seeing a lot of complaints in this thread, I decided to have a look at trying to fix the scraper.

I originally quietly abandoned the scraper because the v1 tvdb api was supposedly going to be discontinued, and I didn't have a good way to integrate the v2 api without completely rewriting everything, so I was expecting the scraper to just kind of die a death.  Instead, of course, the v1 api never stopped, and the scraper kind of muddled on, but by then my interest (in a lot of things) had passed.  A while ago though, I did do a complete rewrite of the tvdb scraper (as it turns out, purely for my own entertainment, not public consumption) which made the whole thing a lot more modular, and it turned out it was surprisingly easy to bodge it into the anidb scraper and get working.

At the moment it's not in a fit state to push to the repo - really, the code is a mess, tabs and spaces colliding, functions just copy-pasted willy-nilly, little bits of debugging here and there, etc. (plus I really need a minute to remember what I need to do to publish it.)
It also needs testing, anyway, so rather than wait until I get it looking all pretty, I figured I'd just upload the zip files straight away, and let you have at it.

[metadata.common.anidb.net] v1.2.1 (temp version number for testing)
[metadata.tvshows.anidb.net.mod] v2.4.1 (temp version number for testing)
No update for the movie scraper.  All it's important stuff is in the common library anyway.

(Install the common library first, as the tv scraper now depends on the updated version.)

Fixes:
  • TVDB/TMDB API keys updated to current ones.  I'm assuming this might have been an issue.
  • TVDB artwork restored - posters, banners, fanart (using V2 api - so no zip errors)
  • TVDB episode details restored - episode plots, thumbnails, uniqueids
  • Some tweaks to the plot filtering to fix the somewhat overzealous editing <-- CHECK THIS PLEASE
  • Decoupled the language selection from the Official/main name selection.  You can select "main" name and still have a different language for episode plots and titles (where available - English is the automatic fallback).  The series/movie plot is still from the anidb description, so only in English, that's just how that goes.
  • GetArt function added - I see some people pointing out these "errors" in the log.  It's not really an error, the GetArt function is a way for Kodi to update the artwork automatically, independent of a scan, it's primary purpose was so the user didn't have to refresh a series every time a new season was added in order to get the new season posters.  
    Since anidb shows are all one season, it didn't seem that necessary, plus the function would need the anidb id number to work, but at the time it was possible to switch the scraper to use the tvdb id (for compatibility with extrafanart addons and such), but there is no way for the function to reasonably distinguish which id it is given, since either way it's just some random number.  The scraper now ignores the tvdbid setting, and uses named uniqueids for both anidb and tvdb, and always defaults to anidb (the more up-to-date addons should look for the tvdb uniqueid now, so it doesn't matter if it's not the default), but if you were using the tvdbid before, some oddities may arise... possibly?  Refreshing should put the right uniqueids in, and fix anything.
  • There are other fixes/tweaks I idly made for myself in the past, I think, mainly in the genres and tagging areas.  General tagging is still off the table, but location, source, and audience tagging should work again (if they didn't(?)).
    Also if you see anything appear as a genre that definitely shouldn't be one, report it and it can be filtered.
  • By necessity for the tvdb api, the cachepersistence has been lowered from 48 hours to 24.  This shouldn't cause issues with anidb, 48 hours was always a little overkill, but... be aware.

My plan is to fix everything up the week after next, when I have some time, and push it around the 15th(ish).
You have until then to report any errors in the zip files or bugfixes I've missed, and to request any reasonable/achievable new features to consider.
A "reasonable/achievable" new feature would be e.g. adding a "disable ratings" setting  (consider that on the todo list).  Any tvdb-related features might also be doable with the new setup.
Unreasonable new features would be anything involving major rewrites, e.g. integrating themoviedb tv information, or just saying "make thing work better" without clearly stating what "better" is, or stuff that is outside the scraper's control. 
Awesome, welcome back. I'll check it out.
Reply


Messages In This Thread
RE: - by scudlee - 2013-10-12, 17:42
RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - by fighuass - 2020-02-01, 23:01
Logout Mark Read Team Forum Stats Members Help
[MOD] AniDB.net scrapers for TV shows and Movies8