Kodi Community Forum
Release [MOD] AniDB.net scrapers for TV shows and Movies - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305)
+----- Thread: Release [MOD] AniDB.net scrapers for TV shows and Movies (/showthread.php?tid=142835)



RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-02

Jarvis is a little too old for me to consider continue supporting. 
If it had been Krypton that was broken, I'd be tempted to come up with a workaround, but I've checked and multiple rating support was added in Krypton.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - NemRe - 2020-02-02

@scudlee It's okay, thanks anyway! ^^


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - Shogunreaper - 2020-02-04

thank you for updating! i was trying to use tmm to manually scrape stuff but that is such a pain for anime.

edit: just tried it, it works but it seems to be missing network and runtime information. All that appears for me is first aired and the synopsis.

also the genre tags seem to not get scraped, all it says is "anime" for me.

https://i.imgur.com/OwfAwDQ.jpg

i'm using krypton (since the dsplayer fork hasn't been updated)


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-04

Can't duplicate on Leia:
Image
There is another oddity in your image which suggests there's something going wrong on your end, which is the completely unfiltered series plot. 
It shouldn't matter what version you're running, that should just work.  The leading asterisk, the URL, The whole "Based on..." part now, too, should all be tidied up.

I'd need to see a debug log of you refreshing the show to be able to help further.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - Shogunreaper - 2020-02-06

(2020-02-04, 21:28)scudlee Wrote: Can't duplicate on Leia:
Image
There is another oddity in your image which suggests there's something going wrong on your end, which is the completely unfiltered series plot. 
It shouldn't matter what version you're running, that should just work.  The leading asterisk, the URL, The whole "Based on..." part now, too, should all be tidied up.

I'd need to see a debug log of you refreshing the show to be able to help further.

okay heres the log from when i started the rescan

https://pastebin.com/raw/0YmRH1BV


RE: [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-06

@Shogunreaper 
Code:
17:39:50.501 T:11024 DEBUG: VideoInfoScanner: Found matching full NFO file: H:\Anime\Enen no Shouboutai\tvshow.nfo
There's your issue.  You're not using the scraper.  All the details are just coming from the nfo files.
How or when they were generated, I don't know, but you need to delete them (and the episode nfo files) first, then try again.

If the problem still persists, I'll need to see another log of you refreshing without the nfo files.


RE: [MOD] AniDB.net scrapers for TV shows and Movies - fireba11 - 2020-02-06

@scudlee welcome back :-D
The fixes work nicely for me, thanks.

PS: If you're really motivated, is there a way to get https://anidb.net/anime/10729 to scrape? My guess is the problem is with the quotes, i never managed to get that added even with .nfo files -_-


RE: [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-06

@fireba11 See this post in the thread, where I mention the cause of, and possible workarounds for, that very issue.

(tl;dr:  I named my folder "Ore ga Ojou-sama Gakkou ni Shomin S_ample Toshite Gets-sareta Ken" and it worked fine.)


RE: [MOD] AniDB.net scrapers for TV shows and Movies - fireba11 - 2020-02-07

Indeed, somehow i missed that. Thanks, worked nicely.


RE: [MOD] AniDB.net scrapers for TV shows and Movies - stylus7 - 2020-02-14

@scudlee Some shows are still getting scraped with notes before the actual plot, the ones that don't start with "* Based on a..." (like 13299, 15024, 14076).

Also the the "* Based on a..." line is now at the end of the description. Would it possible not to have them at all? Thanks.


RE: [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-15

Okay, so quick update,  I've tidied the code up, and I've added in a setting to "Fetch Ratings" (@stylus7), and also added a "Use Pre-Krypton ratings" option (@NemRe ) (ehh... I figure, why not?).
I've also expanded the filtering of "based on" to all opening lines starting with a "*", and added a setting ("Keep source line in plot") to allow you to remove it completely (@stylus7 ).  I like having it at the end, so the default is true.

I'm a little behind where I wanted to be, as I foolishly rescanned my entire library and got myself banned from anidb (twice!).  The second time even after cranking the delay parameter way up.  So, fair warning to anyone planning on doing a full rescan.
Note the delay parameter is a measure of how much busywork the scraper does during a scrape to slow it down, not a direct measure of time. The more powerful your processor, the higher it needs to be to have any noticeable effect.  Even 250 for me was only adding less than 0.2s delay.  You may want that to be in 1-2 second range to be safe, especially if you have a lot of series with only a few episodes each. (It's also non-linear, so larger numbers should dramatically increase the delay.  In theory.)

Expect the scraper updates early Monday.


RE: [MOD] AniDB.net scrapers for TV shows and Movies - fighuass - 2020-02-15

(2020-02-15, 14:07)scudlee Wrote: Okay, so quick update,  I've tidied the code up, and I've added in a setting to "Fetch Ratings" (@stylus7), and also added a "Use Pre-Krypton ratings" option (@NemRe ) (ehh... I figure, why not?).
I've also expanded the filtering of "based on" to all opening lines starting with a "*", and added a setting ("Keep source line in plot") to allow you to remove it completely (@stylus7 ).  I like having it at the end, so the default is true.

I'm a little behind where I wanted to be, as I foolishly rescanned my entire library and got myself banned from anidb (twice!).  The second time even after cranking the delay parameter way up.  So, fair warning to anyone planning on doing a full rescan.
Note the delay parameter is a measure of how much busywork the scraper does during a scrape to slow it down, not a direct measure of time. The more powerful your processor, the higher it needs to be to have any noticeable effect.  Even 250 for me was only adding less than 0.2s delay.  You may want that to be in 1-2 second range to be safe, especially if you have a lot of series with only a few episodes each. (It's also non-linear, so larger numbers should dramatically increase the delay.  In theory.)

Expect the scraper updates early Monday.
Awesome, great work!


RE: [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-17

New Features in 2.5.0 (TV) and 2.3.0 (Movies)

The biggest feature is that it should all be working again. All the tvdb details and artwork are now fetched from the v2 api, neatly avoiding the zip issue.
Other fixes include improved plot filtering, minor genre tweaks, using the newer rating format, and uniqueids, some cosmetic improvements to the search results.
General tagging settings have been removed, but you can still add tags for source, location, and audience.
Audience tags should also allow for sorting of hentai titles by tags, since they should all be tagged "Audience: 18 restricted", so it could be used as a playlist/node rule (for example).

Other new settings that were added are mentioned in my previous post (so just scroll up).
All settings are documented in the readme.md files included in the addon folders, and are also visible on their github pages.
(There's a typo on the "Pre-Krypton ratings" description. It should say "for Jarvis and older", not "for Gotham and older". The alphabet is hard, guys.)


RE: [MOD] AniDB.net scrapers for TV shows and Movies - fighuass - 2020-02-17

(2020-02-17, 13:18)scudlee Wrote: New Features in 2.5.0 (TV) and 2.3.0 (Movies)

The biggest feature is that it should all be working again. All the tvdb details and artwork are now fetched from the v2 api, neatly avoiding the zip issue.
Other fixes include improved plot filtering, minor genre tweaks, using the newer rating format, and uniqueids, some cosmetic improvements to the search results.
General tagging settings have been removed, but you can still add tags for source, location, and audience.
Audience tags should also allow for sorting of hentai titles by tags, since they should all be tagged "Audience: 18 restricted", so it could be used as a playlist/node rule (for example).

Other new settings that were added are mentioned in my previous post (so just scroll up).
All settings are documented in the readme.md files included in the addon folders, and are also visible on their githubpages.
(There's a typo on the "Pre-Krypton ratings" description. It should say "for Jarvis and older", not "for Gotham and older". The alphabet is hard, guys.)
Once again thanks for your work, but I have a problem, I'm trying to install the addons but I keep getting an error about the addon failing to install a dependency (tried to manually install it and through your repo), is this just me or...?


RE: [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2020-02-17

If you're manually installing, you would need to install the common library first, as the two scrapers are both dependent on the new version (1.3.0).

I had assumed installing through the repo would resolve the dependency automatically.  Maybe not. 
I didn't really get to test that part though, because I had everything installed already (obviously).