Kodi Community Forum
Release [MOD] AniDB.net scrapers for TV shows and Movies - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: TV Show Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=305)
+----- Thread: Release [MOD] AniDB.net scrapers for TV shows and Movies (/showthread.php?tid=142835)



RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - ZERO <ibis> - 2013-05-14

My issue was that I did not "update" the dabase but instead copynpasted the only one in and gave it the correct name. I ran an upgrade on the database and that resolved my issue.

Thanks for adding the delay pram for the tv shows. All my shows have .nfo links to the correct anidb index to ensure they always scrap to the correct destination. Currently on my local system if I try to scrap my entire database in as a test I get banned pretty fast without the delay it will praise over 3.6TB of shows (over 550 series) in a min or 2. I assume that most people do not run into this issue b/c their raid array is not that fast or not having the .nfo links makes auto updating much slower.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2013-05-14

@eurofag: Could you check C:\Users\Eurofag\AppData\Roaming\XBMC\cache\scrapers\metadata.tvshows.anidb.net.mod and see if there's an animetitles.xml there. It should be about 3MB in size (so don't post the contents).

I have a feeling it's there but broken. If that's right, deleting it might fix your problems.

@ZERO <ibis>: I'd say it's the nfo files (forgot about that possibility), since they will bypass the search (and so speed things along).


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - User 149741 - 2013-05-14

Apparently it works now, thanks.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - ZERO &lt;ibis&gt; - 2013-05-19

Any eta on 2.0?


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2013-05-19

Hopefully this evening (it's 10:00am currently, for reference). A couple of last minute SNAFUs. One thing that had previously been working suddenly decided not to work (now fixed), and one thing I just swapped around isn't working (currently fixing) (fixed - a stupid stray "+" wreaking havoc).

Note: This will be a download and manual install only for now. Once a few suckers volunteers have tested it and pointed out any bugs, I'll push it to the repo for everybody.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2013-05-19

...Oh wow...

I've just realized another major source of ban occurrences.

DO NOT USE THE GOOGLE SEARCH OPTION!

Every hit in the Google search causes a call to AniDB (to get the proper titles). Too many calls too fast = ban.

Can't believe I didn't notice this before! (I even rewrote/reorganized the code for 2.0.0.)

I'm going to have to gut the Google search out for the RC... I've already got half a work-around in mind, but I want to stick to getting it out by today.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - jbsoum - 2013-05-19

Hey all, I've been back and forth about XBMC for awhile, and one of the major hurdles for me is anime. The thing that keeps me locked into MP at the moment is the My Anime 3 plugin for MediaPortal. I like this plugin because it scrapes data from anidb via file hashes, so whether or not my files get recognized is independent of naming convention. Every now and then there is a corrupt file hash, but that's where scraping by name is a nifty backup.

Is that something that's possible in XBMC?


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2013-05-19

Not currently. There might be some possibility in the advent of Heimdall, but that's a way off (probably).

The best compromise is to use an AniDB client to use the hashes to rename the files to an XBMC compatible format first. There's details on that in the first post.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - jbsoum - 2013-05-19

(2013-05-19, 17:17)scudlee Wrote: Not currently. There might be some possibility in the advent of Heimdall, but that's a way off (probably).

The best compromise is to use an AniDB client to use the hashes to rename the files to an XBMC compatible format first. There's details on that in the first post.

That is a shameSad

So I would basically have to rename all of my files, and then rename things as I acquire them. Perhaps I'll wait a bit longer to make the switch. Thanks for the info!


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - scudlee - 2013-05-19

Okay, here it is:
AniDB.net [MOD] scrapers 2.0.0rc1
Official Release is now available. Download the repo in the first post.

Inside the zip file you'll find:
metadata.common.anidb.net
metadata.movies.anidb.net.mod
metadata.tvshows.anidb.net.mod

To install just copy all three folders directly into your addons folder (on Windows this is %APPDATA%/XBMC/addons/, other OSes will vary, but it should be near your userdata folder (wiki)). Don't copy them into your program folder.


I will do a full feature list with the official release, but there (perhaps) surprisingly aren't that many new features, the major work was on rewriting and reorganizing the code into the shared common scraper library. All the various details are now in their own separate functions, making it much easier to maintain and add new stuff (the debug logs are much more readable too).

Highlights:
TV:
The only major changes to TV are the inclusion of season artwork (I added this since linking a movie to a TV show will put you in a season view, so nice to have artwork if available), and the delay parameter to slow things down for people getting banned (defaults to 1, so be sure to change that if needed).

Movies:
Trailer support (via themoviedb.org)
Certification support (again via themoviedb.org)
Movie set support - Movie sets are now by default taken from an anime-movieset-list.xml instead of themoviedb.org. You can switch back to using themoviedb.org sets if you want, or use both with one as back up. If you already turned off sets, this will be respected. It's also possible to use a personal set list... But more details on that later. As with the mapping list, any help maintaining this would be greatly appreciated (again, details to come).

Other changes:
The mapping list URL and the animetitles.xml URL will be reset to their defaults, so if you've changed either of them, you'll need to redo it.
The genre handling has been completely rewritten, so if you changed the "Return only Genres with weight" setting, you'll need to change the new "Minimum Genre weight" setting to the corresponding value. If you notice any oddities with the genres please let me know, it should be more-or-less identical.
Posters and banners are now always scraped, so those settings have been removed.
Google search has been removed (see my previous post).
The scrapers are now no longer Eden-compatible. It's Frodo or broke! (Literally).



Things not yet done: Updating the Changelog and adding READMEs. (@swe-divx: Before you ask, it will be with the official 2.0.0 release. Wink)


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - ZERO &lt;ibis&gt; - 2013-05-19

(2013-05-19, 16:53)jbsoum Wrote: Hey all, I've been back and forth about XBMC for awhile, and one of the major hurdles for me is anime. The thing that keeps me locked into MP at the moment is the My Anime 3 plugin for MediaPortal. I like this plugin because it scrapes data from anidb via file hashes, so whether or not my files get recognized is independent of naming convention. Every now and then there is a corrupt file hash, but that's where scraping by name is a nifty backup.

Is that something that's possible in XBMC?

What I would recommend is creating a tvshow.nfo file with contents of for example: aid=5571 for Hidamari Sketch X 365 so that you have 100% assurance that the correct show is scrapped for each directory. Yes you will need to rename shows but I have found that not using hashed based checking is great for those times when you need to mux your own versions.

When I first converted my collection over for XBMC I used Lammer Context Menu this program address back renaming to your context menu and many other useful features.

I recently manually created the nfo files for my over 577 series and am glad that I am able to re scrap my library so fast.

If you ever do make the switch these may interest you:
Back Row Frodo Anime Mod
MPC-HC Ultimate Setup Guide
Back Row Custom category in home menu
View the last page if you just want your anime label to replace tvshows all together so you can get the submenu options ect.
External players
Required fix for using MPC-HC with xbmc

If your really into anime and want the best playback and library options your going to want to read up on those links. I hope this helps you one day make the switch!


On another note, please post that test version when ready and I will try it out on my main system (which I use for xbmc testing) I will have it scrap my entire collection and it either works without banning me or blows up lol.

EDIT: I see that you made your post for version 2 when I typed this but I only know saw it. Will check it out as soon as I get a chance Big Grin


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - jbsoum - 2013-05-20

(2013-05-19, 20:28)ZERO &lt;ibis&gt; Wrote:
(2013-05-19, 16:53)jbsoum Wrote: Hey all, I've been back and forth about XBMC for awhile, and one of the major hurdles for me is anime. The thing that keeps me locked into MP at the moment is the My Anime 3 plugin for MediaPortal. I like this plugin because it scrapes data from anidb via file hashes, so whether or not my files get recognized is independent of naming convention. Every now and then there is a corrupt file hash, but that's where scraping by name is a nifty backup.

Is that something that's possible in XBMC?

What I would recommend is creating a tvshow.nfo file with contents of for example: aid=5571 for Hidamari Sketch X 365 so that you have 100% assurance that the correct show is scrapped for each directory. Yes you will need to rename shows but I have found that not using hashed based checking is great for those times when you need to mux your own versions.

When I first converted my collection over for XBMC I used Lammer Context Menu this program address back renaming to your context menu and many other useful features.

I recently manually created the nfo files for my over 577 series and am glad that I am able to re scrap my library so fast.

If you ever do make the switch these may interest you:
Back Row Frodo Anime Mod
MPC-HC Ultimate Setup Guide
Back Row Custom category in home menu
View the last page if you just want your anime label to replace tvshows all together so you can get the submenu options ect.
External players
Required fix for using MPC-HC with xbmc

If your really into anime and want the best playback and library options your going to want to read up on those links. I hope this helps you one day make the switch!


On another note, please post that test version when ready and I will try it out on my main system (which I use for xbmc testing) I will have it scrap my entire collection and it either works without banning me or blows up lol.

EDIT: I see that you made your post for version 2 when I typed this but I only know saw it. Will check it out as soon as I get a chance Big Grin

Well I wouldn't mind initially renaming everything once more, but I download new episodes and shows via rss feeds a lot. So if I was to make the switch, it seems like I wouldn't be able to download these automatically anymore. Instead, I'd have to be cautious about the naming of subdirectories and files for everything I downloaded.

Aside from the file hashing to add episodes to my database, I've been really spoiled by all of the features of the My Anime 3. XBMC does have a pretty interface and more plugin options than MediaPortal, which is why I've been strongly considering a switch lately.

Yes, I am very much into anime, I think I have about 3TB of it as of now. That's why anime management is so important to me in a media center.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - ZERO &lt;ibis&gt; - 2013-05-20

If you use the .nfo files as you should always the only thing that matters is that the shows and subsidiary have the same name and that you follow the s01exx naming system. I also use rss but dl manually b/c for some groups I like to read the release notes n shit. I then even set that as the rss feed in xbmc so that I can even see on my tvpc when there is a new ep out: http://pipes.yahoo.com/pipes/pipe.run?_id=7cc4e2a417c77f74356e7285f1a0bb08&_render=rss

I am actually going to be making a vid of my build later this week and will send it to you as well so you can get an idea of what is possible with xbmc.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - jbsoum - 2013-05-20

I do not use nfo files. What benefit is there to using them? If it's just for the sake of adding things to my database then I really don't have a need for that in MP.

The automatic addition of files to a database is something I really rely on. I think giving that up would be an honest deal breaker for me.

I would appreciate the video! I'm not trying to be closed minded, so I would really like to know what I can do with my anime collection in XBMC!

For the sake of sharing links:

https://code.google.com/p/jmm/

That is what I am currently using now. It's called Japanese Media Manager, but it's a suite of programs that My Anime 3 (what I use the most) is a part of. It's really the only thing about MediaPortal that I love. I, as well as a few others, have been trying to nudge the programmer to support it, but he says that it would be difficult to work around the fact that XBMC won't let My Anime 3 have it's own database.


RE: [RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies - ZERO &lt;ibis&gt; - 2013-05-20

I tested the beta version and for back row fan art view it is now for some series not displaying the description of the series and instead shows for example in the case of "Brother, Dear Brother" it will display an image instead which is really annoying and I am unable to yet figure out how to disable it.

Out of 577 series the following are effected by this issue:
Brother, Dear Brother
Senki Zesshou Symphogear
Toward the Terra

I am not sure yet as to why only these 3 are effected.

On another note would it be possible to have unique cast images per series and use the images of the characters?