[RELEASE] [MOD] AniDB.net scrapers for TV shows and Movies

  Thread Rating:
  • 8 Vote(s) - 4.5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
ZERO <ibis> Offline
Senior Member
Posts: 275
Joined: May 2010
Reputation: 1
Post: #796
Does anyone know a safe delay perm value? The default when I installed the scraper was 1 and I changed that value to 125 as per the first post in the thread but I still got banned.
find quote
Griever92 Offline
Junior Member
Posts: 8
Joined: Feb 2013
Reputation: 0
Post: #797
Recently update to v17 which somehow managed to nuke a bunch of my scraped info for shows.
I only managed to refresh 2 separate shows before getting the all too familiar ban message.

Still the one recurrent issue I've been unable to solve all these years.
Appreciate any suggestions/solutions, as I now have roughly 200 series to re-scrape, and getting banned every 2 is not an attractive option.
find quote
ZERO <ibis> Offline
Senior Member
Posts: 275
Joined: May 2010
Reputation: 1
Post: #798
I set it to 300 and it worked without me getting banned. I think mine loads really fast because I have the tvshow.nfo file with the aid values for every entry so I needed a big delay.
find quote
techterry Offline
Junior Member
Posts: 8
Joined: Jun 2015
Reputation: 0
Post: #799
i just upgraded to krypton on a linux system running ubuntu and i cant get this addon to install from superrepo. i select the addon and click install and it kicks me out of that window back to the listing of scrapers to choose to install. is this addon updated for krypton or is it something funky with my setup
find quote
gates Offline
Senior Member
Posts: 173
Joined: Jun 2010
Reputation: 3
Location: France
Post: #800
Use scudlee repo (see the first post).
(This post was last modified: 2017-02-15 20:14 by gates.)
find quote
Big2hd Offline
Junior Member
Posts: 21
Joined: Mar 2013
Reputation: 0
Post: #801
Hi scraper for some reason will find series info but not individual episodes of series http://pastebin.com/yp86McYU
find quote
Razzorx1 Offline
Junior Member
Posts: 6
Joined: Apr 2014
Reputation: 0
Post: #802
Hi, I've been using this scraper since XBMC 12 on Windows 7x64 and I had no problems.

After upgrading to Windows 10, Kodi now crashes every time I try to scrape something with your addon.
I tried using 15.1 and 17.1, the crash happens on both. Also, scraping the same folder with e.g. TVDB does work without problems.
Here is the log.
find quote
OtomoAkira Offline
Junior Member
Posts: 5
Joined: Sep 2014
Reputation: 0
Post: #803
@scudlee : The scraper is very good (Xem by takoi is able to identify many more titles, but the lack of integration with the AniDB genre & tags system is a showstopper).
The only problem is that it doesn't seem to remember the items already scraped. And every time I scrape my anime folders, it starts from the beginning scanning thousands of files, even If I added only ten or so new titles.
Is it normal? It's a pain to have to wait 15 minutes the scraper every time I add a new anime show to Kodi.
Note that it doesn't seem to forget the show that he has recognized. The scraper rescans only those titles that he didn't recognize. But unfortunately I have a very big collection and almost 40% of all titles are never been recognized by it. So hundreds of files are scanned again and again every time, even if it has already checked them with AniDB and found nothing. It seems that the files already scanned are not marked to avoid scraping them twice.
Is this only a problem with my installation or a limit of the scraper? And is there a way to avoid this? Thank you in advance!
(This post was last modified: 2017-04-18 22:41 by OtomoAkira.)
find quote
gates Offline
Senior Member
Posts: 173
Joined: Jun 2010
Reputation: 3
Location: France
Post: #804
hi OtomoAkira

It's not related to the scrapper.
Kodi (itself) try to scan all the show it dont know (even if already seen/failed), and use the configured scraper

I have 150+ anime/mangas, and they are all recognized.
Maybe you have to rename the failed anime with anidDB "main title" of the show
find quote
Razzorx1 Offline
Junior Member
Posts: 6
Joined: Apr 2014
Reputation: 0
Post: #805
@OtomoAkira:

You can try to rename the folders based on the main name of every show.
That should work most of the time, but I had a few cases where that wasn't working.

The other possibility is to search the anime on AniDB, copy the id and search it on scudlee's anime-list and use the name listed next to it, as it sometimes differs from the name listed on AniDB.
If the anime id can't be found on the list, scudlee hasn't added it yet, so you can't add that anime in Kodi.
find quote
costi Offline
Senior Member
Posts: 104
Joined: Jul 2010
Reputation: 0
Post: #806
(2017-02-08 01:23)ZERO &amp;lt;ibis&amp;gt; Wrote:  Does anyone know a safe delay perm value? The default when I installed the scraper was 1 and I changed that value to 125 as per the first post in the thread but I still got banned.

I second this question - what is the safe value nowadays? I'm at 150 but I still get banned, even after waiting a week between updates.
find quote
OtomoAkira Offline
Junior Member
Posts: 5
Joined: Sep 2014
Reputation: 0
Post: #807
(2017-04-19 09:53)gates Wrote:  it is not related to the scrapper.
Kodi (itself) try to scan all the show it dont know (even if already seen/failed), and use the configured scraper

This is a bad news. Why KODI has this stupid behaviour by default? Is there a mod or add on for Kodi that can avoid this useless behaviour? Rescanning all files all the times is demanding for my hd and I risk to be banned from AniDB!

Quote:I have 150+ anime/mangas, and they are all recognized.
Maybe you have to rename the failed anime with anidDB "main title" of the show

This is not feasible for me. I have more than two thousands files and most of them have a japanese title. I have no idea what they are. This a task too demanding for doing it manually. This is why a scraper is needed.
If the scraper was smarter (for example avoiding to include in the search the fansub group or the video details like codec used, mp4, sub, dub, eng, 480p, dvd, etc. or any other common tag in the file name, and it would interpret season and episodes numbers translating them to the standard S00E000 format, and using releases lists from releaser groups like XEM does) then the problem would not exist. But instead it sends the plain title as a search string to AniDB with no processing and consequently a lot of anime titles are not recognized just because there are fansub groups names in it.
find quote
Razzorx1 Offline
Junior Member
Posts: 6
Joined: Apr 2014
Reputation: 0
Post: #808
(2017-04-21 13:01)OtomoAkira Wrote:  This is a bad news. Why KODI has this stupid behaviour by default? Is there a mod or add on for Kodi that can avoid this useless behaviour? Rescanning all files all the times is demanding for my hd and I risk to be banned from AniDB!

Rescanning doesn't ban you if the show cannot be found, as it goes through the custom anime-list from scudlee first. If it doesn't find the show there, it won't event send a request to AniDB.

(2017-04-21 13:01)OtomoAkira Wrote:  This is not feasible for me. I have more than two thousands files and most of them have a japanese title. I have no idea what they are. This a task too demanding for doing it manually. This is why a scraper is needed.
If the scraper was smarter (for example avoiding to include in the search the fansub group or the video details like codec used, mp4, sub, dub, eng, 480p, dvd, etc. or any other common tag in the file name, and it would interpret season and episodes numbers translating them to the standard S00E000 format, and using releases lists from releaser groups like XEM does) then the problem would not exist. But instead it sends the plain title as a search string to AniDB with no processing and consequently a lot of anime titles are not recognized just because there are fansub groups names in it.

The names of the files are basically irrelevant, as it mostly searches by folder names. If your folder is named something like "[FFF] Hibike! Euphonium [BD][1080p-FLAC]", it will still only search for "Hibike! Euphonium" as it automatically excludes [ and (.
Also, you don't need the s01e01 format, you can set a custom regex to detect filenames which will work with the original names from the sub groups.
find quote
thefran Offline
Junior Member
Posts: 1
Joined: Apr 2017
Reputation: 0
Post: #809
I've tried pointing the thing at my anime folder. The result is: my library is completely empty. Creating an advancedsettings file as recommended in the OP fixes nothing.
(This post was last modified: 2017-04-22 11:48 by thefran.)
find quote
OtomoAkira Offline
Junior Member
Posts: 5
Joined: Sep 2014
Reputation: 0
Post: #810
(2017-04-21 17:38)Razzorx1 Wrote:  Rescanning doesn't ban you if the show cannot be found, as it goes through the custom anime-list from scudlee first. If it doesn't find the show there, it won't event send a request to AniDB.

Good to know, thanks. But there is still the need of a way to stop Kodi to scan repeatedly the same files. It is very painful to watch while it rescans all those files every time, and it is also very stressful for the HD.

Quote:The names of the files are basically irrelevant, as it mostly searches by folder names. If your folder is named something like "[FFF] Hibike! Euphonium [BD][1080p-FLAC]", it will still only search for "Hibike! Euphonium" as it automatically excludes [ and (.

I meant folder names, of course, not file names. Sorry. And sadly in my collection there are hundreds of folders with names with tags but with no parenthesis. If the scraper would be able to filter those tags and especially the fansub group names and codecs, without the need of parenthesis, then it will become really useful.
But currently it is only capable of identifying 50% of an average anime collection.

Does scudlee have any plan to fix this?

Quote:Also, you don't need the s01e01 format, you can set a custom regex to detect filenames which will work with the original names from the sub groups.

But the series of my collection seems to use many differents numbering systems. How can I write a regex that works for all?
(This post was last modified: 2017-04-22 23:20 by OtomoAkira.)
find quote
Post Reply