Kodi Community Forum - Light IMDb Ratings Update 5.1.2

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74

I Have one issue to report. After some time scanning the progress bar disappears and if I try to run again it says "its already running please wait" the thing is that it is taking more than 4 hours for 700 TV Shows...too long

(2017-10-19, 17:24)djhifi Wrote: [ -> ]I Have one issue to report. After some time scanning the progress bar disappears and if I try to run again it says "its already running please wait" the thing is that it is taking more than 4 hours for 700 TV Shows...too long

Yes, this is socket timeout problem. After that addon crashed and not clean info about self-running.

(2017-10-19, 17:30)dziobak Wrote: [ -> ]
(2017-10-19, 17:24)djhifi Wrote: [ -> ]I Have one issue to report. After some time scanning the progress bar disappears and if I try to run again it says "its already running please wait" the thing is that it is taking more than 4 hours for 700 TV Shows...too long

Yes, this is socket timeout problem. After that addon crashed and not clean info about self-running.

How can I test it to report it?

EDIT:

axlt, sorry but no further info is added in the first post that I can see Smile

Fast fix (I try this now) with infinite (not good) loop:

https://pastebin.com/fywJCJ8P

As soon as I get home. Altough probably axlt is going to see this first.

Next bug:
Error Type: <type 'exceptions.IndexError'>
Error Contents: list index out of range

https://pastebin.com/mQFx3gur

In this movie there is no rating (IMDB: Needs 5 Ratings)

----------
fast fix:
https://pastebin.com/PggnNjAS

How many movies and shows do you have dziobak?

(2017-10-19, 20:10)djhifi Wrote: [ -> ]How many movies and shows do you have dziobak?

This is bug in IMDB page parsing with no rating (votes < 5), not in movies/shows number in library.
This bug shows on 85% of progress, so I try now from start. When all will be ok, then I drop addon 3.0 with my fixes. Or maybe Axlt will be first Wink

Light IMDb Ratings Update 3.0.0 with fixes:
- fixed socket timeout
- fixed error when no IMDB rating

[link removed, 3.1.0 is available]

IMDB responses are slow, with large library addon works few hours, so new ideas:
- skip watched movies/tvshows/episodes (config option for this)
- threads for IMDB queries

Next bug (in tvshow part):

TVDB has no IMDBId for series/episode, get_IMDb_ID gives empty result (no None like expected), then:

00:42:21.528 T:1458353056 DEBUG: [Light IMDb Ratings Update] - Obtaining IMDb ID from TheTVDB site...
00:42:22.574 T:1458353056 DEBUG: [Light IMDb Ratings Update] - IMDb ID:
00:42:22.912 T:1458353056 DEBUG: [Light IMDb Ratings Update] - Error accessing IMDb site (HTTP Error 404: Not Found)

fast fix:
update_main.py, line 135
- if IMDb == None:
+ if IMDb == "" or IMDb == None:

After night addon successfully finished work:
07:00:36.741 T:1552933792 DEBUG: [Light IMDb Ratings Update] - Ratings update for TV Shows finished

Light IMDb Ratings Update 3.0.1:
- fixed socket timeout
- fixed error when no IMDB rating (votes < 5)
- fixed error when TheTVDB has no IMDBId for tvshow/episode

[link removed, 3.1.0 is available]

IMDB responses are slow, with large library addon works few hours, so new ideas:
- skip watched movies/tvshows/episodes (config option for this)
- threads for IMDB queries
- limit queries for movies/tvshow from selected years (i.e. 1 year - 2017 only, 2 years - 2016+2017) - rating not changes for old (config option for this)

You are the man! Im testing the addon on my 700+ TV Shows first.

just one thing, when you say: "IMDB responses are slow, with large library addon works few hours" you mean the addon takes A LOT of time on big collections, is that it?

I have 5000+ movies and 700+ TV Shows Tongue

All those changes you made are not in the gui, so we need to edit which .py or thoise are just ideas?

PS: I have an idea as well, introduce a "resume" feature. Some old RATINGS addons had it. It is useful for big collections.

EDIT:

Update, Im running the scan in my TV Shows, its going but "Casual" (tt4577466 its a show) it's always showing up. For example: "almost human" then "casual" "almost human" again, then "casual", and moves to the next show, "showname" "casual" "showname" "casual" moves to the next show again.

Im gonna check the log. But all the IDs on my database are ok for that show. Im trying to find it in the normal logs but cant even see the addon working wtf.

Light IMDb Ratings Update version 3.1.0 AKA "Speed Runner"

Very high speed improvement version. In my case full library refreshing changed from over 10h to 1h (RPi2, over 800 movies and 130 tvshows).

Changes:
- Added threading for IMDb requests (defaults for 4 threads)
- Removed BeautifulSoup (very slow on RPi2, 5-6s for every request), changed HTML processing to few regexp
- Changed IMDb site to akas
- Added gzip support to IMDb requests -> less bytes = fast response

Fixes:
- fixed crash when socket timeout (loop added)
- fixed crash when no IMDB rating (votes < 5)
- fixed crash when TheTVDB has no IMDBId for tvshow/episode
- sometimes (for old shows?) TheTVDB returns IMDb_ID without leading "tt" - fixed
- changed behavior when errors on IMDb connection (try few times then return None -> skip problematic entry instead of doing infinite loop)
- fixed other minor bugs

Download 3.1.0_fixes_5 (temporary):
[3.1.0 version for download is on first page]

@djhihi
try above version first, maybe speed will be ok for full scanning;
if not, better idea is skipping old (or watched) entries than implement resume (for few reasons)

Code:
03:18:40.389 T:545031934176   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--

                                            - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!

                                           Error Type: <type 'exceptions.IndentationError'>

                                           Error Contents: ('unindent does not match any outer indentation level', ('/storage/.kodi/addons/script.light.imdb.ratings.update/resources/core/imdb_scraper.py', 50, 24, '      if do_loop == 10:\n'))

                                           Traceback (most recent call last):

                                             File "/storage/.kodi/addons/script.light.imdb.ratings.update/context.py", line 10, in <module>

                                               from resources.core import update_context

                                             File "/storage/.kodi/addons/script.light.imdb.ratings.update/resources/core/update_context.py", line 13, in <module>

                                               from imdb_scraper import parse_IMDb_page

                                           IndentationError: ('unindent does not match any outer indentation level', ('/storage/.kodi/addons/script.light.imdb.ratings.update/resources/core/imdb_scraper.py', 50, 24, '      if do_loop == 10:\n'))

                                           -->End of Python script error report<--

03:18:45.738 T:545193951456   ERROR: GetDirectory - Error getting

03:18:54.812 T:547992162304   ERROR: Previous line repeats 1 times.

I installed version 3.1.0 AKA "Speed Runner" and tried to update a tv show. Am I the only one getting this error?
Libreelec 8.1.2 (Odroid_C2.aarch64)
Thank you for your work so far!

(2017-10-21, 13:16)Glottis Wrote: [ -> ]I installed version 3.1.0 AKA "Speed Runner" and tried to update a tv show. Am I the only one getting this error?
Libreelec 8.1.2 (Odroid_C2.aarch64)
Thank you for your work so far!

Pls re-download, I changed this version few times (small bugs fixed), actual has fixes_5 in zip name.