Kodi Community Forum

(2020-11-18, 21:58)olympia Wrote: [ -> ]Can you please tell me what is unstable at the moment with the non-Python scraper? I would like to fix it.

I don't know if it is any more, I was referring to the multiple recent updates that were needed to make it work again, while the Python one was working fine without any need for fixes.

Just thought I'd chime in here.

I also noticed the issue of Cast not available when scraping movies when the issue started almost a week ago.  

TMDB common scraper v3.2.8 seemed to have addressed that issue.  

I've since updated to TMDB common scraper v3.2.9 and now 3.2.10.

I'm not sure if v3.2.9 or 3.2.10 introduced a new issue or if TMDB made some additional changes on their end that may have messed up the scraping process further.

But what I'm noticing now is that when I scrape some movies, the first actor/actress in the cast list is missing completely.  So for example, I scraped the movie Bowling for Columbine after I renamed the file.  Michael Moore should be the first actor in the cast list according to TMDB.  When Kodi does the scrape, Michael Moore is nowhere to be found.  I'm not sure if it's the same issue that Karellen indicated in post #18 of this thread, because Bowling for Columbine is a documentary and for most of the cast, their character names just say Himself or Herself.  So it's possible that Michael Moore was bumped off the list, however, his character name of Himself was given to the next actor on the list which is George H. W. Bush who also has a character name of Himself.  Hope this makes sense.

Now if I select the Extended info add-on, it pulls up all the movie details, and Michael Moore is indeed listed as the first actor in the cast list with a character name of Himself.

Lets make things even more interesting.  I re-scraped the movie Lord of War where Nicolas Cage is the first actor listed.  The movie rescraped just fine.  Nicolas Cage is still the first actor listed with the correct character name of Yuri Orlov.

I'm guessing you'll want logs.  I'm also guessing you'll want verbose logging for the libcURL enabled, so I took the liberty of enabling that.  Here are the logs:

https://paste.kodi.tv/emecomotog.kodi

I rescraped Bowling for Columbine at 1:42PM Eastern Time on 11/19/2020.  Confirmed that Michael Moore is still missing from the cast list.  But when I go to Extended info, Michael Moore is listed there.

I rescraped Lord of War at 1:44PM Eastern Time on 11/19/2020.  Confirmed the cast list scraped just fine and Nicolas Cage is the first cast member on the list.

This is a snippet of what I found in the logs:

2020-11-19 13:43:26.872 T:22082 WARNING: using DialogBusy from python results in nop now
2020-11-19 13:43:30.468 T:22082   ERROR: NEWADDON Unknown Video Info Key "file"
2020-11-19 13:43:30.584 T:22082   ERROR: Previous line repeats 3 times.
2020-11-19 13:43:30.584 T:22082   ERROR: NEWADDON Unknown Video Info Key "media_type"
2020-11-19 13:43:30.820 T:22103   ERROR: Previous line repeats 19 times.
2020-11-19 13:43:30.820 T:22103   ERROR: Exception in thread Thread-3:
                                            Traceback (most recent call last):
                                              File "/home/jenkins/workspace/Android-ARM-Leia/tools/depends/xbmc-depends/arm-linux-androideabi-21-release/lib/python2.7/threading.py", line 801, in __bootstrap_inner
                                              File "/home/jenkins/workspace/Android-ARM-Leia/tools/depends/xbmc-depends/arm-linux-androideabi-21-release/lib/python2.7/threading.py", line 754, in run
                                              File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.extendedinfo/resources/lib/dialogs/DialogBaseInfo.py", line 259, in get_youtube_vids
                                                self.yt_listitems = youtube.search(search_str, limit=15)
                                              File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.module.kodi65/lib/kodi65/youtube.py", line 199, in search
                                                listitems = handle_videos(results["items"], extended=extended)
                                            KeyError: 'items'
2020-11-19 13:43:30.890 T:22101   ERROR: DoWork - Direct texture file loading failed for resource://resource.images.studios.white/Iconolatry Productions Inc..png
2020-11-19 13:43:30.895 T:22101   ERROR: DoWork - Direct texture file loading failed for resource://resource.images.studios.white/Salter Street Films International.png
2020-11-19 13:43:30.896 T:22101   ERROR: DoWork - Direct texture file loading failed for resource://resource.images.studios.white/TiMe Film- und TV-Produktions GmbH.png
2020-11-19 13:43:30.904 T:22099   ERROR: DoWork - Direct texture file loading failed for resource://resource.images.studios.white/Vif Babelsberger Filmproduktion GmbH & Co. Zweite KG.png
2020-11-19 13:43:30.911 T:22099   ERROR: DoWork - Direct texture file loading failed for resource://resource.images.studios.white/United Broadcasting Inc..png
2020-11-19 13:43:30.915 T:22102   ERROR: DoWork - Direct texture file loading failed for resource://resource.images.studios.white/Dog Eat Dog Films.png
2020-11-19 13:43:42.524 T:22082 WARNING: CPythonInvoker(3, /storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.extendedinfo/default.py): the python script "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.extendedinfo/default.py" has left several classes in memory that we couldn't clean up. The classes include: N9XBMCAddon7xbmcgui10DialogBusyE
2020-11-19 13:43:55.599 T:22099  NOTICE: CDVDVideoCodecFFmpeg::Open() Using codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
2020-11-19 13:45:14.494 T:21835 WARNING: Create - unsupported protocol(image) in image://Browse/
2020-11-19 13:45:14.494 T:21835   ERROR: Exists - Error checking for image://Browse/

I'll be honest, I'm not sure if the log lines are related to the scraping issue.  I'm just going by log lines that were captured around the time I reproduced the issue.

So just for my understanding, the issue that started about a month ago where movie posters failed to scrape was due to a change on CloudFare.  Now this particular issue where the cast list fails to scrape is due to a change on TMBD's side?  I believe it was mentioned there were changes to certain changes to properties for the cast list?  Whatever happened to the saying "If it's not broken, don't fix it!"

I'm also having an issue with scraping TV shows.  But I'm done troubleshooting for now.  That'll have to be dealt with another day.

(2020-11-19, 21:23)Crashtastic Wrote: [ -> ]I'm guessing you'll want logs.  I'm also guessing you'll want verbose logging for the libcURL enabled, so I took the liberty of enabling that.  Here are the logs:
https://paste.kodi.tv/emecomotog.kodi

You must have uploaded the wrong log as debug mode is not enabled, there is no libcurl, and there is no scraping.

@Crashtastic this should have been a VERY VERY rare issue, so messing up the scraping process further is a bit of an overstatement. I am not even sure if you could have provided an other example movie for this case.
However, this has been fixed with tmdb common scraper v3.2.11 and I hope the change because of this single movie was not actually now messing up anything else.

(2020-11-19, 21:49)Karellen Wrote: [ -> ]

(2020-11-19, 21:23)Crashtastic Wrote: [ -> ]I'm guessing you'll want logs.  I'm also guessing you'll want verbose logging for the libcURL enabled, so I took the liberty of enabling that.  Here are the logs:
emecomotog.kodi (paste)

You must have uploaded the wrong log as debug mode is not enabled, there is no libcurl, and there is no scraping.

Okay.  I didn't enable debug logging.  I just enabled component logging for libcURL specifically.  I went to Settings > Event Log > Settings > Debug > Specify component-specific logging > selected Verbose logging for the libcURL library (HTTP(S), DAV).

If I was supposed to Enable debug logging on top of this as well, then my apologies, I didn't do that.  

Also, I remember installing an add-on that allowed me to just select Upload or something like that to upload the logs, but I couldn't find it, so opened up a file explorer app and dug down to the Android/Data/org.xbmc.kodi/files/.kodi/temp directory and grabbed the kodi.log file from there.  

If this wasn't the correct steps, then I apologize.  I'll have to do it over.

(2020-11-19, 22:16)olympia Wrote: [ -> ]@Crashtastic this should have been a VERY VERY rare issue, so messing up the scraping process further is a bit of an overstatement. I am not even sure if you could have provided an other example movie for this case.
However, this has been fixed with tmdb common scraper v3.2.11 and I hope the change because of this single movie was not actually now messing up anything else.

I'll agree.  It's likely a very rare issue.  So maybe my statement is a little exaggerated.  My bad.  

However, would you believe me if I said I can find another example movie for this particular case?  Same issue happened with the movie Sicko which also happens to be a Michael Moore movie.  Michael Moore is missing from the cast list.  

Just to see if the issue may be affecting other Michael Moore movies, I tried re-scraping Fahrenheit 9/11.  No Michael Moore there either.  I tried re-scraping Fahrenheit 11/9.  Well, Michael Moore is missing from the cast list on that one as well.  Is this a Michael Moore issue?  Is it possible that his documentaries have upset someone at TMDB?  I'm just joking.  But it is a strange issue indeed.  However, it's not that big a deal if it's just affecting Michael Moore.  

If this peaks your interest and you want to look into it further, let me know.  If not, I totally understand.  Not everybody likes Michael Moore anyways.

(2020-11-19, 22:49)Crashtastic Wrote: [ -> ]If this peaks your interest and you want to look into it further, let me know.  If not, I totally understand.  Not everybody likes Michael Moore anyways.

As I said, I already fixed the issue in v3.2.11 of TMDB Common scraper

The issue was with Michael Moore is that TMDB included a new property for the cast list: "known_for_department" whereas the values can be "Acting", "Directing", whatever.
Initially I only included cast members from department "Acting" into the cast list of Kodi and since Michael Moore is known for "Directing" department, he was left out.
In v3.2.11 I added cast members from all departments, so everybody can be an actor now. Hence I said I hope this will not create more issues, in exchange to get back Michael Moore onto the map

I'm having issues scraping movie posters and fan art using both Kodi and media companion. Using either of these provides me with only imdb movie posters (showing the cast at the bottom etc.) Instead of providing the custom art that's available on tndb. Also, when using media companion to change poster and fanart (alternative) as well as change movie option, no posters are downloaded and the alternative poster scraper option shows no results for tndb. I'm not seeing media companion support, I'm just providing this information to possibly help confirm it is tmdb that's the culprit. Even using my personal api didn't help. I updated media companion, tried a fresh install of mc, and tried forcing Kodi to scrape from tmdb instead of using the .nfo+artwork to see if that helps and the issue remained regardless.

Quite an oddity occurring for a movie I recorded at the weekend that I just tried adding to my Kodi library.
Movie title is SUS (2010) and is listed here on the TMDB:-
https://www.themoviedb.org/movie/51918-sus

When adding to Kodi, the TMDB scraper added title as 'The Secret in their Eyes'  (2009). as listed here:-
https://www.themoviedb.org/movie/25376-e...e-sus-ojos
Strangely enough, this title is also listed as the first movie when you do a search of SUS on the TMDB site.
I tried doing a manual refresh 2-3 times, but kept returning this title when I selected SUS (2010). I then tried using upper case text when doing a manual search and a title SUS (1980) was listed. Although this was the wrong year, when I selected it the right movie appeared in Kodi much to my surprise, ie SUS (2010).

So yes, the problem is solved, but I'm curious why this might have happened? Currently using Kodi 18.8.

I was literally about to ask the same thing.  I've been moving shows from old 3TB drives to new 8TB drives so was re-scanning my TV Shows and The Blacklist wouln't scrape. tried everything I could think of, swapped to TVMaze and found it first time.

(2020-11-24, 16:58)Dumyat Wrote: [ -> ]Quite an oddity occurring for a movie I recorded at the weekend that I just tried adding to my Kodi library.
Movie title is SUS (2010) and is listed here on the TMDB:-
https://www.themoviedb.org/movie/51918-sus

When adding to Kodi, the TMDB scraper added title as 'The Secret in their Eyes'  (2009). as listed here:-
https://www.themoviedb.org/movie/25376-e...e-sus-ojos
Strangely enough, this title is also listed as the first movie when you do a search of SUS on the TMDB site.
I tried doing a manual refresh 2-3 times, but kept returning this title when I selected SUS (2010). I then tried using upper case text when doing a manual search and a title SUS (1980) was listed. Although this was the wrong year, when I selected it the right movie appeared in Kodi much to my surprise, ie SUS (2010).

So yes, the problem is solved, but I'm curious why this might have happened? Currently using Kodi 18.8.

I'm getting the same thing with a lot of titles. Started happening last night. Roughly half the titles are identified wrong and when you try a manual refresh, selecting the right title, it still reverts to something else entirely. Was working correctly a few days ago, so something might have changed on TMDB end.

I should mention this is with 3.2.11. Some examples.

Right Title => Scrapped Title

Triangle (2009) => The Truth Behind: The Bermuda Triangle (2009)
Transsiberian (2008) => Trans-Siberian by Private Train (1601)
Mother (2009) => My Mother's Coat (2009)
Extract (2009) => Basic Brewing - Introduction to Extract Home Brewing (2005)
Departures (2008) => The Museum Of Departures (2015)

Again, doing a manual refresh and selecting the correct title is always returning the wrong title.

(2020-11-24, 19:49)cidtek Wrote: [ -> ]I'm getting the same thing with a lot of titles. Started happening last night. Roughly half the titles are identified wrong and when you try a manual refresh, selecting the right title, it still reverts to something else entirely. Was working correctly a few days ago, so something might have changed on TMDB end.

We are looking into it... https://forum.kodi.tv/showthread.php?tid...pid2990893

(2020-11-20, 22:23)olympia Wrote: [ -> ]

(2020-11-19, 22:49)Crashtastic Wrote: [ -> ]If this peaks your interest and you want to look into it further, let me know.  If not, I totally understand.  Not everybody likes Michael Moore anyways.

As I said, I already fixed the issue in v3.2.11 of TMDB Common scraper

The issue was with Michael Moore is that TMDB included a new property for the cast list: "known_for_department" whereas the values can be "Acting", "Directing", whatever.
Initially I only included cast members from department "Acting" into the cast list of Kodi and since Michael Moore is known for "Directing" department, he was left out.
In v3.2.11 I added cast members from all departments, so everybody can be an actor now. Hence I said I hope this will not create more issues, in exchange to get back Michael Moore onto the map

My apologies.  I must have missed the part about it being fixed in v3.2.11.  It was a long day and I was tired at the time.  I appreciate the quick updates.  Thank you.

Although, in a small way, I was hoping it stayed broken, so I'd have an excuse to blame Michael Moore for the issue.  Just kidding.  

A fix has been released. See here... https://forum.kodi.tv/showthread.php?tid...pid2991031