Kodi Community Forum

Full Version: Movie scraper not working in XBMC Gotham 13.1
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3
(2020-08-25, 10:57)Karellen Wrote: [ -> ]@dozerplex @fourthreetwo

I am not familiar with v13, but where you enable Debug Mode in settings, is there an option there for Component Logging? If yes, look for libcURL and enable it and provide a log the same as your previous log.

Looks like this in current versions of Kodi... https://kodi.wiki/view/Log_file/Easy#Ena...ic_logging

Thanks

@dozerplex can you use Kodi Paste Site. The service you are using has no line numbers which makes it hard when a few of us are trying to review it.
Hi @Karellen Thank you for taking the time to try and help. Below are two new logs. There is no such thing as Component Logging in Gotham. There is however something called Verbose Logging, for this i have the following choices:

SMB library
CURL library (http, dav)
CMYTH library
FFMPEG libraries
RTMP library
DBUS library
JSON-RPC requests
AUDIO component

...sadly i have little understanding of what all this means but i made one log using the CURL setting (since your suggestion had curl in the name Smile) and also one "standard" log. Both with version 4.1.9 of the scraper. Not sure it matters but it's showing as 4.1.9 (blacklisted) when i check the rollbacks.

Should it be important i'm on XBMC 13.0 Git:20140504-0d1d803

Log WITH CURL:
https://paste.kodi.tv/afadepahef.kodi

Normal log:
https://paste.kodi.tv/aropixetir.kodi

Thanks again for your time and help, very much appreciated! Nod
Morning! Sorry for the late reply. 

https://paste.kodi.tv/ozavudakiz.kodi

Logged using Verbose (CURL). 

If it helps I can get Wireshark running to see if that can delve a bit deeper into what url it is failing to hit?
Thanks @fourthreetwo
You assumed correctly. Unfortunately you did not scrape a movie, so there was nothing to see.

Thanks @dozerplex
That is exactly what is needed. Running Wireshark would be a good move. It might provide more info for when the Europe based developers come on-line again tomorrow.
@dozerplex @fourthreetwo

Our core developer @Fuzzard has looked at the issue and has stated this is not an issue with Kodi or the scraper. It is an issue either with your ISP / DNS provider or TheMovieDB. It is possible that the servers in your part of the world have not synced with whatever changes have been made at TheMovieDB and hopefully it may resolve itself in a few days.

Further he believes that TheMovieDB have implemented changes described here... https://codefetti.com/how-to-fix-tlsv1-a...l-version/ If this is the case, then your old v13 install is too old to be updated to the new protocol.

I have also posted at TheMovieDB so you can monitor their response... https://www.themoviedb.org/talk/5f45a22f...0033c5e926

I think that is as far as we can go.
(2020-08-26, 02:01)Karellen Wrote: [ -> ]@dozerplex @fourthreetwo

Our core developer @Fuzzard has looked at the issue and has stated this is not an issue with Kodi or the scraper. It is an issue either with your ISP / DNS provider or TheMovieDB. It is possible that the servers in your part of the world have not synced with whatever changes have been made at TheMovieDB and hopefully it may resolve itself in a few days.

Further he believes that TheMovieDB have implemented changes described here... https://codefetti.com/how-to-fix-tlsv1-a...l-version/ If this is the case, then your old v13 install is too old to be updated to the new protocol.

I have also posted at TheMovieDB so you can monitor their response... https://www.themoviedb.org/talk/5f45a22f...0033c5e926

I think that is as far as we can go.
Hey Karellen, 

Thanks for getting back to us. 

I've just grabbed a Wireshark log if you still believe it will help? Let me know if you have a preferred host I can dump it to. I can see a lot of TLSv1.2 Application Data entries when I start scraping, so I believe you're right and TMDB have made changes to their protocol connections. Seeing as I'm running Kodi on the same machine, using Google DNS, I'm sure it isn't geographical. 

I've been an enthusiast since the original Xbox and an avid daily user since Eden and I can't thank you and @Fuzzard enough for looking into this for us. You and I have had some discussions in the past and it's always been a pleasant experience.

Looks like I'll be migrating to NFO's unless you have a better suggestion Tongue 

Thanks again guys, really appreciate it.
Thanks for the kind words @dozerplex

You can post the Wireshark log to Kodi Paste Site or to any site you prefer as long as we don't need to sign up for an account. At this stage, the log will only be out of curiosity for us as there is nothing we can do.

Probably NFO files is the best way, but I am concerned by the number of differences between a v13 nfo and database structure compared to the v18 nfo and database structure. There have been a lot of changes since v13, and one of those changes affect the critical UniqueID field which is a big concern for TV Shows.

If you have a Kodi generated v13 NFO file handy (from a Kodi export), I can check the critical tags and let you know if v18 nfo files are backwards compatible with v13 database.
(2020-08-26, 01:33)Karellen Wrote: [ -> ]Thanks @fourthreetwo
You assumed correctly. Unfortunately you did not scrape a movie, so there was nothing to see.

Thanks @dozerplex
That is exactly what is needed. Running Wireshark would be a good move. It might provide more info for when the Europe based developers come on-line again tomorrow.

Hi @Karellen well that was stupid of me, sorry about that.  Good that you got what you needed from @dozerplex  Smile
(2020-08-26, 02:51)Karellen Wrote: [ -> ]Thanks for the kind words @dozerplex

You can post the Wireshark log to Kodi Paste Site or to any site you prefer as long as we don't need to sign up for an account. At this stage, the log will only be out of curiosity for us as there is nothing we can do.

Probably NFO files is the best way, but I am concerned by the number of differences between a v13 nfo and database structure compared to the v18 nfo and database structure. There have been a lot of changes since v13, and one of those changes affect the critical UniqueID field which is a big concern for TV Shows.

If you have a Kodi generated v13 NFO file handy (from a Kodi export), I can check the critical tags and let you know if v18 nfo files are backwards compatible with v13 database.

Sorry for the late reply, got called into work. 

I'm having success with Ember Media Manager to generate NFO's for V13 so I can just use that until that also becomes Legacy software and stops working. Hopefully it hasn't already with this TMDB update... 

RE V18 NFO compatibility, I don't want to bother you guys anymore than I have, so I'll just run a test on my end at some stage and see how I go!

Thanks again for the support!
Thank you @Karellen @Fuzzard and @dozerplex for your time and effort in trying to get this resolved. I will follow the thread at themoviedb and see what happens, thank for posting the issue there. I will also start using NFO's, at least until the TV scraper stops working. If/when that happens it might be time for a change of hardware. Smile

Thanks again to all involved for you friendly and helpful approach, much appreciated.
One last (stupid) question for now while we're on the subject. When using NFO's, should themoviedb scraper be disabled or is that irrelevant?
(2020-08-26, 08:49)fourthreetwo Wrote: [ -> ]One last (stupid) question for now while we're on the subject. When using NFO's, should themoviedb scraper be disabled or is that irrelevant?
It is irrelevant. Leave it enabled, it does not matter. Kodi scans the Source looking for the movie file, when it finds the file, it then looks for the nfo file and artwork. Then, and only then, if it does not find either or both, it calls up the scraper.

So if you have both nfo files and artwork, the scraper is never used.
(2020-08-26, 09:14)Karellen Wrote: [ -> ]
(2020-08-26, 08:49)fourthreetwo Wrote: [ -> ]One last (stupid) question for now while we're on the subject. When using NFO's, should themoviedb scraper be disabled or is that irrelevant?
It is irrelevant. Leave it enabled, it does not matter. Kodi scans the Source looking for the movie file, when it finds the file, it then looks for the nfo file and artwork. Then, and only then, if it does not find either or both, it calls up the scraper.

So if you have both nfo files and artwork, the scraper is never used.
Great, thanks again for all your help.
Hi again @Karellen, the thread at themoviedb appears to have been removed. Just curious if you know if anything was posted? I’ve made the move to NFO’s and it’s working great so all good here regardless. Smile
(2020-08-27, 22:33)fourthreetwo Wrote: [ -> ]Hi again @Karellen, the thread at themoviedb appears to have been removed. Just curious if you know if anything was posted? I’ve made the move to NFO’s and it’s working great so all good here regardless.
I have asked the Moderators there to make the thread public again. Hopefully it will be visible in the next day or so. The outcome is that TheMovieDB updated to TLS 1.1 back in March 2019, so you had a good run while it lasted. It should have broken back then.
@fourthreetwo The best workaround for the TMDB I've discovered has been switching to Universal Movie Scraper - It will grab the info/synopsis of movies but you'll need to manually change Fanart/Poster/Banner/Whatever manually which I usually did anyway as Fanart.tv has higher quality files. 

Seems to be the most painless workaround.
Pages: 1 2 3