Kodi Community Forum

Full Version: TheMovieDB movie scraper - PYTHON version
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12
(2021-01-23, 01:43)sualk Wrote: [ -> ]Another great option would be to set a fallback language for fanart.tv like in Artwork Beef.
There is. Check the settings for the scraper.
(2021-01-23, 01:56)Karellen Wrote: [ -> ]
(2021-01-23, 01:43)sualk Wrote: [ -> ]Another great option would be to set a fallback language for fanart.tv like in Artwork Beef.
There is. Check the settings for the scraper.

Thanks for the answer. I can only find the setting for the preferred language in the scraper fanart.tv settings, but not for a fallback language. I am using version 1.3.1
Ah, fallback language. Sorry, scanned your post too quickly.

Nope, there is not one, but the developer will see your suggestion above and will decide.
(2021-01-23, 03:25)Karellen Wrote: [ -> ]Ah, fallback language. Sorry, scanned your post too quickly.

Nope, there is not one, but the developer will see your suggestion above and will decide.

Great !!!!
How do I get the IMDB ID when using this scraper?
using this will just give me the TMDB ID
xbmc.getInfoLabel("VideoPlayer.IMDBNumber") ## e.g. returns 209112
I'm getting MB and MB of the below lines in the log using this scraper on Kodi 19

Code:
2021-03-03 15:07:02.537 T:408   WARNING <CAddonSettings[metadata.themoviedb.org.python]>: failed to load value "en" for setting language
2021-03-03 15:07:02.538 T:408   WARNING <CSettingsManager>: unable to read value of setting "language"

And bit confused as 1st post says this doesnt work on 19, but it is included with 19...
(2021-03-03, 04:07)matthuisman Wrote: [ -> ]And bit confused as 1st post says this doesnt work on 19, but it is included with 19...
Thanks. I have deleted those comments. They were added in the early days of this scrapers developement.

(2021-03-03, 04:07)matthuisman Wrote: [ -> ]I'm getting MB and MB of the below lines in the log using this scraper on Kodi 19

Code:
2021-03-03 15:07:02.537 T:408   WARNING <CAddonSettings[metadata.themoviedb.org.python]>: failed to load value "en" for setting language
2021-03-03 15:07:02.538 T:408   WARNING <CSettingsManager>: unable to read value of setting "language"
You must have had this installed for quite a while. There were some updates to language codes forced by TheMovieDB. Can you follow this... https://forum.kodi.tv/showthread.php?tid...pid2983546 obviously for this scraper, not the ones mentioned in the post.
What is MB?
oh, im using headless kodi so starting from "scratch" just with default add-on settings

metadata.themoviedb.org.python v1.3.1+matrix.1 installed

and the default settings its created are below
Looks like language is correct

i also got same sort of errors using the TvDB python scraper...
maybe its a headless issue

oh wait, I know!

to change the scraper, I'm editing the SQL db directly as you cant change scraper via web ui (i dont think)
So, just changing "metadata.themoviedb.org" to "metadata.themoviedb.org.python" under strScraper

I see the settings are listing next to it in the DB row.
So, itll still have the old settings in the DB itself.
I'll just need to edit the scraper via a non-headless instance to get the settings in the db row correct Smile

UPDATE: yes that sorted it.
Log is a lot better now.

Just noticed it seems to try to scan new movies twice?
Code:
2021-03-03 16:00:07.730 T:491      INFO <general>: initializing python engine.
2021-03-03 16:00:08.007 T:491      INFO <general>: [metadata.themoviedb.org.python]: Find movie with title 'Adventures With Purpose Egypt' from year '2008'
2021-03-03 16:00:13.372 T:491      INFO <general>: CPythonInvoker(2, /usr/share/kodi/addons/metadata.themoviedb.org.python/python/scraper.py): script successfully run
2021-03-03 16:00:13.375 T:491      INFO <general>: initializing python engine.
2021-03-03 16:00:13.381 T:491      INFO <general>: [metadata.themoviedb.org.python]: Find movie with title 'Adventures With Purpose Egypt' from year '2008'
2021-03-03 16:00:18.493 T:491      INFO <general>: CPythonInvoker(2, /usr/share/kodi/addons/metadata.themoviedb.org.python/python/scraper.py): script successfully run
2021-03-03 16:00:18.493 T:490   WARNING <general>: No information found for item 'smb://192.168.20.3/sharedfolders/pool/Movies/Adventures With Purpose Egypt (2008)/Adventures With Purpose Egypt (2008).mp4', it won't be added to the library.
Is that normal?

And Python scrapers seem a lot slower. I guess if its needing to start up Python for each movie - that's expected?
im getting nothing but errors when trying to scrape with the python scraper.. seems largely imdb related
Quote:2021-03-17 15:20:49.741 T:5331 ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'socket.timeout'>
Error Contents: The read operation timed out
Traceback (most recent call last):
File "/storage/.kodi/addons/metadata.themoviedb.org.python/python/scraper.py", line 175, in <module>
run()
File "/storage/.kodi/addons/metadata.themoviedb.org.python/python/scraper.py", line 164, in run
enddir = not get_details(parse_lookup_string(params["url"]), params['handle'], settings)
File "/storage/.kodi/addons/metadata.themoviedb.org.python/python/scraper.py", line 102, in get_details
imdbinfo = get_imdb_details(details['uniqueids'])
File "/storage/.kodi/addons/metadata.themoviedb.org.python/python/lib/tmdbscraper/imdbratings.py", line 34, in get_details
votes, rating, top250 = _get_ratinginfo(imdb_id)
File "/storage/.kodi/addons/metadata.themoviedb.org.python/python/lib/tmdbscraper/imdbratings.py", line 38, in _get_ratinginfo
response = api_utils.load_info(IMDB_RATINGS_URL.format(imdb_id), default = '', resp_type='text')
File "/storage/.kodi/addons/metadata.themoviedb.org.python/python/lib/tmdbscraper/api_utils.py", line 62, in load_info
response = urlopen(req)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
File "/usr/lib/python3.8/urllib/request.py", line 525, in open
File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
File "/usr/lib/python3.8/urllib/request.py", line 1358, in do_open
File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
File "/usr/lib/python3.8/http/client.py", line 307, in begin
File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
File "/usr/lib/python3.8/socket.py", line 669, in readinto
File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
File "/usr/lib/python3.8/ssl.py", line 1099, in read
socket.timeout: The read operation timed out
-->End of Python script error report<--
just non stop errors
Longer log extract https://cloud.dimspace.xyz/s/W6mS4eAdcZ2FAwQ

and a lot of locking up as well, and crashing my device during scrape. summat not right (using coreelec fwiw)
(2021-03-17, 17:28)dm21912 Wrote: [ -> ]Longer log extract https://cloud.dimspace.xyz/s/W6mS4eAdcZ2FAwQ
Oh geez,, what a terrible site. Please use Kodi Paste Site and post the full log thanks.
(2021-03-17, 20:26)Karellen Wrote: [ -> ]
(2021-03-17, 17:28)dm21912 Wrote: [ -> ]Longer log extract https://cloud.dimspace.xyz/s/W6mS4eAdcZ2FAwQ
Oh geez,, what a terrible site. Please use Kodi Paste Site and post the full log thanks.
you dont even know what nextcloud is? its my personal file host on my home server, it was quicker and easier to just use that as the pertinent parts of the log was already here in text (md) form and the kodi log uploader in matrix does not work, it crashes every time you try to use it. 

no full log because I've now switched scrapers and don't want to mess things up scraping with a broken scraper. That extract gives all the information that's needed. It covers everything from the start of the scrape to when i pulled the log. 

it looks no different https://paste.kodi.tv/ufegipukol.kodi just has pretty colours.. 

but thanks for just complaining about a cloud host rather than suggesting solutions Wink

cheers.
"socket.timeout: The read operation timed out"

your connection is timing out trying to reach the imdb website: https://www.imdb.com/title/{imdb_id}/
Maybe IMDB was down at the time or your internet was having trouble reaching it
(2021-03-18, 00:51)dm21912 Wrote: [ -> ]it looks no different https://paste.kodi.tv/ufegipukol.kodi just has pretty colours.. 
Haha, you are joking, right?

Go on, tell me you can read your one easily and comfortably....
Image

Image

Anyway, what matthuisman said
Is this addon having issues in Kodi 19? It's very slow, it gets stuck at 50% and if I try to exit Kodi while it's running it just locks up, I have to force kill Kodi (on Android, Chromecast with Google TV to be exact). The database also ends up with duplicates, same movies shows up as 1080p and 480 SD even though there is a single file. Any settings I should be checking? I tried reinstalling Kodi and starting clean and it does not help.
(2021-03-27, 07:56)sirmedia Wrote: [ -> ]it gets stuck at 50% and if I try to exit Kodi while it's running it just locks up, I have to force kill Kodi (on Android, Chromecast with Google TV to be exact). The database also ends up with duplicates, same movies shows up as 1080p and 480 SD even though there is a single file. Any settings I should be checking? I tried reinstalling Kodi and starting clean and it does not help.
We'll need a Debug Log that captures the problem you describe.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12