Kodi Community Forum
Release Universal Movie Scraper - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Movie Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=302)
+----- Thread: Release Universal Movie Scraper (/showthread.php?tid=129821)



RE: [Release] Universal Movie Scraper - Living4ever - 2017-03-31

Hello @ all,

I have a "problem" using the Universal Movie Scraper...

In the scraper setting I have chosen "Preferred Tagline Language = DE". But if I scrape a movie now with no german tagline in TMDB, the scraper uses the english tagline... For example the movie "Colonia Dignidad".

Can anybody help me? Is it possible to disable the fallback language if the german tagline field is empty? Or are there any other workarounds?

Thanks for your help Big Grin


RE: [Release] Universal Movie Scraper - Ipaddle - 2017-04-08

Hi

I have trouble scraping this movie: https://www.themoviedb.org/movie/381288-split

There are several split movies to choose from, but I can not find this one.


RE: [Release] Universal Movie Scraper - truthlesshero - 2017-04-09

It seems like the rotten tomatoes rating scraper is broken. Did RT change the way to retrieve it again? I know there was a problem with it for a few months last year..


RE: [Release] Universal Movie Scraper - freakout87 - 2017-04-09

I've encountered the same issue as truthlesshero: http://forum.kodi.tv/showthread.php?tid=311047&pid=2560523#pid2560523

FWIW here's the debug log:

Quote:12:08:13.353 T:8920 DEBUG: scraper: GetRTRatingById returned <details><url function="ParseRTTomAll" cache="4276820-omdbapi.html">http://www.omdbapi.com/?i=tt4276820&r=json&tomatoes=true</url></details>

12:08:13.353 T:8920 DEBUG: CurlFile::Open(19CE6FC0) http://www.omdbapi.com/?i=tt4276820&r=js...atoes=true

12:08:13.399 T:8920 DEBUG: CScraperUrl::Get: Using "UTF-8" charset for "http://www.omdbapi.com/?i=tt4276820&r=json&tomatoes=true"

12:08:13.403 T:8920 DEBUG: scraper: ParseRTTomAll returned <details><rating max="100">N/A</rating><votes>N/A</votes></details>



RE: [Release] Universal Movie Scraper - freakout87 - 2017-04-10

^^ I sent an email to the developer of OMDB regarding this issue. He says that there have been changes made to the API which will require an update to UMS to parse.

https://www.patreon.com/posts/rating-changes-8417367


RE: [Release] Universal Movie Scraper - justin150 - 2017-04-10

How do you uninstall this add on? The relevant sections in the add - on information for uninstall or disable are greyed out.

I have 2 machines running Kodi which I am using for testing. Machine 1 only uses The Movie Database add on which correctly identifies all the movies and assigns the correct posters. Machine 2 has the Universal scraper and mangles roughly 10 titles at least one of which is not fixable (DVD of stage version of Cats)


RE: [Release] Universal Movie Scraper - justin150 - 2017-04-12

Further to my last post I am now getting very confused and deeply irritated.

A manual search on both IMDB and The Movies Database finds the right movie instantly. I have even put in a nfo file which the scraper completely ignores Trying to refresh the movie information does not work because whilst it will let me change the title the scraper refuses to accept that there is a movie called Cats even though, as I say both IMDB and The Movies Database clearly show it. I has fixed that problem so that it uses the locally stored information on a refresh (I would have thought that should be the default option and only if no local info should you go to internet)

Now I face another problem - getting it to use the correct poster/fan art. I thought that would be simple after all in refresh information there is an option for choose art. It is hopeless no matter what I click it endlessly cycles through a series of windows and never actually adds anything. I even tried changing the nfo file so that it pointed to the correct title on movies database - it completely ignored that). Anyone got any suggestions


RE: [Release] Universal Movie Scraper - MasterCATZ - 2017-04-24

(2017-04-08, 16:01)Ipaddle Wrote: Hi

I have trouble scraping this movie: https://www.themoviedb.org/movie/381288-split

There are several split movies to choose from, but I can not find this one.


everyone is haveing problems with those ones

there are 3x movies same name same year ... and it's our fault it's not working for some reason, I can not even get them loaded by using the unique IMDB ID Either

https://www.themoviedb.org/talk/58fbcef9c3a36879f4012770

and because of trying to get those movies corectly linked
now my entire database is stuffed as I switched from themoviedb to this Universal Scraper in the hope it would work better, now most of my movies have been linked to wrong movies as for some reason it decided movies from 50 years ago were better matching !!

not to mention now I have duplicates of every movie because EVERY damn file in each movie folder has been listed the movie ...
( I had a lot of DVD/BluRay rips in VOB form etc .. )


RE: [Release] Universal Movie Scraper - Ipaddle - 2017-04-26

(2017-04-24, 15:05)MasterCATZ Wrote:
(2017-04-08, 16:01)Ipaddle Wrote: Hi

I have trouble scraping this movie: https://www.themoviedb.org/movie/381288-split

There are several split movies to choose from, but I can not find this one.


everyone is haveing problems with those ones

there are 3x movies same name same year ... and it's our fault it's not working for some reason, I can not even get them loaded by using the unique IMDB ID Either

https://www.themoviedb.org/talk/58fbcef9c3a36879f4012770

and because of trying to get those movies corectly linked
now my entire database is stuffed as I switched from themoviedb to this Universal Scraper in the hope it would work better, now most of my movies have been linked to wrong movies as for some reason it decided movies from 50 years ago were better matching !!

not to mention now I have duplicates of every movie because EVERY damn file in each movie folder has been listed the movie ...
( I had a lot of DVD/BluRay rips in VOB form etc .. )

I found this post at themoviedb.org:

For Kodi (Krypton, should be the same in older versions):

If your movie is scraped incorrectly, go to the movies content menu, click "Information". Within "Information" click "Refresh". A window opens where you can chose between titles, but instead click on the "Manual" button. Now enter the correct movie's imdb ID (e.g. tt0120338 for Titanic, it's always the tt number in the imdb URL). The correct movie title will now be listed. Click on the title and voilĂ  the movie will now be scraped correctly.

The same also works for movies that won't be scraped at all (e.g. short films). You can find those failed entries in your Event Log. Here you just click the movie, the refresh window opens automatically. Same procedure as above.

I hope this helps, had the same problem some time go. Took me a while to figure it out. Cheers!

Edit: Just tested it with a Split stub out of couriosity. It worked.


That worked!


RE: [Release] Universal Movie Scraper - Jackanory - 2017-04-26

I recognize that the scraper only imports the 'above the line'* cast from IMDB ... enough is (normally) enough! Sometimes though, a movie's stars end up below the line, and are not scraped into our library. Case in point, Jessica Chastain is listed as a star in Interstellar, but her cast entry is below the line, so she doesn't appear in my library.

What I would propose, is that when scraping, both the Stars field and the Cast field are scraped, then those fields are merged, and that merged list is imported into Kodi. Sounds simple enough, and would certainly increase usability, for those of us that use the cast field.

* By above the line, I refer to those cast members who are displayed on IMDB without having to click on 'See full cast'.


RE: [Release] Universal Movie Scraper - willemd - 2017-05-31

(2017-04-26, 22:49)Jackanory Wrote: I recognize that the scraper only imports the 'above the line'* cast from IMDB ... enough is (normally) enough! Sometimes though, a movie's stars end up below the line, and are not scraped into our library. Case in point, Jessica Chastain is listed as a star in Interstellar, but her cast entry is below the line, so she doesn't appear in my library.

What I would propose, is that when scraping, both the Stars field and the Cast field are scraped, then those fields are merged, and that merged list is imported into Kodi. Sounds simple enough, and would certainly increase usability, for those of us that use the cast field.

* By above the line, I refer to those cast members who are displayed on IMDB without having to click on 'See full cast'.

See http://forum.kodi.tv/showthread.php?tid=181488
I think this solves your problem.


RE: [Release] Universal Movie Scraper - willemd - 2017-05-31

I think the scraper has a small bug in reading IMDb plot outlines which contain a hyperlink.

For example: https://www.imdb.com/title/tt3263904/
The plot outline ("The story of Chesley Sullenberger, an American pilot who became a hero ... etc. etc.") contains a link to Chesley Sullenberger as a character (or a real person in the IMDb database). The scraper discards the link and anything after it. So the plot outline in the Kodi database becomes "The story of ". Bad, bad behaviour!

For now I've solved it with making .nfo files with an <outline> entry, but I'm not sure all Kodi users will be able to do so.


Other examples: https://www.imdb.com/title/tt0056172/ or https://www.imdb.com/title/tt1619029


RE: [Release] Universal Movie Scraper - truthlesshero - 2017-06-01

Does anyone know if there's any update on the add-on to make changes that include the rotten tomatoes ratings again?


RE: [Release] Universal Movie Scraper - hac_ - 2017-06-03

(2017-05-31, 12:03)willemd Wrote: I think the scraper has a small bug in reading IMDb plot outlines which contain a hyperlink.

For example: https://www.imdb.com/title/tt3263904/
The plot outline ("The story of Chesley Sullenberger, an American pilot who became a hero ... etc. etc.") contains a link to Chesley Sullenberger as a character (or a real person in the IMDb database). The scraper discards the link and anything after it. So the plot outline in the Kodi database becomes "The story of ". Bad, bad behaviour!

For now I've solved it with making .nfo files with an <outline> entry, but I'm not sure all Kodi users will be able to do so.


Other examples: https://www.imdb.com/title/tt0056172/ or https://www.imdb.com/title/tt1619029

Problem is this regexp in the metadata.common.imdb.com plugin, imdb.row 125-127

Quote:<RegExp input="$$1" output="&lt;outline&gt;\1&lt;/outline&gt;" dest="2">
<expression fixchars="1" trim="1">itemprop=&quot;description&quot;&gt;([^&lt;]+)&lt;</expression>
</RegExp>

Current regexp will stop when it finds the first <, but it should actually stop only on </div>. After that we need to filter out anything inside <> pairs to get rid of the a href html tags. I tried but couldn't figure out how to achieve that filtering in this format.


RE: [Release] Universal Movie Scraper - scudlee - 2017-06-03

html/xml tags are automatically cleaned out by default.
You'd need to add (e.g.) noclean="1" as an attribute to the <expression> for that not to happen (on the first capturing group).

Code:
<RegExp input="$$1" output="&lt;outline&gt;\1&lt;/outline&gt;" dest="2">
<expression fixchars="1" trim="1">itemprop=&quot;description&quot;&gt;(.+?)&lt;/div&gt;</expression>
</RegExp>
Would probably do it.