[olympia]

Whiile the parseIMDBplot function seems correct, the cached file it runs against is not. GetIMDBPlotById is returning tt052013-main-html to work with.
If you search the actual cache file, in my case %appdata%\Kodi\cache\scrapers\metadata.universal\tt052013-main-html
the plot is missing from the file. So the problem is with the web page saver not the scraper.

I experienced the same for a good while now. Not sure what is the root cause of this. For sure it's not the "web page saver" as I also experience this if I look for the page source in a browser.

[monisriz]

Looking for some clarity on the intended behavior when setting the options in the addon in the Ratings section.

My options/settings:

Get default ratings from => IMDB
Also get ratings from Rotten Tomatoes => Checked
- Rating generated using => TomatoMeter All Critics
Also get ratings from MetaCritic => Unchecked
Also get ratings from  TMDB => Unchecked

I have added my OMDB api key. IMDB ratings are being set as "default" as expected. However, for Rotten Tomatoes, what I see is that "Audience" ratings are also being scraped and getting added to my MySQL db (instead of just "Critics" rating).

Looking at the options and selections above, when it comes to RT-related ratings, shouldn't ONLY the "TomatoMeter All Critics" ratings (associated with the tomato icon) be scraped and other ratings (popcorn icon) be ignored?



A note after some more checking:

When RT is selected as default rating provider, all four RT related ratings are scrapped even if "Also get non-default RT ratings" option is unchecked. The rating type selected under "Ratings generated using" sets the correct rating type as default.

It seems regardless of RT related options chosen, if RT is selected as one of the ratings provider (default or not), all four RT ratings are scraped to the library?

[monisriz]

delete.

[monisriz]

Whiile the parseIMDBplot function seems correct, the cached file it runs against is not. GetIMDBPlotById is returning tt052013-main-html to work with.
If you search the actual cache file, in my case %appdata%\Kodi\cache\scrapers\metadata.universal\tt052013-main-html
the plot is missing from the file. So the problem is with the web page saver not the scraper.

I experienced the same for a good while now. Not sure what is the root cause of this. For sure it's not the "web page saver" as I also experience this if I look for the page source in a browser.

IMDB has changed the layout and some of the info (e.g. tagline) is no longer on *.main.html. However, *.reference.html has a much simpler layout with all the relevant info available on it. Minor tweaking will fix it though and I believe someone a couple of posts earlier has the solution.

[itwasallablur]

Where do you add the fixes? I'm using a Nvidia shield.

[Karellen]

Where do you add the fixes? I'm using a Nvidia shield.

If you are referring to the code in this post... https://forum.kodi.tv/showthread.php?tid...pid3101319

Then it completely replaces the existing code in the imdb.xml file of metadata.common.imdb.com located in the Kodi data folder (wiki) / addons folder

You also have the choice to use a different scraper.

[staindrocks]

Where do you add the fixes? I'm using a Nvidia shield.

If you are referring to the code in this post... https://forum.kodi.tv/showthread.php?tid...pid3101319

Then it completely replaces the existing code in the imdb.xml file of metadata.common.imdb.com located in the Kodi data folder (wiki) / addons folder

You also have the choice to use a different scraper.

So why aren't you implementing this change and offering it in an update?  Does it not solve the issue consistently enough, or does it break other things... basically, what is the downside to doing it this way?

[Karellen]

So why aren't you implementing this change and offering it in an update?  Does it not solve the issue consistently enough, or does it break other things... basically, what is the downside to doing it this way?

I have no idea why it has not been fixed in the official download. Maybe @olympia has some issue with it?

[Jackanory]

So why aren't you implementing this change and offering it in an update?  Does it not solve the issue consistently enough, or does it break other things... basically, what is the downside to doing it this way?

I have no idea why it has not been fixed in the official download. Maybe @olympia has some issue with it?

Perhaps a solution is being sought, that will trickle down to those of us that rely on earlier versions. For instance my 18.6 installation is stuck at version 5.5.10 of the Universal scraper, and hasn't scraped a plot for several weeks!

I'm considering writing a update query to copy plot outline to plot, where ... appropriate.

[JJarmush]

Hi
is there any chance I run the universal movie scraper without kodi running? The idea is that each day the scraper is launched silently, detect any change in the video or music database, and stores the relevant data locally so when i launch kodi everything is already up-to date.

A less polished alternative would be to

1. launch kodi SILENTLY (from win or Android): then the scrapers would automatically launch since I would enabled in kodi "launch the scraper anytime kodi starts".
   
2.  Shut down that kodi instance as soon as the scraper is done (by the way: does the scraper return an exit code each time the scraping is completed ? and where the exit code may be possibly stored ?)
   

Is there a command line (windows and/or Android) that I would put in a cron bash/bat file so to repeat the command line (run kodi in the background and shut it when scraper is done) everyday?
thanks

[At2010]

Hello again.

https://paste.kodi.tv/qanufefaga.kodi

This is my latest effort to improve imdb.xml. It implements fixes for the Plot and Tagline functions to use xx-reference.html instead of xx-main.html. I have verified this works consistently for a variety of test movies.
functions changed:
GetIMDBPlotById
ParseIMDBPlot
GetIMDBTaglineById
ParseIMDBTagline

To use, copy the pastebin code to a new imdb.xml. Backup and replace the imdb.xml file in %appdata%\Kodi\addons\metadata.common.imdb.com

@

[Karellen]

Thanks @At2010

Are you in a position to create a PR here... https://github.com/xbmc/repo-scrapers/pulls

[At2010]

Hi
is there any chance I run the universal movie scraper without kodi running? The idea is that each day the scraper is launched silently, detect any change in the video or music database, and stores the relevant data locally so when i launch kodi everything is already up-to date.

Beware of using Auto update on startup with this addon. IMDB still uses scraper functions on a web pages that change whenever Amazon wants. This can lead to a corruption or loss of information provided. I would recommend the TheMovieDB python addon for an automated process. There is also a Program addon in the repository called Library Auto Update by robweber that updates media libraries on a timer (so it says). I have not tried it but it may work for you.

[At2010]

Thanks @At2010

Are you in a position to create a PR here... https://github.com/xbmc/repo-scrapers/pulls

I have a github id, yes (at2010) . But i have not done anything for Kodi. Is there a how-to for information provider pull requests?
Thanks,
at2010

[Karellen]

Is there a how-to for information provider pull requests?

Not anything comprehensive. Try section 3 here... https://kodi.wiki/view/Submitting_Add-ons#Pull_requests
Maybe @olympia can help if you ask specific questions.
If you make a mistake, don't stress. One of the developers will let you know how it should have been done.