2016-08-06, 04:55
Finally done this using Universal Movie Scraper and TMM. It took me several days to scrape, verity, and aggregate the four rating data in an excel sheet.
Here's what I've come to know through the process.
- TMM's Rotten Tomatoes scraper has some serious issues in detecting movies. Almost half of movies can't be identified. It can't use already known IMDb ID or TMDb ID, purely guessing out of filename. Furthermore, RT site seems to use US release date as movie year. Also after around one thousand movies, you keep getting 403 Forbidden error due to their API volume limit. So it's almost unusable.
- Universal Movie Scraper, Kodi addon, too can't use already known IMDb ID for searching. For this reason, scraping was taking hours and I was left with lots of un/mis-identified movies for each rating scrape. Fortunately it exports IMDb ID for each movie, which allows you to compare with your original IMDB rating database exported from TMM. After figuring out what those misidentified movies actually are, I had to rescrape each movie using Kodi. Yes, repeat the same process for each rating scrape to the point that you practically memorize the wrong/right movie title list.
- OMDb API, for some reason, does not return RT ratings for some movies (there's no pattern but especially for many recent movies). It amounted to about 8~9% of my collection, which I had to find ratings manually from the RT site.
Though I've finally got a nice rating comparison chart, keeping this data up to date is unthinkable. For new movies to my collection, probably I will add each rating manually to the spreadsheet.
Again, it would be awesome if TMM could scrape those ratings from OMDb API using IMDB ID. Then it won't take much time to scrape and there will be no misidentified movies. Also updating ratings could be done with just one click.
Regarding the single rating tag issue, I would suggest those additional ratings be written as custom tags something like this:
<metaScore>100</metaScore>
<tomatoMeter>99</tomatoMeter>
<tomatoReviews>84</tomatoReviews>
<tomatoUserMeter>98</tomatoUserMeter>
<tomatoUserReviews>728130</tomatoUserReviews>
Then the user could export all of them at once. If the user wants to use one of them for TMM rating display, renaming those tags to <rating> and <votes> tags (via bulk editing NFO files) would do. Of course, TMM could offer such rating selection function.
Here's what I've come to know through the process.
- TMM's Rotten Tomatoes scraper has some serious issues in detecting movies. Almost half of movies can't be identified. It can't use already known IMDb ID or TMDb ID, purely guessing out of filename. Furthermore, RT site seems to use US release date as movie year. Also after around one thousand movies, you keep getting 403 Forbidden error due to their API volume limit. So it's almost unusable.
- Universal Movie Scraper, Kodi addon, too can't use already known IMDb ID for searching. For this reason, scraping was taking hours and I was left with lots of un/mis-identified movies for each rating scrape. Fortunately it exports IMDb ID for each movie, which allows you to compare with your original IMDB rating database exported from TMM. After figuring out what those misidentified movies actually are, I had to rescrape each movie using Kodi. Yes, repeat the same process for each rating scrape to the point that you practically memorize the wrong/right movie title list.
- OMDb API, for some reason, does not return RT ratings for some movies (there's no pattern but especially for many recent movies). It amounted to about 8~9% of my collection, which I had to find ratings manually from the RT site.
Though I've finally got a nice rating comparison chart, keeping this data up to date is unthinkable. For new movies to my collection, probably I will add each rating manually to the spreadsheet.
Again, it would be awesome if TMM could scrape those ratings from OMDb API using IMDB ID. Then it won't take much time to scrape and there will be no misidentified movies. Also updating ratings could be done with just one click.
Regarding the single rating tag issue, I would suggest those additional ratings be written as custom tags something like this:
<metaScore>100</metaScore>
<tomatoMeter>99</tomatoMeter>
<tomatoReviews>84</tomatoReviews>
<tomatoUserMeter>98</tomatoUserMeter>
<tomatoUserReviews>728130</tomatoUserReviews>
Then the user could export all of them at once. If the user wants to use one of them for TMM rating display, renaming those tags to <rating> and <votes> tags (via bulk editing NFO files) would do. Of course, TMM could offer such rating selection function.
