Need the IMDb repository or packages to fix scraping issues.
#1
Hi,

Does anyone know where I can get hold of the IMDb repository?

I need access to all the different version updates in its zip format.

The latest update removes the support for fallback to IMDb trailers. I have a lot of old movies that don't scrape with trailers from hdtrailers.net so I would like to be able to continue to scrape from IMDb.

I updated the to the latest IMDb scraper without knowing about the changelog and the removal of the IMDb fallback. I tried to rollback but I managed to completly F'up my library. I decided to reinstall xbmc and have since then troubleshooted it for hours with various combinations. Yesterday I finally located the issue. For some strange reason "metadata.common.imdb.com" is automatically updated to version 2.1.0 (I'm not updating add-ons automatically). Once that happens the IMDb trailer scraping gets broken. First I thought it was "metadata.imdb.com" that was the issue but when I write protected "metadata.common.imdb.com" with version 2.0.8 and updated "metadata.imdb.com" to 2.1.8 scraping trailers still worked (old movies showed up with trailers).

The problem I have now is that a few movies does not scrape the fanart e.g Grindhouse, The Cove & Wallstreet 1987 etc. If I try to manually update the movie there are no fanarts available either. These are all movies that I use to have fanarts before I completely reinstalled XBMC.

So these are my findings:

- metadata.common.imdb.com (2.0.8) and metadata.imdb (2.1.8) scrapes trailers for 9.5/10 movies but has fanart and in some cases movie identification problems.
- metadata.common.imdb.com (2.1.0) and metadata.imdb (2.1.8 or later) scrapes 5/10 movies (basically only new movies because of the IMDb trailer removal) but has less fanart problems.

That's my story and I'm hoping that I can fix this with a later version of "metadata.imdb.com" prior to the removal of IMDb trailers.

If there's no repository I would be more than happy if anyone could provide me with the zip's prior to the latest IMDb update. They are located in 'Library\Application Support\XBMC\addons\packages' on my Mac Mini 2010 but I assume that Windows users can find them in '\XBMC\addons\packages'.

Br
Blumman
Reply
#2
All the gits for addons can be found at sf.net/projects/xbmc
Reply
#3
Hmmm... imdb trailer fallback has been removed because those links are not permanent. I.e. if you scrape a trailer link from imdb, it will only work for a while, then gets invalid, makes it completely unusable. Is it not what you've experienced?
Reply
#4
Smile 
olympia Wrote:Hmmm... imdb trailer fallback has been removed because those links are not permanent. I.e. if you scrape a trailer link from imdb, it will only work for a while, then gets invalid, makes it completely unusable. Is it not what you've experienced?

You might be right. However the only movie that had a trailer, where the trailer pointed to a removed yahoo trailer was Avatar. Else I haven't experienced such issues. It's not like I watch all of the old trailers so I wouldn't know for sure. It's more that it's nifty to have them there once you want to see them, it's a great feature. E.g Star Wars, Back to the Future and new movies works fine. Smile

I began to look in to the xml files to see if I could understand what was going on and I noticed that I was missing "metadata.common.themoviedb.org". It looked like IMDb needed this to get fanarts so added the addon. After a while I realized that the IMDb addon already downloaded posters and fanarts from TMDb (I unchecked all poster and fanarts sources in IMDb settings and I was left off with IMDb thumbs and nothing else). Then I re-scrapped my library with TMDb activated and I compared the local posters/fanarts with TMDb/MPDb, they matched.

I have rolled back to metadata.imdb.com (2.1.8) and write-protected metadata.common.imdb (2.0.8) as it gets overwritten when you start xbmc.

As far as the fanart goes I'm not on the right track. 81% of my 720p movies had fanarts after scraping. 71% for the 1080p movies the rest I had to add manually. Every single 1080p move had fanarts prior to my re-installation and they physically exist on TMDb. :confused2:

There must be something that I had in my old install that I'm missing now but I can't figure out what it is.

I'm going to search a bit more in the forum, it might be related to Aeon MQ2. If I provide a debug log, can I remove my SMB password from the log?
Reply
#5
Can you try to play a trailer for a movie you scraped long time ago with imdb scraper?
Reply
#6
olympia Wrote:Can you try to play a trailer for a movie you scraped long time ago with imdb scraper?

Since I had to reinstall xbmc I'm unable to do so since my library is fresh. I know for a fact that Avatar did not work a few months ago and it does not work today either i.e after a new IMDb scrape.

I thought IMDb hosted their own trailers but I can see that Avatar is picked up from Yahoo. From XWMM I can see that other movie trailers that have been scraped with IMDb have it's source from e.g totaleclips.com & amazonimdb. The majority is picked up from Yahoo and totaleclips.com.

I find it strange that none have it's source from hd-trailers.net. My understanding of enabling hd-trailers.net in IMDb setting is that hd-trailers.net should be used as a primary source for trailers. If a trailer is not found it should fall back to IMDb.

I'm going to re-organize my library once the root cause to the fanart issue has been located. At least one other user have similar issues as me. In the meantime, if you want me to test something with the trailers I'd be happy to assist.
Reply
#7
I explained what is the "root cause" of your fanart issue in that other thread.

Concerning the trailers:
- hd-trailers = yahoo (hd-trailers is collecting links mainly from yahoo, it doesn't host them)
- no, I can't ask you to do any test now. The only valuable test would've been to try to play a trailer which has been surely scraped from imdb (amazonimdb).
Reply
#8
Well if it helps I could re-scrape my library with hd-trailers.net disabled and see how many trailers I get with the IMDb fallback and if they work. It would be interesting to see if Avatar worked after that...
Reply

Logout Mark Read Team Forum Stats Members Help
Need the IMDb repository or packages to fix scraping issues.0