Does XBMC and XBMC Media Center end-users have permission to scrape imdb.com website?

  Thread Rating:
  • 1 Votes - 1 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Closed
Waffa Offline
Posting Freak
Posts: 1,163
Joined: May 2008
Reputation: 7
Location: Netherlands
Post: #16
Jester Wrote:BIG NOTICE !

From 17033 the IMDB scraper DOES NOT WORK ANY LONGER !


they do not allow scraping their web site and XBMC is complying with this.

the new default scraper is themoviedb.org which needs help filling, so please go there and help them (and thus yourself) out Smile

Does this also means that Media Companion and Movie info plus are not allowed to scrape there anymore?
(This post was last modified: 2009-01-12 22:33 by Waffa.)
find
fekker Offline
Posting Freak
Posts: 1,545
Joined: Oct 2008
Reputation: 30
Post: #17
My cached data is pulled from the imdb list files, it does not create an online database, is not a commercial application, it's not charging any fees, It does reference IMDB as the source of that data with links to there site.

MIP's cached data is in compliance with the imdb list file usage.

Quote:All data and software released by Internet Movie Database Ltd is
freely available to anyone within certain limitations. You are
encouraged to quote subsets of the database in USENET articles,
movie related FAQs, magazine articles etc. We do ask, however,
that you make reference to the source of the data and provide a
pointer to the database for the benefit of the reader.

MIP also does a standard http query and downloads the same full html page as any other browser, the difference is in how I display that data as I parse it into a different format again with reference to IMDB as the source of that data.
find
fekker Offline
Posting Freak
Posts: 1,545
Joined: Oct 2008
Reputation: 30
Post: #18
My cached data is pulled from the imdb list files, it does not create an online database, is not a commercial application, it's not charging any fees, It does reference IMDB as the source of that data with links to there site.

MIP's cached data is in compliance with the imdb list file usage.

Quote:All data and software released by Internet Movie Database Ltd is
freely available to anyone within certain limitations. You are
encouraged to quote subsets of the database in USENET articles,
movie related FAQs, magazine articles etc. We do ask, however,
that you make reference to the source of the data and provide a
pointer to the database for the benefit of the reader.

MIP also does a standard http query and downloads the same full html page as any other browser, the difference is in how I display that data as I parse it into a different format again with reference to IMDB as the source of that data.
find
nekrosoft13 Offline
Fan
Posts: 509
Joined: Dec 2008
Reputation: 1
Post: #19
ultrabrutal Wrote:The first thing that springs to mind when looking at the site, is... Who owns the copyright of fanart and posters? I think there is a grey area there also I'm afraid Sad

offtopic: spiff, have looked into the api info. looks easy to use. any indications on speed vs the others?

can you please be quite, once you are done everything will be a violation.
find
jmarshall Offline
Team-XBMC Developer
Posts: 26,221
Joined: Oct 2003
Reputation: 178
Post: #20
Thanks for the heads up on the license issues. Team XBMC will contact IMDb directly and ask for their approval if they are willing to give it.

For the meantime, take the opportunity to contribute to an open database such as themoviedb. If each of us take the time (an hour or so ain't that much) to contribute 10 movies from our collections (obviously NOT just resubmitting IMDb data!) then their collection will swell extremely quickly.

After all, look at how awesome thetvdb is now, thanks to user contributions from people like you and me.

I'll post back once I hear back from IMDb.

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find
ultrabrutal Offline
Posting Freak
Posts: 952
Joined: Feb 2005
Reputation: 0
Location: South of Heaven
Post: #21
nekrosoft13 Wrote:can you please be quite, once you are done everything will be a violation.

You forgot the smiley

jmarshall Wrote:Thanks for the heads up on the license issues

Np I guess. Never thought that spiff would be that radical. Let's just hope imdb guys are cool
find
warwon Offline
Senior Member
Posts: 178
Joined: Dec 2008
Reputation: 0
Post: #22
I like to know more as to why this has to change.

The main reason I use these applications is to pull data from a website such as IMDB.

They have never sent out letters, emails, or anything to other projects.

Mythtv, Meedios are not reporting any issues.

Time to time IMDB changes that function, of scrapping but the coders keep up.

Frak.
find
Waffa Offline
Posting Freak
Posts: 1,163
Joined: May 2008
Reputation: 7
Location: Netherlands
Post: #23
fekker Thanx for the reply and aplication.
find
LaTropa64 Offline
Fan
Posts: 649
Joined: Nov 2005
Reputation: 0
Post: #24
fekker Wrote:My cached data is pulled from the imdb list files, it does not create an online database, is not a commercial application, it's not charging any fees, It does reference IMDB as the source of that data with links to there site.

MIP's cached data is in compliance with the imdb list file usage.



MIP also does a standard http query and downloads the same full html page as any other browser, the difference is in how I display that data as I parse it into a different format again with reference to IMDB as the source of that data.
I'm going to agree with this. I'm essentially just using XBMC to access imdb.com instead of FireFox or Internet Explorer.
find
jmarshall Offline
Team-XBMC Developer
Posts: 26,221
Joined: Oct 2003
Reputation: 178
Post: #25
As I said in the other thread, Team XBMC has contacted IMDb and will leave the IMDb scraper disabled until we hear from them. Note that we were not contacted from IMDb, but are simply responding to a users' request.

It is unclear to the team as to whether we're doing anything wrong or not, so given that, the wisest action is to cease distributing the scraper until we are certain that we are doing things correctly.

In the meantime, please spend some time contributing data to themoviedb so that it's available to others. If we all spend an hour or two submitting data (not just copy + pasted from IMDb, ofcourse!) on some of our own movies, it will benefit the entire community.

Thanks for your understanding.

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
(This post was last modified: 2009-01-13 00:28 by jmarshall.)
find
rodalpho Offline
Fan
Posts: 637
Joined: Nov 2006
Reputation: 17
Post: #26
Not that any of us would knowingly violate their TOS, and I certainly didn't do it myself, but just for academic purposes, do you mind if I post how to reenable the scraper?
find
warwon Offline
Senior Member
Posts: 178
Joined: Dec 2008
Reputation: 0
Post: #27
This is odd, oh well no more updating until it gets back.

I refuse to lose that ability for imdb Tongue
find
mr.b Offline
Senior Member
Posts: 165
Joined: May 2007
Reputation: 0
Post: #28
not saying hoorah or anything for this latest development (although it is reversable if need be), but this may lead to some speed improvements if we end up with a setup that periodically keeps a local cached copy of imdb and periodically updates it (http://www.imdb.com/interfaces).
find
ultrabrutal Offline
Posting Freak
Posts: 952
Joined: Feb 2005
Reputation: 0
Location: South of Heaven
Post: #29
rodalpho Wrote:just for academic purposes, do you mind if I post how to reenable the scraper?

Well you look at what spiff did in the revision where he disabled the scraper. Undo this to the source and rebuild
find
Gamester17 Offline
Team-XBMC Forum Moderator
Posts: 10,523
Joined: Sep 2003
Reputation: 10
Location: Sweden
Post: #30
jmarshall Wrote:As I said in the other thread...
For reference, here is a link to that other thread => http://forum.xbmc.org/showthread.php?tid=43626

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find