IMDB Scraper using iPhone API

  Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
ntlord Offline
Junior Member
Posts: 24
Joined: Aug 2008
Reputation: 0
Post: #1

After hearing about this new IMDB iPhone app, it got me thinking about what type of API it might be using. I figure if we switch to something like this, it might give us a bit more stability then the current webpage scrapers out there.

I did a quick tcpdump from my iphone when it connects to IMDB using the newly released app. It looks like it is using some HTTP API which returns everything in the form of comma, delimited text.

It does a GET to

with the following variables

q=this is the search terms
sig= this is some sort of signature. I'm not sure if this is unique to the phone or something else.

I don't really have time right now to look at this deeper. But this should be enough to get somebody started, if they so chose and have access to an iPhone.

find quote
compcentral Offline
Posts: 629
Joined: Apr 2010
Reputation: 2
Post: #2

Doesn't seem to provide much back other than a list of search results with basic info. You can actually send the request via a web browser and timestamp and sig appear to be optional.


Search for "shrek":

Response from server:

{"copyright":"For use only by clients authorized in writing by IMDb. Authors and users of unauthorized clients accept full legal exposure/liability for their actions.","data":{"results":[{"label":"Popular Titles","list":[{"tconst":"tt0126029","type":"feature","title":"Shrek","image":{"width":334,"url":"","height":475},"principals":[{"nconst":"nm0000196","name":"Mike Myers"},{"nconst":"nm0000552","name":"Eddie Murphy"}],"year":"2001"},{"tconst":"tt0298148","type":"feature","title":"Shrek 2","image":{"width":319,"url":"","height":475},"principals":[{"nconst":"nm0000196","name":"Mike Myers"},{"nconst":"nm0000552","name":"Eddie Murphy"},{"nconst":"nm0000139","name":"Cameron Diaz"}],"year":"2004"},{"tconst":"tt0413267","type":"feature","title":"Shrek the Third","image":{"width":1382,"url":"","height":2048},"principals":[{"nconst":"nm0000196","name":"Mike Myers"},{"nconst":"nm0000139","name":"Cameron Diaz"},{"nconst":"nm0000552","name":"Eddie Murphy"}],"year":"2007"},{"tconst":"tt0892791","type":"feature","title":"Shrek Forever After","image":{"width":400,"url":"","height":593},"principals":[{"nconst":"nm0000196","name":"Mike Myers"},{"nconst":"nm0000139","name":"Cameron Diaz"},{"nconst":"nm0000552","name":"Eddie Murphy"}],"year":"2010"}]},{"label":"Names (Exact Matches)","list":[{"nconst":"nm1277920","known_for":"Transportation Department, \"Catch Me If You Can\"","name":"Mark Webb"},{"nconst":"nm1903847","known_for":"Actor, \"Lies & Alibis\"","name":"Sergio Bruna","image":{"width":450,"url":"","height":527}},{"nconst":"nm3331096","known_for":"Self, \"Big Brother\"","name":"David Sín"}]},{"label":"Titles (Partial Matches)","list":[{"tconst":"tt0897387","type":"short","title":"Shrek the Halls","image":{"width":155,"url":"","height":200},"year":"2007"},{"tconst":"tt0360985","type":"short","title":"Shrek 4-D","image":{"width":500,"url":"","height":365},"principals":[{"nconst":"nm0000196","name":"Mike Myers"},{"nconst":"nm0000552","name":"Eddie Murphy"},{"nconst":"nm0000139","name":"Cameron Diaz"}],"year":"2003"},{"tconst":"tt0307461","type":"short","title":"Shrek in the Swamp Karaoke Dance Party","year":"2001"},{"tconst":"tt0410595","type":"video_game","title":"Shrek 2","principals":[{"nconst":"nm0332227"},{"nconst":"nm0276253","name":"Holly Fields"},{"nconst":"nm0609137","name":"Mark Moseley"}],"year":"2004"},{"tconst":"tt0290741","type":"tv_special","title":"Creating a Fairy Tale World: The Making of 'Shrek'","year":"2001"},{"tconst":"tt0928070","type":"video_game","title":"Shrek: Smash n' Crash Racing","year":"2006"},{"tconst":"tt1024946","type":"video_game","title":"Shrek the Third","year":"2007"},{"tconst":"tt1056092","type":"feature","title":"100 Things You Always Wanted to Know About Shrek But Were Afraid to Ask So We're Gonna Tell You Anyway Movie Special","year":"2007"},{"tconst":"tt0488580","type":"video","title":"Meet the Cast of Shrek 2","year":"2004"},{"tconst":"tt0286967","type":"video_game","title":"Shrek: Fairy Tale Freakdown","year":"2001"},{"tconst":"tt0488589","type":"video","title":"The Music of Shrek 2","year":"2004"},{"tconst":"tt0475753","type":"video","title":"The Tech of Shrek 2","year":"2004"},{"tconst":"tt1087816","type":"video","title":"The Tech of Shrek the Third","year":"2007"},{"tconst":"tt1640561","type":"video_game","title":"Shrek Forever After: The Game","year":"2010"},{"tconst":"tt1055313","type":"tv_special","title":"Shrek the Third: UK Premiere Special","year":"2007"}]},{"label":"Names (Partial Matches)","list":[{"nconst":"nm3318147","known_for":"Self, \"Episode #1.3\"","name":"Tony 'Shrek' Organ"},{"nconst":"nm3026404","known_for":"Miscellaneous Crew, \"Bad\"","name":"Steven Shrek"}]}]}}
find quote
bleze Offline
Senior Member
Posts: 223
Joined: Nov 2009
Reputation: 0
Post: #3
"copyright":"For use only by clients authorized in writing by IMDb. Authors and users of unauthorized clients accept full legal exposure/liability for their actions."
find quote
CrashX Offline
Posting Freak
Posts: 1,165
Joined: Jan 2009
Reputation: 2
Post: #4
It looks like all you need is q.

The normal XBMC log IS NOT a debug log, to enable debug logging you must toggle it on under XBMC Settings - System or in advancedsettings.xml. Use XBMC Debug Log Addon to retrieve it.
find quote
m.savazzi Offline
Posts: 1,070
Joined: Mar 2008
Reputation: 32
Post: #5
apart from being illegal it appears to be a JSON transaction

If you think I'm useful please use the +/- button to raise my reputation
find quote
takoi Offline
Team-Kodi Member
Posts: 1,223
Joined: Oct 2009
Reputation: 25
Location: Norway
Post: #6
Doesn't work anymore, and "illegal"? Taking any info from imdb is "illegal"
find quote