themoviedb scraper now partly broken
#1
Exclamation 
Hi all

running on the last stable linux release am I right in thinking a change to the themoviedb api has now broken fan art thumbnails and other information.

Recent scrapes have picked up some details, notably title, year and some info but thumbnails and fan art dont appear even though they are on the site (I've added them in some cases).

Checking the log shows lines such as

DEBUG: scraper: GetTMDBFanart returned <details><fanart></fanart></details>

so it would appear that although the initial query to themoviedb has brought back lots of xml nodes of

<poster id="4bc919a1017a3c57fe009d8d">
<image url="http://i1.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-original.jpg" width="1000" size="original" height="1500"/>
<image url="http://i2.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-mid.jpg" width="500" size="mid" height="750"/>
<image url="http://i1.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-cover.jpg" width="185" size="cover" height="278"/>
<image url="http://i3.themoviedb.org/posters/d8d/4bc919a1017a3c57fe009d8d/beetle-juice-thumb.jpg" width="92" size="thumb" height="138"/>
</poster>


and

<backdrop id="4bc9199e017a3c57fe009d6d">
<image url="http://i2.themoviedb.org/backdrops/d6d/4bc9199e017a3c57fe009d6d/beetle-juice-original.jpg" width="1920" size="original" height="1080"/>
<image url="http://i2.themoviedb.org/backdrops/d6d/4bc9199e017a3c57fe009d6d/beetle-juice-poster.jpg" width="780" size="poster" height="439"/>
<image url="http://i1.themoviedb.org/backdrops/d6d/4bc9199e017a3c57fe009d6d/beetle-juice-thumb.jpg" width="300" size="thumb" height="169"/>
</backdrop>

They aren't being parsed properly anymore. I took a look at share/bin/xbmc/scripts/scrapers/video/tmdb.xml but although it looks like a regexp nodelist I couldnt seem to fix anything in there.

Any ideas, this is a pain in the rear when it goes wrong especially since I'm still trying to add a backlog of dvd's over into the system

Thanks

Ed
Reply
#2
Wondering the same. Using the live rc1. I just tried to rebuild my movie library and I'm getting weird results. It's not pulling any fanart or posters, although its getting synopsis and cast. A handful of movies work correctly, and all the others exhibit the problematic behavior.
Reply
#3
debug logs please.
Reply
#4
here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly
Reply
#5
Thanks! Working now, was having the same issue as the guys above but only just noticed it about 10 minutes ago when I tried to add a new movie. Thanks again.
Reply
#6
Still not 100%, is it possible not all areas have been updated?

*Prior to modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...

Scraper not receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

*After modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

Scraper not receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...
Reply
#7
aptalca Wrote:here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly

Thanks, this made some progress. Still not fully working. Some still experience the same problem, others work great. I'll have to grab a debug later.
Reply
#8
The problem is TMDB changed their API so that now the ID field for images is a GUID, not a simple number. The current REGEX is matching ID on numeric digits only, but now it can contain the alphas a-f as well. I am looking for at fixing this - I'll post a new Xml file if no one else gets to it first. Shouldn't be a difficult fix.
Reply
#9
Can someone try the following xml?

http://pastebin.com/DTCa46NN

Again, replace your common TMDB.XML only, not the main one. Let me know if it works for you.
Reply
#10
The latest XML doesn't fix the problem on my system. The file is also a lot bigger than the original file. Is that correct?
Reply
#11
AaronD Wrote:Can someone try the following xml?

http://pastebin.com/DTCa46NN

Again, replace your common TMDB.XML only, not the main one. Let me know if it works for you.

Thanks for the try AaronD, but it does not resolve the issue on my system...
Missng i'd say one third of the FanArt and Thumbs for my library.
Sad
Reply
#12
aptalca Wrote:here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly

I'm not having the same issues as everyone else. This version has updated 100% of my movies (~260 that I've transfered so far). So just an FYI. Using 9.11 live install.
Reply
#13
aptalca Wrote:here's a working version of common tmdb.xml

http://pastebin.com/f1Um9i2Y


Replace your system\scrapers\video\common\tmdb.xml

I repeat, do not replace your main tmdb.xml, but the one in the common folder with this one


This will fix both tmdb and imdb scrapers as long as you have imdb set up to download fanart from tmdb.

The reason it broke is tmdb changed their api, search results for fanart have an added "width="xxx"" element in there that prevents the scraper from parsing the link correctly

Thanks aptalca and AaronD for the quick work on this!!

The quoted above fixed the problem for me, but only for the four most recent movies I've added. Don't want to blow away my entire library just yet.

thumbsup!
Reply
#14
odin836 Wrote:Still not 100%, is it possible not all areas have been updated?

*Prior to modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...

Scraper not receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

*After modification of tmdb.xml
Scraper receiving Thumbs and FanArt for:
2 Fast 2 Furious, 3:10 to Yuma, 21 Grams ...

Scraper not receiving Thumbs and FanArt for:
16 Blocks, 50 First Dates, A Clockwork Orange, Aeon Flux ...

I just checked the format the api delivered for "16 blocks", "50 first dates" and "Aeon Flux" and they all conform to the new style. tmdb's database might have had been partially updated before you tried to scrape, but looks like they should work now.

FYI. I didn't rescrape my entire library to test it, I just rescraped the latest few movies in my library.
Reply
#15
Thanks so much! Was pulling my hair out all night...updated XML file worked like a charm!
Reply
 
Thread Rating:
  • 0 Vote(s) - 0 Average



Logout Mark Read Team Forum Stats Members Help
themoviedb scraper now partly broken00