Kodi Community Forum
Mixed XML & URLl video NFO bug? - Printable Version

+- Kodi Community Forum (http://forum.kodi.tv)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Scraper Development (/forumdisplay.php?fid=60)
+--- Thread: Mixed XML & URLl video NFO bug? (/showthread.php?tid=53135)

Pages: 1 2 3


Mixed XML & URLl video NFO bug? - pletopia - 2009-06-18 20:41

this is something i noticed a while back but never got around to reporting it but since its still present in builds (i have r21082 at the moment running)

basically i REALLY like the idea of mixed url and xml nfo's .. it lets me have the abililty to specify certain fields in the db while having xbmc import from imdb.com the latest info for other fields when i upgrade or want to reload

anyway .. what i found is that some (not all tags) get pulled and entered into the db from both the nfo and imdb

i created a nfo file with MIP and added a imdb.com url to the movies page at the end of the file .. after i imported the movie into xbmc i open up the myvideos34.db file in SQLiteSpy and took a look at the movie in the movie table

looking through the columns i noticed some have dupes
genre (Action / Adventure / Thriller / Action / Adventure / Thriller)
director (Lee Tamahori / Lee Tamahori)
studio (Eon Productions / Eon Productions)

the only reason i bring this up is that i want to be able to set studio manually in some movies since some skins now have studio flags and not every default scrape from imdb gives a good studio (like in my example i rather set studio to United Artists

when i manually set studio in the nfo file <studio>United Artists</studio> and i looked into the db i had "Eon Productions / United Artists" in the column

hope this makes sense

copy of my test nfo file to fine duplicates
Code:
<?xml version="1.0" encoding="utf-8"?>
<movie xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <title>007 (2002) Die Another Day</title>
  <rating>6.0</rating>
  <year>2002</year>
  <top250 />
  <votes>59,322</votes>
  <outline>James Bond is sent to investigate the connection between a North Korean terrorist and a diamond mogul who is funding the development of an international space weapon. </outline>
  <plot>Pierce Brosnan gives one last mission as James Bond. Starting off in North Korea, Bond is betrayed and captured. 14 months later, Bond is set free, but traded for Zao who was captured by MI6. When back in his world, Bond sets off to track down Zao. Bond gets caught up in yet another scheme which sends him to millionaire Gustav Graves. Another MI6 agent known as Miranda Frost is also posing as a friend of Graves. Bond is invited to a presentation held by Graves about a satellite found in space which can project a huge laser beam. Bond must stop this madman with a fellow American agent, known as Jinx. Whilst Bond tries to stop Graves and Zao, will he finally reveal who betrayed him?</plot>
  <tagline>Events don't get any bigger than... </tagline>
  <runtime>133 min </runtime>
  <thumb />
  <fanart url="http://www.themoviedb.org/" />
  <mpaa> Rated PG-13 for action violence and sexuality. (theatrical version); Rated PG-13 for intense sequences of action violence, and sexual content including innuendo. (2006 video version)</mpaa>
  <playcount />
  <File />
  <Path />
  <filenameandpath />
  <id>tt0246460</id>
  <genre>Action / Adventure / Thriller</genre>
  <credits>Lee Tamahori / Ian Fleming (characters) / Neal Purvis (written by) &amp;  / Robert Wade (written by)</credits>
  <director>Lee Tamahori</director>
  <premiered />
  <status />
  <studio>Eon Productions</studio>
  <album />
  <trailer />
  <watched>no</watched>
  <actor>
    <name>Pierce Brosnan</name>
    <role>James Bond</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMTIwMzU0NzAyOF5BMl5BanBnXkFtZTcwODg5NjYyMQ@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Halle Berry</name>
    <role>Jinx</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMjAzNDc1MzMyNF5BMl5BanBnXkFtZTcwMzk3NTYyMQ@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Toby Stephens</name>
    <role>Gustav Graves</role>
    <thumb>http://i.media-imdb.com/images/tn15/addtiny.gif</thumb>
  </actor>
  <actor>
    <name>Rosamund Pike</name>
    <role>Miranda Frost</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMjA1MzM1MDA2Nl5BMl5BanBnXkFtZTcwNzQxMzEyMQ@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Rick Yune</name>
    <role>Zao</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMTY0MDU2OTkyMV5BMl5BanBnXkFtZTcwOTIzMDQ0Mg@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Michael Madsen</name>
    <role>Damian Falco</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMjE3NTEyMzI2Nl5BMl5BanBnXkFtZTcwMTY0MDkzMQ@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Will Yun Lee</name>
    <role>Colonel Moon</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMjA1MDUxMzEwNV5BMl5BanBnXkFtZTcwNTc2NjcyMQ@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Kenneth Tsang</name>
    <role>General Moon</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BMTU4ODYwODg4Ml5BMl5BanBnXkFtZTcwMDE0MTUxMQ@@._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Mikhail Gorevoy</name>
    <role>Vlad</role>
    <thumb>http://i.media-imdb.com/images/tn15/addtiny.gif</thumb>
  </actor>
  <actor>
    <name>Lawrence Makoare</name>
    <role>Mr. Kil</role>
    <thumb>http://i.media-imdb.com/images/tn15/addtiny.gif</thumb>
  </actor>
  <actor>
    <name>Colin Salmon</name>
    <role>Charles Robinson</role>
    <thumb>http://ia.media-imdb.com/images/M/MV5BOTE3OTg3MDYyM15BMl5BanBnXkFtZTYwODUwODI4._V1._SY275_SX400_.jpg</thumb>
  </actor>
  <actor>
    <name>Samantha Bond</name>
    <role>Miss Moneypenny</role>
    <thumb>http://i.media-imdb.com/images/tn15/addtiny.gif</thumb>
  </actor>
</movie>
http://www.imdb.com/title/tt0246460/

copy of my test nfo file to inject "United Artists" instead of "Eon Productions"
Code:
<?xml version="1.0" encoding="utf-8"?>
<movie xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <title>007 (2002) Die Another Day</title>
  <studio>unitedartists</studio>
</movie>
http://www.imdb.com/title/tt0246460/

with this file .. i only get the studio column duplicated .. not the genre or director


- spiff - 2009-06-18 20:50

those are additive tags... come up with a sane way to handle it for all cases and i'll consider it.


- pletopia - 2009-06-19 00:37

umm .. what you mean sane way ??

basically it should be that if there is a xml tag present for specific column in the db .. it shouldn't be pulled from imdb

it works for all other tags (title, plot, rating, votes, etc) .. i don't see why it shouldn't pull data properly from both sources for most columns but randomly pull it for those three

i only bring this up cause these "additive tags" as you call them are actually quite useful

studio has now become important due to skins having studio flags and this will let you specify which ones will get displayed

genre recently i've noticed has also become more important because in my opinion imdb is too liberal with there genre categorization and i would like to limit my movies to specific genre's which might not coincide with what imdb has them listed as


- jmarshall - 2009-06-19 01:56

I believe he means that it's not necessarily an all or nothing thing. One could either completely ignore any genres scraped online if a genre is present in the XML, or one could append it to the list. This has implications for scrapers that are chained together as well. I presume that the code currently does the latter.

It thus would require modification, which may or may not be simple.

Perhaps a <genre append="true"> might be a solution?

Cheers,
Jonathan


- pletopia - 2009-06-19 07:52

ahh .. now i understand what spiff meant by additive .. i'm curious how feasible it would be to make a <xml append = false> setting for advancedsettings.xml


- spiff - 2009-06-19 17:57

hmm, i think the solution would be a <genre clear="true">. the specified xml information is applied after we have scraped. pletopia, you haven't quite grasped the problem yet if you think an advancedsetting will help Smile

see, what i mean with additive is that when we encounter such a tag we add it to whatever is already around. this is necessary since you can have multiple genre's in the xml, and due to that genre's may in theory come from multiple scraper functions (the chaining jmarshall is refering to). the specified xml in the mixed nfo is treated just like a normal chain and loaded after the scraping process has finished. by specifying clear="true"/append="false" (whatever we choose) on the first genre in your mixed nfo you can clear those added earlier


- spiff - 2009-06-19 18:20

http://trac.xbmc.org/ticket/6790

i'm gone for the weekend, feel free to test it


- pletopia - 2009-06-20 15:43

okay .. i got ya Smile

so first the info is pulled from imdb.com THEN it gets pulled from xml nfo .. gotcha


- Paradise - 2009-06-24 00:22

Hm, i think my english is good, but i don't really understand that or how to include or handle this now.

Also see http://forum.xbmc.org/showthread.php?tid=53325 , cause therer are still other questions not answered.


- spiff - 2009-06-24 07:40

if your english is so good, you go read google if you do not understand the concept of a diff


- Paradise - 2009-06-24 13:49

There is no need for googleing for diff in english, cause its the same in german.

I mean where to i have to set clear="true"? Just one time or in <genre>, <credits>, <studio>...., everywhere.

Also i can't find a VideoInfoTag file on my windows.


- spiff - 2009-06-24 21:18

then what is the problem? i have already explained how it works AND that it is only a diff on trac at this point.

and VideoInfoTag is (shockingly) a SOURCE file. you know, what we apply those german diffs to ;P


- sho - 2009-06-25 12:29

The non programmer version:

This is an issue that has just been discovered.
The fix or workaround is being contemplated and is not part of the program yet.
If you have the skills you can get the diff that spiff graciously made and apply it to the source and compile your own XBMC version.
Otherwise you have to wait patiently until this or another approach will be put into the mainline code and a binary is released.


- Paradise - 2009-06-25 17:16

Ok, thanks sho.

For me a diff is a file comparison utility. And i didn't know that VideoInfoTag.cpp is something in the source. I tought its a file in XBMC. Also i thought "version changed from 9.04 "Babylon" to SVN" means it is in the latest SVN.


- Aenima99x - 2009-07-01 05:34

Spiff - I'm running svn ver. 21304 compiled today and I'm still getting duplicates in *some* fields when using nfo's with url's. This used to work before, I'm just not sure when it quit working. This is on Jaunty 32-bit.
I've also deleted the video db just to ensure there were no problems there.
Debug log here
Thanks