[Release] Universal Movie Scraper

  Thread Rating:
  • 7 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Szymon_Zy Offline
Junior Member
Posts: 22
Joined: Nov 2013
Reputation: 0
Post: #1216
Is it possible to manually update movie infos and for those that are wrongly discovered (old polish movies) add ttXXXXXX as title in search box so the scrapper will find correct movie? I have real problems with movies form '50-'70 of XX century. Files have correct names and IMDB also have the same name for example "Kryptonim Nektar.mkv" and nothing found, "Ostatni Kurs" and "Last Cab to Darvin". I don't see way to manually set propper imdb id.
(This post was last modified: 2017-02-06 23:17 by Szymon_Zy.)
find quote
T-E-N Offline
Junior Member
Posts: 1
Joined: Feb 2017
Reputation: 0
Lightbulb  Teaching a new pony older tricks (of the VDR variety) - RE: Universal Movie Scraper
Post: #1217
(2012-04-24 15:15)olympia Wrote:  release a new scraper, called 'Universal Scraper'
  • Currently it supports 3 sites: IMDb, TMDb, Trakt.tv and Rotten Tomatoes
The only scraper to support IMDb in Kodi 17 ?

Like many Europeans, I have a decade's trove of DVB recordings (legal; we pay both a TV tax and blank-media levy for the "privilege") made on VDR, weighing Terabytes and structured in one of the following ways:
/var/lib/video.00/Film/2014-12- 00002.ts index info
/var/lib/video.00/%Next/2009-12- 002.vdr index.vdr info.vdr marks.vdr resume.vdr
The latter typically being older recordings in the PES flavor of MPEG-2.

The recordings are stored in chunks of 2GB max. to ensure portability across file systems.

I.e. the structure usually in /var/lib/video.00 is like this:
    001.vdr or 00001.ts and subsequent

We have a hard time getting KODI to scrape these, cf. http://forum.kodi.tv/showthread.php?tid=216204 (http://www.vdr-portal.de/board60-linux/b...der-beste/, http://www.vdr-portal.de/board60-linux/b...anreichern, http://www.vdr-portal.de/board60-linux/b...it-fanart/, https://www.heise.de/forum/heise-online/...ng_3210315 for anyone actually inclined to read German).

To provide http://kodi.wiki/view/NFO_files/Movies#V...ning_a_URL, one daunting one-liner has been contrived:
while read -r r; do if [ ! -f $r/$(basename $(dirname $r)).nfo ]; then echo $(basename $(dirname $r)) >$r/$(basename $(dirname $r)).nfo; fi; done < <(dirname $(find /var/lib/video.00 -name '0*.vdr' -or -name '0*.ts') | uniq -u)
This aims to achieve something like described at the above URL: 'If you use the "Movies are in separate folders that match the movie title" scraper setting Kodi will use the first nfo file it finds in the folder (other than the .nfo files described above) and apply it to any valid video file it finds in the same folder."
However without an easy API to get (out of bash, other than trying a two-stage grab of e.g. http://akas.imdb.com/find?q=Avatar&s=tt possibly without &exact=true, then RegEx to <td class="result_text"> <a href="/title/tt0499549/) the most likely deeplink from IMDb by (occasionally truncated) name, it can only write the latter rather than an URL into the NFO, which gets ignored in scraping:
A database full of movies misidentified to have "001" in their name is the result, and their 2GB chunks are not combined into one single KODI database entry (still no better way to fix the latter than advancedsettings.xml? E.g. in each NFO file individually for the respective directory since moviestacking will be path-specific!).

Can you recommend what (besides the name of the directory's parent) a minimal NFO written this way should contain in this case for Universal (or any other) Movie Scraper to get the contents (mostly) right?

On the other hand, Universal Movie Scraper might be taught to do its magic on these structures as well:
Underscores and escaped special characters you'll probably have a habit to ignore from other formats already.
Similar for the leading % which simply indicates advertising has been cut out.

info or info.vdr are descriptions similar to NFO which you may want to parse for this line:
T One particular movie's title
I.e. the title is:
grep '^T\ ' /var/lib/video.00/One_particular_movie/2*/info* | sed 's/^T\ \(.*\)/\1/g'
Some iconv may need to be applied where older titles aren't UTF-8 yet.
(This post was last modified: 2017-02-21 00:24 by T-E-N.)
find quote
iMak Offline
Senior Member
Posts: 232
Joined: Feb 2016
Reputation: 0
Post: #1218
The Addon is not available in Kodi 18 repository.
find quote
Post Reply