[Release] Universal Movie Scraper

  Thread Rating:
  • 7 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
wgstarks Offline
Posting Freak
Posts: 1,599
Joined: Jun 2013
Reputation: 12
Location: North Carolina, USA
Post: #1006
(2015-07-28 07:53)seftongillis Wrote:  Hello,

I wonder if you could help me, I have an issue with an nfo file. I installed the VideoExtras addon and successfully scraped a movie in its own folder with an extra in its own Extras folder. However, when trying to scrape another movie of the same set, with a videoextras.nfo file inside its folder pointing to the other Extras directory (shared extras), the scraping fails (no title, no cast, no crew, etc.). I suspect it's because the nfo file is interpreted to have information about the movie rather than the extras. If I remove the nfo file the movie is scraped properly but the extra is not associated with the movie. I understand nfo files are meant to be ignored when they don't have the same name as the movie file, but in this case the videoextras.nfo file is always picked up by this scraper.

Here's the videoextras.nfo file:

Code:
<?xml version='1.0' encoding='UTF-8'?>
<videoextras>
    <directory>..\Raiders of the Lost Ark (1981).DVD [en, es-lat, sub] [xtra]\Extras</directory>
</videoextras>

I store the movies in VIDEO_TS folders inside the movie folders, along with the Extras folders when they apply. However, just to make sure this wasn't the problem, I tested the scraper using a single movie file in a folder, rather than a DVD structure, and the problem persisted.

Any help will be greatly appreciated.

Have you tried creating a new "ignore" in advanced settings? Just a wild guess, but might work.

[Image: all-thin-banner.jpg]
MacMini5,2 HTPC (Helix 14.0/Aeon Nox 5.5/MacOSX 10.8.5)/Onkyo TXNR609/Panny65VT30
find quote
seftongillis Offline
Junior Member
Posts: 22
Joined: Jan 2012
Reputation: 0
Post: #1007
Do you mean, an "exclude from scan"?

These are the excludes added by VideoExtras to advancedsettings.xml:
Code:
<!-- VideoExtras: Section Start -->
      <excludefromscan action="append">
        <regexp>/Extras/</regexp>
        <regexp>[\\/]Extras[\\/]</regexp>
      </excludefromscan>
      <excludetvshowsfromscan action="append">
        <regexp>/Extras/</regexp>
        <regexp>[\\/]Extras[\\/]</regexp>
      </excludetvshowsfromscan>
      <!-- VideoExtras: Section End -->

It is with these settings that the scraper reads the videoextras.nfo file as having information about the movie. I wasn't sure what ignore condition to add, so I tried this:
Code:
<regexp>/videoextras.nfo/</regexp>

The movie is scraped properly but the extras are ignored.
find quote
wgstarks Offline
Posting Freak
Posts: 1,599
Joined: Jun 2013
Reputation: 12
Location: North Carolina, USA
Post: #1008
(2015-07-28 21:56)seftongillis Wrote:  Do you mean, an "exclude from scan"?

These are the excludes added by VideoExtras to advancedsettings.xml:
Code:
<!-- VideoExtras: Section Start -->
      <excludefromscan action="append">
        <regexp>/Extras/</regexp>
        <regexp>[\\/]Extras[\\/]</regexp>
      </excludefromscan>
      <excludetvshowsfromscan action="append">
        <regexp>/Extras/</regexp>
        <regexp>[\\/]Extras[\\/]</regexp>
      </excludetvshowsfromscan>
      <!-- VideoExtras: Section End -->

It is with these settings that the scraper reads the videoextras.nfo file as having information about the movie. I wasn't sure what ignore condition to add, so I tried this:
Code:
<regexp>/videoextras.nfo/</regexp>

The movie is scraped properly but the extras are ignored.
That should do the trick then. I think the VideoExtras addon uses a seperate scan. This is really just a work around. Not sure why UMS would pick up a file named videoextras.nfo.

[Image: all-thin-banner.jpg]
MacMini5,2 HTPC (Helix 14.0/Aeon Nox 5.5/MacOSX 10.8.5)/Onkyo TXNR609/Panny65VT30
find quote
seftongillis Offline
Junior Member
Posts: 22
Joined: Jan 2012
Reputation: 0
Post: #1009
Oh, should have made myself more clear. I mean, the scraper scans fine but the extras are ignored by VideoExtras, so my problem is yet unsolved. I wouldn't know for sure, but this makes me think VideoExtras actually depends on the movie scraper instead of using its own.
(This post was last modified: 2015-07-28 22:33 by seftongillis.)
find quote
donkamillo Offline
Junior Member
Posts: 43
Joined: Mar 2011
Reputation: 0
Location: Germany
Post: #1010
Hi!

It seams that I can no longer scrape movies with the universal movie scraper. I tried to re-add a movie from a NFO file, but absolutely nothing happens.

In the log file I see always the same two lines after trying that:

Code:
14:11:31 T:140099370014720 WARNING: failed to load scraper XML from /home/mediacenter/.kodi/addons/metadata.universal/universal.xml
14:11:31 T:140099370014720   ERROR: exception in CApplication::FrameMove()

Anyone have an idea what that might be? A debug log is here:

http://xbmclogs.com/pkc7k2ocd

Regards!
find quote
donkamillo Offline
Junior Member
Posts: 43
Joined: Mar 2011
Reputation: 0
Location: Germany
Post: #1011
OK, it works again! :-) I deleted the .kodi folder to start from scratch. There were so many problems that a clean configuration was the easyest way to go. But as I installed all the plugins and addons, some dependencies were obviously not installed. I noticed that some folders, like metadata.common.imd.com were missing. I retreaved them from an older backup. At least the NFO files are imported correctly.

Why the dependencies were not installed I cannot say. At least its working again! :-)

Regards!
find quote
Goran65 Offline
Junior Member
Posts: 1
Joined: Jul 2015
Reputation: 0
Post: #1012
(2015-07-17 12:39)muzzakus Wrote:  I'm getting cannot contact remote server with this. Anybody else? Have been using this successfully since the beginning.

I am having the same problem.
Can any one help?

Best
Goran
find quote
finalmakerr Offline
Junior Member
Posts: 45
Joined: Dec 2013
Reputation: 0
Post: #1013
I had this problem long time ago.
I suggest to try the follow steps:
1. Uninstall addon and it's related addons (see the addon.xml file - Sections import).
2. Remove the addon userdata folder.
3. Install the addon back and check if issue is gone.
4. You may also try to scrape a different folder and see if problem solved.
(This post was last modified: 2015-08-01 14:49 by finalmakerr.)
find quote
GStacks13 Offline
Junior Member
Posts: 15
Joined: May 2015
Reputation: 0
Post: #1014
I'm having a similar issue ever since I upgraded my computer to Windows 10. I decided to go for a fresh install, which meant starting from scratch with Kodi. But when I installed UMS, the installation failed, saying dependencies couldn't be installed. I actually got it to install once (don't ask me how), but then it would tell me it couldn't contact the remote server. I uninstalled, and now it's not letting me reinstall.

Anyone have any luck solving this issue?
find quote
Dave the Minion Offline
Member
Posts: 87
Joined: Jun 2015
Reputation: 0
Post: #1015
I am having a new problem and I'm not sure if it's AD related or an issue with the Universal Scraper.

When I scrape a new movie into my database when I've already created all the local extra art I want (logo, clearart and disc) Artwork Downloader would easily allow me to manually add these arts and when I accessed the artwork options I only had 2; local art or browse:

When I run AD in automatic mode the proper local art is added no problem.

Now when I go to quickly add these arts I've given a ton of options in these arts that all seem to be posters and banners and others things that are not the proper types of art let alone anything local. Is it AD doing something odd, or, as I suspect, the Universal Scraper populating thumbnail fields incorrectly?

Thanks!
find quote
DeCaf Offline
Junior Member
Posts: 5
Joined: Jul 2012
Reputation: 0
Post: #1016
I just created a fresh installation of Kodi Isengard, and I proceeded to then install the Universal Movie Scraper add-on, without changing any configuration. Then I attempted to add a source with movies, and scrape it, but it seems to terminate the scraping at the first Movie. Examining the log reveals the following lines:

Code:
17:03:06 T:3772   DEBUG: VideoInfoScanner: Scanning dir 'smb://192.168.0.5/Storage/Video/Movies/' as not in the database
17:03:08 T:436   DEBUG: ------ Window Deinit (Pointer.xml) ------
17:03:08 T:3772   DEBUG: VideoInfoScanner: No (new) information was found in dir smb://192.168.0.5/Storage/Video/Movies/
17:03:08 T:3772   DEBUG: VideoInfoScanner: Scanning dir 'smb://192.168.0.5/Storage/Video/Movies/3 Idiots (2009)/' as not in the database
17:03:08 T:3772   DEBUG: CUtil::ExcludeFileOrFolder: File 'smb://192.168.0.5/Storage/Video/Movies/3 Idiots (2009)/3.Idiots.2009-trailer.mp4' excluded. (Matches exclude rule RegExp:'-trailer')
17:03:08 T:3772 WARNING: failed to load scraper XML from C:\Users\HTPC\AppData\Roaming\Kodi\addons\metadata.universal\universal.xml
17:03:08 T:3772   ERROR: VideoInfoScanner: Exception while scanning.
17:03:08 T:3772   DEBUG: CAnnouncementManager - Announcement: OnScanFinished from xbmc
17:03:08 T:3772   DEBUG: GOT ANNOUNCEMENT, type: 16, from xbmc, message OnScanFinished

Any ideas on how to solve this?

EDIT:

After much fiddling I discovered that if I copied the addon directory metadata.common.impa.com from an old (and working) installation of Kodi to the new installation, the Universal Movie Scraper works again. Shouldn't this be included per default when installing Universal Movie Scraper if it is required?
(This post was last modified: 2015-08-01 17:51 by DeCaf.)
find quote
rksn Offline
Junior Member
Posts: 1
Joined: Aug 2015
Reputation: 0
Post: #1017
(2015-08-01 17:17)DeCaf Wrote:  I just created a fresh installation of Kodi Isengard, and I proceeded to then install the Universal Movie Scraper add-on, without changing any configuration. Then I attempted to add a source with movies, and scrape it, but it seems to terminate the scraping at the first Movie. Examining the log reveals the following lines:

Code:
17:03:06 T:3772   DEBUG: VideoInfoScanner: Scanning dir 'smb://192.168.0.5/Storage/Video/Movies/' as not in the database
17:03:08 T:436   DEBUG: ------ Window Deinit (Pointer.xml) ------
17:03:08 T:3772   DEBUG: VideoInfoScanner: No (new) information was found in dir smb://192.168.0.5/Storage/Video/Movies/
17:03:08 T:3772   DEBUG: VideoInfoScanner: Scanning dir 'smb://192.168.0.5/Storage/Video/Movies/3 Idiots (2009)/' as not in the database
17:03:08 T:3772   DEBUG: CUtil::ExcludeFileOrFolder: File 'smb://192.168.0.5/Storage/Video/Movies/3 Idiots (2009)/3.Idiots.2009-trailer.mp4' excluded. (Matches exclude rule RegExp:'-trailer')
17:03:08 T:3772 WARNING: failed to load scraper XML from C:\Users\HTPC\AppData\Roaming\Kodi\addons\metadata.universal\universal.xml
17:03:08 T:3772   ERROR: VideoInfoScanner: Exception while scanning.
17:03:08 T:3772   DEBUG: CAnnouncementManager - Announcement: OnScanFinished from xbmc
17:03:08 T:3772   DEBUG: GOT ANNOUNCEMENT, type: 16, from xbmc, message OnScanFinished

Any ideas on how to solve this?

EDIT:

After much fiddling I discovered that if I copied the addon directory metadata.common.impa.com from an old (and working) installation of Kodi to the new installation, the Universal Movie Scraper works again. Shouldn't this be included per default when installing Universal Movie Scraper if it is required?

Exactly the same issue here. I had no old installation, so i downloaded the impa scraper somewhere (just google for it) and dropped it into the addons folder. Now it works just fine. This issue is new for me, I never had any problems with the universal movie scraper before.
find quote
mahidi Offline
Junior Member
Posts: 1
Joined: Aug 2015
Reputation: 0
Post: #1018
Hi all,

I reinstalled my systems (Intel NUC with Openelec) with the latest version of Openelec with Kodi 15 RC2.
When i tried to add new movies it showed me the message that the scraper was down.
I found this error in the log file of Kodi:

"WARNING: failed to load scraper XML from /storage/.kodi/addons/metadata.universal/universal.xml"

I put in the settings of UMS to use multiple sites for the information, so i looked in the addon file for the required addons.
After opening the file addon.xml that was located in /.kodi/addons/metadata.universal, i found the following:

<requires>
<import addon="xbmc.metadata" version="2.1.0"/>
<import addon="metadata.common.fanart.tv" version="3.1.0"/>
<import addon="metadata.common.hdtrailers.net" version="2.0.5"/>
<import addon="metadata.common.imdb.com" version="2.7.8"/>
<import addon="metadata.common.impa.com" version="1.1.2"/>
<import addon="metadata.common.movieposterdb.com" version="1.1.1"/>
<import addon="metadata.common.ofdb.de" version="1.0.3"/>
<import addon="metadata.common.port.hu" version="1.0.3"/>
<import addon="metadata.common.rt.com" version="1.4.0"/>
<import addon="metadata.common.themoviedb.org" version="2.12.1"/>
<import addon="metadata.common.trakt.tv" version="1.1.2"/>
</requires>

I was missing the first 4 when looking in to the addons folder (./kodi/addons) and i tried to install them through the addons option within Kodi.
But they weren't there (yet).

Since i've reinstalled my systems with the latest openelec version and i didn't created a backup (i wanted to start fresh), i couldn't get them from a earlier backup.

So what i did was this.

1. check which addons your missing
2a. copy them from an earlier created backup
2b. download them from a Kodi ftp or mirror site (http://ftp.bit.nl/pub/xbmc/addons/helix/)
3. extract the files in to the same name as the download file
4. copy the folders to /.kodi/addons folder
5. restart Kodi or System (restart did it for me)
6. try to rescan for new movies

The movies we're added fine like i was used to and no errors we're found in the log file from Kodi.

I hope this will work also for you guys.

Mahidi
(This post was last modified: Yesterday 02:18 by mahidi.)
find quote
Post Reply