[Release] Universal Movie Scraper

  Thread Rating:
  • 7 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
tars Offline
Senior Member
Posts: 161
Joined: Aug 2015
Reputation: 1
Post: #1171
OK. Thanks for replying.
find quote
drfresh Offline
Junior Member
Posts: 1
Joined: Aug 2011
Reputation: 0
Post: #1172
I don't know if this issue has been mentioned before. If so I'm sorry - I did browse and search in order to see...

I have problems scrabing many short movies from IMDB. Here is but a few examples; I have many more of the same kind:

http://www.imdb.com/title/tt1850374/reference - filename: "Hawaiian Vacation (2011).1080p.mkv"

http://www.imdb.com/title/tt0034670/reference - filename: "Donald's Snow Fight (1942).1080p.mkv"

http://www.imdb.com/title/tt0043400/reference - filename: "Cheese Chasers (1951).1080p.mkv"


Is there some solution to this?

Thanks in advance.
find quote
wgstarks Offline
Posting Freak
Posts: 1,929
Joined: Jun 2013
Reputation: 16
Location: North Carolina, USA
Post: #1173
(2016-08-09 14:05)drfresh Wrote:  I don't know if this issue has been mentioned before. If so I'm sorry - I did browse and search in order to see...

I have problems scrabing many short movies from IMDB. Here is but a few examples; I have many more of the same kind:

http://www.imdb.com/title/tt1850374/reference - filename: "Hawaiian Vacation (2011).1080p.mkv"

http://www.imdb.com/title/tt0034670/reference - filename: "Donald's Snow Fight (1942).1080p.mkv"

http://www.imdb.com/title/tt0043400/reference - filename: "Cheese Chasers (1951).1080p.mkv"


Is there some solution to this?

Thanks in advance.

You probably need to enable short films in the settings for this video source (not the addon itself). Sorry, can't remember exactly what the setting is. Something like "All Titles".

Edit: It would probably also help if your file names match the title used by IMDB.

MacMini 5,2 HTPC (Jarvis 16.0/Aeon Madnox/MacOSX 10.10.5)/Onkyo TXNR609/Panny65VT30
(This post was last modified: 2016-08-09 14:29 by wgstarks.)
find quote
tars Offline
Senior Member
Posts: 161
Joined: Aug 2015
Reputation: 1
Post: #1174
(2016-08-09 14:05)drfresh Wrote:  I don't know if this issue has been mentioned before. If so I'm sorry - I did browse and search in order to see...

I have problems scrabing many short movies from IMDB. Here is but a few examples; I have many more of the same kind:

http://www.imdb.com/title/tt1850374/reference - filename: "Hawaiian Vacation (2011).1080p.mkv"

http://www.imdb.com/title/tt0034670/reference - filename: "Donald's Snow Fight (1942).1080p.mkv"

http://www.imdb.com/title/tt0043400/reference - filename: "Cheese Chasers (1951).1080p.mkv"


Is there some solution to this?

Thanks in advance.


To find those short movies, you need to tick "Include All Movie Categories" setting under "Search" tab of this scraper settings window.
find quote
firewater Offline
Junior Member
Posts: 12
Joined: Mar 2016
Reputation: 0
Post: #1175
Okay, so as I guessed on a previous page, making this scrapper detect any content that isn't recognized for not being on any database, or having a mismatched name, is rather easy. You simply have to paste the following code at the top of universal.xml, right after

Code:
<scraper framework="1.1" date="2014-07-24">

and before

Code:
<NfoUrl dest="3">

Code:
<CreateSearchUrl dest="3" clearbuffers="no">
        <RegExp dest="3" output="&lt;url&gt;http://search.yahoo.com/search?p=$$7&lt;/url&gt;" input="$$7">
            <RegExp dest="7" output="\1" input="$$1">
                <expression noclean="1" trim="1">(.+)</expression>
            </RegExp>
            <expression noclean="1"></expression>
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults dest="8" clearbuffers="no">
        <RegExp dest="8" output="&lt;results sorted=&quot;yes&quot;&gt;\1&lt;/results&gt;" input="$$5">
            <RegExp dest="5" output="&lt;entity&gt;&lt;title&gt;\1&lt;/title&gt;&lt;url&gt;http://search.yahoo.com/search?p=$$7&lt;/url&gt;&lt;/entity&gt;" input="$$1">
                <expression noclean="1" trim="1">&lt;title&gt;([^"]*)\-\sYahoo Search</expression>
            </RegExp>
            <expression noclean="1"></expression>
        </RegExp>
    </GetSearchResults>
    <GetDetails dest="3" clearbuffers="no">
        <RegExp dest="3" output="&lt;details&gt;\1&lt;/details&gt;" input="$$5">
            <RegExp dest="5+" output="&lt;title&gt;\1&lt;/title&gt;" input="$$1">
                <expression noclean="1" trim="1">&lt;title&gt;(.+)\s\-\sYahoo Search Results&lt;/title&gt;</expression>
            </RegExp>
            <expression noclean="1" trim="1"></expression>
        </RegExp>
    </GetDetails>

The code is from an addon called Stupid Movie Scrapper by "mrjwm2", so all credit goes to him. I believe Universal will fall back into this scrapper only when none of the other ones find a match, and so the video will be added to the MOVIES section using its filename as the title, and a thumbnail as the fanart. I do not know how to make it so the quality of the thumbnail is better than the default one (which is very poor), so if somebody knows about that, please tell me so.
(This post was last modified: 2016-08-21 06:49 by firewater.)
find quote
pjotr69 Offline
Junior Member
Posts: 6
Joined: Sep 2014
Reputation: 0
Post: #1176
Im using UMS and im quite happy in general with the preformence but sometime a serie or movie isn't picket up.
sometime manualy renew solves the problem some times it picks the wrong info.

A movie (single videofile) isn't that big problem add a xxx.nfo file and it appears but with a serie or non- official serie (self made group of videocontent) i can't get it appear in the serie library nor in the video. only in folder manualy searched and started.

It is not a big thing but it would be nice if i can manualy add to "serie" or video list.

How do i get it to work?
find quote
wgstarks Offline
Posting Freak
Posts: 1,929
Joined: Jun 2013
Reputation: 16
Location: North Carolina, USA
Post: #1177
(2016-08-23 13:00)pjotr69 Wrote:  Im using UMS and im quite happy in general with the preformence but sometime a serie or movie isn't picket up.
sometime manualy renew solves the problem some times it picks the wrong info.

A movie (single videofile) isn't that big problem add a xxx.nfo file and it appears but with a serie or non- official serie (self made group of videocontent) i can't get it appear in the serie library nor in the video. only in folder manualy searched and started.

It is not a big thing but it would be nice if i can manualy add to "serie" or video list.

How do i get it to work?

If you want UMS to scrape the info you need to add the series to TVDB since that's the source for all TV shows. Another alternative would be to use an nfo file (wiki) containing all the info. There are several library managers (wiki) that can help with this option.

MacMini 5,2 HTPC (Jarvis 16.0/Aeon Madnox/MacOSX 10.10.5)/Onkyo TXNR609/Panny65VT30
(This post was last modified: 2016-08-23 15:04 by wgstarks.)
find quote
pjotr69 Offline
Junior Member
Posts: 6
Joined: Sep 2014
Reputation: 0
Post: #1178
TVDB i think i have that source in "series" folder scraper. (it automatic choose that after choosing typ of content series/movie/

nfo file adding something like this?:

<episodedetails>
<title>My TV Episode</title>
<season>1</season>
<episode>1</episode>
<plot>he best episode in the world</plot>
<playcount>0</playcount>
<lastplayed></lastplayed>
<epbookmark>200</epbookmark> <!-- For media files containing multiple episodes,
where value is the time where the next episode begins in seconds -->
<displayseason>3</displayseason> <!-- For TV show specials, determines how the episode is sorted in the series -->
<displayepisode>4096</displayepisode> (this number i don't get)
</episodedetails>

so if i do make a s01e01 yadahjahda and a s01e02 yodi yodi
Do i make this .nfo for every episode separate or just for one in this folder to get the serie visible?
(it looks like for every episode one in my eye's)
(This post was last modified: 2016-08-23 21:12 by pjotr69.)
find quote
wgstarks Offline
Posting Freak
Posts: 1,929
Joined: Jun 2013
Reputation: 16
Location: North Carolina, USA
Post: #1179
I don't really mess with tv show nfo's much, but I would think you would want to do something like a TV Show nfo file for the series and then a episode nfo file for each episode.

Example: you could have a series "Home Movies" with whatever info you want to include. And then the episodes would be the individual videos with their own descriptions (year, plot, actors etc).

The examples shown in the wiki provide most of the XML tags but afaik not all tags are required. You can just use the ones you want usually.

<displayepisode> would be the episode number you want to be displayed if different from the actual episode number (my guess). If your skin supports that.

MacMini 5,2 HTPC (Jarvis 16.0/Aeon Madnox/MacOSX 10.10.5)/Onkyo TXNR609/Panny65VT30
find quote
pjotr69 Offline
Junior Member
Posts: 6
Joined: Sep 2014
Reputation: 0
Post: #1180
Thanks, i will give it a try when i have time later this week.
If it works i will post this for others.
find quote
wgstarks Offline
Posting Freak
Posts: 1,929
Joined: Jun 2013
Reputation: 16
Location: North Carolina, USA
Post: #1181
(2016-08-23 23:17)pjotr69 Wrote:  Thanks, i will give it a try when i have time later this week.
If it works i will post this for others.

Honestly, I think it is off topic for this thread since it has nothing to do with UMS. It's also already posted in the wiki. If any of the series are actually production tv shows that are not listed in TVDB please consider adding them so that others can benefit.

MacMini 5,2 HTPC (Jarvis 16.0/Aeon Madnox/MacOSX 10.10.5)/Onkyo TXNR609/Panny65VT30
find quote
pjotr69 Offline
Junior Member
Posts: 6
Joined: Sep 2014
Reputation: 0
Post: #1182
(2016-08-24 01:09)wgstarks Wrote:  
(2016-08-23 23:17)pjotr69 Wrote:  Thanks, i will give it a try when i have time later this week.
If it works i will post this for others.

Honestly, I think it is off topic for this thread since it has nothing to do with UMS. It's also already posted in the wiki.

ok sorry, i understand, its something to do with a database not the program?, (i didn't named series and used "home movie" because of eh caution)
The home movies are also thrue.

[/quote] TVDB please consider adding them so that others can benefit. [/quote]
ok don't know how but i think "see Wiki"?

ps Where can i post a found glitsh from the confluence skin? Jarvis 16.1
after a error and a reboot the shortcuts are gone from the home screen and needs to be selected again in systems.
find quote
nickr Offline
Super Moderator
Posts: 15,339
Joined: May 2009
Reputation: 308
Location: Christchurch NZ
Post: #1183
Adding a series to thetvdb is easy. Join, add. You benefit from that community, try contributing something.

If I have helped you or increased your knowledge, click the 'thank user' button to give thanks :) (People with less than 20 posts won't see the "Thank you" button.)
find quote
wgstarks Offline
Posting Freak
Posts: 1,929
Joined: Jun 2013
Reputation: 16
Location: North Carolina, USA
Post: #1184
(2016-08-24 08:08)pjotr69 Wrote:  ps Where can i post a found glitsh from the confluence skin? Jarvis 16.1
after a error and a reboot the shortcuts are gone from the home screen and needs to be selected again in systems.
In the Confluence skin support thread.

MacMini 5,2 HTPC (Jarvis 16.0/Aeon Madnox/MacOSX 10.10.5)/Onkyo TXNR609/Panny65VT30
find quote
CooperCGN Offline
Junior Member
Posts: 21
Joined: Aug 2016
Reputation: 0
Post: #1185
I have a problem with Kodi v17. I scraped some new movies which definitely have tags on TMDB but the tags don't show up. It's not skin related (using Estuary and Estuary MOD) as when I add those movies manually to the tags they are displayed correctly. Tags option in UMS settings is enabled of course. Any idea what I'm doing wrong? It seems as UMS isn't fetching them at all.
find quote
Post Reply