Stupid movie scraper stops working on my Rasperry Pi after upgrading to kodi 18.02
#1
Hi forum.

I'm having problem with the scraper "stupid movie database" that I found years ago and has been doing the job fine for my by just adding the filename into the library. (I have a lot of unknown short film and documentary that I wanted to have in my library).

But now after upgrading my kodi on my Rasperry to 18.02 its stops working and have the message "cannot connect to remote server".
Downgrading to 18.01, its fine again.

So what has been introduced in the 18.02 that tha scraper dosnt like on the rasperry?

I have made the same installation on a windows 10 PC and here the scraper works fine on 18.02.

I hope that there is some one that can find the problem:

Here is the scraper code:
xml:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<scraper name="Stupid Movie Database" content="movies" thumb="icon.png" framework="1.1" language="en">
    <CreateSearchUrl dest="3" clearbuffers="no">
        <RegExp dest="3" output="&lt;url&gt;http://search.yahoo.com/search?p=$$7&lt;/url&gt;" input="$$7">
            <RegExp dest="7" output="\1" input="$$1">
                <expression noclean="1" trim="1">(.+)</expression>
            </RegExp>
            <expression noclean="1"></expression>
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults dest="8" clearbuffers="no">
        <RegExp dest="8" output="&lt;results sorted=&quot;yes&quot;&gt;\1&lt;/results&gt;" input="$$5">
            <RegExp dest="5" output="&lt;entity&gt;&lt;title&gt;\1&lt;/title&gt;&lt;url&gt;http://search.yahoo.com/search?p=$$7&lt;/url&gt;&lt;/entity&gt;" input="$$1">
                <expression noclean="1" trim="1">&lt;title&gt;([^"]*)\-\sYahoo Search</expression>
            </RegExp>
            <expression noclean="1"></expression>
        </RegExp>
    </GetSearchResults>
    <GetDetails dest="3" clearbuffers="no">
        <RegExp dest="3" output="&lt;details&gt;\1&lt;/details&gt;" input="$$5">
            <RegExp dest="5+" output="&lt;title&gt;\1&lt;/title&gt;" input="$$1">
                <expression noclean="1" trim="1">&lt;title&gt;(.+)\s\-\sYahoo Search Results&lt;/title&gt;</expression>
            </RegExp>
            <expression noclean="1" trim="1"></expression>
        </RegExp>
    </GetDetails>
</scraper>
Reply
#2
...And that is Kodi 18.2 off cause, and not 18.02 :-)
Reply
#3
I found the source on github here... https://github.com/mrjwm2/metadata.movie.stupid

I downloaded the zip file (green button), installed via zip and scraped some random movie and it worked.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#4
(2019-06-24, 11:04)Karellen Wrote: I found the source on github here... https://github.com/mrjwm2/metadata.movie.stupid

I downloaded the zip file (green button), installed via zip and scraped some random movie and it worked.

Really strange. Its not working for me no matter what I do. Clean installed rasperry with Libreelec 19.02 (kodi18.2) and it is still the same. Could it be some default settings in the kodi on rasperry that I need to change?
Reply
#5
Provide a Debug Log and I can have a look. Post the full log though, not a snippet. The add-on logs some basic processes as shown in my log below. Last line confirms it was added.

xml:
DEBUG: VideoInfoScanner: Fetching url 'http://search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere' using Stupid Movie Scraper scraper (content: 'movies')
DEBUG: Thread VideoInfoDownloader start, auto delete: false
DEBUG: ADDON::CScraper::GetVideoDetails: Reading movie 'http://search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere' using Stupid Movie Scraper scraper (file: 'C:\KODI\KODI- v18.3 nightlies\portable_data\addons\metadata.movie.stupid', content: 'movies', version: '1.1.0')
DEBUG: CurlFile::Open(0x26da985f3f0) http://search.yahoo.com/search?p=The%20H...20Premiere
DEBUG: CCurlFile::Open - effective URL: <https://au.search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere>
DEBUG: CScraperUrl::Get: Using "UTF-8" charset for HTML "http://search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere"
DEBUG: scraper: GetDetails returned <details><title>The House (2017) - Premiere</title></details>
DEBUG: Thread VideoInfoDownloader 5140 terminating
DEBUG: VideoInfoScanner: Adding new item to movies:C:\KODI\Test- Movies\The House (2017) - Premiere_20190622_21022253.ts
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#6
(2019-06-24, 12:02)Karellen Wrote: Provide a Debug Log and I can have a look. Post the full log though, not a snippet. The add-on logs some basic processes as shown in my log below. Last line confirms it was added.

xml:
DEBUG: VideoInfoScanner: Fetching url 'http://search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere' using Stupid Movie Scraper scraper (content: 'movies')
DEBUG: Thread VideoInfoDownloader start, auto delete: false
DEBUG: ADDON::CScraper::GetVideoDetails: Reading movie 'http://search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere' using Stupid Movie Scraper scraper (file: 'C:\KODI\KODI- v18.3 nightlies\portable_data\addons\metadata.movie.stupid', content: 'movies', version: '1.1.0')
DEBUG: CurlFile::Open(0x26da985f3f0) http://search.yahoo.com/search?p=The%20H...20Premiere
DEBUG: CCurlFile::Open - effective URL: <https://au.search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere>
DEBUG: CScraperUrl::Get: Using "UTF-8" charset for HTML "http://search.yahoo.com/search?p=The%20House%20(2017)%20-%20Premiere"
DEBUG: scraper: GetDetails returned <details><title>The House (2017) - Premiere</title></details>
DEBUG: Thread VideoInfoDownloader 5140 terminating
DEBUG: VideoInfoScanner: Adding new item to movies:C:\KODI\Test- Movies\The House (2017) - Premiere_20190622_21022253.ts

HI again and thanks for your effort. This is what I found in the log.


2019-06-24 09:14:32.392 T:1216275312  NOTICE: VideoInfoScanner: Starting scan ..
2019-06-24 09:14:53.185 T:1216275312   ERROR: CCurlFile::FillBuffer - Failed: HTTP returned error 400
2019-06-24 09:14:53.185 T:1216275312   ERROR: CCurlFile::Open failed with code 400 for http://search.yahoo.com/search?p=Pink%20...The%20Wall:
2019-06-24 09:14:53.185 T:1216275312   ERROR: Run: Unable to parse web site
2019-06-24 09:14:56.440 T:1216275312 WARNING: No information found for item 'nfs://xx.xx.xx.xx/volume1/video/Udenlandske serier/Nye hentede/Pink Floyd - About The Wall.mp4', it won't be added to the library.
2019-06-24 09:14:58.616 T:1216275312   ERROR: CCurlFile::FillBuffer - Failed: HTTP returned error 400
2019-06-24 09:14:58.616 T:1216275312   ERROR: CCurlFile::Open failed with code 400 for http://search.yahoo.com/search?p=Pink%20...he%20Wall2:
2019-06-24 09:14:58.616 T:1216275312   ERROR: Run: Unable to parse web site
2019-06-24 09:15:00.290 T:1216275312 WARNING: No information found for item 'nfs://xx.xx.xx.xx/volume1/video/Udenlandske serier/Nye hentede/Pink Floyd - About The Wall2.mp4', it won't be added to the library.
Reply
#7
If you click on the yahoo link in your code snippet, can you access the search results page?
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#8
(2019-06-24, 12:32)Karellen Wrote: If you click on the yahoo link in your code snippet, can you access the search results page?
It open yahoo but didnt find a result.
Reply
#9
If I open the Yahoo link from your log I get

Image
Reply
#10
(2019-06-24, 13:48)jjd-uk Wrote: If I open the Yahoo link from your log I get

Image
Yes this is also what I see when I open the link. But Why is the result not copied to the library?
Reply
#11
(2019-06-24, 19:27)Longfors Wrote: Yes this is also what I see when I open the link. But Why is the result not copied to the library?
I do not know. This is a question better asked of the developer of the add-on.

The developer is @mrjwm2, but he has not logged in since 2016. Also this is the thread for the scraper here... https://forum.kodi.tv/showthread.php?tid=249546

You could also try leaving an issue report on the github repository... https://github.com/mrjwm2/metadata.movie.stupid
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#12
Found the problem and solved it.

Apparently is kodi 18.2 on a Rasperry not allowed to connect to http sites.
Just adding s to http (https) on the two lines in the scraper code does the trick :-)


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<scraper name="Stupid Movie Database" content="movies" thumb="icon.png" framework="1.1" language="en">
<CreateSearchUrl dest="3" clearbuffers="no">
<RegExp dest="3" output="&lt;url&gt;https://search.yahoo.com/search?p=$$7&lt;/url&gt;" input="$$7">
<RegExp dest="7" output="\1" input="$$1">
<expression noclean="1" trim="1">(.+)</expression>
</RegExp>
<expression noclean="1"></expression>
</RegExp>
</CreateSearchUrl>
<GetSearchResults dest="8" clearbuffers="no">
<RegExp dest="8" output="&lt;results sorted=&quot;yes&quot;&gt;\1&lt;/results&gt;" input="$$5">
<RegExp dest="5" output="&lt;entity&gt;&lt;title&gt;\1&lt;/title&gt;&lt;url&gt;https://search.yahoo.com/search?p=$$7&lt;/url&gt;&lt;/entity&gt;" input="$$1">
<expression noclean="1" trim="1">&lt;title&gt;([^"]*)\-\sYahoo Search</expression>
</RegExp>
<expression noclean="1"></expression>
</RegExp>
</GetSearchResults>
<GetDetails dest="3" clearbuffers="no">
<RegExp dest="3" output="&lt;details&gt;\1&lt;/details&gt;" input="$$5">
<RegExp dest="5+" output="&lt;title&gt;\1&lt;/title&gt;" input="$$1">
<expression noclean="1" trim="1">&lt;title&gt;(.+)\s\-\sYahoo Search Results&lt;/title&gt;</expression>
</RegExp>
<expression noclean="1" trim="1"></expression>
</RegExp>
</GetDetails>
</scraper>
Reply
#13
Great!! Thanks for letting us know the fix.

Also thanks as I did not know this scraper was still around and working.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#14
Does this scraper work for TV Shows (or can the scraper be adapted for TV Shows)?
Reply
#15
(2020-08-17, 17:26)elsmandino Wrote: Does this scraper work for TV Shows (or can the scraper be adapted for TV Shows)?

I don't think this scraper works at all anymore, but when it did it was for Movies only. As for adapting it, I have no idea.

Which tv shows are you trying to scrape?
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply

Logout Mark Read Team Forum Stats Members Help
Stupid movie scraper stops working on my Rasperry Pi after upgrading to kodi 18.020