Kodi Community Forum
Need help in writin a really simple scraper - Printable Version

+- Kodi Community Forum (http://forum.kodi.tv)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Scraper Development (/forumdisplay.php?fid=60)
+--- Thread: Need help in writin a really simple scraper (/showthread.php?tid=116584)



Scraper issues with a really simple ones - Finalspace - 2011-12-09 23:38

Hello, i am trying to write a simple scraper which is simply a pass-through interface for my local webservice.
Every content which requires the scraper (SearchURL, SearchResults, Details, EpisodeGuideURL) will be returned from the webservice.

My current implementation returns the correct search url: "http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchou".

From wireshark i can see the HTTP GET is succeeded but XBMC UI displays "Unable to connect to remote server".

Here are the scraper parts from the xbmc log file:

Code:
22:29:55 T:140615347619712   DEBUG: ------ Window Init (DialogVideoScan.xml) ------
22:29:55 T:140615347619712    INFO: Loading skin file: DialogVideoScan.xml
22:29:55 T:140614672590592   DEBUG: Thread VIDEO::CVideoInfoScanner start, auto delete: 0
22:29:55 T:140614672590592  NOTICE: VideoInfoScanner: Starting scan ..
22:29:55 T:140614672590592   DEBUG: VideoInfoScanner: No NFO file found. Using title search for '/home/final/animedirs/Kami-sama no Memo-chou/'
22:29:55 T:140614672590592   DEBUG: FindMovie: Searching for 'Kami-sama no Memo-chou' using Anime Plugin scraper (path: '/home/final/.xbmc/addons/metadata.animeplugin', content: 'tvshows', version: '1.0.0')
22:29:55 T:140614672590592   DEBUG: scraper: CreateSearchUrl returned <url>http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchou</url>
22:29:55 T:140614672590592   DEBUG: FileCurl::Open(0x7fe3679c10e0) http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchou
22:29:55 T:140614672590592   DEBUG: scraper: GetSearchResults returned http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchuo
22:29:55 T:140614672590592   ERROR: Run: Unable to parse XML

Here is returned XML Content from the search url:

Code:
<?xml version="1.0" encoding="iso-8859-1" standalone="yes"?><results><entity><title>kami-sama no memo-chuo</title><url>http://localhost:8080/Test/detail.jsp?did=1</url></entity></results>

And it seems that xbmc cannot parse that xml, why? Where is the error?

Here are the really simple scraper code:

Code:
<?xml version="1.0" encoding="UTF-8"?>
<scraper name="Anime Plugin" framework="1.1" date="2011-12-09" content="tvshows">
    <CreateSearchUrl clearbuffers="no" dest="4">
        <RegExp input="$$1" output="&lt;url&gt;http://localhost:8080/Test/search.jsp?title=\1&lt;/url&gt;" dest="4">
            <expression></expression>
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults clearbuffers="no" dest="4">
        <RegExp input="$$4" output="\1" dest="4">
            <expression></expression>
        </RegExp>
    </GetSearchResults>
    <GetDetails dest="3">
        <RegExp>
            <expression></expression>
        </RegExp>
    </GetDetails>
    <EpisodeGuideUrl dest="3">
        <RegExp>
            <expression></expression>
        </RegExp>
    </EpisodeGuideUrl>
</scraper>

Please help!


- Finalspace - 2011-12-10 10:58

Ok, i fixed a lot and now it works in xbmc 10.1.

But in XBMC unstable ppa, the scraper does not work Sad

Code:
<?xml version="1.0" encoding="utf-8"?>
<scraper framework="1.1" date="2011-12-10" name="MyScrapper" content="generic" language="en">
    <CreateSearchUrl dest="3">
        <RegExp input="$$1" output="&lt;url&gt;http://localhost:8080/Webservice/search.jsp?title=\1&lt;/url&gt;" dest="3">
            <expression noclean="1" />
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults dest="3">
        <RegExp input="$$1" output="\1" dest="3">
            <expression noclean="1" />
        </RegExp>
    </GetSearchResults>
    <GetDetails dest="3">
        <RegExp input="$$1" output="\1" dest="3">
            <expression noclean="1" />
        </RegExp>
    </GetDetails>
</scraper>



- mkortstiege - 2011-12-11 13:48

Make sure YourScrapper has a valid addon.xml