Need help in writin a really simple scraper

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Finalspace Offline
Junior Member
Posts: 43
Joined: Jan 2011
Reputation: 0
Post: #1
Hello, i am trying to write a simple scraper which is simply a pass-through interface for my local webservice.
Every content which requires the scraper (SearchURL, SearchResults, Details, EpisodeGuideURL) will be returned from the webservice.

My current implementation returns the correct search url: "http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchou".

From wireshark i can see the HTTP GET is succeeded but XBMC UI displays "Unable to connect to remote server".

Here are the scraper parts from the xbmc log file:

Code:
22:29:55 T:140615347619712   DEBUG: ------ Window Init (DialogVideoScan.xml) ------
22:29:55 T:140615347619712    INFO: Loading skin file: DialogVideoScan.xml
22:29:55 T:140614672590592   DEBUG: Thread VIDEO::CVideoInfoScanner start, auto delete: 0
22:29:55 T:140614672590592  NOTICE: VideoInfoScanner: Starting scan ..
22:29:55 T:140614672590592   DEBUG: VideoInfoScanner: No NFO file found. Using title search for '/home/final/animedirs/Kami-sama no Memo-chou/'
22:29:55 T:140614672590592   DEBUG: FindMovie: Searching for 'Kami-sama no Memo-chou' using Anime Plugin scraper (path: '/home/final/.xbmc/addons/metadata.animeplugin', content: 'tvshows', version: '1.0.0')
22:29:55 T:140614672590592   DEBUG: scraper: CreateSearchUrl returned <url>http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchou</url>
22:29:55 T:140614672590592   DEBUG: FileCurl::Open(0x7fe3679c10e0) http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchou
22:29:55 T:140614672590592   DEBUG: scraper: GetSearchResults returned http://localhost:8080/Test/search.jsp?title=kami%2dsama%20no%20memo%2dchuo
22:29:55 T:140614672590592   ERROR: Run: Unable to parse XML

Here is returned XML Content from the search url:

Code:
<?xml version="1.0" encoding="iso-8859-1" standalone="yes"?><results><entity><title>kami-sama no memo-chuo</title><url>http://localhost:8080/Test/detail.jsp?did=1</url></entity></results>

And it seems that xbmc cannot parse that xml, why? Where is the error?

Here are the really simple scraper code:

Code:
<?xml version="1.0" encoding="UTF-8"?>
<scraper name="Anime Plugin" framework="1.1" date="2011-12-09" content="tvshows">
    <CreateSearchUrl clearbuffers="no" dest="4">
        <RegExp input="$$1" output="&lt;url&gt;http://localhost:8080/Test/search.jsp?title=\1&lt;/url&gt;" dest="4">
            <expression></expression>
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults clearbuffers="no" dest="4">
        <RegExp input="$$4" output="\1" dest="4">
            <expression></expression>
        </RegExp>
    </GetSearchResults>
    <GetDetails dest="3">
        <RegExp>
            <expression></expression>
        </RegExp>
    </GetDetails>
    <EpisodeGuideUrl dest="3">
        <RegExp>
            <expression></expression>
        </RegExp>
    </EpisodeGuideUrl>
</scraper>

Please help!
(This post was last modified: 2011-12-09 23:42 by Finalspace.)
find quote
Finalspace Offline
Junior Member
Posts: 43
Joined: Jan 2011
Reputation: 0
Post: #2
Ok, i fixed a lot and now it works in xbmc 10.1.

But in XBMC unstable ppa, the scraper does not work Sad

Code:
<?xml version="1.0" encoding="utf-8"?>
<scraper framework="1.1" date="2011-12-10" name="MyScrapper" content="generic" language="en">
    <CreateSearchUrl dest="3">
        <RegExp input="$$1" output="&lt;url&gt;http://localhost:8080/Webservice/search.jsp?title=\1&lt;/url&gt;" dest="3">
            <expression noclean="1" />
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults dest="3">
        <RegExp input="$$1" output="\1" dest="3">
            <expression noclean="1" />
        </RegExp>
    </GetSearchResults>
    <GetDetails dest="3">
        <RegExp input="$$1" output="\1" dest="3">
            <expression noclean="1" />
        </RegExp>
    </GetDetails>
</scraper>
(This post was last modified: 2011-12-10 11:00 by Finalspace.)
find quote
mkortstiege Offline
Team-Kodi Developer
Posts: 2,965
Joined: Jan 2008
Reputation: 8
Location: Germany
Post: #3
Make sure YourScrapper has a valid addon.xml

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules
For troubleshooting and bug reporting please make sure you read this first.
find quote