Scraper for own site
#1
I would like to have the scraper for my own site (i.e. I can control what the site returns).

Is there any scraper template which I can use to simplify my work?

Since this is my own site, as far as I understand, I don't need to parse it - I can return results in the needed format (json, xml, etc.)
Reading the docs, looks like I can just return the info in the required format. For ex., the movie info will be in xml:
Code:
<details>
       <title></title>
       <year></year>
       <director></director>
       <top250></top250>
       <mpaa></mpaa>
       <tagline></tagline>
       <runtime></runtime>
       <thumb></thumb>
       <credits></credits>
       <rating></rating>
       <votes></votes>
       <genre></genre>
       <actor>
           <name></name>
           <role></role>
       </actor>
       <outline></outline>
       <plot></plot>
   </details>
Reply
#2
Hmm. Started to work on the own scraper and the first problem I faced with is XBMC logic to 'optimize' filename by cutting extension, DVDRip etc. So `$$1` at `CreateSearchUrl` doesn't contain original filename. How can I get it?

The next question - how to get the same filename at `GetSearchResults`?
OK, I can generate search URL with this filename, but can I get search URL at at `GetSearchResults`?
Reply
#3
Further investigation shows that CleanString function does such clean up (https://github.com/xbmc/xbmc/search?q=Cl...ef=cmdform). It could be controlled with advanced settings (http://wiki.xbmc.org/?title=Advancedsett...eanstrings). But doesn't look like very good (and user-friendly approach). So, still looking for a better solution
Reply
#4
Not sure how to log a feature request (to pass original filename in $$3), found many similar requests (https://www.google.ru/search?ie=UTF-8&hl...l+filename)
Reply
#5
Why do you need it?
Filename usually contain name of movie or music track.
XBMC does a good job on cleaning it from some junk.
Reply
#6
I have my own website with movies description and filenames. So, I know for sure where to take movie description for particular file based on its filename (the link will look like http://example.com/Some.file.name.2014.D...mebody.avi). I was thinking about search by what XBMC returns (http://example.com/some%20file%20name), but it will not work properly if user modified CleanString behavior in the advanced settings. And really I don't know how to do fuzzy search on the server side...

While looking for the solution, I found several similar requests from others...
Reply



Logout Mark Read Team Forum Stats Members Help
Scraper for own site1
This forum uses Lukasz Tkacz MyBB addons.