Kodi Community Forum
Scraper for own site - Printable Version

Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=60)
--- Thread: Scraper for own site (/showthread.php?tid=189005)

Scraper for own site - and7ey - 2014-03-13

I would like to have the scraper for my own site (i.e. I can control what the site returns).

Is there any scraper template which I can use to simplify my work?

Since this is my own site, as far as I understand, I don't need to parse it - I can return results in the needed format (json, xml, etc.)
Reading the docs, looks like I can just return the info in the required format. For ex., the movie info will be in xml:

RE: Scraper for own site - and7ey - 2014-03-13

Hmm. Started to work on the own scraper and the first problem I faced with is XBMC logic to 'optimize' filename by cutting extension, DVDRip etc. So `$$1` at `CreateSearchUrl` doesn't contain original filename. How can I get it?

The next question - how to get the same filename at `GetSearchResults`?
OK, I can generate search URL with this filename, but can I get search URL at at `GetSearchResults`?

RE: Scraper for own site - and7ey - 2014-03-13

Further investigation shows that CleanString function does such clean up (https://github.com/xbmc/xbmc/search?q=CleanString&ref=cmdform). It could be controlled with advanced settings (http://wiki.xbmc.org/?title=Advancedsettings.xml#cleanstrings). But doesn't look like very good (and user-friendly approach). So, still looking for a better solution

RE: Scraper for own site - and7ey - 2014-03-14

Not sure how to log a feature request (to pass original filename in $$3), found many similar requests (https://www.google.ru/search?ie=UTF-8&hl=ru&q=xbmc%20scraper%20filename#hl=ru&newwindow=1&q=CreateSearchUrl+filename)

RE: Scraper for own site - Karlson2k - 2014-03-14

Why do you need it?
Filename usually contain name of movie or music track.
XBMC does a good job on cleaning it from some junk.

RE: Scraper for own site - and7ey - 2014-03-14

I have my own website with movies description and filenames. So, I know for sure where to take movie description for particular file based on its filename (the link will look like http://example.com/Some.file.name.2014.DVDRip.By.Somebody.avi). I was thinking about search by what XBMC returns (http://example.com/some%20file%20name), but it will not work properly if user modified CleanString behavior in the advanced settings. And really I don't know how to do fuzzy search on the server side...

While looking for the solution, I found several similar requests from others...

This forum uses Lukasz Tkacz MyBB addons.