Kodi Community Forum
[RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted! - Printable Version

Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=60)
--- Thread: [RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted! (/showthread.php?tid=52929)

Pages: 1 2 3 4 5 6 7 8 9 10 11

- Nicezia - 2009-12-29

DuMbGuM Wrote:Just a quick question guys wondering if you can help me out, I was trying to use this tool to edit the tvdb scraper to scrape episode ratings from tv.com but have gotten completely lost trying to do it. Any idea if this had already been done or how I might do it? I'm not sure if it's a simple task or not.

I just want the same tvdb scraper but one which scrapes all episode ratings from tv.com as the ratings there are much more comprehensive.

Thanks, and nice work on this by the way!

I'm actually personally working on a rewrite of the tvdb scraper (since some of the way its written makes it incompatible with ScraperXML Editor)

My suggestion would to be to make an include file to get the tv.com ratings. You'd have to find a way to be able to find the exact tvshow and episode from the tv.com given the tvshow/episode and find the right page given the tvdb episode & tvshow title.

- DuMbGuM - 2009-12-30

Ah okay, good thinking, i'll give it a shot thanks for the help

- wikid - 2010-01-05

Hi, I just created these two threads, that maybe should help a little bit in regards to helping the development and debugging your scraper:






- flobbes - 2010-01-27

I cant get the editor running neither under windows nor linux.

Using Linux I at least get an error message, dont know if it helps you any further.


- Nicezia - 2010-01-28

it seems you did get the editor running since its gotten to the point of moving between treenodes (on the scraper tree), when does this error occur?

- flobbes - 2010-01-28

Nicezia Wrote:it seems you did get the editor running since its gotten to the point of moving between treenodes (on the scraper tree), when does this error occur?

I See the Editor and it immediatly crashes before I can do anything.
If I delete the config files, a dialog to choose the scraper dir appears.
After doing that it crashes.

Under Windows I get this error:


- Nicezia - 2010-01-28

delete your old config.xml in the app folder, the configuration format has changed

- flobbes - 2010-01-29

Im starting it from a clean install, dont have an old .cfg

Just wanted to mention when I delete the created config files, I at least see the "choose scraper dir" dialog, after doing that the program crashes.

Edit: Tried it with an empty dir under windows, that worked.
Wow the program is really looking nice!
Grats and thanks for your effort!

- Nicezia - 2010-01-29

then it should ask you for your scraper folder before it even starts did you at least get that far?

- Nicezia - 2010-01-29

oh i see it says you do get that far, are you choosing the base folder? (the structure must match XBMC's folder ./video and ./music with common folders as subfolders of these... or just choose the xbmc/system/scrapers folder)

- flobbes - 2010-01-29

At the moment im using an empty folder, that works fine.

But I have a problem, using the tester.

THe program crashes when enabling "buffer edit". And it crashes when I load a file from cache and start the testing.

Error enabling buffer edit:

- Nicezia - 2010-01-29

uploading 3.6.0 now should fix that problem

also this version fixes all scraper testing (tv show scraper whould work for both tvshow and tvepisode testing now)

- flobbes - 2010-01-30

Nice Buffer edit works now.

When testing my own scrapers, I always get:

Could not download webpage.
If you were running auto mode it will be halted

Error returned:
Ung├╝ltiger URI: Das URI-Schema ist ung├╝ltig

The last one translates to:

Invalid URI: The URI-syntax is invalid

Any idea what the problem could be?

- Nicezia - 2010-01-30

i need to see the url its trying to download in the way the scraper returns it in order to know why

and if you're trying to use gzip, that's still something i haven't added as of yet, since i don't truly know how to handle gzipped webpages, still learning when it comes to programming internet interaction.

- flobbes - 2010-01-30

Its a scraper for a german adult site, so not safe for work.

The CreateSearchUrl is:


The scraper (and my others are working fine) with xbmc, but the editor gives my that error message on all my scrapers.

The whole scraper can be found here:

This forum uses Lukasz Tkacz MyBB addons.