Kodi Community Forum

Full Version: ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work...
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Leading and trailing I should think, yes.
Well its up.

The good: Everything works for me tested all scrapers on it and it retrieves all info asked for.

The Bad: Haven't finished adding the logging as of yet. There's sure to be some bugs running on other systems

The Ugly: There are no programs up to demonstrate its usage, though I'm working on that and should be done by tomorrow this time. This program i'm working on can also double as a scraper tester (Pure console program) and won't neccessarily be using the object's i've made for GUI programs(which are mostly objects to save and restore settings at runtime)
Just a little note, Adultcdmovies doesn't exist anymore so you might want to remove it from the svn
Great work, looking forward to some kind of test app just to show how it works to normal users like in v1.0

Also do you have a list of the working XBMC scrapers? I've had a look at the Wiki and SVN but couldn't find anything.
EDIT: I guessed the names from the svn and posted a list here

If the XBMC scrapers do become the standard for other HTPC apps, would people want a site that collects them all in one place and shows if they are working/broken?
zag2me Wrote:Great work, looking forward to some kind of test app just to show how it works to normal users like in v1.0

Also do you have a list of the working XBMC scrapers? I've had a look at the Wiki and SVN but couldn't find anything.
EDIT: I guessed the names from the svn and posted a list here

If the XBMC scrapers do become the standard for other HTPC apps, would people want a site that collects them all in one place and shows if they are working/broken?


Currently uploading a few edits and a console test app that i've just finished up. It displays the function calls, as far as the actual gui usage of the settings, i have a tiny little bit of work to do on it at the moment, currently all scrapers will only run with default settings.
Ok there's an app up in the svn that demonstrates the use of the library,

The release of this app is up on Sourceforge download page, it can be used as a scraper tester. currently has a few problems with foreign language scrapers mainly the korean chinese and russian scrapers but not many
Tried the console app and it works great for movies and tv shows.

Couldn't get it to work with any of the music scrapers though. It crashes on any search.
hmm it works on mine,
did you get any error message?
Sure

Image

This is with the allmusic and lastfm scrapers.
zag2me Wrote:Tried the console app and it works great for movies and tv shows.

Couldn't get it to work with any of the music scrapers though. It crashes on any search.

There should be a log file, run the music scraper again and send me a copy of the log.
[email protected]
Code:
No Settings Found for conditional comparison RegExp will execute
No Settings Found for conditional comparison RegExp will execute
No Settings Found for conditional comparison RegExp will execute
No match found for term<TITLE>allmusic[ (]+ ([^>]*) > Overview
No Settings Found for conditional comparison RegExp will execute

Thats what shows in the log when it crashes. Searching for prodigy with allmusic.xml
Well i think i just found the problem

damned ampersands are still haunting me. I'm going to have to implement a nasty(but workable) hack to parse the xml

I'm going to make a function to change "&" to &amp; and then go back and change &amp;quot;, &amp;apos; &amp;gt; &amp;lt; &amp;amp; to their respective entities.

Done. And believe it or not that worked!
I need to write a utility to change encoding to utf8 hwever, so it'll be a while before my next update,
zag2me Wrote:
Code:
No Settings Found for conditional comparison RegExp will execute
No Settings Found for conditional comparison RegExp will execute
No Settings Found for conditional comparison RegExp will execute
No match found for term<TITLE>allmusic[ (]+ ([^>]*) > Overview
No Settings Found for conditional comparison RegExp will execute

Thats what shows in the log when it crashes. Searching for prodigy with allmusic.xml

can you send me a list of the scrapers that crash it?
Nicezia Wrote:can you send me a list of the scrapers that crash it?

The only 2 scrapers that crash are

lastfm.xml

and

allmusic.xml

All the others work as far as I can see.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22