[RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted!

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
seedzero Offline
Fan
Posts: 553
Joined: Mar 2009
Reputation: 0
Post: #31
spiff Wrote:the includes wont make it one bit easier for you to add anything. only difference is the function's code would be in a different .xml file

Not directly no, but I've been working with a text editor thus far. Also modifying the existing imdb scraper for ratings is not really a good idea as there would be two scrapers with nearly the same content. It's probably moreso the stepping functions that will help me as I haven't been able to glean much from the logs.
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #32
well my whole thing (the reason why i started making the editor) is because i could never keep track mentally of what was where, and visualizing what should be (and knowing whether what i am trying to put) in the buffers is actually there, was a priority for me.
That and stupid mistakes like not putting a semicolon on the end of a entity.

somehow all that mushroomed into me creating a complete external scraper library before i actually understood enough about the internal workings of the scraper code to attempt to be able to write the editor.

I'm not completely satisfied with the edtor yet, and i'm doing alot of work on it at the moment (since until they add the new features to scraper code i'm actually and god knows i never thought i'd see the day i'd say this - I'm caught up to the current XBMC code in svn.

As you can see i've added quite a few features in just the last couple of days.

I'm now trying to whittle the scraper editor down to a literally fool-proof construction tool. weeding out the possibility of malformed xml, conditional mispells, stray open tags, entity handling...

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #33
uhm, why don't you make a 'use rottentomato rating' setting and use that. having a completely new scraper is a waste
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #34
Spiff i have some time on my hands, going to go through the working scrapers that haven't been updated (for the thumbnail changes) and fix those... plus work on some include files. tmdb-fanart i've already made an include file for. (because i kinda got tired of waiting to test a official-made include file Wink)

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #35
cool. much appreciated Smile
find quote
seedzero Offline
Fan
Posts: 553
Joined: Mar 2009
Reputation: 0
Post: #36
spiff Wrote:uhm, why don't you make a 'use rottentomato rating' setting and use that. having a completely new scraper is a waste

I was going that route but I couldn't get my custom function to work properly.
I've managed to get most things working in a stand alone scraper, it's helped me get a better handle on how things work, it's just the custom function that's been causing grief.
I'm actually grabbing more than just the rating now, getting the synopsis, actors, etc. Is it worthwhile if I go that route or should I just do it as a setting like you've suggested?

Sorry for hijacking the thread a little, I don't wanna create a seperate thread until there's something substantial to show.
(This post was last modified: 2009-08-04 23:02 by seedzero.)
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #37
if its more than the rating, it makes sense to have it as a separate one
find quote
Daniel Malmgren Offline
Senior Member
Posts: 192
Joined: Jul 2009
Reputation: 0
Location: Sweden
Post: #38
When I open my filmdelta scraper (the version that's in svn) with the editor (version 2.1.153.16) it crashes with the following:

Code:
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object
  at ScraperXMLEditor.ScraperLib.ScraperFunction.Deserialize (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.ScraperFunction..ctor (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper.Deserialize () [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper..ctor (System.String xmlFilePath) [0x00000]
  at ScraperXMLEditor.Form1.OpenScraper () [0x00000]
  at ScraperXMLEditor.Form1.openToolStripButton_Click (System.Object sender, System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripButton.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.HandleClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.FireEvent (System.EventArgs e, ToolStripItemEventType met) [0x00000]
  at (wrapper remoting-invoke-with-check) System.Windows.Forms.ToolStripItem:FireEvent (System.EventArgs,System.Windows.Forms.ToolStripItemEventType)
  at System.Windows.Forms.ToolStrip.OnMouseUp (System.Windows.Forms.MouseEventArgs mea) [0x00000]
  at System.Windows.Forms.Control.WmLButtonUp (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ScrollableControl.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ToolStrip.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlWindowTarget.OnMessage (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlNativeWindow.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.NativeWindow.WndProc (IntPtr hWnd, Msg msg, IntPtr wParam, IntPtr lParam) [0x00000]
  at System.Windows.Forms.XplatUIX11.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.XplatUI.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.Application.RunLoop (Boolean Modal, System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.Form mainForm) [0x00000]
  at ScraperXMLEditor.Program.Main () [0x00000]

I can't find any errors within the xml. Is it my fault or the editor being buggy?

/Daniel
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #39
Daniel Malmgren Wrote:When I open my filmdelta scraper (the version that's in svn) with the editor (version 2.1.153.16) it crashes with the following:

Code:
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object
  at ScraperXMLEditor.ScraperLib.ScraperFunction.Deserialize (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.ScraperFunction..ctor (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper.Deserialize () [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper..ctor (System.String xmlFilePath) [0x00000]
  at ScraperXMLEditor.Form1.OpenScraper () [0x00000]
  at ScraperXMLEditor.Form1.openToolStripButton_Click (System.Object sender, System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripButton.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.HandleClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.FireEvent (System.EventArgs e, ToolStripItemEventType met) [0x00000]
  at (wrapper remoting-invoke-with-check) System.Windows.Forms.ToolStripItem:FireEvent (System.EventArgs,System.Windows.Forms.ToolStripItemEventType)
  at System.Windows.Forms.ToolStrip.OnMouseUp (System.Windows.Forms.MouseEventArgs mea) [0x00000]
  at System.Windows.Forms.Control.WmLButtonUp (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ScrollableControl.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ToolStrip.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlWindowTarget.OnMessage (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlNativeWindow.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.NativeWindow.WndProc (IntPtr hWnd, Msg msg, IntPtr wParam, IntPtr lParam) [0x00000]
  at System.Windows.Forms.XplatUIX11.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.XplatUI.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.Application.RunLoop (Boolean Modal, System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.Form mainForm) [0x00000]
  at ScraperXMLEditor.Program.Main () [0x00000]

I can't find any errors within the xml. Is it my fault or the editor being buggy?

/Daniel


would have to see the current state scraper to figure that out but probably something i haven't accounted for

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
find quote
Daniel Malmgren Offline
Senior Member
Posts: 192
Joined: Jul 2009
Reputation: 0
Location: Sweden
Post: #40
Nicezia Wrote:would have to see the current state scraper to figure that out but probably something i haven't accounted for

As I said, it's the version in svn. Haven't changed a byte Wink

/Daniel
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #41
hmmm, app error, will have it fixed in next version (there's not much error checking in the current ScraperXML Editor release, the version i'm working on corrects that...

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
find quote
Daniel Malmgren Offline
Senior Member
Posts: 192
Joined: Jul 2009
Reputation: 0
Location: Sweden
Post: #42
Nicezia Wrote:hmmm, app error, will have it fixed in next version (there's not much error checking in the current ScraperXML Editor release, the version i'm working on corrects that...

Thanks mate. Looking forward to it Nod

/Daniel
find quote
flobbes Offline
Senior Member
Posts: 131
Joined: Mar 2009
Reputation: 0
Post: #43
Which Wine and which mono version are you using, cant get in running under linux and have to use a virtual machine.

Code:
wine: Call from 0x7b8455b0 to unimplemented function gdiplus.dll.GdipCreateFontFamilyFromName, aborting
err:seh:raise_exception Unhandled exception code c00000fd flags 0 addr 0x68f5d3a4
find quote
Daniel Malmgren Offline
Senior Member
Posts: 192
Joined: Jul 2009
Reputation: 0
Location: Sweden
Post: #44
flobbes Wrote:Which Wine and which mono version are you using, cant get in running under linux and have to use a virtual machine.

Code:
wine: Call from 0x7b8455b0 to unimplemented function gdiplus.dll.GdipCreateFontFamilyFromName, aborting
err:seh:raise_exception Unhandled exception code c00000fd flags 0 addr 0x68f5d3a4

I don't use wine at all. Just "mono scraperxmled.exe" does the trick.

/Daniel
find quote
flobbes Offline
Senior Member
Posts: 131
Joined: Mar 2009
Reputation: 0
Post: #45
Great its working thanks, so I dont have to run my virtual machine with windows everytime anymore Smile
find quote
Post Reply