Developing an Amazon Movie Scraper
#7
jelockwood Wrote:Been there and read it already.

It may have enough information for people already expert in writing regex but for the majority of us it still does not help. What is really needed is some examples in the Wiki.

What I would suggest would be that for the CreateSearchUrl it should include an example Url and describe how it is created.

Likewise, for the GetSearchResults it should show a URL returned as a result of using the CreateSearchUrl and describe how the example regex extracts the needed information.

I am not expecting it to cover every eventuality or source, but it does not have any examples at all. This is why I found it much more helpful to look at an existing scraper (the FilmAffinity one) and compare the xml code with what appears in a web-browser doing the same thing.

Don't get me wrong I think the developers of XBMC have done a great job but like most/all open-source projects (especially Linux people) they assume all the users have the same level of programming expertise as themselves. I am not a full-time programmer but I have found bugs in some open-source projects and submitted successful fixes and I still find the documentation less than desirable.

Hmm, just had a thought, part of the difficulty is that the only way to 'test' a scraper is to use it within XBMC, and you don't get any detailed feedback, it either works or it does not. However I do have a utility to test regex against sample text, this might at least help me test the extracting portion.

Anyway, if any one else is interested, the original Amazon information I posted may assist in a group effort.
what utility is it that you're using to "test" the scraper?
Reply


Messages In This Thread
[No subject] - by blittan - 2008-07-26, 16:30
[No subject] - by jelockwood - 2008-07-28, 12:04
[No subject] - by jmarshall - 2008-07-28, 12:10
[No subject] - by spiff - 2008-07-28, 12:50
[No subject] - by jelockwood - 2008-07-28, 12:57
[No subject] - by flipped cracker - 2008-08-06, 02:12
[No subject] - by spiff - 2008-08-06, 11:39
[No subject] - by DonJ - 2008-08-06, 14:42
[No subject] - by jelockwood - 2008-08-07, 03:30
[No subject] - by ShortySco - 2008-08-07, 04:24
[No subject] - by spiff - 2008-08-07, 10:09
[No subject] - by jelockwood - 2008-08-11, 13:25
[No subject] - by spiff - 2008-08-11, 14:22
[No subject] - by spiff - 2008-08-17, 22:18
[No subject] - by jelockwood - 2008-08-18, 00:57
[No subject] - by Gaarv - 2008-08-18, 10:41
[No subject] - by spiff - 2008-08-18, 12:01
[No subject] - by C-Quel - 2008-08-19, 22:47
[No subject] - by jelockwood - 2008-08-21, 00:30
Good news! - by jelockwood - 2008-08-23, 15:19
[No subject] - by jelockwood - 2008-08-23, 21:19
[No subject] - by C-Quel - 2008-08-24, 16:27
[No subject] - by jelockwood - 2008-08-25, 19:43
[No subject] - by w00dst0ck - 2008-08-26, 09:32
[No subject] - by w00dst0ck - 2008-08-26, 13:17
[No subject] - by jelockwood - 2008-09-20, 04:44
[No subject] - by w00dst0ck - 2008-10-01, 17:11
[No subject] - by gyrene2083 - 2008-12-11, 04:40
[No subject] - by jelockwood - 2008-12-11, 21:12
[No subject] - by spiff - 2008-12-14, 16:34
Scraper broken? - by jelockwood - 2009-01-11, 07:10
[No subject] - by C-Quel - 2009-01-12, 00:55
[No subject] - by jelockwood - 2009-01-12, 04:38
[No subject] - by mkortstiege - 2009-01-12, 09:47
[No subject] - by ultrabrutal - 2009-01-12, 10:16
[No subject] - by spiff - 2009-01-12, 13:38
[No subject] - by nekrosoft13 - 2009-01-12, 23:27
[No subject] - by jelockwood - 2009-01-13, 00:20
[No subject] - by Gamester17 - 2009-01-13, 11:19
[No subject] - by jelockwood - 2009-01-13, 12:30
[No subject] - by Gamester17 - 2009-01-13, 13:27
[No subject] - by C-Quel - 2009-01-13, 20:07
[No subject] - by ultrabrutal - 2009-01-13, 20:42
[No subject] - by Clumsy - 2009-01-13, 23:21
[No subject] - by azido - 2009-01-14, 11:30
[No subject] - by ultrabrutal - 2009-01-14, 13:28
[No subject] - by Nuka1195 - 2009-01-14, 16:03
[No subject] - by ultrabrutal - 2009-01-14, 16:14
[No subject] - by azido - 2009-01-14, 17:24
[No subject] - by ultrabrutal - 2009-01-14, 17:29
[No subject] - by azido - 2009-01-14, 17:51
[No subject] - by ultrabrutal - 2009-01-14, 17:57
[No subject] - by XavHorneT - 2009-01-22, 12:47
[No subject] - by jelockwood - 2009-01-23, 16:38
[No subject] - by XavHorneT - 2009-01-24, 16:33
[No subject] - by joolz - 2009-01-24, 18:48
[No subject] - by joolz - 2009-02-09, 00:10
Logout Mark Read Team Forum Stats Members Help
Developing an Amazon Movie Scraper1