2013-03-31, 18:34
Thanks for pushing this!
I think that a good first step would be to make heimdall available as a dependency in the repos, so any python addon can import and use it.
With this we can hopefully have plugins or scripts test it out and we will probably get tons of feedback on what works and what doesn't, or whats confusing and what isn't :)
I think you outlined a few confusing parts in the gsoc thread. Basically I've been unsure on lending from semantic web during the development, and it might be valid to switch to a simpler document representation instead (with simple property names which isn't globally unique as semantic web predicates are.).
As for using heimdall for movie, tv show and music scraping I think we have a bit further to go. But the first step will greatly help this though!
As I see it to get heimdall to build the pipeline for movies, tv shows and music we first need to get python scripts as a possible scraper, then by extension to step 1 we could at the very least try heimdall and see how it works.
This is a vital step I think, we must retain the ability to write a scraper in the old way. Regexp is an incredibly powerful tool for language processing and scraping and the old pipelines are battle tested and those guys have spent incredibly much work on perfecting that pipeline, which is something we can't throw away :)
Heimdall is, IMO, a way to build the pipelines dynamically and empower the scraper writer to write the scraping steps in any way he finds it best!
I think that a good first step would be to make heimdall available as a dependency in the repos, so any python addon can import and use it.
With this we can hopefully have plugins or scripts test it out and we will probably get tons of feedback on what works and what doesn't, or whats confusing and what isn't :)
I think you outlined a few confusing parts in the gsoc thread. Basically I've been unsure on lending from semantic web during the development, and it might be valid to switch to a simpler document representation instead (with simple property names which isn't globally unique as semantic web predicates are.).
As for using heimdall for movie, tv show and music scraping I think we have a bit further to go. But the first step will greatly help this though!
As I see it to get heimdall to build the pipeline for movies, tv shows and music we first need to get python scripts as a possible scraper, then by extension to step 1 we could at the very least try heimdall and see how it works.
(2013-03-30, 23:57)garbear Wrote: * Backwards compatibility might be retained if a heimdall module was written that loads xbmc scraper xmls
This is a vital step I think, we must retain the ability to write a scraper in the old way. Regexp is an incredibly powerful tool for language processing and scraping and the old pipelines are battle tested and those guys have spent incredibly much work on perfecting that pipeline, which is something we can't throw away :)
Heimdall is, IMO, a way to build the pipelines dynamically and empower the scraper writer to write the scraping steps in any way he finds it best!
