2015-09-11, 08:53
(2015-09-07, 11:50)corona Wrote:(2015-09-07, 10:38)eirki Wrote:(2015-09-07, 03:01)corona Wrote: alternatively, I plan on integrating pythonjs into my soon-to-be-released selenium based webdriver plugin, such that it can be used to load pages in the background and run all the js to get all the visible content, ie let it handle the ajax calls.
Did you mean do write PhantomJS here? (I tried googling for "pythonjs" but the results didn't seem super relevant). It seems to me like Selenium is the way to go, with a PhantomJS webdriver to scrape content and a Chrome (maybe IE) webdriver to watch it.
I've been using a self-made PhantomJS based solution to scrape my Netflix "My List" into my Kodi Library for the past couple of months and I am very happy it. I'll be looking to polish it and make it publicly available if there is any interest.
Yeah that is what I meant, my webdriver plugin already leverages selenium to get better cross platform control of chrome, and I've even worked on being able to keep chrome in the background to use to get url data rather than using python requests library (to get around ssl issues etc).
There's serious difficulties keeping a persistent chrome&selenium around though, so initially the plugin's going to just start a new chrome each time to watch a show (like the current chrome launcher) but I could spin up a selenium&phantomjs to handle queries where I need js to be executed and/or ajax calls made to get a full page worth of information to display.
If I have more luck keeping a persistent selenium&chrome in the background though that'll be better than having to ship a binary of phantomjs with the plugin.
will this work with chromium under openelec too?