[eirki]

alternatively, I plan on integrating pythonjs into my soon-to-be-released selenium based webdriver plugin, such that it can be used to load pages in the background and run all the js to get all the visible content, ie let it handle the ajax calls.

Did you mean do write PhantomJS here? (I tried googling for "pythonjs" but the results didn't seem super relevant). It seems to me like Selenium is the way to go, with a PhantomJS webdriver to scrape content and a Chrome (maybe IE) webdriver to watch it.

I've been using a self-made PhantomJS based solution to scrape my Netflix "My List" into my Kodi Library for the past couple of months and I am very happy it. I'll be looking to polish it and make it publicly available if there is any interest.