on-the-fly info scraping
#1
I've written an add-on that scrapes a website for the video links and displays the information in a nice format. the process is:
1) pull the available shows from "http://website.url/list"
2) when a show is selected, open "http://website.url/showname" and display available episodes
3) when an episode is selected, open "http://website.url/showname/episode" and display available sources
4) when a source is selected, play that source

What I'm wanting to do is at step 1 be able to display the MovieInformation window (when requested through the context menu) with a summary of information that's found at "http://website.url/showname". Here's what I've tried so far:
1) Filling the listItem.setInfo as the list is populated at step 1 above. This takes a (very) long time since the addon will browse out to each "/showname/" url and process the page for information.

2) Calling out to a separate script that is passed the show's url and then parses the information and then creates a window displaying that info. This seems to be the best method, but requires me to create my own window, which xbmc crashes randomly when that window is created or destroyed.

3) Calling out to a separate script and creating a listItem in that script, filling it with the information needed, and then calling ActivateWindow(MovieInformation). The window displays, but the information is not present.

In summary, here are my questions:
1) Can ActivateWindow(MovieInformation) be passed any arguments, such as a listItem, or the individual Info items that it requires?
2) Why is my window crashing XBMC?
3) Is there a better way to get MovieInformation on request, rather than populating a listItem as it's created?


The following code is called from a context menu entry created as such:
Code:
{'context': [('TEST', 'XBMC.RunScript(' + DIR_HOME + '/resources/lib/episode_info.py' + ',' + self + ',' + str(self._urljoin(show['href'])))}

Script that is called in attempt2 with a url, then builds the information, displaying it in it's own window (This is what crashes XBMC):
Code:
import xbmc, xbmcgui, xbmcaddon
import os, sys

from BeautifulSoup import BeautifulSoup as BS, BeautifulStoneSoup as BSS, SoupStrainer as SS
from xbmcvideoplugin import urlread

print 'AS --> ep_info.py --> testing :: entering with: ' + str(len(sys.argv))
print 'AS --> ep_info.py :: args: ' + str(sys.argv)
for i in range(0, len(sys.argv)):
    print str(type(sys.argv[i]))
#xbmc.executebuiltin('ActivateWindow(MovieInformation)')

url = sys.argv[2]

#import xbmc, xbmcgui

#get actioncodes from keymap.xml
ACTION_PREVIOUS_MENU = 10

class winEpisodeInfo(xbmcgui.Window):
  def __init__(self):
    page = BS(urlread(url), convertEntities=BSS.ALL_ENTITIES)
#    print 'AS --> episode_info.py :: page: ' + page.prettify()
    series_info = BS(str(page.find('dl')))

    series_title = series_info.find(text='Title:')
    while getattr(series_title, 'name', None) != 'dd':
        series_title = series_title.next

    series_type = series_info.find(text='Type:')
    while getattr(series_type, 'name', None) != 'dd':
        series_type = series_type.next

    series_genre = series_info.findAll('a')
    genre_temp = str(series_genre[0].renderContents())
    for i in range(1, len(series_genre)):
        genre_temp = genre_temp + ', ' + str(series_genre[i].renderContents())
    series_genre = genre_temp
    del genre_temp

    series_summary = page.find(text='Summary')
    while getattr(series_summary, 'name', None) != 'p':
        series_summary = series_summary.next

    series_image = page.find('img', id='poster')
    series_image = series_image['src']


    self.addControl(xbmcgui.ControlImage(150,100,0,0, str(series_image)))

    self.title = xbmcgui.ControlTextBox(500,100,300,30, textColor='0xFFFFFFFF')
    self.addControl(self.title)
    self.title.setText(str(series_title.renderContents()))

    self.media = xbmcgui.ControlTextBox(500,150,300,30, textColor='0xFFFFFFFF')
    self.addControl(self.media)
    self.media.setText(str(series_type.renderContents()))

    self.genre = xbmcgui.ControlTextBox(500,200,400,50, textColor='0xFFFFFFFF')
    self.addControl(self.genre)
    self.genre.setText(str(series_genre))

    self.summary = xbmcgui.ControlTextBox(500, 300, 600, 300, font='font13', textColor='0xFFFFFFFF')
    self.addControl(self.summary)
    self.summary.setText(str(series_summary.renderContents()))


#    self.strActionInfo = xbmcgui.ControlLabel(250, 80, 200, 200, '', 'font14', '0xFFBBBBFF')
#    self.addControl(self.strActionInfo)
#    self.strActionInfo.setLabel('Push BACK to quit')
#    self.list = xbmcgui.ControlList(200, 150, 300, 400)
#    self.addControl(self.list)
#    self.list.addItem('Item 1')
#    self.list.addItem('Item 2')
#    self.list.addItem('Item 3')
#    self.setFocus(self.list)

  def onAction(self, action):
    if action == ACTION_PREVIOUS_MENU:
      self.close()

mydisplay = winEpisodeInfo()
mydisplay.doModal()
del mydisplay
Reply
#2
Is this going to work on any website?
Reply
#3
No. I've written it specifically for one website.
Reply
#4
1. Nope.
2. No idea - debug log may give you some info?
3. Your only option would be grabbing the info in a background thread (or populating a database on first run or similar) and then telling xbmc to refresh the listing now that you have the new info.

I agree it would be nice if the plugin could be called to supply further info, but currently that is unsupported (being able to call the movie info window with a param would be nice as well, though I doubt that would give you what you want as any such interface would be unlikely to accept a listitem object). A patch would be welcome.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply

Logout Mark Read Team Forum Stats Members Help
on-the-fly info scraping0