lonewolf74 Wrote:Can anyone confirm whether this plugin works with the 1-25-09 SVN build of T3CH?
As a follow-up... I thought the T3CH Upgrader script was working, but it wasn't. My big clue was that I was still seeing the T3CH splash screen and not the XBMC splash screen. (Another clue: no option for Video Plugin under the Videos section of XBMC.)
I would recommend others who are having problems to try replacing their T3CH build via FTP (preferably from the UnleashX dash so no XBMC files are in use) and then trying the scripts again.
Doing this, and then trying the Crichton version of the Hulu plugin, worked fine for me.
a few questions for the devs....
what are your thoughts on keeping a local cache updated on a regular basis by cron or similar scheduler? Do the video paths remain constant or do they change on a per session basis (requiring that we make requests for them every time the script is run)? I haven't done any scripts/plugins for xbmc, but I have done some python so I may be able to lend a hand if there is interest in this type of thing.
quick edit -- primary reason for bringing this up, is that on my appletv, it takes considerably longer for it to get results back/processed than it does on my mac desktop. I haven't dug into where the time is being spent yet in the script. Maybe i should start there and see if I can find any way to speed it up.
--Nick
rwparris2:-
Does your Hulu Plugin work on Xbox, Linux, Windows ? I look through your fix and it is well structured. I want to do the same for the CBS Plugin. The current plugin for CBS doesn't work on Linux ..
CrashX Wrote:rwparris2:-
Does your Hulu Plugin work on Xbox, Linux, Windows ? I look through your fix and it is well structured. I want to do the same for the CBS Plugin. The current plugin for CBS doesn't work on Linux ..
yes, it I test on windows linux & xbox, haven't tried mac but from the quote below I assume it works there as well.
mr.b Wrote:a few questions for the devs....
what are your thoughts on keeping a local cache updated on a regular basis by cron or similar scheduler? Do the video paths remain constant or do they change on a per session basis (requiring that we make requests for them every time the script is run)? I haven't done any scripts/plugins for xbmc, but I have done some python so I may be able to lend a hand if there is interest in this type of thing.
quick edit -- primary reason for bringing this up, is that on my appletv, it takes considerably longer for it to get results back/processed than it does on my mac desktop. I haven't dug into where the time is being spent yet in the script. Maybe i should start there and see if I can find any way to speed it up.
--Nick
Turning off descriptions and plots for tv shows and movies should speed it up significantly. On my xbox it takes 30 secs to get all shows and their descriptions, but only maybe 2 secs if I leave off descriptions.
The thing that takes so long is getting show descriptions and plots, since they're kept on a seperate webpage. Basically you have to open two completely different pages for each show movie or episode, and that is where the bulk of the time is spent. Another bottleneck is the fact that I parse the webages using beautifulsoup. I'll up a new rev later using minimalsoup, which I
percieve to be faster than beautifulsoup but I haven't profiled it yet. If I were to use element tree it would be slightly faster, but not significantly. lxml would be significantly faster (I'd guess 7-10x faster) but I'm not sure it is feasable to distribute it since it is written in c. I welcome anyone to look into it for me. Again though, the vast majority of the time is spent opening all the necessary pages, and anything done to speed up parsing would only have a small effect.
However, it is possible to have the plugin keep all of the shows & movies and their plots in an xml file stored locally. Then, every time you enter a category, it quickly parses the webpage and your local xml, and only downloads the information you don't have yet. I actually started this last night, but I was tired and couldn't get it to work properly. It is feasible, and I'll continue to work on that when I have time.
Yep lxml is a speedy beast. The reason is it is written in c instead of pure python. This is also the reason it is hard to distribute in plugins... it isn't plug 'n' play like beautifulsoup.
Besides, when I profiled the scraping of the TV Shows page with get plots, BeautifulSoup only took up about 5% of that time, so any optimizations in scraping will be nearly negligible.
what about opening a socket and shoving multiple page requests in per connection? I forget the term, but it's something allowed for in modern HTTP standards. People doing mass scraping employ it and say it improves performance in a big way. I've found a reference implementation in php before.... I'll dig around and see if I can't find something in python.
Just looked it up -- its Http 1.1 pipelining
mr.b Wrote:what about opening a socket and shoving multiple page requests in per connection? I forget the term, but it's something allowed for in modern HTTP standards. People doing mass scraping employ it and say it improves performance in a big way. I've found a reference implementation in php before.... I'll dig around and see if I can't find something in python.
Just looked it up -- its Http 1.1 pipelining
Do you know of a (pure python, so I can actually use it) lib that supports pipelining?
I suppose threading the requests could work, but I don't know if it's worth it...
Feel free to work on this yourself, using the xbmc-addons to submit any patches you may have.
I don't think their will be any major benefit in using different parser for HTML. By the look of the Hulu Script, I see that we download and parse for each show .. Downloading from the web would contribute alot more then actual parsing .. I say we looking at 70% for downloading and 30% for parsing in terms of time.
What we need to look into caching information into a file and that I belive is what rwparris2 is working on ? What is the current status ?
CrashX Wrote:If the caching does help much, we than try using Threads. We use threads to do the downloading and the same time parse the data:-
http://www.wellho.net/solutions/python-p...ample.html
Thanks for the example.
Yes I have a version here that keeps everything in an xml file, but as of right now it's buggy as heck. I'll have time this weekend to finish this up & add searching & hopefully finish the logging in stuff.
BTW if anyone wants to help with the logging in & queuing stuffs, feel free. I don't use those features on Hulu so my motivation is pretty low
I appreciate all of you all spending time on this. Is there a link I can get the latest fruits of your labor? The latest and greatest plugin source that is.
Thanks much.