Kodi Community Forum

Full Version: [RELEASE] iPlayer - BBC Video and Music Plugin (UK only!)
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi, thanks for your work on fixing this. The HD streams are working great for me.
However, is there any chance of getting the none HD BBC1 streams added? The SD streams have the regional content, and the HD streams do not.
2.6.6 released

download: https://drive.google.com/file/d/0B_knGio...sp=sharing

changes:

2.6.6
* use SetResolvedUrl for playback - fixes playback from information window
and will allow easier calling from other plugins / web service etc
* various code cleanups / fixes
sorry, found a few more issues. Need to fix up live radio.
2.6.7 released

download: https://drive.google.com/file/d/0B_knGio...sp=sharing

changes:

2.6.7
* support for mediaselector v5 (and settings to fallback to 4)
* fixed live radio
* new elementtree based listparser to replace horrible regexp hackery
* fixed radio shows missing summary info
* include aired / episode number for episodes
* drop script.module.elementtree dependency and python 2.4 support
* various code cleanups / fixes
Many thanks for all the work, however version 2.6.7 is now a *lot* slower when entering the catchup channels than 2.6.6 - for instance with a 1GHz Pi it now takes over 30 seconds before the listing is displayed after clicking the BBC One icon.

The extra time is spent processing the data, with the CPU maxed out at 100% for all of this time.

This is the debug log when selecting BBC One catchup with 2.6.7:
Code:
04:55:40 14527.781250 T:3041038336   DEBUG: LIRC: Update - NEW at 313195:160 0 KEY_OK_UP devinput (KEY_OK_UP)
04:55:41 14528.676758 T:2748314688    INFO: plugin.video.iplayer: Version: 2.6.7
04:55:41 14528.676758 T:2748314688    INFO: plugin.video.iplayer: Subtitles dir: /storage/.kodi/userdata/addon_data/plugin.video.iplayer/Subtitles
04:55:41 14528.680664 T:2748314688    INFO: plugin.video.iplayer: (<iplayer2.feed object at 0xa1d7c650>, 'list', None, None, None, None, None, None, None, None, None, None, None, None)
04:55:41 14528.681641 T:2748314688    INFO: plugin.video.iplayer: Read RSS: http://www.bbc.co.uk/iplayer/ion/listview/masterbrand/bbc_one/service_type/tv/format/xml
04:55:41 14528.681641 T:2748314688    INFO: plugin.video.iplayer: Feed URL not in cache, requesting...
04:55:41 14528.872070 T:2748314688    INFO: plugin.video.iplayer: URL Fetch took 0.18 sec for http://www.bbc.co.uk/iplayer/ion/listview/masterbrand/bbc_one/service_type/tv/format/xml
04:56:03 14550.994141 T:3041038336   DEBUG: SECTION:UnloadDelayed(DLL: special://xbmcbin/system/ImageLib-arm.so)
04:56:03 14550.994141 T:3041038336   DEBUG: Unloading: /ImageLib-arm.so
04:56:09 14557.271484 T:2691691584   DEBUG: Thread JobWorker 2691691584 terminating (autodelete)
04:56:09 14557.272461 T:2758800448   DEBUG: Thread JobWorker 2758800448 terminating (autodelete)
04:56:09 14557.272461 T:2879386688   DEBUG: Thread JobWorker 2879386688 terminating (autodelete)
04:56:13 14560.515625 T:2748314688    INFO: plugin.video.iplayer: Found 337 entries
04:56:13 14560.833984 T:2862609472   DEBUG: WaitOnScriptResult- plugin returned successfully
04:56:13 14560.839844 T:2748314688    INFO: CPythonInvoker(12, /storage/.kodi/addons/plugin.video.iplayer/default.py): script successfully run
04:56:13 14560.862305 T:3041038336   DEBUG: Saving fileitems [plugin://plugin.video.iplayer/?feed_channel=bbc_one&listing=list]
04:56:13 14560.876953 T:3041038336   DEBUG:   -- items: 114, sort method: 0, ascending: false
04:56:13 14560.879883 T:3041038336    INFO: MYSQL server has gone. Will try 4 more attempt(s) to reconnect.
04:56:13 14561.008789 T:2879386688  NOTICE: Thread BackgroundLoader start, auto delete: false
04:56:14 14561.509766 T:2879386688   DEBUG: Thread BackgroundLoader 2879386688 terminating
04:56:14 14561.517578 T:3041038336   DEBUG: ------ Window Deinit (DialogBusy.xml) ------
04:56:14 14562.047852 T:2748314688    INFO: Python script stopped
04:56:14 14562.055664 T:2748314688   DEBUG: Thread LanguageInvoker 2748314688 terminating

It took 32 seconds to process 337 entries.

Navigating into pretty much any list, for example "Antiques Roadshow (5)" or "Alex Salmond - A Rebel's Journey (1)" both take another 30+ seconds each.

With 2.6.6, the same query/processing takes only 2 seconds - this is entering BBC One (catchup):
Code:
05:04:06 15033.764648 T:3040964608   DEBUG: LIRC: Update - NEW at 92607:160 0 KEY_OK_UP devinput (KEY_OK_UP)
05:04:07 15034.715820 T:2761294912    INFO: plugin.video.iplayer: Version: 2.6.6
05:04:07 15034.715820 T:2761294912    INFO: plugin.video.iplayer: Subtitles dir: /storage/.kodi/userdata/addon_data/plugin.video.iplayer/Subtitles
05:04:07 15034.719727 T:2761294912    INFO: plugin.video.iplayer: (<iplayer2.feed object at 0x4670a10>, 'list', None, None, None, None, None, None, None, None, None, None, None, None)
05:04:07 15034.720703 T:2761294912    INFO: plugin.video.iplayer: Read RSS: http://www.bbc.co.uk/iplayer/ion/listview/masterbrand/bbc_one/service_type/tv/format/xml
05:04:07 15034.720703 T:2761294912    INFO: plugin.video.iplayer: Feed URL not in cache, requesting...
05:04:07 15035.000000 T:2761294912    INFO: plugin.video.iplayer: URL Fetch took 0.27 sec for http://www.bbc.co.uk/iplayer/ion/listview/masterbrand/bbc_one/service_type/tv/format/xml
05:04:09 15036.833008 T:2761294912    INFO: plugin.video.iplayer: Found 337 entries
05:04:09 15037.173828 T:2705323072   DEBUG: WaitOnScriptResult- plugin returned successfully
05:04:09 15037.174805 T:2761294912    INFO: CPythonInvoker(10, /storage/.kodi/addons/plugin.video.iplayer/default.py): script successfully run
05:04:09 15037.216797 T:3040964608   DEBUG: Saving fileitems [plugin://plugin.video.iplayer/?feed_channel=bbc_one&listing=list]
05:04:09 15037.231445 T:3040964608   DEBUG:   -- items: 114, sort method: 0, ascending: false
05:04:10 15037.355469 T:2751460416  NOTICE: Thread BackgroundLoader start, auto delete: false
05:04:10 15037.735352 T:2761294912    INFO: Python script stopped

It appears that a very significant XML inefficiency has been introduced with this latest version.
Yep, however I am not going to parse XML with regular expressions just because it's quicker on a slow machine. It was a ghastly pile of crap that needed fixing.

There are a few problems here, partly because the thing parses the entire XML everytime (and a large xml at that), and partly because we should be using JSON rather than XML. I am working towards this.

in the meantime I think a fix would be to use cElementTree which should be quicker. I'll do a test.

Long term, we need a new stable fast api (or have some service running somewhere serving us up a decent api), and a lot of code refactoring so it doesn't need to keep parsing stuff everytime. Maybe some local mysqlite db.
I'm just about to profile the regex to see which expression is costing the most - will upload the results shortly.

If JSON is on the horizon then I'd agree that spending any more time on XML is a waste - anyone finding 30+ second delays unacceptable may be better off sticking with 2.6.6 for now.

What about caching the results, there's CommonCache - could that help?
additional caching could probably help, but first the code needs refactoring somewhat. I've been trying to pull it into shape a bit, as it's suffering from mess that has accumulated over the years. rewriting the parser to use ET was a good exercise as it now has a lot less lines of code and is readable, which will make it a lot easier when switching over to json.
Some very quick regex profiling analysis (see here for details) - this is just for entering the BBC One catchup channel:
Code:
Method           Freq     regex vs.   re     Avg +/- us  |                re (min/max/avg/total)            
c.compile         368         0 vs     368    1269.3625  |     63.8962 /  74696.0640 /  1269.3625 /  0.4671s
c.findall         674         0 vs     674     101.5341  |     56.9820 /   3571.9872 /   101.5341 /  0.0684s
c.match          1012         0 vs    1012      73.1910  |     39.1006 /   4981.0410 /    73.1910 /  0.0741s
c.search          337         0 vs     337      80.8911  |     30.9944 /   3283.0238 /    80.8911 /  0.0273s
d.sub               2         0 vs       2   14349.4606  |   4667.0437 /  24031.8775 / 14349.4606 /  0.0287s
============================================================================================================
TOTAL            2393         0 vs    2393       0.6656s |                                           0.6656s

ELAPSED TIME less re   :   34.9420s
ELAPSED TIME less regex:   35.6076s
ELAPSED TIME TOTAL     :   35.6076s
PERF LOGGING OVERHEAD  :    2.8955s (included in above elapsed times)

On the face of it, regex parsing doesn't seem to be the problem (in total, it accounts for only 0.6646 seconds of processing time) so the problem is elsewhere, presumably the XML parsing but maybe some other list processing.

For now I'll stick with 2.6.6, but always happy to test out any other version you may have. Smile
you misunderstood, I replaced regexp parsing with elementtree (I wasn't saying it was the fault of the regular expressions but the replacement code). the new code is slower and doesn't use regular expressions. The speed is due to the speed of elementtree. Blame python ;-) Unfortunately cElementTree which should be multiple times faster is being problematic, so I'm not using that currently.

Python is horribly slow on the pi anyway (pythons fault or just due to resources being used by xbmc ?). If you run a pi, you should also be patient - the original Xbox is far quicker for Python code.
(2014-11-22, 07:53)exobuzz Wrote: [ -> ]you misunderstood, I replaced regexp parsing with elementtree. the new code is slower and doesn't use regular expressions. The speed is due to the speed of elementtree.

Right. Oh well. Tip to add-on developers - avoid heavy use of Elementree. Wink

(2014-11-22, 07:53)exobuzz Wrote: [ -> ]Python is horribly slow on the pi anyway (pythons fault or just due to resources being used by xbmc ?). If you run a pi, you should also be patient - the original Xbox is far quicker for Python code.

Sure, I'm not under any illusion that the Pi is a speed demon, but when it took 2 seconds in 2.6.6 and now takes 32 seconds in 2.6.7, it's not really the fault of the hardware - ElementTree is clearly a less than ideal solution. Avoiding XML longer term is the best solution.

(By the way, I'm not moaning or ungrateful - many thanks for all your efforts. I'll stick with 2.6.6 for now as it seems to work best for my needs - catchup/live TV).
I am getting similar results to @Milhouse on my RPi (OpenELEC 4.2.1), however, the Search function has never been faster...
Approximately 2 seconds to search for "Antiques Roadshow" or "Alex Salmond" (I usually prefer to search, rather than browse).

@exobuzz, I don't know if this is the right forum for this request, but I find the "iPlayer A-Z" feature on that other guy's BBC iPlayer (0.0.7) extremely useful and very fast to locate a show by name (regardless of what channel or catagory it may appear in BBC's maze).
I guess I am making a feature request, but I mention it here, only because you appear to be doing some major re-tooling.

I appreciate all the work you (and all the other contributors) are doing on this plugin (as my only BBC fix, here in the colonies).

Thanks.
(2014-11-22, 08:00)mentat Wrote: [ -> ](as my only BBC fix, here in the colonies).

Thanks.

Colonies? You Canuck's don't even have the Union Jack in your flag LOL.
(2014-11-22, 08:14)nickr Wrote: [ -> ]
(2014-11-22, 08:00)mentat Wrote: [ -> ](as my only BBC fix, here in the colonies).

Thanks.

Colonies? You Canuck's don't even have the Union Jack in your flag LOL.

Still a loyal member of the Commonwealth (and Queen Elizabeth smiles at me from every $20 bill).
Yep and ours too!

She is getting an update next year. At least they aren't replacing her with Lorde like some idiot suggested...