Release - Initial release: Inter Process Communication package for Kodi

Release Initial release: Inter Process Communication package for Kodi - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=26)
+--- Thread: Release Initial release: Inter Process Communication package for Kodi (/showthread.php?tid=209171)

Initial release: Inter Process Communication package for Kodi - KenV99 - 2014-11-18

Hi,

One issue I have occasionally encountered is the desire to share data between addons or scripts. Think of plugins and background services or hotkey runscripts and background services.

Although there are a number of solutions for this, I decided to package Pyro4 for Kodi and then use it to create a datastore that addons can use to store and retrieve data from. It is in two parts: a script module and a separate service.

The script module is minimally changed from Pyro4 and is provided if any other ppl want to create their own remote objects directly.
The service implements a working datastore which can be used 'as-is' by addons to store and retrieve objects (anything pickleable).

Pyro4 is an all-python package for Python Remote Objects. It is authored by Irmen de Jong. Details about the original package can be found here: https://pythonhosted.org/Pyro4/index.html

Docs regarding my implementation can be found here: http://kenv99.github.io/service.ipcdatastore/
The download links are also in the docs.

It has been tested to work on a number of platforms. If you have any issues or questions, please post back.
If it is well-received, I plan to submit it for inclusion in the official repo.

I appreciate any and all feedback!

RE: Initial release: Inter Process Communication package for Kodi - Karnagious - 2014-11-18

Can it handle listitems?

RE: Initial release: Inter Process Communication package for Kodi - KenV99 - 2014-11-18

(2014-11-18, 01:29)Karnagious Wrote: Can it handle listitems?

I haven't tested it on a listitem itself, but if you try:

Code:
...

    import pickle

    test_obj = one_of_your_list_items

    x = pickle.dumps(test_obj)

And you don't get an exception, you should be good to go.

RE: Initial release: Inter Process Communication package for Kodi - Karnagious - 2014-11-18

That's the part that gives me an error.

Someone said something about it being because the ListItem is actually a C object, not a python object.

I haven't tried pickling just the __dict__, so maybe that would be something to try. I will be able to check out the code later.

Good work anyway. There are quite a few threads about passing data between processes.

RE: Initial release: Inter Process Communication package for Kodi - KenV99 - 2014-11-18

(2014-11-18, 03:15)Karnagious Wrote: That's the part that gives me an error.

Someone said something about it being because the ListItem is actually a C object, not a python object.

I haven't tried pickling just the __dict__, so maybe that would be something to try. I will be able to check out the code later.

Good work anyway. There are quite a few threads about passing data between processes.

I played around with it a bit this morning. Unfortunately, not only are ListItems not picklable, they do not have a __dict__.
The output of dir(xbmc.ListItem) is:

Code:
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'addContextMenuItems', 'addStreamInfo', 'getLabel', 'getLabel2', 'getProperty', 'getdescription', 'getduration', 'getfilename', 'isSelected', 'select', 'setArt', 'setIconImage', 'setInfo', 'setLabel', 'setLabel2', 'setMimeType', 'setPath', 'setProperty', 'setThumbnailImage']

I was hoping there might be a simple way to convert a ListItem into something pickleable, but without anyway of iterating over attributes and having several "setters without getters", I am at loss Sad

RE: Initial release: Inter Process Communication package for Kodi - ironic_monkey - 2014-11-18

somebody needs to implement __reduce__ on the objects on the XBMC side, and then you have to use cPickle instead of pickle.

RE: Initial release: Inter Process Communication package for Kodi - Karnagious - 2014-11-18

I did try that, but I didnt use cPickle. Will give that a go, thanks!

EDIT: Actually I think I did use cPickle. But I just added a __reduce__ method to my sub-class, which should have worked, right?

RE: Initial release: Inter Process Communication package for Kodi - KenV99 - 2014-11-19

With a little more tinkering, I found that I can pickle ListItem using the 'highest protocol', HOWEVER, it doesn't save any state information Sad

Code:
import xbmcgui

import cPickle as pickle

def test():

    y = xbmcgui.ListItem('test', 'test', path='c:\\temp')

    try:

        x = pickle.dumps(y, pickle.HIGHEST_PROTOCOL)

        z = pickle.loads(x)

        print "z: " + str(z)

        print "dir(z) :" + str(dir(z))

        print "y.getLabel(): " + y.getLabel()

        print "z.getLabel(): " + z.getLabel() # Check to see if state was saved

        z.setLabel('test2') # Check that obj behaves as expected

        print "z.getLabel() after set: " + z.getLabel()

    except Exception as e:

        print 'Exception: ' + e.message

if __name__ == '__main__':

    test()

Yields:

Code:
z: <xbmcgui.ListItem object at 0x1B9F3D70>

dir(z) :['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'addContextMenuItems', 'addStreamInfo', 'getLabel', 'getLabel2', 'getProperty', 'getdescription', 'getduration', 'getfilename', 'isSelected', 'select', 'setArt', 'setIconImage', 'setInfo', 'setLabel', 'setLabel2', 'setMimeType', 'setPath', 'setProperty', 'setThumbnailImage']

y.getLabel(): test

z.getLabel(): 

z.getLabel() after set: test2

So as ironic_monkey pointed out, perhaps someone that works on the kodi SWIG code could either put some code into __reduce_ex__ or
create __getstate__ and __setstate__ for us. I won't hold my breath Wink

RE: Initial release: Inter Process Communication package for Kodi - removed151214 - 2014-12-07

First of all, let me say what a great idea to implement something like this - inter process communication has long been a large limitation of Kodi, and it's great to see an attempt to solve this Smile

However, I hope you don't mind a question - assuming you've done the profiling on this (and if you haven't please do say, I'll do it myself - but if I can save time by using someone else's results, I will!) have you got any data on what the performance is like in communicating between processes?

I'm working hard on a new type of widget with a content-based recommendation system built in, and have implemented sockets within them to pass the request for the widget (and most importantly the Kodi 'handle' of the list the widgets are being returned to), so a script can get the request for the widget, but a service can actually return them.

The idea works surprisingly well, but there have been a few technical issues actually getting it working reliably (primarily due to my lack of experience in this area.) So, do you know/have you tested how quickly a variable can be passed from one process to another? Thanks. Smile

RE: Initial release: Inter Process Communication package for Kodi - KenV99 - 2014-12-07

I haven't run extensive benchmarks for latency and bandwidth.
My initial tests on W764 with a [email protected] showed a latency of under 10ms and a bandwidth of about 32Mb/s using a 256 byte random string being read repeatedly (with caching turned off, obviously) with the server and client on the same machine. This was using the datastore object - performance is dependent on the shared object's complexity, so if you share a simpler, more optimized object, you may have better performance.

There are more extensive performance tests available on the Pyro4 site, but it's down at the moment.

https://pythonhosted.org/Pyro4/intro.html#performance

Hope this helps...

RE: Initial release: Inter Process Communication package for Kodi - removed151214 - 2014-12-07

Helps a lot, many thanks. Will start moving my code over to use this Smile

RE: Initial release: Inter Process Communication package for Kodi - xnappo - 2014-12-19

Do you have a feel how fast this would be vs. using Window properties? We are sharing data using this ugly hack (written with replacing it with something better in mind):

https://github.com/MediaBrowser/MediaBrowser.Kodi/blob/master/resources/lib/Database.py

xnappo

RE: Initial release: Inter Process Communication package for Kodi - agb2 - 2016-11-29

I realise that this is an old thread, but - is this addon Krypton compatible?
I've downloaded from github and will try installing, just thought I'd ask :-}