+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=26)
+--- Thread: "import" python modules aka library addons (/showthread.php?tid=251555)
"import" python modules aka library addons - eldon - 2015-12-10
hi,
i would like to know if there's a proper way to import/add python libraries to a kodi (15) addon ?
when i say python libraries i do mean regular python libraries, unrelated to kodi.
I'm running a python daemon (deluged) on Openelec6 (raspi and other arm devices) alongside kodi.
Kodi can still be used to interact with the daemon mostly to modify its basic (first run) configuration, but nothing else, maybe i'll also make it start / stop the daemon through systemd calls.
That python program requires many libraries (python and binaries) that are not shipped with kodi (or openelec) and i'm simply trying to include them in my addon.
i'll describe what i'm doing on my Openelec6 setup (i understand it should be platform independent but i'll address that later).
I also have to say that i have no real knowledge of git repositories nor kodi repositories which to my surprise do have some library addons but i have not yet found where they came from or how they were build.
So to add package libraries i have chrooted into an arch distro on the device, and installed the missing packages inside the chroot.
The python version is different from the host one but not significantly (2.7.3 vs 2.7.10), and at the moment the python program does run on the host, but the way i've added those python libraries to the host is very messy.
Indeed i simply copied the python module files (looking at the package files list in the chroot) inside my addonname/lib/ directory. It does work for some of the modules but for others, in sub directories it didn't.
i've then found a couple of existing script.module... addons (zope interface) from kodi repositories and noticed that the __init__.py are very different from the ones from the chroot packages, so i'm gessing it's the __init__ fault that some modules are not recognized by python on the host.
i did read about those __init__.py in the wiki but no real tutorial describing how to write them, i've started to look at python docs and well i'm getting lost in there..
I have a second question, once i've found how to add python libraries properly, i wanted to know if it was preferred to split every library into a different script.module... addon or if it made no difference putting everything inside my addon lib dir ?
thx for your input
RE: "import" python modules aka library addons - Roman_V_M - 2015-12-10
It's totally up to you how your organize your library modules. You can include them into your addon or, if the libraries re-used among several addons, wrap them into script.module.* addons. I also recommend to read about module and package imports in Python 2 (In Python 3 things are slightly different, so you need to focus on Python 2).
This is if your library modules consist only of pure Python scripts. If you need to use binary modules, you need to supply a statically compiled binary module for every platform you are going to support. Add to this the fact that on Android using binary modules is a complicated matter because of broken import mechanism for binary modules in Kodi-Python for Android and permission issues on some Android devices. So I strongly advise against using binary modules in your addons unless there are no other options.
RE: "import" python modules aka library addons - tadly - 2015-12-10
It is strongly recommended to use modules from the kodi repository. For a list of available modules head over to http://kodi.wiki/view/Category:Add-on_libraries/modules
As told by Roman_V_M, if there are modules missing, you should create a script.module.* for it and submit them alongside your add-on to the official repository.
This will benefit everyone
As example on how to create a module just look at the structure of existing ones.
e.g. https://github.com/xbmc/repo-scripts/tree/gotham/script.module.httplib2
RE: "import" python modules aka library addons - eldon - 2015-12-11
Thanks for your answers, i'm still learning about addons as well as trying to check which libraries i actually need.
I was also wondering how or if kodi checks which libraries are already available on the system (not as script.module addons) and then decides if the current addon being installed has to get the dependencies or not (see my last comment below).
deluged will require some binaries as well unfortunately (libtorrent and geoip), so yes i understand i would have to managed different archs on the repository. And maybe it's not a nice thing to say but Android was never on my mind for this project.
I will definitely build script.module.library addons for each python library as it is a complete mess at the moment.
It will be a bit of a pain as some python libs actually depend on other python libs but well, it's how it is and as long as it works, i simply hope that the system PATH and LD_LIBRARY_PATH has no size limit because at some point i'm pretty sure i'd break it :)
I do understand the kodi addon structure and dependencies system, it's not very complicated and Openelec having no package manager makes it mandatory to put your resources in addons or at least "pseudo" addons for the time being.
I did check existing library addons like zope interface which was not working with my simple copy/paste and it is apparently only a matter of respecting the directory structure and proper __init__.py presence.
i'm reading that python document to get a better understanding of modules and import process :
https://docs.python.org/2/tutorial/modules.html
it's quite insightful but did not really answer some of my questions, for example :
Quote:The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.
That's one thing i was wondering about, since we're actually adding some libraries to the system, at least on openelec which builds system PATH / LD_LIBRARY_PATH to include all addons/lib or bin directories, i was expecting overlaps, various paths having potentially the same python library, and i'm not quite sure how python handles that, maybe simply registers the first one it finds and ignores the other one ?
I'm starting to wonder if that setup (having everything in addons libs) would only work on openelec and not other linux systems.
Thx again for your help, i'll probably come back asking about repositories at a later time, or if something goes horribly wrong with all those addons libraries :)
RE: "import" python modules aka library addons - Roman_V_M - 2015-12-11
2 eldon
Quote:I was also wondering how or if kodi checks which libraries are already available on the system (not as script.module addons) and then decides if the current addon being installed has to get the dependencies or not (see my last comment below).
From the point of view of addons, Kodi does not check anything. A Python runtime sees all modules (by 'modules' I mean packages too) available in its standard library plus all modules located in directories listed in sys.path list. In Kodi's built-in Python (or Kodi-Python) several additional xbmc* modules are added to Python runtime namespace in addition to the standard library. Also an addon additionally sees all modules listed in its addon.xml as dependencies plus all modules in directories explicitly added to sys.path in your code.
The above is true on platforms where Kodi has a built-in Python runtime, i.e. almost all supported platforms, except for desktop Linuxes. Kodi for desktop Linuxes uses the system-wide Python 2 so Python on those systems also sees all modules installed in system-wide Python 2's site-packages directory.
Quote:That's one thing i was wondering about, since we're actually adding some libraries to the system, at least on openelec which builds system PATH / LD_LIBRARY_PATH to include all addons/lib or bin directories, i was expecting overlaps, various paths having potentially the same python library, and i'm not quite sure how python handles that, maybe simply registers the first one it finds and ignores the other one ?
I guess you are confusing Python modules (both scripted and binary) and binary shared libraries. They are different things (although on *nix systems Python binary modules have the same .so extension). Although, a binary Python module may rely on an external binary shared library, but it's not a good practice as it complicates things even more. Ideally, binary Python modules should be built with static linking.
RE: "import" python modules aka library addons - tjost - 2018-02-17
Hello, I do know this is old. but I want to implement the Python-numpy Module.
Do have someone an idea on how to?
RE: "import" python modules aka library addons - Roman_V_M - 2018-02-17
(2018-02-17, 00:46)tjost Wrote: Hello, I do know this is old. but I want to implement the Python-numpy Module.
Do have someone an idea on how to?
Get a compiled wheel for your platform from PyPI, unpack it with any ZIP unpacker and use in your addon as described above.
RE: "import" python modules aka library addons - tjost - 2018-02-17
I'm very sorry but it is still confusing.
I use a raspberry pi 3 so I have to download this "numpy-1.14.0.zip (md5, pgp)"?
and do I need to install it via setup.py or just create a add-on an copy the folders to my_addon/lib?
i´m a beginner. so sorry.
RE: "import" python modules aka library addons - Roman_V_M - 2018-02-17
(2018-02-17, 12:02)tjost Wrote: I use a raspberry pi 3 so I have to download this "numpy-1.14.0.zip (md5, pgp)"?
No. This is the source code. Nympy is a binary library and you need to either compile it for your target platform or obtain a pre-compilded wheel. There's piwheels project that hosts pre-compiled wheels for different Python libraries, including Numpy: https://www.piwheels.hostedpi.com/simple/numpy/
(2018-02-17, 12:02)tjost Wrote: create a add-on an copy the folders to my_addon/lib?
This.
RE: "import" python modules aka library addons - yannbrrd - 2019-09-12
(2018-02-17, 20:53)Roman_V_M Wrote:Hi,(2018-02-17, 12:02)tjost Wrote: I use a raspberry pi 3 so I have to download this "numpy-1.14.0.zip (md5, pgp)"?
No. This is the source code. Nympy is a binary library and you need to either compile it for your target platform or obtain a pre-compilded wheel. There's piwheels project that hosts pre-compiled wheels for different Python libraries, including Numpy: https://www.piwheels.hostedpi.com/simple/numpy/
(2018-02-17, 12:02)tjost Wrote: create a add-on an copy the folders to my_addon/lib?
This.
I'm trying to create gTTS module. How can I test it in my add-on before submitting ?
https://github.com/YannBrrd/repo-scripts/tree/gtts/script.module.gtts