Why are ".pyo" files created when importing own modules in addons?
#1
Hello,

I thought it may be a good idea to group out some code into small modules. So I created a "resources/lib" directory, added it to "sys.path" and now import the small .py modules.

But for every import a new ".pyo" file is created every time Kodi starts.

I've disabled this using "sys.dont_write_bytecode", but why are they created? Why are they created every time? Is it recommended to use "sub-modules" at all and if so, what are the recommendations when doing so?
Reply
#2
why would you need to compile them each time over and over again? If nothing changes it's faster to already have the .py files compiled
Read/follow the forum rules.
For troubleshooting and bug reporting, read this first
Interested in seeing some YouTube videos about Kodi? Go here and subscribe
Reply
#3
You are right. They are, in fact, only created once.

But what would I do if this Addon is meant to be installed globally? Meaning that the user itself can't write the "pyo" files.

Is there some "build command" to create the files in advance or is it OK if the files just can't be written?
Reply
#4
This is how a Python interpreter works. Source code is compiled into bytecode which, in its turn, executed by the Python interpreter. Normally, bytecode for imported modules is saved for future use and the next time imports modules from bytecode files unless the source files have been changed. Only then the changed files are re-compiled.

From Kodi addon perspective, it's better to divide your code into logical parts and put those parts into separate modules, leaving in the main script only some minimal code needed to launch your addon. Such approach not only makes your code more maintainable but also speeds up addon start on slower systems, like Pi.
Reply
#5
Thanks for this information. I'll keep some basic code in the main file but try to separate logical parts into new modules.

But what about the "pyo" files if the Addon is installed system-global? Should I pre-create them to install them to the same system-global place or is it better to not do that? If this is recommeded: How to directly convert ".py" to ".pyo" in, for example, a Makefile?
Reply
#6
What do mean with "global"?
Read/follow the forum rules.
For troubleshooting and bug reporting, read this first
Interested in seeing some YouTube videos about Kodi? Go here and subscribe
Reply
#7
Normally, you don't have to deal with bytecode files because they are maintained automatically by a Python interpreter. If you don't know what they are for, then you can safely forget about them.
Reply
#8
(2017-01-21, 15:00)Martijn Wrote: What do mean with "global"?

/usr/share/kodi/addons/$MYADDONNAME

And: Yes there are good reasons for doing this as my Addon requires some code which has to be written in C++. I've created this as Python Addon so it can be directly loaded. But as a compile process is required, this also means that it would be a good idea if this specific Addon could be packaged by the distributor and installed globally.

Edit: See here for complete code:
https://github.com/M-Reimer/script.service.graphlcd

The C++ part interfaces to a library which is needed to talk to graphical LCD's.
Reply
#9
(2017-01-21, 15:15)M-Reimer Wrote:
(2017-01-21, 15:00)Martijn Wrote: What do mean with "global"?

/usr/share/kodi/addons/$MYADDONNAME

And: Yes there are good reasons for doing this as my Addon requires some code which has to be written in C++. I've created this as Python Addon so it can be directly loaded. But as a compile process is required, this also means that it would be a good idea if this specific Addon could be packaged by the distributor and installed globally.

Edit: See here for complete code:
https://github.com/M-Reimer/script.service.graphlcd

The C++ part interfaces to a library which is needed to talk to graphical LCD's.

Building binary extension modules has absolutely nothing to do with Python bytecode. If you addon needs to include some binary modules you must ship them pre-compiled for your target system. Normally, Python packages (non-Kodi) are built with distutils/setuptools build system, but for a Kodi addon you can use whatever build system you like.
Reply
#10
Every architecture may be possible so I keep this to the distributor. That's why I want to install to /usr/share/kodi/addons.

But I still don't know if I have to handle the pyo files in some way if the .py files are placed at a location not writeable by the user. Or is it not supported to install Python addons "system-wide" at all? At least for my Addon this seems to work well...
Reply
#11
hmm.... interesting question.

addons that ship with kodi (service.xbmc.versioncheck for instance) are installed globally.
bytecode for those modules can't be written to disk, as that would require root privileges.
Do not PM or e-mail Team-Kodi members directly asking for support.
Always read the Forum rules, Kodi online-manual, FAQ, Help and Search the forum before posting.
Reply
#12
(2017-01-21, 15:36)M-Reimer Wrote: Every architecture may be possible so I keep this to the distributor. That's why I want to install to /usr/share/kodi/addons.

But I still don't know if I have to handle the pyo files in some way if the .py files are placed at a location not writeable by the user. Or is it not supported to install Python addons "system-wide" at all? At least for my Addon this seems to work well...

There's no such thing as "installing addons system-wide". Technically it's possible but it's a hack. The only supported methods of installing addons are online repositories or ZIP files. Also, in Kodi 17 Krypton such "drop-in" installs no longer work. You need to enable an addon manually from Kodi after copying it to the addons directory.

As for compiling bytecode files for distribution on a read-only filesystem, this must be done by the very same Python interpreter that will run those bytecode files, because Python bytecode is not compatible even between minor Python versions. Usually, the Python standard library shipped with Kodi is compiled into bytecode for read-only filesystems during Kodi compilation.
Reply
#13
(2017-01-21, 15:48)ronie Wrote: hmm.... interesting question.

addons that ship with kodi (service.xbmc.versioncheck for instance) are installed globally.
bytecode for those modules can't be written to disk, as that would require root privileges.

Compilation to bytecode could be done in-memory, for example, if modules are imported from a .ZIP or an. EGG package.
Reply
#14
(2017-01-21, 15:55)Roman_V_M Wrote:
(2017-01-21, 15:48)ronie Wrote: hmm.... interesting question.

addons that ship with kodi (service.xbmc.versioncheck for instance) are installed globally.
bytecode for those modules can't be written to disk, as that would require root privileges.

Compilation to bytecode could be done in-memory, for example, if modules are imported from a .ZIP or an. EGG package.

i get that, but it has to be done each time you run the addon.
as you mentioned previously, this will lead to a slower start.

shouldn't kodi generate the bytecode for all included addons, either during compile or at install time?
Do not PM or e-mail Team-Kodi members directly asking for support.
Always read the Forum rules, Kodi online-manual, FAQ, Help and Search the forum before posting.
Reply
#15
(2017-01-21, 15:51)Roman_V_M Wrote: There's no such thing as "installing addons system-wide". Technically it's possible but it's a hack. The only supported methods of installing addons are online repositories or ZIP files. Also, in Kodi 17 Krypton such "drop-in" installs no longer work. You need to enable an addon manually from Kodi after copying it to the addons directory.

This is absolutely OK as long as I can do the "pre-install" using the system package manager.

In fact I would prefer to also have this in Jarvis. Currently my Addon gets installed, directly activated and the user isn't even able to disable it.

PVR-Addons are installed in a similar way and have to be enabled by the user first.

Can I get the same behaviour with my Python based Addon? Can I "pre-install" it and the user has it directly available in his Addon list, but has to enable it first? Or is this only possible with Kodi 17 Krypton?
Reply

Logout Mark Read Team Forum Stats Members Help
Why are ".pyo" files created when importing own modules in addons?0