Kodi Community Forum

Full Version: Addons compiled from 18.2 source don't support DESTDIR and missing .so files
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
To start, this post is about addons that are part of the official 18.2 source download. No 3rd-party addons are being downloaded separately. I think this is the best forum for this since the addon forum seems to be about individual addons, not all of them.

I've been trying to get a working SlackBuild (a shell script that automates building software into an installable package on Slackware and ) for kodi v18 now that I'm running a new enough mesa to build it. kodi compiles without issue, but I found I was missing inputstream.adaptive, as apparently these aren't compiled automatically anymore. Once I figured that out, I was able to add the command to compile the addons to the script, but it seems it doesn't support DESTDIR. That makes it really difficult to use this to package as you're now writing to the main system, possibly overwriting what might already be there.

Is there a way I'm just missing to support DESTDIR or something similar? I want to run it from /usr/, but I want to create the package from /tmp/.

The second issue is once I got all that done, I fire up kodi, get all the addons installed for Netflix, log in, start browsing, and try and play a video, but I get nothing. Once I look into the logs, I find that it is complaining about a missing inputstream.adaptive.so.2.3.18. I check my package and it doesn't exist. I then search the source tree everything was compiled in and I found the .so files in tools/depends/target/binary-addons/native/inputstream.adaptive-prefix/src/inputstream.adaptive-build/ and was able to manually copy them into the package... but shouldn't this be automatic while building the addons? Looking through tools/depends/target/binary-addons/native/, I found many other addons that have .so files that aren't being installed when building the addons. Should I be manually copying all of these .so files into their various addon folders? The addons that contain .so files are:
Code:
audiodecoder.2sf-prefix
audiodecoder.asap-prefix
audiodecoder.dumb-prefix
audiodecoder.fluidsynth-prefix
audiodecoder.gme-prefix
audiodecoder.gsf-prefix
audiodecoder.modplug-prefix
audiodecoder.ncsf-prefix
audiodecoder.nosefart-prefix
audiodecoder.openmpt-prefix
audiodecoder.organya-prefix
audiodecoder.qsf-prefix
audiodecoder.sidplay-prefix
audiodecoder.snesapu-prefix
audiodecoder.ssf-prefix
audiodecoder.stsound-prefix
audiodecoder.timidity-prefix
audiodecoder.upse-prefix
audiodecoder.usf-prefix
audiodecoder.vgmstream-prefix
audiodecoder.wsr-prefix
audioencoder.flac-prefix
audioencoder.lame-prefix
audioencoder.vorbis-prefix
audioencoder.wav-prefix
game.libretro-prefix
imagedecoder.heif-prefix
imagedecoder.mpo-prefix
imagedecoder.raw-prefix
inputstream.adaptive-prefix
inputstream.rtmp-prefix
peripheral.joystick-prefix
peripheral.xarcade-prefix
pvr.argustv-prefix
pvr.demo-prefix
pvr.dvblink-prefix
pvr.dvbviewer-prefix
pvr.filmon-prefix
pvr.hdhomerun-prefix
pvr.hts-prefix
pvr.iptvsimple-prefix
pvr.mediaportal.tvserver-prefix
pvr.mythtv-prefix
pvr.nextpvr-prefix
pvr.njoy-prefix
pvr.octonet-prefix
pvr.pctv-prefix
pvr.sledovanitv.cz-prefix
pvr.stalker-prefix
pvr.teleboy-prefix
pvr.vbox-prefix
pvr.vdr.vnsi-prefix
pvr.vuplus-prefix
pvr.wmc-prefix
pvr.zattoo-prefix
screensaver.asteroids-prefix
screensaver.asterwave-prefix
screensaver.biogenesis-prefix
screensaver.cpblobs-prefix
screensaver.greynetic-prefix
screensaver.matrixtrails-prefix
screensaver.pingpong-prefix
screensaver.pyro-prefix
screensaver.stars-prefix
screensavers.rsxs-prefix
vfs.libarchive-prefix
vfs.nfs-prefix
vfs.rar-prefix
vfs.sacd-prefix
vfs.sftp-prefix
visualization.fishbmc-prefix
visualization.goom-prefix
visualization.projectm-prefix
visualization.shadertoy-prefix
visualization.spectrum-prefix
visualization.waveform-prefix

I'm open to any suggestions or guidance. It wouldn't surprise me if this information is already documented somewhere, but my extensive google-fu just didn't seem to be finding it, but it is possibly because of how common the search terms are.

Thanks for your time!

EDIT: In case it's needed, my cmake options and how I'm compiling kodi are:

Code:
mkdir -p build
cd build
  cmake \
    -DCMAKE_C_FLAGS:STRING="$SLKCFLAGS" \
    -DCMAKE_CXX_FLAGS:STRING="$SLKCFLAGS" \
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DENABLE_INTERNAL_FMT=ON \
    -DENABLE_INTERNAL_FSTRCMP=ON \
    -DCMAKE_BUILD_TYPE=Release ..
  make

  make install DESTDIR=$PKG
cd ..

and I'm compiling and preparing the addons via:

Code:
make -C tools/depends/target/binary-addons PREFIX=/usr
cp -r /usr/share/kodi/addons/* $PKG/usr/share/kodi/addons/.
cp tools/depends/target/binary-addons/native/inputstream.adaptive-prefix/src/inputstream.adaptive-build/inputstream.adaptive.so* $PKG/usr/share/kodi/addons/inputstream.adaptive/.
rm -r /usr/share/kodi/
it's possible that the way you're building binary addons does not support DESTDIR, since it wasn't designed to be used for "packaging".
The correct way to package binary addons is to build each one individually, without using the the tools/depends/target/binary-addons helper Makefile.

first make sure kodi has been built and installed, including header files and cmake helper scripts.
then:
git clone some/addons
mkdir build && cd build
cmake ..
make
make install DESTDIR=...
I figured that the code provided under the linux compiling doesn't support DESTDIR. It's unfortunate, but it is what it is. Hopefully this will be addressed at some point to make things easier for packagers.

But the bigger issue is even when I build the addons using the code from the README.Linux.md guide on github the process isn't properly building at least some of those binary addons.

With at least inputstream.adaptive, it leaves out the .so files from the final install, so the addon fails to function. This is only known once you try and run something using the addons and figure it out from the logs. You need to manually move those files into /usr/lib{64}/kodi/addons/inputstream.adaptive/ for the addon to even work. Shouldn't that be part of the build process? If these addons are "included" with the kodi source (I actually just realized that they aren't actually included and are downloaded according to cmake/addons/bootstrap/repositories/binary-addons.txt), if there are errors with the resulting addons is it something that is able to be adjusted or is it more of a one-type-fits-most build system?

There's 84 addons (if I counted right) that are linked in the repo-binary-addons github repo. Since it seems unreasonable to need to build all of them separately and it seems that not all are needed by most, is there a recommended list somewhere on what should be built on a standard install? Based on the code provided on the README.Linux.md, I assumed these were "stock" addons and should be included in most installs, but now I'm starting to think that is not the case.

Thanks for any insight you can provide. I guess I still have a lot more to learn about this new build system...
(2019-06-03, 08:05)bassmadrigal Wrote: [ -> ]I figured that the code provided under the linux compiling doesn't support DESTDIR. It's unfortunate, but it is what it is. Hopefully this will be addressed at some point to make things easier for packagers.

Packagers don't use our depends system, as I already said above.
(2019-06-03, 08:05)bassmadrigal Wrote: [ -> ]But the bigger issue is even when I build the addons using the code from the README.Linux.md guide on github the process isn't properly building at least some of those binary addons.

With at least inputstream.adaptive, it leaves out the .so files from the final install, so the addon fails to function. This is only known once you try and run something using the addons and figure it out from the logs. You need to manually move those files into /usr/lib{64}/kodi/addons/inputstream.adaptive/ for the addon to even work. Shouldn't that be part of the build process? If these addons are "included" with the kodi source (I actually just realized that they aren't actually included and are downloaded according to cmake/addons/bootstrap/repositories/binary-addons.txt), if there are errors with the resulting addons is it something that is able to be adjusted or is it more of a one-type-fits-most build system?
I cannot reproduce:
Code:
$ make -j8 ADDONS=inputstream.adaptive PREFIX=/tmp/kodi-test
$ ll /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so*
lrwxrwxrwx 1 wolfgang wolfgang 28 Jun 3 09:35 /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so -> inputstream.adaptive.so.19.0
lrwxrwxrwx 1 wolfgang wolfgang 30 Jun 3 09:35 /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.19.0 -> inputstream.adaptive.so.2.3.19
-rw-r--r-- 1 wolfgang wolfgang 2572760 Jun 3 09:35 /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.2.3.19
 
(2019-06-03, 08:05)bassmadrigal Wrote: [ -> ]There's 84 addons (if I counted right) that are linked in the repo-binary-addons github repo. Since it seems unreasonable to need to build all of them separately and it seems that not all are needed by most, is there a recommended list somewhere on what should be built on a standard install? Based on the code provided on the README.Linux.md, I assumed these were "stock" addons and should be included in most installs, but now I'm starting to think that is not the case.

Thanks for any insight you can provide. I guess I still have a lot more to learn about this new build system...
They are addons for a reason, so no, they are not included(besides on platforms where it's technically impossible to install via addon repo). You usually want at least inputstream.* and probably the most popular pvr addons.
(2019-06-03, 09:42)wsnipex Wrote: [ -> ]
(2019-06-03, 08:05)bassmadrigal Wrote: [ -> ]I figured that the code provided under the linux compiling doesn't support DESTDIR. It's unfortunate, but it is what it is. Hopefully this will be addressed at some point to make things easier for packagers.

Packagers don't use our depends system, as I already said above. 

That's unfortunate. I'm a packager and was hoping to use your depends system so I wouldn't have to package all the addons separately. It seems like if your system is already able to download and handle dependencies that it would be nice to build off of it rather than reinventing the wheel. I wonder if others aren't using it because of this limitation or if they'd rather provide addons individually?
 
(2019-06-03, 09:42)wsnipex Wrote: [ -> ]
(2019-06-03, 08:05)bassmadrigal Wrote: [ -> ]But the bigger issue is even when I build the addons using the code from the README.Linux.md guide on github the process isn't properly building at least some of those binary addons.

With at least inputstream.adaptive, it leaves out the .so files from the final install, so the addon fails to function. This is only known once you try and run something using the addons and figure it out from the logs. You need to manually move those files into /usr/lib{64}/kodi/addons/inputstream.adaptive/ for the addon to even work. Shouldn't that be part of the build process? If these addons are "included" with the kodi source (I actually just realized that they aren't actually included and are downloaded according to cmake/addons/bootstrap/repositories/binary-addons.txt), if there are errors with the resulting addons is it something that is able to be adjusted or is it more of a one-type-fits-most build system?
I cannot reproduce:
Code:
$ make -j8 ADDONS=inputstream.adaptive PREFIX=/tmp/kodi-test
$ ll /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so*
lrwxrwxrwx 1 wolfgang wolfgang 28 Jun 3 09:35 /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so -> inputstream.adaptive.so.19.0
lrwxrwxrwx 1 wolfgang wolfgang 30 Jun 3 09:35 /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.19.0 -> inputstream.adaptive.so.2.3.19
-rw-r--r-- 1 wolfgang wolfgang 2572760 Jun 3 09:35 /tmp/kodi-test/lib/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.2.3.19

I just tried this and confirmed this works fine. This lead me to realize that it seems the libraries are being built and installed, but they were installed to the wrong location. On my build system, everything (including kodi) is under /usr/lib64/, but all libraries for addons were installed to /usr/lib/. The library locations for the addons seem to not be hardcoded for an absolute path to this location as they don't function when kodi is built against /usr/lib64/ and the addon libraries end up in /usr/lib/, but moving everything to /usr/lib64/ seems to fix the problem.

Is there anyway to change this? My distro has all 32bit packages use /usr/lib/ and all 64bit packages use /usr/lib64/. I futilely tried passing LIBDIR to make, but I realize the only way to change that is probably a cmake option that is passed by kodi's addon build system. I've tried grepping through the source files, but there are so many potential places that might be where to adjust it and the few I've tried had no effect. Any suggestions on where I could start?
 
(2019-06-03, 09:42)wsnipex Wrote: [ -> ]
(2019-06-03, 08:05)bassmadrigal Wrote: [ -> ]There's 84 addons (if I counted right) that are linked in the repo-binary-addons github repo. Since it seems unreasonable to need to build all of them separately and it seems that not all are needed by most, is there a recommended list somewhere on what should be built on a standard install? Based on the code provided on the README.Linux.md, I assumed these were "stock" addons and should be included in most installs, but now I'm starting to think that is not the case.

Thanks for any insight you can provide. I guess I still have a lot more to learn about this new build system...
They are addons for a reason, so no, they are not included(besides on platforms where it's technically impossible to install via addon repo). You usually want at least inputstream.* and probably the most popular pvr addons.  

 Maybe this portion could be updated in the wiki? I know this new build system is quite a bit different than before and I was under the impression all addons should be built based on the text of the wiki.

Thanks for your time so far and for the (possibly inadvertent) help in learning more about kodi's new addon build system.
Then you are the first packager ever to use the depends system. The reason why it's usually frowned upon by distros is that we download build and statically link all libraries needed by the addons.
This violates the packaging policy of all distros I know (and I maintain our ubuntu packages as well), because they require the use of distro provided, dynamic libraries.


Because of the above nobody ever bothered to support advanced install options like --libdir, --datadir etc in that wrapper Makefile.
It's non-trivial to do since you have to pass those options through multiple layers of Makefiles/autotools/cmake
You can however build tar or zipballs of addons, or you just can copy the compiled libs to the correct place for your distro. As long as they end up where kodi expects them($LIBDIR/kodi/addons/), it will work.
It would require quite a bit of work to follow my distro's requirements for packaging (but statically linking things is not specifically mentioned as being frowned upon), but to me, it still seemed easier than hunting down all the dependencies themselves and building individual packages for them. But I under understand the team's position behind not supporting packaging options with it.

However, it does seem like not supporting at least --libdir could cause potential problems for people who aren't even trying to package it, but are still trying to follow their distro's file convention (in my case, using /usr/lib64/ on 64bit systems and /usr/lib/ on 32bit systems). But maybe these use cases are infrequent enough that it just isn't worth doing. I hope that's not the case, but being a Slackware user, we do things differently than a lot of distros (like being systemd free), so I'm use to my use cases not always being considered/implemented by developers.

Thanks for your time and effort in explaining everything!