[Sam.Nazarko]

Hi

I wanted to ask some questions about add-on packaging in Kodi, particularly with the strict delineation between Kodi as a core application and the addons (binary) that ship with it.

With language files, we (OSMC) like to ship them in advance. I know that Kodi can dynamically download them on demand, but we would like to provide proper internationalisation without depending on an active network connection. I could not find a way of downloading these files easily from mirrors.kodi.tv. Can a symlink to languagename-latest.zip be provided? Or better yet, languagename-KODIVERSION.zip (where KODIVERSION is Isengard / Jarvis, etc?) At the moment I have this hacky way of pulling language files from a Kodi mirror that supports Apache based directory listing:

Code:

languages=$(wget ${base_url} -O- | grep resource.language. | sed -e 's/<a/\n<a/g' | sed -e 's/<a .*href=['"'"'"]//' -e 's/["'"'"'].*$//' -e '/^$/ d' | sed '/tr/d' | sed 's/resource.language.//' | tr -d /)
if [ $? != 0 ]; then echo "Can't get list of languages" && exit 1; fi
for language in ${languages}
do
echo "Downloading language file for ${language}"
language_file=$(wget ${base_url}/resource.language.${language} -O- | grep -o '<a .*href=.*>' | sed -e 's/<a/\n<a/g' | sed -e 's/<a .*href=['"'"'"]//' -e 's/["'"'"'].*$//' -e '/^$/ d' | tail -n 1)
if [ $? != 0 ]; then echo "Can't determine language file" && exit 1; fi
wget ${base_url}/resource.language.${language}/${language_file}
if [ $? != 0 ]; then echo "Couldn't download language file ${language_file}" && exit 1; fi
unzip ${language_file}
rm ${language_file}
done

mirrors.kodi.tv does not always correctly resolve, and does not always return mirrors that use the same directory listing format. As a result, we found a good mirror (Leaseweb) and pinned it at that. But we wish there was a simpler solution.. The reason for 'scraping', is that I don't wish to maintain a list of language versions and a list of all languages.

Another enquiry I have is about binary addons when Kodi is mid release cycle. For example, OSMC is now about to support Apple TV. There was an issue with audiodecoder.lame on i686. This was fixed upstream in Kodi's binary addons repository. I took to patching the addon reference in Isengard tree, but I quickly realised that although this was fixed, another commit, presumably for Jarvis was introduced. 'addon.xml' had been renamed to 'addon.xml.in'. audiodecoder.lame is not branched, so I had to fork it, and revert this commit: https://github.com/samnazarko/audioencod...e037997d75. So my question is: how should we handle these addons post-Kodi release? I see that *some* addons are branched to Isengard and master, and some are not. Are there plans to unify this? We can work on this downstream if we must -- is there a nice way to apply a patch post-download to the source? Not too familiar with the new cmake depends system.

My final question: why are Git URLs used in the addon's depends file? This means a git clone is used for each addon. This can fail under QEMU environments (pthreads...). I fix this with:

Code:

git_to_archive()
{
file_contents=$(cat $1)
if grep -q github.com $1
then
PKG_NAME=$(echo $file_contents | cut -f 1 -d " ")
GIT_REPO=$(echo $file_contents | cut -f 2 -d " ")
GIT_REV=$(echo $file_contents | cut -f 3 -d " ")
GIT_URL=$(echo ${GIT_REPO}/archive/${GIT_REV}.zip)
echo "${PKG_NAME} ${GIT_URL}" > $1
fi
}

And we call it for each addon. But I am more curious as to why a clone is used versus the downloading of a tarball. I am not sure of your cloning parameters (depth for example), but bandwidth can be saved with a ZIP. Would you accept a PR to use ZIPs instead? ZIPs via GitHub allow the same parameters you currently use -- namely, specifying a git revision.

Thanks,

Sam

[ironic_monkey]

binary addons are a moving target until they are finished. eventually binary addons will be like any other addons, updateable mid cycle, installable from repos and sane for downstreams but until then all bets are off as to how they are handled. you cannot put a restriction of stability on something under construction.

git urls aren't really necessary but some people actually develop those addons and then it's very convenient with the support for git, local directories etc. but anyways, again, moving target. in newer code, tarballs are actually used. not friggan zips but real tarballs.

[Sam.Nazarko]

binary addons are a moving target until they are finished.

Thanks for the reply.

Can you clarify the above? 'lame' should be considered finished, or at least updatable, because it is used in Isengard which is currently the stable version.

you cannot put a restriction of stability on something under construction

OK, but it seems that with addon API changes, like the migration of 'addon.xml' -> 'addon.xml.in', the addon repository should be branched. If this was done, then Isengard could have contained the fix and master could have been used to move forward with the API changes.

Sam

[ironic_monkey]

the encoders themself are finished as such, the build system macros they rely on were not.

yes, i do agree that ideally the addons should have branched to keep the isengard support intact (in fact they were in helix as well). that is the intended way to go about such changes. but somebody forgot about that part. because the whole thing is under heavy changes, it probably slipped their mind. that was all i tried to state.

[wsnipex]

re git URLs:
we actually convert them to tarballs by default, see
https://github.com/xbmc/xbmc/blob/master...#L280-L281
the downloaded tarballs are also cached. Adding an option to only download, but not build the addons would be rather easy.

[Sam.Nazarko]

Hi,

I did not know that. Thanks.

Sam

[Sam.Nazarko]

re git URLs:
we actually convert them to tarballs by default, see
https://github.com/xbmc/xbmc/blob/master...#L280-L281
the downloaded tarballs are also cached. Adding an option to only download, but not build the addons would be rather easy.

Seems this is not always the case, at least in Isengard

Code:

-- Processing /mnt/package/mediacenter-osmc/src/xbmc-02e7013889e08d608363f2909ebeccdb9ea3b7c9/project/cmake/addons/depends/common/kodi-platform/kodi-platform.txt
-- kodi-platform url: https://github.com/xbmc/kodi-platform
-- kodi-platform depends: kodi;tinyxml;platform
-- Could NOT find Git (missing:  GIT_EXECUTABLE)
CMake Error at /usr/share/cmake-3.0/Modules/ExternalProject.cmake:1314 (message):
  error: could not find git for clone of kodi-platform

Sam

[ironic_monkey]

if the addon points to a branch/tag and not a hash, git is used to resolve the hash, used for tarball caching purposes.

[Sam.Nazarko]

Thanks for clarifying

Sam