[jugendhacker]

Hello,

I have a Addon Source hosted on github an want to use a continous integration to test if PRs are working or not. Is there a way to do this or do I need to test every PR manually before merging it?

[Roman_V_M]

Addons are no different than any other Python programs so nothing prevents you from using unit/integration tests in a CI environment. Obviously, you will need to mock Kodi Python API calls.

[jugendhacker]

Is there a mock for the API where I can start from or do I need to start there from scratch?

[Roman_V_M]

There are Kodistubs: https://github.com/romanvm/Kodistubs They are meant for writing code but can be used in tests as well. E.g. you can pass a function/class from Kodistubs as spec argument for a mocked object: https://docs.python.org/3/library/unitte....mock.Mock

[vl_maksime]

You can look at my 'tests.py' in this addon: https://github.com/vlmaksime/plugin.video.ivi.client
I'm modified Roman's Kodistubs (https://github.com/vlmaksime/Kodistubs/tree/develop) to emulate KODI actions. For example, You can set and read addon settings.
If you set

python:
xbmc._set_log_level(1)

you'll see all operations at KODI api.

[jugendhacker]

You can look at my 'tests.py' in this addon: https://github.com/vlmaksime/plugin.video.ivi.client
I'm modified Roman's Kodistubs (https://github.com/vlmaksime/Kodistubs/tree/develop) to emulate KODI actions. For example, You can set and read addon settings.
If you set

python:
xbmc._set_log_level(1)

you'll see all operations at KODI api.

Oh ok that sounds nice, thank you

[Roman_V_M]

However, you need to understand that such tests that rely heavily on mocking some external API that is out of your control are not very reliable by themselves. They are only as good as your mocks and you may end up testing your mocks and not your primary code. So in my opinion in case of Kodi addons unit tests are good only for catching basic things such as syntax errors and typos, and not for any serious functional testing.

[jugendhacker]

The problem is I think that for good testing kodi needs a way to start itself from cli and do the unit tests inside of kodi...

[spiff]

which is the why of the discussion about mocking.

[jugendhacker]

No if kodi natively can run unittests it isn't needed to mock anything...

[spiff]

you can't fire up kodi, a media center, on a CI node. you have to mock.

[jugendhacker]

You could do this but kodi isn't made for because there is no cli

[spiff]

fine, if you want to argue like that, yes, if somebody rewrote shitloads of code in kodi, then you could do it.

[jugendhacker]

That's the problem with the plugin desgin, I can't test them...
So I end up testing my mocking or I could manually test all PRs...

[czukowski]

Total Kodi newbie here, and I was wondering the same thing. To keep the topic alive and on a constructive note, I think there should be a separation (as much as possible) of an actual functional add-on code and the code that uses C modules imported from the Kodi binary, perhaps in form of an adapter. Then writing testable code should become much more manageable. Does anybody know whether any existing add-on already employs such design?