[MastaG]

Hi there,

I've built Kodi 19 from the master branch on F33.
In order to succesfully build against Python 3.9 I'm using this patch from RPM Fusion:
https://github.com/rpmfusion/kodi/blob/m...thon.patch

However when attempting to run it, it will briefly show the mainscreen.
But quickly segfaults with:

Stack trace of thread 10399:
                #0  0x00000000b6573b18 PyEval_ReleaseLock (libpython3.9.so.1.0 + 0x3db18)

Unfortunately my system (Odroid XU4) has limited RAM and therefore I cannot compile a debug build to get the full stack trace.
The last thing the crash log says is: CPythonInvoker(0, /home/odroid/.kodi/addons/service.xbmc.versioncheck/resources/lib/runner.py): script successfully run

So I guess the segfault occurs right after the versioncheck addon has been started.

I hope someone could take a look at the above patch and tell me a bit more.

[asavah]

I haven't built kodi with Python 3.9 so I can not confirm, but that patch looks incomplete.
Python 3.8.5 PR was merged recently. Try building Kodi without any half-assed third-party patches.

[wsnipex]

that's what happens when distros patch stuff, just to make it compile with their versions of libs.

[MastaG]

Thanks for pointing out the PR.
I've reverted the distro patch, because it now builds fine.
I guess this: https://github.com/xbmc/xbmc/commit/8b52...141032636f fixed it.

However it still segfaults after showing the main screen:
0  0x00000000b6521b18 PyEval_ReleaseLock (libpython3.9.so.1.0 + 0x3db18)

Right after:
INFO <general>: CPythonInvoker(0, /home/odroid/.kodi/addons/service.xbmc.versioncheck/resources/lib/runner.py): script successfully run


I guess if I can figure out how to disable LTO perhaps I can try to compile a debug build without running out of RAM.

[ronie]

i can confirm kodi crashes on Fedora 33 / Python 3.9rc1

crashlog: https://paste.kodi.tv/xubaqakuvi.kodi

[MastaG]

i can confirm kodi crashes on Fedora 33 / Python 3.9rc1

crashlog: https://paste.kodi.tv/xubaqakuvi.kodi

Thanks for getting us a complete stack trace bro!

This means it isn't caused by any 'half-assed' distro patches or arm32 bugs.

[Fuzzard]

https://bugs.python.org/issue41165

subsequent backport to 3.9 
https://github.com/python/cpython/pull/21345

PyEval_ReleaseLock is removed deprecated. Havent jumped in any further personally, but thats probably a starting point.

I'll be honest, i personally probably wont be diving into this at this stage. 3.9 is still RC, and i cant imagine we will be bumping python again during this release cycle.

[MastaG]

So it's deprecated, but they didn't change the actual function right?
So what causes the segfault then?

[Fuzzard]

As i said, a starting point. Look to replace the usage of it.

[Fuzzard]

https://github.com/ninia/jep/issues/229

Some potential discussion and alternate path than using pyeval_releaselock

[MastaG]

Yes, the segfault when calling pyeval_releaselock in Python 3.9 seems to happen for other projects as well, see: https://sourceware.org/pipermail/gdb-pat...69110.html
They seem to end up moving to PyEval_SaveThread.

So even though the function got deprecated since python 3.2, they also broke it in 3.9 I guess?

[asavah]

So even though the function got deprecated since python 3.2, they also broke it in 3.9 I guess?

Yep and that is big problem for current Kodi code.

Here exactly https://github.com/xbmc/xbmc/blob/master...#L560-L591 

Documentation for Py_EndInterpreter() explicitly states:

void Py_EndInterpreter(PyThreadState *tstate)
Destroy the (sub-)interpreter represented by the given thread state. The given thread state must be the current thread state. See the discussion of thread states below.
When the call returns, the current thread state is NULL.
All thread states associated with this interpreter are destroyed. (The global interpreter lock must be held before calling this function and is still held when it returns.) 
Py_FinalizeEx() will destroy all sub-interpreters that haven’t been explicitly destroyed at that point.

And thats is a problem - one just can't swap PyEval_ReleaseLock() for PyEval_SaveThread() in the code linked above because thread state will be NULL and Kodi will crash (runtime tested).
But we still need to release lock but we can't because PyEval_ReleaseLock() will currently crash with 3.9bX .

That's a bug that needs to be fixed by core python devs or another mechanism for releasing the lock must be found, but that's well above my paygrade.
We can just hope that they fix the crash before releasing 3.9 ...

[asavah]

@MastaG could you try this 3/4-assed patch that drops releaselock alltogether?
https://gist.githubusercontent.com/asava...Lock.patch
Works fine for me but on python 3.8.5

@Fuzzard could you please take a look if the patch looks sane to you?

[ronie]

i've tested it with python 3.9 and can confirm it works fine.
thx!

[asavah]

@ronie Thx for confirming.
I'm not sure how all this embedded python threading machinery works ... I can read and understand the code and the docs, but my actual c++ code-writing skills are close to 0. )
If this fixes 3.9 and does not break 3.7.x/3.8.x (so far at least on linux with 3.8 it works ok) I will PR it.