[TheJezstarr]

I thought I had, but looks like there was a rogue one left behind.

A quick test (I have to be up in five hours for work) looks as though this has fixed the issue.

[kereltje]

Excellent!
Please post a new debug log when you have a bit more time, so we can check if you've actually had the error and it's now properly handled; ITVX is a little inconsistent in what it returns when lists are empty.
Don't forget to set 'log level' to debug in viwX's settings.

[TheJezstarr]

Still getting occasional crashes, but they seem to be unrelated to this. Here's a log where it seems to be working:
kodi.log

[kereltje]

Can't tell from this log - debug logging was not enabled in Kodi's settings, so vital info is missing.
Also ensure in viwx's settings log level is set to debug.

[TheJezstarr]

Oops, sorry. Debug was enabled in viwx, but not kodi, I've done it again this morning, but the whole log is too big for the pastebin, so I'm looking to put it somewhere else.

I think the relevant part is here, but I'll come back with full log anyway.

[kereltje]

What you got here is ITVX returning HTTP status 404, which is another way of telling there are no items. The content of this response is not parsed as JSON, so it doesn't prove that our workaround is actually working.

What we need is ITVX sending HTTP status 200 with the textual content 'no continue watching items', which will cause the JSONDecodeError exception when parsed as JSON.
In the first log you posted ITVX also returned HTTP status 404 to the first request, but returned status 200 with a text in subsequent requests. I've seen this kind of rather erratic behaviour with other requests as well, so it's no real surprise.

Maybe it is already somewhere further down in the new log, or you can try to force these requests by opening viwX and navigate to 'My ITVX' -> 'Continue Watching'.
If ITVX return status 404, you'll get a notification in Kodi with 'Not Found' in the text.
If the JSON error occurs and it's now properly handled, you should get a notification with a text like 'Received invalid data'.
Just try a few times until you hit the JSON error.

[TheJezstarr]

Just my luck, I'm only getting the 404 now. I might create another account and try with that (in case it is because my wife watched Douglas is Cancelled and it's changed something in the data?)

[kereltje]

Don't bother. I just just did a whole lot of tests and each time now ITVX returns HTTP status 404. They must have changed something after the weekend.
If your wife did watch Douglas is Cancelled you should not get any error at all. Douglas is Cancelled should simply be on the list, unless she just finished watching the last episode.

Unfortunately, we don't have a simple way to test anymore. I can only advice to check kodi log and gdb trace for JsonDecodeErrors to find out whether this workaround is actually working. I'm not so sure it does, actually.
Even so, it's only a workaround, not a solution - you still have the issue of simplejson crashing Kodi when it's calling str.count(),  while doing the same in interactive mode completes fine. All a bit mysterious to me and the underlying problem might be the cause of other errors. 
Perhaps it's wise to file an issue at the Kodi from Debian project. But in order to investigate someone must be able to replicate the problem. Can you advise @basilgello?

[basilgello]

I was thinking about setting up a test build of addon which would return mocked responses from local server we control. This way we could fuzz stuff and make it reproducible. FTR, I tested on bookworm container until I realized James is using sid now. Gonna try sid as well, maybe I am lucky enough to get the crash.

[kereltje]

I was thinking about setting up a test build of addon which would return mocked responses from local server we control. This way we could fuzz stuff and make it reproducible. 

Excellent idea! I don't think that setting up a web-server is quite necessary at this stage. 
Here is an addon with some simple tests to try for induce the error on TheJezstarr's system. Can also be downloaded as zip from https://dimkroon.net/tests/plugin.video...._0.0.1.zip

@TheJezstarr, could you install this test plugin, and run each test until Kodi crashes. Please try with, and without the patch of the requests package we did earlier. No need to enable debug logging for this test, but do post a log please.
Disregard possible Kodi messages regarding streams that failed to play. All test should run without error, except the last one, on which Kodi should show a notification to check the log.

[TheJezstarr]

Right, so this is with the original simpleJSON. No crash, but there were messages.

This is with simpleJSON removed.

[kereltje]

Thanks, interesting. Need to have a good look, but haven't much time at the moment.
One thing I did notice: in the last logs Kodi uses python 3.12.6 while your test results in this post reported python 3.12.5. Has your system been updated over the weekend, or does Kodi run a different binary? Can you check again what version you run now in interactive mode?

[TheJezstarr]

Someone did a full update a few days ago - the culprit was trying to update Zotero and forgot to untick the python updates.

[kereltje]

OK, that's one mystery solved.

The tests failed at exactly the same line of code as with the original crashes, only now throwing a python exception, but not the expected JSONDecodeError. Although it did not crash Kodi (which is probably because of the python update), it is still the same thing that goes wrong. Just with a little less dramatic effect.

The good thing is we've managed to reproduce the issue without doing an actual request, but somehow the requests package does affect the tests.

I've updated the tests to investigate the role of request and log a bit more info. The plugin is now named "Simplejson crash test" and also available as zip from https://dimkroon.net/tests/plugin.video...._0.0.4.zip
Can try these test and post a log again?

And can you try the following from an interactive python session and post the output?

python:


>>> import requests

>>> import simplejson

>>> requests.__version__

>>> simplejson.loads('no content')

>>> requests.get('https://google.com')

>>> simplejson.loads('no content')

Pretty much the same you did earlier, but now with requests imported, just to check if it has the same effect as in Kodi. Simplejson.loads() should normally throw a JSONDecodeError, but in Kodi you get a TypeError.

[TheJezstarr]

Really not sure what happened here; I was certain I posted a reply on Tuesday. Anyway, here it is again:

python:


>>> import requests

>>> import simplejson

>>> requests.__version__

'2.32.3'

>>> simplejson.loads('no content')

Traceback (most recent call last):

 File "<stdin>", line 1, in <module>

 File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 514, in loads

   return _default_decoder.decode(s)

          ^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 386, in decode

   obj, end = self.raw_decode(s)

              ^^^^^^^^^^^^^^^^^^

 File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 416, in raw_decode

   return self.scan_once(s, idx=_w(s, idx).end())

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

>>> requests.get('https://google.com')

<Response [200]>

>>> simplejson.loads('no content')

Traceback (most recent call last):

 File "<stdin>", line 1, in <module>

 File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 514, in loads

   return _default_decoder.decode(s)

          ^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 386, in decode

   obj, end = self.raw_decode(s)

              ^^^^^^^^^^^^^^^^^^

 File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 416, in raw_decode

   return self.scan_once(s, idx=_w(s, idx).end())

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Tests done without simplejson here.
With simplejson back in, it actually crashed on test no.3 and 4 without errors in the log, so I did a GDB:

gdb:
#3 Frame 0x7fffbf005510, for file /usr/lib/python3/dist-packages/simplejson/errors.py, line 7, in linecol (doc='no continue watching items', pos=0)

lineno = doc.count('\n', 0, pos) + 1

#3 Frame 0x7fffbf005438, for file /usr/lib/python3/dist-packages/simplejson/errors.py, line 16, in errmsg (msg='Expecting value', doc='no continue watching items', pos=0, end=None)

lineno, colno = linecol(doc, pos)

#3 Frame 0x7fffbf005380, for file /usr/lib/python3/dist-packages/simplejson/errors.py, line 41, in __init__ (self=<JSONDecodeError at remote 0x7fff542c9be0>, msg='Expecting value', doc='no continue watching items', pos=0, end=None)

ValueError.__init__(self, errmsg(msg, doc, pos, end=end))

#18 Frame 0x7fffbf0052d0, for file /usr/lib/python3/dist-packages/simplejson/decoder.py, line 416, in raw_decode (self=<JSONDecoder(encoding='utf-8', object_hook=None, object_pairs_hook=None, parse_float=<type at remote 0x7ffff7b58600>, parse_int=<type at remote 0x7ffff7b5adc0>, parse_constant=None, strict=True, parse_object=<function at remote 0x7fff4972ab60>, parse_array=<function at remote 0x7fff4972ac00>, parse_string=<built-in method scanstring of module object at remote 0x7fff6eb09030>, memo={}, scan_once=<simplejson._speedups.Scanner at remote 0x7fff542ca960>) at remote 0x7fff6c3d9490>, s='no continue watching items', idx=0, _w=<builtin_method at remote 0x7fff6c28c3b0>, _PY3=True, ord0=110)

return self.scan_once(s, idx=_w(s, idx).end())

#18 Frame 0x7fffbf005218, for file /usr/lib/python3/dist-packages/simplejson/decoder.py, line 386, in decode (self=<JSONDecoder(encoding='utf-8', object_hook=None, object_pairs_hook=None, parse_float=<type at remote 0x7ffff7b58600>, parse_int=<type at remote 0x7ffff7b5adc0>, parse_constant=None, strict=True, parse_object=<function at remote 0x7fff4972ab60>, parse_array=<function at remote 0x7fff4972ac00>, parse_string=<built-in method scanstring of module object at remote 0x7fff6eb09030>, memo={}, scan_once=<simplejson._speedups.Scanner at remote 0x7fff542ca960>) at remote 0x7fff6c3d9490>, s='no continue watching items', _w=<builtin_method at remote 0x7fff54283dd0>, _PY3=True)

obj, end = self.raw_decode(s)

#18 Frame 0x7fffbf005150, for file /usr/lib/python3/dist-packages/simplejson/__init__.py, line 514, in loads (s='no continue watching items', encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, use_decimal=False, allow_nan=False, kw={})

return _default_decoder.decode(s)

#18 Frame 0x7fffbf0050b8, for file /home/james/.kodi/addons/plugin.video.smpljsoncrashtest/addon.py, line 97, in test_simpjson_loads (simplejson=<module at remote 0x7fff542804f0>, requests=None)

simplejson.loads(DOC)

#18 Frame 0x7fffbf005020, for file /home/james/.kodi/addons/plugin.video.smpljsoncrashtest/addon.py, line 189, in <module> () callb()