[popcornmix]

Can you stop kodi with:

Code:

killall -SIGSEGV kodi.bin

That should result in a crashlog that may indiciate if a deadlock has occurred.
Also try enabling CEC in component specific debugging before capturing the kodi.log - it might show what the last CEC events were.

[borderite]

@popcornmix

Unfortunately, "killall -SIGSEGV kodi.bin" generates no crash log, though Kodi's entry in the process list becomes "[kodi.bin]" rather than the usual form like:

/usr/lib/kodi/kodi.bin --standalone -fs --lircdev /run/li...

(successfully killed, but it becamse zombie?). The kodi.log obtained after enabling CEC in component specific debugging is available at:

http://sprunge.us/JCib

This log file is probably useless, as nothing was added to it after the volume control button was used.

Here are a few more observations:

1. As soon as the volume control button is used, the logging information on the screen freezes.
2. When I use the "poweroff" command via ssh, the ssh terminal is immediately disconnected (as you would expect), and RPi3 seems to shut down quickly (steady red LED and no yellow LED), but the TV screen still shows the frozen Kodi.

Please let me know if there is anything I can try.

[Milhouse]

Unfortunately, "killall -SIGSEGV kodi.bin" generates no crash log, though Kodi's entry in the process list becomes "[kodi.bin]" rather than the usual form like:

That should have resulted in a crashlog (once kodi has restarted). If you have no crash log, did you see any errors on the command line when killing kodi? If not, did kodi restart?

Can you try the command again, wait for Kodi to restart, and then run:

Code:

cat /storage/.kodi/temp/kodi_crash.log | pastebinit

and post the link.

[borderite]

@Milhouse

After "killall -SIGSEGV kodi.bin", kodi.bin becomes a zombie and never terminates. (I waited more than two hours). This keeps kodi from restarting.
So, I manually executed the gdm line in /usr/lib/kodi/kodi.sh to get the crash log. Please find it at:

http://sprunge.us/ZaOh

[borderite]

I have manually generated the full crash log and pasted it to:

http://sprunge.us/fPHh

I hope that it will give you hints about how to resolve the issue.

[popcornmix]

I don't see anything abnormal there.
What do "dmesg" and "vcdbg log msg" show after the problem?

[borderite]

Here is the output of dmesg:

http://sprunge.us/PQTO

"vcdbg log msg" got stuck in the state "D" (uninterruptible sleep), yielding nothing.

[borderite]

Actually, once the TV volume is changed, even vcdbg with no arguments hangs. Is GPU dead or something?

[popcornmix]

Actually, once the TV volume is changed, even vcdbg with no arguments hangs. Is GPU dead or something?

Yes, it sounds like that. I've improved vcdbg so it should still work after the gpu had died and may provide some useful information.
This fix is only in the latest Milhouse nightly build.
Can you test that build and assuming it hangs, can you grab output of "vcdbg log msg".

[borderite]

OK, here is the output of vcdbg log msg:

http://sprunge.us/MHeD

[popcornmix]

Thanks. Can you download:
https://github.com/raspberrypi/firmware/...f?raw=true
https://github.com/raspberrypi/firmware/...t?raw=true

and rename/replace the start.elf and fixup.dat on your sdcard? That will include assert logging which may indicate if something unexpected has occurred.

If that doesn't show any new line in log, can you add:
logging_level=0xffffffff

to config.txt and reboot then capture a new log when it hangs.

[borderite]

Thanks for your prompt reply. The output with the modified start.elf and fixup.dat is at:

http://sprunge.us/UJQS

As it does not seem to me that it shows anything interesting (though I am not really sure what to seek), I have also increased the log level as you suggested and got the output at:

http://sprunge.us/JYga

This one contains quite a few lines that start with "HDMI: CEC"; in particular, the log ended with such lines. I hope that it will help you diagnose the problem.

[popcornmix]

Thanks for your prompt reply. The output with the modified start.elf and fixup.dat is at:

http://sprunge.us/UJQS

Actually can you also run:
vcdbg log assert

with the modified firmware - there may be additional messages in there.

[borderite]

Here you go!

http://sprunge.us/LUVF

Change of the TV volume added the last item in the log (the one at 204234.257).

[popcornmix]

The final assert in that log is very interesting. It shows a latch (semaphore) is being acquired with interrupts disabled.
That is a big no-no as it won't be possible to wake up again, so is almost certainly the reason for your freeze.

However I really need to know where that is being called from (latches are obtained everywhere in the firmware code).
Now with a dump of memory, I can probably work out a backtrace that will provide the answers.

However a GB of RAM will be painful to extract and upload.

Can you try adding "total_mem=256" to config.txt and check if you can still reproduce the issue?
That will behave like a very early 256M Pi1, which may make kodi unusable slow but if you can still provoke the issue then you'll need to dump less RAM.

The command you now want is:

Code:

vcdbg save dump.bin 0 0x10000000

(this may produce "Failed to get memory size via ioctl" messages, but they appear to be harmless).
That should produce a 256M file which if you can get it to me (e.g. google drive or dropbox) I can examine.
Also report output of "vcgencmd version" to be sure I'm comparing with same firmware version.