[FernetMenta]

I still need to understand why a UI, that's purpose is to display data, wants to do something other than wait for the data it is going to display to arrive before rendering?

There is a very big difference in waiting in terms of a synchronous blocking call or in an asynchronous way. If you do processing on the main thread, the application becomes unresponsive for the time of processing. It won't render and it won't accept any user input. That also means that you can't even display a busy dialog, the application is just dead during the time of processing. Background video or visualisations also dead.

Not long ago I fixed some issue where a DB operation was hooked into OnPlaybackStarted (or some announcements hooked into this callback). That call blocked the Raspberry Pi for approximately 4 seconds.

The solution is rather simple and can already be found in Kodi. Keep the application alive and wait asynchronously. Give the user also a chance to cancel the operation.

[Montellese]

I can understand why cleaning the database or installing an addon are done asynchronously, there is no need for the user to wait for those.

That's not true in all cases. You can manually start database cleaning from settings and you will get a blocking dialog with a cancel button. The same goes for installing addons from a "choose addon" dialog where the user wants to wait for the chosen addon to be installed immediately.

Part of what confuses me is that for say the slow songs node now, all on same thread, I see a whirling cursor and I can back out from it using Esc or Back Tab. The UI does not appear to freeze, or is that an illusion?

This is because VFS access from CGUIMediaWindow is already done asynchronously. You tell the VFS to fetch some path (which in your case results in accessing the database for all songs) and you let it do it in another thread so that the UI thread can show the busy dialog/spinner and also be able to cancel it based on user input.

[DaveBlake]

Sorry for repeatedly asking "why" and "what is gained", but I find the requirement to wait asynchronously every time the UI gets data to display, even just fetching a single record, totally surprizing.

In the few places I can create a delay in the UI receiving the data it needs it is always possible to cancel the operation from the keyboard interrupt. Yes the songs node does block simultaneous video playback (haven't managed to stop music yet), but if the user is browsing their music or video library the UI is full of info. Is uninterrupted playback in that situation a requirement? It seems that it is.

Anyway I know that if Fernet and Montellesse, let alone anyone else (and that does seem to be everyone), think that getting data for the UI should have a separate thread then that is the way it will be done.

The solution is rather simple and can already be found in Kodi.

Marvellous, should be easy for someone to make that change then.
I wonder why it has not all happened already?

Are we talking one data getting/setting thread? New thread per read/write? What is the current thread/process design anyway? All in the code somewhere no doubt. At the end of this I can only conclude I am not the best person for that job, because I would need more than Fernet's cryptic comment to go on. Perhaps this thread puts it on the table more prominently for someone else to take up.

[DaveBlake]

Part of what confuses me is that for say the slow songs node now, all on same thread, I see a whirling cursor and I can back out from it using Esc or Back Tab. The UI does not appear to freeze, or is that an illusion?

This is because VFS access from CGUIMediaWindow is already done asynchronously. You tell the VFS to fetch some path (which in your case results in accessing the database for all songs) and you let it do it in another thread so that the UI thread can show the busy dialog/spinner and also be able to cancel it based on user input.

Ah, thanks for that helpful explanation. I thought maybe it was a matter of keyboard intrrupts, I'll have a look at CGUIMediaWindow.

[popcornmix]

Sorry for repeatedly asking "why" and "what is gained", but I find the requirement to wait asynchronously every time the UI gets data to display, even just fetching a single record, totally surprizing.

In an ideal world the GUI thread cycles round at 60Hz and nothing should block that thread. You never want to miss a frame update that may cause a dropped video frame, or a judder in the screensaver, or a stutter on the RSS scrolling.
Even worse a network request (e.g. to a MySQL database) may be very slow. It may even have gone offline and take 30s (or even forever) to timeout. If the UI thread is blocked then nothing can happen, including pressing the cancel button.

So, any work that may block should be in a different thread. The UI thread, every 1/60th of a second will render with the latest information it has available. For slow requests it may render an empty list or a progress bar for many seconds. But as soon as the worker thread has the data, the next iteration of the UI will render the final data.

The difference is during those seconds, the UI is updating as normal. The progress bar is animating. Video can still play. Button presses are handled and the user can cancel the operation.

[popcornmix]

Here is an example of a slow operation being called from UI thread being moved into a worker thread using CJobManager:
https://github.com/xbmc/xbmc/pull/8766

[FernetMenta]

It is also a matter of encapsulation. The UI is just a client of this functionality (like JSON-RPC, Python, etc). Clients do not want to know where the data comes from and a dependency to a source like a database should be avoided. If the UI accesses DB directly, other clients need to do so as well.

Kodi's UI started with pulling every info directly from each component, regardless how deep this was encapsulated. There is still code that pulls information directly from VideoPlayers demuxer, which is very bad. We can't change the application from one day to another but in small steps. For VideoPlayer the design is changing from a polling model to a event driven model. VP pushes all UI relevant data to a cache the UI and other clients can read from:
https://github.com/xbmc/xbmc/blob/master...acheCore.h

This cache will contain state information in terms of what data has been changed. No need anymore for the UI to compare every single parameter for a difference.

[da-anda]

Just out of curiosity - is this cache really the way forward? The UI will again be polling from it, wouldn't it? Also, it's again a static core component. IMO the UI should manage it's own cache by grabbing the data from a dedicated event system. Any component, like JSON-RPC or remote clients via sockets could listen for those events and perform necessary actions. I don't want this to be overengineered, but when I think of headless Kodi and a probably web based UI, it would also be better to get new data pushed to those clients instead of polling it once again. Just a thought.

[FernetMenta]

Just out of curiosity - is this cache really the way forward? The UI will again be polling from it, wouldn't it? Also, it's again a static core component. IMO the UI should manage it's own cache by grabbing the data from a dedicated event system. Any component, like JSON-RPC or remote clients via sockets could listen for those events and perform necessary actions. I don't want this to be overengineered, but when I think of headless Kodi and a probably web based UI, it would also be better to get new data pushed to those clients instead of polling it once again. Just a thought.

No, it is definitely not a step back. Components like VP will never push data into components it does not know because those may block the call.

EDIT: More time now so I can be more verbose. If desired the cache can be made an active object and send data to subscribers. Note what I said above: we can't change the app in a single day. The challenge is not to outline an ideal architecture but a step by step approach to get to one.

[Memphiz]

Make the cache an active object and you have what i call a datamodel approach. Which works well for sure