[gujal]

Ever since I upgraded to 18.5 (also happens on 19 nightly) I have an issue with search in most addons, such as youtube, dailymotion, etc ( any addon with a search function)
When you select search and put in a search term, it goes searches and displays a list of videos which is fine.
Now when you play any video and it finishes, instead of showing the list of videos, it now pops up the keyboard input for search again
Pressing OK leads to spinner wheel for ever
Pressing Esc kicks you out of the addon completely back to Kodi menu
Only way to getback into youtube in such case is to put some search term on the keyboard entry, let it search and then without playing any video press Back

Anyone else facing this issue? (From what I can see this is some thing to do with Directory Container Refresh in Kodi 18.5 and above)
How do I fix this in the addon as there is no issue with the addon in Kodi 18.4 or below. Please advise

[ronie]

i can confirm this issue with youtube + kodi v19.

i've checked if there was anything obvious in the Debug Log, and this line caught my attention:
2020-01-20 11:14:47.067 T:22465   DEBUG: Clearing cached fileitems [plugin://plugin.video.youtube/play/]

looking at the changes between 18.4 and 18.5, this one seemed to be the most likely cause:
https://github.com/xbmc/xbmc/pull/16806

reverting that PR indeed fixes the problem.

i have no idea if plugins have to update their code due to those changes or whether it needs to be fixed in kodi.

@DaveBlake

[gujal]

Thanks for looking at this @ronie 
If it needs to be fixed in the addon, please advise on how and I will work on a fix for Dailymotion

[gujal]

@ronie This patch seems to have affected more than the search menus
For example "Recently Updated" in the main menu, lists the addons and the dates they were updated, now if we do a check for updates from the left menu and if some addons do get updated, you can see the "downloading" against the addon name and when it finishes the date still shows the old date and not current date. You have to restart Kodi to see the correct date on the "Recently Updated" menu
Second, some directories where we setup the view mode, for example wall on the main menu of dailymotion is stuck throughout the addon even if the next directory it shows is meant to be iconwall for example. and as you go in and out of the menus, many times it defaults back to the list view, happens in youtube addon as well
Can you please check with @DaveBlake if addon developers need to code differently from now on and if so what needs to be done on the addon side. Thanks

I see that in the latest beta of youtube addon container refresh is being controlled in code for avoiding the search issue

Code:

https://github.com/jdf76/plugin.video.youtube/commit/71709be5fbb2f30e03c5b2b65bef5417651c96c8

[DaveBlake]

Sorry if there has been unintended consequences for addons from the fix of library watched status issues, it is obviously more fragile than I realised. I'll take a look and see what can be done, it is not a matter of just reverting because watched state was an even bigger issue.

[lorkadiscovery]

Sorry if there has been unintended consequences for addons from the fix of library watched status issues, it is obviously more fragile than I realised. I'll take a look and see what can be done, it is not a matter of just reverting because watched state was an even bigger issue.

The problem that's currently occurring here is that addons such as YouTube here https://forum.kodi.tv/showthread.php?tid...4&page=132 are having to bypass this with Container.Updates everywhere.

This in itself is a "hack" in my opinion as a container update requires the list item folder to be set to false to work so we are not now searching into a new container but instead updating the existing container which is not great practice.

We are potentially going to end up here with a fix for this PR being made but some addons such as YouTube finding work around when we should just wait for input on you for this PR.

What would you advise? Do we try and bypass the issues with a container.update or wait to see where this goes and what you find?

Thanks for the reply.

[V8MEM]

Using Container.* is not needed to avoid the search dialog popup. Use the normal plugin directories and the and the result will be cached then when moving up via ".." will use the cached result. People who care about the tickmarks should not cache the plugin results at all.

[DaveBlake]

It seems to be that plugins have been relying on an accidental flaw that was preventing cached items being cleared when the item is updated. This flaw was causing the item to display the wrong watched status  and has been fixed by https://github.com/xbmc/xbmc/pull/16806 which correctly clears cache for updated items including plugin items.

Update an item and the cache for that item will be found and cleared, however I am far less clear on how plugins best implement a search function or why removeal of cache for an updated item is an issue. I don't even use any video addons in real life so someone is going to have to explain the addon needs to me in more detail.

[V8MEM]

The plugin caching is per directory not per item

python:
 xbmcplugin.endOfDirectory(handle[, succeeded, updateListing, cacheToDisc]) 

cacheToDisc [opt] bool - True=Folder will cache if extended time(default)/False=this folder will never cache to disc.

With this change in 18.5 kodi Player is no longer respecting the cacheToDisc parameter.
This wrong watched status is a bug in the plugins people are using that should be using cacheToDisc=False for directories that contain items that will be updated.

Kodi should respect the cacheToDisc parameter or remove this cache altogether if it is not going to be used anyway.

[DaveBlake]

Yes all caching is per directory and not per item. But I am not sure this is about problems in respecting the cacheToDisc parameter, nor am I sure that plugins should use cacheToDisc=False for directories that contain items that will be updated. While the PR that that has been accused of introducing this issue to 18.5 is mostly about cache clearing, I think this could be misleading us, is this related to cache at all?  Then again there is a lot I am unsure about in this area, but I am willing to try to figure it out and to learn.

I think it is about marking watch status of an item causing the entire list to refresh (and to clear cache, but not using cache will not prevent the refresh). Having used Search and played one of the items on the resulting list, the status of the item is marked as watched, the list gets refreshed and this refresh causes the search dialog to be redisplayed.

I just tested the Dailymotion.com plugin v2.4.1 (mentioned in the OP, would have tried youtube too but I believe it already has a workaround) on v18.4 (prior to the PR implicated) and it also behaves as the OP describes - when a video finished the search dialog is redisplayed.

Can someone else confirm the issue is in v18.4 as well please (and therefore not caused by PR16806).

[DaveBlake]

OK, seems that there are 2 related problems for video addons that use search
a) mark an item as watched causes the search dialog to be redisplayed - happens in version back to 17.6 at least
b) play an item and when it finishes the search dialog to be redisplayed  only happens in 18.5 onwards

I had problems finding something to play from DailyMortion that didn't cause 403 errors, but evetually managed to test using Vimeo. Hence I found issue a), but was not correctly testing for b) just assuming that marking was hitting the same issue as playback finishing (it marks watched). Got it now, but isn't video stuff fun (not).

[V8MEM]

Doubt any dev wants this burden of an overly complex cache system to handle this dynamic watched status correctly.

Plugins that need cache already use python modules to cache and with <reuselanguageinvoker/> navigation in plugins is fast.
cacheToDisc default could be changed to False with minimal impact and the directory cache reverted back to being strictly static that would not be deleted by Player actions so that items that actually need it, like Search, can use it.

Sure there are many bad plugins out there and this change could result in increased load on web services, but this is already the case with the change in 18.5 where the cache is deleted after every playback and never used.

[DaveBlake]

Yeap, this does feel like playing Jenga. I got sucked in because I tried to fix a v18 regression (that users were upset about but no other dev was picking up) where the watched status for library items was not being updated on screen, but video is not my thing at all. Kodi list caching was designed when memory was limited, and IMO the whole list based GUI approach is in need of a redesign, but we are where we are.

With current filelitem list design to display an updated item (after playing it or marking it as watched etc.) Kodi has to refresh the entire list, in fact all lists that the item could be in.  It also has to clear any cache for the list(s) before refreshing otherwise the old data would be shown. This was broken in with the player/GUI/application interface changes in v18. Previously in many cases the GUI list item got updated directly and the application caught up and no reading cache was involved. But that was the tail wagging the dog, so now to the update the display needs the cache clearing and list refreshed.

I don't think that having some option whereby the cache is kept and so the display does not get updated is viable, sooner or later some one will ask why played items are not immediately shown as watched. Maybe users are less worried about immediate watched status display updates for plugin items than for llibrary items, but still it isn't the way to go since Kodi bothers to hold if something has been watched it should also show it.

That means the way forwards is to trigger the recreation of the search results list (with updated watched status read from the db) on refresh. Saved searches do that, the item list has a path like "plugin://plugin.video.youtube/kodion/search/query/?q=horsepower". But the path for  a new serach is "plugin://plugin.video.youtube/kodion/search/input/", and so you get the search dialog displayed. So I am looking at how that can be done i.e. setting the item list path when the list is filled with serach results like it is when a saved search is displayed. Any suggestions along these lines?

[V8MEM]

It is a matter of avoiding the search input endpoint in plugin history
xbmcplugin.endOfDirectory(updateListing=True) , Container.Update(plugin,replace) or ActivateWindow/ ReplaceWindow(window,dir) but these are likely not meant to be used by plugins.

If this doesn't work with remote apps, then a script entry could be used, this would require a search history file/db

Container.Update

python:
@plugin.route("/search_cntu")

def search_cntu():

    dialog = xbmcgui.Dialog()

    keyword = dialog.input("Search")

    xbmcplugin.endOfDirectory(plugin.handle, cacheToDisc=False)

    if keyword:

        xbmc.executebuiltin("Container.Update({0})".format(plugin.url_for(search, keyword=keyword)))

    else:

        xbmc.executebuiltin("Container.Update({0},replace)".format(plugin.url_for(root)))



@plugin.route("/search")

def search():

    keyword = plugin.args["keyword"][0]

    list_items = []

    for item in range(5):

        title = "result {} {}".format(keyword, item)

        li = ListItem(title)

        li.setProperty("IsPlayable", "true")

        li.setInfo(type="Video", infoLabels={"Title": title, "mediatype": "video"})

        list_items.append(("plugin://plugin.video.youtube/play/?video_id=dQw4w9WgXcQ", li, False))

    xbmcplugin.addDirectoryItems(plugin.handle, list_items)

    xbmcplugin.endOfDirectory(plugin.handle, updateListing=True, cacheToDisc=False)

Search with history and script entry

python:
@plugin.route("/search_history")

def search_history():

    list_items = []

    search_history = []

    new = ListItem("New Search")

    list_items.append((plugin.url_for(new_search), new, False))  # ! not a folder ! could also be in context menu

    if os.path.exists(history_file):

        with io.open(history_file, "r", encoding="utf-8") as f:

            try:

                search_history = json.loads(f.read())

            except:

                search_history = []

    for item in search_history:

        li = ListItem(item)

        list_items.append((plugin.url_for(search2, keyword=item), li, True))



    xbmcplugin.addDirectoryItems(plugin.handle, list_items)

    xbmcplugin.endOfDirectory(plugin.handle, cacheToDisc=False)



@plugin.route("/new_search")

def new_search():

    search_history = []

    dialog = xbmcgui.Dialog()

    keyword = dialog.input("Search")

    if keyword:

        if os.path.exists(history_file):

            with io.open(history_file, "r", encoding="utf-8") as f:

                try:

                    search_history = json.loads(f.read())

                except:

                    search_history = []

        search_history.append(keyword)

        with io.open(history_file, "w", encoding="utf-8") as f:

            f.write(json.dumps(search_history).decode("utf-8"))

    xbmc.executebuiltin("Container.Refresh()")





@plugin.route("/search2")

def search2():

    keyword = plugin.args["keyword"][0]

    list_items = []

    for item in range(5):

        title = "result2 {} {}".format(keyword, item)

        li = ListItem(title)

        li.setProperty("IsPlayable", "true")

        li.setInfo(type="Video", infoLabels={"Title": title, "mediatype": "video"})

        list_items.append(("plugin://plugin.video.youtube/play/?video_id=dQw4w9WgXcQ", li, False))

    xbmcplugin.addDirectoryItems(plugin.handle, list_items)

    xbmcplugin.endOfDirectory(plugin.handle, cacheToDisc=False)