[devkid]

Lets say I want to set the watched flags for 100 movies by JSON RPCs calls. I do this by calling "VideoLibrary.SetMovieDetails" for every movie via xbmc.executeJSONRPC in python. But doing this for lots of movies takes a significant amount of time. Any ideas what I could do to speed things up?
For example it is possible to fire multiple RPCs in a single call or something?

Thank you guys!

[Montellese]

I'm not 100% sure but xbmc.executeJSONRPC should also support JSON-RPC batch requests.

[devkid]

Thank you, exactly what I as looking for. I just had no idea how it was called some my google-fu failed.

Anyway, I tried it but it seems XBMC does only process the first request from the batch although the docs say batch requests are support (if I understand it correctly):
http://wiki.xbmc.org/index.php?title=JSON-RPC_API/v6

But it seems XBMC ignores all further request. Here is what I tried:

Code:

17:16:58 T:8392   DEBUG: JSONRPC: Incoming request: {"jsonrpc": "2.0", "method": "VideoLibrary.SetEpisodeDetails", "params": {"episodeid" : 131, "playcount" : 1, "lastplayed" : "2013-04-20 17:01:59" }, "id": 5},{"jsonrpc": "2.0", "method": "VideoLibrary.SetEpisodeDetails", "params": {"episodeid" : 129, "playcount" : 1, "lastplayed" : "2013-04-20 17:01:59" }, "id": 6}
17:16:58 T:8392   DEBUG: JSONRPC: Calling videolibrary.setepisodedetails
17:16:58 T:8392   DEBUG: JSONRPC: Type mismatch in type
17:16:58 T:9160   DEBUG: Previous line repeats 1 times.
17:16:58 T:9160   DEBUG: CGUIMediaWindow::GetDirectory (addons://sources/executable/)
17:16:58 T:9160   DEBUG:   ParentPath = [addons://sources/executable/]
17:16:58 T:1064  NOTICE: Thread Background Loader start, auto delete: false
17:16:58 T:1064   DEBUG: Thread Background Loader 1064 terminating
17:16:58 T:8392   DEBUG: CAnnouncementManager - Announcement: OnUpdate from xbmc
17:16:58 T:8392   DEBUG: GOT ANNOUNCEMENT, type: 16, from xbmc, message OnUpdate
17:16:58 T:8392   DEBUG: script.watched.states: Response to SetEpisodesDetails: {'jsonrpc': '2.0', 'id': 5, 'result': 'OK'}

So the last lines show the RPC response but it only contains a response to the first request. Is there anything wrong with my request? Or is it XBMC that just really isnt processing all requests from the batch?

As a sidenote: I get "JSONRPC: Type mismatch in type" for all those request (even when not batching). But I dont get whats wrong. XBMC does not tell which the error is about?
I googled for it and it seems other people get it too: http://forum.xbmc.org/showthread.php?tid...pid1237510

[Montellese]

Can you post your batch request (maybe with only 2-3 requests in it)?
Like I said I'm not sure if it is supported through python but it certainly is through HTTP and TCP.

Concerning the "JSONRPC: Type mismatch in type" log, just ignore it. I think I'm gonna remove it completely as it confuses people too much and there seem to be too few cases where it may be useful.

[devkid]

Can you post your batch request (maybe with only 2-3 requests in it)?
Like I said I'm not sure if it is supported through python but it certainly is through HTTP and TCP.

Sorry, I am not sure what you mean? The "raw" request string is visible in my previous post in the first line of the code-box. It contains a batch of two requests (or at least supposed to ).

If you mean the python code, it looks like this:

Code:

def setEpisodesDetails(episodes):
    log("Firing setEpisodesDetails batch. Length: %d" % len(episodes))
    
    if len(episodes) == 0:
        return

    reqStr = ''
    i=5
    for e in episodes:
        reqStr += '{"jsonrpc": "2.0", "method": "VideoLibrary.SetEpisodeDetails", "params": {"episodeid" : %d, "playcount" : %s, "lastplayed" : "%s" }, "id": %d},' % (e['episodeid'], e['playcount'], e['lastplayed'], i)
        i+=1
    reqStr = reqStr[:-1] #removes trailing comma
    episodeDetailRes = eval(xbmc.executeJSONRPC(reqStr))
    log("Response to SetEpisodesDetails: %s" % episodeDetailRes)

Concerning the "JSONRPC: Type mismatch in type" log, just ignore it. I think I'm gonna remove it completely as it confuses people too much and there seem to be too few cases where it may be useful.

Ok thanks, good to know.

[Tolriq]

You need to put [] around the queries to be in batch mode.

[devkid]

You need to put [] around the queries to be in batch mode.

Thank you mate! That did the trick!
I read the JSON RPC specification and silly me did not see that brackets in the examples...

[devkid]

The sad part of the story: Batching the SetMovieDetails requests does not speed up the process significantly. If I bundle 20 requests into a single RPC it takes quite some time for XBMC to process that RPC. So the bottleneck seems to be somewhere else... nevermind...

[Tolriq]

Yes sadly it is on Xbmc side each SetFooDetails refresh 100% of the data with lot's of queries even when you just want to change resume point or watched status.

Some optimizations could be made on this part but I'd need validation from Montellese to go on
(For example UpdateVideoTag return a bool to know that at least one thing is changed and should update the details or not with a little change for playcount / lastplayed).

[N3MIS15]

I think the bottleneck is that the gui gets updated every time. IIRC it was pretty fast before the gui updating was added.
In a perfect world the gui would only be updated once for each batch request. We have had this problem with the trakt addon.
Syncing thousands of episodes (batched per show IIRC) can be painful and sometimes even crash xbmc (or at least it seems like it does).

[Montellese]

Would it help if you could apply a SetFooDetails() call to multiple video items instead of a single one like using an SQL WHERE condition? Or is the use case really that you have to perform different property changes for specific video items?

The problem with updating the GUI should only be a problem if you are actually viewing a media item list as otherwise it doesn't have to do any updating. Just updating once for a single batch request would probably result in a very ugly hack because the batch request handling has no clue about what is in every single request (and there's no reason that every request should be for the same method) so it doesn't even know that a GUI update will happen. The request itself doesn't know that it's part of a batch request and therefore doesn't know anything about any other requests.

[N3MIS15]

Maybe have another method for updating gui, GUI.Refresh or something? or have a param for it in the setfoodeatils methods "updategui": true
either way would help batch requests, a seperate method could just be called after you send the requests. and an "inline" param could just be tacked on to the last request.

I have no idea how feasible those suggestions are XBMC side.

[topfs2]

Maybe have another method for updating gui, GUI.Refresh or something? or have a param for it in the setfoodeatils methods "updategui": true
either way would help batch requests, a seperate method could just be called after you send the requests. and an "inline" param could just be tacked on to the last request.

I have no idea how feasible those suggestions are XBMC side.

Just an FYI the latter is how Banshee has it in its API. When you create a song, or update they have a bool flag "updateGUI" you can control on save.
Ideally it shouldn't be needed, but I guess it adds a bit of extra locking?

[Martijn]

I would definitely welcome and updateGUI option in the setFoo.* methods and default it to true (if possible of course)

[Montellese]

Well technically it's certainly possible but there are two issues:

1. I don't like it because it may leave users with invalid or outdated information on the screen.
   
2. Adding such a parameter to the SetFooDetails method definitions in the JSON schema is no problem in general but it will mess up the parameter ordering if we ever have to add an additional parameter to these methods. This is no problem for anyone using "by-name" parameters but will be for anyone using "by-position" parameters. Moving it right after the "fooid" parameter would break backwards compatibility.