Req Virtual Controllers through JSON-RPC
#1
Hi there,

first I want to thank garbear and all developers for working on game branch! Nice job. Hope it stays in v18 branch!

I have a feature sugesstion: Sometimes if friends visit you, you would like to play a multiplayer game, but you have just a limited amount of controllers. In this case it would be nice, if you can use available smartphones as a virtual controller/gamepad.

Therefore my suggestion:
Please provide an option to add virtual controllers by kodis JSON-RPC API.


A virtual controller should be configurable the same way as a common controller, since they can emulate different layouts and a player could prefer a specific key assignment.
I hope this is possible?

Maybe a few questions regarding this feature:
  • Will a virtual controller through JSON-RPC suitable, since there might be a delay for the requests?
  • Is it in general possible to redirect virtual controller from RPC to the libretro input system?
  • What do you think about this idea? Or am I the only one interested in it?
Reply
#2
Yes, extending the joystick API to JSON is something I have planned. The phone can register itself as a virtual joystick. It will then be usable in games.

The JSON device will be able to emulate every controller profile - when Kodi wants a port with a particular controller type, the phone can either show a layout specific to that controller, or show a set of generic controls which Kodi will automatically translate into the correct controller profile.

We'll need a two-way interface for rumble and switching controller profiles when different games are loaded. I'm not sure if JSON-RPC provides a way to do callbacks, so we may have to implement bidirectional communication on to pof the API.
Reply
#3
A bidirectional communication might be possible by HTTP/2 server push, which enables the webserver to push information to the client.
If my research is correct, Kodi uses libmicrohttpd, which 'is HTTP 1.1 compliant' - this means (I believe) that it won't support server push.

Another option would be to open a socket somehow else - but I believe that is not really a goot solution.


In my opinion rumble is optional and could be a feature of this feature, so for a first implementation not necessary.

And is it really neded for switching controller profiles?
Reply
#4
We already offer a JSON-RPC access point over raw TCP on port 9090 which supports sending notifications from Kodi to JSON-RPC clients. But right now we don't support addressed notifications i.e. all notifications are sent to all controllers.

Are we sure that using JSON-RPC is the right choice for something like input where minimal delay is crucial? What about using EventServer which is "designed" for input event handling?
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#5
or maybe the websocket
Reply
#6
(2017-01-05, 14:37)Montellese Wrote: We already offer a JSON-RPC access point over raw TCP on port 9090 which supports sending notifications from Kodi to JSON-RPC clients.
Hmm, do we really need a bidirectional channel? Of course this would not be a perfect solution, but for some games like 2048 sufficient. So assuming the JSON RPC gamecontroller as simplified version would be sufficient. For getting the full gaming experience "real" gamecontrollers are needed, but for testing?


(2017-01-05, 14:37)Montellese Wrote: Are we sure that using JSON-RPC is the right choice for something like input where minimal delay is crucial? What about using EventServer which is "designed" for input event handling?
Of course JSON-RPC is not the best solution for those tasks. But, as mentioned, suffient for some games. From my point of view (but I have not really an idea of the complexity) the JSON RPC implementation is easier to implement than e.g. an additional event server. So assuming such an implementation as one way to create virtual game controllers there could be two ways in parallel, e.g. we implement the JSON RPC approach in near future and an EventServer 'in the future'...



@garbear: I noticed your great work on the PlayerManager and the multiple gamecontroller profiles. Would it be possible to extend the PlayerManager for those virtual devices?
What workload do you expect for implementing the JSON-RPC controller feature? Would it be possible to implement a proof of concept or is it to much work?
Reply
#7
(2017-08-13, 22:58)muehlmeyer Wrote: @garbear: I noticed your great work on the PlayerManager and the multiple gamecontroller profiles. Would it be possible to extend the PlayerManager for those virtual devices?
What workload do you expect for implementing the JSON-RPC controller feature? Would it be possible to implement a proof of concept or is it to much work?

Game controller profiles are an abstraction between what the OS driver sees (anonymous buttons and axes) and the input sent to games. The idea is to make it easy to add new joystick drivers, such as a JSON-RPC one, without restricting the controller layout. The JSON-RPC client can use a standard layout, and the system will translate it to the desired controller profile. I didn't envision a JSON-RPC client with multiple profiles, but ofc. this is trivial to expand on the current system.

Implementing a JSON-RPC controller is very straight-forward. We just need a bridge between the JSON-RPC api and the peripherals subsystem. If a client was willing to use the API, I could implement it on the server side.
Reply
#8
Since https://yatse.tv/redmine/issues/1082 Smile

We discussed that a long time ago Garbear, from a performance point of view I think that first implementing EventServer support would be best as the protocol already support everything needed + is very fast and would not be limited by JSON RPC threads limits / http overheads impacts.

https://github.com/xbmc/xbmc/pull/8807#i...-170231046 for the last discussion can't remember where we first talked about that before.

What is needed is just some special profiles names that can be mapped from ES to your code, or eventually a more advanced wrapper that can address all your current controllers layout. (Just not sure how you name things internally and if it will be easy to do, not enough info on your side after the controlled input addons migration)

I'm totally willing to add support for that in Yatse as I'm actually eagerly waiting for that, since my wife have made me sell my arcade cabinet and will love to play again to KOF / Metal Slug if they are supported by retroplayer (Not have found time to actually really test you work).
(Can't promise I'll go to more than a pad / joystick and a few buttons as handling tons of stuff would be way too much work for something that will still be a niche thing until a long time).
Reply
#9
I'd rather not extend the life of ES, it's very poorly integrated into Kodi and JSON-RPC is the way of the future. If HTTP adds too much overhead, is WebSockets an option? Are there other transports with lower overhead?
Reply
#10
ES is here to stay because many things does not work without it.

To kill ES, we need:
- a way to control Kodi when in profile selection screen
- a way to properly call all Kodi actions accessible from keymaps
- an access to keymaps and advancedsettings (That was explicitly refused many times, still for no valid reasons).

Since all those will never happens as some people does not want it, this does not really leave options.

I do not know performances of WS in current implementation, but:
HTTP / JSON overhead are really important and also one reason to use EventServer for remote controls.
TCP connexion currently have threadings issues, so no sure it could handle the necessary request speed (Not talking about impacts on normal usage of that port).

I know some fear EventServer and integration could be better (And would have been if, well you know history Wink ).
But if you exclude bidirectional that is not really mandatory, raw UDP will always be faster and IMO when talking about remote controlling a video game over network every ms count.

Everything is here and works, just needs a link to your new controller plugins. Most is already prepared and even if some were removed, there's still some controller support integrated as users of EventServer can send all necessary Xbox 1 controller for example and it will act exactly as if there was a physical controller. Just missing the link bits to Retroplayer.

If WS can achieve the necessary load + low latency, it may worth a try, but I currently have no time to implement a new protocol for testing something that may in the end not be performant enough, and in that case will wait for a working POC with acceptable performance before investing time on this.

But it would still need a way to use WS without JSON overhead, encoding and decoding data to JSON for fast controlling is a total nonsense. This would mean creating a new protocol whatever transport is chosen.
ES have the transport and the protocol all very optimized for the exact purpose of what we want to achieve.


Edit: I forget to add that JSON RPC is not currently the future, as there's no more maintainers like me. Nearly everything that is recently added to Kodi could have used some JSON-RPC integration but all devs totally ignore that part.
Leading to remotes not being fully able to follow what Kodi offers and things will only get worse.
Reply
#11
Unpleasantries split to https://forum.kodi.tv/showthread.php?tid=319979
Reply
#12
(2017-08-15, 18:54)Tolriq Wrote: Most is already prepared and even if some were removed, there's still some controller support integrated as users of EventServer can send all necessary Xbox 1 controller for example and it will act exactly as if there was a physical controller. Just missing the link bits to Retroplayer.

Unfortunately this isn't the case. (and the fact that it's Xbox 1 should tip you off that this is long dead code.) ES hooks into the existing keymap system. Joysticks use another system entirely, and keymapping is built on top of joysticks. This is because keymapping lacks a fundamental feature - the ability to send "releases". To make ES work for controlling games, we would have to integrate it into the underlying joystick layer. Expanding ES would be an option if it didn't affect the rest of Kodi, but integrating ES deeper into Kodi isn't the way to go.

Does JSON-RPC support UDP sockets? Would that solve our speed issues?
Reply
#13
No JSON-RPC doesn't support UDP. It works through HTTP and TCP. TCP should be a little faster, but i bet that a lot of users don't use it right now and just go through HTTP (less network issues).

Furthermore, JSON-RPC as a protocol has a lot of overhead, no matter the transport. It really isn't suited to something that needs to be responsive, but we can try and see how it goes.

It will probably even work acceptably, just makes me cringe to think of all the wasted power...
Reply
#14
as for the speed issues via JSON. Does that really matter much? I mean, a touch screen as gamepad is a bad gameplay experiences in every possible way and nobody will play any response critical games with such a remote anyways. And for casual gaming it shouldn't be too much of an issue, right? But if we can get rid of some overhead, the better ofc.

Would BSON be an option? Should be a bit faster than JSON de/encoding.
Reply
#15
(2017-08-16, 10:32)syncd Wrote: It will probably even work acceptably, just makes me cringe to think of all the wasted power...

I'm not sure if I follow this bit. Do you mean like JSON is a choke point, so the powerful phone/tablet can't effectively communicate with the powerful Kodi gaming system?
Reply

Logout Mark Read Team Forum Stats Members Help
Virtual Controllers through JSON-RPC0