ACCEPTED: NVIDIA GameStream
#1
Hello, Anthony, a 3rd year Computer Science and Engineering student looking for an emulation related project to tackle. I know there is a thread going on already, I'd just like to formalize my own separate proposal here though.

NVIDIA GameStream
Name: Anthony Miyaguchi
Forum/Email: acmiyaguchi; [email protected]
Summary:
Quote:NVIDIA GameStream™ technology brings high-fidelity, low-latency PC gaming to your NVIDIA SHIELD device. It harnesses the power of GeForce® GTX™ graphics cards to stream your games from your PC to your SHIELD device over the Internet, unleashing PC gaming around the house or on the go.
The protocol has been reverse-engineered by the folks at Limelight and would make a perfect addition to the gaming ecosystem being nutured by RetroPlayer. A GameStream client addon would allow people to stream their desktop games to Kodi-enabled devices, allowing access to a wide variety of games, including Steam.

How I will achieve this:
By using an open source implementation of the GameStream API that has been developed by Limelight, I hope to create an addon and translation layer that will act as the mediator between GameStream and Kodi's RetroPlayer. Most of the development would be using a common C library and the Game API in Retroplayer.

Benefits:
With this in place, the Kodi-enabled device only has to match the system requirements needed by the GameStream protocol, which was developed for the portable NVIDIA Shield in the first place. This would allow htpc's to also act as a thin client for desktop games.

Goals:
  • Integrate Limelight GameStream API into a binary plugin
  • Create a client in kodi around the GameStream API that utilizes the Game API
  • Ensure cross-platform support by providing build support for multiple systems
  • Providing a buildsystem that is consistent with other RetroPlayer plugins

What does it touch in Kodi:
Game API, Addon system. Input, Video, and Audio streams would need to be dealt with.

Requirements:
C, C++

Thanks, let me know if there are any suggestions that would help make this proposal better.

Status: Student Passed
Code: https://github.com/acmiyaguchi/game.moonlight
Reply
#2
Looks pretty good. Any chance you have reference to any past coding experience, possibly on github or elsewhere?
Reply
#3
I have some stuff on my github. You might have to sift through some garbage repositories I have on there of some hackathons gone awry. I've done a little bit of work on garbear's input refactorization last year, so I've touched the Kodi (or XBMC at the time) codebase a little bit. I did a little bit of work on a XCB binding in rust before school started up last year, with a small window manager demo, which was probably the last non academic thing I've coded.
Reply
#4
I wonder if it wouldn't make more sense to add it as an alternative to retroplayer, i..e hook into the player stuff very similarly to how retroplayer has done instead of adding it to retroplayer. Could obviously extract stuff from retroplayer into a super class shared between gamestreamplayer and retroplayer.

Other than that I think it sounds like a very interesting project. Have you tried the limelight code and seen if it works? How active is community around the open source gamestream api? Mostly asking as it would be a shame if you got it working and 4 months later nvidia alters something and none of the limelight folks fix it Smile (Perhaps our community is ready to step in then though)

Cheers,
Tobias
If you have problems please read this before posting

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.

Image

"Well Im gonna download the code and look at it a bit but I'm certainly not a really good C/C++ programer but I'd help as much as I can, I mostly write in C#."
Reply
#5
(2015-03-11, 00:20)acmiyaguchi Wrote: By using an open source implementation of the GameStream API that has been developed by Limelight
Cool, I would just be sure to first checkup on the potential open source licensing compatibility issues mentioned in the other thread.

(2015-03-11, 00:20)acmiyaguchi Wrote: I hope to create an addon and translation layer that will act as the mediator between GameStream and Kodi's RetroPlayer. Most of the development would be using a common C library and the Game API in Retroplayer.
Did you mean to do it similar to what garbear did with his Libretro Wrapper library for the Game API instead of like topfs2 suggested above?


For everyone else's reference, below is a diagram of garbear's layout design for how RetroPlayer and the Game API fits into Kodi today:

Image

This is the current layout of the RetroOlayer project. It is centered around two APIs, the Game API (connects RetroPlayer to Game Clients) and the Hardware API (connects Game Clients to input). The Hardware API moves all of the joystick code out of XBMC and into binary addons. I have also started working on support for the Retrode and Retrode-like devices through the media reader sub-API.

You'll notice the only functional difference is that RetroPlayer is no longer connected to any input. I've re-written the joystick API two or three times, and the end result was never satisfactory because I couldn't separate the input from RetroPlayer. Now, with input gone, the complexity of RetroPlayer is greatly reduced, leading to more maintainable code.

Putting as much code as possible behind APIs is generally a good strategy. I learned this rather late in the development of RetroPlayer, so there are still quite a few parts that are entangled. however, looking at this diagram, I think I'm on the right track - components are finally only communicating with their API, and not with other components.
Reply
#6
@topfs2: I believe the Game API already abstracts away things that would make a GameStream addon fit in well. Also, the Limelight community as of now seems to be active enough, there is active development on their android and iOS ports.

Quote:Did you mean to do it similar to what garbear did with his Libretro Wrapper library for the Game API instead of like topfs2 suggested above?

Yes, that would be the most logical and isolated way to develop everything. This way, the addon would be more of less independent from the rest of RetroPlayer. I think as discussed before, most of the licensing issues are a relative non-issue due to it being a binary plugin.
Reply
#7
I must admit I would personally prefer a Steam in-home streaming plugin, but this project seems much more likely to be successful, since the protocol is already known.
If the addon is correctly designed, though, it could be possible to implement valve's solution one day, without touching a lot of the common functionality.
I need to improve my English skills, feel free to correct me ;-)
My GSOC 2014 post CANCELLED 2015 one
Reply
#8
Hey anthony,

as you know, I'll be the mentor for this project. excuse my lack of comments so far, we've talked about this proposal and so far and it matches exactly what I had in mind.

(2015-03-11, 00:49)natethomas Wrote: Looks pretty good. Any chance you have reference to any past coding experience, possibly on github or elsewhere?

I'd like to point out that anthony has contributed to RetroPlayer before. About a year and a half ago he helped advance my joystick work: https://github.com/garbear/xbmc/pull/13 . The code was a little rough then, but I've seen your recent work and I'm sure this is a project you can handle.
Reply
#9
Hi, developer of Limelight here.

Glad to see this project has some traction. I think integration Kodi integration is a neat idea. I'd be happy to help in any way possible.

As you've figured out, the limelight-common-c library is your best bet for integration. You'll find limelight-common-c isn't an exact replacement for limelight-common (Java).

The HTTP(S) part of the protocol isn't implemented in the library because I didn't want to complicate things by depending on an HTTP library, since the original clients that used this (iOS and Windows RT) had ways to do HTTPS in managed code. Nvidia's use of HTTPS is complicated by the fact that they use TLS client authentication. Since you're considering a C++-based binding to common-c, I'll take a look at adding HTTP support to common-c to make your life easier.

HTTPS client authentication also complicates testing in a browser. For testing, I've written a small Java program that relays to GFE for testing HTTP requests in a browser (among other things). I'll send that your way if you want it.

For some reference on the HTTP-based protocol:
Most HTTP functions: https://github.com/limelight-stream/lime...vHTTP.java
Pairing functionality: https://github.com/limelight-stream/lime...nager.java
Reply
#10
Very cool, I'm glad to see resources for this project. If this project gets past the proposal stage (and even if it doesn't), I'll definitely make sure to hit you up. I'll take a look into it in more detail soon and update a list of checkpoints I'll hit along the way, its nice to know the potential pitfalls before they sneak up on me during development.
Reply
#11
(2015-03-19, 22:31)cgutman Wrote: As you've figured out, the limelight-common-c library is your best bet for integration. You'll find limelight-common-c isn't an exact replacement for limelight-common (Java).

The HTTP(S) part of the protocol isn't implemented in the library because I didn't want to complicate things by depending on an HTTP library, since the original clients that used this (iOS and Windows RT) had ways to do HTTPS in managed code. Nvidia's use of HTTPS is complicated by the fact that they use TLS client authentication. Since you're considering a C++-based binding to common-c, I'll take a look at adding HTTP support to common-c to make your life easier.
FYI, Kodi already have both libCURL and GnuTLS (and OpenSSL) libraries as dependencies: http://curl.haxx.se/libcurl/

This CURL client library provides a C API that supports HTTPS with TLS (and SSL): http://curl.haxx.se/docs/ssl-compared.html
Reply
#12
(2015-03-20, 08:54)acmiyaguchi Wrote: Very cool, I'm glad to see resources for this project. If this project gets past the proposal stage (and even if it doesn't), I'll definitely make sure to hit you up. I'll take a look into it in more detail soon and update a list of checkpoints I'll hit along the way, its nice to know the potential pitfalls before they sneak up on me during development.

Major problems will be enabling hardware acceleration on PC. Limelight currently is totally software based reading from their release on xda
Reply
#13
(2015-03-22, 18:05)evilsephiroth Wrote: Major problems will be enabling hardware acceleration on PC. Limelight currently is totally software based reading from their release on xda

What specifically are you talking about? Do you mean hardware decoding on the client side?
Reply
#14
(2015-03-23, 01:42)Dark_Slayer Wrote:
(2015-03-22, 18:05)evilsephiroth Wrote: Major problems will be enabling hardware acceleration on PC. Limelight currently is totally software based reading from their release on xda

What specifically are you talking about? Do you mean hardware decoding on the client side?

Nothing prevents hardware decoding in either limelight common libraries. Both current common-c clients use hardware decoding. The common library just delivers a raw H264 stream. What the client does with the stream is up to the implementation.

Limelight-PC uses software decoding because ffmpeg's CPU decoder was much easier than writing the 3 or more hardware acceleration bindings required for Windows, OS X, and Linux. I imagine Kodi has these already, which considerably simplifies the work required.

The iOS, Windows RT, and Android clients all use hardware accelerated decoding.
Reply
#15
(2015-03-23, 02:24)cgutman Wrote:
(2015-03-23, 01:42)Dark_Slayer Wrote:
(2015-03-22, 18:05)evilsephiroth Wrote: Major problems will be enabling hardware acceleration on PC. Limelight currently is totally software based reading from their release on xda

What specifically are you talking about? Do you mean hardware decoding on the client side?

Nothing prevents hardware decoding in either limelight common libraries. Both current common-c clients use hardware decoding. The common library just delivers a raw H264 stream. What the client does with the stream is up to the implementation.

Limelight-PC uses software decoding because ffmpeg's CPU decoder was much easier than writing the 3 or more hardware acceleration bindings required for Windows, OS X, and Linux. I imagine Kodi has these already, which considerably simplifies the work required.

The iOS, Windows RT, and Android clients all use hardware accelerated decoding.

Exactly :-) I hope integration won't be a difficult task. Consider me in for Pull Request if you share your project on GitHub :-)
Reply

Logout Mark Read Team Forum Stats Members Help
ACCEPTED: NVIDIA GameStream2