[Proposal] Video/Audio call API + tox binding

  Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Mykola Offline
Junior Member
Posts: 3
Joined: Mar 2015
Reputation: 0
Post: #1
Hello guys, I would like to participate you this summer, and here’s my idea:

Video/Audio call API + tox binding

Name: Mykola Dolhyi
Email: [email protected]

Summary:
Most smart TVs now have integrated Skype client and it’s handy, especially if you are not talking face to face but with your kids/family/friends. I suppose it would be quite useful to have an option to make/receive calls from media center. For now if someone want’s to make a call from his HTPC he needs to toggle from kodi to skype or utox and backwards and it’s definitely not OK. The idea is to add a new API to bind different VoIP/IM messengers and a new dedicated tab, which will hold all your contact and messages from different sources and allow you to make a call, or receive one. Text messages is not too handy for this kind of device, but as many people use wireless keyboards to control their media centers it should be ok either. As you know Skype is a piece of proprietary software without public API (SkypeKit is closed), moreover you can’t rely on it to keep your conversation private, so it’s not an option. But what we have is a shiny-new secure p2p protocol called Tox, and I suppose it’ll fit just fine.

How will I achieve this:
The project will consist of three parts:
First: develop some kind of messaging API (or integrate some existing one), will provide unified access to contacts and accounts, chats, video and audio calls.
Second: develop a GUI part for Kodi, this part will show list of contacts by accounts, view chat messages and make video calls. When a call is made or received video stream from a local web-cam as well as audio stream are submitted to API for future transition to client, incoming streams are submitted to kodi for playback.
Third: create a binding of toxcore and toxav to this API

What will the project focus on:
The main focus will be set on a well-built API, as a most important part, which is not desirable to change in future.

Benefits:
  • Kodi becomes a more self-containing platform;
  • Developers get a unified API to create their own bindings to different IMs;
  • Widespread tox among different kinds of users;

Dependencies: libopus and libvpx (are already added to kodi), toxcore, toxav, toxdns, NaCL or libsodium

Requirements: C, C++, web camera, microphone

Possible mentors:

So, what do you guys think about all of this?
--
Background:
My name is Mykola Dolhyi, I’m from Ukraine. Am a third year student of Kyiv National University of Taras Shevchenko on a Mathematics department. Besides of some mathematics background I possess some coding skills in C++, C#, Java, Python, several functional languages (ML,Racket,Haskell), have a little experience in parallel computing with CUDA and OpenCL, databases (MS SQL, MySQL, MongoDB), GUI frameworks (Qt Quick,WPF,GTK+,Swing,WinApi,WinForms). I have some experience in game development with OpenGL (both desktop and mobile), used to play a bit with gstreamer (especially transcoding and video streaming parts).
(This post was last modified: 2015-03-19 23:48 by Mykola.)
find quote
garbear Offline
Team-Kodi Developer
Posts: 1,864
Joined: Dec 2010
Reputation: 116
Location: city of angels
Post: #2
I've thought of possibly hacking a skype add-on into the Game API. the API supports audio, video, webcam (no mic yet but wisler might cover this in his GSoC work) and input so the pieces are all there

in the interest of time, and because we know students like to be as lazy as possible Wink can you think of any elements of RetroPlayer that you can repurpose for your project?
find quote
RockerC Offline
Posting Freak
Posts: 1,516
Joined: May 2011
Reputation: 31
Post: #3
Recommend to begin by checking out the work that Amitn and Shanee did previously, it also had an API concept that was suppose to support multiple VoIP/SIP-clients:

http://forum.kodi.tv/showthread.php?tid=123620

https://github.com/amitn/xbmc

Might still give you some good ideas even if you can't reach anyone of them. Anyway, wouldn't hurt to try to reach out to see if you can build on their code?


Probably good idea to start with an free and open source VoIP/SIP frameworks as a reference client for Kodi before looking at closed protocols like Skype.

OpenSIPS or Linphone could maybe make a good reference client as it is popular and already have cross-platform support, but there are many more:

http://www.opensips.org
http://www.linphone.org
http://www.voip-info.org/wiki/view/Open+...P+Software
http://en.wikipedia.org/wiki/List_of_SIP_software
(This post was last modified: 2015-03-20 15:25 by RockerC.)
find quote
Mykola Offline
Junior Member
Posts: 3
Joined: Mar 2015
Reputation: 0
Post: #4
(2015-03-20 02:33)garbear Wrote:  I've thought of possibly hacking a skype add-on into the Game API.

Hacking a Skype addon seems impossible at the moment, as Microsoft discontinued both Desktop and SkypeKit api's at the beginning of the last year. Google Talk after rebrending to Hangouts got rid of XMPP protocol and now uses some proprietary stuff, whose official api provides only bells and whistles to their client app.

(2015-03-20 02:33)garbear Wrote:  The API supports audio, video, webcam (no mic yet but wisler might cover this in his GSoC work) and input so the pieces are all there

So you suggest me to make it on top of the RetroPlayer? Maybe just like some game client relying on game api? It'll get input, and supply images for the video stream, by the way does it support video stream playback?
find quote
Mykola Offline
Junior Member
Posts: 3
Joined: Mar 2015
Reputation: 0
Post: #5
(2015-03-20 15:05)RockerC Wrote:  Recommend to begin by checking out the work that Amitn and Shanee did previously, it also had an API concept that was suppose to support multiple VoIP/SIP-clients:

http://forum.kodi.tv/showthread.php?tid=123620
https://github.com/amitn/xbmc

I've tried contacting amitn, but he hadn't any public activity for the past few monthes. Moreover, his repository on github doesn't contain any relevant code and the build won't give much as, you know, it won't work.
(This post was last modified: 2015-03-23 20:43 by Mykola.)
find quote
garbear Offline
Team-Kodi Developer
Posts: 1,864
Joined: Dec 2010
Reputation: 116
Location: city of angels
Post: #6
(2015-03-23 19:51)Mykola Wrote:  Hacking a Skype addon seems impossible at the moment, as Microsoft discontinued both Desktop and SkypeKit api's at the beginning of the last year. Google Talk after rebrending to Hangouts got rid of XMPP protocol and now uses some proprietary stuff, whose official api provides only bells and whistles to their client app.

way to rain on my parade Wink

(2015-03-20 02:33)garbear Wrote:  So you suggest me to make it on top of the RetroPlayer? Maybe just like some game client relying on game api? It'll get input, and supply images for the video stream, by the way does it support video stream playback?

it will after GameStream support is added
find quote
bam80 Offline
Senior Member
Posts: 105
Joined: Dec 2011
Reputation: 1
Location: Russia
Post: #7
What happened with GameStream support?
Mykola, are you still going to implement your ideas?
find quote