• 1(current)
  • 2
  • 3
  • 4
  • 5
  • 8
ACCEPTED: On-the-Fly Transcoding
#1
Hello everyone. I'm interested in working on Kodi at this year's GSoC. Going through the project proposals I noticed that no one opted for implementing the transcoding functionality so far. Still I would find it important to have such a feature when it comes to streaming video/audio to a mobile device that is potentially missing the required codec/capabilities. Therefore an option to choose the format to receive would be very helpful.

My proposal is the following:

Transcoding Support

Name: Mark Muth
forum/e-mail: [email protected]

Summary:
Support for on-the-fly transcoding of media on the Kodi server makes it possible to stream content to low-profile devices such as embedded and mobile systems.

How will I achieve this:
Using existing software such as ffmpeg, which is already been used within Kodi, the server can be extended to use available video and audio encoders to perform transcoding.

What will the project focus on:
There should be layer to support transcoding prior to streaming the content. The requesting client should have the possibility to request the content as-is or in any supported format.

Benefits:
Users would be able to enjoy their content on many more devices that might currently lack support for the contents format.

Goals:
1.1 A first step would be to correctly integrate a software package with transcoding capabilities and to make use of it within the Kodi server.
1.2 Also an additional layer for transcoding needs to be inserted before the actual streaming takes place. A first draft would forward the content as-is (without performing actual transcoding)
1.3 The API has to be extended for requesting a certain file format/codec before actual streaming of the content begins.
2. Now the previous steps would be brought together to perform real transcoding.

What does it touch in Kodi:
Not quite sure how Kodi is structured yet. But obviously more functionality of ffmpeg or possibly other transcoding software needs to be aggregated. And the streaming API needs to be enhanced.

Requirements:
I think good programming skills in C++ are mandatory.
Knowledge of Kodi and ffmpeg preferable.

Any feedback on this is welcomed. To me this sound like a fair piece of work for GSoC. Correct me if I'm wrong.

Cheers,
Mark

Status: Student Passed
Code: https://github.com/MarkMuth/xbmc/commits/gsoc2015-2
Reply
#2
This is definitely something a lot of people want. Could you talk a bit about your dev background? The project is definitely not a beginner project.
Reply
#3
Assuming you have the skills this has a high chance of being accepted, mainly because we all really want it Smile

You may want to look at how NAS devices are doing it:

http://docs.qnap.com/nas/4.1/Home/en/ind...gement.htm
http://www.digitalreviews.net/reviews/st...iewed.html
Reply
#4
Cool! Though how about fleshing out a little by also adding support for DLNA Media Format Profiles to the UPnP server and client side via MIME Type profiles?

See these two previous proposal here http://forum.kodi.tv/showthread.php?tid=162225 and here http://forum.kodi.tv/showthread.php?tid=127949

And checkout related pull request about adding initial support for custom UPnP MIME Types for practical implementation https://github.com/xbmc/xbmc/pull/6179

MIME Types and Media Format Profile to support DLNA specifications have also been discussed before here http://forum.kodi.tv/showthread.php?tid=187231

You probably already thought of this, I was just highlight that if something like this is not defined in the project scope then to where are you going to transcode?
Reply
#5
(2015-03-17, 12:00)RockerC Wrote: Cool! Though how about fleshing out a little by also adding support for DLNA Media Format Profiles to the UPnP server and client side via MIME Type profiles?

See these two previous proposal here http://forum.kodi.tv/showthread.php?tid=162225 and here http://forum.kodi.tv/showthread.php?tid=127949

And checkout related pull request about adding initial support for custom UPnP MIME Types for practical implementation https://github.com/xbmc/xbmc/pull/6179

MIME Types and Media Format Profile to support DLNA specifications have also been discussed before here http://forum.kodi.tv/showthread.php?tid=187231

You probably already thought of this, I was just highlight that if something like this is not defined in the project scope then to where are you going to transcode?

Transcoding could also be done through our normal webserver which tools like the Chorus webinterface use to play video in the browser. IMO that would be the easier initial approach than a fully working UPnP profile system.
I fully agree that transcoding is probably most useful for our UPnP stuff but it's also much more difficult to integrate because we use a third party library which is not that easy to modify. It is much easier to add some simple HTTP GET options to one of the URLs in our webserver and use that for development and testing than first having to define and agree upon a UPnP profile structure, hack that into the UPnP Platinum SDK and integrate it into Kodi.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#6
(2015-03-17, 13:44)Montellese Wrote: Transcoding could also be done through our normal webserver which tools like the Chorus webinterface use to play video in the browser. IMO that would be the easier initial approach than a fully working UPnP profile system.
You might be right, and that could also lead to cool place-shifting functionality as suggested here http://forum.kodi.tv/showthread.php?tid=56561

That could enable not only web interfaces but also Android and iOS remote apps such as Yatse to also act media player clients to Kodi as well.
Reply
#7
Thank's for the feedback. I used the time to look through the previous threads about this topic.

Now I have the following remarks:

@RockerC: Indeed I saw it more like it was suggested by Montellese and UPnP was not my first priority.

First step would be to implement transcoding and test it via the webserver.
I thought about choosing different containers and video/audio codecs and quality attributes such as resolution through different HTTP requests.

Using this kind of API at least mobile phones could stream media properly. And this could be used in many of the existing remote apps to stream media content directly into the app.

Once this works fine one could think about hardware based transcoding and UPnP support. However I guess all of this could be a little overkill for one GSoC project.

@natethomas: I gathered programming skills in C++ at projects at the university and during a six month internship at a company in an embedded software development department where I created a rather big piece of software that helped other devs to debug their applications and devices over the network. I also did a part-time job where I created modules for OpenFOAM. Although these are very widespread topics I always got new insights into C++ and related software and to multiple development platforms such as Eclipse, Visual Studio and Qt Creator on Linux and/or Windows.

Hope this helps to clarify my proposal.
Reply
#8
(2015-03-17, 19:43)RockerC Wrote:
(2015-03-17, 13:44)Montellese Wrote: Transcoding could also be done through our normal webserver which tools like the Chorus webinterface use to play video in the browser. IMO that would be the easier initial approach than a fully working UPnP profile system.
You might be right, and that could also lead to cool place-shifting functionality as suggested here http://forum.kodi.tv/showthread.php?tid=56561

That could enable not only web interfaces but also Android and iOS remote apps such as Yatse to also act media player clients to Kodi as well.

Yup. I think this on its own would make a great project. Just create a transcoding system which provides APIs (through JSONRPC or WebServer), which is cleanly encapsulated enough so it possibly could be reused by python addons etc and core.
Could consider support for a client to propose multiple stuff it supports and the transcode system picks the least harsh to produce. For example sometimes only a demuxing is needed (For example uPnP were we could demux in subtitles) in a transcode and sometimes a full re coding is needed, were the latter takes much much more power.

I think aiming for the above have a far higher successrate and would give the community lots to work on, and it would be the final puzzle needed for headless kodi. By experience its not good to bite more off than you can chew, its easy to envision a very big project and its very sad not to have time to finish it Smile

If there is time there are a number of possible next steps.
1) A number of NAS supports hardware transcoding, for example Synology DS214play, adding support for that would be very useful
2) Adding it to UPnP, and possibly doing it with profiles as suggested.
If you have problems please read this before posting

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.

Image

"Well Im gonna download the code and look at it a bit but I'm certainly not a really good C/C++ programer but I'd help as much as I can, I mostly write in C#."
Reply
#9
this could finally be another good use case for our vfs pipe which could be used as the "file" for the transcoded data
AppleTV4/iPhone/iPod/iPad: HowTo find debug logs and everything else which the devs like so much: click here
HowTo setup NFS for Kodi: NFS (wiki)
HowTo configure avahi (zeroconf): Avahi_Zeroconf (wiki)
READ THE IOS FAQ!: iOS FAQ (wiki)
Reply
#10
(2015-03-17, 22:45)mANm Wrote: @RockerC: Indeed I saw it more like it was suggested by Montellese and UPnP was not my first priority.

First step would be to implement transcoding and test it via the webserver.
I thought about choosing different containers and video/audio codecs and quality attributes such as resolution through different HTTP requests.

Using this kind of API at least mobile phones could stream media properly. And this could be used in many of the existing remote apps to stream media content directly into the app.
Cool, but still suggest that you implement MIME Type profiles for device interoperability, as the server-side need to know what formats/codecs/resolutions is supported on the client-side.

As topfs2 noted, sometimes only remuxing is needed, like for example if a video is encoded in H.264 but the stream is in an MKV container then demux and mux to MP4 might be enough.

Even if your transcoding system will initially be limited to only providing APIs for JSON RPC or Web Server I think that you will still want the the client (player) begin communication with handshake where it state exactly what formats/codecs/resolutions it is capable to playback. And to achieve this I suggest you use DLNA's Media Format Profiles standard as a basis, and as a proof-of-concept maybe copy the XML profile files for clients from Plex Media Server as suggested here https://github.com/xbmc/xbmc/pull/6179 and see https://forums.plex.tv/index.php/topic/4...a-devices/ or for a specific example http://nine.plugins.plexapp.com/system/b...ofiles.xml.

The basic idea with Media Format Profiles is that the client (player) begin its connection with the server (transcoding system) by sending a manifest with a capability specification lists. And if you implement this using the standard DLNA Media Format Profiles then it will also be easier to later reuse your transcoding system for the UPnP server in Kodi.

That said, no matter which solution you choose I think that device interoperability and selection of formats to transcode to should be done fully automatic from the end-user perspective.
Reply
#11
IMO the first step is to get transcoding working for as many cases as possible. If that is working adding a layer on top (or rather in front) of that which uses an XML file to determine what to transcode to doesn't really have anything to do with the transcoding itself. It's "just" a decision making process based on a list of facts.

We have to be careful not to try to do too many things at the beginning and end up not finishing any of it. Better start with "pure" transcoding (including simple remuxing et al.) and then integrate it into specific use cases like UPnP/DLNA streaming etc and keep going until the time is up. If the time is up after transcoding has been finished that's still better than having started with the logic to parse an XML file to determine what to transcode to and then start working on the transcoding and not being able to finish it before the time is up.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#12
(2015-03-18, 18:45)RockerC Wrote: That said, no matter which solution you choose I think that device interoperability and selection of formats to transcode to should be done fully automatic from the end-user perspective.

I totally agree with you. This should be the final goal and also hardware transcoding should not be forgotten. But initially (software) transcoding itself and access to it should be realized and support for UPnP, DLNA, XML-client profiles and hardware acceleration should definitively come afterwards but kept in mind during the design and implementation. Since I find it hard to estimate how much effort each of those parts will take I'll probably propose only the first two things (transcoding and browser/JSON-RPC access) to GSoC and mention the others as optional, if that's possible.

Please tell me if you wouldn't agree.

Thanks for your feedback,
Mark
Reply
#13
That should be fine.

There was a GSoC project about transcoding (that wasn't done) before (2013 IIRC) so maybe you can take a look there as well.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#14
Yep. In a first instance, I'd suggest to agree on a single, hardcoded, profile (e.g h264 + aac) to maximize the chances to reach the transcoding goal for most of potential clients.

We could then build on that, even after GSOC, to add the niceties.
Reply
#15
Actually from lot's of user feedback the transcoding needs are 33%/33%/33% :

- Video to local devices or limited devices so h264 / mp3 (or aac) (aac is not supported correctly by some old devices)
- Video to WAN : So main need is to limit the bw so lower the video profile and mp3 for audio (mostly watch tv shows on the go)
- Audio to WAN so flac / high quality mp3 to lower quality mp3 for all low bandwith users.

This should cover 90% of actual needs and be better for the future. (And for WAN need UPNP is not good)
Reply
  • 1(current)
  • 2
  • 3
  • 4
  • 5
  • 8

Logout Mark Read Team Forum Stats Members Help
ACCEPTED: On-the-Fly Transcoding1