[peak3d]

Hi,

I'm currently working on DASH stream import (QT-MP4 (MOV)) into kodi (video section).

1.) Is it possible to pass video and audio sreams as separate streams into kodi?
I know that PVR Demuxer can handle this but it is restricted to TV section (this is my information)

For sure I could mux the streams together, but in case the stream bitrate / reolution changes this is a lot overhead:
- the backend I'm using has to rebuild headers for both streams
- kodi has to restart both streams also only one stream (audio or video) has changed

For me the optimal solution would be the possibility to push some streams to codi known that DTS / PTS matches and let odi do the rest :-)

Looking forward about some feedback!

[FernetMenta]

What is your definition of a stream? PVR demuxers handle mpeg-ts which can have a large number of different stream. This is not much different to ffmpeg demuxer.

[peak3d]

Thanx FernetMenta for your response,

I had short contacht with opdenkamp and understood it that the PVR Demuxer has multiple instances each fed with a single stream.
In sum the addon passes multiple streams to kodi. This is against what we can do now: send one "file" or one URL. wich forms a stream with multiple channels (tracks in my MOV stream)

Following situation makes me some headache: I have an video mp4 stream and an audio mp4 stream, segmented.
Unfortunately the video has a slightly different sampling rate as the video, I have 2 more segments (=~4secs) for audio then for video.
For sure I can handle this to be well weighted in the muxer, bit it is all work and done for the fact that the streams will be separeted in kodi.

I would love to call kodi video player with 2 (or more) URL's, one for video and one for audio (and maybe some more (text??)

Is there already a solution?

[FernetMenta]

Well, the streams handled by PVR demuxer are actually the streams of a program in mpegts. VideoPlayer get an more abstract type of stream from a demuxer and does not really care where those come from.

I still don't completely understand what you intend to do. Maybe you are looking for multiple demuxers being active at the same time. This is something Ace is working on: https://github.com/ace20022/xbmc/tree/multidemux

[peak3d]

the situation is quite simple:

From a streaming platform I receive information about the location of single track streams: a few for video (differ in resolutions), a few for audio (differ in languages / sampling rates / codec).
What I have are a bunch of single streams (one URL for each) from them I know that they fit together (in meaning of PTS).

I want the user let choose the video quality / audio language and want to pass these two matching URL's to Kodi.
Kodi should keep track over PTS matching in the same manner as it does it currently with the demuxer (audio leads, video will be adjusted).

I hope this makes it a little bit clearer - I'll look in the ticket you mentioned.....

The MultiStream Demuxer from link above seems to be exactly the thing what I'm looking for.
Any idea if / when this will be included into the mainline tree?

[FernetMenta]

Get in contact with Ace. The reason why he started this was to integrate external audio streams. I think that is pretty much the same as you are trying to achieve. Feel free to open an new stream in this subsection of the forum.

[Ace]

spiff and I have done a poc for dash: https://github.com/ace20022/xbmc/pull/4
It's the next thing on my list to pick up again. If I understand you correctly you have an addon and that should pass multiple urls to kodi, right?
So no c++ help from your side?

[peak3d]

I'm pure C / C++, frontend is done by others who make it better than me (and this is not very difficult)

I'm currently on an backend (workflow like TVHeadend) for dash streams, including adaptive streaming.
The source will be released in the next couple of days.


As far as I have seen your multistream solution looks clear and straight forward.
I'm currently fighting with getting resolution changes smooth in kodi.
My hope is that a multistream solution would solve some aspects wich are hard to handle in CDVDDemuxerFFMPeg (especially the different dash segment sizes)

Give me some days and I'll take a deeper look on the links you provided......

[peak3d]

b.t.w.: libdash is the hell on earth, 1.) they claim that their solution is fast - its a lie.
it takes 10 seconds (on my i7) to load and parse a 3MB mpd file. They don't support gzipped mpd. Its one line in curl.....

Using expat as a parser and enable curl gzip option does the same in 0.5 secs.

[Ace]

Don't expect too much, these are only quick pocs. There are other problems like unique stream ids, I have another branch for that which isn't fully functional yet.

[peak3d]

I did this task directly in ffmpeg because it was easier - remove old AVStreams and replace them with new ones if there is a codec change - 4 lines code.

But one thing remains: Kodi canot be smooth because you will ever see the time gap if the decoder reinitializes.
Also this are one some 10th milliseconds, it is visible.

There is only one real solution for the future: be able to open at least 2 video decoders so the 2nd one is ready if the packet data of the stream after the change will be processed.
I'm not sure if this is something wich hardware is able to....

Maybe one can use ffmpeg for the time of a resolution / codec switch and turn to h/w if the decoder is up and running.

2nd solution would be implementing a frame buffer for the time of 1/2 second or whatever so decoded image data would be buffered - I think this is not the best choice for small embedded boards like rpi.

But - it's running and if resolution changes not so often it doesn't matter.

Question is only if kodi acepts my changes in ffmpeg (and FFMPEGDemuxer)

[FernetMenta]

Not sure but you have have doing something wrong. Kodi can change decoder on the fly without any noticeable gap. This happens a lot on live tv where codec parameters of audio or video stream can change.

[peak3d]

Smooth means for me that there is not a single frame is lost - because you'll see it:

1.) There seems to be a bug in calculating the PTS in CheckStreamChange (fixing this reduces the gap from 0.5 secs to <100 ms)
2.) Video decoder is set up at time the packet with the new resolution will be processed, This costs some 10-th ms.
3.) HW-Scaler is completely reinitialized at the time of the switch -> this cost currently the most time (~30 ms)

These are ~2 missing frames in my example, but they are very remarkable.

I have solved 1 and 2 (pull request comes soon), but 3 (start up an HW scaler at time the new packet is read - this is some seconds before it is displayed) is some more work (and I'm not familiar with this).
Multiple Decoders can already be allocated / initialized side by side, HW scalers not.

As written above: I have some minor changes in ffmpeg wich allows multiple MOV Atoms inside one stream (and fills the correct duration in case the stream is not seekable).
The change is only a few lines - but where can I place the diff for this to be reviewed from your side?

Code:

$ diff ffmpeg/libavformat/mov.c ffmpeg_vc/libavformat/mov.c
3992c3992,3999
<             }
---
>                       }
>                       else if (a.type == MKTAG('m', 'o', 'o', 'v') && c->found_moov)
>                       {
>                               while (c->fc->nb_streams > 0)
>                                       ff_free_stream(c->fc, c->fc->streams[c->fc->nb_streams - 1]);
>                               c->found_moov = 0;
>                               c->found_mdat = 0;
>                       }
4544c4551,4553
<     }
---
>               if (mov->duration && mov->time_scale && st->time_base.den)
>                       st->duration = (mov->duration * mov->time_scale / st->time_base.den);
>       }

Thanks for the discussion!

Edit: In real life the HW scaler issue will not be such important as I try to switch to lower quality inside one resolution block (In my example I have for example 3 qualities for 720p)

[FernetMenta]

I am looking forward to seeing your PR. In theory the behaviour is as follows:

- demuxer detects a change
- stream player queues a message for the codec change
- at this time a couple of frames are queued in the render buffers
- player opens codec, old codec stays alive (on platforms that support this)
- new codec starts its job and delivers pictures to render.
- after display of last frame of old codec, ref count goes to zero and old decoder is destructed.

[peak3d]

https://github.com/xbmc/xbmc/pull/8830

thanks for your explanation! The concept you described above is a great base for keeping gaps as small as possible.
There was only one thing I had to add: decode the first packet whaen the new decoder is set up because ffmpeg determines with the first packet how it will be processed.

On my system its directX and when you decode the first packet ffmpeg sets up the DXVA decoder already at the time the changed packet arrives.

The same would be really good with h/w scaler, but this is a lot of work and I think in real life there are not too many resolution changes (for my example)

Also I think there is one big difference between live-TV and my case: Stream changes in LiveTV will happen if something major changes: a new episode or a new film or whatever.
It is not too important if there will be some video frames lost.
Im my case its a continuous video, the change happens anywhen and therefore already some frames will disdurb the perfection.

I'll provide the stream I'm testing with but therefore you need the ffmpeg patch applied (a new one because I made some small modifications)
If you tell me where to place it (attach it to the PR??) I'll do.