Thread Rating:
  • 0 Vote(s) - 0 Average
3D Bluray
#1
So, now that MVC decoding support is popping up, it's actually worth at looking at supporting 3D BD. Let's gather information here.

3D BD support is 3-tiered:

1) We need an MVC decoder
- RPI2 now has it, if I read correctly
- I received an (alpha/beta?) firmware from Minix for amlogic S812 where it's working too, for an MVC stream inside MKV (mk3d?)
- Software-wise, there is nothing in ffmpeg, but there is an h264 reference implementation that supports it: ldecod (http://iphome.hhi.de/suehring/tml/)

2) We need .ssif demuxing support:
.ssif's are the 3D BD files containing 2 multiplexed m2ts streams; 1 for the "normal" h264, left-eye stream, 1 for the MVC part.
The only interesting info I could find is the actual 3D BD patent Wink : http://www.google.com/patents/EP2395770A2
As far as I understand it so far, we probably just have to merge the 2 streams to produce an MVC stream. I'm working on that part, now.

3) We need access to the .ssif files thru libbluray
I haven't had a look, yet, but I assume that shouldn't be a problem.

More info you might have welcome Wink

[EDIT] User @Vanek pointed me to an attempt at a MVC s/w decoder in ffmpeg: https://github.com/Britz/FFmpeg/commits/...264Context
Reply
#2
Currently on Pi you need to include "3D.SBS" or "3D.TAB" in filename, or identify 3D SBS/TAB in video OSD.
As the file is not really SBS/TAB it would make sense to also support "3D.MVC".

We need to decide whether to render the GUI as half-SBS, half-TAB, full-SBS or full-TAB.
full-SBS/full-TAB is pretty expensive (16M for the framebuffer, and on Pi it is triple buffered). It would also reduce the gui FPS in this mode, so currently we use "half" modes on Pi.

It would be nice to detect 3D support (through ffmpeg) before playback starts, so matching filenames is not required and file is automatically identified.
Automatically detecting if file is left-right or right-left would be useful. It appears the MVC bitstream doesn't contain that information, but I suspect it is somewhere within ISO.

We do have someone who has been investigating .ssif parsing, but work is currently stalled.
If you can work on this now, then feel free, although please keep us informed of progress.
Our plan was to handle this in libblueray.

I suspect you've discovered this, but the blu-ray folder structure contains a .ssif file which includes the complete movie, interleaved as a chunk of right-eye data, followed by a chunk of left-eye + audio + subs. There are also files like "0000.m2ts" and "0001.m2ts" which are virtual files that use hard links to ssif file. The lower number if left-eye + audio + subs and the number + 1 is right eye.

Our view was that reading the ssif file directly would likely result in more efficient file/network accesses compared to trying to merge the 0000.m2ts / 0001.m2ts files.

The chunk size was quite large, meaning that from seeking, you would have to read the whole right-eye chunk (which contains many video frames), and then after the first picture from left-eye chunk could return the first complete left+right+audio+subs chunk. Back of the envelope calculations suggested that was okay (right-eye bitrate is typically about half the left-eye's which helps). Even on 100Mb/s network seek time should be below 1 second.

E.g. ssif stream looks like:
[ R R R R R R R R ] [ L L L L L L L L ] [ R R R R R R R R ] [ L L L L L L L L ] [ R R R R R R R R ] [ L L L L L L L L ]
L : left eye + audio + subs
R: right eye

After reading:
[ R R R R R R R R ] [ L ]
You can produce [ LR ], and as each L picture is parsed you emit a new [ LR ]

We believe this new [ LR LR LR LR LR LR LR LR ] stream will be handled by ffmpeg's mt2s parser.

A minor ffmpeg patch is needed to handle MVC inside mp4 files.
Reply
#3
(2015-05-08, 14:53)popcornmix Wrote: We need to decide whether to render the GUI as half-SBS, half-TAB, full-SBS or full-TAB.
full-SBS/full-TAB is pretty expensive (16M for the framebuffer, and on Pi it is triple buffered). It would also reduce the gui FPS in this mode, so currently we use "half" modes on Pi.
HTAB is probably the most compatible. Full requires frame packing if not mistaken, and TAB is better for passive 3D TV.

(2015-05-08, 14:53)popcornmix Wrote: It would be nice to detect 3D support (through ffmpeg) before playback starts, so matching filenames is not required and file is automatically identified.
Automatically detecting if file is left-right or right-left would be useful. It appears the MVC bitstream doesn't contain that information, but I suspect it is somewhere within ISO.
I've got an embryo of an ffmpeg patch with detects MVC based on the h264 NAL encountered (14, 15, 20 -> MVC).
I would somewhat assume that the eye order is always the same for BD3D, isn't it?

(2015-05-08, 14:53)popcornmix Wrote: E.g. ssif stream looks like:
[ R R R R R R R R ] [ L L L L L L L L ] [ R R R R R R R R ] [ L L L L L L L L ] [ R R R R R R R R ] [ L L L L L L L L ]
L : left eye + audio + subs
R: right eye

After reading:
[ R R R R R R R R ] [ L ]
You can produce [ LR ], and as each L picture is parsed you emit a new [ LR ]

We believe this new [ LR LR LR LR LR LR LR LR ] stream will be handled by ffmpeg's mt2s parser.

Do you mean the sequence (I assume those are packets) is
[ R1 R2 R3 R4 R5 R6 R7 R8 ] [ L1 L2 L3 L4 L5 L6 L7 L8 ]
and that
[L1 R1] [L2 R2] ... has to go out of the demuxer?
Reply
#4
(2015-05-08, 15:11)Koying Wrote: I would somewhat assume that the eye order is always the same for BD3D, isn't it?

No, "The Day After Tomorrow" is encoded with right eye as the main stream, and left eye as the MVC addition.
I believe there are a couple of dozen similarly encoded movies.

Quote:Do you mean the sequence (I assume those are packets) is
[ R1 R2 R3 R4 R5 R6 R7 R8 ] [ L1 L2 L3 L4 L5 L6 L7 L8 ]
and that
[L1 R1] [L2 R2] ... has to go out of the demuxer?

That is my understanding. I'll see if I can get @deborah_c to confirm as she has a better understanding of this than me.
Reply
#5
Getting somewhere.

With a .ssif, if I change the stream type of the right-eye (0x20) to AVMEDIA_TYPE_DATA in ffmpeg, and set the stream "need_parsing" to AVSTREAM_PARSE_NONE, I get the packets in a plain CDVDDemuxFFmpeg (m2ts).
The proper stream can be detected by pStream->id == 0x1012, which also (if not mistaken) unequivocally make the program an MVC one.

With a random .ssif, I indeed first get 100 packets of 0x1012, then x packets of 0x1011 (left-eye) and sound, then more 0x1012.

Next step will be to buffer the 0x1012's, then build LR packets as 0x1011 are coming.
Reply
#6
Sounds plausible. I'm happy to test any patches when you think you have something that might work.
Reply
#7
Proof of concept for ssif playback is working Smile
Some cleanups and special cases to handle, still...
Reply
#8
Still? Hahah! Great work!!
f**k..... started editing without sudo | M.K.

Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#9
Some patches:
ffmpeg: https://github.com/koying/FFmpeg/commit/...5bbad55c0b
Kodi aml mvc: https://github.com/koying/xbmc/commit/94...eb795108f5
Kodi demux: https://github.com/koying/xbmc/commit/ee...ce5ac94458

This "kind of" work, but I'm not convinced it's the way to go regarding demuxing. I feel it works "by accident":
- The interleaved SSIF is not a proper M2TS, so libavformat is quite confused, especially regarding program tables
- I have no idea how we could manage seeking this way
Reply
#10
Cool. I could mount an iso file as a loop device and then navigate to the ssif file and it plays as MVC on the Pi.
Seeking kind of worked. I get a few second of 2D playback then it switches back to 3D... It does sometimes stall for a few seconds after seeking.

For proper seeking, I suspect that there are indexes into the ssif file.
I think the clpi/mpls contain indexes into the m2ts file, and as the ssif is a remux of two m2ts files, you may be able to add the offsets from the two m2ts indexes to get an offset into ssif file?

If not then I believe the normal way of seeking a non-indexed transport or program stream is binary search and parsing enough packets to get a timestamp. Keep seeking until you hit the desired timestamp. Obviously this will be less efficient than making use of indexes if that is possible.

I do wonder if the demuxer code should be in libbluray. You want to be able to play the ISO file directly, rather than extract the ssif file from the ISO.
Reply
#11
(2015-05-16, 20:10)popcornmix Wrote: Seeking kind of worked. I get a few second of 2D playback then it switches back to 3D... It does sometimes stall for a few seconds after seeking.

For proper seeking, I suspect that there are indexes into the ssif file.
I think the clpi/mpls contain indexes into the m2ts file, and as the ssif is a remux of two m2ts files, you may be able to add the offsets from the two m2ts indexes to get an offset into ssif file?

If not then I believe the normal way of seeking a non-indexed transport or program stream is binary search and parsing enough packets to get a timestamp. Keep seeking until you hit the desired timestamp. Obviously this will be less efficient than making use of indexes if that is possible.

I suspect it worked because the rpi mvc decoder is forgiving. It might not be the case for others.
What I have in mind is to intercept seek requests on 0x1011 (the main h264, the one Kodi sees), and redirect them to 0x1012 (the MVC).
This way, the MVC buffer could be refilled and a proper MVC stream sent to the decoders after seek.

(2015-05-16, 20:10)popcornmix Wrote: I do wonder if the demuxer code should be in libbluray. You want to be able to play the ISO file directly, rather than extract the ssif file from the ISO.

I didn't look into this at all, yet, but my assumption would be that we can just use libbluray to get access to the main SSIF, the same way we get access to the main M2TS, can't we?
Beside the menu things, I see libbluray as a filesystem accessor, but I can be completely wrong.
Reply
#12
(2015-05-17, 09:52)Koying Wrote: I didn't look into this at all, yet, but my assumption would be that we can just use libbluray to get access to the main SSIF, the same way we get access to the main M2TS, can't we?
Beside the menu things, I see libbluray as a filesystem accessor, but I can be completely wrong.

I've not dug in too closely, but my understanding was that libbluray currently opens the (left eye) m2ts file and needs to be changed to open the ssif file, so some changes to libbluray are required. I had imagined merging the left/right nal units to be done in libbluray, but no objections to doing it in dvdplayer's demuxer if that makes more sense to you.
Reply
#13
(2015-05-17, 14:41)popcornmix Wrote:
(2015-05-17, 09:52)Koying Wrote: I didn't look into this at all, yet, but my assumption would be that we can just use libbluray to get access to the main SSIF, the same way we get access to the main M2TS, can't we?
Beside the menu things, I see libbluray as a filesystem accessor, but I can be completely wrong.

I've not dug in too closely, but my understanding was that libbluray currently opens the (left eye) m2ts file and needs to be changed to open the ssif file, so some changes to libbluray are required. I had imagined merging the left/right nal units to be done in libbluray, but no objections to doing it in dvdplayer's demuxer if that makes more sense to you.

Well, merging the 2 m2ts in libbluray might be conceptually the best solution, but that would basically mean muxing a new stream from the left and right ones.
That would be a heck of a job, probably out-of scope of libbluray. and probably wasted resources, too (demuxing in libbluray -> muxing in libbluray -> demuxing in Kodi)

IMO, the clean way would be to have a specific Kodi demuxer which makes out a valid MVC stream out of the left-right ones by reading them in parallel. We then loose single ssif files playback, but it probably doesn't matter.
Not a simple task, either...
Reply
#14
(2015-05-17, 15:11)Koying Wrote: IMO, the clean way would be to have a specific Kodi demuxer which makes out a valid MVC stream out of the left-right ones by reading them in parallel. We then loose single ssif files playback, but it probably doesn't matter.
Not a simple task, either...

You mean alternate reading the two m2ts files? I have a concern that will be less efficient on file/network bandwidth than reading the single ssif file (due to thrashing file caches).
With MVC going up to 60Mbit/s which is close to the limit of, say SMB over a 100Mb/s ethernet then I feel this is the part of the system you want to be most efficient (even if extra cpu work is required to recombine the streams).

Now we can't be sure how much difference this makes, but it seems clear that reading linearly from a single file is going to be optimal, and reading from multiple files and/or seeking is going to make this worse.

This is the reason for the ssif file. For an optical BluRay player reading linearly is efficient. Seeking between two files is expensive, so I suspect the "two file" scheme wouldn't be feasible on an optical BluRay player.
Reply
#15
Okay. Let's continue the ssif route, then, and try to fix the seek issue.
Reply



3D Bluray00