libstagefright - Experimental hardware video decoding builds - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32) +--- Forum: Kodi Application (https://forum.kodi.tv/forumdisplay.php?fid=93) +---- Forum: Android Development (https://forum.kodi.tv/forumdisplay.php?fid=184) +---- Thread: libstagefright - Experimental hardware video decoding builds (/showthread.php?tid=152005) |
RE: libstagefright - Experimental hardware video decoding builds - Koying - 2013-02-23 (2013-02-23, 09:59)fun_ Wrote: I thought, buffer is prepared on main memory by software(your code). then, you can prepare tweaked buffer which has MODxx width/height for RK (you can calculate it from metadata), and you can pass it as a output buffer for hardware decoder.That would be, more or less, the "pure" OMX way. libstagefright is one level above and does all the buffers allocation. So, if Rockchip requires specific quirks, it should be in there... RE: libstagefright - Experimental hardware video decoding builds - Herman.Chen - 2013-02-23 (2013-02-23, 05:43)Koying Wrote: Hi Herman,The OMXCodec::Create has a flags argument, we mainly use two type: kSoftwareCodecsOnly = 8, kHardwareCodecsOnly = 16, If kHardwareCodecsOnly is used OMX component will convert video frame from yuv to RGBA and copy to GraphicBuffer which is allocated from NativeWindow. The disavantage is that the colour conversion consumes bandwidth and the conversion and display are working serially. If kSoftwareCodecsOnly is used OMX component will output a private VPU_FRAME structure and use SoftwareRender to direct display YUV frame on framebuffer by hwc. Both of these two path have a limitation on NativeWindow, the NativeWindow buffer width has to be 32 aligned height has to be 16 aligned by native_window_set_buffers_geometry. Then use crop function native_window_set_crop to display correctly. There is an error when OMX component create GraphicBuffer from NativeWindow, NativeWindow width and height are not set correctly. We fix the bug and send the libstagefright.so below. Please download and check whether can play 1080p normally. libstagefright.so RE: libstagefright - Experimental hardware video decoding builds - fun_ - 2013-02-23 (2013-02-23, 10:44)Koying Wrote:(2013-02-23, 09:59)fun_ Wrote: I thought, buffer is prepared on main memory by software(your code). then, you can prepare tweaked buffer which has MODxx width/height for RK (you can calculate it from metadata), and you can pass it as a output buffer for hardware decoder.That would be, more or less, the "pure" OMX way. I thought native window (passed as last argument of OMXCodec::Create() at https://github.com/koying/xbmc/blob/32e99c78388ae993112c18a889b9e89471a48691/xbmc/cores/dvdplayer/DVDCodecs/Video/StageFrightVideo.cpp#L886) has a buffer for output. it's in your code, not in libstagefright. but probably I misunderstood it. sorry. RE: libstagefright - Experimental hardware video decoding builds - Herman.Chen - 2013-02-23 (2013-02-23, 09:34)Koying Wrote: We don't use a "real" surface, but a SurfaceTexture, which wraps a GL texture. emm, that is normal. Because we have modified the SoftwareRenderer to adapt to VPU_FRAME structure This is the patch we made: diff --git a/media/libstagefright/include/SoftwareRenderer.h b/media/libstagefright/include/SoftwareRenderer.h index 7ab0042..994c2e1 100644 --- a/media/libstagefright/include/SoftwareRenderer.h +++ b/media/libstagefright/include/SoftwareRenderer.h @@ -21,7 +21,8 @@ #include <media/stagefright/ColorConverter.h> #include <utils/RefBase.h> #include <system/window.h> - +#include <utils/Vector.h> +#include "vpu_global.h" namespace android { struct MetaData; @@ -48,7 +49,13 @@ private: int32_t mWidth, mHeight; int32_t mCropLeft, mCropTop, mCropRight, mCropBottom; int32_t mCropWidth, mCropHeight; + Vector<VPU_FRAME*> mStructId; + uint32_t mLastbuf; + bool init_Flag; + int32_t rga_fd; + int32_t power_fd; + int32_t mHttpFlag; SoftwareRenderer(const SoftwareRenderer &); SoftwareRenderer &operator=(const SoftwareRenderer &); }; RE: libstagefright - Experimental hardware video decoding builds - fun_ - 2013-02-23 (2013-02-23, 10:51)Herman.Chen Wrote: There is an error when OMX component create GraphicBuffer from NativeWindow, NativeWindow width and height are not set correctly. We fix the bug and send the libstagefright.so below. Please download and check whether can play 1080p normally. this libstagefright.so is for both 4.0 and 4.1? does RK2918 have same hardware decoder and same bug? RE: libstagefright - Experimental hardware video decoding builds - fun_ - 2013-02-23 (2013-02-23, 11:03)Herman.Chen Wrote: emm, that is normal. Because we have modified the SoftwareRenderer to adapt to VPU_FRAME structure can you provide vpu_global.h or "typedef struct tVPU_FRAME { ... } VPU_FRAME;" part under free/open source license? I can see it in hardware/rk29/libon2/ in RK's ICS source, but license is not clear. EDIT: ah, sorry, it also exists in your libvpu.tar. but license is not clear too... ---- btw, I'm not sure this kind of device specific hack can be merged to XBMC. koying? RE: libstagefright - Experimental hardware video decoding builds - Koying - 2013-02-23 (2013-02-23, 10:51)Herman.Chen Wrote:The good news is that your updated libstagefright.so prevents the vpu from crashing(2013-02-23, 05:43)Koying Wrote: Hi Herman,The OMXCodec::Create has a flags argument, we mainly use two type: The bad news is that performance is very poor above 720p Doesn't the rk3066 have HW YUV -> RGB conversion? We indeed use kHardwareCodecsOnly to indicate that we only wants HW accelerated codecs. I don't think setting this flag should influence the way the frame is rendered, though. On other platforms, if you don't supply an ANativeWindow to OMXCodec::Create, the buffers are created inside libstagefright rather than using the ones of the surface. You then get access to the frame content via MediaBuffer::data() rather than MediaBuffer::graphicbuffer(). Those have to be in the origin color system, e.g. yuv420, which might solve the color conversion performance issue as we are using GL shaders to blit the YUV content. Does your SoftwareRenderer patch means you're returning a opaque struct rather than the actual yuv data? RE: libstagefright - Experimental hardware video decoding builds - Koying - 2013-02-23 (2013-02-23, 11:26)fun_ Wrote: btw, I'm not sure this kind of device specific hack can be merged to XBMC. koying?It would fall off the scope of having a common Android HW decoding support, for sure. It could be made into a Rockchip specific "codec", as was done for AmLogic, but that would be another story... RE: libstagefright - Experimental hardware video decoding builds - fishpepper - 2013-02-23 device: hardkernel ordoid u2: - 1.7GHz Exynos4412 Prime Cortex-A9 Quad-core processor with PoP (Package on Package) 2Gbyte LPDDR2 880Mega Data Rate - Mali-400 Quad Core 440MHz running cyanogenmod 10.1 (android 4.2) http://www.xbmclogs.com/show.php?id=1132 issues: stuttering playback of HD files (for example H264 - MPGE-4 (part 10) (H264), 1280x720, 50fps, planar 4:2:0 YUV, Aufo: MPEG 1/2/3 192kbit/s) looks very strange. could be described as playing some frames slow, than some frames faster, then frames slower again. if there is constant motion in a scene (e.g. camera moves) it looks like the motions is alternating between slow and fast. relly strange regards, simon RE: libstagefright - Experimental hardware video decoding builds - CruNcher - 2013-02-23 i pushed the libstagefright.so onto this tablet no boot possible anymore (though older sdk 3 firmware from november) will try it with a december firmware. Yep December firmware works Finally no Crash anymore on the Zelda Medley even with both cores active , not sure about the Performance it stucks a little here and there But at least it's stable now and decent playback is now possible with Youtube 1080p complexity, will test more The second run also looks better with less stucks in the GL render pipeline for this tablet firmware Yep 4 runs now no crash And yes also Mx Players H/W+ Mode benefits from this bugfix we can just hope that it finds it way to tablets and other RK3066 devices fast after seeking it takes some seconds to get stable again, though not with Mx Players H/W+ Mode seeking is very performant (better frame buffering) here (instant). I pushed it hard seeking forth and back like crazy no crash, though it's clear for Performance and whenever possible use and prefer the Native HW mode (lowest overhead) Especially when trying to save battery on the go don't use XBMC or MX Player H/W+ Mode it's anyway crazy todo so in that scenario (to much overhead) So this fixes the most annoying issues for RK3066 + 1080p crashes + modxx render issues perfect so far in terms of outside stability, performance is another issue RE: libstagefright - Experimental hardware video decoding builds - Herman.Chen - 2013-02-23 (2013-02-23, 11:09)fun_ Wrote:(2013-02-23, 10:51)Herman.Chen Wrote: There is an error when OMX component create GraphicBuffer from NativeWindow, NativeWindow width and height are not set correctly. We fix the bug and send the libstagefright.so below. Please download and check whether can play 1080p normally. This libstagefright.so is for 4.1, not for 4.0. RK2918 uses different GPU, do not have the same bug. This bug is caused by GPU limit not hardware decoder. (2013-02-23, 11:26)fun_ Wrote: can you provide vpu_global.h or "typedef struct tVPU_FRAME { ... } VPU_FRAME;" part under free/open source license? VPU_FRAME part structure is writen by Rockchip. It has no specified license. We decide to open it, but we do not add license on it so far. VPU_FRAME structure is a private path, and it is a hack for andrdoid framework and our hardware combination. In my opinion as a SW engineer it is not very good for a general software. I know this private path is quite ugly but it works. (2013-02-23, 12:43)Koying Wrote: The good news is that your updated libstagefright.so prevents the vpu from crashing Glad to hear the lib can help to resolve your problem. RK3066 has rga to do YUV -> RGB conversion. It has a very good performance and easy interface. Both kHardwareCodecsOnly and kSoftwareCodecsOnly use HW to decode video, but the difference is in the output. One is standard MediaBuffer with GraphicBuffer (in 4.1), The other is in for private path to direct render to FB which can make 1080p more smooth. MediaBuffer::data() is for 4.0 and MediaBuffer::graphicbuffer() first appear in 4.1 for more direct connect between surface and decoder. Yes, you are right. SoftwareRenderer patch indeed skip the color conversion and direct use hwc to composite yuv data to FB. The opaque struct contains the phycial address of video frame and some other information to make it possible to do the direct blending.Finally it comes to a zero copy. The early version of Samsung code use similiar strategy. RE: libstagefright - Experimental hardware video decoding builds - fun_ - 2013-02-23 (2013-02-23, 16:12)Herman.Chen Wrote:(2013-02-23, 11:09)fun_ Wrote: this libstagefright.so is for both 4.0 and 4.1? I see. thank you! (2013-02-23, 16:12)Herman.Chen Wrote: VPU_FRAME part structure is writen by Rockchip. It has no specified license. We decide to open it, but we do not add license on it so far. "decide to open" it's really great. it will help open source community like XBMC. people want to get Rockchip devices to use XBMC then, about license, I think it should be specified. some open source project may not accept any code if it's unclear. I recommend Apache license as like as Android http://source.android.com/source/licenses.html RE: libstagefright - Experimental hardware video decoding builds - Koying - 2013-02-23 Let's try to recap... (2013-02-23, 16:12)Herman.Chen Wrote: RK3066 has rga to do YUV -> RGB conversion. It has a very good performance and easy interface.I'm a bit lost. Then why did you say color conversion YUV420 -> RGB induced reduced performance? Does that happen in CPU memory? (2013-02-23, 16:12)Herman.Chen Wrote: Both kHardwareCodecsOnly and kSoftwareCodecsOnly use HW to decode video, but the difference is in the output. One is standard MediaBuffer with GraphicBuffer (in 4.1), The other is in for private path to direct render to FB which can make 1080p more smooth.So, it is not possible to use private path and kHardwareCodecsOnly, right? Or can we still get a tVPU_FRAME via MediaBuffer::data() if we don't specify a native window? (2013-02-23, 16:12)Herman.Chen Wrote: Yes, you are right. SoftwareRenderer patch indeed skip the color conversion and direct use hwc to composite yuv data to FB. The opaque struct contains the phycial address of video frame and some other information to make it possible to do the direct blending.Finally it comes to a zero copy. The early version of Samsung code use similiar strategy.Would you mind publishing the diff on SoftwareRenderer.cpp, too? RE: libstagefright - Experimental hardware video decoding builds - fun_ - 2013-02-23 (2013-02-23, 16:12)Herman.Chen Wrote: RK3066 has rga to do YUV -> RGB conversion. It has a very good performance and easy interface. (2013-02-23, 16:12)Herman.Chen Wrote: Yes, you are right. SoftwareRenderer patch indeed skip the color conversion and direct use hwc to composite yuv data to FB. The opaque struct contains the phycial address of video frame and some other information to make it possible to do the direct blending.Finally it comes to a zero copy. The early version of Samsung code use similiar strategy. is decoded frame always converted and composed to RGB framebuffer in zero copy path? I think most stick/STB products with RK3066 use 1280x720 framebuffer. then, any 1080p videos are downscaled to 720p for framebuffer, and lastly upscaled to 1080p for 1080p TV? or 1920x1080 YUV framebuffer can be overlayed on top of 1280x720 RGB(UI) framebuffer? (this question will be off-topic, but I want to know how "normal video player apps" works on RK3066 because I want to know difference between other apps and XBMC...) RE: libstagefright - Experimental hardware video decoding builds - CruNcher - 2013-02-23 Hmm there is a issue Chen a AVC Blu-Ray stream .m2ts shows strange frame jumping (reorder issue) in the beginning for around 35 seconds either with H/W+ or XBMC no frame jumps with the kSoftwareCodecsOnly path (RKPlayer,MX Player Hardware Mode, Archos Video Player). Maybe it's a parser issue i guess both player use in these modes the libav parser not yours. Though it happens only on the first load seeking back doesn't show these frame jumps anymore. Mpeg-2 Blu-Ray stream .m2ts still fallsback to ff-mpeg2video, is the vpu mpeg-2 decoder not accessible via libstagefright as neither mx player can use it in H/W+ mode ? Also .m2ts and .mkv VC-1 fallback to ff-vc1 so currently only H.264 seems to work accelerated for RK3066, which at least grants most web based plugin support by now tested and work now Vimeo 1080p Youtube 1080p Apple Quicktime Trailer 1080p Lot of Apple Quicktime 1080p content tough wont work sufficient yet, especially Game content Promo Videos that use very high framerates so most H.264 based Web Platforms should work now Mpeg-4 Part 2 ASP 1080p 23.976 stf-mpeg4 also works though frame drop issues |