[BUG] Slow Chinese subtitle rendering
#1
Currently working on fixing the chinese subtitle stuttering problem. Since this is my first time working on the source code, I figured it might be a good idea to start a thread, just to make sure I'm not making bad assumptions/breaking stuff/engaging in general retardedness.

I've done a quick profiling of the code with instruments, with subtitles displaying -
Image

The spikes in the graph occur whenever subtitles are rendered, spiking the CPU at 100%, from a baseline of about 20%( this is with SD content ). All of the extra CPU usage is coming solely from rendering the subtitles.

Comparing it against just rendering english subtitles, I can see that they use more or less the same code path, with the same functions eating up most of the time. However, with english, even though I'm seeing spikes here and there, it is nowhere near as bad as with chinese.

From what I'm seeing in the profiler, it looks like there is a lot of GL related memcpying going on, which is taking a large percentage of the time. So I'll think I will investigate there first.
Reply
#2
Reason is simple: Chinese == lots + lots of characters, thus you need either a huge-arse texture to keep 'em all cached, or you're constantly clearing + recaching them. You'll see things like "out of memory caching character <foo>" in the debug log - see the code in CacheCharacter.

I suggest move to a vector of textures. Patch most welcome Smile

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#3
Yes, I'm stepping through the code right now. Watching a show with typical chinese subtitles, about 15mins, it's at about 1000 cached characters. Which means it's making at obnoxiously large texture( bigger than a 1080p screen if it is making an atlas ), which, even if it were allowed, would be a very cache unfriendly cache Smile

Since I'm not John Carmack, doing a font MEGATEXTURE is out of the question.

I think that doing a true type font shader is the correct solution.

Advantages -
Subtitles are still beautiful, in all languages.
Memcpy thrashing goes away.
Massive performance improvement.

Disadvantages -
Some chipsets, like the PowerVR chipsets in iDevices, absolutely suck at clipping in pixel shaders. Which means that shaders like the one described at research.microsoft.com/en-us/um/people/cloop/loopblinn05.pdf are going to perform very poorly on that kind of hardware. Or netbooks that use that hardware.

But that's probably the way to go. In the short term, I'm going to keep stepping through code, and see if there is some kind of quick band-aid fix that can be done to make the Chinese subtitle rendering not quite so painful. I've noticed that chinese DVDs have absolutley horrid quality subtitles( and subtitles display without stuttering on chinese DVDs), so perhaps there is an easy trade-off of quality for memory that can be done somewhere.
Reply
#4
Quick fix is just using a bunch of textures. Not sure how many you'll actually need. Only problem is you may end up thrashing through textures all the time.

We're seriously considering switching from a letter cache to a string cache (i.e. GUIFontTTF would just use freetype to render the entire string to a texture, and GUITextLayout would hold the texture) but obviously whatever works best is best.

As you say, the main disadvantage of switching purely to TTF rendering with a shader is the embedded platforms, which my guess are going to become increasingly important in the next few minutes :p

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#5
Think I'll just skip doing a quick fix in favor of vectorizing the fonts. Having done some research, it looks like that is the way to go.

I'm also going to skip doing the TTF shader at first. I've set up a test app, and am focusing on extracting the outline from the font. Which will then be tessellated.

There are solutions for this already, like FTGL, but since it looks like xbmc has enough dependencies already, I'm just going to extract them using freetype, and try to keep it fairly modular in case freetype is ditched in the future.

The advantage of this approach is that there will be room to play with when it comes to optimization. Will work on lower end hardware, and perform well, and the anti-aliasing approach that makes sense for that hardware can be used. Whereas on higher-end hardware, a different preprocessing stage can be used on the glyphs, and use an approach similar to the one in the paper, for infinite resolution subtitles.

I'll post again when I have something to show.
Reply
#6
Just a quick update, nothing to show just yet though.

Main obstacle right now is finding tessellation code for non-convex 2D polygons that I can lift, that is GPL/LGPL(to avoid dependencies), and will work across all platforms.

First instinct was to go with the gluTess functions, however, they are not available on OpenGL ES or DirectX, and the open source implementations are under non-GPL licensees.

CGAL offered a slight glimmer of hope, as it has functions for partitioning a 2D non-convex polygon into convex polygons, which could then be easily triangulated. It is under both LGPL and QPL, and just my luck, the functions for partitioning 2D polygons are under the QPL.

So that's where I'm at right now, I'm going to keep looking around, since I'd like to avoid coding non-convex polygon triangulation by myself( would take quite a bit of time to accomplish it correctly and efficiently ).
Reply
#7
just a quick confirmation. I have come across CGAL before (for constrained Delaunay triangulation) and it is a very good/efficient library for this kind of thing (perhaps the only one), it is used in matlab too.
Reply
#8
Any update about this topic?
Reply
#9
Sounds like you are doing it wrong. Just create them for the scene, no need to cache them all. Caching might make sense for ascii.
WinLIRC, the Windows port of LIRC (Linux Infrared Remote Control):
http://winlirc.sourceforge.net
Active development of the WinLIRC package:
http://sourceforge.net/projects/winlirc/
LIRC config files for supported remote controls:
http://lirc.sourceforge.net/remotes/
Reply
#10
It's the lack of a cache that's causing the issue - when things are cached it's fine. When we run out of cache -> bam!

It could well be issues with the way we render the fonts to the texture in the first place - it's not done via pbo or anything like that.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#11
Version 9.11 does not have this issue , why don't you check it out, what was changed between 10.0 and 9.11 about subtitle rendering ?
Reply
#12
I think this is related to what is discussed here. Using a different font than the one supplied with XBMC seems to fix or at least negligate the issue (I am on an ancient single core Atom 230 and have not noticed any subtitle stutter since).
Reply
#13
a11599 Wrote:I think this is related to what is discussed here. Using a different font than the one supplied with XBMC seems to fix or at least negligate the issue (I am on an ancient single core Atom 230 and have not noticed any subtitle stutter since).

No, I don't think so, I tried some fonts on Mac OS X, still the same problem.
Reply
#14
any progress?
Reply
#15
How about doing it the way Media Player Classic does it - caching strings, instead of characters, about 10 seconds before displaying? The dsplayer branch of XBMC does it this way, and I am still using the dsplayer version of XBMC to this day, for this reason. The only catch is that I must lower the resolution of the cache to 960 by 540 pixels. However, it still looks really nice, even on 1080p displays, and my Acer 3610 can handle Chinese subtitles without stuttering at all.
Reply

Logout Mark Read Team Forum Stats Members Help
[BUG] Slow Chinese subtitle rendering0