Hello Kasbah,
Yes, the differences shouldn't be that big between OpenGL and OpenGL ES 2.0, I think. Some things like 3D textures, which Rudd seems to use for storing the decoded picture buffer may have to be changed to multiple 2D textures and so on, but the algorithms should be the same anyway.
I did stumble upon your Git repository a while ago, and compiled the code using that. I don't have anything to complain about there. It compiles
I was under the impression that it does do motion compensation on the GPU. I haven't looked at h264.c (in libavcodec) in much detail, but it seems like the implementation first decodes the first I-frame on the CPU, and transfers that frame to the 3D texture dpb_tex. This texture will then be used as a reference frame when doing motion comp. on the GPU.
It seems to perform the actual motion compensation in render_mbs (calls render_one_block properly for different block types), render_one_block (puts motion vectors and block coordinates in vertices and texture coordinates), and draw_mbs (actually executes things on the GPU).
Draw_mbs() seems to use six different shaders to do motion compensation in six passes. The remaining black blocks seem to be intra coded blocks, which are not handled by the GPU in that code.
glutMainLoop in gpu_motion() never returns which is the reason for it not doing anything after the second frame.
Hopefully we can learn something from each other during our projects. I don't have a real deadline yet, but am supposed to come up with some kind of benchmarks and demo before July. I do not expect to have a full h.264 GPU implementation by that time though
Right now I'm focusing on porting the code to OpenGL ES 2.0 and SDL, since GLUT and GLEW do not seem to support ES.