[wesk05]

Are you saying that that your test file is RGB encoded, or am I reading this wrong?

No, it is YUV/YCbCr. Apparently, Kodi needs information about the color space. Without that information, it seems to default to ?601 matrix for RGB -> YCbCr conversion. I guess even for Mediacodec (Surface) rendering Kodi does YCbCr ->RGB ->YCbCr conversions. I could be wrong about this, but the output was certainly messed up with the file that didn't have defined primaries/matrix/transfer.

I have two sets of 10-bit files. One for HDR and another for 4K Rec.709. When I made those changes, I forgot to put in the SEI/VUI details.