[fritsch]

I'm not confused by how the "Prefer VAAPI Render Method" and "Use Sw Filter" settings work... but I'm still on Gotham, so I just pick the one that's staring me in the face. :-)

But, I do know that using VAAPI Rendering offers a noticeable improvement on CPU usage, which is why I had been eagerly waiting for someone to fix the hanging bug... but now, I'm learning there doesn't seem to be any way to use VAAPI Rendering AND avoid banding. Also, that the only way to use it with Limited RGB range involves double range conversion. Is the double conversion transparent (such that the second conversion doesn't make banding worse)?

It's quite hard to answer. In fact, when we use the vaPutSurface path a Full RGB conversion is done, we can "hint" 701 or 601 color format, which we do. Kodi does not touch this data anymore, as we use the VAAPi Upload to display those.
The OS driver then does the conversion back to "limited range".

We know, that this sucks like hell - and as we both, Fernetmenta and me are running VAAPI at home (Fernet in full RGB) and me (with a clamping AVR) with OS set to limited, really want to solve that by moving towards EGL, but this is not that easy, cause we don't have a Reference Clock for now, though the newly developed DRM Reference Clock could work.

Yeah, there is nothing we can "talk correct" or something - it's just fucked up as is. And the EGL codepath for vaapi is not even yet in the stable vaapi drivers, bleeding edge - nobody cared (on vendor side) for a much too long time.