Kodi Community Forum

Full Version: nVidia graphics: Any way to disable VDPAU hardware decoding for 4:2:2 playback only?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3
I attempted to upgrade a system with nVidia graphics to Ubuntu 18.04 and Kodi 17.6.  The problem I am having is that if I leave VDPAU hardware acceleration enabled, any 4:2:2 content I attempt to play is a hot mess, only screen flashes of mostly green. However it works fine for any other format and does not put nearly as much load on the CPU.  What I have pretty much deduced is that the nVidia hardware in this system is not capable of decoding the 4:2:2 format.  in my previous install (Ubuntu 14.04 and Kodi Isengard) Kodi did not seem to have any problem figuring out that it could not use hardware decoding for 4:2:2, and switched to software decoding for that format.  But Kodi 18 doesn't seem to be able to detect that.

I am just wondering if there is some setting or switch or configuration option that I am missing, that would tell Kodi not to try to use VDPAU hardware acceleration on 4:2:2 content only.

This is what MediaInfo reports for the video details on a file that will not play properly with VDPAU acceleration enabled:

Video
ID                                       : 142 (0x8E)
Menu ID                                  : 4 (0x4)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:2:[email protected]
Format settings                          : CABAC / 2 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 2 frames
Format settings, GOP                     : M=3, N=36
Codec ID                                 : 27
Duration                                 : 1 h 3 min
Bit rate mode                            : Constant
Bit rate                                 : 15.2 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan type, store method                  : Separated fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.245
Stream size                              : 6.72 GiB (90%)


As you can see, the bit depth is 8 bits, so it's not an issue with 10 bit video, it is solely a problem with 4:2:2 video as far as I can determine.  I have another system with Intel graphics and on that system I have no problem playing such files, even with all the VAAPI settings enabled, so this seems to be a particular issue with nVidia graphics.  Any suggestions on how I can fix this (other that "disable VDPAU hardware acceleration and just use software decoding for everything", because that drives up CPU usage and increases fan noise) would be much appreciated.  I should also mention that this computer is a type that doesn't have card slots, and the nVidia graphics are integrated on the main board, so I cannot swap out video cards.
return false based on the hints you receive in vdpau decoder. PR it upstream.
(2018-08-18, 14:12)fritsch Wrote: [ -> ]return false based on the hints you receive in vdpau decoder. PR it upstream.
I honestly did not understand any of that, it made no sense to me at all.  Could you please explain without using jargon or abbreviations?
please open a trac ticket and provide a short sample to reproduce
Well since no one actually answered my question, I decided to try completely removing Kodi 17.6 and installing the Leia nightly.  Unfortunately that was even worse - several channels that would play fine in 17.6 with VDPAU enabled would not play in the Leia beta, and those were only 720p 4:2:0 channels.  This was with the same operating system, nVidia driver, etc. - the only difference was the Kodi version.  I would suggest that anyone that is an alpha or beta tester and that has older nVidia hardware probably should test several different channels and program sources, particularly those in less-used formats.

So anyway, I went back to 17.6 and decided to use a workaround.  I first configured Kodi to use VDPAU hardware decoding, exited, and copied the guisettings.xml to a directory.  I then started Kodi again, configured it to NOT use VDPAU hardware decoding, and copied the guisettings.xml to a different directory.  I have been using a small bash script to start Kodi so that I can start it by pressing a button on my remote, so I duplicated that and set a different button to run the second script, but now in each script prior to starting Kodi it copies back one of the two saved guisettings.xml files.  So if I push one button it starts Kodi with VDPAU hardware decoding enabled, and if I push the other it starts it with hardware decoding disabled.  This isn't an ideal solution by any means, but it sort of solves the problem, without having to understand an obtuse reply that make no sense at all to me and that did not even attempt to answer my question.

It seems to me as though newer versions of Kodi have a real issue with determining which files can be processed with VDPAU hardware acceleration and which cannot.  I think in some cases it might be because the hardware on the board, or maybe the nVidia driver, is lying about its capabilities - for example, the hardware claims it can do 4:2:2 but in reality it cannot.  Providing a sample file would not help unless it was played on very similar hardware.  If you have a nVidia chipset that claims it can do 4:2:2 and actually can do 4:2:2 then you would probably not see the problem.  And for reasons I cannot really explain but that have to do with ownership issues, I'm unable to provide a sample file, but that might not help anyway if you are using newer nVidia hardware.  If Kodi had more ways to specify that hardware decoding should not be used for certain specific formats, it might address this issue.  I did note there were a couple of additional VDPAU settings in guisettings.xml that were not exposed in the Kodi settings, but setting those to false didn't make any difference with this issue.  But i really have no idea why Leia would not play some recordings that Krypton could play just fine, or why Krypton won't play recordings that played with no problem in Isengard, with VDPAU hardware decoding enabled, and I'm not a technically-inclined user of Kodi.
You didn't get a direct answer to your question because your question is wrong.
The right thing to do is to fix the problem, not to hide it by disabling something which can only be disabled in source code, that's what "obtuse" reply meant.
The answer is: you can't disable it without modifyng the source code, there is neither a setting not a "magic button"  for that
You are claiming you have some old nvidia hardware without stating which exactly, you don't even tell us which distro you are using, and of course providing a debug log seems to be an impossible task.

FFS, that's why you were asked to provide a sample and create a bug report on the trac with the relevant debug log(s) and actual hardware info, so the issue can be looked at and possibly fixed for EVERY nvidia user.
(2018-08-19, 02:18)asavah Wrote: [ -> ]You didn't get a direct answer to your question because your question is wrong.
The right thing to do is to fix the problem, not to hide it by disabling something which can only be disabled in source code, that's what "obtuse" reply meant.
The answer is: you can't disable it without modifyng the source code, there is neither a setting not a "magic button"  for that
You are claiming you have some old nvidia hardware without stating which exactly, you don't even tell us which distro you are using, and of course providing a debug log seems to be an impossible task.

FFS, that's why you were asked to provide a sample and create a bug report on the trac with the relevant debug log(s) and actual hardware info, so the issue can be looked at and possibly fixed for EVERY nvidia user.
This is why I hate with a passion trying to ask a question in this forum.  You ask a simple question, no one will answer it, and then if someone finally does they feel they need to berate you.

I did not state what nVidia hardware it is because I do not know what nVidia hardware it is.  It's embedded nVidia graphics in an ASRock Vision 3D computer.  It's not on a separate video card.  That's all I know about the hardware.

The very first sentence of my first post was "I attempted to upgrade a system with nVidia graphics to Ubuntu 18.04 and Kodi 17.6."  So yes, I did tell you what distro I am using.

Nobody said a single word about a debug log until your post but that's not what I was asking anyway.  I was asking if a particular setting existed.  But no, I don't do debug logs because I don't know how to do them and I don't know what's in them.  The last time anyone mentioned them in response to something I had asked, which was MANY months ago, I was told you could only post them publicly on a server where anybody could read them, and that you could not send them directly to a developer.  Again, I don't know what's in them, but it could be information I don't want to share publicly, so that's not going to happen.  Kodi is the only piece of software I've ever used where the developers want to see debug logs but make it so inconvenient for people to provide them, or at least that used to be the case, not sure what the situation is now.

If they want to fix this for EVERY nvidia user that would be great, maybe someone who's not in my particular situation and that is far more versed in the ways of Linux than I am can provide the information that will allow them to do that.  But I can't, and I can't even really explain why, as I said it has to do with ownership and that's all I can say.  However, why would asking if a setting exists obligate me to provide information that I either can't or don't want to provide?  I don't get the logic that says that if I ask whether a setting exists, I'm now on the hook to read minds and provide information that I either don't understand or that hasn't been asked for.

Sometimes hacky workarounds are the easiest solution.  They may not be the RIGHT solution, but if that's the only way you can understand how to work around the problem, that's what you do.  It doesn't help when people use jargon, acronyms, or abbreviations that are meaningless to the ordinary user.
(2018-08-19, 03:16)xbmclinuxuser Wrote: [ -> ]Nobody said a single word about a debug log until your post but that's not what I was asking anyway.  I was asking if a particular setting existed.  But no, I don't do debug logs because I don't know how to do them and I don't know what's in them.
I'm sorry... It does seem a little strange to me, when someone who signed up on this forum on some 5.5 years ago and posted almost 400 messages on the Kodi forum, is totally unaware of something which is referred to so many times as a kodi.log or a debug log (wiki) file. It must have come across the threads you posted in at some point in all that time. You know, the stupid questions are those not asked. So, simply ask what the debug log (wiki) is for.

The Kodi log/debug file, especially when 'debugging' is enabled inside the Kodi application for extra details, will provide a sh!tload of technical information during a Kodi session, so the Kodi developers or others can likely see where things go wrong. It will provide all technical info of the device (OS, cpu, gpu, etcetera) itself at the start, and all subsequent details from the video and/or music files you play. If there is no log file, we won't know where or how things went wrong. The proverbial crystal ball is not part of our tools to fix things in Kodi. Smile
(2018-08-19, 01:31)xbmclinuxuser Wrote: [ -> ]Well since no one actually answered my question, I decided to try completely removing Kodi 17.6 and installing the Leia nightly.  Unfortunately that was even worse - several channels that would play fine in 17.6 with VDPAU enabled would not play in the Leia beta, and those were only 720p 4:2:0 channels.  This was with the same operating system, nVidia driver, etc. - the only difference was the Kodi version.  I would suggest that anyone that is an alpha or beta tester and that has older nVidia hardware probably should test several different channels and program sources, particularly those in less-used formats.

So anyway, I went back to 17.6 and decided to use a workaround.  I first configured Kodi to use VDPAU hardware decoding, exited, and copied the guisettings.xml to a directory.  I then started Kodi again, configured it to NOT use VDPAU hardware decoding, and copied the guisettings.xml to a different directory.  I have been using a small bash script to start Kodi so that I can start it by pressing a button on my remote, so I duplicated that and set a different button to run the second script, but now in each script prior to starting Kodi it copies back one of the two saved guisettings.xml files.  So if I push one button it starts Kodi with VDPAU hardware decoding enabled, and if I push the other it starts it with hardware decoding disabled.  This isn't an ideal solution by any means, but it sort of solves the problem, without having to understand an obtuse reply that make no sense at all to me and that did not even attempt to answer my question.

It seems to me as though newer versions of Kodi have a real issue with determining which files can be processed with VDPAU hardware acceleration and which cannot.  I think in some cases it might be because the hardware on the board, or maybe the nVidia driver, is lying about its capabilities - for example, the hardware claims it can do 4:2:2 but in reality it cannot.  Providing a sample file would not help unless it was played on very similar hardware.  If you have a nVidia chipset that claims it can do 4:2:2 and actually can do 4:2:2 then you would probably not see the problem.  And for reasons I cannot really explain but that have to do with ownership issues, I'm unable to provide a sample file, but that might not help anyway if you are using newer nVidia hardware.  If Kodi had more ways to specify that hardware decoding should not be used for certain specific formats, it might address this issue.  I did note there were a couple of additional VDPAU settings in guisettings.xml that were not exposed in the Kodi settings, but setting those to false didn't make any difference with this issue.  But i really have no idea why Leia would not play some recordings that Krypton could play just fine, or why Krypton won't play recordings that played with no problem in Isengard, with VDPAU hardware decoding enabled, and I'm not a technically-inclined user of Kodi.
 I really don't get why you invest so much time for such a wall of text while it is totally clear that nvidia on vdpau can only decode 4:2:0 - you need to distinguish between answers that "answer your question" and answers you don't understand!

Here is what I have written:
Code:
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
index 72c8384885..1cfe5338a2 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
@@ -878,6 +878,9 @@ bool CDecoder::ConfigVDPAU(AVCodecContext* avctx, int ref_frames)
 
   ReadFormatOf(avctx->codec_id, vdp_decoder_profile, m_vdpauConfig.vdpChromaType);
 
+  if (avctx->pix_fmt == AV_PIX_FMT_YUV422P)
+    return false;
+
   if (avctx->codec_id == AV_CODEC_ID_H264)
   {
     m_vdpauConfig.maxReferences = ref_frames;

I simple google would have found for you in this forum: https://forum.kodi.tv/showthread.php?tid...pid1428125
(2018-08-19, 09:56)fritsch Wrote: [ -> ]Here is what I have written:
Code:
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
index 72c8384885..1cfe5338a2 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
@@ -878,6 +878,9 @@ bool CDecoder::ConfigVDPAU(AVCodecContext* avctx, int ref_frames)
 
   ReadFormatOf(avctx->codec_id, vdp_decoder_profile, m_vdpauConfig.vdpChromaType);
 
+  if (avctx->pix_fmt == AV_PIX_FMT_YUV422P)
+    return false;
+
   if (avctx->codec_id == AV_CODEC_ID_H264)
   {
     m_vdpauConfig.maxReferences = ref_frames;
Nice try but wrong Smile

Best chances to get this fixed is to follow my request form above.
(2018-08-19, 10:08)FernetMenta Wrote: [ -> ]
(2018-08-19, 09:56)fritsch Wrote: [ -> ]Here is what I have written:
Code:
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
index 72c8384885..1cfe5338a2 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
@@ -878,6 +878,9 @@ bool CDecoder::ConfigVDPAU(AVCodecContext* avctx, int ref_frames)
 
   ReadFormatOf(avctx->codec_id, vdp_decoder_profile, m_vdpauConfig.vdpChromaType);
 
+  if (avctx->pix_fmt == AV_PIX_FMT_YUV422P)
+    return false;
+
   if (avctx->codec_id == AV_CODEC_ID_H264)
   {
     m_vdpauConfig.maxReferences = ref_frames;
Nice try but wrong Smile

Best chances to get this fixed is to follow my request form above. 
 Works here, does not for you?

Really nice would be if the decoder would not even be opened if the pix_fmt does not work ... but there are other decoders like VAAPI that can do 4:2:2 h264, this would need some more changes in DVDVideoCodecFFmpeg. I will give you a sample in some minutes.
(2018-08-19, 10:20)fritsch Wrote: [ -> ]
(2018-08-19, 10:08)FernetMenta Wrote: [ -> ]
(2018-08-19, 09:56)fritsch Wrote: [ -> ]Here is what I have written:
Code:
diff --git a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
index 72c8384885..1cfe5338a2 100644
--- a/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
+++ b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp
@@ -878,6 +878,9 @@ bool CDecoder::ConfigVDPAU(AVCodecContext* avctx, int ref_frames)
 
   ReadFormatOf(avctx->codec_id, vdp_decoder_profile, m_vdpauConfig.vdpChromaType);
 
+  if (avctx->pix_fmt == AV_PIX_FMT_YUV422P)
+    return false;
+
   if (avctx->codec_id == AV_CODEC_ID_H264)
   {
     m_vdpauConfig.maxReferences = ref_frames;
Nice try but wrong Smile

Best chances to get this fixed is to follow my request form above.  
 Works here, does not for you?

Really nice would be if the decoder would not even be opened if the pix_fmt does not work ... but there are other decoders like VAAPI that can do 4:2:2 h264, this would need some more changes in DVDVideoCodecFFmpeg. I will give you a sample in some minutes. 
I am 100% sure that it does not work for you either. pix_fmt at this location is always AV_PIX_FMT_VDPAU or vdpau won't be created
(2018-08-19, 08:43)Klojum Wrote: [ -> ]
(2018-08-19, 03:16)xbmclinuxuser Wrote: [ -> ]Nobody said a single word about a debug log until your post but that's not what I was asking anyway.  I was asking if a particular setting existed.  But no, I don't do debug logs because I don't know how to do them and I don't know what's in them.
I'm sorry... It does seem a little strange to me, when someone who signed up on this forum on some 5.5 years ago and posted almost 400 messages on the Kodi forum, is totally unaware of something which is referred to so many times as a kodi.log or a debug log (wiki) file. It must have come across the threads you posted in at some point in all that time. You know, the stupid questions are those not asked. So, simply ask what the debug log (wiki) is for.
It seems a little strange to me that no one can grasp the concept that I was asking if a setting existed, not attempting to report a bug.  If I had said this was a bug and I wanted it fixed, then I suppose you would be justified in asking for that level pf information, but I avoid doing that in this forum, and the main reason is to be found on that wiki page:
Quote:1 Important Notes
Do NOT post your logs directly in a forum. Always use a paste site like https://pastebin.com or https://paste.ubuntu.com . Post the link to your pasted log in the forum. Details are in the following pages.
So you want me to post the log publicly where anyone in the world could read it.  And that's bad enough, but then it says:
Quote:Provide FULL Logs only. No Partial logs or snippets. The only redactions should be personal information.
So basically you want people to create what will undoubtedly be a very long log file, then go through it line by line to try to find any sensitive information that should be redacted (and I have a hunch my idea of what constitutes personal information and yours might be rather different), and then post it on the wide open Internet.  Maybe you don't have a problem with doing that, but I do.  I especially do when I was just asking a question, and not even trying to report a bug.

Seriously, if you guys want to receive debug logs, you should make sure that Kodi isn't including personal information in the first place, but mainly you need to find a way to let people send them to just the Kodi developers without making them readable to just anyone and everyone who happens along.
Jep you are right. Worked for me if you changed the == to != but stops hw accel for everything :-).

Btw originally I wanted to use the hints, here it should not be vdpauified or vaapified.
Pages: 1 2 3