Kodi Community Forum
Improving commercial break skip support for MythTV - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=26)
+---- Forum: PVR (https://forum.kodi.tv/forumdisplay.php?fid=136)
+---- Thread: Improving commercial break skip support for MythTV (/showthread.php?tid=131016)

Pages: 1 2 3 4


Improving commercial break skip support for MythTV - dteirney - 2012-05-08

I've heard that a number of people may be experiencing commercial skips happening in the wrong place in XBMC where it works correctly in mythfrontend.

If this has happened for you are you able to post:
1) A link to a full debug log opening the recording that has the incorrect skip behaviour (please just that file - full debug log files are long enough as it is).
2) A list of the time periods that the commercial break should have been (to the nearest 5 seconds should be fine)
3) What sort of tuner the recording was recorded using, e.g. DVB-T, Analog, Cable via USB Tuner.
4) Whether you know if there are any frame rate changes in the playback, e.g. I've heard that some stations play commercial breaks at a different frame rate to the broadcast content. If this happens, unfortunately, there is unlikely to be a workaround.

If it's possible to improve the behaviour I will. However, mythbackend stores frame markers for the commercial break locations and XBMC needs time markers to perform seeking so there is a little bit of heuristic black magic to get the right frame rate to use to convert from the frame markers to timestamps.

For interest sake as well, consider having a look at the advanced EDL settings in XBMC to help remove false positive commercial breaks. http://wiki.xbmc.org/index.php?title=Userdata/advancedsettings.xml#.3Cedl.3E

And for those that are also interested I've already submitted a patch that appears to resolve the 10 second hang that is experienced when a commerical break is skipped.


RE: Improving commercial break skip support for MythTV - dteirney - 2012-05-13

No takers so far so I'll relax the requirements somewhat. If you are having problems with commercials being skipped at the wrong time please post a debug log showing playback of the recording that is problematic.


RE: Improving commercial break skip support for MythTV - tsp42 - 2012-05-13

If it is any help the mythtv MySQL table recordedmarkup contains the framerate of the recordings. It would be interesting if the people that experience problems could post the content of that table to see if mythtv stores when the framerate changes in the recordings.


RE: Improving commercial break skip support for MythTV - dteirney - 2012-05-14

(2012-05-13, 13:30)tsp42 Wrote: If it is any help the mythtv MySQL table recordedmarkup contains the framerate of the recordings. It would be interesting if the people that experience problems could post the content of that table to see if mythtv stores when the framerate changes in the recordings.

Do you mean this table? http://www.mythtv.org/wiki/Recordedmarkup_table

That table stores the actual frame number for each type of mark that gets stored. The offset column (apparently) used to contain the associated timestamp but that was deprecated ages ago and was empty when I looked at using it several years ago.




RE: Improving commercial break skip support for MythTV - tsp42 - 2012-05-14

(2012-05-14, 10:28)dteirney Wrote:
(2012-05-13, 13:30)tsp42 Wrote: If it is any help the mythtv MySQL table recordedmarkup contains the framerate of the recordings. It would be interesting if the people that experience problems could post the content of that table to see if mythtv stores when the framerate changes in the recordings.

Do you mean this table? http://www.mythtv.org/wiki/Recordedmarkup_table

That table stores the actual frame number for each type of mark that gets stored. The offset column (apparently) used to contain the associated timestamp but that was deprecated ages ago and was empty when I looked at using it several years ago.
Yes at least in my version 0.24 it contains all kind of information about the recordings. The rows with type == 32 contains the framerate.
See https://github.com/MythTV/mythtv/blob/master/mythtv/libs/libmyth/programtypes.h
Here is the actual commit that added the frame rate to the markup table: https://github.com/MythTV/mythtv/commit/9ef808914de16deb14f26d37f57b2f65f8cec2e5


RE: Improving commercial break skip support for MythTV - dteirney - 2012-05-15

Interesting. Just out of interest how many 32 entries does it have for each recording?

The automated seeking in the XBMC player is time based. Without being able to calculate the correct time for the start and end positions of the commercial breaks before the playback starts everything gets messy. If someone knows some magic to calculate the actual playback time that corresponds to a frame marker using whatever information is currently in the database that might be an avenue to explore. Mythfrontend obviously manages (maybe) but I believe it's player uses the recordedseek table for some information.

The current approach will never work properly for recordings that have a change in frame rate in them. Tracking frame rate changes in the player will not help because as soon as a manual skip is done you don't have any idea what was skipped over so the calculations could be all up in the air again.


RE: Improving commercial break skip support for MythTV - tsp42 - 2012-05-15

It only has one entry per recording but the frame rate doesn't change during the recordings.
According to the commit previous linked every frame rate change should be included in the table.

The following pseudo-code could be used to convert the frame marker to times in seconds:
Code:
//Parameters
//framemarker: frame marker to convert to time
//framerates: array of frame rates from the recordedmarkup table
//marks: matching array of frame marks from the recordmarkup table
//length: length of the two arrays
//
//Retval: time in seconds corrensponding to the framemarker
time_t markerToTime(long long framemarker,int framerates[],long long marks[],int length)
{
    time_t retval=0;
    if(length==1) //constant framerate
        return(framemarker/framerates[0]);
    for(int i=1;i<length;i++)
    {
        if(framemarker>=marks[i])
            retval += (marks[i]-marks[i-1])/framerates[i-1];
        else
            return(retval + (framemarker-marks[i-1])/framerates[i-1]);
    }
    return(retval);
}



RE: Improving commercial break skip support for MythTV - dteirney - 2012-05-16

I'll recheck the mythbackend code that is used to calculate the frame rate in the first place. If it's doing the same as XBMC, which I think it was when I initially looked, then the frame rate would probably still need manipulation like it is using the current heuristics within XBMC.

The equivalent frame rate that was used by mythcommflag is what is needed, not what is reported in the video stream header information.

Having said that if the database will indeed contain an entry each time the framerate changes that would help. The heuristics could just move into the calculation.


RE: Improving commercial break skip support for MythTV - tsp42 - 2012-05-16

Looks like MythTV scans each transport stream packet for the frame rate while saving the recording:
https://github.com/MythTV/mythtv/blob/master/mythtv/libs/libmythtv/dtvrecorder.cpp



RE: Improving commercial break skip support for MythTV - varneyb - 2012-06-11

(2012-05-08, 14:42)dteirney Wrote: 1) A link to a full debug log opening the recording that has the incorrect skip behaviour (please just that file - full debug log files are long enough as it is).
2) A list of the time periods that the commercial break should have been (to the nearest 5 seconds should be fine)
3) What sort of tuner the recording was recorded using, e.g. DVB-T, Analog, Cable via USB Tuner.
4) Whether you know if there are any frame rate changes in the playback, e.g. I've heard that some stations play commercial breaks at a different frame rate to the broadcast content. If this happens, unfortunately, there is unlikely to be a workaround.

1) I'll have to work on getting you a log, but I know that I do get the following WARNING:
WARNING: ReadComskip - Frame rate not in Comskip file. Using detected frames per second: 29.970

2) I have found that if I use a skip list generated by comskip.exe instead of mythcommflag, it works. For E02 of Unforgettable:
Here is the mythcommflag skip list:
FILE PROCESSING COMPLETE
------------------------
31137 36371
47736 53922
68679 72980
88667 92785


And here is the comskip skip list:
FILE PROCESSING COMPLETE 107718 FRAMES AT 2997
-------------------
1 206
10030 15825
36923 42438
56113 62638
80338 84924
98699 107718

3) Over the air (ATSC) tuner (HDHOMERUN)
4) I don't know, but I know that if I use the comskip list, I get the skips in the right place. However, there is a substantial delay (15-20 seconds) when it skips during which XBMC is completely unresponsive. This has happened with EDEN both on XBMCBUNTU and Windows Vista (I did not have this problem with Dharma on Vista).




RE: Improving commercial break skip support for MythTV - dteirney - 2012-06-11

(2012-06-11, 03:19)varneyb Wrote:
(2012-05-08, 14:42)dteirney Wrote: 1) A link to a full debug log opening the recording that has the incorrect skip behaviour (please just that file - full debug log files are long enough as it is).
2) A list of the time periods that the commercial break should have been (to the nearest 5 seconds should be fine)
3) What sort of tuner the recording was recorded using, e.g. DVB-T, Analog, Cable via USB Tuner.
4) Whether you know if there are any frame rate changes in the playback, e.g. I've heard that some stations play commercial breaks at a different frame rate to the broadcast content. If this happens, unfortunately, there is unlikely to be a workaround.

1) I'll have to work on getting you a log, but I know that I do get the following WARNING:
WARNING: ReadComskip - Frame rate not in Comskip file. Using detected frames per second: 29.970

2) I have found that if I use a skip list generated by comskip.exe instead of mythcommflag, it works. For E02 of Unforgettable:
Here is the mythcommflag skip list:
FILE PROCESSING COMPLETE
------------------------
31137 36371
47736 53922
68679 72980
88667 92785


And here is the comskip skip list:
FILE PROCESSING COMPLETE 107718 FRAMES AT 2997
-------------------
1 206
10030 15825
36923 42438
56113 62638
80338 84924
98699 107718

3) Over the air (ATSC) tuner (HDHOMERUN)
4) I don't know, but I know that if I use the comskip list, I get the skips in the right place. However, there is a substantial delay (15-20 seconds) when it skips during which XBMC is completely unresponsive. This has happened with EDEN both on XBMCBUNTU and Windows Vista (I did not have this problem with Dharma on Vista).

1) The detected frame rate of 29.97 matches the frame rate in the comskip.exe generated file, e.g. the 2997 part. So that means that the commercial skip tools are finding different commercial break locations. What commercial detection method are you using in mythbackend? I use Logo Detection for New Zealand transmissions. All the other methods were hopeless.

2) You shouldn't get the warning re the Framerate not being in the comskip file for that file. 2997 means 29.97fps. How are you playing those recordings?

4) There is a known issue that is currently being worked through for the skip delay. https://github.com/xbmc/xbmc/pull/944 (still needs a lot more work with help from another XBMC developer to sort out what introduced the problem).





RE: Improving commercial break skip support for MythTV - varneyb - 2012-06-11

(2012-06-11, 09:15)dteirney Wrote: 1) The detected frame rate of 29.97 matches the frame rate in the comskip.exe generated file, e.g. the 2997 part. So that means that the commercial skip tools are finding different commercial break locations. What commercial detection method are you using in mythbackend? I use Logo Detection for New Zealand transmissions. All the other methods were hopeless.

2) You shouldn't get the warning re the Framerate not being in the comskip file for that file. 2997 means 29.97fps. How are you playing those recordings?

4) There is a known issue that is currently being worked through for the skip delay. https://github.com/xbmc/xbmc/pull/944 (still needs a lot more work with help from another XBMC developer to sort out what introduced the problem).

1) I'm just using the default - I can try other methods and see what I get. Note, however that when I try to remove commercials with mythtranscode, it removes the wrong sections (i.e. still leaves some of the commercials in and removes some of the show) regardless of whether I use the mythcommflag or comskip skip lists to generate the cut list for the file.
2) I'm not sure I understand the question. I'm playing them in XBMC on XBMCBuntu.
4) Thanks for the info.


RE: Improving commercial break skip support for MythTV - tdavis - 2013-01-11

Bringing this backup from the dead..

Using these two PR's:

https://github.com/xbmc/xbmc/pull/1743
https://github.com/fetzerch/xbmc-pvr-addons/pull/51

You can build a version of Frodo & pvr.Mythtv that uses the MythTV commercial skips.

It works great on a fixed frame rate mpeg, which is the default on OTA here.

Using a CableCard enabled Prime on Comcast, however.. that is not true. When you view using Mythtvfrontend, the breaks are dead on. When using pvr & xbmc, they are 15 or more seconds off (it skips early, and ends early).

Playing back the files using mplayer shows this:

demux_mpg: 24000/1001fps progressive NTSC content detected, switching framerate.
A:14575.0 V:14575.0 A-V: 0.022 ct: 0.051 1938/1583 22% 0% 1.1% 0 0

demux_mpg: 30000/1001fps NTSC content detected, switching framerate.
A:14575.0 V:14575.0 A-V: 0.025 ct: 0.053 1939/1584 22% 0% 1.1% 0 0
Warning! FPS changed 47.952 -> 59.940 (-11.988010) [7]
A:14576.2 V:14576.2 A-V: 0.016 ct: 0.083 2010/1654 23% 0% 1.1% 0 0

demux_mpg: 24000/1001fps progressive NTSC content detected, switching framerate.
A:14576.2 V:14576.2 A-V: 0.015 ct: 0.085 2013/1655 23% 0% 1.1% 0 0

demux_mpg: 30000/1001fps NTSC content detected, switching framerate.
A:14576.2 V:14576.2 A-V: 0.041 ct: 0.087 2014/1656 23% 0% 1.1% 0 0
Warning! FPS changed 47.952 -> 59.940 (-11.988010) [7]
A:14576.5 V:14576.6 A-V: -0.013 ct: 0.075 2031/1668 23% 0% 1.1% 0 0

and no, those are not at scene changes or commercial breaks..

The EDL/comskip in the two mentioned PRs uses a fixed frame rate; mythtv db shows the videos have a fixed frame rate; but they do not.

I'm working on getting a sample of LiveTV that has a commercial break for testing purposes that has varying frame rates..




RE: Improving commercial break skip support for MythTV - LSU Jonno - 2013-01-13

Curious what could be causing this. I have a HDHR Prime on Comcast and am getting these same huge time shifts during commercial skips. I've been aware of this constant frame rate switching since I built my HTPC 2 years ago.

If you have an ATI video card and enable "Dynamic Contrast" on the ATI Catalyst Control Center you can visually see how often these frame rate changes happen. Your screen will change brightness consistent with the FR changes...It happens 10-30 times per hour long TV program.

Anyway, assuming the FR changes aren't consistent in one direction, wouldn't you expect them to sort of cancel themselves out resulting in no actual, or maybe no perceived shift when assuming a constant frame rate multiplied by total frames to come up with a shift time?


RE: Improving commercial break skip support for MythTV - dteirney - 2013-01-13

If the frame rate of the video changes, then the time markers calculated from the initial frame rate and frame markers from mythbackend will all be wrong after the first frame rate switch. The amount they are off will depend on how different the frame rate is and for how long that different frame rate plays for. Without knowing all of the places where the frame rate changes to / from it is not possible to pre-calculate the correct time markers. Of course, all of this assumes that the commercial break locations were correctly calculated by the commercial flagging tool in the first place.