Comskip for Linux released - Printable Version

Comskip for Linux released - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: PVR & Live TV Support (https://forum.kodi.tv/forumdisplay.php?fid=167)
+---- Forum: Tvheadend (https://forum.kodi.tv/forumdisplay.php?fid=168)
+---- Thread: Comskip for Linux released (/showthread.php?tid=150084)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

RE: Comskip for Linux released - Prof Yaffle - 2013-01-03

I'm going to have to do some serious tuning to make this work for me, but that's no-one's fault - the original comskip on Windows struggles as well, too many different DOGs to fight with as much as anything, I think. But this is doing what it set out to do, it seems, and cbxbiker61 is being very responsive to this thread - so thanks again for all the great work on this.

Now, how's H.264 support coming along? :-P

RE: Comskip for Linux released - MarcAngelo - 2013-01-04

I must agree that cbxbiker61 is very responsive on this thread and very helpful in finding solutions.
Big hand of applause for that!! :):)

I'm a bit confused at the moment though...
When I process an mkv with offset 0.0, the start is sometimes correct, but skipping over the commercials seems sometimes 30 secs later. (tv show is already playing 30 secs after commercials ended).

When I process an mkv with offset 30.0, the start is sometimes aprox. 30 secs later, but skipping over the commercials is correct.

At the end, I had to use "edl_mode=0", since value "3" (Commercial Breaks) seems to be ignored by XBMC (no commercial skipping).
With value "0" the time of the video is adjusted (shorter), this seems correct.

I make the recordings from DVB-T provider "Digitenne" (KPN) in Holland, so I guess al video's should be PAL 25fps.
Is there a way to calculate the values in the .edl or .txt file to the actual times, so I can manually check this in another player,
for example VLC? (to check if detected times for commercials is correct).

Thanks!

Marc

RE: Comskip for Linux released - cbxbiker61 - 2013-01-07

I've release 0.92c.

Generation of .csv files should now work (useful for tuning comskip).

More code has been converted to C++11.

Commercial detection should behave the same as before.

Once the code cleanup is finished, we can begin to look at more improvements to commercial detection.

RE: Comskip for Linux released - cbxbiker61 - 2013-01-08

(2013-01-03, 12:45)Prof Yaffle Wrote: I'm going to have to do some serious tuning to make this work for me, but that's no-one's fault - the original comskip on Windows struggles as well, too many different DOGs to fight with as much as anything, I think. But this is doing what it set out to do, it seems, and cbxbiker61 is being very responsive to this thread - so thanks again for all the great work on this.

Now, how's H.264 support coming along? :-P

Theoretically H.264 should work from mkv files. I haven't tested it though, since most of the source material starts as mpeg. H264 would be a likely target after running comskip, i.e. start with mpeg, locate commercials, then transcode to h264 with commercials removed.

RE: Comskip for Linux released - MarcAngelo - 2013-01-08

Hey Biker!

Thanks for the update, but this is the first version I'm unable to compile.
I use the procedure for ffmpeg < 0.9.

make -j8
make all-am
make[1]: Entering directory `/var/tmp/comskip-0.92c'
g++ -DHAVE_CONFIG_H -I. -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -g -O2 -std=gnu++0x -MT comskip-mpeg2dec.o -MD -MP -MF .deps/comskip-mpeg2dec.Tpo -c -o comskip-mpeg2dec.o `test -f 'mpeg2dec.cpp' || echo './'`mpeg2dec.cpp
g++ -DHAVE_CONFIG_H -I. -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -g -O2 -std=gnu++0x -MT comskip-comskip.o -MD -MP -MF .deps/comskip-comskip.Tpo -c -o comskip-comskip.o `test -f 'comskip.cpp' || echo './'`comskip.cpp
g++ -DHAVE_CONFIG_H -I. -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -g -O2 -std=gnu++0x -MT comskip-BlackFrame.o -MD -MP -MF .deps/comskip-BlackFrame.Tpo -c -o comskip-BlackFrame.o `test -f 'BlackFrame.cpp' || echo './'`BlackFrame.cpp
g++ -DHAVE_CONFIG_H -I. -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -g -O2 -std=gnu++0x -MT comskip-ProcessCsv.o -MD -MP -MF .deps/comskip-ProcessCsv.Tpo -c -o comskip-ProcessCsv.o `test -f 'ProcessCsv.cpp' || echo './'`ProcessCsv.cpp
g++ -DHAVE_CONFIG_H -I. -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -g -O2 -std=gnu++0x -MT comskip-ProcessArInfo.o -MD -MP -MF .deps/comskip-ProcessArInfo.Tpo -c -o comskip-ProcessArInfo.o `test -f 'ProcessArInfo.cpp' || echo './'`ProcessArInfo.cpp
g++ -DHAVE_CONFIG_H -I. -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -I/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include -g -O2 -std=gnu++0x -MT comskip-LoadSettings.o -MD -MP -MF .deps/comskip-LoadSettings.Tpo -c -o comskip-LoadSettings.o `test -f 'LoadSettings.cpp' || echo './'`LoadSettings.cpp
mv -f .deps/comskip-BlackFrame.Tpo .deps/comskip-BlackFrame.Po
In file included from ProcessArInfo.cpp:1:0:
comskip.h:247:7: error: looser throw specifier for ‘virtual CS::FileException::~FileException()’
/usr/include/c++/4.6/exception:65:13: error: overriding ‘virtual std::exception::~exception() throw ()’
make[1]: *** [comskip-ProcessArInfo.o] Error 1
make[1]: *** Waiting for unfinished jobs....
In file included from ProcessCsv.cpp:3:0:
comskip.h:247:7: error: looser throw specifier for ‘virtual CS::FileException::~FileException()’
/usr/include/c++/4.6/exception:65:13: error: overriding ‘virtual std::exception::~exception() throw ()’
In file included from LoadSettings.cpp:11:0:
comskip.h:247:7: error: looser throw specifier for ‘virtual CS::FileException::~FileException()’
/usr/include/c++/4.6/exception:65:13: error: overriding ‘virtual std::exception::~exception() throw ()’
ProcessCsv.cpp: In member function ‘void CS::CommercialSkipper::processCsv(std: Confused

tring)’:
ProcessCsv.cpp:41:36: warning: ignoring return value of ‘char* fgets(char*, int, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
In file included from mpeg2dec.cpp:66:0:
comskip.h:247:7: error: looser throw specifier for ‘virtual CS::FileException::~FileException()’
/usr/include/c++/4.6/exception:65:13: error: overriding ‘virtual std::exception::~exception() throw ()’
make[1]: *** [comskip-ProcessCsv.o] Error 1
In file included from comskip.cpp:42:0:
comskip.h:247:7: error: looser throw specifier for ‘virtual CS::FileException::~FileException()’
/usr/include/c++/4.6/exception:65:13: error: overriding ‘virtual std::exception::~exception() throw ()’
mpeg2dec.cpp: In function ‘void {anonymous}::audio_packet_process({anonymous}::VideoState*, AVPacket*)’:
mpeg2dec.cpp:576:18: warning: ‘int avcodec_decode_audio3(AVCodecContext*, int16_t*, int*, AVPacket*)’ is deprecated (declared at /var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include/libavcodec/avcodec.h:3852) [-Wdeprecated-declarations]
mpeg2dec.cpp:577:75: warning: ‘int avcodec_decode_audio3(AVCodecContext*, int16_t*, int*, AVPacket*)’ is deprecated (declared at /var/tmp/ComSkipWork/ffmpeg-1.0.1-install/include/libavcodec/avcodec.h:3852) [-Wdeprecated-declarations]
LoadSettings.cpp: In member function ‘void CS::CommercialSkipper::LoadCutScene(const string&)’:
LoadSettings.cpp:1556:43: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
LoadSettings.cpp: In member function ‘void CS::CommercialSkipper::loadSettings(int, char**)’:
LoadSettings.cpp:54:10: warning: ignoring return value of ‘int nice(int)’, declared with attribute warn_unused_result [-Wunused-result]
make[1]: *** [comskip-LoadSettings.o] Error 1
comskip.cpp: In member function ‘int CS::CommercialSkipper::inputReffer(const char*, bool)’:
comskip.cpp:12460:32: warning: ignoring return value of ‘char* fgets(char*, int, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
comskip.cpp:12480:32: warning: ignoring return value of ‘char* fgets(char*, int, FILE*)’, declared with attribute warn_unused_result [-Wunused-result]
make[1]: *** [comskip-mpeg2dec.o] Error 1
comskip.cpp: In member function ‘bool CS::CommercialSkipper::outputCleanMpg()’:
comskip.cpp:7349:46: warning: ignoring return value of ‘ssize_t write(int, const void*, size_t)’, declared with attribute warn_unused_result [-Wunused-result]
comskip.cpp:7351:72: warning: ignoring return value of ‘ssize_t write(int, const void*, size_t)’, declared with attribute warn_unused_result [-Wunused-result]
comskip.cpp: In member function ‘void CS::CommercialSkipper::openOutputFiles()’:
comskip.cpp:5829:22: warning: ignoring return value of ‘char* getcwd(char*, size_t)’, declared with attribute warn_unused_result [-Wunused-result]
comskip.cpp:5862:22: warning: ignoring return value of ‘char* getcwd(char*, size_t)’, declared with attribute warn_unused_result [-Wunused-result]
make[1]: *** [comskip-comskip.o] Error 1
make[1]: Leaving directory `/var/tmp/comskip-0.92c'
make: *** [all] Error 2

Marc

RE: Comskip for Linux released - cbxbiker61 - 2013-01-08

0.92d should take care of that, as well as eliminating some of the other warnings (g++ 4.7 doesn't post some of those warning).

RE: Comskip for Linux released - Prof Yaffle - 2013-01-08

(2013-01-08, 00:06)cbxbiker61 Wrote: Theoretically H.264 should work from mkv files. I haven't tested it though, since most of the source material starts as mpeg. H264 would be a likely target after running comskip, i.e. start with mpeg, locate commercials, then transcode to h264 with commercials removed.

The H.264 comment was part jest, part serious - I'll test to see if your code does happen to support this through whatever library calls you're using, though. HD recordings (on both UK DVB-S2 and -T2) use this by default, otherwise yes, you're right - you'd Handbrake the files as a final step.

RE: Comskip for Linux released - MarcAngelo - 2013-01-09

This is weird...

I've processed an already recorded .mkv with 0.92d.
No matter if I use offset 0.0 or 30.0, the skip is done on the same position.
It also skips to the same position. Seems like the offset makes no difference?

I run the latest 11.0 "Eden" version, where the "30 second bug" for [b].edl[/b] files is already fixed.

Marc

RE: Comskip for Linux released - Prof Yaffle - 2013-01-09

There's a fix out for the 30s bug in tvheadend, though you'll need to download and compile the master branch yourself to get it...

https://www.lonelycoder.com/redmine/issues/1495

RE: Comskip for Linux released - dhaskwn - 2013-01-10

This version doesn't seem to detect commercials as well as the windows version under wine. (I'm getting ~8min of commercials with wine and ~1min with this version on the same file).

Were there major changes to the code?
Are other's seeing something similar?
The ini files seem smilar between the two versions, is there anything I can do to improve performance?

Also somewhat off-topic, I don't have commerical skpping working in XBMC frodo under live-tv->recordings, only when I create a share to the recorded tv folder. Is anyone else seeing something similar?

RE: Comskip for Linux released - schumi2004 - 2013-01-10

Anyone tried to run this on a NAS and do i need a updated TVH for it?
I own a Synology NAS and can't upgrade TVH because of GCC limitations from Synology toolchain.
Currently i'm running TVH 3.2.18 and if i need to upgrade this to get Comskip working then it's a no-go for me unfortunately

RE: Comskip for Linux released - MarcAngelo - 2013-01-10

(2013-01-09, 23:13)Prof Yaffle Wrote: There's a fix out for the 30s bug in tvheadend, though you'll need to download and compile the master branch yourself to get it...

I downloaded the masterbranch, but unfortunatly it's unable to compile due to errors during compilation:

______________________________________________________________________

# ./configure --datadir=/usr/share
checking for cc execinfo.h ... ok
checking for cc -mmmx ... ok
checking for cc -msse2 ... ok
checking for cc getloadavg ... ok
checking for py module gzip ... ok
checking for bzip2 ... ok
checking for pkg openssl ... ok
checking for pkg zlib ... ok
checking for pkg avahi-client ... ok
checking for pkg libavcodec <=55.0.0 ... ok
checking for pkg libavcodec >=52.96.0 ... ok
checking for pkg libavutil >=50.43.0 ... ok
checking for pkg libavformat <=55.0.0 ... ok
checking for pkg libavformat >=50.43.0 ... ok
Fetching dvb-scan files... done
checking for pkg libcurl ... ok

Compiler:
Using C compiler: cc
Build for arch: i686

Binaries:
Using PYTHON: python

Options:
cwc: yes
v4l: yes
linuxdvb: yes
dvbscan: yes
timeshift: yes
imagecache: yes
avahi: yes
zlib: yes
libav: yes
bundle: no
dvbcsa: no
execinfo: yes
mmx: yes
sse2: yes
getloadavg: yes
py_gzip: yes
bin_bzip2: yes
ssl: yes

Packages:
openssl: 1.0.0e
zlib: 1.2.3.4
avahi-client: 0.6.30
libavcodec: 53.6.0
libavutil: 51.7.0
libavformat: 53.3.0
libcurl: 7.21.6

Installation paths:
Prefix: /usr/local
Binaries: ${prefix}/bin
Libraries: ${prefix}/lib
Data files: /usr/share
Man pages: ${datadir}/man

Final Binary:
/usr/src/tvheadend-master/build.linux/tvheadend

Tvheadend Data Directory:
/usr/share/tvheadend

______________________________________________________________________

# make
CC build.linux/src/main.o
CC build.linux/src/utils.o
CC build.linux/src/wrappers.o
CC build.linux/src/version.o
CC build.linux/src/access.o
CC build.linux/src/dtable.o
CC build.linux/src/tcp.o
CC build.linux/src/http.o
CC build.linux/src/notify.o
CC build.linux/src/file.o
CC build.linux/src/epg.o
CC build.linux/src/epgdb.o
CC build.linux/src/epggrab.o
CC build.linux/src/spawn.o
CC build.linux/src/packet.o
CC build.linux/src/streaming.o
CC build.linux/src/teletext.o
CC build.linux/src/channels.o
CC build.linux/src/subscriptions.o
CC build.linux/src/service.o
CC build.linux/src/psi.o
CC build.linux/src/parsers.o
CC build.linux/src/parser_h264.o
CC build.linux/src/parser_latm.o
CC build.linux/src/tsdemux.o
CC build.linux/src/bitstream.o
CC build.linux/src/htsp_server.o
CC build.linux/src/serviceprobe.o
CC build.linux/src/htsmsg.o
CC build.linux/src/htsmsg_binary.o
CC build.linux/src/htsmsg_json.o
CC build.linux/src/htsmsg_xml.o
CC build.linux/src/misc/dbl.o
CC build.linux/src/misc/json.o
CC build.linux/src/settings.o
CC build.linux/src/htsbuf.o
CC build.linux/src/trap.o
CC build.linux/src/avg.o
CC build.linux/src/htsstr.o
CC build.linux/src/rawtsinput.o
CC build.linux/src/iptv_input.o
CC build.linux/src/avc.o
CC build.linux/src/huffman.o
CC build.linux/src/filebundle.o
CC build.linux/src/config2.o
CC build.linux/src/lang_codes.o
CC build.linux/src/lang_str.o
CC build.linux/src/imagecache.o
CC build.linux/src/epggrab/module.o
CC build.linux/src/epggrab/channel.o
CC build.linux/src/epggrab/module/pyepg.o
CC build.linux/src/epggrab/module/xmltv.o
CC build.linux/src/plumbing/tsfix.o
CC build.linux/src/plumbing/globalheaders.o
CC build.linux/src/dvr/dvr_db.o
CC build.linux/src/dvr/dvr_rec.o
CC build.linux/src/dvr/dvr_autorec.o
CC build.linux/src/webui/webui.o
CC build.linux/src/webui/comet.o
CC build.linux/src/webui/extjs.o
CC build.linux/src/webui/simpleui.o
CC build.linux/src/webui/statedump.o
CC build.linux/src/webui/html.o
CC build.linux/src/muxer.o
CC build.linux/src/muxer/muxer_pass.o
CC build.linux/src/muxer/muxer_tvh.o
CC build.linux/src/muxer/tvh/ebml.o
CC build.linux/src/muxer/tvh/mkmux.o
CC build.linux/src/epggrab/otamux.o
CC build.linux/src/epggrab/module/eit.o
CC build.linux/src/epggrab/module/opentv.o
CC build.linux/src/epggrab/support/freesat_huffman.o
CC build.linux/src/timeshift.o
CC build.linux/src/timeshift/timeshift_filemgr.o
CC build.linux/src/timeshift/timeshift_writer.o
CC build.linux/src/timeshift/timeshift_reader.o
CC build.linux/src/dvb/dvb.o
CC build.linux/src/dvb/dvb_support.o
CC build.linux/src/dvb/dvb_charset.o
CC build.linux/src/dvb/dvb_fe.o
CC build.linux/src/dvb/dvb_tables.o
CC build.linux/src/dvb/diseqc.o
CC build.linux/src/dvb/dvb_adapter.o
CC build.linux/src/dvb/dvb_multiplex.o
CC build.linux/src/dvb/dvb_service.o
CC build.linux/src/dvb/dvb_preconf.o
CC build.linux/src/dvb/dvb_satconf.o
CC build.linux/src/dvb/dvb_input_filtered.o
CC build.linux/src/dvb/dvb_input_raw.o
CC build.linux/src/webui/extjs_dvb.o
CC build.linux/src/muxes.o
CC build.linux/src/v4l.o
CC build.linux/src/webui/extjs_v4l.o
CC build.linux/src/avahi.o
CC build.linux/src/libav.o
CC build.linux/src/muxer/muxer_libav.o
/usr/src/tvheadend-master/src/muxer/muxer_libav.c: In function ‘lav_muxer_add_stream’:
/usr/src/tvheadend-master/src/muxer/muxer_libav.c:69:3: error: implicit declaration of function ‘avformat_new_stream’ [-Werror=implicit-function-declaration]
/usr/src/tvheadend-master/src/muxer/muxer_libav.c:69:6: error: assignment makes pointer from integer without a cast [-Werror]
cc1: all warnings being treated as errors

make: *** [build.linux/src/muxer/muxer_libav.o] Error 1
#
______________________________________________________________________

Could you point me in the right direction?

Thanks in advance!

Marc

RE: Comskip for Linux released - cbxbiker61 - 2013-01-11

(2013-01-10, 22:10)schumi2004 Wrote: Anyone tried to run this on a NAS and do i need a updated TVH for it?
I own a Synology NAS and can't upgrade TVH because of GCC limitations from Synology toolchain.
Currently i'm running TVH 3.2.18 and if i need to upgrade this to get Comskip working then it's a no-go for me unfortunately

Tvheadend 3.2.18 works great for me. Configure tvheadend to output to .ts files.

I've compiled and tested my comskip versions on an armv7l (CuBox) system. It compiles clean and seems to work fine (much slower than on my FX-8350 box).

RE: Comskip for Linux released - Prof Yaffle - 2013-01-11

@MarcAngelo - I had the same problem - it's because (I suspect) your ffmpeg version is too old. If you don't fancy updating system-wide, or pointing tvheadend to the updated version you inevitably installed while compiling comskip, try ./configure --disable-libav instead before make. I don't know what this really leaves out, I think it's some new muxing code.

@schumi2004 - you don't need to do anything with tvheadend, comskip is standalone with no dependencies between them. You could run them on separate boxes if you wished...

RE: Comskip for Linux released - MarcAngelo - 2013-01-11

(2013-01-11, 00:51)Prof Yaffle Wrote: @MarcAngelo - I had the same problem - it's because (I suspect) your ffmpeg version is too old. If you don't fancy updating system-wide, or pointing tvheadend to the updated version you inevitably installed while compiling comskip, try ./configure --disable-libav instead before make. I don't know what this really leaves out, I think it's some new muxing code.

Well...
I was able to compile tvheadend using the --disable-libav, but I did found something that's changed... Smile

In my current version I had the following in the webinterface Tab "Configuration" >> "Digital Video Recorder".

For "Media container", I could select "Matroska" or "TS".
With the new compile version I have now "Matroska" and "Same as Source (Pass through)", but a test-recording shows
that "Same as Source (Pass through)" creates a .TS file Wink

Marc

(2012-12-30, 21:10)cbxbiker61 Wrote: If your system ffmpeg is older than 0.9:

Code:
cd /var/tmp mkdir ComSkipWork wget http://ffmpeg.org/releases/ffmpeg-1.0.1.tar.bz2 tar xf ffmpeg-1.0.1.tar.bz2 cd ffmpeg-1.0.1 ./configure --prefix=/var/tmp/ComSkipWork/ffmpeg-1.0.1-install make -j8 make install cd .. wget http://www.xilka.com/xilka/source/comskip-0.92d.tar.xz tar xf comskip-0.92d.tar.xz cd comskip-0.92d PKG_CONFIG_PATH=/var/tmp/ComSkipWork/ffmpeg-1.0.1-install/lib/pkgconfig ./configure --prefix=/usr make -j8 sudo make install

@Prof Yaffle:
Do you think it might be possible to compile tvheadend against ffmpeg like comskip shows for ffmpeg < 0.9 in the first post?
This is what I used to get Comskip compiled.
Perhaps an option to get it compiled without --disable-libav ??

Marc