• 1
  • 62
  • 63
  • 64(current)
  • 65
  • 66
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings
just discovered this script as i was looking for a solution to watch my myth recordings via xbmc without resorting to any PVR build or mythbox, etc. tried those and never really liked them. just moved and no longer have access to analog for my old PVR-250 and PVR-500 tuners (was using comcast DTA's before) so my myth system is effectively useless now but is still working superbly as a mysql and squeezebox server.

anyways here's the problem i'm running into, got all my recordings moved via symlinks and imported into my XBMC libraries. on my older Eden box the XBMC notifications works, but my Frodo RC2 box the notifications fail. i have all the same settings on both (HTTP on port 81). is this a known issue i've overlooked? i believe i read somewhere that there have been some changes in this regard between the two versions.
Reply
Just browsed the source code-See Below:
It appears the web server HTTP Api was deprecated
http://wiki.xbmc.org/index.php?title=Web...r_HTTP_API

This section needs to be rewritten to use the new JSON RPS Interface as that is all that works in Frodo... I may take a crack at it tonight and see what I come up with.
http://wiki.xbmc.org/index.php?title=JSON_RPC


#####XBMC COMMUNICATIONS#####
#This function Automates communication with XBMC
XBMCAutomate () {
#Send notification to XBMC, Update Library, Clean Library

if [ "$XBMCNotify" = "Enabled" -o "$XBMCUpdate" = "Enabled" -o "$XBMCClean" = "Enabled" ]; then
for XBMCIP in ${XBMCIPs[@]}
do
echo "SENDING REQUESTED COMMANDS TO:$XBMCIP"
test "$XBMCNotify" = "Enabled" && curl -L -s -m3 --connect-timeout 5 "http://"$XBMCIP"/xbmcCmds/xbmcHttp?command=ExecBuiltIn(Notification(mythical%20Librarian%2Cadding%20show%20$tvdbshowname%20to%20library))" > /dev/null 2>&1
test "$XBMCUpdate" = "Enabled" && curl -L -s -m3 --connect-timeout 5 "http://"$XBMCIP"/xbmcCmds/xbmcHttp?command=ExecBuiltIn(UpdateLibrary(video))" > /dev/null 2>&1
test "$XBMCClean" = "Enabled" && curl -L -s -m3 --connect-timeout 5 "http://"$XBMCIP"/xbmcCmds/xbmcHttp?command=ExecBuiltIn(CleanLibrary(video))" > /dev/null 2>&1
done
fi
return 0
}


Reply
(2013-01-17, 23:13)gthompson20 Wrote: Just browsed the source code-See Below:
It appears the web server HTTP Api was deprecated
http://wiki.xbmc.org/index.php?title=Web...r_HTTP_API

This section needs to be rewritten to use the new JSON RPS Interface as that is all that works in Frodo... I may take a crack at it tonight and see what I come up with.
http://wiki.xbmc.org/index.php?title=JSON_RPC


#####XBMC COMMUNICATIONS#####
#This function Automates communication with XBMC
XBMCAutomate () {
#Send notification to XBMC, Update Library, Clean Library

if [ "$XBMCNotify" = "Enabled" -o "$XBMCUpdate" = "Enabled" -o "$XBMCClean" = "Enabled" ]; then
for XBMCIP in ${XBMCIPs[@]}
do
echo "SENDING REQUESTED COMMANDS TO:$XBMCIP"
test "$XBMCNotify" = "Enabled" && curl -L -s -m3 --connect-timeout 5 "http://"$XBMCIP"/xbmcCmds/xbmcHttp?command=ExecBuiltIn(Notification(mythical%20Librarian%2Cadding%20show%20$tvdbshowname%20to%20library))" > /dev/null 2>&1
test "$XBMCUpdate" = "Enabled" && curl -L -s -m3 --connect-timeout 5 "http://"$XBMCIP"/xbmcCmds/xbmcHttp?command=ExecBuiltIn(UpdateLibrary(video))" > /dev/null 2>&1
test "$XBMCClean" = "Enabled" && curl -L -s -m3 --connect-timeout 5 "http://"$XBMCIP"/xbmcCmds/xbmcHttp?command=ExecBuiltIn(CleanLibrary(video))" > /dev/null 2>&1
done
fi
return 0
}

Added to the repository. Please test with SVN 950
(2012-11-11, 05:06)sdsnyr94 Wrote:
Quote:I don't know what handbrake does. You can launch handbrake as a mythicalLibraian job if you'd like. User Jobs are documented in the wiki with it's own page.

I am running Handbrake as a JobSuccessfull job..... My issue was that with 'SYMLINK=Disabled', all the Database info was being removed PRIOR to the launch of the User Job... so according to MythBackend there was no longer a MythTV User Job running (because you have now just removed all information about that program), and it would begin the next MythTV User Job.

Is there a way that you can delay the removal of DataBase information until after the JobSuccessfull user job has completed?

The Symlink=Disabled is kinda a depricated feature. There is a script called MythDataDeleter. http://www.mythtv.org/wiki/MythDataDeleter We wrote that right after mythicalLibrarian switched to MythDataGrabber. MythDataDeleter will do what you want.
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply
I just tried to install mythicalLibrarian on my mac mini running OSX 10.6.8 an MythTV 0.25, with XBMC Frodo. Since my MythTV install is from macports with everything rooted in /opt/local, the install was a total failure.

I can tell you that SVN 950 failed to communicate with XBMC 12.0, and I double checked my webserver being on and able to be controlled remotely.

I think the paths and python bindings for the macports install could be fixed, but it will be involved. If you have time and interest for this, let me know and I will help.
Reply
(2013-01-19, 08:17)outleradam Wrote: The Symlink=Disabled is kinda a depricated feature. There is a script called MythDataDeleter. http://www.mythtv.org/wiki/MythDataDeleter We wrote that right after mythicalLibrarian switched to MythDataGrabber. MythDataDeleter will do what you want.

So, Symlink=Disabled doesn't seem to be deleting my database entries.

How do I go about implementing MythDataDeleter?
Reply
(2013-03-06, 08:56)ubuntite Wrote:
(2013-01-19, 08:17)outleradam Wrote: The Symlink=Disabled is kinda a depricated feature. There is a script called MythDataDeleter. http://www.mythtv.org/wiki/MythDataDeleter We wrote that right after mythicalLibrarian switched to MythDataGrabber. MythDataDeleter will do what you want.

So, Symlink=Disabled doesn't seem to be deleting my database entries.

How do I go about implementing MythDataDeleter?

In case this might help someone, this is what I did.

My goal was to:
  1. move a recorded show to my NAS
  2. transcode it to an x264 .mp4 & delete the .mpg
  3. delete the related .txt and .nfo files on the NAS that get generated for the show
  4. delete the show from the myth database
  5. delete the show's thumbnails from the myth recordings directory
The first this I did was to make sure to set Symlink=Move in mythicalLibrarian.

Next I copied this code into /usr/local/bin/MythDataDeleter and did a sudo chmod 755 on it.

Then I inserted the following code into mythicalLibrarian directly underneath "#Insert Custom User Job here" within the "#Successful Completion of mythicalLibrarian" section of the "### USER JOBS ###" section.

Code:
ffmpeg -i "$MoveDir/$ShowFileName.$OriginalExt" -vcodec libx264 -vpre veryfast -deinterlace "$MoveDir/$ShowFileName.mp4"
rm "$MoveDir/$ShowFileName.$OriginalExt"
rm "$MoveDir/$ShowFileName".txt
rm "$MoveDir/$ShowFileName".nfo
/usr/local/bin/MythDataDeleter --filename="$InputFileBasename" --DBPin="$DBPin" --DBHostName="$DBHostName" --DBName="$MySQLMythDb" --DBUserName="$MySQLuser" --DBPassword="$MySQLpass"
rm "$originaldirname/$FileBaseName".*

The next thing I would like to accomplish is to automatically remove the commercial breaks when I do the transcoding. Can anyone give me some assistance? Thanks.
Reply
(2013-03-07, 22:24)ubuntite Wrote: The next thing I would like to accomplish is to automatically remove the commercial breaks when I do the transcoding. Can anyone give me some assistance? Thanks.

Thanks for posting that ubuntite, very interesting.

For commercial removal, mythtranscode worked great for me using a version of this User Job script in 0.25, but was broken by the 0.26 version of mythtranscode (see this bug).

Other people have been using Lossless Cut in 0.26 to do the cutting and exporting, but it reportedly has some issues with HDHomeRun recordings, and since that is what I use for a tuner, I haven't tried to configure Lossless Cut on my system.

Since you seem to know what you are doing more than I do, maybe one of those two options will work for you.

/Greg
Reply
(2013-03-07, 22:38)GregoryK Wrote:
(2013-03-07, 22:24)ubuntite Wrote: The next thing I would like to accomplish is to automatically remove the commercial breaks when I do the transcoding. Can anyone give me some assistance? Thanks.

Thanks for posting that ubuntite, very interesting.

For commercial removal, mythtranscode worked great for me using a version of this User Job script in 0.25, but was broken by the 0.26 version of mythtranscode (see this bug).

Other people have been using Lossless Cut in 0.26 to do the cutting and exporting, but it reportedly has some issues with HDHomeRun recordings, and since that is what I use for a tuner, I haven't tried to configure Lossless Cut on my system.

Since you seem to know what you are doing more than I do, maybe one of those two options will work for you.

/Greg

I briefly tried removecommercials, but didn't have any luck. I took into consideration the syntax changes of mythcommflag and mythtranscode (-c to --chanid and -s to --starttime), and used mythutil instead of mythcommflag where appropriate.

Anyone have removecommercials working with 0.25?
Reply
I made a few changes to the RemoveCommercials script, I'll share it here for a while -> http://ubuntuone.com/5VVhAYIp9V0M24yMwxGZPx

I had trouble with the original script not playing nice if running multiple user jobs, so I added nice-ness to the process. I also changed it so that the original file is overwritten instead of being renamed .old. Maybe some other minor changes, can't remember, it's been a while.

I run this script as a MythTV userjob before mythicalLibrarian runs.
Code:
/path/to/commercialremover.sh %DIR% %FILE% %CHANID% %STARTTIME%

I do commercial flagging when the recording starts so, if you want to use this script to run it you will have to un-comment the commercial flagging lines.

On my hardware it takes about five minuets for a thirty minuet recording to complete and file size is reduced by about a third.

As GregoryK said MythTV 0.26 broke the mythtranscode Angry that was a deal breaker for me, sticking with 0.25.
Reply
(2013-03-09, 15:48)divingmule Wrote: I made a few changes to the RemoveCommercials script, I'll share it here for a while -> http://ubuntuone.com/5VVhAYIp9V0M24yMwxGZPx

I had trouble with the original script not playing nice if running multiple user jobs, so I added nice-ness to the process. I also changed it so that the original file is overwritten instead of being renamed .old. Maybe some other minor changes, can't remember, it's been a while.

I run this script as a MythTV userjob before mythicalLibrarian runs.
Code:
/path/to/commercialremover.sh %DIR% %FILE% %CHANID% %STARTTIME%

I do commercial flagging when the recording starts so, if you want to use this script to run it you will have to un-comment the commercial flagging lines.

On my hardware it takes about five minuets for a thirty minuet recording to complete and file size is reduced by about a third.

As GregoryK said MythTV 0.26 broke the mythtranscode Angry that was a deal breaker for me, sticking with 0.25.

Thanks for this. I was excited to try your script to see if I would have more luck than with the version I was using. I more closely followed the action this time and noticed a very strange thing. The "Flag Commercials" job that I set to run when the recording starts actually ended before the recording finished.

I got a bunch of these errors:
Code:
Decoder ringbuffer.cpp:1023 (WaitForReadsAllowed) RingBuf(/var/lib/mythtv/recordings/1003_20130309141500.mpg): Taking too long to be allowed to read..

followed by bunch of these:
Code:
Decoder avformatdecoder.cpp:4396 (GetFrame) decoding error#012#011#011#011eno: Broken pipe (32)

I am running my myth-backend on a low-powered nettop with a bunch of other servers running in the backend. Could it be that I am running out of horsepower? Can I tune mythcommflag to be more patient?
Reply
I don't know, one might assume so :-) do you run the ffmpeg transcode on the same hardware? Maybe try setting the flagging to run after the recording ends or run it from the script.
Reply
(2013-03-10, 16:00)divingmule Wrote: I don't know, one might assume so :-) do you run the ffmpeg transcode on the same hardware? Maybe try setting the flagging to run after the recording ends or run it from the script.

Hmm... Doesn't seem to matter how I run mythcommflag, it fails. I tried running it after a recording, and I tried running it from the script, but each time it fails with a "decoding error" and a "broken pipe (32)" error.

Obviously this is a mythtv issue. I guess an update somewhere along the way broke mythcommflag. Anyone having similar issues?
Reply
Let me respond to my own posts and please forgive me for straying way off topic. I know this has nothing, or at most very little, to do with mythicalLibrarian. The commercialremover script is the user job that I run immediately before mythicalLibrarian.

My problems with flagging commercials seem to be due to either a non-existant or a corrupt seektable. It seems that because I am running a low-powered nettop with a bunch of services and servers, creation of the seektable may not be reliable.

My solution is to make the following changes to the commercialremover script. I added

Code:
mythcommflag --chanid $CHAN --starttime $START --rebuild

(which (re)builds the seektable) immediately before

Code:
mythcommflag --chanid $CHAN --starttime $START --method 7

(which actually flags commercials) in the commercialremover script . I do not flag commercials while recording. You'll note that I explicitly state the commercial flagging method (7=all), so as not to have to rely on this parameter being correctly set within the mythbackend setup.

Next, to get the commercialremover script running properly under myth 0.25, I had to make a change to the line responsible for transcoding, from

Code:
mythtranscode --honorcutlist --showprogress -i $VIDEODIR/$FILENAME -o $VIDEODIR/$FILENAME.tmp

to

Code:
mythtranscode --chanid $CHAN --starttime $START --mpeg2 --honorcutlist --showprogress --outfile $VIDEODIR/$FILENAME.tmp

Not only did I need to change how the recording was passed to the script, but it turned out to be important to specify the "--mpeg2" parameter (which initiates a lossless transcode) if you hadn't previously specified a transcode profile in the myth backed (which I hadn't).

I hope this helps someone.
Reply
It looks like an episode with a ":" (colon) in its name causes mythicalLibrarian to crap out when trying to move the file.

Is this a known problem? Is there a fix?
Reply
My MythicalLibrarian setup appears to work with most of my programs, but fails on a handful. Some of the failures are on "news" shows, which might be a systemic problem however, it fails processing America's Test Kitchen -- which I think should be working. Here is the relative part of output.log

Code:
SEARCHING: www.TheTvDb.com SHOW NAME: America's Test Kitchen From Cook's Illustrated EPISODE: Skewered and Wrapped
...
###################DEBUG MODE ENABLED####################
...
CONFIDENCE:0 LOGIC PATH:  Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Failed; Attemted Title match on Fuzzy Logic: Failed; Information not obtainable; Could not set Sxx and Exx, Failed;  Guide Data did not match showing;
...

PROGRAMID:EP005502520281- CHANNEL ID:3131- CATEGORY:How-to- GOFORDOOVER:0-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0.0 RATING:0
ZAP2IT SERIES ID:550252- MATCHED TVDB SERIES ID:-
PLOT: Grilled beef satay; saute pans; Chinese chicken lettuce wraps.
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB

Could the problem be that the TVDB doesn't have a Zap2It ID for America's Test Kitchen?

I've checked the Schedules Direct information in the MythTV guide, and it shows "Episode Information" correctly (I think) as "1320" (with the numbers changing based on the episode). Isn't that what ML uses to identify the episode?

These showings don't match the "original" run dates (as they are reruns), but ML works with my other re-running shows. Any ideas?
Reply
  • 1
  • 62
  • 63
  • 64(current)
  • 65
  • 66

Logout Mark Read Team Forum Stats Members Help
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings3