Release - myth2kodi -- generate Kodi friendly file-names from MythTV recordings.

  Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #46
(2017-03-01 16:37)jctennis Wrote:  The new version is working great.

I was thinking it might be a good idea to include some type of installer script in the download so everything could be done quickly instead of having to manually do every install step with each upgrade. I took a swing at it but I couldn't program my way out of a paper bag so I'll post what I got so far here so you can take a look. My biggest headache is getting sed to do what I want it to do. It is horribly rudimentary and has no error checking to speak of but it is my best attempt to help:
Code:
#!/bin/bash
# Ask the user for their m2kdir directory
echo What Directory do you use for your m2kdir?
read var1
#Replace m2kdir default with contents of variable
sed -i "s|$HOME/.myth2kodi|$var1|g" myth2kodi #This command does NOT work currently because I did something wrong. If I delete the $ before HOME it works but leaves the $ behind, runing the path provided by the variable
# Copy all scripts to proper directory
sudo cp myth2kodi /usr/local/bin/
sudo cp bashlogging /usr/local/bin/
sudo cp mythdb_access /usr/local/bin/
sudo cp m2k_notify /usr/local/bin/
#Set Permissions
sudo chmod o+rx /usr/local/bin/myth2kodi
sudo chmod o+rx /usr/local/bin/mythdb_access
sudo chmod o+rx /usr/local/bin/m2k_notify

Yeah, I have considered this... The reason that I have not, so far,
added an install script is that adding one that works robustly will
require a lot of testing and checking. I will most likely do this
eventually, but for the moment I have things I want to focus on that
I consider higher priority.

From experience I know that a simple script, like the one you propose,
will cause more problems than it solves -- at least, if I officially
add it to the myth2kodi repository and people use it blindly. That
being said, I understand the desire to automate the process. So, for
the sed in your script, you were close, just off with the quoting a
bit, what you want is:
Code:
sed -i 's|$HOME/.myth2kodi|'"$var1"'|g' myth2kodi
Single quotes prevent the expansion of parameters, so '$HOME' will remain
'$HOME' instead of expanding to '/home/mythtv' or whatever. You want
var1 to expand, so it is in double quotes "$var1". The only other thing
to know is that if you put strings against one another with no spaces
then they basically get treated like one string, that is:
'str1'"str2"'str3' == 'str1str2str3'.

Though, to avoid replacing the default value in the comment line above the
m2kdir= expression, you could be more specific and including the m2kdir=
in your pattern, so:
Code:
sed -i 's|m2kdir="$HOME/.myth2kodi"|m2kdir="'"$var1"'"|g' myth2kodi
will only substitute the setting of m2kdir. The key thing to know here is
that if you put single quotes inside double quotes, or double quotes inside
single quotes, then the quotes inside are just treated as characters in your
string and do not have a special meaning. So the double quotes in
m2kdir="$HOME/.myth2kodi" are just part of the string and do not
cause parameter expansion. Whereas with m2kdir="'"$var1"'", the double
quotes closest to $var1 allow parameter expansion and disappear while
the next set of double quotes are inside the adjacent single quote blocks and
so remain part of the string being substituted.
(This post was last modified: 2017-03-02 07:28 by stuartk.)
find quote
jctennis Offline
Senior Member
Posts: 141
Joined: Feb 2013
Reputation: 7
Post: #47
(2017-03-02 07:17)stuartk Wrote:  
(2017-03-01 16:37)jctennis Wrote:  The new version is working great.

I was thinking it might be a good idea to include some type of installer script in the download so everything could be done quickly instead of having to manually do every install step with each upgrade. I took a swing at it but I couldn't program my way out of a paper bag so I'll post what I got so far here so you can take a look. My biggest headache is getting sed to do what I want it to do. It is horribly rudimentary and has no error checking to speak of but it is my best attempt to help:
Code:
#!/bin/bash
# Ask the user for their m2kdir directory
echo What Directory do you use for your m2kdir?
read var1
#Replace m2kdir default with contents of variable
sed -i "s|$HOME/.myth2kodi|$var1|g" myth2kodi #This command does NOT work currently because I did something wrong. If I delete the $ before HOME it works but leaves the $ behind, runing the path provided by the variable
# Copy all scripts to proper directory
sudo cp myth2kodi /usr/local/bin/
sudo cp bashlogging /usr/local/bin/
sudo cp mythdb_access /usr/local/bin/
sudo cp m2k_notify /usr/local/bin/
#Set Permissions
sudo chmod o+rx /usr/local/bin/myth2kodi
sudo chmod o+rx /usr/local/bin/mythdb_access
sudo chmod o+rx /usr/local/bin/m2k_notify

Yeah, I have considered this... The reason that I have not, so far,
added an install script is that adding one that works robustly will
require a lot of testing and checking. I will most likely do this
eventually, but for the moment I have things I want to focus on that
I consider higher priority.

From experience I know that a simple script, like the one you propose,
will cause more problems than it solves -- at least, if I officially
add it to the myth2kodi repository and people use it blindly. That
being said, I understand the desire to automate the process. So, for
the sed in your script, you were close, just off with the quoting a
bit, what you want is:
Code:
sed -i 's|$HOME/.myth2kodi|'"$var1"'|g' myth2kodi
Single quotes prevent the expansion of parameters, so '$HOME' will remain
'$HOME' instead of expanding to '/home/mythtv' or whatever. You want
var1 to expand, so it is in double quotes "$var1". The only other thing
to know is that if you put strings against one another with no spaces
then they basically get treated like one string, that is:
'str1'"str2"'str3' == 'str1str2str3'.

Though, to avoid replacing the default value in the comment line above the
m2kdir= expression, you could be more specific and including the m2kdir=
in your pattern, so:
Code:
sed -i 's|m2kdir="$HOME/.myth2kodi"|m2kdir="'"$var1"'"|g' myth2kodi
will only substitute the setting of m2kdir. The key thing to know here is
that if you put single quotes inside double quotes, or double quotes inside
single quotes, then the quotes inside are just treated as characters in your
string and do not have a special meaning. So the double quotes in
m2kdir="$HOME/.myth2kodi" are just part of the string and do not
cause parameter expansion. Whereas with m2kdir="'"$var1"'", the double
quotes closest to $var1 allow parameter expansion and disappear while
the next set of double quotes are inside the adjacent single quote blocks and
so remain part of the string being substituted.
Thanks. I figured my script was way too simplistic to be included. I'll keep it on my end so I can just pop it in the folder when I update. Speaking of which, is it safe to just do a git pull when you release a new update rather than go through the download tarball and extract song and dance?

Really appreciate the help with sed. I couldn't figure out how to deal with that variable and still handle the rest. I have been using Linux exclusively for more than a decade but I am a total script kiddie. I never bothered to learn more of the robust command line features because honestly I have never really had much need to. I can usually find something existing and just build on it but I was at a loss with this one.

Anyway, whenever you decide to give an install script a whack I am more than happy to do some error testing from the *buntu neck of the woods.

Sent from my Nexus 6 (typie typie)

[Image: Forum_Signature.jpg]
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #48
(2017-03-02 14:14)jctennis Wrote:  Thanks. I figured my script was way too simplistic to be included. I'll keep it on my end so I can just pop it in the folder when I update. Speaking of which, is it safe to just do a git pull when you release a new update rather than go through the download tarball and extract song and dance?

Short answer is yes, but there is a caveat.
You need to be a little careful in how you go about it, because there are two potential issues to avoid (more on that below), you want an approach something like:
Code:
#Make sure you are in master
git checkout master

#Update
git pull

#Check for the latest release name
git tag --list

#Create a branch from that release tag and switch to it, currently
git checkout tags/v1.3.1 -b jct-v1.3.1

#Run your install script from the release branch you created
...

#Discard or commit the changes your script made on this branch, to discard:
git checkout -- myth2kodi

There are two things you are trying to avoid with this approach:

1) I will occasionally make a release when I know things are stable and I am about to do something that has a good chance of breaking stuff and will require thorough testing. So, shortly after a release, with shortly measured in minutes to hours, the master branch will have code that is likely to have issues. This is why you want to access a release tag rather than just using the current state of the repository.

2) The way your script (at least the version you posted) is written the sed does its work in-place, and so if you run it directly in master you will have changed master's version of myth2kodi, leading to an increased risk of encountering merge conflicts and such when you next want to pull. This is why, even if you want the latest repo version and not a release, you should create and switch to a branch other than master before running your script.
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #49
(2017-03-02 14:14)jctennis Wrote:  Anyway, whenever you decide to give an install script a whack I am more than happy to do some error testing from the *buntu neck of the woods.

Your suggestion of an install script became an itch in the back of my mind...
So, if you are still open to doing some testing it is in the repo now.

While it works safely on my system for all the cases I could think to check, just to be extra safe, before testing it is probably a good idea that you back-up your myth2kodi working directory -- or if possible test the install script on a different machine.

From the main myth2kodi directory of your clone of the repo, as the user who will be running myth2kodi, just run:
Code:
./install.sh
It should work from either a repo, where by default it will install the latest release, or from a release tar-ball.
As I have not made a release including it yet, to test from a tar-ball just copy the install.sh file from the repo into the unpacked release directory and run it from there.
It is interactive by default, there is also a non-interactive 'quick' install option that just does a default install.
The interactive mode allows you to, optionally, specify non-default install and working directories.
If it fails during install then any temporary files it created, including the install log, should be in /tmp.
After a successful install, the install log is moved to myth2kodi's working directory and, unless debugging is enabled, any other temporary files are deleted.

Let me know if anything is not clear, or you have any issues, or if by some miracle it "just works"...
find quote
jctennis Offline
Senior Member
Posts: 141
Joined: Feb 2013
Reputation: 7
Post: #50
(2017-03-08 03:03)stuartk Wrote:  
(2017-03-02 14:14)jctennis Wrote:  Anyway, whenever you decide to give an install script a whack I am more than happy to do some error testing from the *buntu neck of the woods.

Your suggestion of an install script became an itch in the back of my mind...
So, if you are still open to doing some testing it is in the repo now.

While it works safely on my system for all the cases I could think to check, just to be extra safe, before testing it is probably a good idea that you back-up your myth2kodi working directory -- or if possible test the install script on a different machine.

From the main myth2kodi directory of your clone of the repo, as the user who will be running myth2kodi, just run:
Code:
./install.sh
It should work from either a repo, where by default it will install the latest release, or from a release tar-ball.
As I have not made a release including it yet, to test from a tar-ball just copy the install.sh file from the repo into the unpacked release directory and run it from there.
It is interactive by default, there is also a non-interactive 'quick' install option that just does a default install.
The interactive mode allows you to, optionally, specify non-default install and working directories.
If it fails during install then any temporary files it created, including the install log, should be in /tmp.
After a successful install, the install log is moved to myth2kodi's working directory and, unless debugging is enabled, any other temporary files are deleted.

Let me know if anything is not clear, or you have any issues, or if by some miracle it "just works"...
I'll put it through it's paces tomorrow. I'll have some time. I certainly didn't expect anything this quickly. Kudos!

Sent from my Nexus 6 (typie typie)

[Image: Forum_Signature.jpg]
find quote
jctennis Offline
Senior Member
Posts: 141
Joined: Feb 2013
Reputation: 7
Post: #51
(2017-03-08 03:03)stuartk Wrote:  
(2017-03-02 14:14)jctennis Wrote:  Anyway, whenever you decide to give an install script a whack I am more than happy to do some error testing from the *buntu neck of the woods.

Your suggestion of an install script became an itch in the back of my mind...
So, if you are still open to doing some testing it is in the repo now.

While it works safely on my system for all the cases I could think to check, just to be extra safe, before testing it is probably a good idea that you back-up your myth2kodi working directory -- or if possible test the install script on a different machine.

From the main myth2kodi directory of your clone of the repo, as the user who will be running myth2kodi, just run:
Code:
./install.sh
It should work from either a repo, where by default it will install the latest release, or from a release tar-ball.
As I have not made a release including it yet, to test from a tar-ball just copy the install.sh file from the repo into the unpacked release directory and run it from there.
It is interactive by default, there is also a non-interactive 'quick' install option that just does a default install.
The interactive mode allows you to, optionally, specify non-default install and working directories.
If it fails during install then any temporary files it created, including the install log, should be in /tmp.
After a successful install, the install log is moved to myth2kodi's working directory and, unless debugging is enabled, any other temporary files are deleted.

Let me know if anything is not clear, or you have any issues, or if by some miracle it "just works"...
Sorry for the late response... I had to get into a few projects I didn't expect. The script seems to be working very well. The only problem I could foresee so far id if someone has a non-standard Librarian. I don't, but somewhere out there someone probably does. Otherwise, so far so good.

[Image: Forum_Signature.jpg]
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #52
(2017-03-10 16:08)jctennis Wrote:  Sorry for the late response... I had to get into a few projects I didn't expect. The script seems to be working very well. The only problem I could foresee so far id if someone has a non-standard Librarian. I don't, but somewhere out there someone probably does. Otherwise, so far so good.

No need to apologise, I appreciate the feedback.

The script does set the Librarian in the myth2kodi.conf it generates as part of the install (provided working directory set-up is requested), at least when I run it.
Is this not working on your system?
Or is there some other issue with a non-standard Librarian that I am missing?

If you already have a myth2kodi.conf, the install script does not overwrite the existing one but rather moves the install generated .conf to a file name like
Code:
$m2kdir/m2k_install_<DateAndTime>_myth2kodi.conf
and adds a warning to the install log pointing out that you need to check that your existing myth2kodi.conf is compatible with the new install.

So, with a fresh install, you should get a myth2kodi.conf that sets the Librarian user setting correctly. And when updating an existing install, one that has an existing myth2kodi.conf, you get a warning in the install log and a copy of the myth2kodi.conf that the install script would have installed placed in your working directory.
find quote
jctennis Offline
Senior Member
Posts: 141
Joined: Feb 2013
Reputation: 7
Post: #53
Ok... time for me to get really needy and annoying. Feel free to tell me to quit bugging you if I am asking too much.

I have damn near filled my 5TB hard drive with recordings (made oh so easy by this project!) and now I am going to need to get some transcoding action underway or I am going to have to start deleting things. I am playing with handbrake and it seems to do a better job of compressing my recordings than mythtranscode. My question is how to go about using handbrake to shrink my recordings and not screw up the simlinks and mythtv database you have preserved with your project.

[Image: Forum_Signature.jpg]
find quote
jctennis Offline
Senior Member
Posts: 141
Joined: Feb 2013
Reputation: 7
Post: #54
(2017-03-11 01:30)stuartk Wrote:  
(2017-03-10 16:08)jctennis Wrote:  Sorry for the late response... I had to get into a few projects I didn't expect. The script seems to be working very well. The only problem I could foresee so far id if someone has a non-standard Librarian. I don't, but somewhere out there someone probably does. Otherwise, so far so good.

No need to apologise, I appreciate the feedback.

The script does set the Librarian in the myth2kodi.conf it generates as part of the install (provided working directory set-up is requested), at least when I run it.
Is this not working on your system?
Or is there some other issue with a non-standard Librarian that I am missing?

If you already have a myth2kodi.conf, the install script does not overwrite the existing one but rather moves the install generated .conf to a file name like
Code:
$m2kdir/m2k_install_<DateAndTime>_myth2kodi.conf
and adds a warning to the install log pointing out that you need to check that your existing myth2kodi.conf is compatible with the new install.

So, with a fresh install, you should get a myth2kodi.conf that sets the Librarian user setting correctly. And when updating an existing install, one that has an existing myth2kodi.conf, you get a warning in the install log and a copy of the myth2kodi.conf that the install script would have installed placed in your working directory.

I re-ran it and it works as expected. I was just being bone-headed.

[Image: Forum_Signature.jpg]
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #55
(2017-03-11 05:32)jctennis Wrote:  Ok... time for me to get really needy and annoying. Feel free to tell me to quit bugging you if I am asking too much.

I have damn near filled my 5TB hard drive with recordings (made oh so easy by this project!) and now I am going to need to get some transcoding action underway or I am going to have to start deleting things. I am playing with handbrake and it seems to do a better job of compressing my recordings than mythtranscode. My question is how to go about using handbrake to shrink my recordings and not screw up the simlinks and mythtv database you have preserved with your project.


To your exact request, the short answer is that this is not something I ever figured out for myself, but then as I only watch recordings via Kodi I went a slightly different way.

Currently I use handbrake, because, as you say, it does a better job than mythtranscode, and also I was already familiar with Handbrake before I started using MythTV.
The approach I take is that, for recordings I know have been correctly comflagged and moved, I first "disconnect" the recording from MythTV, then I run Handbrake and, once I've checked I'm happy with the transcoded version, I manually remove (rm) the pre-transcoding recording file. This is actually the reason I added the --disconnect flag in the first place. As long as you set the Framerate to same-as-source in Handbrake the comflagging should still be aligned with the transcoded recording. But this approach does not work if you really want to keep the recording connected to MythTV, for example, if you want to also be able to use the MythTV-frontend to view recordings.


If you decide you are ok disconnecting recordings from MythTV and are considering following the approach that I do, then if possible you should ideally wait until the next release of myth2kodi.
The reason being that I am currently in the process of refining the --disconnect and the deletion of MythTV-DB entries in general. The current minor issue that I am resolving is that the --disconnect flag currently does a hard delete of the MythTV-DB entry, this is because it uses the direct MySQL delete method. What this means is that, after disconnect, MythTV has no knowledge that the recording ever existed. Which has the minor inconvenience that MythTV now lacks the knowledge necessary to prevent rerecording the same episode again. This is not a major issue for me because my guide data is so bad I already get a lot of duplicate recordings. But your guide data seems quite good and so you will be weakening MythTV's duplicate recording protection. What I am in the process of doing is making sure that, if DATABASE_ACCESS is set as PythonBindings (which is the default/preferred setting) then the Python-bindings delete method is used for disconnect as well. The Python-bindings delete is the same as deleting via mythweb or the MythTV-frontend in that it doesn't actually completely delete the MythTV-DB entry but rather moves the entry to an "oldrecorded" table, so MythTV views it as deleted but also remembers that in the past it recorded that episode and so should not record it again in the future.


EDIT: Disregard the previous paragraph, turns out I'd misunderstood a detail in how the MythTV-DB is set-up. Luckily it was a good sort of misunderstanding as it just led me to be overly conservative. Even the MySQL-based database-entry deletion, used for --disconnect in the current release of myth2kodi (v1.3.1), does not damage MythTV's ability to remember all old recordings and use that info to prevent duplicates. So it can be used without the caveat of the previous paragraph. That being said, I've already made the changes that mean all MythTV-DB accesses will respect the DATABASE_ACCESS setting, and I'm currently at the point of testing before making the next release -- which will include these changes as well as quite a few other minor fixes and improvements and should be available in the next few days.
(This post was last modified: 2017-03-14 16:15 by stuartk.)
find quote
jctennis Offline
Senior Member
Posts: 141
Joined: Feb 2013
Reputation: 7
Post: #56
Ok, so I should wait until the next release before I do too much is what I am getting from this. I also only watch recordings via Kodi, but yes, the schedules direct data is very good so I rarely have to deal with dupes. My main goal was to try and avoid having all my transcoded files be re-recorded as I often use a "record all episodes" rule for quite a few things (especially children's shows so they can watch them at their leisure.)

I am trying to automate as much of the process as possible for the wife acceptance factor. I currently have an inotfywait script watching my recordings folder (it doesn't do anything yet but output messages to the terminal because I was waiting to hear from you before I did anything rash but it does work.) It sounds like you have it pretty well figured out and working to that end.

ETA: Do you know how to set the framerate as same-as-source using the CLI? I can't seem to find any documentation about that.

[Image: Forum_Signature.jpg]
(This post was last modified: 2017-03-11 16:29 by jctennis.)
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #57
(2017-03-11 15:56)jctennis Wrote:  ETA: Do you know how to set the framerate as same-as-source using the CLI? I can't seem to find any documentation about that.

Transcoding from CLI again falls into the category of something I haven't gotten around to resolving fully myself yet. As I manually verify the recordings rather than just transcoding them automatically, queueing transcoding via the HandBrake GUI isn't a major issue for me, especially as so far I've only bothered doing so for HD movies where a single recording can be 12-14GB and the transcoding gets down to ~3GB with no obvious loss in quality (at least when played through the TV). Having a large NAS has made me lazy.

That being said, I did look into it in the past and seem to remember that from the command line the frame-rate part was just a matter of not specifying an specific frame-rate, in which case HandBrake defaults to the sources frame-rate. Also, I think you want to set --vfr, to allow for the source having variable frame-rate, and if you're using some preset config then make sure it does not explicitly or implicitly set a frame-rate.
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #58
Updated Release Available (v1.4.0):

New Features:
  • Add install.sh -- supports installation from release tar-ball or repo clone.
  • Support single recording undo with myth2kodi --undo '/path/to/filename.ext'
    • Recording can be specified as either MythTV name or myth2kodi name.
    • Old functionality of undoing all processed recordings now requires command:
      myth2kodi --undo 'all'
  • Add user setting RSS_DIRECTORY -- was previously hard-coded in generaterss().
  • Rename DirTracking user setting as DIR_TRACKING_CLEANUP -- this new name more closely matches the effect of the setting.

Bug Fixes
  • Fix removal of doover entry for --delete and --disconnect of recordings processed in LINK mode.
  • Make sure DailyReport contains consistent time and date for recordings processed near midnight.
  • Fix logging for myth2kodi --diagnostics.
  • Add missing comments to default settings for recently added user variables in myth2kodi.conf.
  • Fix edge-cases for fuzzy episode name matching.

Other Changes
  • Make all deletions of MythTV-DB entries respect DATABASE_ACCESS.
  • Add extra checks to calls to myth2kodi --movie.
  • Revise how we test for Sxx an Exx being set -- main difference is now S00 is allowed where it was previously excluded, S00 is the "season" used for special episodes.
  • Strip any trailing / for directories passed to myth2kodi --comskip and myth2kodi --scan.
  • Standardise permissions setting.
    • Working directory files = 644 (rw-r--r--).
    • Working directory directories = 755 (rwxr-xr-x).
    • Kodi Library files = 664 (rw-rw-r--).
    • Kodi Library directories = 775 (rwxrwxr-x).
  • Add single quotes around parameter expansions in debugging statements -- easier to identify empty variables.
  • General clean-up.
The links provided in the original post of this thread always point to the latest version of myth2kodi.

Thanks to jctennis for the feedback.
find quote
mackie Offline
Junior Member
Posts: 6
Joined: Mar 2017
Reputation: 1
Post: #59
Thank you so much for this!

I came here to learn if what I want to do is supported and if so, what the best way to go about it is. I was happy to see you mostly answered my questions in reply #55 and recently took care of one of my concerns (Make all deletions of MythTV-DB entries respect DATABASE_ACCESS). I'm grabbing the updated tarball now to use in my developing workflow.

First, my process, then a request for recommenations Smile :

1. MythTV auto commflags episodes after recording.
2. Use the editor in mythfrontend to generate and save a proper cutlist.
3. User Job script which does a lossless transcode using mythtranscode and --honorcutlist and then uses Handbrake to transcode to x264 MKV. At this point, I'm hoping to add m2k to my User Job to finish up the automation, but might need to put it into a separate User Job to give time for sanity checks on the transcoded files before deleting recordings.

So far my tests are working, but am I doing the following in the "right" way? Should I be using --disconnect as stuartk does in post #55?
Code:
myth2kodi /path/to/handbrakeoutputfile.mkv <title> <subtitle> (title and subtitle are already available as variables in my User Job) (PROCESS_RECORDING_MODE='MOVE', SYMLINK='Disabled', CommercialMarkup='Disabled', GuideDataType='SchedulesDirect'
myth2kodi --delete /path/to/mythtv/recordings/<chanid>_<starttimeutc>.mpg (these variables also already available)

As I said, things seem to be working - the recorded show disappears from MythTV and goes to my Kodi storage, but I get paranoid.

Otherwise, I've experienced a couple of quirks with my Ubuntu 14.04.3 LTS system.
1. Since my mythtv ($Librarian) user is set to nologin, I do most m2k tasks with `sudo -u mythtv ...`. As a result, the install.sh script fails for me when it tries to sudo as the mythtv user instead of my admin user. Not really a problem to copy over and edit files manually.
2. Ubuntu really doesn't like your use of realpath at all Smile --diagnostics didn't detect realpath, so I installed realpath as available in the regular repository and it passed --diagnostics, but complains as shown below when I run a --delete. I'm hoping the last two `rm` lines of this output are caused by the realpath error. If I can't satisfactorily find a good version of realpath, I'll try to experiment with changing myth2kodi.sh to remove the --no-symlinks. At first glance, it seemed like where realpath is used I can do that safely since I'm not using symlinks.

Code:
[email protected]:/home/mythtv/.myth2kodi$ sudo -u mythtv myth2kodi --delete /mnt/myth/rec2/recordings/1051_20170204020000.mpg
WARNING: Attempting to delete recording: '/mnt/myth/rec2/recordings/1051_20170204020000.mpg'
         This will PERMANENTLY delete the recording, associated files
         and the associated MythTV database entry.
         Are you sure you want to continue?
  Requested permanent deletion: 'y' to continue or 'n' to exit... y/(n):>y
WARNING: find_file_link_pairs(): Cannot find link to: '/mnt/myth/rec2/recordings/1051_20170204020000.mpg'
         This probably means that the file was not successfully processed by myth2kodi.
realpath: unrecognized option '--no-symlinks'
Usage:
realpath [-s|--strip] [-z|--zero] filename ...
realpath -h|--help
realpath -v|--version
INFO: delete_mythtv_database_entry_mysql(): REMOVING: '1051_20170204020000.mpg'; THUMBNAILS; DATABASE ENTRIES.
rm: cannot remove ‘.’: Is a directory
rm: cannot remove ‘..’: Is a directory
Thank you again for all the hard work that went into these scripts. I'm glad I don't have to reinvent a poorer quality wheel myself!
find quote
stuartk Offline
Junior Member
Posts: 43
Joined: Dec 2016
Reputation: 9
Post: #60
(2017-03-19 04:48)mackie Wrote:  Thank you so much for this!

I came here to learn if what I want to do is supported and if so, what the best way to go about it is. I was happy to see you mostly answered my questions in reply #55 and recently took care of one of my concerns (Make all deletions of MythTV-DB entries respect DATABASE_ACCESS). I'm grabbing the updated tarball now to use in my developing workflow.

First, my process, then a request for recommenations Smile :

1. MythTV auto commflags episodes after recording.
2. Use the editor in mythfrontend to generate and save a proper cutlist.
3. User Job script which does a lossless transcode using mythtranscode and --honorcutlist and then uses Handbrake to transcode to x264 MKV. At this point, I'm hoping to add m2k to my User Job to finish up the automation, but might need to put it into a separate User Job to give time for sanity checks on the transcoded files before deleting recordings.

So far my tests are working, but am I doing the following in the "right" way? Should I be using --disconnect as stuartk does in post #55?
Code:
myth2kodi /path/to/handbrakeoutputfile.mkv <title> <subtitle> (title and subtitle are already available as variables in my User Job) (PROCESS_RECORDING_MODE='MOVE', SYMLINK='Disabled', CommercialMarkup='Disabled', GuideDataType='SchedulesDirect'
myth2kodi --delete /path/to/mythtv/recordings/<chanid>_<starttimeutc>.mpg (these variables also already available)

As I said, things seem to be working - the recorded show disappears from MythTV and goes to my Kodi storage, but I get paranoid.

I have never done things in the way you describe, so I'm uncertain about an aspect of MythTV's behaviour here:
Does calling Handbrake as part of a MythTV User Job result in MythTV associating "/path/to/handbrakeoutputfile.mkv" with the recording?
For example, if you run myth2kodi --recording-info "/path/to/handbrakeoutputfile.mkv" does it actually return data on the recording or does it issue some sort of "recording not found" error?

I assume it must be associated, otherwise when myth2kodi attempts to process "/path/to/handbrakeoutputfile.mkv" it would fail with a database access error.

If the "/path/to/handbrakeoutputfile.mkv" file is tracked by MythTV, then you need to be aware that if myth2kodi was not confident about a MOVE when SYMLINK is Disabled, it tries to fall-back to SYMLINK=Enabled, so that the processing can still be undone.
NOTE: --delete completely deletes a recording, including following symbolic links if they are present, so if you had MOVEd with SYMLINKing Enabled then using --delete will remove all trace of the recording. So in this case, you really do not want to run --delete as it will completely delete the recording.

In the case that myth2kodi is confident in its processing, then, with SYMLINK=Disabled the recording is deleted from MythTV, making your call to myth2kodi --delete "/path/to/mythtv/recordings/<chanid>_<starttimeutc>.mpg" redundant. If it is not confident, then myth2kodi attempts to add symbolic-link and your --delete will most likely delete the recording entirely -- which I assume is not what you want.

There are two ways to proceed:
1) Continue as you are, that is MOVE with SYMLINKing Disabled. Then, for any recordings that have symlinks added because myth2kodi was not confident in its processing, you can manually check the recordings for which there has been a link created from /path/to/handbrakeoutputfile.mkv to the moved file and, if you are happy with myth2kodi's processing (the moved recording has the correct name), run myth2kodi --disconnect to permanently disconnect the recording from MythTV. The --disconnect flag is basically the same as --delete with the key exception that it severs the link between MythTV and the recording file before calling delete, so that the myth2kodi processed file remains; or
2) A more conservative option, if you are concerned about myth2kodi getting things wrong, would be to run in MOVE mode with SYMLINKing Enabled, and then manually check each recording processed by myth2kodi before using --disconnect.

Either way, do not use --delete unless you really want to permanently delete the recording and information associated with it.


(2017-03-19 04:48)mackie Wrote:  Otherwise, I've experienced a couple of quirks with my Ubuntu 14.04.3 LTS system.
1. Since my mythtv ($Librarian) user is set to nologin, I do most m2k tasks with `sudo -u mythtv ...`. As a result, the install.sh script fails for me when it tries to sudo as the mythtv user instead of my admin user. Not really a problem to copy over and edit files manually.
2. Ubuntu really doesn't like your use of realpath at all Smile --diagnostics didn't detect realpath, so I installed realpath as available in the regular repository and it passed --diagnostics, but complains as shown below when I run a --delete. I'm hoping the last two `rm` lines of this output are caused by the realpath error. If I can't satisfactorily find a good version of realpath, I'll try to experiment with changing myth2kodi.sh to remove the --no-symlinks. At first glance, it seemed like where realpath is used I can do that safely since I'm not using symlinks.

Code:
[email protected]:/home/mythtv/.myth2kodi$ sudo -u mythtv myth2kodi --delete /mnt/myth/rec2/recordings/1051_20170204020000.mpg
WARNING: Attempting to delete recording: '/mnt/myth/rec2/recordings/1051_20170204020000.mpg'
         This will PERMANENTLY delete the recording, associated files
         and the associated MythTV database entry.
         Are you sure you want to continue?
  Requested permanent deletion: 'y' to continue or 'n' to exit... y/(n):>y
WARNING: find_file_link_pairs(): Cannot find link to: '/mnt/myth/rec2/recordings/1051_20170204020000.mpg'
         This probably means that the file was not successfully processed by myth2kodi.
realpath: unrecognized option '--no-symlinks'
Usage:
realpath [-s|--strip] [-z|--zero] filename ...
realpath -h|--help
realpath -v|--version
INFO: delete_mythtv_database_entry_mysql(): REMOVING: '1051_20170204020000.mpg'; THUMBNAILS; DATABASE ENTRIES.
rm: cannot remove ‘.’: Is a directory
rm: cannot remove ‘..’: Is a directory
Thank you again for all the hard work that went into these scripts. I'm glad I don't have to reinvent a poorer quality wheel myself!

1. I'll look into whether there is a clean way to handle your use-case for the install script. Do I understand correctly that you are running:
Code:
sudo -u mythtv ./install.sh
and then attempting to do a system wide install (that is, install to /usr/local/bin)? EDIT: I've had a look at this and I can't reproduce an issue when running the install script using sudo -u from another user... Could you post the install log from when it failed for you? Or better yet, enable debugging (set LOGLEVEL=3 on line 66 of install.sh) and try again and post that log?

2. The "realpath: unrecognized option '--no-symlinks'" is an Ubuntu specific bug I was not aware of, it looks like it should be easily fixed as "--no-symlinks", at least on my system, is just another name for the same command as "-s" or "--strip". I'll include the change in an bug-fix release. And yes, hopefully the the two rm errors are just a result of realpath having failed, but I'll look into it and revise the code to protect against this in the future.
(This post was last modified: 2017-03-19 14:29 by stuartk.)
find quote
Post Reply