Kodi Community Forum

Full Version: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
outleradam Wrote:Try:
Code:
sudo mythicalLibrarian --undomythicalLibrarian --scan mpg /path_to/files
to undo and rescan the library which should rebuild all of the tracking files.

[...]After a total undo, --doover jobs --undo jobs should be deleted, assuming all library files are located in the same dir. file/folder.tracking are taken care of as usual by the maintenance routine.

I actually have my library files in three separate volumes:

/var/lib/mythtv/recordings/
/mnt/video1/store/
/mnt/video2/store/

Given that, should I run the --undo once, then three separate --scan mpg [path/to/files/], one for each of the above directories? I think that would be right.

Thanks as always,

/Greg
For some reason in my last post it would not register my new lines. It merged them all into a single paragraph, even after I edited it.So yeah. You do --undo once as all users you've run mythicalLibrarian under, then --scan three times.Wow... still not letting me start a new paragraph
Adam:

I hit the permissionsCheck errors again when doing the --scan after running --undo. I was running as user mythtv

Here's the link to pastebin of output.log.

Suggestions?

/Greg
Code:
[email protected]:/home/mythtv# sudo su mythtv
$ mythicalLibrarian --scan mpg /mnt/video1/store
The code says you ran as root, so you may have typed sudo mythicalLibrarian --scan. Never sudo a file operation. I would put a check in place, but I cannot figure out how to descriminate between root and sudo. Some people like to run as root so I'm not going to force running as non-root.

Code:
[email protected]:~$ whoami
mythtv
[email protected]:~$ sudo whoami
root
Adam:

But I did do sudo su mythtv before running the --scan; I never logged in as root.

EDIT: turns out as you surmised above, I used sudo mythicalLibrarian --scan. I'll re-run as mythtv and not sudoer. Thanks for clarifying

Here are the terminal commands I used to get to the output.log I just posted:

Code:
[email protected]:~$ sudo su mythtv
$ sudo mythicalLibrarian --undo
[sudo] password for mythtv:
--You have entered undo mode.
<<BIG SNIP OF OUTPUT>>
Removing mythicalLibrarian database from:/home/mythtv/.mythicalLibrarian/Top Gear
done.
$ sudo mythicalLibrarian --scan mpg /mnt/video1/store
ls: unrecognized option '--scan'
Try `ls --help' for more information.
--You have entered scan Mode.
mythicalLibrarian will conduct a scan of the folder:
/mnt/video1/store
searching for file types:mpg. Each file will be
processed through mythicalLibrarian. This mode will
only work when database access functional.
please scan a single file to test capabilities before
running the scan mode of operation using the following
eg. mythicalLibrarian /path/to/file.ext
mythicalLibrarian will first run the doovers to prevent
clobbering and creation of unnescessary links.
This operation can be reversed using the following command:
sudo mythicalLibrarian --undo
Data loss can occour if this operation is interupted.
Would you like mythicalLibrarian to conduct a scan? (y)/n>y
--You have entered doover mode.
In this mode of operation, mythicalLibrarian will rerun
mythicalLibrarain against all files which failed on the
last run of mythicalLibrarian.  It is important that this
process does not get interupted or data loss may occour.
  Press 'y' to continue or 'n' to exit..... y/(n):>yNo Doover jobs exist.
/mnt/video1/store mpg
Revision 583 Tue Sep 14 21:18:06 PDT 2010
<<SNIP OF OUTPUT>>
I made a few more permissions changes tonight.

On a side note, if mythtv is not in sudoers, it will be added and you will be prompted to enter a password for user mythtv. I will be reviewing this process to see how it goes. Let me know if there are any problems with it. I hope this is not an inconvenience and it could be bypassed if not necessary for everyone.
mythicalLibrarian revision 600 to stable.
Tonight I will be performing some revisions that will support mutiple instances of mythicalLibrarian at a time. mythicalLibrarian's TvDb entries and comskip will no longer be located in the mythicalLibrarian working directory. This will cause disruption in the SVN so feel free to use Stable until the problem is resolved. This may take days.
Just to finalize my end of this... your suggestions worked like a charm!

I am having issues with /etc/mythicalLibrarian/JobSucessful being brought in correctly when I update, but I will double check a few things on my side and if I run into problems I will post in the other thread.

Thanks for your help!
Two major changes in the SVN happened last night. First, A new committer. barney_1, aka szczys, a writer for hackaday.com, has posted his first commit. He is a skilled python coder and a long-time user/contributer to the mythicalLibrarian project. He is responsible for several features in mythicalLibrarian including the RSS Feeds and the ability of mythicalLibrarian to change the responsibility of MythTV from a back-end server to a dummy recording device.
Seccond, in the SVN you will notice a new folder called PythonBindings. This folder is the start of a new method in which mythicalLibrarian will access the MythTV database. The MythTV Python Bindings were introduced in version .23 in order to allow for upcomming changes in the early development cycle of version .25. The .25 changes will disassociate the file name from the recording data. This change will mean mythicalLibrarian would explicitly fail all recordings as insufficient information. The MythTV Python Bindings will allow for data to be extracted from the MythTV library without any version dependency.
The plan is to replace the version dependent mySQL commands in mythicalLibrarians functions GrabTitleAndSubtitle() and GetMythTVDatabase() with a python script written by barney_1. This script will access the universal MythTV python bindings and dump the database which MythTV has obtained to a file which will be readable by mythicalLibrarian. Currently we are hammering out the interface details, but this script should be usable as a stand-alone script which can be used to grab all database information MythTV has stored on a recording.
The transition should be smooth from direct mySQL to abstracted python queries and it will allow mythicalLibrarian to support every future version of MythTV without changes to mythicalLibrarian's code.
Code:
/usr/local/bin/mythicalLibrarian: line 599: syntax error near unexpected token `fi'
/usr/local/bin/mythicalLibrarian: line 599: `        fi'

Using the mythicalLibrarian --update command. All the previous updates in this manner have worked like a charm.

Will I need to reinstall from scratch, or can I modify the script to recover when/if a patch is committed?

Thanks for all your hard work on this!
Sorry, SVN is sorta screwy right now. I will commit a fix tonight. Barney_1 and I are working on the MythTV python bindings.To run mythicalSetup and update to stable type:
Code:
cd ~/.mythicalLibrarian/mythicalSetup
sudo ./mythicalSetup.sh
mythicalLibrarian --update is just the quick way of downloading and running mythicalSetup.
outleradam Wrote:Sorry, SVN is sorta screwy right now. I will commit a fix tonight. Barney_1 and I are working on the MythTV python bindings.To run mythicalSetup and update to stable type:
Code:
cd ~/.mythicalLibrarian/mythicalSetup
sudo ./mythicalSetup.sh
mythicalLibrarian --update is just the quick way of downloading and running mythicalSetup.

Sweet. Looks like I'm back on track. Thanks again!
Firstly, I would like to take my hat off to you for the excellent work you have done so far. This project has completely streamlined my MythTV/XBMC experience. That said, I am having a few problems...

1. I am ending up with about half of my shows in the 'Showings' directory. Is this normal? They are show such as 'Late Night With Jimmy Fallon', 'The Amazing Race 17', 'Ultimate Engineering', 'Larry King Live', 'The Colbert Report' and others. What determines whether a show ends up in 'Episodes' and when it ends up in 'Showings'? I have had some tv shows where some of its episodes have ended up in 'Episodes' and some in 'Showings', such as 'Late Night With Jimmy Fallon'.

2. Speaking of 'Jimmy Fallon', I have a couple of episodes that have ended up in 'Episodes', with a season number and episode number, but they did not get added to the library after an update. They look correctly named, but do not get added.

3. I have about eight XBMC installs in my home! Laptops, desktops, Revo and a bunch of XBOXes. I would like to automatically update all of their libraries, but I am having problems. I was able to send notifications to one of my XBMC instances, and then send the Update Library command successfully with a single XBMC IP address in /usr/local/bin/mythicalLibrarian. But when I added a bunch of XBMC IP addresses to /usr/local/bin/mythicalLibrarian, I only get the notification, but not the Update happening.

4. Lastly, if I wanted to completely start over, what do I need to delete? In other words, it would be nice to see a complete listing of all the files that get installed so that one could remove them for a complete uninstall. There seems to be files in /home/USER/.mythicalLibrarian as well as files in /home/mythtv/.mythicalLibrarian. Are the files in /home/USER/.mythicalLibrarian used? I run mythicalLibrarian as a user job from within MythTV, so I assume the version of mythicalLibrarian in /home/mythtv is the one that gets used. If so, do I need to keep the one in /home/USER? What is it for?

Keep up the great work!
ubuntite Wrote:Firstly, I would like to take my hat off to you for the excellent work you have done so far. This project has completely streamlined my MythTV/XBMC experience. That said, I am having a few problems...
Thank you. I will try to help resolve the issues
Quote:1. I am ending up with about half of my shows in the 'Showings' directory. Is this normal? They are show such as 'Late Night With Jimmy Fallon', 'The Amazing Race 17', 'Ultimate Engineering', 'Larry King Live', 'The Colbert Report' and others. What determines whether a show ends up in 'Episodes' and when it ends up in 'Showings'? I have had some tv shows where some of its episodes have ended up in 'Episodes' and some in 'Showings', such as 'Late Night With Jimmy Fallon'.
When information cannot be obtained from TheTvDb, or if the episode has no identifiable information (no subtitle, or original airdate), it is a showing. mythicalLibrarian makes decisions based on available information. The show ID for generic programming, that which cannot be classified as an episode in any way, comes marked with a showID of SH*****.

If mythicalLibrarian determines that there is suitable information, but it cannot pull the episode or season from TheTvDb, it will mark it as what I call an "epi-show". Epi-shows get put into the --doover que. You can have mythicalLibrarian rerun the doover que by typing mythicalLibrarian --doover. Generic programming is already categorized as best as it can be and will not be rerun
Quote:2. Speaking of 'Jimmy Fallon', I have a couple of episodes that have ended up in 'Episodes', with a season number and episode number, but they did not get added to the library after an update. They look correctly named, but do not get added.
Please post the name and path of the episode. I can try to recreate the problem on my end.
Quote:3. I have about eight XBMC installs in my home! Laptops, desktops, Revo and a bunch of XBOXes. I would like to automatically update all of their libraries, but I am having problems. I was able to send notifications to one of my XBMC instances, and then send the Update Library command successfully with a single XBMC IP address in /usr/local/bin/mythicalLibrarian. But when I added a bunch of XBMC IP addresses to /usr/local/bin/mythicalLibrarian, I only get the notification, but not the Update happening.
You need to have the source set up on each XBMC installation. If XBMC is receiving communications messages, but there are no library updates, then there is a problem with the source. mythicalLibrarian sends two messages, one says "display a message" the other says "update your sources".
Quote:4. Lastly, if I wanted to completely start over, what do I need to delete? In other words, it would be nice to see a complete listing of all the files that get installed so that one could remove them for a complete uninstall. There seems to be files in /home/USER/.mythicalLibrarian as well as files in /home/mythtv/.mythicalLibrarian. Are the files in /home/USER/.mythicalLibrarian used? I run mythicalLibrarian as a user job from within MythTV, so I assume the version of mythicalLibrarian in /home/mythtv is the one that gets used. If so, do I need to keep the one in /home/USER? What is it for?
if you wanted to completely remove mythicalLibrarian, you simply run mythicalLibrarian --undo (reverts all changes), delete the user job from mythtv, delete the ~/.mythicalLibrarian folder, and finally delete the file from /usr/local/bin.

The ~/.mythicalLibrarian folder contains mythicalLibrarian's working files. It is the storage space for mythicalLibrarian's lookup files, the --doover que, --undo que, created folder tracking, and the comskip/NFO file tracking. It is also home to the DailyReport(a log of all files generated) and the output.log file(a detailed account of what happened during processing of each file). You can see what shows were processed each day by looking at the Daily Report folder. If you want to find out more about the episode, you can look at the output.log file. I wouldn't recommend changing anything in the folder, but it is good information.
Quote:Keep up the great work!
Thanks, I'm trying. Hopefully we can get everything squared away in time for mythtv .25