Kodi Community Forum

Full Version: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi Adam:

I have a quick showTranslations question. I am using a showTranslations file to try to get mythicalLibrarian to recognize some of my files that don't match theTVdb.com's file names. Some of the entries are working when I run mythicalLibrarian, but some are not.

Here's one particular example: my mythtv recordings contain episodes of Curious George on PBS. Mythtv's guide data calls it "Curious George". TheTVDB calls it "Curious George (2006)" (link to thetvdb.com entry here.)

In my showTranslations file I have this entry:
Code:
Curious George = Curious George (2006)

When I run mythicalLibrarian on the file, here's the output, which indicates (I'm pretty sure) that it is not using the showTranslation name to do the lookup, and so treats it as an unrecognized Showing:
Code:
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/mnt/video1/store/Episodes-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/mnt/video1/store/Episodes-
USE ORIGINAL DIR:Enabled NOTIFICATIONS:Disabled
INPUT SHOW NAME:Curious George- LOCAL SHOW NAME TRANSLATION:- ShowName:Curious George
DATABASE UPDATED:- TVDB LAST UPDATED:- CURRENT:-
RESOLVED SERIES ID:207041- RESOVED SHOW NAME:Curious George-
INPUT EPISODE NAME:Housebound!; Curious George Rides a Bike- Absolute EPISODE NUMBER:- RESOLVED EPISODE NAME:Housebound-
SEASON:- EPISODE:- SYMLINK MODE:MOVE- FILESIZE: 0kB-
CONFIDENCE:-4 LOGIC PATH:  Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Sucessful; Attemped match on OriginalAirDate: Failed; Attemted SubTitle match on Standard logic: Failed; Attemted SubTitle match on Fuzzy logic: Failed; Could not set Sxx and Exx, Failed;  Guide Data did not match showing;
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:386164696kB- WORKINGDIR:1 FREE:23176444kB-
MOVEDIRWRITABLE:1- FREE:386164696kB- ALTERNATEMOVEDIR:1- FREE:386164696kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:386164696kB-ALTERNATESHOWDIRWRITABLE:1- FREE:386164696kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:386164696kB- ALTERNATEMOVIEDIR:1- FREE:386164696kB-
DATABASE TYPE:-
RECSTART:2009-10-11 08:30:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2007-02-19-
PROGRAMID:EP008466780026- CHANNEL ID:1709- CATEGORY:Children- GOFORDOOVER:0-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0.0 RATING:0
ZAP2IT SERIES ID:846678- MATCHED TVDB SERIES ID:-
PLOT: George breaks his leg and gets a cast; George hits a rock while delivering papers, bending his bicycle's wheel out of shape.
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB
CREATING FOLDER: /mnt/video1/store/Showings/Curious George
MOVING FILE: '/mnt/video1/store/1709_20091011083000.mpg' to '/mnt/video1/store/Showings/Curious George/Curious George S0E0 (Housebound Recorded [email protected] on 1709).mpg'
@@@@@@@@@@@@@OPERATION COMPLETE Sun Jan 16 18:14:14 PST 2011 @@@@@@@@@@@@@@@@

Any ideas why it is not applying the showTranslation? Are there any escape characters or quotation marks I should be using?

Thanks, as always.

/Greg
Sorry it took me so long to get back to you on this. I lost my phone and got a new one. Usually I get notifications within 15 minutes.

I am not aware of this problem. I have a couple of things going on tonight. I will check it out tomorrow.

On first inspection, it is trying to locate the file by the name /home/mythtv/.mythicalLibrarian/showtranslations. If the file is not there, then it won't work. I will have it check the /etc/mythicalLibrarian folder. Linux is case sensitive, are you using capitol letters in the showtranslations file?

It's been a while since I needed showTranslations. Zap2it IDs take care of what ShowTranslations used to take care of. Add EP00846678 as the zap2it ID on TheTvDb under the Curious George 2006.

Let me know if the file is properly located and my documentation is wrong, or if the ~/.mythicalLibrarian/showtranslations file is simply not working.
Adam:

Thanks for the reply.

Your documentation says to use "showTranslations" not "showtranslations" as the file name for the translations. I'll try adding a file named "showtranslations" as well and see if that helps.

The Curious George and other shows that I am having trouble with don't seem to have Zap2It IDs on thetvdb, so I will add those to the extent I can find them. Maybe I'll try that first. If I do, should I run --doMaintenance before running --doover again?

/gkk
You should not need to do that. Only the bulky databases are archived. The searches are not archived.

Ill update release a new stable soon. I will grandfather the old naming convention and the new one will be either /etc/mythicalLibrarian or ~/.mythicalLibrarian showTranslations
Howdy, need help please...

My wife is about to kill me because all the recordings have disappeared, quite bizarre but what I think has happened is a debian/myth upgrade has broken/started something and now the run job I have with mythicalLibrarian (which I am a newbie with) has broken things. All the recordings are still here, nicely arranged, but no symlink has been produced so myth can't track them. About two weeks of recordings suddenly disappeared from Myth. So...
A. Is there any way I can rebuild the symlinks?
B. How can I get this working again?
C. (newbie question) Will the recordings get deleted as my disk fills up?

Here is a log part where a symlink failed to create...

2011-01-22 19:59:31.054 JobQueue: Started mythicalLibrarian for "Good Game: SP" recorded from channel 1023 at 2011-01-22T19:31:00
Revision 721 Wed Oct 20 16:06:28 CST 2010
Accessing MythTV DataBase:
RECSTART: DATE:
PROGRAMID: ShowCategory:
PLOT:
REMOVING ORPHAN /srv/MythTV/Showings/Angelina Ballerina/Angelina Ballerina S0E0 (Angelina and the Musical Theatre Recorded on ).nfo
/usr/local/bin/mythicalLibrarian: line 317: /srv/MythTV/arbitraryfile.ext: Input/output error
mythtv TRIED TO WRITE TO /srv/MythTV --FAILED
QUICK FIX: sudo chown mythtv "/srv/MythTV"
JOB: mythicalLibrarian '/srv/MythTV/1023_20110122193100.mpg' 'Good Game: SP' ''
WARNING: EPISODE INFORMATION COULD NOT BE OBTAINED
HANDLING AS A GENERAL SHOWING
Please consider helping out and adding to thetvdb.
TARGET SET:/srv/MythTV/Showings/Good Game SP/Good Game SP S0E0 ( Recorded on )
/usr/local/bin/mythicalLibrarian: line 929: [: too many arguments
CREATING SYMLINK IN MOVE MODE
MOVED: /srv/MythTV/1023_20110122193100.mpg
TO: /srv/MythTV/Showings/Good Game SP/Good Game SP S0E0 ( Recorded on ).mpg
SENDING REQUESTED COMMANDS TO:192.168.1.13:8080
RSS ENTRY SUCCESSFULLY CREATED: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: MoveModeSuccessful
I've never seen this problem before.

You are running quite an old version of mythicalLibrarian. Newer versions are in the 900's revisions you're running in the 700's

Since you just upgraded, I don't know what is going on... it may be a database error, it may be mythtv, it may be filesystem, it may be mythicalLibrarian, it may be just about anything...

do the following:
1. before running undo, verify that the recordings information is still in mythtv. Do not proceed if the Episode information on the files is not in mythweb, or mythtv.
2.
Code:
su mythtv
mythicalLibrarian --undo
mythicalLibrarian --update
mythicalLibrairan --doMaintenance
mythicalLibrarian --scan mpg /srv/MythTV
Since it seems that there is quite a bit of pressure on you... I'll be on instant messenger all day. You can catch me on MSN with my username at hotmail, or aim with my username. Try the stuff above and then if it does not work, we can chat to get things right.
outleradam Wrote:1. before running undo, verify that the recordings information is still in mythtv. Do not proceed if the Episode information on the files is not in mythweb, or mythtv.

I've just got home, unfortunately the information no longer seems to exist in Myth which is even more puzzling. I recall deleting a few by hand that had no associated file, but not where near the full amount missing. Perhaps myth cleans up itself.

So given this is my only option now to correct the problem, move the files to where we can view them directly and look to a better future?

I'll hold off on upgrading it until I hear from you again.
By all means do the upgrade.


Add the sources in XBMC and you should be squared away to watch the shows. They are exported and ready for XBMC. I don't know what happened to mythtv. It does run a cleanup at midnight. So you may have manually moved the files, breaking the symlink?

Do the steps above with the exception of undo.
Use the default options for upgrade
Share and set the XBMC sources to your recording dir.
For some reason DBHostName was incorrectly set, even after rerunning set-up. It seemed to have a <return> so looked something like:
DBHostName=#localhost
192.168.1.99
Perhaps I created that fault when I re-ran setup, otherwise how could it have been correctly finding the show names at all earlier? In any case after an update the --scan correctly created sym-links now. It does seem like it should stop and wave some bigger flags when something like a database connection can't be made.

Currently I'm giving Plex a run and haven't been running XBMC for a bit, but I like mythicalLibrarian and think it still holds value to me.

Thanks for you help Smile Perhaps I should request debian-multimedia repos carry mythicalLibrarian so its kept up to date automatically.
hmm, spoke too soon. I was watching this one, mythicalLbrarian appeared to have completed its job correctly, I saw the sym-link but then when I checked again a few minutes later it was missing as is any mention of it in MythTV itself!? So perhaps mythicalLbrarian is not to blame after all, sigh.

*edit*
Blush I think I know what it all was now, the myth partition was near full so naturally myth was trying to make room by deleting whatever it could, however it wasn't set to follow sym-links when deleting so naturally the problem became worse with time. I'm moving things around to free up space, switch on "follow sym-links" when deleting and I'm confident this will cure the problem.

Sorry for wasting your time, although it has helped me update and improve things here.

Thanks
I know what's going on... Inside of MythTV setup there is an option for "Follow Symlinks on Delete"

You need to check that box. You've run out of space and every 15 minutes MythTV is trying to make more space by deleting shows marked for deletion.
Just finished up a robot santa for the hackaday.com santapede competition, and i've got no projects lined up right now. This thread has been quiet for a while. Any problems? Let me know.
I'm trying to set this up for the 1st time and I hit a problem with my primary folders. They're on my NAS under a smb share. The path is //datastore/videos/TV Shows. I'm getting errors during the setup:

/usr/local/bin/mythicalLibrarian: line 5: Shows: command not found
/usr/local/bin/mythicalLibrarian: line 54: Shows: command not found

and then later

QUICK FIX: mkdir "" ; chmod -R 775 ""
USER SETTING |FLAG|Folder name
MoveDir |0 |
AlternateMoveDir |1 |/myth_data/Episodes
PrimaryMovieDir |1 |/myth_data/Movies
AlternateMovieDir|1 |/myth_data/Movies
PrimaryShowDir |0 |
AlternateShowDir |1 |/myth_data/Showings
mythicalLibrarian|1 |/root/.mythicalLibrarian
Critical Filesystem checks ***FAILED***

So apparently the script doesn't handle spaces in the paths. Should I enter it as "//datastore/videos/TV Shows" with the double quotes?


Also, it would be nice if this supported direct update of the centralized MySQL database for XBMC (or does it?).
The following is a method which should squash 99% of any problems that I can think of off hand.
Code:
sudo apt-get install libmyth-python python-lxml agrep dialog libnotify-bin curl
mkdir ~/.mythicalLibrarian
mkdir ~/.mythicalLibrarian/mythicalSetup
cd ~/.mythicalLibrarian/mythicalSetup
curl [url=http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh]http:[url="http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh"]//mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh[/url][/url] > ./mythicalSetup
chmod +x ./mythicalSetup
sudo ./myhticalSetup
sudo su mythtv
sudo ./mythicalSetup
If this does not work, please get me the results of these commands
Code:
whoami
uname -a
and I need the friendly name of the operating system.

mythicalLibrarian is required to be run on the machine which records the shows.

mythicalLibrarian works with files and sends notifications to different machines. If you have your database pointed at an IPaddress:port/share, then each machine should be able to read from that share on your network. It is compatable with centralized MySQL and will send the command to update the database to any machine you desire.
Any problems? I'm working on installing ubuntu and XBMC on my android right now.