Kodi Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- Kodi Community Forum (http://forum.kodi.tv)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- kvandesteeg - 2010-05-15 22:32

I'm getting further. I can't tell if this is success or not

Code:
Press any key to continue to online testing....
/usr/local/bin/mythicalLibrarian: line 187: syntax error near unexpected token `('
/usr/local/bin/mythicalLibrarian: line 187: `Tradi    ionalNamingConven    ions () {'
chown: htpc: Invalid argument
Please try again.  If problem persists, please post here: http://forum.xbmc.org/showthread.php?tid=65644
./mythicalSetup.sh: line 568: mysql: command not found
./mythicalSetup.sh: line 568: mysql: command not found
./mythicalSetup.sh: line 568: mysql: command not found
./mythicalSetup.sh: line 568: mysql: command not found
ADDING JOB to slot 1
./mythicalSetup.sh: line 578: mysql: command not found
permissions were set for user: htpc
mythicalLibrarian is located in /usr/local/bin
'mythicalLibrarian --help' for more information
Done.



- outleradam - 2010-05-15 23:38

I've been making some serious progress today. For those of you who used the command
Code:
port install agrep
please run
Code:
port uninstall agrep
port install tre
There are ALOT of problems (mostly uncritical errors) still left to work out for Darwin's installer. But, mythicalLibrarian should rename as is in the SVN and the installer is working. I'm still working on mythtv database interface. I do not have mythtv installed yet.



Does the command mysql work for anyone who has mythtv in the terminal?


- outleradam - 2010-05-16 02:03

as of build 444 mythicalLibrarian is operational on OSX to mythSExx standards.

For those of you who don't know, mythSExx was the predecessor to mythicalLibrarian and BashSExx was the predecessor to it. It was not designed with the mythTV database in mind, nor did it have many of the features which make mythicalLibrarian such a great program. As of now, I am stopping all development on the mythSExx project as mythicalLibrarian takes enough time on it's own.

Build 444 was sucessfully able to rename a file on OSX given the filename, title, and subtitle of the show. It was built from the easy installer as well.
I would say that this is a major success. I am going to install mythtv on my mac now and see if it can integrate with the database. This may pose some problems which should be able to be worked around.

Most of the problems I'm coming across is related to BASH being an older version in OS X then it is on Linux. So I'm having to rewrite commands here and there in an older format so that they work properly.

Please, I am new to mac so any input is appreciated. Please test out mythicalLibrarian using the SVN.

Here is my output.log from this last test.

Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Sat May 15 18:41:11 CDT 2010@@@@@@@@@@@@@@
Revision 444 Sat May 15 18:37:30 CDT 2010
REMOVING ORPHAN FOLDER:./Episodes/The End of the Year Show/Season 1
%%%NO DATABASE INFORMATION. CHECK LOGIN/PASS OR FILE %%%%%
SEARCHING: www.TheTvDb.com SHOW NAME: the simpsons EPISODE: Blowfish
FILE NAME: /Users/adam/file.ext
SEARCH FOUND:The Simpsons ID#: 71663
DEFINED ABSOLOUTE EPISODE NUMBER: 37
###################DEBUG MODE ENABLED####################
MY USER NAME:adam-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/Users/adam/.mythicalLibrarian-
MOVE DIR:/Users/adam/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/Users/adam/Episodes-
USE ORIGINAL DIR:Enabled NOTIFICATIONS:Enabled DEBUG MODE:Enabled-
INPUT SHOW NAME:the simpsons- LOCAL SHOW NAME TRANSLATION:- ShowName:the simpsons
DATABASE UPDATED:- TVDB LAST UPDATED:1274037113- CURRENT:1273967113-
RESOLVED SERIES ID:71663- RESOVED SHOW NAME:The Simpsons-
INPUT EPISODE NAME:Blowfish- ABSOLOUTE EPISODE NUMBER:37- RESOLVED EPISODE NAME:One Fish, Two Fish, Blowfish, Blue Fish-
SEASON:S02- EPISODE:E11- SYMLINK MODE:MOVE- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:98999464kB- WORKINGDIR:1 FREE:98999464kB-
MOVEDIRWRITABLE:1- FREE:98999464kB- ALTERNATEMOVEDIR:1- FREE:98999464kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:98999464kB-ALTERNATESHOWDIRWRITABLE:1- FREE:98999464kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:98999464kB- ALTERNATEMOVIEDIR:1- FREE:98999464kB-
DATABASE TYPE:-
RECSTART:- MOVIE YEAR:- ORIGINAL SERIES DATE:-
PROGRAMID:- CHANNEL ID:- CATEGORY:- GOFORDOOVER:1-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:- STARS: RATING:0
ZAP2IT SERIES ID:- MATCHED TVDB SERIES ID:-
PLOT:
####################END OF DEBUG LOG#####################
CREATING FOLDER: /Users/adam/Episodes/The Simpsons
MOVING FILE: '/Users/adam/file.ext' to '/Users/adam/Episodes/The Simpsons/Season 2/The Simpsons S02E11 (One Fish, Two Fish, Blowfish, Blue Fish).ext'
@@@@@@@@@@@@@OPERATION COMPLETE Sat May 15 18:41:12 CDT 2010 @@@@@@@@@@@@@@@@



444 mythicalSetup - kvandesteeg - 2010-05-16 04:17

new log attached

I expected the notifications and XBMC to fail (said no to both during install- i use PLEX)
I'm surprised about the filesystem issue, i ran under sudo su

other note -
under osx the home directory is /Users/username instead of /home/username



Code:
mkdir: /home/mythtv: Operation not supported                                    
mkdir: /home/mythtv: No such file or directory
mkdir: /home/mythtv: No such file or directory
mkdir: /home/mythtv: No such file or directory
chown: htpc: Invalid argument
chown: htpc: Invalid argument
mkdir: /home/mythtv: No such file or directory
mkdir: /home/mythtv: No such file or directory

mythicalLibrarian will now conduct mythicalDiagnostics
Press any key to continue to online testing....
mkdir: /var/root/.mythicalLibrarian: Permission denied
/usr/local/bin/mythicalLibrarian: line 1355: /var/root/.mythicalLibrarian/output.log: Permission denied
--You have entered diagnostic mode
mythicalLibrarian will now conduct a series of tests.
press any key to verify installed packages...
package 'curl' is installed
package 'agrep' or 'tre-agrep' is installed
/usr/local/bin/mythicalLibrarian: line 1067: [: too many arguments
Installed packages check ***PASSED***
Press any key to perform Desktop Message test...
Performing librarian-notify-send test
This test verifies that the message was sent to the DBUS.
This test does not verify the user has a valid desktop open.
sudo: /usr/local/bin/librarian-notify-send: command not found
Desktop Notifications test ***FAILED***
Press any key to perform XBMC Notification test...
Performing XBMC Notifications test
XBMC Communications ***FAILED***
Press any key to test file system permissions...
Testing file system permissions
df: /home/mythtv/Episodes: No such file or directory
df: /var/root/.mythicalLibrarian: Permission denied
UNUSABLE SPACE-CHECK:/home/mythtv/Episodes
UNUSABLE SPACE-CHECK:/home/mythtv/Movies
UNUSABLE SPACE-CHECK:/home/mythtv/Showings
UNUSABLE SPACE-CHECK:/var/root/.mythicalLibrarian
USER SETTING     |FLAG|Folder name
MoveDir          |1   |/Volumes/pvr/tv
AlternateMoveDir |0   |/home/mythtv/Episodes
PrimaryMovieDir  |1   |/Users/htpc/Movies/ota
AlternateMovieDir|0   |/home/mythtv/Movies
PrimaryShowDir   |1   |/Volumes/pvr/tv
AlternateShowDir |0   |/home/mythtv/Showings
mythicalLibrarian|0   |/var/root/.mythicalLibrarian
Critical Filesystem checks ***FAILED***
OVERALL FAILURE
press any key to continue on to the final results...
  All tests complete            RESULTS
  Installed Packages:        ***PASSED*** libnotify-bin
  Notifications Test:        ***FAILED***
  XBMC Communications:       ***FAILED***
  File System Checks:        ***FAILED***
  mythicalTesting:           COULD NOT COMPLETE
  Overall mythicalReadiness: ***FAILURE***
-----End of diagnostics-----
chown: htpc: Invalid argument
Please try again.  If problem persists, please post here: http://forum.xbmc.org/showthread.php?tid=65644
./mythicalSetup.sh: line 573: mysql: command not found
./mythicalSetup.sh: line 573: mysql: command not found
./mythicalSetup.sh: line 573: mysql: command not found
./mythicalSetup.sh: line 573: mysql: command not found
ADDING JOB to slot 1
./mythicalSetup.sh: line 583: mysql: command not found
permissions were set for user: htpc
mythicalLibrarian is located in /usr/local/bin
'mythicalLibrarian --help' for more information
Done.



- kvandesteeg - 2010-05-16 23:50

diagnostics still not Passed
how do i change the alternative directories?

Code:
mini:bin htpc$ ./mythicalLibrarian -m
--You have entered diagnostic mode
mythicalLibrarian will now conduct a series of tests.
press any key to verify installed packages...
package 'curl' is installed
package 'agrep' or 'tre-agrep' is installed
./mythicalLibrarian: line 1067: [: too many arguments
Installed packages check ***PASSED***
Press any key to perform Desktop Message test...
Performing librarian-notify-send test
This test verifies that the message was sent to the DBUS.
This test does not verify the user has a valid desktop open.
sudo: /usr/local/bin/librarian-notify-send: command not found
Desktop Notifications test ***FAILED***
Press any key to perform XBMC Notification test...
Performing XBMC Notifications test
XBMC Communications ***FAILED***
Press any key to test file system permissions...
Testing file system permissions
df: /home/mythtv/Episodes: No such file or directory
UNUSABLE SPACE-CHECK:/home/mythtv/Episodes
UNUSABLE SPACE-CHECK:/home/mythtv/Movies
UNUSABLE SPACE-CHECK:/home/mythtv/Showings
USER SETTING     |FLAG|Folder name
MoveDir          |1   |/Volumes/pvr/tv
AlternateMoveDir |0   |/home/mythtv/Episodes
PrimaryMovieDir  |1   |/Users/htpc/Movies/ota
AlternateMovieDir|0   |/home/mythtv/Movies
PrimaryShowDir   |1   |/Volumes/pvr/tv
AlternateShowDir |0   |/home/mythtv/Showings
mythicalLibrarian|1   |/Users/htpc/.mythicalLibrarian
Critical Filesystem checks ***FAILED***
OVERALL FAILURE
press any key to continue on to the final results...
  All tests complete            RESULTS
  Installed Packages:        ***PASSED*** libnotify-bin
  Notifications Test:        ***FAILED***
  XBMC Communications:       ***FAILED***
  File System Checks:        ***FAILED***
  mythicalTesting:           COULD NOT COMPLETE
  Overall mythicalReadiness: ***FAILURE***
-----End of diagnostics-----



- outleradam - 2010-05-17 02:40

Alternate dirs holding it up?

Give me a few and I'll code up some alternate dir selections. I use original dir.


- outleradam - 2010-05-17 02:57

I'm working on it now, and about to test, however, if you edit /usr/local/bin and change the lines which say AlternateShowDir=, AlternateMoveDir= and AlternateMovieDir= you will be in business. That's the old-skool way of doing it. mythicalSetup is the easy way.


- outleradam - 2010-05-17 03:19

Just added another method for you to use.

Global Settings for mythicalLibrarian have been implemented.

Code:
echo "AlternateMoveDir=~/episodes">/etc/mythicalLibrarian/globalSettings
echo "AlternateShowDir=~/Showings">etc/mythicalLibrarian/globalSettings
echo "AlternateMovieDir=~/Movies"/etc/mythicalLibrarian/globalSettings
you may have to mkdir /etc and mkdir /etc/mythicalLibrarian

Global Settings will override any mythicalLibrarian setting.

Code:
#######################USER SETTINGS##########################
###Stand-alone mode values###
#SYMLINK has 3 modes.  MOVE|LINK|Disabled: Default=MOVE
#Create symlink in original dir from file after 'MOVE' | Do not move, just create a sym'LINK' | move the file, symlinking is 'Disabled'
SYMLINK=MOVE
#MoveDir is the folder which mythicalLibrarian will move the file.  No trailing / is accepted eg. "~/videos"
MoveDir="/home/mythtv/Episodes"  #<------THIS VALUE MUST BE SET-------
#AlternateMoveDir will act as a seccondary MoveDir if the primary MoveDir fails.  No trailing / is accepted eg. "~/videos"
AlternateMoveDir=/home/mythtv/Episodes
#If UseOriginalDir is Enabled, original dir will override MoveDir.  Useful for multiple recording dirs.
#UseOriginalDir will separate generic shows from episodes and movies. Enabled|Disabled
UseOriginalDir=Enabled
#When Enabled, mythicalLibrarian will move the file to a folder of the same name as the show. This is not affected by UseOriginalDir. Enabled|Disabled
UseShowNameAsDir=Enabled
#Internet access Timeout in seconds: Default Timeout=50 (seconds)
Timeout=50
#Update database time in secconds, Longer duration means faster processing time and less strain on TheTvDb. Default='84000' (1 day)
UpdateDatabase=1
#mythicalLibrarian working file dir: Default=~/mythicalLibrarian (home/username/mythicalLibraian)
mythicalLibrarian=~/.mythicalLibrarian
#FailSafe mode will enable symlinks to be formed in FailSafeDir if the move or symlink operation fails. Enabled|Disabled
FailSafeMode=Enabled
#FailSafeDir is used when the file cannot be moved to the MoveDir. FailSafe will not create folders. eg. /home/username
FailSafeDir="/home/mythtv/FailSafe"  #<------THIS VALUE MUST BE SET-------
#DirTracking will check for and remove the folders created by mythicalLibrarian
DirTracking=Enabled
#the following line contains the API key from www.TheTvDb.Com. Default project code: 6DF511BB2A64E0E9
APIkey=6DF511BB2A64E0E9
#Language setting
Language=en
  
###Database settings###
#MythTV MYSQL access allows addition of movies, comskip data, and improves accuracy of episode recognition.
#Database access Enabled|Disabled
Database=Enabled
#Database Type Default=MythTV
DatabaseType=MythTV
#Guide data type: SchedulesDirect currently covers DataDirect, MC2XML and others.  NoLookup will bypass tvdb scanning
#Options: SchedulesDirect|NoLookup
GuideDataType=SchedulesDirect
#MySQL User name: Default="mythtv"
MySQLuser="mythtv"
#MySQL Password: Default="mythtv"
MySQLpass="mythtv" #<------THIS VALUE MUST BE SET-------
#MySQL Myth Database: Default="mythconverg"
MySQLMythDb=mythconverg
#Primary Movie Dir. mythicalLibrarian will attempt to move to this dir first. No trailing / is accepted eg. "~/videos"
PrimaryMovieDir="/home/mythtv/Movies" #<------THIS VALUE MUST BE SET-------
#AlternateMoveDir will act as a Seccondary move dir if the primary moive dir fails
AlternateMovieDir="/home/mythtv/Movies"
#ShowStopper = Enabled prevents generic shows and unrecognized episodes from being processed
ShowStopper="Enabled"
#Primary Show Dir. mythicalLibrarian will attempt to move to this dir first. No trailing / is accepted eg. "~/videos"
PrimaryShowDir="/home/mythtv/Showings" #<------THIS VALUE MUST BE SET-------
#AlternateMoveDir will act as a Seccondary move dir if the primary moive dir fails
AlternateShowDir="/home/mythtv/Showings" #<------THIS VALUE MUST BE SET-------
#CommercialMarkup will generate comskip files for recordings when they are moved. Enabled|Disabled
CommercialMarkup=Enabled
#CommercialMarkupCleanup will execute a maintenance routine which will remove comskip files if they are not needed
CommercialMarkupCleanup=Enabled

###Reporting/Communications###
#Enables debug mode.  This is a verbose mode of logging which should be used for troubleshooting.  Enabled|Disabled
DEBUGMODE=Enabled
#DailyReport provides a local log of shows added to your library per day. Enabled|Disabled
DailyReport=Enabled
#Notify tells mythicalLibrarian to send a notification to GNOME Desktop upon completion. Enabled|Disabled
Notify=Enabled
#If notifications are enabled, NotifyUserName should be the same as the user logged into the GNOME Session. (your username)
NotifyUserName="MYUSERNAME" #<------THIS VALUE MUST BE SET-------
#Send a notification to XBMC to Update library upon Successful move job Enabled|Disabled
XBMCUpdate=Enabled
#Send a notification to XBMC to cleanup the library upon Successful move job Enabled|Disabled
XBMCClean=Enabled
#Send Notifications to XBMC UI when library is updated Enabled|Disabled
XBMCNotify=Enabled
#Ip Address and port for XBMC Notifications Eg.XBMCIPs=( "192.168.1.110:8080" "192.168.1.111:8080" "XBOX:8080" )
XBMCIPs=( "192.168.1.110:8080" "XBOX:8080" ) #<------THIS VALUE MUST BE SET-------
#maxItems controls the number of items in the RSS. RSS Can be activated by creating a folder in /var/www/mythical-rss.  
maxItems=8  

#########################USER SETTINGS##########################



- kvandesteeg - 2010-05-17 04:40

Got it version 444 working from the command line.

I gave up on the setup script - the filesystem test was failing because of running the script under sudo su
running the diagnostics under sudo, it worked as expected

i also had to add mysql's path to /etc/profile
Code:
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/mysql/bin"

Now that it's working ... how do i change it? Big Grin
specifically for movies, i don't want directory/movie title/movie title(year).mpg i'd rather have directory/movie title (year).mpg

Was that just because i ran it from the command line?

I haven't seen the output from a live recoding from MythTV yet.

I haven't gotten the RSS fixed yet, but i'm using mythweb's rss feed for now.


Permissions Error - krschuerman - 2010-05-17 05:52

I just started using mythicalLibrarian and ran into an issue when working with recordings in one particular folder in my MythTV storage group.

Excerpt from the output.log

Code:
MOVING FILE: '/home/shared/media/MythTV.Storage/1065_20100423073000.mp4' to '/home/shared/media/Episodes/Mickey Mouse Clubhouse/Season 1/Mickey Mouse Clubhouse S01E16 (Plutos Puppy-sitting Adventure).Storage/1065_20100423073000.mp4'

PERMISSION ERROR WHILE MOVING

It took me a while to figure out why this was failing, turns out that it didn't like the "." in the folder name. I changed MythTV.Storage to MythTV_Storage and the error went away.

Sherm


- outleradam - 2010-05-17 06:22

^^ Very interesting. Can you post the entire output.log please? I would really like to see it. A period in the name should not effect normal operations. mythicalLibrarian should be looking for the last period in the name, not the first.


kvandesteeg Wrote:Got it version 444 working from the command line.

I gave up on the setup script - the filesystem test was failing because of running the script under sudo su
running the diagnostics under sudo, it worked as expected

i also had to add mysql's path to /etc/profile
Code:
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/mysql/bin"
Now that it's working ... how do i change it? Big Grin
specifically for movies, i don't want directory/movie title/movie title(year).mpg i'd rather have directory/movie title (year).mpg

Was that just because i ran it from the command line?

I haven't seen the output from a live recoding from MythTV yet.

I haven't gotten the RSS fixed yet, but i'm using mythweb's rss feed for now.

Rss feeds are currently located at /var/www/mythical-rss mythicalLibrarian checks for that folder. If you create that folder, it will start creating RSS feeds.

Where does mac keep it's apache server?


- kvandesteeg - 2010-05-17 13:50

outleradam Wrote:Rss feeds are currently located at /var/www/mythical-rss mythicalLibrarian checks for that folder. If you create that folder, it will start creating RSS feeds.

Where does mac keep it's apache server?
content is shared from
/Library/WebServer
apache is configured in
/etc/apache2/

how do i change the naming convention on the files, by type?


- outleradam - 2010-05-17 14:37

I keep the naming convention fixed for simplicity. I'm not troubleshooting people's naming conventions. If you want to go in and manually edit the code, you can find the section called categoricNamingConventions () and traditionalNamingConventions () and you can make your changes there. You can find the variables you have to work with at the bottom of the user settings/user jobs section. This is unsupported and will be reverted on the next update to it's defaults.


- kvandesteeg - 2010-05-17 15:33

I think i found a setting that might do what I want. If i understand the setting
UseShowNameAsDir=Enabled

Setting it to disabled ought to flatten the output structure - which will work for me.


- outleradam - 2010-05-17 21:01

You can set that perminantly by using the command

echo "UseShowNameAsDir=Enabled">/etc/mythicalLibrarian/globalSettings