Kodi Community Forum

Full Version: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
outleradam Wrote:ok... then lets try it this way....

Code:
x=`mysql -umythtv -pmythtv -e 'use mythconverg; select basename from recorded where starttime not like "" ;'|sed -n 2p`
mysql -umythtv -pmythtv -e "use mythconverg; select * from recorded where basename like '$x';"
MythDataGrabber $x
mythicalLibrarian $x
Please copy the output of this into a pastebin.

What's a pastebin?
pastebin.com. It's a website for posting what some might consider clutter. It keeps large amounts of what may seem like garbage to most people out of the forums. I'd like to see the information, but it will be very ugly to post it here. you just go on pastebin.com, copy, paste, possibly enter a captcha, then paste the URL it gives you here and I can see the pages of data.
outleradam Wrote:pastebin.com. It's a website for posting what some might consider clutter. It keeps large amounts of what may seem like garbage to most people out of the forums. I'd like to see the information, but it will be very ugly to post it here. you just go on pastebin.com, copy, paste, possibly enter a captcha, then paste the URL it gives you here and I can see the pages of data.

Well, I think I will just paste it here. There isn't much data!?

Code:
x=`mysql -umythtv -pmythtv -e 'use mythconverg; select basename from recorded where starttime not like "" ;'|sed -n 2p`
mysql -umythtv -pmythtv -e "use mythconverg; select * from recorded where basename like '$x';"
+--------+---------------------+---------------------+----------------+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+----------+----------+----------+---------+---------+------------+-------------+----------+----------+------------+----------------+---------------------+------------+-------+-----------------+-----------------+----------+--------+---------------+------------+-------------+-------------+-------------------------+---------------------+---------------------+-----------+---------+-----------+------------+---------+--------------+---------------------+
| chanid | starttime           | endtime             | title          | subtitle                               | description                                                                                                             | category | hostname | bookmark | editing | cutlist | autoexpire | commflagged | recgroup | recordid | seriesid   | programid      | lastmodified        | filesize   | stars | previouslyshown | originalairdate | preserve | findid | deletepending | transcoder | timestretch | recpriority | basename                | progstart           | progend             | playgroup | profile | duplicate | transcoded | watched | storagegroup | bookmarkupdate      |
+--------+---------------------+---------------------+----------------+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+----------+----------+----------+---------+---------+------------+-------------+----------+----------+------------+----------------+---------------------+------------+-------+-----------------+-----------------+----------+--------+---------------+------------+-------------+-------------+-------------------------+---------------------+---------------------+-----------+---------+-----------+------------+---------+--------------+---------------------+
|   1036 | 2010-11-03 12:30:00 | 2010-11-03 13:00:00 | Hannah Montana | Me & Mr. Jonas & Mr. Jonas & Mr. Jonas | Miley is jealous when her father spends time with the Jonas Brothers; Jackson takes aim at the world pogo stick record. | Sitcom   | Revo     |        0 |       0 |       0 |          1 |           1 | Default  |       83 | EP00798440 | EP007984400043 | 2010-11-03 13:00:21 | 1165070336 |     0 |               1 | 2007-08-17      |        0 |      0 |             0 |          0 |           1 |           1 | 1036_20101103123000.mpg | 2010-11-03 12:30:00 | 2010-11-03 13:00:00 | Default   | Default |         1 |          0 |       0 | Default      | 0000-00-00 00:00:00 |
+--------+---------------------+---------------------+----------------+----------------------------------------+-------------------------------------------------------------------------------------------------------------------------+----------+----------+----------+---------+---------+------------+-------------+----------+----------+------------+----------------+---------------------+------------+-------+-----------------+-----------------+----------+--------+---------------+------------+-------------+-------------+-------------------------+---------------------+---------------------+-----------+---------+-----------+------------+---------+--------------+---------------------+
MythDataGrabber $x
Establishing database connection
Traceback (most recent call last):
  File "/usr/local/bin/MythDataGrabber", line 201, in <module>
    rec = db.searchRecorded(basename=options['filename']).next()
  File "/usr/lib/python2.6/dist-packages/MythTV/MythFunc.py", line 60, in __call__
    % (self.__name__, key))  
TypeError: searchRecorded got an unexpected keyword arguemnt 'basename'
mythicalLibrarian $x
mythicalLibrarian --:invalid pattern

usage: mythicalLibrarian 'path/to/file' ['title'] ['subtitle']
usage: mythicalLibrarian --mode [parameter] [parameter]
items in brackets are [context sensitve]
ensure you have set the user settings set properly and run
--mythicalDiagnostics before using mythicalLibrarian
in --scan mode or as a UserJob.
For more information type: mythicalLibrarian --help or --special
done.
I didn't think you'd make your terminal wide enough to handle the table. it would have been multiple lines of stuff.

You will not need to use the python bindings for this at all. Your version will handle the 'old skool' methods. You may have a bad password in mythicalSetup.
ok... try this:
Code:
x=`mysql -umythtv -pmythtv -e 'use mythconverg; select basename from recorded where starttime not like "" ;'|sed -n 2p`
filename=`find $x|sed -n 1p`
mythicalLibrarian "$filename"
outleradam Wrote:I didn't think you'd make your terminal wide enough to handle the table. it would have been multiple lines of stuff.

You will not need to use the python bindings for this at all. Your version will handle the 'old skool' methods. You may have a bad password in mythicalSetup.
ok... try this:
Code:
x=`mysql -umythtv -pmythtv -e 'use mythconverg; select basename from recorded where starttime not like "" ;'|sed -n 2p`
filename=`find $x|sed -n 1p`
mythicalLibrarian "$filename"

Code:
x=`mysql -umythtv -pmythtv -e 'use mythconverg; select basename from recorded where starttime not like "" ;'|sed -n 2p`
filename=`find $x|sed -n 1p`
find: `1036_20101103123000.mpg': No such file or directory
mythicalLibrarian "$filename"
mythicalLibrarian --:invalid pattern

usage: mythicalLibrarian 'path/to/file' ['title'] ['subtitle']
usage: mythicalLibrarian --mode [parameter] [parameter]
items in brackets are [context sensitve]
ensure you have set the user settings set properly and run
--mythicalDiagnostics before using mythicalLibrarian
in --scan mode or as a UserJob.
For more information type: mythicalLibrarian --help or --special
done.
And I am not using python bindings.
can you please email me the file /usr/local/bin/mythicalLibrarian? I'd like to look at it and see if I can reproduce /identify the problems.

it's my xbmc.org profile name @hotmail.com <to prevent spam scanners.

I will work with it tomorrow.
issues are fixed in the SVN. I tested pretty well. This was an issue that stemmed from consolidation of database information in preparation for python bindings. Review your logs while using mythicalLibrarian.

Since you are using SYMLINK=Disabled, please make sure that any S0E0 episodes are not deleted. After you run a scan do a --doover to see what's left over. mythicalLibrarian will not delete database information even when comanded in Disabled mode when there is not enough information to identify an episode.

Let me know if there are any problems with the SVN and I will be committing to stable shortly.
outleradam Wrote:issues are fixed in the SVN. I tested pretty well. This was an issue that stemmed from consolidation of database information in preparation for python bindings. Review your logs while using mythicalLibrarian.

Since you are using SYMLINK=Disabled, please make sure that any S0E0 episodes are not deleted. After you run a scan do a --doover to see what's left over. mythicalLibrarian will not delete database information even when comanded in Disabled mode when there is not enough information to identify an episode.

Let me know if there are any problems with the SVN and I will be committing to stable shortly.

So far, it looks good! One problem still remains: when SYMLINK=Disabled, the DB entries are still not being deleted.
Sorry it took 3 days. I am sorting out making a Linux kernel work with a proprietary home security system. It is detecting cameras as floppy disks...

Try the current svn. I can't test but it will provide information.
outleradam Wrote:Sorry it took 3 days. I am sorting out making a Linux kernel work with a proprietary home security system. It is detecting cameras as floppy disks...

Try the current svn. I can't test but it will provide information.

Nope. Not yet...
Post the debug log.
outleradam Wrote:Post the debug log.

Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Thu Nov 11 05:00:29 PST [email protected]@@@@@@@@@@@@@
Revision 762 Wed Nov 10 21:55:28 PST 2010 on Linux
PERFORMING MAINTENANCE ROUTINE
SEARCHING: www.TheTvDb.com SHOW NAME: Conan EPISODE:
DATE: 2010-11-10 FILE NAME: /var/lib/mythtv/recordings/1042_20101111040000.mpg
NO ZAP2IT ID FOUND. USING FUZZY LOGIC FOR TITLE RECOGNITION.
Please update TheTvDb.com
FUZZY LOGIC SHOW NAME:  ID:
SEARCH FOUND: ID#:
SERIES ID WAS NOT FOUND. THETVDB MAY BE DOWN. TRY USING A showTranslations
DB ENTIRES- RECSTART:2010-11-11 04:00:00- MOVIE:- ORIGAIRDATE:2010-11-10- CHID:1042- CAT:Talk-
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/media/recordings/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/media/recordings/Episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled
INPUT SHOW NAME:Conan- LOCAL SHOW NAME TRANSLATION:- ShowName:Conan
DATABASE UPDATED:- TVDB LAST UPDATED:- CURRENT:-
RESOLVED SERIES ID:- RESOVED SHOW NAME:-
INPUT EPISODE NAME:- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:-
SEASON:- EPISODE:- SYMLINK MODE:Disabled- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:36947044kB- WORKINGDIR:1 FREE:36947044kB-
MOVEDIRWRITABLE:1- FREE:12866778092kB- ALTERNATEMOVEDIR:1- FREE:12866778092kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:12866778092kB-ALTERNATESHOWDIRWRITABLE:1- FREE:12866778092kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:12866778092kB- ALTERNATEMOVIEDIR:1- FREE:12866778092kB-
DATABASE TYPE:schedulesdirect1-
RECSTART:2010-11-11 04:00:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2010-11-10-
PROGRAMID:EP013323920003- CHANNEL ID:1042- CATEGORY:Talk- GOFORDOOVER:0-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:1332392- MATCHED TVDB SERIES ID:-
PLOT: Actor Jon Hamm; actress Charlyne Yi; Fistful of Mercy performs.
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB
GENERIC GUIDE DATA WAS SUPPLIED TYPE: Series With Episode Data- Conan,
WARNING:%%%%%%%%%%PROGRAM GUIDE DATA IS NOT COMPLETE%%%%%%%%%%%%%%%%%%%%
MOVING FILE: '/var/lib/mythtv/recordings/1042_20101111040000.mpg' to '/media/recordings/Showings/Conan/Conan S0E0 ( Recorded [email protected] on 1042).mpg'
@@@@@@@@@@@@@OPERATION COMPLETE Thu Nov 11 05:02:43 PST 2010 @@@@@@@@@@@@@@@@
Please run mythicalLibrarian --doover. It should now recognize that showing...



It loooks like your internet connection was down at the time.
Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Thu Nov 11 17:25:14 CST [email protected]@@@@@@@@@@@@@
Revision 761 Sat Nov 6 20:38:08 CDT 2010 on Linux
REMOVING ORPHAN FOLDER:/root/.mythicalLibrarian/Episodes/Conan (2010)
REMOVING ORPHAN FOLDER:/root/Episodes/Conan (2010)/Season 1
SEARCHING: www.TheTvDb.com SHOW NAME: conan EPISODE:
DATE: 2010-11-10 FILE NAME: testfile.ext
SEARCH FOUND:Conan (2010) ID#: 194751
MATCH FOUND BASED ON Zap2itID:Conan (2010) ID:194751
###################DEBUG MODE ENABLED####################
MY USER NAME:root-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/root/.mythicalLibrarian-
MOVE DIR:/root/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/root/Episodes-
USE ORIGINAL DIR:Enabled NOTIFICATIONS:Disabled
INPUT SHOW NAME:conan- LOCAL SHOW NAME TRANSLATION:- ShowName:conan
DATABASE UPDATED:- TVDB LAST UPDATED:1289581813- CURRENT:1289518834-
RESOLVED SERIES ID:194751- RESOVED SHOW NAME:Conan (2010)-
INPUT EPISODE NAME:- ABSOLOUTE EPISODE NUMBER:3- RESOLVED EPISODE NAME:Dead Men Don't Wear Spanx-
SEASON:S01- EPISODE:E03- SYMLINK MODE:MOVE- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:53905904kB- WORKINGDIR:1 FREE:53905904kB-
MOVEDIRWRITABLE:1- FREE:53905932kB- ALTERNATEMOVEDIR:1- FREE:53905920kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:53905920kB-ALTERNATESHOWDIRWRITABLE:1- FREE:53905912kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:53905920kB- ALTERNATEMOVIEDIR:1- FREE:53905920kB-
DATABASE TYPE:-
RECSTART:- MOVIE YEAR:- ORIGINAL SERIES DATE:2010-11-10-
PROGRAMID:EP013323920003- CHANNEL ID:1042- CATEGORY:- GOFORDOOVER:0-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS: RATING:0
ZAP2IT SERIES ID:1332392- MATCHED TVDB SERIES ID:194751-
PLOT:
####################END OF DEBUG LOG#####################
mythicalLibrarian will not delete relevant mythtv database entries and pictures unless a positive match was made.

Do not update right now. I'm revising title matching for a serious speed increase. It is not stable.
The SVN is back up. It has been refactored for very quick television show recognition.
outleradam Wrote:Please run mythicalLibrarian --doover. It should now recognize that showing...

It loooks like your internet connection was down at the time.

mythicalLibrarian will not delete relevant mythtv database entries and pictures unless a positive match was made.

It is doing the same thing for ALL my recordings, whether it gets a positive match or not.

Regarding --doovers, is there any way to automate this? What about performing a --doover every time mythicalLibrarian gets run, or at least have a setting for this functionality.