Kodi Community Forum

Full Version: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Per chance, are you running the development version of Ubuntu?

my XBMC computer with mythtv on ubuntu 9.10 works great for auto-setup
my personal computer with ubuntu 10.04 development does not work for mythicalLibrarian's auto-setup.

My computer asks me if I want to download or continue, then when I continue, it rolls over and over with the downloading. Every once in a while, about 1 in 20 times it will work properly.
no, i'm running mythbuntu 9.10 which is built on ubuntu 9.10.

I've not upgraded to 10.04 yet.
this is pretty odd then. For some reason XBMC-Live works with mythicalLibrarian but not 9.10 or 10.04. Must be an extra dependency somewhere.
Do the following
Code:
sudo su
then
Code:
apt-get install curl
mkdir ~/.mythicalLibrarian && mkdir ~/.mythicalLibrarian/mythicalSetup && cd ~/.mythicalLibrarian/mythicalSetup
curl [url=http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh][color=#0000ff]http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh[/color][/url]>./mythicalSetup.sh
chmod +x ./mythicalSetup.sh && ./mythicalSetup.sh
I'm working on a non-su, sudo only version.

The glitch is that I run as root on my media center. It works fine when running as root, but not sudo root.
Upstream has pushed out recent revisions to sudo so I may have been exploiting a bug before. I'll work on this tomorrow.
outleradam Wrote:SVN 310 was just committed.
Code:
/MoveDir/Episode Title/Season number/Episode Title SxxExx (subtitle).ext
Testing now

I was able to get the script manually installed and after a few trial runs (Thanks much for that "Undo" command) it appears to be working properly in 10.04. I only ran it in scan mode on my MythTV .23 recording folder(var/lib/mythtv/recordings and have not setup the userjob yet. The install script, like others, went in a loop. The setup program in the previous version ran fine. Other than correcting permission errors on the folders the only issue I had was caused by the "useorigdir" defaulting to Enabled which caused all the recordings to be moved to a folders under
var/lib/mythtv/recordings..but again..you know that<grin>.

I can verify that:
1. If series parent folder (SeriesName) does not exist it creates it.
2. If series Season folder does not exist it creates it.
3. If series Season folder exists it uses it.
4. Files are moved to proper season folder.
5. File determined to be unknown(not in thetvdb) moved to /home/mythtv/Showings
6. Links to moved files created in /var/lib/mythtv/recordings and play in myth

Possible issue: I had XBMC opened in window mode when I tested and it locked up. I have no idea of the cause and whether it's related to mythicalLibrarian operation.

One question though. When Mythtv "Deletes" the recordings because they are expired will it just delete the link and not the actual file which is now in the XBMC library?
Thank you for reminding me about that. So far on my todo list

1. isolate permission issues with auto-install
2. create processing for Japanese database
3. auto-installer should set the follow symlinks when deleting option in mythtv (mythsetup option)
4. add pictures to categoric recordings
I installed the newest SVN on a test machine, and it looks like the new myth:\\ protocol supports recording groups. Once I got the font issues resolved (no Japanese fonts) the recordings are all viewable in XBMC, and divided into recording groups.

I will keep an eye on the progress of your project here. It is very interesting.

Thank you for all your work.
Not really a bug but last night I recorded Heroes twice. Once on analog cable via PVR-150 and once with my HDHomeRun. I do this because I'm in a fringe (not to be confused with Fringe on Fox) area and NBC doesn't alway receive well. The analog version was found in tvdb and moved to the series folder but the HD copy was not and was moved to /showings/Heroes S0E0. As both filenames are basically the same it seemed odd.

One good thing I found out about XBMC. If you have multiple folders for the same show on different drives it logically combines them. If you have a folders with 2 seasons mixed together and the same series in another folder with Season 3 and Season 4 folders XBMC combines them properly. This means that your new version will not impact previous installations where all seasons were placed into one folder.
Adam:

I'm back. mythicalLibrarian has been working great for me lately, with one minor exception: it can't seem to get episode information for Top Gear (recorded off of BBC America.) It looks like the lookup gets the correct TVDB and Zap2It series info, but then can't get the episode information, even tho thetvdb.org has the episode info listed.

Here's the most recent example, recorded yesterday. From my --doover output:

Code:
REMOVING ORPHAN /var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162).txt
REMOVING ORPHAN /var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162).nfo
Accessing MythTV DataBase:
COMMERCIAL DATA START:22413 STOP:25397
COMMERCIAL DATA START:40185 STOP:45438
COMMERCIAL DATA START:60991 STOP:65393
COMMERCIAL DATA START:75297 STOP:79097
COMMERCIAL DATA START:87473 STOP:91847
RECSTART:2010-04-26 18:00:00 MOVIEYEAR: SERIESDATE:2010-04-26
PROGRAMID:EP003351720253 ShowCategory:Auto STARS:0 RATING:0
PLOT: Jeremy makes a film about the BMW X6.
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405700
MATCH FOUND BASED ON Zap2itID:Top Gear ID:74608
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear Australia
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405702
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear classic
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272461988
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear Russia
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405702
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear Xtra
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405703
DB ABSOLOUTE Exx NR:-1 BASED ON ORIG AIR DATE:2010-04-26
FUZZY Exx NUMBER:
EPISODE: ABSOLUTE NUMBER:
JOB: mythicalLibrarian '/var/lib/mythtv/recordings/1162_20100426180000.mpg' 'Top Gear' ''
WARNING: EPISODE INFORMATION COULD NOT BE OBTAINED
HANDLING AS A GENERAL SHOWING
Please consider helping out and adding to thetvdb.
TARGET SET:/var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162)
CREATING SYMLINK IN MOVE MODE
MOVED: /var/lib/mythtv/recordings/1162_20100426180000.mpg
TO: /var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162).mpg
/usr/local/bin/mythicalLibrarian: line 788: [: missing `]'
SENDING REQUESTED COMMANDS TO:192.168.1.10:8080
RSS ENTRY SUCCESSFULLY CREATED: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: MoveModeSuccessful
done.

Here's the entry on thetvdb.org that mythicalLibrarian couldn't parse: http://thetvdb.com/?tab=episode&seriesid...4561&lid=7

(Note: I was running SVN 306 until today, when I --updated to the current SVN (352) and then ran the --doover which generated the output quoted above.)

Any ideas? Thanks!

/gkk
This isn't really Mythical but I know you've done a lot of other stuff. Say someone downloads 3 seasons of a show and they need to bulk rename them.
Original format might be something like "Dr.Who.Return.of.the.Daleks(XVID).MKV". I know Mythical can do it if I manually parse the series and episode name on the command line but is there a method whereby we can scan a directory and attempt a tvdb match then rename the file with SxxExx so that XBMC can properly import them? I guess if the filenames are consistent in that the episode name always starts at a specific position it might be possible to create a c script to parse out the Series and Episode names then run Mythical in non DB file mode. I used to use a program called TheRenamer on Windows but did'nt see a similar program for Linux.
^^ mythicalLibrarian categorizes recorded content only. There is another renamer somewhere in this Supplemental Tools forum which specializes in renaming downloaded files. I'm trying to keep everything legal and basically like a "super tivo". I do not want to get wrapped up in RIAA issues.

Dobyken Wrote:Not really a bug but last night I recorded Heroes twice. Once on analog cable via PVR-150 and once with my HDHomeRun. I do this because I'm in a fringe (not to be confused with Fringe on Fox) area and NBC doesn't alway receive well. The analog version was found in tvdb and moved to the series folder but the HD copy was not and was moved to /showings/Heroes S0E0. As both filenames are basically the same it seemed odd.

One good thing I found out about XBMC. If you have multiple folders for the same show on different drives it logically combines them. If you have a folders with 2 seasons mixed together and the same series in another folder with Season 3 and Season 4 folders XBMC combines them properly. This means that your new version will not impact previous installations where all seasons were placed into one folder.
If the information was there, then you should be able to
Code:
sudo mythicalLibrarian --doover
. If that does not work, then try deleting the ~/.mythicalLibrarian/DatabaseFolders then --doover.


GregoryK Wrote:Adam:

I'm back. mythicalLibrarian has been working great for me lately, with one minor exception: it can't seem to get episode information for Top Gear (recorded off of BBC America.) It looks like the lookup gets the correct TVDB and Zap2It series info, but then can't get the episode information, even tho thetvdb.org has the episode info listed.

Here's the most recent example, recorded yesterday. From my --doover output:

Code:
REMOVING ORPHAN /var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162).txt
REMOVING ORPHAN /var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162).nfo
Accessing MythTV DataBase:
COMMERCIAL DATA START:22413 STOP:25397
COMMERCIAL DATA START:40185 STOP:45438
COMMERCIAL DATA START:60991 STOP:65393
COMMERCIAL DATA START:75297 STOP:79097
COMMERCIAL DATA START:87473 STOP:91847
RECSTART:2010-04-26 18:00:00 MOVIEYEAR: SERIESDATE:2010-04-26
PROGRAMID:EP003351720253 ShowCategory:Auto STARS:0 RATING:0
PLOT: Jeremy makes a film about the BMW X6.
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405700
MATCH FOUND BASED ON Zap2itID:Top Gear ID:74608
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear Australia
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405702
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear classic
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272461988
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear Russia
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405702
TESTING FOR ZAP2IT SERIES ID MATCH:Top Gear Xtra
DATABASE IS MAINTAINED. TIME IS:1272393745 NEXT UPDATE IS:1272405703
DB ABSOLOUTE Exx NR:-1 BASED ON ORIG AIR DATE:2010-04-26
FUZZY Exx NUMBER:
EPISODE: ABSOLUTE NUMBER:
JOB: mythicalLibrarian '/var/lib/mythtv/recordings/1162_20100426180000.mpg' 'Top Gear' ''
WARNING: EPISODE INFORMATION COULD NOT BE OBTAINED
HANDLING AS A GENERAL SHOWING
Please consider helping out and adding to thetvdb.
TARGET SET:/var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162)
CREATING SYMLINK IN MOVE MODE
MOVED: /var/lib/mythtv/recordings/1162_20100426180000.mpg
TO: /var/lib/mythtv/recordings/Showings/Top Gear/Top Gear S0E0 (Recorded [email protected] on 1162).mpg
/usr/local/bin/mythicalLibrarian: line 788: [: missing `]'
SENDING REQUESTED COMMANDS TO:192.168.1.10:8080
RSS ENTRY SUCCESSFULLY CREATED: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: MoveModeSuccessful
done.

Here's the entry on thetvdb.org that mythicalLibrarian couldn't parse: http://thetvdb.com/?tab=episode&seriesid...4561&lid=7

(Note: I was running SVN 306 until today, when I --updated to the current SVN (352) and then ran the --doover which generated the output quoted above.)

Any ideas? Thanks!

/gkk
I can't see it right now from work, but I will check on that when I get home. Try to delete your ~/.mythicalLibrarian/DatabaseFolders then do a --doover

I think I should add a --FlushDatabase option to force mythicalLibrarian to rebuild all TVDB entries and have that executed as a last step on --update. This would help with mythicalLibrarian database upgrades.
Quote:I can't see it right now from work, but I will check on that when I get home. Try to delete your ~/.mythicalLibrarian/DatabaseFolders then do a --doover

I don't have a directory named ~/.mythicalLibrarian/DatabaseFolders (??) My ~/.mythicalLibrarian directory has a bunch of directories with show names, e.g.:

Code:
$ cd /home/mythtv/.mythicalLibrarian
$ ls -al
total 2196
drwxrwxr-x 47 mythtv mythtv    4096 2010-04-27 23:54 .
drwxr-xr-x 11 mythtv mythtv    4096 2010-04-27 10:42 ..
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-19 08:55 30 Rock
drwxrwxr-x  2 mythtv mythtv    4096 2010-03-11 22:14 A Night at the Office
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-19 08:56 Battlestar Galactica (1978)
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-19 08:56 Battlestar Galactica (2003)
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-25 22:11 Call Me Mister
drwxr-xr-x  2 mythtv mythtv    4096 2010-04-27 11:00 Clever V stupid
-rw-r--r--  1 mythtv mythtv   47093 2010-04-27 23:54 created.tracking
-rw-r--r--  1 mythtv mythtv   33374 2010-03-22 13:08 created.tracking.old
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-19 08:55 Curious George (2006)
drwxrwxr-x  2 mythtv mythtv    4096 2010-04-27 11:00 DailyReport
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-19 08:57 Damages
drwxr-xr-x  2 mythtv mythtv    4096 2010-03-18 13:23 Dinosaur Train
-rw-r--r--  1 mythtv mythtv    1499 2010-04-27 23:54 dir.tracking
-rw-r--r--  1 mythtv mythtv   24386 2010-04-27 11:30 doover.sh
-rw-r--r--  1 mythtv mythtv   88239 2010-04-27 11:28 doover.sh.old
[...snip...]

Thoughts?

/gkk
Those folders, with the exception of daily report and mythicalSetup are the database folders. I will make an easy button for it.
As of SVN 369 updater is working properly.

Also, --doMaintenance will delete all MythicalLibrarian created database folders as well as flush the disconnected folder and file cache. I'm going to call that a day!