Kodi Community Forum
SortTV: Sort TV episodes, movies, and music into directories for xbmc (Linux/Win/Mac) - Printable Version

+- Kodi Community Forum (http://forum.kodi.tv)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (/forumdisplay.php?fid=116)
+--- Thread: SortTV: Sort TV episodes, movies, and music into directories for xbmc (Linux/Win/Mac) (/showthread.php?tid=75949)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45


- cliffe - 2011-01-19 07:39

rflores2323 Wrote:Just wondering if you were going to port this to an official addon? I think it would be alot easier for users if it was an addon and you could configure it thru xbmc etc.. is this in your plans?

To be honest, I hadn't given it much thought. I like the idea of a front end in Python, with the Perl SortTV script in the same directory. That way you could install and configure and run SortTV through xbmc. The front-end should be relatively straightforward to code. Basically it could just edit the config, launch SortTV.pl, and maybe also setup scheduled sorts.

Does anyone reading this want to have a go at coding the front-end?


- e2zippo - 2011-01-19 11:39

cliffe Wrote:Woah. No that is not normal...

Try adding the command line argument "--no-network" and let me know how long that takes.

Perl on Windows (strawberry) seems to take a really long time to recover when it tries to connect to a non-existing network host.

It could be trying to connect to xbmc. If no-network speeds it up, try commenting that line out of the config.

Let me know how that goes.

Woot, what a difference, I didn't even have time to blink, it moved about 1gb instantly, so I guess it tries to connect to XBMC.

I suppose XBMC must be running for it to work, and I didn't have it running, Rolleyes

Is this correct btw?

xbmc-web-server:localhost:8080

Thanks a lot for your fast support!


[HACK/MOD] Make SortTV ignore non-video files - TravellingGuy - 2011-01-20 04:43

I'm a sysadmin, not a programmer, when I dabble, it's usually PHP, not Perl. And regex confounds me.

Anyway, I do want to make this better and make it a config option. I quickly did it this way, as it served my purposes for the short term.

This little bit of code is my very first draft at having sorttv ignore anything except .mkv and .avi files. i.e. it won't try and sort a file unless it has a .mkv or .avi extension.

After this line (line 122 in sorttv 1.19):
Code:
my $filename = filename($file);

Put these lines:
Code:
# Let .mkv and .avi files pass through (get sorted), skip any other filetypes
    if (-f $file && $filename =~ /(.*)mkv/ ) {
        #
    } elsif (-f $file && $filename =~ /(.*)avi/ ) {
        #
    } elsif (-f $file) {
        out("std", "IGNORING FILE: $filename\n");
        next;
    }

The end result:
Code:
        my $filename = filename($file);
        # Let .mkv and .avi files pass through, any other filetypes, don't do anything with
        if (-f $file && $filename =~ /(.*)mkv/ ) {
            #
        } elsif (-f $file && $filename =~ /(.*)avi/ ) {
            #
        } elsif (-f $file) {
            out("std", "IGNORING FILE: $filename\n");
            next;
        }
        # Regex for tv show season directory
        if(-l $file) {



Shell script to help sort video files - TravellingGuy - 2011-01-20 04:52

Hey guys.

Thanks to Cliffe and his most excellent sorttv (and my crap hack...see post above this), I have a littleLinux (probably work on Mac too) shell script I use to extract media (if it's RAR'd), sort it, then have Transmission move completed downloads to a seeding directory, where they sit until the ratio is hit. Thought I'd share, in case anyone else can make use of it.

Code:
#! /bin/sh

# Things to do:
#   - ignore sample video files (usually included in rar archives)

# Unrar any rar'd files.
# I ran into a rar archive where every rar file had .rar on the end (instead of the more normal .rar, .r01, .r02....).
# Use "-o-" switch to make unrar automatically not prompt and not overwrite destination file
find /home/todd/Download/Torrents/Downloaded -iname "*.rar" -execdir unrar x -o- '{}' ';'

/home/todd/sorttv/sorttv.pl

# either hard-code the MOVEDIR variable here...
MOVEDIR=/home/todd/Download/Torrents/Seeding # the folder to move completed downloads to

# use transmission-remote to get torrent list from transmission-remote list
# use first sed to delete first / last line of output, and remove leading spaces
# use cut to get first field from each line
# use second sed to remove any * beside the torrent number (it will have a * if the torrent has an error)
TORRENTLIST=`transmission-remote --auth=username:password --list | sed -e '1d;$d;s/^ *//' | cut --only-delimited --delimiter=" " --fields=1 | sed -e 's/*//'`

# for each torrent in the list
for TORRENTID in $TORRENTLIST
do
    echo "---------------------------------------------------------------------------------------------------------------------"
    TORRENT_NAME=`transmission-remote --auth=username:password --torrent $TORRENTID --info | grep "Name: "`

    # check if torrent download is completed
    DL_COMPLETED=`transmission-remote --auth=username:password --torrent $TORRENTID --info | grep "Percent Done: 100%"`

    # If 100% of torrent is downloaded, move to Seeding directory
    if [ "$DL_COMPLETED" != "" ]; then
        echo "$TORRENT_NAME is complete, moving to $MOVEDIR."
        transmission-remote --auth=username:password --torrent $TORRENTID --move $MOVEDIR
    fi
done

Notes: Replace username and password with your Transmission username and password (if you've set one). The Transmission part of this, I actually found somewhere else on the internet, some guy had written a script to move sorted files into a seeding directory. I just grabbed what he did and modified it to fit my needs.


- Holdengts - 2011-01-22 11:01

This script is awesome.
I had some trouble getting it going but have succeeded in the end in most part.
One suggestion I have is to make it so you can define all the TV directories you wish to sort to, and the movie directory's. I was thinking that if it could be like in XBMC in the way that you can create a source with multiple folders, but only in reverse, where you create a destination that has multiple folders.
As i have two external USB hdd that i use to store my TV episodes on, and a external for my dvd rip movies and a 3 drive stripe raid array for my HD movies.
I can not get the script to check both drives for existing TV Series. It will check one and not the other, so when it checks which ever drive and not find the folder it will create it, which means that i end up with a folder on each drive for the same TV series.
If there is a way to set it to scan both folders / drives already can someone please point me in the right direction.


- nitrag - 2011-01-22 18:14

First of all, awesome work. SortTV is a great! great! script. Currently I have two scripts
1) Moves finished torrent files to "complete" directory
2) SortTV sorts the "complete" directory

Voila!

Is there a way of incorporating the two? (I use torrentflux-b4rt)

Code:
#!/usr/bin/perl

my $home="/home/torrents/matador/";
my $tv_home="/home/torrents/complete/";
my $completed = `/var/www/torrentflux/bin/fluxcli.php transfers | grep Done | awk '{print \$2}'`;
my @torrents = split(/\n/,$completed);
my $dir,$download,$show_name;
my @results;


# Get possible locations for torrents. Replace spaces for ease of processing.
#$dir = `ls $tv_home | awk '{print \$9,\$10}' | sed 's/ /./g'`;
$dir = `ls -ap $tv_home | grep / | sed 's/ /./g'`;
my @directories = split(/\n/,$dir);


foreach (@torrents) {
    $download = $_;
    # remove .torrent extension.
    $download =~ s/(.+)\.[^.]+$/$1/;
  
    `mv $home/$download $tv_home/$download`;
    `/var/www/torrentflux/bin/fluxcli.php delete $download.torrent`;
}

The code checks that the torrent status is "Done" then moves the file and removes the torrent from download. Small bug, it only looks for a file. If it sees a folder instead, it hiccups and still deletes the torrent. Maybe someone could improve/fix that for me?


- Holdengts - 2011-01-24 09:08

nitrag Wrote:First of all, awesome work. SortTV is a great! great! script. Currently I have two scripts
1) Moves finished torrent files to "complete" directory
2) SortTV sorts the "complete" directory

Voila!

Is there a way of incorporating the two? (I use torrentflux-b4rt)

Code:
#!/usr/bin/perl

my $home="/home/torrents/matador/";
my $tv_home="/home/torrents/complete/";
my $completed = `/var/www/torrentflux/bin/fluxcli.php transfers | grep Done | awk '{print \$2}'`;
my @torrents = split(/\n/,$completed);
my $dir,$download,$show_name;
my @results;


# Get possible locations for torrents. Replace spaces for ease of processing.
#$dir = `ls $tv_home | awk '{print \$9,\$10}' | sed 's/ /./g'`;
$dir = `ls -ap $tv_home | grep / | sed 's/ /./g'`;
my @directories = split(/\n/,$dir);


foreach (@torrents) {
    $download = $_;
    # remove .torrent extension.
    $download =~ s/(.+)\.[^.]+$/$1/;
  
    `mv $home/$download $tv_home/$download`;
    `/var/www/torrentflux/bin/fluxcli.php delete $download.torrent`;
}

Hi
The code checks that the torrent status is "Done" then moves the file and removes the torrent from download. Small bug, it only looks for a file. If it sees a folder instead, it hiccups and still deletes the torrent. Maybe someone could improve/fix that for me?


Does this only do its thing if the torrent has finished seeding or when the torrent is 100% downloaded.

I have the sorttv script running (awesome script) and it does not move the files until i remove the torrent from utorrent.
Which is a pain as i am trying to get everything automated and not have to do anything (unless i have to).

Cheers
holdengts


- e2zippo - 2011-01-24 19:14

Holdengts Wrote:Does this only do its thing if the torrent has finished seeding or when the torrent is 100% downloaded.

I have the sorttv script running (awesome script) and it does not move the files until i remove the torrent from utorrent.
Which is a pain as i am trying to get everything automated and not have to do anything (unless i have to).

Cheers
holdengts

Couldn't agree more, the script can't move the file when it's in use by utorrent (not the scripts fault).
But is there anything that can be done about it?


- nitrag - 2011-01-26 01:44

Correct me if I'm wrong (I tested this on like ver 1.0 of sorttv) but in Linux it doesn't tell that the file is being written to, it just moves it. I'm using SortTV in a torrentflux-b4rt setup.

The above is for when the torrent is completely finished ("Done"). Leeching -> Seeding (X%) - > Done.


- Helioy - 2011-01-28 23:44

Hi Cliffe,
Thanks for your script. Took me hell of a time getting to work on Windows Smile
For people reading you have to download dmake here: http://johnbokma.com/perl/make-for-windows.html and copy it into ~ C:\Perl64\bin
before trying to install and compile stuff.

Now to my question:
In order to use the proper/repack stuff do I need to let the script to rename my tv shows?

I just want the script to look in a folder and check if proper is present and if proper IS present then remove non-proper. No rename no nothing.

Scenario:
Quote:Show.S01E13.720p.HDTV.X264-.mkv <-- below removes this
Show.Family.S01E13.PROPER.720p.HDTV.x264-.mkv / Show.Family.S01E13.REPACK.720p.HDTV.x264-.mkv <-- below removes this
Show.Family.S01E13.REAL.PROPER.720p.HDTV.x264-.mkv

Is that possible? I think it only works if I use rename and it checks if file is present etc. and then overwrite. Would it be possible to add support to non-renamed files?


I also got this error:
Quote:Unrecognized escape \F passed through in regex; marked by <-- HERE in m/C:\F <-- HERE oer// at sorttv.pl line 121.

Config:
Quote:directory-to-sort:C:\Foer\
directory-to-sort-into:C:\Efter\
log-file:C:\sorttv.log



- rflores2323 - 2011-01-29 00:48

anyway you can add unrar support to be able to go into folders and unrar and then move to another folder, and then rename ?


- SumOne - 2011-01-31 00:29

Cliffe,

First thank you, and the other contributors as well, for the great work done on this script.

That said, I am having an issue with files names in particular format that has me stumped at the moment... it seems like it is parsing the year in the filename as the Season number instead of picking it up as a date. I have tried running sorttv with match-files-named-showname-episodetitle set to both TRUE and FALSE... but that seems to make no difference.

I think the output below provides pretty good detail around what is happening, if you or anyone else can help i would appreciate it. Also, if there is additional information I should supply please let me know.

Thanks!

*** Update: zeroing in on line 188 from the first error message led me to change season-double-digits from TRUE to FALSE, and everything worked fine. That provides me a decent workaround for now, and hopefully it will be useful to anyone else hitting the same issue. If I can figure out a way to patch the code to eliminate the issue I'll gen a patch and link to it from here.

Quote:/usr/bin/perl /home/xbmc/.sorttv/sorttv.pl --directory-to-sort:~/BT/completed --directory-to-sort-into:/freenas/TV --xbmc-webserver:localhost:8080 --match-files-named-showname-episodetitle:FALSE

Quote:INFO: trying to move The.Daily.Show season 2011 episode 7
INFO: making show directory: /freenas/TV/The Daily Show with Jon Stewart
DOWNLOAD: downloading images for The Daily Show with Jon Stewart
Argument "2011.01.12" isn't numeric in sprintf at /home/xbmc/.sorttv/sorttv.pl line 188.
INFO: trying to move The.Daily.Show season 2011 episode 7
INFO: found a matching show:
/freenas/TV/The Daily Show with Jon Stewart
INFO: making season directory: /freenas/TV/The Daily Show with Jon Stewart/Season 2011
DOWNLOAD: downloading season image for The Daily Show with Jon Stewart
INFO: found a matching show:
/freenas/TV/The Daily Show with Jon Stewart
INFO: found a matching season:
/freenas/TV/The Daily Show with Jon Stewart/Season 2011
INFO: Fetching episode title for The Daily Show with Jon Stewart Season 2011 Episode 7.
Warning: TBDB::API: Invalid season 2011 for 71256
WARN: Could not get episode title for The Daily Show with Jon Stewart Season 2011 Episode 7.
COPY: sorting /home/xbmc/BT/completed/The.Daily.Show.2011.01.12.Gov.Tim.Pawlenty.HDTV.XviD-FQM to /freenas/TV/The Daily Show with Jon Stewart/Season 2011/The.Daily.Show.with.Jon.Stewart.S2011E07
Warning: TBDB::API: Invalid season 2011 for 71256



New version available - cliffe - 2011-02-07 19:20

Thanks for the patience. I have been pretty busy, what with submitting my PhD thesis (yay!) Big Grin

Anyway, more good news: here is v1.20 of SortTV!

This new version has these new features:
  • --whitelist:pattern
    [INDENT]Only copy if the file matches one of these patterns
    Uses shell-like simple pattern matches (eg *.avi)
    This argument can be repeated to add more rules[/INDENT]
  • --blacklist:pattern
    [INDENT]Don't copy if the file matches one of these patterns
    Uses shell-like simple pattern matches (eg *.avi)
    This argument can be repeated to add more rules[/INDENT]
  • --extract-compressed-before-sorting:[TRUE|FALSE]
    [INDENT]Extracts the contents of archives (.zip, .rar) into the directory-to-sort while sorting
    If "rar" and "unzip" programs are available they are used. If you don't have them you will need to install them on your system or this feature won't work.
    If not specified, TRUE[/INDENT]
  • Provides a warning if downloading images on Windows (since this feature currently doesn't work).
  • Should handle directory names correctly (Windows style slashes and special characters such as "-").
  • Fix for date named formatting.

This addresses things brought up recently, and adds some features you guys wanted.

As always, your testing, bug reports, and feedback are appreciated.

Thanks to everyone who have donated, your encouragement is really appreciated! If you find this software helpful, and haven’t already shown your support, please consider a $5 donation, which will give us both warm fuzzy feelings. Click for more information:
http://sourceforge.net/donate/index.php?group_id=330009

You can get the new version from:
https://sourceforge.net/projects/sorttv/

TravellingGuy Wrote:I'm a sysadmin, not a programmer, when I dabble, it's usually PHP, not Perl. And regex confounds me.

Anyway, I do want to make this better and make it a config option. I quickly did it this way, as it served my purposes for the short term.

This little bit of code is my very first draft at having sorttv ignore anything except .mkv and .avi files. i.e. it won't try and sort a file unless it has a .mkv or .avi extension.

Thanks. Since you demonstrated that this is a feature that is really wanted, I have added this to SortTV. I didn't end up using your code, but thanks for your contribution. Please feel free to compare your solution to mine and discuss.

Holdengts Wrote:This script is awesome.
I had some trouble getting it going but have succeeded in the end in most part.
One suggestion I have is to make it so you can define all the TV directories you wish to sort to, and the movie directory's. I was thinking that if it could be like in XBMC in the way that you can create a source with multiple folders, but only in reverse, where you create a destination that has multiple folders.
As i have two external USB hdd that i use to store my TV episodes on, and a external for my dvd rip movies and a 3 drive stripe raid array for my HD movies.
I can not get the script to check both drives for existing TV Series. It will check one and not the other, so when it checks which ever drive and not find the folder it will create it, which means that i end up with a folder on each drive for the same TV series.
If there is a way to set it to scan both folders / drives already can someone please point me in the right direction.

Check out this post.

TravellingGuy Wrote:Hey guys.

Thanks to Cliffe and his most excellent sorttv (and my crap hack...see post above this), I have a littleLinux (probably work on Mac too) shell script I use to extract media (if it's RAR'd), sort it, then have Transmission move completed downloads to a seeding directory, where they sit until the ratio is hit. Thought I'd share, in case anyone else can make use of it.

Thanks for sharing, I am sure that other Transmission users will find this helpful. I added some rar support to SortTV, so that should help simplify your script.

nitrag Wrote:First of all, awesome work. SortTV is a great! great! script. Currently I have two scripts
1) Moves finished torrent files to "complete" directory
2) SortTV sorts the "complete" directory

Voila!

Is there a way of incorporating the two? (I use torrentflux-b4rt)

The code checks that the torrent status is "Done" then moves the file and removes the torrent from download. Small bug, it only looks for a file. If it sees a folder instead, it hiccups and still deletes the torrent. Maybe someone could improve/fix that for me?

Thanks. Not sure that it fits in the scope of sorttv.pl, but I am sure that other torrentflux-b4rt users will find that helpful.

Helioy Wrote:Hi Cliffe,
Thanks for your script. Took me hell of a time getting to work on Windows Smile
For people reading you have to download dmake here: http://johnbokma.com/perl/make-for-windows.html and copy it into ~ C:\Perl64\bin
before trying to install and compile stuff.

Now to my question:
In order to use the proper/repack stuff do I need to let the script to rename my tv shows?

I just want the script to look in a folder and check if proper is present and if proper IS present then remove non-proper. No rename no nothing.

Scenario:


Is that possible? I think it only works if I use rename and it checks if file is present etc. and then overwrite. Would it be possible to add support to non-renamed files?


I also got this error:


Config:

Currently the repack overwrite only applies if renaming. Feel free to have a go at adding the feature and sending me a patch.

Not sure why you needed to compile anything manually.

rflores2323 Wrote:anyway you can add unrar support to be able to go into folders and unrar and then move to another folder, and then rename ?

Sure Smile SortTV now can extract within the directory being sorted before the sort occurs.

SumOne Wrote:Cliffe,

First thank you, and the other contributors as well, for the great work done on this script.

That said, I am having an issue with files names in particular format that has me stumped at the moment... it seems like it is parsing the year in the filename as the Season number instead of picking it up as a date. I have tried running sorttv with match-files-named-showname-episodetitle set to both TRUE and FALSE... but that seems to make no difference.

I think the output below provides pretty good detail around what is happening, if you or anyone else can help i would appreciate it. Also, if there is additional information I should supply please let me know.

Thanks!

*** Update: zeroing in on line 188 from the first error message led me to change season-double-digits from TRUE to FALSE, and everything worked fine. That provides me a decent workaround for now, and hopefully it will be useful to anyone else hitting the same issue. If I can figure out a way to patch the code to eliminate the issue I'll gen a patch and link to it from here.

I think this should be fixed now, but let me know if there are still problems.

Enjoy Smile


- vikjon0 - 2011-02-08 19:07

I just tested this and it is working very well. It just took me a few minutes to setup.
Thank you very much!

I need a few tweaks which I could probably hard code into the end of your script but I list it here in case you want to add it.
1) Some series or "mini series"(?) is named part I,III etc. I have e.g. Spartacus.Gods.of.the.Arena.Pt.III.HDTV.XviD.avi. I also remember that "the pacific" was the same. This should be possible to handle I guess?

2) Some stuff is TV but not "episodes" according to this script. Assuming you have one folder for TV and one for Movies you still want to move it to TV.
E.g. The.Daily.Show.2011.02.02.Matthew.Perry.HDTV.XviD-FQM.avi.
EDIT: While the the spillover have to be assumed to be movies.

Perhaps it can be soft coded based on that a file with a full date is normally a daily tv show?
Alternatively I need two "non episodes" folders where the first is for files matching a list of names in the conf file.


- vikjon0 - 2011-02-08 21:07

After re-sorting all my TV files I am even more impressed by the matching logic.

Howver, I seem to have found a small hole, I would'y say it is bugg but it should be possible to cover it.

I see that you can get the name & season from a folder structure /house/season/ and then it is enough if the files have a numer e.g. 2.avi.

However, if they have a stupid name e.g.
House MD - 5 - Mirror Mirror
It goes to "non epi", but if I rename it just "5" it works! Perhaps he is only going for the folder name if he is sure the filename does not indicate something else than the folder? That may make some sense...but if the folder is "season 5" it is pretty safe to assume next level really is the name?