Store Thumbnails with Media - SQL
#1
Hi

I am in the middle of trying to update my XBMC version which I needed help with (http://forum.xbmc.org/showthread.php?tid...pid1708080) and I ran into a small issue.

I don't seem to have enough space, when I started using XBMC it was still very small and SSD Drives were expensive so I only got a 16GB SSD Drive. Now it appears its full, or close to it. The easy solution would be to upgrade I am sure, but I am not in a position to buy anything that isn't necessary at the moment, so I have been doing some cleaning up. I have removed unwanted plugins, cleared cache, all that, and I have just enough space, but I think it would be wise to make more room, so here is where I need a little info.

I have a SQL server set up that has all of my media info stored, on the same server is all the media. I was under the impression that all of the thumbnails etc were stored on the server along with the media, but I noticed the Thumbnails folder in .xbmc and its about 5gb. I'm not sure if this is just a cache, but when I looked at more recent shows on the server, I noticed their thumbnails were not stores along with them. So assuming that its storing the thumbnails on the XBMC PC, is there a way to have it stored on the server along with everything else? I am sure it use todo this because a lot of older shows and movies have folder.jpg etc stored with the media. I did a few searched trying to find info and I did find a wiki which had some vague info but it seemed to suggest it was pre frodo only.

I have Frodo 12.2 (Trying to upgrade to 12.3), if it helps.

EDIT: This is the wiki I found that looks like it is what I need todo. But I am uncertain weather I just add that bit of code adn point it to my server PC or what.

Thank You
Kirt
Reply
#2
Try the script in my sig to "prune" your Thumbnails folder of unnecessary images which will free up some space on your SSD (hard to say how much but could be a GB or more).

If you're really unable to run XBMC comfortably on a 16GB SSD (what else have you got on there taking all the space?) then try using path substitution to relocate your Thumbnails folder to a network share (just don't try sharing the Thumbnails with another XBMC client).

Finally, consider using OpenELEC booted from a small USB drive (256MB will do, or use your SSD). With the entire 16GB SSD dedicated to XBMC storage you'd be very unlikely to fill it up unless you're also storing media files on it.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#3
Hi, I downloaded your script and tried to run one of the commands, but I got a fetal error. Something todo with possible permissions with the textures.db. Just wondering which command/s I should be running and how I might go about allowing access to that file for your script?

I should also point out I checked and the .xbmc folder is about 6.5gb of the 11gb space available on the drive, then the usr and lib files take up another 2 or so gig, so that there is about 1.4gb remaining. I am not sure why the drive size is only 11 and not closer to 16 but I assume there is some overhead and hidden partitions for the OS?
Reply
#4
Run the script as the same user you use to run xbmc. Paste the screen output if you can't fix it, it's hard to work with "something todo with ...".

As for the 11gb issue, what's the output of "df -hT" and also "sudo parted -l"?

5GB is a lot of overhead, it sounds like your drive isn't partitioned correctly or has additional partitions taking up the remaining space.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#5
Hi, very sorry for the late reply, I have been under a lot of load at work at the moment. Sorry for my vagueness, I was at work at the time and couldn't remember the exact error.

For df -hT:
Code:
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/sda1     ext4     11G  8.7G  1.6G  85% /
udev      devtmpfs    2.0G  4.0K  2.0G   1% /dev
tmpfs        tmpfs    806M  416K  805M   1% /run
none         tmpfs    5.0M     0  5.0M   0% /run/lock
none         tmpfs    2.0G     0  2.0G   0% /run/shm
//192.168.0.60/Recording
cifs    1.9T  1.7T  126G  94% /home/citris/mnt/record

For sudo parted -l:
Code:
Model: ATA KINGSTON SS100S2 (scsi)
Disk /dev/sda: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type      File system     Flags
1      1049kB  11.7GB  11.7GB  primary   ext4            boot
2      11.7GB  16.0GB  4294MB  extended
5      11.7GB  16.0GB  4294MB  logical   linux-swap(v1)

Do not ask me what extended is haha, I just set this up using the xbmcBuntu disc, I am very green when it comes to linux.

And as for the error when trying to run your script, I get the following:
Code:
FATAL: The task you wish to perform requires read/write file
       access to the XBMC sqlite3 Texture Cache database.

       The following sqlite3 database could not be opened:
       /root/.xbmc/userdata/Database/Textures13.db

       Check settings in properties file texturecache.cfg,
       or upgrade your XBMC client to use a more recent
       version that supports Textures JSON API.

I only have one user on the MC (not including root I am sure). which is citris, when I log into SSH I use that as my login, and then I use su and login, I have tried using Sudo (Which I am sure is not needed since I am already logged in using su) and that hasn't helped. I am not really sure how I give the script access to read/write to that file.

Thank you
Cirtis
Reply
#6
Disk: You've got two partitions - one (your primary) is 11GB, plus a second (logical/extended) partition for 4GB of swap. So that's where all your space is. Two partitions. You probably don't need the 4GB of swap.

Script: It can't find your Textures13.db file. You're running the script as root, when XBMC is probably run under a normal (non-root) user. Run the script under the same user as you run XBMC and then the script should find Textures13.db.

To see which user is running xbmc, try running "ps aux|grep xbmc" and paste the results. I imagine it will be user "citris". If you login as citris, why are you trying to run everything as root, that's the fastest way to screw things up, particularly file permissions.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#7
(2014-05-29, 04:05)MilhouseVH Wrote: Disk: You've got two partitions - one (your primary) is 11GB, plus a second (logical/extended) partition for 4GB of swap. So that's where all your space is. Two partitions. You probably don't need the 4GB of swap.

Script: It can't find your Textures13.db file. You're running the script as root, when XBMC is probably run under a normal (non-root) user. Run the script under the same user as you run XBMC and then the script should find Textures13.db.

To see which user is running xbmc, try running "ps aux|grep xbmc" and paste the results. I imagine it will be user "citris". If you login as citris, why are you trying to run everything as root, that's the fastest way to screw things up, particularly file permissions.

I'm not 100% on what exactly the swap drive is for, if you believe its not needed, (and I believe it probably isn't) is there a way to delete that partition and extend the main one?

As for running things as root, probably because I am a noob haha. when I login to SSH it just ask for UN and pass and I just give it citris and its Pass, but it shows as root@citris-mediaplayer so I guess I am doing that wrong.

EDIT: heh, just realized what i was doing haha, yeah logged in as citris and its all good for that script.
Reply
#8
(2014-05-29, 04:14)citris.87 Wrote: I'm not 100% on what exactly the swap drive is for, if you believe its not needed, (and I believe it probably isn't) is there a way to delete that partition and extend the main one?

There is, with tools such as gparted, but it's a a little tricky to perform that kind of operation on a running system drive. Try doing some Google searches, but it may also be easier to wipe it and start again this time without a swap partition. If you can boot from another drive, even a LiveCD (or USB equivalent), then you stand a better chance of resizing your SSD.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#9
(2014-05-29, 04:20)MilhouseVH Wrote: There is, with tools such as gparted, but it's a a little tricky to perform that kind of operation on a running system drive. Try doing some Google searches, but it may also be easier to wipe it and start again this time without a swap partition. If you can boot from another drive, even a LiveCD (or USB equivalent), then you stand a better chance of resizing your SSD.

Thank you, you have been a great help, your script appears to be working now (Its just sorta sitting there with a '>' which I assume means its working) so we will see how much space it frees up, I might only need about 200mb - 400mb. I think I might look into a larger SSD in the near future when I upgrade the server I will use its SSD which is 32gb. Fresh install is never a bad thing, its just setting up the remote and wake on sleep etc is such a pain in the ass haha, there is always a different issue and takes a while to work it out,

Thanx Mate
Kirt
Reply
#10
No, that doesn't sound like it's working. I don't recognise ">", maybe if you pasted the rest of the text around the ">" to give me some context I might have some idea what has happened.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#11
(2014-05-29, 04:29)MilhouseVH Wrote: No, that doesn't sound like it's working. I don't recognise ">", maybe if you pasted the rest of the text around the ">" to give me some context I might have some idea what has happened.

Dammit, my optimism was misplaced... haha

Code:
citris@citris-mediaplayer:~$ ./texturecache.py x "where lastusetime <= '2014-01-01'
>

The command I used and what I got back, litterally all I got was >... haha, what did I do wrong this time, I can confirm that this worked
Code:
./texturecache.py x | sort -t"|" -k6

As a test, to which I got:
Code:
009421|4/408088da.png|0256|0256|0939|2014-05-28 10:01:01|2014-05-27 21:20:09|/home/citris/TVShowIcons/41396908.png
009426|4/48249776.png|0256|0256|1312|2014-05-28 10:01:01|2014-05-27 21:20:09|/home/citris/TVShowIcons/7hd.png
009088|8/8687e59e.png|0256|0256|1057|2014-05-28 10:01:01|2014-05-27 21:20:09|/home/citris/TVShowIcons/7mate.png
009419|a/ac1d711c.png|0256|0256|1051|2014-05-28 10:01:01|2014-05-27 21:20:09|/home/citris/TVShowIcons/7two.png
009424|d/dfd37741.png|0256|0256|1269|2014-05-28 10:01:01|2014-05-27 21:20:09|/home/citris/TVShowIcons/elevenm.png
009422|a/a7acf5a5.png|0132|0400|0996|2014-05-28 10:01:01|2014-05-27 21:20:09|/home/citris/TVShowIcons/GEM_TV_logo.png
024029|3/36ae2b88.jpg|0202|0360|0002|2014-05-28 10:01:01|                   |video@smb://MEDIAHUB/Media 1 - Movies/Downloaded/Non-Stop 2014 BRRip XviD AC3 HebSubs-DownRev/Non-Stop.2014.BRRip.XviD.AC3.HebSubs-DownRev.avi

Obviously there was more then this. What I did see in here which is weird, is some items come from "smb://MEDIAHUB/X/Avatar/banner.jpg" which is where I would like all the thumbs, and then some on this SSD... eh
Reply
#12
In the first case, it's because you're not typing the command correctly - you're missing the closing double-quote.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#13
(2014-05-29, 04:38)MilhouseVH Wrote: In the first case, it's because you're not typing the command correctly - you're missing the closing double-quote.

I hate when its simple... also, you might be interested to know its also missing the " in the thread that script was located on, although I guess I should have picked that up lol. Thanks again mate, it printed out a lot more this time, too much to post it went off the screen but ended in a bunch of numbers

Code:
22780 22781 22782 22783 22784 22786 22787 22788 22789 22790 22791 22792 22793 22794 22795 22796 22797 22798 22799 22800 22801 22802 22803 22804 22805 22806 22807 22808 22809 22810 22811 22812 22813 22814 22815 22816 22817 22818 22819 22820 22821 22822 22823 22824 22825 22826 22827 22828 22829 22830 22831 22832 22833 22834 22835 22836 22837

And so on. I'll play around with some of the commands and see what space I can get.
Reply
#14
So it is - thanks!

The command to view the artwork files that can be automatically "pruned" is "./texturecache.py p", and to automatically remove these same files it's "./texturecache.py P".

Note sure why some of your artwork is located on your SSD and the rest on SMB, use "./texturecache.py jd movies" (or jd tvshows) to view your media library and see where XBMC thinks your artwork should be located. However relocating artwork will most likely involve rescraping your library.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#15
(2014-05-29, 04:49)MilhouseVH Wrote: So it is - thanks!

The command to view the artwork files that can be automatically "pruned" is "./texturecache.py p", and to automatically remove these same files it's "./texturecache.py P".

Note sure why some of your artwork is located on your SSD and the rest on SMB, use "./texturecache.py jd movies" (or jd tvshows) to view your media library and see where XBMC thinks your artwork should be located. However relocating artwork will most likely involve rescraping your library.

Yeah so with that command it seems to want to prune all the files on the server, which is where I wanted the files in the first place, ran thoes commands you mentyiones for TV shows and Movies and I get this (last One):
Code:
{
    "art": {
      "fanart": "image://smb://MEDIAHUB/Media 1 - Movies/All Movies/Movies/-fanart.jpg/",
      "poster": "image://smb://MEDIAHUB/Media 1 - Movies/All Movies/Movies/-poster.jpg/",
      "thumb": "image://smb://MEDIAHUB/Media 1 - Movies/All Movies/Movies/-thumb.jpg/"
    },
    "file": "smb://MEDIAHUB/Media 1 - Movies/All Movies/Movies/.avi",
    "label": "",
    "movieid": 1019,
    "title": ""
  },

and for TV shows (Last one)
Code:
{
            "art": {
              "thumb": "image://video@smb://MEDIAHUB/Media 2 - TV - Music/All TV Series/Tosh.0/Season 06/Tosh.0 - S06E15 - Daniel webchats with a goat, and a bowler who got screwed out of a perfect game gets a web redemption.mkv/",
              "tvshow.banner": "image://smb://MEDIAHUB/Media 2 - TV - Music/All TV Series/Tosh.0/banner.jpg/",
              "tvshow.fanart": "image://smb://MEDIAHUB/Media 2 - TV - Music/All TV Series/Tosh.0/fanart.jpg/",
              "tvshow.poster": "image://smb://MEDIAHUB/Media 2 - TV - Music/All TV Series/Tosh.0/poster.jpg/"
            },
            "episodeid": 6050,
            "file": "smb://MEDIAHUB/Media 2 - TV - Music/All TV Series/Tosh.0/Season 06/Tosh.0 - S06E15 - Daniel webchats with a goat, and a bowler who got screwed out of a perfect game gets a web redemption.mkv",
            "label": "6x15. Daniel and the Goat"
          }

That TV show is probably one of the more recent ones, weird thing is, more recent movies seem to have a different address for ffanart and posters which is
Code:
{
    "art": {
      "fanart": "image://http://image.tmdb.org/t/p/original/zkBSu5jNI9YrAGvvy8cAAc7BZX5.jpg/",
      "poster": "image://http://image.tmdb.org/t/p/original/hFQcC5eSSHqJfk70RdgrMMxGuYt.jpg/"
    },
    "file": "smb://MEDIAHUB/Media 1 - Movies/All Movies/Movies/2014.mp4",
    "label": "",
    "movieid": 1065,
    "title": ""
  },

Hmmm thats still not local tho... I wonder weather these files stored on the actual media player are just left over artifacts from before it was stored on the server...

OK so update, and this is interesting, installed ftp and did a quick look at the files in sftp and found that according to it, the thumbnails folder was last modified on 13/02/2012... So, my question, if I was to delete this folder (which if I remember correctly, is 1-2gb) if this is used by xbmc, but for some reason hasn't been modified in so long, worst case it would just re-scrape the images, correct?

Scratch that, closer look shows that some of them are more recent, the thumbnails seem to be unrelated to the posters and fanart, as they are use numbers lke 26ae2b88.jpg, and when looking at them it appears to be a snapshot of the video... any idea what these are, are they needed? I never see these when browsing media...
Reply

Logout Mark Read Team Forum Stats Members Help
Store Thumbnails with Media - SQL0