Clean up the Thumbnails folder

  Thread Rating:
  • 4 Votes - 4.5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
bradvido88 Offline
Donor
Posts: 800
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #1
This tool does not work for XBMC Frodo or greater. Please use the excellent Texture Cache Maintenance Utility


***See 2nd post for my script to clean up unused thumbnails and fanart***

My Thumbnails folder is growing larger and larger. I'm guessing a lot of the images in there are no longer used, as i have a pretty high turn-around rate for my library content.

Is there any standard way to determine which images in the userdata/Thumbnails folder are no longer used? Or some process to clean them out?

If there isn't a standard way, I will attempt to do it myself, but i need some help. I need to determine what all the valid hash's are, so i can determine which ones can be removed. To do this, I need to determine the CRC hash for every item in XBMC's library that can have a thumb/fanart image.

So far I think I know:
  • Movies, episodes, and music videos use the actual path to the file to generate a CRC hash, for example "smb://server/share/videos/movie/Avatar.mkv" = 1c145e0e. Fanart is stored under the fanart subfolder using the same hash.
  • If the video is a multi-part video, the name only (e.g: stack://part1.ext , part2.ext...) is used to generate the CRC (the path string is ignored)
  • Actor thumbs use the "actor" + [actor name] to generate the CRC, for example Jane Lynch = "actorjayne lynch" = 9f309464
  • Album thumbs are (usually) [album]+[artist] and are also stored in the database under the strThumb field.
  • Album/song thumbs may be taken from a folder.jpg image, and may be generated from the folder path or the song path. More info here on album/song thumbs.
  • Artist fanart is generated using the literal artist name, for example "Metallica" = b18ea041.
  • Artist thumbs are generates using "artist"+[artist name], for example Metallica = "artistmetallica" = 0a2d8dbc.
  • Video Bookmark thumbs store their path in the thumbnailImage field in the database. I'm not sure how it is originally generated
  • TV Show seasons use "season" + [path_to_show] + [season_label]. See here for details
  • TV Show banners/fanart use the literal path to the show, for example "smb://server/TV Shows/Lost/". Fanart is stored under the fanart subfolder using the same hash.
I need to figure out:
  • Are there any other image types I'm missing that get stored in the Thumbnails folder?
I plan on deleting all auto-xxxxxxxx.tbn images since they can be recreated automatically if XBMC needs them.

Thanks!

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
(This post was last modified: 2013-10-25 16:37 by bradvido88.)
find quote
bradvido88 Offline
Donor
Posts: 800
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #2
Well, I ended up writing my own script that cleans up the thumbnails directory.

I had 31,563 thumbs/fanart before cleaning up.
After cleanup, I have 12,967....Much nicer
I've been exploring XBMC for a while now and have not found any thumbs missing Smile

-------------------------------------------------------------------------
Here is my script. You can use it to clean up your own thumbnail directory:
Download XBMC Thumbnail Cleaner - v1.1 (Dharma Compatible)
Download XBMC Thumbnail Cleaner - v1.2 (Eden Compatible)
***Currently no Frodo compatible version is available or planned***



Details:
  • This script will delete unused thumbnails, fanart and textures in the userdata\Thumbnails directory.
  • It can be run in simulation mode if you'd like to test it.
  • It uses data from XBMC's video/music/texture databases, as well as the JSON-RPC interface to get a list of all thumbs that are still used by the libraries. Any thumbs that are no longer used are deleted.
  • Any auto-xxxxxx thumbnails will be deleted since XBMC will automatically re-create them if they are needed.
  • Written in Java. Tested in a Windows environment.
Disclaimer:
If you are not running in simulation mode, I recommend backing up your Thumbnails directory, in case something goes wrong, you can easily revert.
This has only been tested when XBMC's language is set to English.

-------------------------------------------------------------------------
If you have problems, please paste your log and ThumbCleanerConfig.xml to pastebin.com and reply here with a link to them.
Here is what a normal log would look like for reference: http://pastebin.com/raw.php?i=5cd6WEYf (v1.0 log)

***Note:
If you are not running this on a Windows environment, you must specify the path to this program's directory as a command line parameter when starting.

Cheers,
BradVido88

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
(This post was last modified: 2013-05-01 15:21 by bradvido88.)
find quote
ppic Offline
Skilled Python Coder
Posts: 2,711
Joined: Feb 2009
Reputation: 11
Location: France idf
Post: #3
you can use the fileitem.py in my logo downloader script to get hash and path Wink

[Image: widget]
Passion-XBMC Repository Download your SVN skins and addons
TvTunes Addon Download Play your theme while browsing library
TVshow Next Aired display next aired infos from tvrage.com
Logo Downloader Script download Logo/clearart/show thumbs/poster/banner
Bande-Annonce Allociné plugin watch trailers in french and vo
SportLive Script Live score match info in XBMC (not compatible dharma)
find quote
bradvido88 Offline
Donor
Posts: 800
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #4
ppic Wrote:you can use the fileitem.py in my logo downloader script to get hash and path Wink
Thanks. I'm not developing in python, but could still be useful for cross-checking.

Edit: for anyone who need to quickly check XBMC CRC hashs, you can use this on windows: http://forum.xbmc.org/showthread.php?tid=116861

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
(This post was last modified: 2012-03-30 22:51 by bradvido88.)
find quote
tboooe Offline
Fan
Posts: 456
Joined: Dec 2010
Reputation: 0
Post: #5
bradvido88 Wrote:Well, I wrote my own script that cleans up the thumbnails/fanart.

I had 31,202 thumbs/fanart before cleaning up.
After cleanup, I have 11,936....Much nicer Smile
I've been exploring XBMC for a while now and have not found any thumbs missing, so I must have properly determined which CRC hash's are still valid and which can be purged...

If there is much interest in the script, i'll post it.

I am definitely interested in your script! I have the same problem as you and my thumbnail directory is getting way too big (over 3gb). Any help would be greatly appreciated.
find quote
paco Offline
Senior Member
Posts: 238
Joined: Mar 2008
Reputation: 0
Location: Toronto
Post: #6
Add me to the people looking for a solution to this problem. My Thumbnails folder has grown exponentially over the years. I have upgraded my NAS a few times and I've also renamed a few items. It's too bad XBMC can't do this on its own.
find quote
queeup Offline
Posting Freak
Posts: 765
Joined: Feb 2009
Reputation: 16
Post: #7
Yes, come on guys send your Thumb clean codes. I want to use it Smile

@ppic come on write one for us with python also Big Grin
find quote
PatK Offline
Donor
Posts: 3,537
Joined: Mar 2010
Reputation: 79
Location: Canada
Star   
Post: #8
Ditto.... speed and space are always on the forefront when I look deep. Trouble is I get lost, end up wandering into plugins.
find quote
kim1406 Offline
Senior Member
Posts: 171
Joined: Jun 2009
Reputation: 0
Post: #9
looking for the script too. thumbnail directory here is over 8gb.
find quote
bradvido88 Offline
Donor
Posts: 800
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #10
OK, I'll clean out the hard-coding and post it when i have a chance.

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
find quote
ppic Offline
Skilled Python Coder
Posts: 2,711
Joined: Feb 2009
Reputation: 11
Location: France idf
Post: #11
queeup Wrote:Yes, come on guys send your Thumb clean codes. I want to use it Smile

@ppic come on write one for us with python also Big Grin

you're better in python than me here Wink

[Image: widget]
Passion-XBMC Repository Download your SVN skins and addons
TvTunes Addon Download Play your theme while browsing library
TVshow Next Aired display next aired infos from tvrage.com
Logo Downloader Script download Logo/clearart/show thumbs/poster/banner
Bande-Annonce Allociné plugin watch trailers in french and vo
SportLive Script Live score match info in XBMC (not compatible dharma)
find quote
queeup Offline
Posting Freak
Posts: 765
Joined: Feb 2009
Reputation: 16
Post: #12
No waaay, i think you are more lazy then me Wink
(This post was last modified: 2011-03-08 16:15 by queeup.)
find quote
ppic Offline
Skilled Python Coder
Posts: 2,711
Joined: Feb 2009
Reputation: 11
Location: France idf
Post: #13
the simplest way i see here:
make a list of used thumb.
compare for each thumbs found in thumbnail folder if present in the list, if not, then delete it.
this means it have to list all used thumb ( addon icon for example)
at least, except library content thumb, all other used thumb will be regenerated by xbmc.

[Image: widget]
Passion-XBMC Repository Download your SVN skins and addons
TvTunes Addon Download Play your theme while browsing library
TVshow Next Aired display next aired infos from tvrage.com
Logo Downloader Script download Logo/clearart/show thumbs/poster/banner
Bande-Annonce Allociné plugin watch trailers in french and vo
SportLive Script Live score match info in XBMC (not compatible dharma)
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,221
Joined: Oct 2003
Reputation: 178
Post: #14
As soon as video and music thumbs are transferred to the texture cache I'll work on adding the cleanup side of things.

Others are welcome to work on the cleanup side of things if you like - basically just open the Textures.db file and take a look at what we have - we have last used and use count available, so we can easily purge at various points (and ofcourse they'll be automatically re-cached if used again Smile

Good strategies for purging need to be dreamt up - I'm sure you guys can take care of that. Obviously these apply only to addons, pictures etc. at this point (i.e. not music or video thumbs).

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
ppic Offline
Skilled Python Coder
Posts: 2,711
Joined: Feb 2009
Reputation: 11
Location: France idf
Post: #15
jmarshall Wrote:As soon as video and music thumbs are transferred to the texture cache I'll work on adding the cleanup side of things.

Others are welcome to work on the cleanup side of things if you like - basically just open the Textures.db file and take a look at what we have - we have last used and use count available, so we can easily purge at various points (and ofcourse they'll be automatically re-cached if used again Smile

Good strategies for purging need to be dreamt up - I'm sure you guys can take care of that. Obviously these apply only to addons, pictures etc. at this point (i.e. not music or video thumbs).

Cheers,
Jonathan

old habits, i forgot about texture.db !

[Image: widget]
Passion-XBMC Repository Download your SVN skins and addons
TvTunes Addon Download Play your theme while browsing library
TVshow Next Aired display next aired infos from tvrage.com
Logo Downloader Script download Logo/clearart/show thumbs/poster/banner
Bande-Annonce Allociné plugin watch trailers in french and vo
SportLive Script Live score match info in XBMC (not compatible dharma)
find quote
Post Reply