• 1
  • 132
  • 133
  • 134(current)
  • 135
  • 136
  • 197
[RELEASE] Texture Cache Maintenance utility
(2016-08-04, 19:36)Milhouse Wrote: Yes, it needs the "--artwork" parameter, and it's in the original post I linked.

After reading and searching so much, this eluded me Big Grin. Thanks again for your patience, all is good now, and the library is in better shape than ever. Cheers
Reply
Today one of my Harddrives crashed and now i want to obtain a list of Movie and TV Shows that are missing from that crashed HDD but the Drive is Dead :-(

Is this possible with this script?
ASRock Beebox J3160 4GB RAM 120GB SATA SSD - Harmony Elite BT
Intel NUC Kit DN2820FYKH 4GB RAM 120GB SATA SSD - Harmony Smart Control BT
all @ Libreelec Testbuild
HP N54L @ Ubuntu 14.04.4 Minimal Server / MySQL DB
HP N40L @ Ubuntu 14.04.4 Minimal Server
Reply
(2016-08-04, 20:41)john.cord Wrote: Today one of my Harddrives crashed and now i want to obtain a list of Movie and TV Shows that are missing from that crashed HDD but the Drive is Dead :-(

Is this possible with this script?

Code:
texturecache.py query movies "file startswith nfs://192.168.0.3/mnt/share/media/Video"
texturecache.py query tvshows "file startswith nfs://192.168.0.3/mnt/share/media/Video"

Replace "nfs://192.168.0.3/mnt/share/media/Video" with whatever the path is/was for your now deceased spinning rust - you'll get a list of movies and episodes.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hello,

after having moved from my old HTPC to an RPi3 with osmc, I started to try out your tool, because my Textures database and the thumbnails folder have become bigger and bigger.
I have executed your script with the parameter p, which from my understanding should have deleted all outdated entries.

Now when I search for one movie, this is what I get:

Code:
osmc@osmc:~$ ./texturecache.py s "Avatar"
016731|9/99b66ed9.jpg|0273|0185|0001|2015-09-17 19:38:27|2015-09-17 21:38:27|smb://white/filme/Avatar (2009)/Avatar.Aufbruch.nach.Pandora.2009.tbn
016739|6/6e7890bd.jpg|0439|0780|0001|2015-09-17 19:38:34|2015-09-17 21:38:34|smb://white/filme/Avatar (2009)/Avatar.Aufbruch.nach.Pandora.2009-fanart.jpg
017229|4/4d84d582.jpg|0439|0780|0005|2015-12-20 21:30:26|2015-09-19 21:06:59|D:\xbmc_videodb_2015-09-19\movies\Avatar_-_Aufbruch_nach_Pandora_2009-fanart.jpg
017230|5/5a185286.jpg|0273|0185|0007|2015-12-20 21:30:50|2015-09-19 21:06:59|D:\xbmc_videodb_2015-09-19\movies\Avatar_-_Aufbruch_nach_Pandora_2009-poster.jpg
024134|9/99aac533.jpg|0439|0780|0001|2016-01-31 14:31:33|2016-01-31 15:31:33|C:\xbmc_videodb_2016-01-31\movies\Avatar_-_Aufbruch_nach_Pandora_2009-fanart.jpg
024135|8/8e364237.jpg|0273|0185|0001|2016-01-31 14:31:33|2016-01-31 15:31:33|C:\xbmc_videodb_2016-01-31\movies\Avatar_-_Aufbruch_nach_Pandora_2009-poster.jpg
026659|0/0417fdf0.jpg|0273|0185|0003|2016-05-29 05:17:04|2016-05-29 07:16:58|smb://syno/filme/Avatar (2009)/Avatar.Aufbruch.nach.Pandora.2009.tbn
026766|3/36c3308d.jpg|0439|0780|0004|2016-06-03 18:44:08|2016-05-29 07:44:34|C:\xbmc_videodb_2016-05-14\movies\Avatar_-_Aufbruch_nach_Pandora_2009-fanart.jpg
026767|2/215fb789.jpg|0273|0185|0007|2016-06-03 18:44:36|2016-05-29 07:44:34|C:\xbmc_videodb_2016-05-14\movies\Avatar_-_Aufbruch_nach_Pandora_2009-poster.jpg
028957|e/ecb67d09.jpg|0439|0780|0002|2016-05-29 05:17:04|2016-05-29 07:16:58|smb://syno/filme/Avatar (2009)/Avatar.Aufbruch.nach.Pandora.2009-fanart.jpg
031184|d/d74d29b8.jpg|0720|0480|0004|2016-05-28 22:20:53|2016-05-28 20:59:09|smb://syno/filme/Avatar (2009)/poster.jpg
031302|c/c0d1aebc.jpg|1080|1920|0001|2016-05-28 18:59:55|2016-05-28 20:59:55|smb://syno/filme/Avatar (2009)/fanart.jpg
032513|5/5c8a83ca.jpg|0439|0780|0001|2016-08-03 18:08:34|2016-08-03 20:08:34|special://userdata/addon_data/script.artworkorganizer/MovieFanart/Avatar - Aufbruch nach Pandora (2009).jpg
Matching row ids: 16731 16739 17229 17230 24134 24135 26659 26766 26767 28957 31184 31302 32513

smb://syno/filme is the current path where the movie is located, all the other pathes that are listed here (C, D and network share white) are outdated, not even available anymore.
Is there a simple way to ged rid of all this data from entries that are not existing anymore?

And another question: At a former time I once had an issue with duplicated entries after moving files to another share, the reason was that Kodi sources are case-sensitive. I corrected it, but now your script finds two entries for every tv show file, one with smb://syno/ and one with smb://SYNO/.
The search functionality of your script does not seem to be case sensitive, so is there a way to identify the entries with capital letters and delete them?

And a third question: If I delete entries with the option d, the thumbnails get deleted and as it seems also the database entries, but the database file Textures13.db still has exactly the same size, shouldn't it become smaller?
Reply
"p" (lower case) doesn't delete, it's a dry-run option and shows you what would be deleted. If you really want to delete the files, use "P" (upper case). Yes, the command line is case sensitive.

Cleaning you library ("vclean") will remove entries (movies, tvshows) from the media library that are no longer accessible (eg smb://SYNO). Once the entries are removed from the media library you can then prune the cache ("P") of any associated artwork - pruning is case sensitive. The search option is case insensitive as that seemed to be the most useful way to implement this function. You could run the search output through grep to filter your SYNO rows, then delete the remaining rows, but I suspect after using the correct prune option (ie. "P") this won't be necessary.

Regarding Textures13.db becoming smaller after deleting data, it's generally not worth it - the free space will be re-used as new rows are inserted. You can try vacuuming your database but it might cause problems.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks, from the wiki I did not understand that the difference between p and P is simulation.
I also thought that vclean refers to the library like cleaning up the library in Kodi.

So, I ran vclean followed by P. That deleted a lot of outdated entries, also the ones with the capital SYNO letters.

However in total it did not really seem to work the way it should. When looking at the avatar example, after cleaning up as described, this is what is left:
Code:
osmc@osmc:~$ ./texturecache.py s "Avatar"
026766|3/36c3308d.jpg|0439|0780|0004|2016-06-03 18:44:08|2016-05-29 07:44:34|C:\xbmc_videodb_2016-05-14\movies\Avatar_-_Aufbruch_nach_Pandora_2009-fanart.jpg
026767|2/215fb789.jpg|0273|0185|0009|2016-08-05 15:00:39|2016-05-29 07:44:34|C:\xbmc_videodb_2016-05-14\movies\Avatar_-_Aufbruch_nach_Pandora_2009-poster.jpg
Matching row ids: 26766 26767

BUT: A drive C does not exist for the RPi (similar to the drive D: entry which it did delete). On the other side, the pictures from the path where it actually IS located (smb://syno/filme/) were obviously deleted, resulting in a missing cover in Kodi :-(

Any idea what happened?
Reply
I suspect what has happened there is that the movies are still in the media library, ie. they haven't been cleaned. Try:
Code:
./texturecache.py jd movies | grep "C:\\\\"
and if anything is listed then the movies on the C:\ drive are still present.

Kodi will only clean movies from the library if the drive is connected. The directory may not be accessible, as in the case of SYNO, but the drive itself must be connected and available in order for the clean process to remove the movies - this avoids cleaning movies that may be on a removable drive, and which shouldn't be clean (unless the remove drive is connected, etc.).

In the case of moves on "C:\" it's probable that Kodi sees the "C:\" drive as being "not connected" so all your "C:\" movies remain in the library.

I don't have a simple way to remove these movies, but you could at least list them with the following:
Code:
./texturecache.py query movies "file startswith C:\\\\"

and then remove them from your library using the GUI.

Another option would be:
Code:
./texturecache.py jd movies >/tmp/movies.dat

then search through /tmp/movies.dat for "C:\" and, using the matching "movieid" for each movie, ie.55, run:
Code:
./texturecache.py remove movie 55
and this will remove that movie from your library.

Then you can run the prune ("P") option again, and it should remove more redundant artwork from the cache.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks for the explanation, indeed the first command shows a lot of hits, which should explain why those movies are still there. I was not aware of this behaviour for library cleaning.
I think I could also get rid of those by finding out their IDs, searching for "C:\xbmc".
These are leftovers after exporting the library, changing the path and importing it again. So the movies are there, just besides the correct thumbs they also have those outdated ones.

But this does not explain why it also deleted the correct entries. The movie Avatar is in the library, and when I look at the info screen, it also shows the path smb://syno/filme/Avatar (2009). I can actually also play it from there. But somehow the vclean and P deleted those entries, so the movie is still in the library, but its pictures got deleted.

P.S. The remove command is also not in the wiki, I guess it is not updated on the latest version?
Reply
(2016-08-05, 20:33)pigeldi Wrote: But this does not explain why it also deleted the correct entries. The movie Avatar is in the library, and when I look at the info screen, it also shows the path smb://syno/filme/Avatar (2009). I can actually also play it from there. But somehow the vclean and P deleted those entries, so the movie is still in the library, but its pictures got deleted.

vclean is just running the Kodi clean library function, what it deletes is down to Kodi.

As for the prune command, I can't say why it would delete an artwork item without knowing what is in (or was in) your library, and what has been removed from the cache.

Many times I've looked at the output of the prune ("P") command and wondered why it would delete a particular artwork from the cache as I'm sure it should be associated with a movie in the library, only to discover that the prune function is correct and the movie is associated with an alternative artwork item.

Dump your movies with "texturecache.py jd movies >/tmp/movies.dat" and see if any of the pruned artwork is in the movies.dat file - I'm pretty sure you won't find them, which is why they were pruned. If I'm wrong, please upload movies.dat somewhere along with a list of the removed artwork.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2016-08-05, 20:33)pigeldi Wrote: P.S. The remove command is also not in the wiki, I guess it is not updated on the latest version?

Wiki is probably out of date. The first post in this thread is likely to be more up to date. The output of "texturecache.py usage" will tell you exactly what is supported by the script.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
After some more testing indeed it seems that something is corrupt with my Kodi database. What it displayed in the GUI is not what it actually used from the database and thumbnails.
With the help of this thread I have deleted all thumbnails, textures and movies db and scan everything from scratch. Great that our tool can also export and import the watched status, for tv shows it already worked :-)
http://forum.kodi.tv/showthread.php?tid=...pid1860134

Thanks, great work!
Reply
Hello Smile

This tool looks pretty awesome, and it does a multitude of things.. So many things that it's a bit overwhelming to look at without any other knowledge..

Some of my actor thumbnails are VERY low resolution files (like 10x10 pixel).. Is there an easy way with this tool to delete or re-scrape just the low res actor thumbs?

Thanks!
Reply
(2016-08-09, 19:24)jbeige Wrote: Hello Smile

This tool looks pretty awesome, and it does a multitude of things.. So many things that it's a bit overwhelming to look at without any other knowledge..

Some of my actor thumbnails are VERY low resolution files (like 10x10 pixel).. Is there an easy way with this tool to delete or re-scrape just the low res actor thumbs?

Thanks!

Possibly not.

The dimensions of the cast thumbnails are often encoded in the thumbnail urls of remote artwork, so if whatever created your cast thumbs uses 10x10 dimensions that's what you're stuck with. It's also possible to have local cast thumbs, in which case if they're only 10x10 then there's nothing to do but replace them all with better artwork.

You can view the cast urls as follows:
Code:
texturecache.py Jd movies avatar @extrajson.movies=cast

Example:
Code:
"cast": [
      {
        "name": "Sam Worthington",
        "order": 0,
        "role": "Jake Sully",
        "thumbnail": "image://http://ia.media-imdb.com/images/M/MV5BMTc5NTMyMjIwMV5BMl5BanBnXkFtZTcwNTMyNjYwMw@@._V1._SX275_SY400_.jpg/"
      },
      {
        "name": "Zoe Saldana",
        "order": 1,
        "role": "Neytiri (as Zo\u00eb Saldana)",
        "thumbnail": "image://http://ia.media-imdb.com/images/M/MV5BMjA4NDk1NTA1OV5BMl5BanBnXkFtZTcwMTIzMjQ4Ng@@._V1._SX275_SY400_.jpg/"
      },
      {
        "name": "Sigourney Weaver",
        "order": 2,
        "role": "Grace",
        "thumbnail": "image://http://ia.media-imdb.com/images/M/MV5BMTk1MTcyNTE3OV5BMl5BanBnXkFtZTcwMTA0MTMyMw@@._V1._SX275_SY400_.jpg/"
      },

You can see the dimensions SX275_SY400 at the end of each remote url.

Assuming however that your cast url dimensions are greater than 10x10, then forcibly recaching artwork might work (though I'm not sure why you only have 10x10 cached artwork in the first place - check your advancedsettings.xml for fanartres and imageres settings).

The following will re-cache only cast thumbs for the movie Avatar:
Code:
texturecache.py C movies avatar @cache.castthumb=yes @cache.artwork=cast.thumb

Note however that if any remote urls are no longer available then re-caching may mean you "lose" that particular artwork from your cache (this isn't really specific to cast artwork, it's just a general problem when using remote artwork that comes and goes).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I'm trying to use texturecache.py on a ATV4 but so far I've been unsuccessful.

If I've understood it correct it should be possible to enable the webserver on the ATV4, and then run texturecache.py from a computer with the texturecache.cfg host pointing to the ATV4.

Should this be possible or have misunderstood how it works? If it should be possible, anyone know how I should setup everything?

When I run the script texturecache.py finds the webserver, but I get an error when it tries to connect to RPC.
Reply
@manhas yes this is possible. To run the script remotely you need to allow programs on other systems to control Kodi, so enable this setting in Kodi. You may need to restart your ATV4 after enabling the setting for the change to become effective.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
  • 1
  • 132
  • 133
  • 134(current)
  • 135
  • 136
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17