Guest - Testers are needed for the reworked CDateTime core component. See... https://forum.kodi.tv/showthread.php?tid=378981 (September 29) x
  • 1
  • 41
  • 42
  • 43(current)
  • 44
  • 45
  • 197
[RELEASE] Texture Cache Maintenance utility
(2014-01-03, 03:41)MilhouseVH Wrote:
(2014-01-03, 03:15)shadrach47 Wrote: Maybe I'm confused and doing all this for nothing, but technically no art is on my computer.

LOL yes, that could be a problem... Smile

(2014-01-03, 03:15)shadrach47 Wrote: I access it all from a mapped network drive. Could the issue be that it's just pulling the actual location and not the mapped one I set up? I also have a symlink for it in C:\ (was trying it for something else), but it probably would do the same thing.

The computer you are running the mklocal.py script on must to be able to access (read) the artwork, that's what the --local path is for. Just specify a path for --local that allows the computer to access the artwork.

Your C:\ symlink may work, or a UNC, or simply a mapped drive - anything that can access the network location should work.



(2014-01-03, 03:15)shadrach47 Wrote: Maybe I'm not supposed to run mklocal, but I thought you need to if you need to fix wrong cache items
That's one use for it, to correct artwork. Or to convert remote artwork to local (in this case, mklocal.py will download the original artwork and write it into your local media folder, outputting the corrections that need to be applied to your media library). Or to associate non-standard already existing local artwork (clearlogo, clearart, discart etc.) with existing movies or tvshows (although AD will also perform this function, but not if you are using movie name prefixes).

If you just want to simply pre-load the cache on a client you don't need mklocal.py you only need texturecache.py.

mklocal.py works in conjunction with texturecache.py and is really intended to help quickly create or load non-standard artwork (ie. those artwork items not supported by the standard scrapers).

(2014-01-03, 03:15)shadrach47 Wrote: basically instead of re-adding the library to the db.
It's not going to create movies in the library if they don't exist, it's only going to associate artwork with existing movies. In that sense it's just replacing one of many addons that do that kind of job.

Obviously it's up to you if you want to continue down the mklocal.py route, but I should point out that you are trying to process an artwork type called "disc" when it should be called "discart" (which maps to files using the suffix disc). You can specify this as discart:disc (ie. artwork-type:filename-suffix) when specifying artwork, however the version of mklocal.py I just pushed (2 minutes ago) to github will now handle this automatically so just specify "--artwork discart" rather than "--artwork discart:disc".

It was able to work with the symlink not sure why the mapped drive didn't. I was running it to fix wrong artwork in my db and also to add logo and discart to it instead of AD. Thanks for the tip on the discart,I was trying to see what the discart was supposed to be named since disc didnt seem right if logo was clearlogo. Thanks for all your help, awesome utility
Reply
(2014-01-03, 03:50)shadrach47 Wrote: I was trying to see what the discart was supposed to be named since disc didnt seem right if logo was clearlogo.

Normally the artwork type is the same as the filename suffix, but it appears that clearlogo (logo.png) and discart (disc.png) are the two exceptions (and now both are handled automatically by the mklocal.py script).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
This script looks like just what I need. I migrated from a 2TB external drive to a NAS and now want all thumbs belonging to the old externa drive removed to clean up. There were a lot of photo's in there. Just read the first dozen pages but not sure if there is an argument that will delete all files that are gone? On page one there is mention of getting a list of thumbs to delete and then feeding it back to the script but I must have thousands of incorrect thumbs now. In reply two there is mention of a d argument ( "Use the "d" option to remove items from the cache, and then refresh the cache.") but no example is given and I am not really good at this so any help appreciated.
Reply
"P" will delete thumbnail files and corresponding texture cache database rows that are not referenced in your media library
"R" will delete thumbnail files that are no longer referenced in the texture cache database
"Xd" will delete texture cache database rows which reference thumbnail files that no longer exist

I'd run all three - this should leave your cache free of any legacy files and database references:
Code:
./texturecache.py P
./texturecache.py R
./texturecache.py Xd
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks, it is running the first one now! Much appreciated

update: all finished, thanks a lot!
Reply
(2013-03-13, 15:05)MilhouseVH Wrote:
(2013-03-13, 14:50)popcornmix Wrote: (or does this feature exist anywhere else?)

I load all my meta-data from NFO files and locally stored artwork, so I wrote a script that I run on my NAS which parses the NFO files to check for missing plots, and to identify missing artwork.. not aware of any other solution, other than querying the media library database direct.

Is there any chance you could share that? I've got the same sytem here and xbmc keeps loading the empty plot from the nfo file instead of the updated one after the qax operation found results and readds the episode.
Reply
It's mainly just a series of ugly looking greps and find commands, and not something I find necessary to run anymore as there is now the qa/qax option that can spot missing plots and artwork.

Is the qa option not working for you? By default it should be looking for empty plot and mpaa rating on movies, empty plot on tvshows/episodes, missing fanart/poster artwork on movies, fanart/poster/banner/thumb artwork on tvshows & episodes, and missing fanart/thumbnails artwork on artists/albums/songs.

You can configure the QA processing in using the qa.[art|blank|zero].* properties. For instance to verify streamdetails in addition to the default plot on a tvshow episode add:
Code:
qa.blank.tvshows.episode = +streamdetails
Once added, if your streamdetails are blank (empty, missing) on an episode, the episode will "fail" QA (flagged up by qa, and reloaded by qax).

If you only want to be warned whenever a field is missing but not reload the episode, then prefix the field with a question mark (?), eg.
Code:
qa.blank.tvshows.episode = +streamdetails, ?studio
Now the episode will be reloaded if it has no streamdetails, but a missing studio won't cause a reload (though it will be listed during qa).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
It does indeed look and find empty plots, the problem is that XBMC is still using the nfo file with the empty plot after the episode is readded.

What I did was:

Code:
./texturecache.py qax tvshows "hell's kitchen"

Output:

Code:
Episode [Hell's Kitchen, Staffel 11 Episode 11x20          ]: Missing plot
Episode [Hell's Kitchen, Staffel 11 Episode 11x21          ]: Missing plot
Episode [Hell's Kitchen, Staffel 11 Episode 11x22          ]: Missing plot
Season [Hell's Kitchen, Staffel 11                        ]: Missing poster
Rescanning directory: /media/Archiv/TV/Hells Kitchen/Season 11...
Updating Library: New episodeid   705 [Hell's Kitchen S11E20 (4 Chefs Compete)]
Updating Library: New episodeid   706 [Hell's Kitchen S11E21 (2 Chefs Compete)]
Updating Library: New episodeid   707 [Hell's Kitchen S11E22 (Winner Chosen)]

Results in plot still missing, if I manually delete the S11E20 episode nfo it correctly updates it. XBMC always prefers nfo before rescraping from the internet, I guess an option to delete "faulty" nfo's would be needed if that is even possible?
Reply
Hmm, and you're sure the NFO is correctly formatted? In this case there's not much more the script can do, it's identified the episode with the missing plot, removed it and even managed to have XBMC rescrape it (which can be a challenge in itself, what with the fast hashes and all) so if re-scraping the episode XBMC still has no plot then I'd have to suspect the NFO being scraped isn't quite correct. Maybe upload the S11E20 NFO to pastebin?
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I've deleted the E20 episode to check if it's actually updating without nfo, so here is E21 which is still empty:

http://pastebin.com/BzEx5vRy (it is created by sickbeard)
Reply
But there is no plot in your NFO!

You need to delete the NFO(s) and then in Sickbeard rescan (or force full update - it's one of the two, can't remember which) the tvshow in order to regenerate the NFOs this time with plot details. Once you are sure your NFOs all have plots you can run qax and update XBMC.

XBMC will take all of it's information from an NFO (if present) or it will take its information from the internet. It won't, however, take some information from an NFO (the non-blank fields) and then get the rest of the information from the internet. So if you provide an NFO with a blank or missing plot, that is what XBMC will load - an empty plot.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Yes, thats what I'm saying, I wanted to find an easy way to force XBMC to update the empty plot, I thought that was your goal with the qax command, but I guess I got that wrong. Big Grin

edit: I mean to easily -> search for empty plots -> check if nfo file exists -> delete -> reload xbmc episode
Reply
qax will simply identify and remove those media items that fail various quality checks, and then have XBMC reload the items, this time with hopefully more complete information (now that artwork is present either locally or on a remote server, or NFO files/metadata sources have more complete information).

If XBMC is re-reading an incomplete NFO then nothing will change - you need to update the NFO first with the missing information before having XBMC re-scrape it.

In this case Sickbeard is the source of the problem as it sometimes fails to download the plot. Grep'ing your NFOs to identify those that have empty plot elements isn't difficult, but automating Sickbeard to rescan a tvshow so that the NFO can be regenerated with a plot is trickier.

If you want to delete your plot-less NFOs as a solution for missing plots (and become dependent on web sites again, which is the whole point of using NFOs) that's your choice, you can use qa to identify the episodes with metadata problems, manually delete the NFOs then run qax to rescrape the episodes sourcing metadata from the internet but texturecache.py has no direct access to the underlying media files so having it delete the NFO files is impossible (and not IMHO the right solution anyway).

What I do is run qa to identify the episodes with missing plots, manually delete the NFOs, rescan/update the tvshow(s) in Sickbeard to regenerate the NFOs (hopefully with plot), then run qax.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks for the explanation, I will take a look at auto deleting empty nfo files when I got some time to spare. I usually export the xbmc database about once a week so the nfo files should be regenerated once that happens.

edit:

Code:
find /media/Archiv/TV/ -type f -name "*.nfo" -exec grep -r -l '    <plot></plot>' '{}' \; -delete

seems to work well to find nfo files with empty plot, is it possible to run texturecache searching for missing plots only? I've had a look at the readme but excluding certain metadata doesn't seem to be covered
Reply
(2014-01-05, 18:44)MilhouseVH Wrote: I'd run all three - this should leave your cache free of any legacy files and database references:
Code:
./texturecache.py P
./texturecache.py R
./texturecache.py Xd

Thanks for this great utility. My userdata folder had grown to 8GB and my partition was full. I'm running the first command (./texturecache.py P) right now and I see lines like below. I assume it means this path does not exist in my library but I've checked that this file exists.

012102|6/6c2226db.jpg|0240|0360|0004|2013-12-22 17:23:44|2013-12-22 22:51:53|image://%2fmnt<some path>%2fDSC_0091.JPG/transform?size=thumb

I see thousands of such lines (it's running for more than an hour now, I've more than 15000 pictures). Is the utility doing the right thing?
Reply
  • 1
  • 41
  • 42
  • 43(current)
  • 44
  • 45
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17
This forum uses Lukasz Tkacz MyBB addons.