[theowiesengrund]

HI Milhouse,

it might not be the right place to ask, but perhaps you know the answer.
Sometimes i use the "vclean" option in your script, but when cleaning/deleting a movie, XBMC prompts me (i'm using your latest Openelec test builds) if i really want to delete this file from the library. How can i avoid that dialog?

[un1versal]

Disable allow rename and delete in XBMC.

[theowiesengrund]

If you mean "Allow file renaming and deletion" in Settings - Appearance: It is already disabled.

[Milhouse]

HI Milhouse,

it might not be the right place to ask, but perhaps you know the answer.
Sometimes i use the "vclean" option in your script, but when cleaning/deleting a movie, XBMC prompts me (i'm using your latest Openelec test builds) if i really want to delete this file from the library. How can i avoid that dialog?

I don't think there's anything that can be done without changes to XBMC - Montellese indicated he would try to get something into Gotham but I guess he didn't have the time. Unfortunately the GUI prompts tend to make the JSON clean library method pretty much useless.

I've given Montellese a nudge.

[un1versal]

All xbmc settings are already exposed to json, isnt that all you need?

* un1versal says with a lot of uncertainty

[Milhouse]

I don't think there is a setting to prevent the GUI prompt during the video library clean. Besides it would be perfectly reasonable to expect the prompt during a GUI initiated clean, but when the clean is initiated remotely with JSON there should be no prompts whatsoever. So its not really a settings issue as far as I can tell.

[un1versal]

Hes probably got some advancedsettings.xml causing this.

I have used texturecache with vclean many hundreds of times and never seen any such prompts.

[Milhouse]

I've seen it (hence my post in the JSON thread that I linked to previously) and I don't have any as.xml settings that would cause this. It was something that slipped into Gotham a while ago.

[theowiesengrund]

Just for the record: I don't use any advancedsettings.xml file.

[radigast]

I have a very specific issue for a very unique situation.

Background:
I have all movies on external drives that get plugged-in when a movie on that drive is watched, and then unplugged the rest of the time. I scan-in/cache all artwork using texturecache.py. I also have the appropriate switches in texturecache.cfg to enable caching of extrafanart and extrathumbs.

Problem:
Weirdly enough, after scanning and caching all artwork using texturecache.py and then unplugging the drive, both extrafanart and extrathumbs are not shown in XBMC. However, when the drive is plugged back in, both the extrafanart and extrathumbs are shown and (improperly) cached.

Workflow:
1. Check texturecache.cfg settings (LINK) and folder structure (LINK).
Also, the movie database and thumbs cache is entirely empty except for the movie run in this test. Everything is fresh.

2. Plug hard drive in / Scan video into library

Code:

texturecache.py vscan movies "G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]"

vscan LOG

3. Adding Artwork

Code:

texturecache.py jd movies | tools\mklocal.py --singlefolders --local "G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]" --prefix "G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]" --artwork poster fanart clearlogo clearart discart characterart:character banner landscape --verbose --output | texturecache.py set

mklocal.py LOG

4. Caching Artwork

Code:

texturecache.py c

cache LOG

5. Unplug External Drive / Check Artwork

Code:

texturecache.py jd movies "Bug"
texturecache.py s "Bug"

check artwork with drive unplugged LOGS
As the logs show, all extrafanart and extrathumbs have been properly cached according to the logs.
However, screenshots of XBMC show a different story:
Extrathumbs are not cached - Screen 1
Extrafanart not used. Only fanart.jpg is shown as the background. Screen 2

6. Plug-In External Drive / Check Artwork

Code:

texturecache.py jd movies "Bug"
texturecache.py s "Bug"

check artwork with drive plugged-in LOG
As you can see, extrathumbs somehow got rescanned and now show twice in the log. However, I did not run any texturecache.py commands to make this happen.
Additionally, now both extrathumbs and extrafanart work properly!?!
Extrathumbs are now cached - Screen 3
Extrafanart in the extrafanart folder is now cycled through - Screen 4 Screen 5

So, to re-iterate, why aren't either the extrathumbs or extrafanart being properly cached by texturecache.py?

Thanks is advance for any advice, tips, and help!

[Milhouse]

Thanks for the very detailed logs, much appreciated!

First of all, this is my understanding of how extrafanart/extrathumbs works from the XBMC perspective.

extrathumbs/extrafanart are not present in the media library (this is why they're not shown with "jd movies") so what a skin has to do at run time is determine if you have extrathumbs/extrafanart artwork by checking the physical source to see if there is a directory called extrathumbs or extrafanart. If such a directory exists, the skin will then load the thumbs/fanart artwork using whatever art files it finds in your extrathumbs/extrafanart folders.

In your case, although the artwork is in the cache (when the disk is online), when you later view the movie while the disk is offline the skin will not be able to determine if you have an extrathumbs/extrafanart directory, so it will instead display placeholder artwork in any view that requires extrathumbs/extrafanart. I wouldn't be surprised if the skin generates a couple of "directory not found" type errors in the xbmc.log for each movie that is located on a sources that is offline.

Connect the drive/source again, and now the skin will "know" that you have the required directories so it will be able to load the relevant extrathumbs/extrafanart artwork.

In a nutshell, extrathumbs/extrafanart is only going to work while the source is online. This is because the media library knows nothing about extrathumbs/extrafanart - it's a skin-specific solution which requires physical access to the source at the time the movie is viewed.

Preloading the cache with the extrathumbs/extrafanart artwork avoids the whole caching-as-you-browse process but won't allow such artwork to be displayed while the source is offline because the skin has no way of determining that this extra artwork actually exists unless it can access the source.

As for the double caching of the extrathumbs/extrafanart - that looks like a bug (no pun intended!) in the skin.

Note how texturecache.py has cached the extrathumbs (first 4 rows) using a perfectly valid url, but the skin has cached the same artwork using a different (malformed) url:

Code:

070116|a/aa82ea12.jpg|0439|0780|0001|2014-05-05 09:15:28|2014-05-05 16:15:28|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs\thumb1.jpg
070117|7/71954285.jpg|0439|0780|0001|2014-05-05 09:15:28|2014-05-05 16:15:28|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs\thumb2.jpg
070118|3/38982508.jpg|0439|0780|0001|2014-05-05 09:15:28|2014-05-05 16:15:28|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs\thumb3.jpg
070120|c/c37b0e1c.jpg|0439|0780|0001|2014-05-05 09:15:30|2014-05-05 16:15:29|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs\thumb4.jpg

070123|c/c46211dd.jpg|0439|0780|0002|2014-05-05 09:36:19|2014-05-05 16:34:36|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs/thumb1.jpg
070124|5/5678dec7.jpg|0439|0780|0002|2014-05-05 09:36:19|2014-05-05 16:34:36|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs/thumb3.jpg
070125|a/ad9bf5d3.jpg|0439|0780|0002|2014-05-05 09:36:19|2014-05-05 16:34:37|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs/thumb4.jpg
070126|1/1f75b94a.jpg|0439|0780|0002|2014-05-05 09:36:19|2014-05-05 16:34:37|G:\Hard Drive 0\Movies - Kids\A Bugs Life [1998]\extrathumbs/thumb2.jpg

The skin is constructing the url for the extrathumbs artwork using a forward slash when appending the filename to the path and not a backward slash which would be normal (and correct) when using a Windows local source. When XBMC attempts to load the artwork using the malformed url it first looks in the cache and doesn't find anything for the url so the artwork is cached a second time using the malformed url. Subsequent views will find and use the cached artwork corresponding to the malformed urls.

XBMC will tolerate this kind of error in a url (ie. it will find and load the artwork from the source) but given the way the cache works, matching on exact urls, it does mean artwork can be cached multiple times if the url varies in any way. You might now use another skin that doesn't contain this forward-slash error, and this other skin will correctly use the artwork cached by texturecache.py, ignoring the same artwork cached using the malformed url, resulting in a excessive cache "bloat". Pruning your cache will also remove these malformed urls as they won't be recognised and retained.

My advice would be to contact the skin author and highlight the error as it means that anyone switching to this skin will find their cache bloated with double the extrathumbs artwork as the skin won't be able to make use of cached extrathumbs already loaded by other skins (or by texturecache.py). Out of interest, which skin is this?

[radigast]

Thanks for reading through my post wall and responding.

Preloading the cache with the extrathumbs/extrafanart artwork avoids the whole caching-as-you-browse process but won't allow such artwork to be displayed while the source is offline because the skin has no way of determining that this extra artwork actually exists unless it can access the source.

I was thinking the same as you. However, here's the weird thing: If I take the hard drive offline again after the malformed urls have been created, the extrathumbs remain cached and are displayed. It's almost like the skin doesn't like how texturecache.py cached the extrathumbs, so it had to redo the caching...or something. Extrafanart does not show up at all when the hard drive is offline no matter what.

Out of interest, which skin is this?

Aeon NOX Gotham

[Milhouse]

I was thinking the same as you. However, here's the weird thing: If I take the hard drive offline again after the malformed urls have been created, the extrathumbs remain cached and are displayed.

The only way I can think the skin is doing this is by referencing a local cache of filename data, maybe common cache populated while the source is online. I could be wrong though, however the simplest method is to just lookup the directory and see if it exists then use whatever files are inside. More complicated methods might allow some offline capability, I guess.

It's almost like the skin doesn't like how texturecache.py cached the extrathumbs, so it had to redo the caching...or something.

That's because the skin has a bug and is constructing the extrathumbs filename incorrectly - it's redoing the caching because it's not using the correct filename (url) that is already cached.

Extrafanart does not show up at all when the hard drive is offline no matter what.

Odd that it doesn't work the same way as extrathumbs, and it also doesn't seem to suffer from the malformed urls either. Probably best to ask the skin authour what is going on, and would be nice if they fixed the malformed url - that seems to happen quite often, unfortunately.

[Milhouse]

This is where the problem lies in Aeon Nox 4.1.9 (from github):

Code:

./1080i/variables.xml:1814:        <value>$INFO[ListItem.Path,,extrathumbs/thumb1.jpg]</value>
./1080i/variables.xml:1820:        <value>$INFO[ListItem.Path,,extrathumbs/thumb2.jpg]</value>
./1080i/variables.xml:1826:        <value>$INFO[ListItem.Path,,extrathumbs/thumb3.jpg]</value>
./1080i/variables.xml:1832:        <value>$INFO[ListItem.Path,,extrathumbs/thumb4.jpg]</value>

It's joining the path items with forward slashes - my guess is the developer is a Linux user, though if [s]he were to specify the alternative "extrathumbs\\thumb1.jpg" they'd end up creating malformed urls for Linux users and also anyone using network shares, which would be a much wider population of users that are then impacted.

Presumably in a skin there is no way to specify the correct forward/backward slash based on the type of path being joined. Perhaps XBMCs skin support should sanitise urls to avoid this kind of problem.

Looking at the skin code, the reason it may be displaying extrathumbs artwork even when the source is offline is because the skin isn't bothering to check if the extrathumbs directory exists before attempting to display the thumbs artwork - it just barrels ahead and displays the thumb[1-4].jpg artwork irrespective. So once the thumbs have been cached they will continue to appear even when the source is offline. This approach would also mean you will be seeing "file not found" errors in xbmc.log for any movie that doesn't have all 4 thumbs.

extrafanart appears to have been implemented using a different technique to extrathumbs, hence the lack of offline capability (maybe using script.grab.fanart?)

Anyway, long story short the url being formatted for extrathumbs by the skin results in a malformed url for local Windows storage, and whether the skin author can fix it so that it works correctly for all types of storage on all platforms, I don't know.

I could change texturecache.py so that it created malformed urls for extrathumbs but chances are it's then going to be wrong for someone else using a different skin that doesn't generate malformed urls for local Windows storage but which might do so for non-local Windows storage, and expecting users to know that their skin is right or wrong is perhaps asking a bit much.

If you really want to change texturecache.py to create malformed extrathumbs urls, the following patch should work - no real testing, mind:

Code:

diff --git a/texturecache.py b/texturecache.py
index 91f7290..aca4f8d 100755
--- a/texturecache.py
+++ b/texturecache.py
@@ -2081,12 +2081,15 @@ class MyJSONComms(object):
             break

     files = []
+    re_thumb_fix = re.compile("(.*extrathumbs)(\\\\)(thumb.*\.jpg)")
     for dir in dirs:
       data = self.getDirectoryList(dir["file"])
       if "result" in data and "files" in data["result"]:
         for file in data["result"]["files"]:
           if file["filetype"] == "file" and file["file"]:
             if os.path.splitext(file["file"])[1].lower() in [".jpg", ".png", ".tbn"]:
+              thumb_fix = re.match(re_thumb_fix,file["file"])
+              if thumb_fix: file["file"] = "%s/%s" % (thumb_fix.group(1), thumb_fix.group(3))
               files.append({"file": MyUtility.denormalise(file["file"], prefix=True), "type": dir["type"].lower()})

     return files

although changing the skin files at the four points listed at the top of this post might be more appropriate, but neither is a long term fix.

[scott967]

I wanted to give this a try, so i installed python 2.7.6 on Win 7x64. I am running XBMC 13.0. When I run the script, it errors out:

Code:

File "E:\Python27\texturecache.py". line 1418
  data.append({u"textureid": r[0], u"cachedurl": r[1],
                          ^
SyntaxError: invalid syntax

any ideas?

scott s.
.