[Milhouse]

There seem to be two problems.

1. The call to "texturecache.py jd movies", as per the "FATAL" message, so check your RPC server is running and listening on port 9090.

2. The problem with L:\Videos, it shouldn't matter if it's network or local, just so long as it's readable/writeable by your current user. For some reason mklocal.py thinks this path is not available, so you should double check the permissions on this folder.

[Mychale]

Ugh, didn't realize I didnt have Kodi running when I ran that last to get the error message for you.

Once Kodi is running, I get the following error message:

Code:

usage: mklocal.py [-h] [-l DIRECTORY] [-p PATH] [-A PATH] [-i [FILENAME]] [-o [FILENAME]]
                  [--dryrun] [-r] [--ignorebadprefix] [-a TYPE [TYPE ...]]
                  [-c TYPE [TYPE ...]] [-s [TYPE [TYPE ...]]] [-e [TYPE [TYPE ...]]] [-1]
                  [-nk] [--info] [-q | -v]
mklocal.py: error: local DIRECTORY L:\Videos\ does not exist!
Traceback (most recent call last):
  File "C:\texturecache.py", line 7923, in <module>
    main(sys.argv[1:])
  File "C:\texturecache.py", line 7736, in main
    extraFields=_extraFields, query=_query)
  File "C:\texturecache.py", line 4246, in jsonQuery
    jcomms.dumpJSON(data, decode, ensure_ascii)
  File "C:\texturecache.py", line 2592, in dumpJSON
    self.logger.out(json.dumps(data, indent=2, ensure_ascii=ensure_ascii, sort_keys=True), newLine=True)
  File "C:\texturecache.py", line 925, in out
    sys.stdout.write("%-s%s" % (udata, NL))
  File "C:\Python34\lib\codecs.py", line 369, in write
    self.stream.write(data)
OSError: [Errno 22] Invalid argument

C:\>

and I did check the permissions on the folder and it is correct. The weird thing is that this was working fine before, but it has been a few months since I ran it last.

Thank you for the fast reply, and all your help.

[Milhouse]

OSError: [Errno 22] Invalid argument

On Linux this would probably be:

Code:

IOError: [Errno 32] Broken pipe

and just means that, because of the DIRECTORY error resulting in the failure of mklocal.py, there is nothing to consume the output of texturecache.py.

So at least this now just a problem with L:\Videos.

Try the following commands (you may need to adapt the first command so that whatever Python interpreter you have installed on your system is invoked):

Code:

python
import os
os.path.exists("L:\\Videos")

The last line should return "True" if the directory is accessible to Python - if not, there's something wrong with your system/folder/path/permissions.

[Mychale]

Ok, I tried as you suggested, and it returns a False result. So definitely a issue with the folder.

Just a heads up, I am running this on Win7, using python34 installed to the C:\python34 folder.

In windows explorer, I can see the L:\ drive mapped and can access it fine. I have Admin rights on my account, and opening the CMD window using administrator access as well. The folder is readable/writeable to me in explorer. Checking the folder mapping using NET USE, shows them listed, but all are showing as unavailable. Strange since I can see them mapped in explorer and access them. I tried to change to that L: drive in CMD and get the error that the system cannot fine the drive specified.

Thanks for pointing me in the right direction. Now to troubleshoot this issue.

[Milhouse]

[Mychale]

LOL, I love the reply!! That show is my life in video. The wife and I had a good laugh at this reply.

Upon further inspection, it seems the drives were mapped in explorer using the normal account, and when accessing the CMD Admin level, they are shown as Unavailable. Opening a Non elevated CMD window and using Net Use, shows them all listed and OK status. So that is the issue right there.

Now I am able to run this in the non-elevated CMD window without issue. Strange, since Admin rights should show/access everything, or so I thought. It seems there are two tokens created for Admin accounts, and the CMD window keeps them separate no matter what.

Thanks for the laugh.

[calisro]

Hi. I have an issue. I was trying to use the ./texturecache.py along with mklocal.py and I realized I have an issue with how entries are being stored in my database.

I am using DAVS as a source rather than NFS. When my library is scrapped, the entries are added like this:

Code:

"file": "davs://xxxxxx:xxxxxxx@some_host:9999/base/Media/Videos/Movies/The.Riot.Club.%282014%29.720p.bluray.h264.dd5.1/The.Riot.Club.%282014%29.720p.bluray.h264.dd5.1.mkv"

but on the file system it is actually stored like this:

Code:

/data/Media/Videos/Movies/The.Riot.Club.(2014).720p.bluray.h264.dd5.1/The.Riot.Club.(2014).720p.bluray.h264.dd5.1.mkv
/data/Media/Videos/Movies/The.Riot.Club.(2014).720p.bluray.h264.dd5.1/The.Riot.Club.(2014).720p.bluray.h264.dd5.1.nfo
/data/Media/Videos/Movies/The.Riot.Club.(2014).720p.bluray.h264.dd5.1/The.Riot.Club.(2014).720p.bluray.h264.dd5.1-poster.jpg
/data/Media/Videos/Movies/The.Riot.Club.(2014).720p.bluray.h264.dd5.1/The.Riot.Club.(2014).720p.bluray.h264.dd5.1-fanart.jpg

It looks like xbmc is storing the paranthesis as encoded rather than decoding them. This doesn't cause any problems and all of my xbmc boxes work as expected. That being said, when I try to use the tool like this:

Code:

./texturecache.py jr movies | ./mklocal.py --local /data/Media/Videos/Mtest  --prefix davs://xxxxxx:xxxxxxx@some_host:9999/base/Media/Videos/Movies --artwork fanart poster clearlogo clearart --output fixup.dat

It attempts to create directories and file names encoded which isn't correct in my case. Is there anything that can be done? I really would have liked to use the mklocal.py to store images at original quality. I actually think xbmc is probably storing it incorrectly but it is what it is...

I'm not sure if this is fixable as the root of hte problem is xbmc storing them incorrectly and mklocal doesn't know if an entrry is really a parenthasis of a literal '%28'/'%29'.

Thoughts??

[Milhouse]

Try jd instead of jr.

jd is "json decoded" (so "%28" decoded to ")" etc.), jr is "json raw" with no decoding.

[calisro]

Try jd instead of jr.

jd is "json decoded" (so "%28" decoded to ")" etc.), jr is "json raw" with no decoding.

Yes, I tried jd and jr. I was hoping jd would solve it but it doesn't.

Code:

->$ ./texturecache.py jd movies | grep Riot
    "file": "davs://xxxxxx:xxxxxxx@some_host:9999/base/Media/Videos/Movies/The.Riot.Club.%282014%29.720p.bluray.h264.dd5.1/The.Riot.Club.%282014%29.720p.bluray.h264.dd5.1.mkv",
    "label": "The Riot Club",
    "title": "The Riot Club"

I've also tried turning it off and on again just to be sure.

[calisro]

I guess I could do this with sed. This makes me nervous though.


./mklocal.py --local /data/Media/Videos/Mtest --prefix davs://xxxxxx:xxxxxxx@some_host:9999/base/Media/Videos/Movies --artwork fanart poster clearlogo clearart --output fixup.dat && cat fixup.dat | sed -e 's/)/%29/g' -e 's/(/%28/g' > correct.dat

Edit: I'd need to account for all encoded ones in addition to ( and ) which makes me more nervous. I have '&' and ',' as special characters. Ampersand is easy but commas are harder since commas are used as part of JSON so i'd need to use better regexp to do that work.

[Milhouse]

Try v2.0.9 - this will now url decode "file" items for jd and jr.

These are the different j variations now I've reminded myself...

j: original json with no url decoding. ASCII is ensured.
jd: decoded urls, ensure ascii
jr: decoded urls, do not ensure ascii (so you may see unicode data in url fields).

Example:

Code:

$ ./texturecache.py j movies extraordinary
[
  {
    "art": {
      "clearlogo": "image://nfs%3a%2f%2f192.168.0.3%2fmnt%2fshare%2fmedia%2fVideo%2fMoviesHD%2fForeign%20Language%2fThe%20Extraordinary%20Adventures%20of%20Ad%c3%a8le%20Blanc-Sec%20(2010)%5bBDRip%5d-logo.png/",
      "fanart": "image://nfs%3a%2f%2f192.168.0.3%2fmnt%2fshare%2fmedia%2fVideo%2fMoviesHD%2fForeign%20Language%2fThe%20Extraordinary%20Adventures%20of%20Ad%c3%a8le%20Blanc-Sec%20(2010)%5bBDRip%5d-fanart.jpg/",
      "poster": "image://nfs%3a%2f%2f192.168.0.3%2fmnt%2fshare%2fmedia%2fVideo%2fMoviesHD%2fForeign%20Language%2fThe%20Extraordinary%20Adventures%20of%20Ad%c3%a8le%20Blanc-Sec%20(2010)%5bBDRip%5d-poster.jpg/"
    },
    "file": "nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Ad\u00e8le Blanc-Sec (2010)[BDRip].mkv",
    "label": "The Extraordinary Adventures of Ad\u00e8le Blanc-Sec",
    "movieid": 2017,
    "title": "The Extraordinary Adventures of Ad\u00e8le Blanc-Sec"
  }
]
$ ./texturecache.py jd movies extraordinary
[
  {
    "art": {
      "clearlogo": "image://nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Ad\u00e8le Blanc-Sec (2010)[BDRip]-logo.png/",
      "fanart": "image://nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Ad\u00e8le Blanc-Sec (2010)[BDRip]-fanart.jpg/",
      "poster": "image://nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Ad\u00e8le Blanc-Sec (2010)[BDRip]-poster.jpg/"
    },
    "file": "nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Ad\u00e8le Blanc-Sec (2010)[BDRip].mkv",
    "label": "The Extraordinary Adventures of Ad\u00e8le Blanc-Sec",
    "movieid": 2017,
    "title": "The Extraordinary Adventures of Ad\u00e8le Blanc-Sec"
  }
]
$ ./texturecache.py jr movies extraordinary
[
  {
    "art": {
      "clearlogo": "image://nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Adèle Blanc-Sec (2010)[BDRip]-logo.png/",
      "fanart": "image://nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Adèle Blanc-Sec (2010)[BDRip]-fanart.jpg/",
      "poster": "image://nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Adèle Blanc-Sec (2010)[BDRip]-poster.jpg/"
    },
    "file": "nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Foreign Language/The Extraordinary Adventures of Adèle Blanc-Sec (2010)[BDRip].mkv",
    "label": "The Extraordinary Adventures of Adèle Blanc-Sec",
    "movieid": 2017,
    "title": "The Extraordinary Adventures of Adèle Blanc-Sec"
  }
]

Edit: Hopefully this change doesn't have any unwanted side effects...

[calisro]

um. You did that faster than my writing of simple sed statements. Are you a machine?

It looks good at a glance.

Code:

->$ ./texturecache.py jd movies | grep Lock
    "file": "davs://xxxx/base/Media/Videos/Movies/Lock,.Stock.and.Two.Smoking.Barrels.(1998).720p.bluray.h264/Lock,.Stock.and.Two.Smoking.Barrels.(1998).720p.bluray.h264.mp4",
    "label": "Lock, Stock and Two Smoking Barrels",
    "title": "Lock, Stock and Two Smoking Barrels"

->$ ./texturecache.py jd movies | grep Pea
    "file": "davs://xxxx/base/Media/Videos/Movies/Mr..Peabody.&.Sherman.(2014).720p.webdl.h264.aac/Mr..Peabody.&.Sherman.(2014).720p.webdl.h264.aac.mp4",
    "label": "Mr. Peabody & Sherman",
    "title": "Mr. Peabody & Sherman"

[calisro]

In my case it worked 100%. I did a jpg compare across a test and actual directories.

[calisro]

Got a Q. With the mklocal.py script, can you ask that it overwrites existing files? If we've previously done a backup from xbmc and we want to replace those files, it would be difficult otherwise. What do you think?

Also, the change to texturecache.py worked but that only solves half of the problem. The mklocal.py script will read that input and create images correctly now which is GREAT but the resulting fixup.dat file now has decoded URLs in it. I don't think that is safe to re-import back to xbmc as the URLs stored in xbmc were originally encoded.

Its almost like we need a option in mklocal.py to use original URLs with the new base rather than the output URLs but how would mklocal know what the original was as it is just a transformation script and whats getting passed to it is whatever.

EDIT: Or we can just run texturecache.py twice. Once to decode for the images and once to output the correct URLs for making them local in the database. The only caveat is if there was a change (new scan) that occured between those two runs.

[Milhouse]

Got a Q. With the mklocal.py script, can you ask that it overwrites existing files? If we've previously done a backup from xbmc and we want to replace those files, it would be difficult otherwise. What do you think?

It doesn't currently, mainly because it doesn't know if the existing artwork is good/original quality artwork downloaded by a previous run (and which shouldn't be overwritten), or artwork exported from Kodi (that could/should be overwritten).

My advice would be to delete the artwork you want replaced before running mklocal.py, so that new original artwork is downloaded.

Also, the change to texturecache.py worked but that only solves half of the problem. The mklocal.py script will read that input and create images correctly now which is GREAT but the resulting fixup.dat file now has decoded URLs in it. I don't think that is safe to re-import back to xbmc as the URLs stored in xbmc were originally encoded.

Kodi will re-encode the urls before storing them - give it a try on one of your movies, or try it manually at the command line:

Code:

./texturecache py set movie #### art.poster "<url suggested by fixup.dat>"

where #### is the movieid of the movie.

If davs:// urls require addition encode processing then I guess something will need to be done about that (not sure what, exactly).