[Milhouse]

Version 0.2.7:

* Added new QA test, qa.file (yes/no, default:no) to check if the underlying video file still exists. Won't initiate remove/rescan in qax, obviously , and is thus purely informational.

* Added "Duplicate" statistic summary class shown at the end of c/C. This is for when the same artwork is used multiple times. Rather than attempting to re-cache the same item as it is referenced each time by different media (which is pointless, and worse, with the C option could result in the same image being removed then re-cached multiple times), the script now remembers each image item as it is referenced during the current run, and will only cache an image once, quickly ignoring any duplicate references, resulting in improved run-times.

* Tidied up logging, c/C statistics (now with added key), and a little bit of code refactoring (hope it didn't break anything...)

[Milhouse]

Version 0.2.8:

* Added jd and Jd options. Functionally equivalent to the existing j and J options, these new options will also decode encoded URLs.

[Milhouse]

Version 0.2.9:

* Added support for Video Library Tags. Use the "tags" keyword as you would movies/sets/tvshows etc., and optionally filter by a tag name (including partial tag names). Options with tag support: j, J, jd, Jd, c, C, qa and qax.

For example:

Code:

./texturecache.py j tags live-comedy
./texturecache.py c tags anime
./texturecache.py qa tags foreign-language
./texturecache.py qax tags

The last example will process all movies that have at least one tag assigned. Not currently possible to filter by more than one tag.

[charrua]

Version 0.2.9:

* Added support for Video Library Tags. Use the "tags" keyword as you would movies/sets/tvshows etc., and optionally filter by a tag name (including partial tag names). Options with tag support: j, J, jd, Jd, c, C, qa and qax.

For example:

Code:

./texturecache.py j tags live-comedy
./texturecache.py c tags anime
./texturecache.py qa tags foreign-language
./texturecache.py qax tags

The last example will process all movies that have at least one tag assigned. Not currently possible to filter by more than one tag.

Thanks for all the new features!

One more suggestion for the future:
multiple threads for speeding up long tasks

[Milhouse]

One more suggestion for the future:
multiple threads for speeding up long tasks

Run multiple separate processes? At the end of the day, the speed is determined by the server... some systems (eg. Pi) are pretty much maxed out by a single task.

[un1versal]

I have a question regarding cache.

Currently total duration mostly noticeable on TVshows needs to be calculated/cached in Im not sure where this info is cached (I do believe its textures#.db since this file grows as this info is cached) but its a slow process that delays displaying of SD/HD (blue/green) flags in confluence and other flagging is also delayed until total duration is calculated for all materials. To calculate this for TV shows you need to be in library view and under all seasons/episodes (lists all episodes oof all seasons view) so total duration starts being calculated and thus SD/HD and codec info flags start showing up.

Can this script somehow trigger this caching in future?
Particularly where a user doesnt have to navigate around and stay in that root dir waiting for this caching/calculation to take place. its a long monotonous job.# as soon as you leave any directory this caching stops so basically means if you want everything cached its a pita.

uNi

[charrua]

One more suggestion for the future:
multiple threads for speeding up long tasks

Run multiple separate processes? At the end of the day, the speed is determined by the server... some systems (eg. Pi) are pretty much maxed out by a single task.

Specially while using c/C options, I've noticed that the script is slower at making XBMC preload the images than XBMC loading the images (not yet cached) when using the GUI. I think that's because XBMC loads multiple images at once (even when they aren't yet cached) while the script is loading them one by one.
The process is not CPU intensive, probably the speed is determined by the http requests to get the images, which I think could be greatly improved by implementing multiple threads.

[Milhouse]

Specially while using c/C options, I've noticed that the script is slower at making XBMC preload the images than XBMC loading the images (not yet cached) when using the GUI. I think that's because XBMC loads multiple images at once (even when they aren't yet cached) while the script is loading them one by one.
The process is not CPU intensive, probably the speed is determined by the http requests to get the images, which I think could be greatly improved by implementing multiple threads.

I think the default is for 2 threads on XBMC (certainly on the Pi).

You could run the script on a more powerful machine, such as a desktop, or even multiple scripts on a desktop, and point the desktop at an XBMC client. However it's the image conversion that has to be done by the XBMC which is where the load/delays can occur, particularly on a Pi. Not even sure if multiple http downloads are possible (this is what triggers the caching process).

Thing is, though, I really don't want to to get involved in writing threaded code if I can help it! If I run out of ideas I'll kick it around to see what's involved... but no promises!

---

I have a question regarding cache.

Currently total duration mostly noticeable on TVshows needs to be calculated/cached in Im not sure where this info is cached (I do believe its textures#.db since this file grows as this info is cached) but its a slow process that delays displaying of SD/HD (blue/green) flags in confluence and other flagging is also delayed until total duration is calculated for all materials. To calculate this for TV shows you need to be in library view and under all seasons/episodes (lists all episodes oof all seasons view) so total duration starts being calculated and thus SD/HD and codec info flags start showing up.

Can this script somehow trigger this caching in future?
Particularly where a user doesnt have to navigate around and stay in that root dir waiting for this caching/calculation to take place. its a long monotonous job.# as soon as you leave any directory this caching stops so basically means if you want everything cached its a pita.

uNi

Certainly this script will cache the tvshow images, but not entirely sure about the stream details (SD/HD etc.), it may be possible to "pre-load" that information by requesting the stream details via json (can already do this with the J/Jd option and adding streamdetails property to extra.json.tvshows.episode)... I'll investigate shortly, just adding some extra tag support, or give it a try and see if you get valid stream details for newly added tvshows (and then see if they're there in the GUI when you access the Season/Episode list).

[charrua]

Specially while using c/C options, I've noticed that the script is slower at making XBMC preload the images than XBMC loading the images (not yet cached) when using the GUI. I think that's because XBMC loads multiple images at once (even when they aren't yet cached) while the script is loading them one by one.
The process is not CPU intensive, probably the speed is determined by the http requests to get the images, which I think could be greatly improved by implementing multiple threads.

I think the default is for 2 threads on XBMC (certainly on the Pi).
You could run the script on a more powerful machine, such as a desktop, or even multiple scripts on a desktop, and point the desktop at an XBMC client. However it's the image conversion that has to be done by the XBMC which is where the load/delays can occur, particularly on a Pi. Not even sure if multiple http downloads are possible (this is what triggers the caching process).
Thing is, though, I really don't want to to get involved in writing threaded code if I can help it! If I run out of ideas I'll kick it around to see what's involved... but no promises!

Thanks! (only a suggestion anyway... )
I tried running multiple instances of the script simultaneously, and it worked, so it will probably work using multiple threads too, but I understand completely that using threads in a script is not always the easiest way to go.

I have a question regarding cache.
Currently total duration mostly noticeable on TVshows needs to be calculated/cached in Im not sure where this info is cached (I do believe its textures#.db since this file grows as this info is cached) but its a slow process that delays displaying of SD/HD (blue/green) flags in confluence and other flagging is also delayed until total duration is calculated for all materials. To calculate this for TV shows you need to be in library view and under all seasons/episodes (lists all episodes oof all seasons view) so total duration starts being calculated and thus SD/HD and codec info flags start showing up.
Can this script somehow trigger this caching in future?
Particularly where a user doesnt have to navigate around and stay in that root dir waiting for this caching/calculation to take place. its a long monotonous job.# as soon as you leave any directory this caching stops so basically means if you want everything cached its a pita.
uNi

The total duration of media is not cached, it's calculated in real time each time you access a list.
If you want the loading of lists to be faster you can use a skin that doesn't show that info (there are a few).

[un1versal]

Certainly this script will cache the tvshow images, but not entirely sure about the stream details (SD/HD etc.), it may be possible to "pre-load" that information by requesting the stream details via json (can already do this with the J/Jd option and adding streamdetails property to extra.json.tvshows.episode)... I'll investigate shortly, just adding some extra tag support, or give it a try and see if you get valid stream details for newly added tvshows (and then see if they're there in the GUI when you access the Season/Episode list).

Excellent, if you wish to duplicate the behaiviour I described, try this with a TV show that has at least 50 episodes. Remove it from library and re-add it then navigate to TVShows>Library>Tittle>All Seasons you will notice all stream details are being re-cached and not listed immediately, as total duration gets calculated and to make flags show up and they get calculated jump from listing to scroll bar (this triggers refresh of flags).

Would be extra nice if script could trigger the calculation of the total duration as well but am unsure if this is at all possible.

Thanks for response will be keeping an eye.

uNi

[Milhouse]

v0.3.0 is up:

* Added combinatorial logic for tags, eg.:

Code:

./texturecache.py c tags "comedy and standup"
./texturecache.py c tags "action or adventure"

[Milhouse]

Would be extra nice if script could trigger the calculation of the total duration as well but am unsure if this is at all possible.

Thanks for response will be keeping an eye.

uNi

Sadly not having much luck.

I've tried all the methods that return stream details (Files.GetFileDetails, VideoLibrary.GetEpisodeDetails, individually and in combination), even downloading the video file (Files.PrepareDownload then downloading over HTTP, tried Files.Download), and I can't find any JSON method that causes XBMC to extract and return the stream details.

Basically, unless you've played back the video in XBMC (or listed it in the GUI, waiting for the SD/HD tags to appear), then XBMC will refuse to return stream details over JSON, which sucks, as I know the problem you are referring to but it looks like it's being handled specially by the GUI and beyond JSON at this time. Probably should be raised as an issue, as this renders the streamdetails side of JSON nigh on useless - virtually my entire library has no stream details according to JSON.

I don't have any iOS or Android apps here, but I'm guessing they don't show streamdetails for videos that haven't yet been pulled up in the GUI, or never played back,but if they are showing these stream details for new media then there must be a way...

Suggestions welcomed!

---

The total duration of media is not cached, it's calculated in real time each time you access a list.
If you want the loading of lists to be faster you can use a skin that doesn't show that info (there are a few).

It seems this has to be extracted from the media on the fly, if it's not already specified in the NFOs, and this is only done the first time the item is listed (or played back)... once that information is extracted, and stored in the media library, it should be much quicker to calculate the total runtime of a given list of items.

The trick is finding the trigger that causes XBMC to extract this information from specific media items - it certainly doesn't look like this has been implemented for JSON, unfortunately. Grrrr...

[Milhouse]

The trick is finding the trigger that causes XBMC to extract this information from specific media items - it certainly doesn't look like this has been implemented for JSON, unfortunately. Grrrr...

Here's the official answer - can't be done right now, and no easy solution.

[un1versal]

I dont have iOS or android apps either Im in Linux Ubuntu. But yea sure is a shortcomming of JSON, but who knows if it will get implemented anytime soon.

Thanks anyway for looking into this and for prompt replies. Fantastic

While Im here would make some suggestion, it would be great if this was a addon in XBMC which one could use with a remote/keyboard via UI or even better if it was part of a web interface like XWMM.
Perhaps you can even consider using github for development/delivery of anything like this as well...

Thanks for tool anyway though haven't used it I can see this being a part of my maintenance tools for XBMC in future.

uNi

[Milhouse]

Thanks for your kind words, however if you ran the script you might realise why it won't be an addon any time soon!

I'm much happier with it at the command line for now, and I'm really not sure if I'd be able to create a suitable gui for this, plus half it's current functionality would most likely be lost. Hope you find a use for it soon, I'd be interested in your feedback.

I do intend to put it on github eventually.