[Radikaltimes]

is there a way to add clean library to the process?

Adding the function to clean the database is pretty trivial. The problem I found when trying to add this earlier is that there is no way to tell if a database clean operation is happening (like you can with scanning) so it is possible to hose up the xbmc process by executing it while another is already running. I did a test case with a cron expression set to run fairly often, like every 10 minutes, and locked up xbmc. Granted you wouldn't set them off that close together, but what happens if a clean operation takes 1-2 hours and another kicks off?

I've thought about how to get around this and the best I can think of is to only allow the clean database function to execute minimum once every 24 hours. Another option you have is add a setting to the advancedsettings file that does a clean immediately following an update. http://wiki.xbmc.org/index.php?title=Adv...library.3E

I guess that was just a really long way of saying that I've been working on it. Having a variable to check like there is with scanning to tell if it is running or not would be a big help.

perfect ..

[paddycarey]

Hi,

I have a problem where sometimes my network shares don't come up and so on an update library (i have it set to autoclean on update for various reasons) i lose my db.

Would it be possible to add an option to settings to check for the existance of a specified path before doing the scan?

I have it hardcoded in my local copy of the script now but the option may be useful to others.

I just use a simple

Code:

if not xbmcvfs.exists(some_path):
    #skip scan

to check before letting the update run.

[rflores2323]

^ is a great idea as I have had the same problem with my setup and stopped using the addon due to this. If this could be implementedI would get this going on my system again.

[robweber]

Hi,

I have a problem where sometimes my network shares don't come up and so on an update library (i have it set to autoclean on update for various reasons) i lose my db.

Would it be possible to add an option to settings to check for the existance of a specified path before doing the scan?

I have it hardcoded in my local copy of the script now but the option may be useful to others.

I just use a simple

Code:

if not xbmcvfs.exists(some_path):
    #skip scan

to check before letting the update run.

I understand your issue, and definitely sympathize, but I'm unsure how your fix will translate well to users in general. Imagine an instance where you have 4 or 5 different paths as part of your video sources. Issuing the UpdateLibrary() function goes through each of the content sources and scans for new content. Using a xbmcvfs check how does the addon know which of the paths to check before running the update? Should it check all of them? What if one fails but the others succeed - you'll still get an update that will most likely remove the non-existant path (via the auto clean operation)? What if the path legitmately doesn't exist? (ie it should be removed) - you'll never get another update.

I'm not tossing out this idea but it needs some thought before I can be sure it will work for everyone. Thanks for pointing this out though, I had not encountered this before with my setup.

[paddycarey]

One option might be to use the JSON api to get a list of all the configured shares (pretty sure this can be done) and then attempt to write/delete/touch/whatever a small file on each one to make sure they're up. This obviously requires having write access to the shares.

Does the JSON API give a different response when attempting to list a share if it's down as opposed to empty? If so then that would be one simple way to implement this.

[robweber]

One option might be to use the JSON api to get a list of all the configured shares (pretty sure this can be done) and then attempt to write/delete/touch/whatever a small file on each one to make sure they're up. This obviously requires having write access to the shares.

Does the JSON API give a different response when attempting to list a share if it's down as opposed to empty? If so then that would be one simple way to implement this.

I checked the JSON API, you get the path and label for each source; no information on if it is accessible. Although this is a good way to find the available source paths, the problem still remains that this won't fix the cleanDB issue. Examine the following pseudo code:

Code:

foreach source
   if source exists
       run libraryupdate on given path

Seems pretty simple right? Well the advancedsettings cleanonupdate entry is going to try to clean the library after each of those "run libraryupdate" commands. So even if only 1 out of 4 sources is connected, the clean DB process is going to run and wipe out the other three.

This method of updating the database is also highly inefficient. The default UpdateLibrary(video) command will go through all the sources first, then execute the clean command (based on your advanced settings entry). The code above will run the UpdateLibrary(video,path) command on each path separately, and then run the clean command after each individual update. Updating your library with 2 sources would require waiting through an entire clean operation between each path update.

I do have an alternate solution. Since it is really the clean update command, not the update library command, that is removing these sources. Why not check if your paths exist before running the clean command instead? It is doable for the Auto Update addon to check your sources for connectivity and then run a clean operation. If all of your sources aren't connected, the clean operation doesn't run and you don't lose your stuff. This would also get rid of the need for the advancedsettings entry.

Does this sound like a workable solution?

[paddycarey]

Seems pretty simple right? Well the advancedsettings cleanonupdate entry is going to try to clean the library after each of those "run libraryupdate" commands. So even if only 1 out of 4 sources is connected, the clean DB process is going to run and wipe out the other three.

This method of updating the database is also highly inefficient. The default UpdateLibrary(video) command will go through all the sources first, then execute the clean command (based on your advanced settings entry). The code above will run the UpdateLibrary(video,path) command on each path separately, and then run the clean command after each individual update. Updating your library with 2 sources would require waiting through an entire clean operation between each path update.

AFAIK the clean on update will only clean the part of the library that was specified in the update call, e.g. when i update and clean the whole library the clean operation seems to take about 3 minutes but if i only update my tv path then the clean operation only takes a few seconds.

I realise it's a bit inefficient but when the library's only being updated every few hours does it make a lot of difference for those extra few seconds? That being said, if you think your other solution will work better then i'm all for it. The only issue i've encountered doing it that way is that a standalone cleanlibrary call seems to much longer than the same call when run automatically after update.

One simple way to reduce the inefficiency might be to do something like:

Code:

for share in shares:
    if share_up:
        mark share as online
    else:
        mark share as offline

if all_shares_up:
    update whole library
else:
    for shares in alive_shares:
        update share

This way you still check that the shares are all up but don't have to run individual update calls if none are down.

---

One other note, as far as I can tell, when a cleanlibrary is run with cleanonupdate, library.isscanning will return true until that clean has finished.

[robweber]

AFAIK the clean on update will only clean the part of the library that was specified in the update call, e.g. when i update and clean the whole library the clean operation seems to take about 3 minutes but if i only update my tv path then the clean operation only takes a few seconds.

I realise it's a bit inefficient but when the library's only being updated every few hours does it make a lot of difference for those extra few seconds? That being said, if you think your other solution will work better then i'm all for it. The only issue i've encountered doing it that way is that a standalone cleanlibrary call seems to much longer than the same call when run automatically after update.

One simple way to reduce the inefficiency might be to do something like:

Code:

for share in shares:
    if share_up:
        mark share as online
    else:
        mark share as offline

if all_shares_up:
    update whole library
else:
    for shares in alive_shares:
        update share

This way you still check that the shares are all up but don't have to run individual update calls if none are down.

---

One other note, as far as I can tell, when a cleanlibrary is run with cleanonupdate, library.isscanning will return true until that clean has finished.

Thanks for the info - I'll be honest I didn't test the library update path vs whole update command when cleaning the library. Since the default cleanlibrary command doesn't take path parameters I didn't think it would be specific like that.

In any case this is what I've done. I added a "Cleaning" category to the addon settings. You can turn on library cleaning and specify if you want media paths checked first, and a frequency to do the cleaning. By default the clean will happen immediately following an update. If you want to only do daily, weekly, monthly cleaning those options are there too.

A few caveats with how this will work. If you are checking paths and one is down - an error will be thrown and no clean operations will happen. If the updater is not updating your video library, it will not clean the video library. The same with music. The daily/monthly/weekly times are hardcoded for midnight for the video library and 2am for the music library. The options are really stripped down because most people seem to either want a clean after each update; or they don't really care to do one that often, if at all.

I also threw in the "single video path" option I've been working on. You can specify a specific video path and scan that one on a different timer rather than doing the whole library. Below is a link for those who can't wait, although I'll send a repo pull request so everyone will be updated automatically within a few days.

http://xbmclibraryautoupdate.googlecode....update.zip

Thanks a lot for everyone's thoughts and ideas on this. Any feedback on these new changes is welcome.

[LoL]

Is working excellent for me!

Really enjoy the "single video path" scan: could there be an option to add more paths with each their own timers?

Thx!

[robweber]

Is working excellent for me!

Really enjoy the "single video path" scan: could there be an option to add more paths with each their own timers?

Thx!

I think that is a good idea - how many would be appropriate? I tend to think 2-3 but that is because I only have like 2-3 sources on my install! More than 4 or 5 seems excessive, easier to just update the whole library at that point.

Any thoughts?

[LoL]

For me 2-3 would be enough i think. But just in case, wouldn't it be possible to code it so as a user you always have the possibility to add another.

I'm really enjoying the single path update man, its so much faster and feels cleaner

[robweber]

For me 2-3 would be enough i think. But just in case, wouldn't it be possible to code it so as a user you always have the possibility to add another.

I'm really enjoying the single path update man, its so much faster and feels cleaner

Thanks for the input. Letting the user add another within the settings really isn't possible at the moment. The addon settings are a static file that needs to have predefined variable inputs. The only dynamic way to do it would be to have the setup of each path be something that was done within the addon itself (some kind of custom written GUI) or via a config file that user would manually have to edit. Both of those aren't very appealing. Having two different areas of setup doesn't seem very intuitive. I do agree though it would be nice to have dynamic use cases like that.

I'll add some more custom path options to get the total up to three and we can see where that goes. For anyone wondering I also have not forgotten about the request to run a script after an update completes. I've just been busy!

[teevee]

Do you know if there is any way to hide the "Cleaning Up Library" notification window? I have the add-on perform a cleanup after every update, and would rather have not have the notification appear. I have this set to update the library even if something is currently playing, which means the notification will even appear on top of what I'm watching.

[robweber]

I'm not sure there is a way to do this. I tried the "backgroundupdates" option in advanced settings and that didn't seem to do anything for the clean library operation. You might be stuck looking at that - or put in a Request.

[Zapa]

When cleaning the library and checking source paths first, a dvd-drive is registered as a broken source, when there are no dvd in it
is it possible to check if content typs, on the source, is set?

To make this run smoothly, the only solution I have found, is to delete the dvd-drive as source