Kodi Community Forum

Just ran 0.5.2 on my system for the first time - all good (best I can tell!)

My system is Kodi 16.1 on Windows 10 with a MySQL 5.6.29 shared DB.

Database was created originally with XBMC 12.x (Frodo) and since then upgraded through 13.x (Gotham), Helix (14.x), and now up to 16.1 (Jarvis). Yes, I skipped Isengard (15.x) as that seemed a complete lemon of a build for Windows.. Anyway, point is that the database has been around the block a few times
Addon ran very quickly and based on the backup file size, DB only shrunk by 25KB (19,588 -> 19,563KB) but that's not too surprising given I don't play many videos from sources not in my sources.xml or the Apple itunes Movie Trailers addon which I added to the excludes.xml.

One observation I did make that you may want to add to the instructions is that the %AppData%\Kodi\userdata\addon_data\script.database.cleaner
folder doesn't exist by default. You have to change a setting in the addon to make the folder actually appear. You can then proceed to manually create the excludes.xml file.

Thanks for the feedback !!

Yes, if you only ever use the add-on's default settings then it Kodi would have no reason to create the 'script.database.cleaner' directory within 'addon_data'.

I do have a version of the script where you can create an excludes.xml file from within the settings of the add-on, but as it is a bit of an advanced feature, and using Kodi's GUI to build the excludes.xml is a little 'clunky', I've not released it.

I do have a version with a much better interface and rather than use a dialog box, uses a large dialog window with all the information presented in it. Looks much better although the underlying cleaning code is the same. It does have a couple of bug fixes too - With the current version, if you don't clean your database, the add-on still makes a backup. With the new version, this is no longer the case and only databases which are about to be cleaned are backed up.

It still needs BatterPudding to cast an eye over it, but hopefully we can push it out at some point this week.

Installed "Video Database Cleaner" v0.5.2 on my Windows7 system running "Kodi 17.0-Alpha2 (May 19 2016)" with the Confluence skin.
I used the zip install method after allowing "Unknown sources" within <System><Settings><System><Add-ons>.

As expected, it performed without issue on my small test database which contains just 5 movies in 2 sources, no TV recordings and no rtmp/rtsp/http/etc streaming.
When run, a popup to verify sources occured and "Yes" was selected.
Then text appeared on the bottom right of the screen to indicate the script was running.
Something then flashed on the screen (Kodi library clean?) but i was not able to read it before the text on the bottom right of the screen vanished.

In reality, with such a trivial database, it was a non test but a few comments can be made
Obviously the add-on finished it's tasks but no popup indication was provided that the 'run' had completed.
Contrast this to when "Automatically trigger clean library" is deselected before running the script, a pop up occurs to indicate the script has finished which suggests that kodi "clean library" should be (manually) run.

Also, the user doesn't know what was found and fixed within the database as a "Database Cleaner" log could not be located.
Maybe there was nothing changed within the database but if this is the case, some indication should still be provided when "prompt before actually deleting anything" is selected.

And it's not clear if the add-on actually writes it's own add-on specific log or writes to the Kodi log (i don't know what the convention is or should be but clarity is good).
If the add-on writes to the Kodi log, what happens when the add-on "Enable debug" is set but the Kodi "debug logging" is not?

When looking at the add-on information, no version information was provided while auto-update was set (i have a dislike for auto update but that may be a Microsoft windows hangup).

So to recap:

1. a popup is not always produced to indicate when the script is completed.
2. a popup is not produced to indicate nothing was changed (when add-on setting "Prompt before actually deleting anything" is set)
3. it's not clear where the add-on log is saved (kodi log or add-on specific log;where?)
4. it's not clear if the add-on debug flag is dependent on kodi debug flag before some verbose stuff is produced
5. it's not clear what add-on version is installed (unless debugging is turned on and log is checked or changelog is checked - not ideal)
6. and maybe the setting text "Prompt before actually deleting anything" should be "Prompt before making database changes" as some may think their videos are being deleted).

Now it's just to try it on my working DB though i expect no issues since i don't stream

Fair comments.

To address your points in order.

1) A pop-up is always given when the script runs, this is a notification though so as to allow the script to run with no user intervention for those users who are running it on a schedule.
2) Currently, the add-on does not request the paths of the files that are going to be deleted, it just deletes them according to the rules you have set. I could add this functionality though.
3) The add-on writes to the Kodi debug log. If Kodi's debugging is enabled it writes a concise amount to it. If debugging is also enabled in the add-on, it writes a far larger amount to the debug log.
4) See above. This is detailed in the readme file as far as I am aware, and also in the first post of this thread.
5) The version (currently 0.5.3 in the repo) is available from 'Settings -> addons -> my addons -> Program addons -> video database cleaner' in the same way that all other add-ons display their version numbers.

I have a later version that is not yet in the repo that has a much better user interface (if prompt before delete is enabled) and I should be able to add the functionality to create a 'Database Cleaner' log containing this information in Kodi's 'temp' directory to it later today (time permitting). I think I will follow the same convention as Kodi, and have a current log and an 'old' version from the previous run, if any.

Adding a notification for 'nothing to clean' should also be do-able.

Thanks for the feedback, it's good to get a perspective on how others view things !!

Some updates... 0.5.3 was put on the repo this morning. That had the new GUI and features as black_eagle mentioned in his post above, and then this afternoon swapped to 0.5.4 after some further testing.

Nothing nasty in that brief 0.5.3 release. Just spotted that the new database_cleaner.log was not being created correctly in Windows. That has been fixed in 0.5.4.


You now get a nicer summary page of what is about to be done to your database. Then after the cleanup, if you want to know what was removed, you'll find database_cleaner.log in your KODI Temp folder. In Windows this is in %APPDATA%\KODI\Cache\ This log gets replaced on each run with the previous one going into a .old version

When I run this on a Jarvis MySql install it wants to clean MyVideos103 which is a Krypton database. I run Krypton and Jarvis so the database does exist. Hopefully you can pull the currently running database from the .log and then clean the right one
Other than that it's working great, thanks for your effort.

EDIT: OK, if all else fails then RTFM I can set the database name in Settings>Advanced and it works
Still I think as default it should get the database name from the kodi.log and then use it

Hi, i uninstalled the old v0.5.2 addon and then reinstalled v0.5.4 via zip install method.
I run 14.0 (Helix) and 17.0-Alpha2 (Krypton 19May 19 2016) on Windows 7.
In both cases i use confluence

Oddly, the new install remember earlier settings from v0.5.2.
So uninstall, in true windows meaning, is not really uninstall as some remanants are left behind and subsequently used by the new install.

On the new version, clicking 'Programs' on the Kodi home ribbon yields a 'list' of program add-ons but in Helix the items also show the add-on revision to the right side while on Krypton it does not!
Likewise, right clicking the add-on within the above list and selecting (Add-on) Information shows a revision of 0.5.4 within Helix but 'not available' within Krypton.
As such, something must have changed between Helix and Krypton and such add-on revision information seems to have been lost (and not for the better as i like seeing the add-on revisions).

Also, left clicking the add-on within the above 'list' will run the add-on and thus bring up your add-on info page (allowing abort or clean) but there are some UI issues:

- there is no 'X' on your add-on info page so the abort button is the only way to kill it (for consistancy with other Kodi popups shouldn't there be an 'X' on the top right of the window?)
- your add-on info page has the highlighting offset from the button text but it also doesn't seem to follow a Kodi convention (since it looks different from other pages your add-on produces).
- in essence, when running the add-on, your information page is a pre-run summary so it's better to differentiate it from the add-on information page (the one that allows add-on configuration, update, run, uninstall, etc).
- your add-on information (pre-run summary) page does not indicate all add-on settings (by design?) but 'automatically trigger clean library' is an odd ommission.

Such UI issues are common within Kodi and Kodi add-ons, so it's nothing new here but it's not ideal.
For example, right click any addon and select 'information' and then 'configuration' has the same result as right clicking any add-on and selecting 'settings' (not good to use two terms for the same object).
Also, right click add-on and select 'settings' then 'changelog' and the popup will have the 'X' on the top left hand side rather than on the top right hand side as is the case with most other popups (and it doesn't look like other popups)
It's as if there is no Kodi add-on design guide to provide guidance to add-on authors (if there is one, it may not be not well enough enforced).

I like your pre run sumary but by allowing configuration changes on this page itself, it would mean one does not need to jump between a configuration page and a run page.
You could still have a configuration page that sets run time defaults that one could overruled at run time if desired
In other words, the add-on conditions are set each and every time it is run using a pre run page (for want of a better term).
Don't know about Kodi add-ons and what is allowed or not but it seems cleaner to set the run conditions at run time...

Other than the above minor points, it's looking good.

Thanks for your efforts

@Dilligaf: We have also had those thoughts, and found no way to get KODI to tell us the database version as an add-on. So the "hunt for it" method is used. (This technique is not unique to us) We are bending add-on rules by directly editing the database. The KODI add-on API tries to get us to use API functions for talking to the database and nothing in there fits the needs of database maintenance.

@skylarking: thanks for feedback. Many of your points are actually core KODI behaviours and not something we can change with the add-on

On the "uninstalling" note. All applications, especially on Windows, leave the settings behind as it is usually expected that you will reinstall. This is just standard KODI behaviour and not in our control.

I am not sure about your comments on behaviour in Krypton. Is this something specific to our add-on in Krypton? I'll install a few more add-ons and compare on my test box. I do remember seeing version numbers at one point, but not sure where you are talking about without repeating it on screen.

With the list of UI issues I'll let black_eagle comment on those. The GUI creating in KODI is still being expanded on. So we'll see what can be tweaked. It is pretty limited though.

You can use ESCAPE and BACKSPACE to also dismiss that summary dialog. The button text highlight being offset is known about, currently only happens on Windows.

Quote:Such UI issues are common within Kodi and Kodi add-ons, so it's nothing new here but it's not ideal.
For example, right click any addon and select 'information' and then 'configuration' has the same result as right clicking any add-on and selecting 'settings' (not good to use two terms for the same object).
Also, right click add-on and select 'settings' then 'changelog' and the popup will have the 'X' on the top left hand side rather than on the top right hand side as is the case with most other popups (and it doesn't look like other popups)

This is a good example of what is out of our control as add-on writers. You are just describing standard KODI inconsistencies.

Quote:It's as if there is no Kodi add-on design guide to provide guidance to add-on authors (if there is one, it may not be not well enough enforced).

HAHAHAHAHAHAHAHAHAHA.... sorry. That just made me laugh. You think there is a manual? There isn't exactly much out there. There are some basic guide lines in the wiki, but otherwise we are running on common sense.

We also have the extra fun that our add-on is doing maintenance directly to KODI's database which is not something really supported by the current add-on system. But as KODI is so flexible it has been possible for us to make this work.


The settings page is a standard part of addons, so the buttons and GUI follows a standard pattern. With the summary page black_eagle has hand crafted it from bits of sticky back plastic and an old washing-up bottle. There isn't as much guidance for that kind of large pop-up window. This is also why the window is currently static and not a place you can make changes. There are thoughts of making it a bit more interactive, but there are limitations as to what can be done with our add-on. We'll see.

Thanks for you feedback. Glad to see it is mainly the cosmetics causing you problems. These are going to have more changes over time.

(2016-05-30, 09:05)skylarking Wrote: [ -> ]Hi, i uninstalled the old v0.5.2 addon and then reinstalled v0.5.4 via zip install method.
I run 14.0 (Helix) and 17.0-Alpha2 (Krypton 19May 19 2016) on Windows 7.
In both cases i use confluence

I am testing with Krypton now and comparing how our addon works alongside something official like Video Node Editor.

I am testing in Estuary skin as that is the new default. I assume Confluence behaves the same in Helix\Jarvis\Krypton as we work independent of the skin.

Trying to reproduce your issues.

(2016-05-30, 09:05)skylarking Wrote: [ -> ]On the new version, clicking 'Programs' on the Kodi home ribbon yields a 'list' of program add-ons but in Helix the items also show the add-on revision to the right side while on Krypton it does not!
Likewise, right clicking the add-on within the above list and selecting (Add-on) Information shows a revision of 0.5.4 within Helix but 'not available' within Krypton.
As such, something must have changed between Helix and Krypton and such add-on revision information seems to have been lost (and not for the better as i like seeing the add-on revisions).

These are Krypton issues and not our add-on. I assume every other add-on is doing the same for you?

When testing in Krypton I am finding the version details appearing in the correct places. Though I am finding odd issues with Krypton. It has clearly been developed with an allergy to the mouse as Right Clicks are generally ignored. So one has to revert to the letter <i> to bring up information. Or <c> to bring up a context menu.

I don't want to drag our add-on off topic as these are issues that need reporting in the Krypton development threads.

Update: After spending a chunk of today with KODI Krusty I can see it is very Alpha. I completely agree with your comments here, but we need to move this discussion into a Krusty Korner, but I can't work out where that should be.

When looking at that page of Add-ons I'd expect to be able to Right Click and then find Run, Information and Settings on the menu. Yet bizarrely Run is left to the user to assume that is the "Click" action. I've also been tripping over all kinds of bugs with the Add-ons handling - and I am only trying to play with two addons.

(2016-05-30, 10:52)BatterPudding Wrote: [ -> ]Thanks for feedback. Many of your points are actually core KODI behaviours and not something we can change with the add-on

Yeah i thought as much
In part i commented so that someone within the Kodi core group could see that at least one user, me, sees UI consistancy as an issue.

(2016-05-30, 10:52)BatterPudding Wrote: [ -> ]On the "uninstalling" note. All applications, especially on Windows, leave the settings behind as it is usually expected that you will reinstall. This is just standard KODI behaviour and not in our control.

It's always been a pet hate that uninstall any application on windows leaves way too much stuff behind and some applications are worse than others.
In my view, an uninstall should at least offer the option that everything changed and/or added is reverted/removed and the system state is as if nothing was installed in the first place
Again, it seems a core Kodi issue so nothing for you guys to worry about.
Maybe core could consider raising the bar w.r.t. uninstall options some time in the future

(2016-05-30, 10:52)BatterPudding Wrote: [ -> ]I am not sure about your comments on behaviour in Krypton. Is this something specific to our add-on in Krypton? I'll install a few more add-ons and compare on my test box. I do remember seeing version numbers at one point, but not sure where you are talking about without repeating it on screen.

I did some more testing.
With Helix using Confluence skin where <Program> <Add-on>is selected, when view is set to list, the add-on revision is shown.
With Krypton using Confluence skin where <Program> <Add-on> is selected, it will show more or less information depending on the view set.
With view set to Info or Info2, the add-on author and add-on revision is shown.
With view set to BigList, List or Thumbnail, no add-on revision is shown.
I had the view set to List (which may be the default as i don't remember changing it), so it was this changed behaviour between Helix and Krypton that threw me

As i didn't have many add-ons, i thought it could have been some add-on implementation issue you guys may be interested in.
But i've since reinstalled a few add-ons and they all seem to behave in the same way w.r.t. what is shown within teh various views
So again it seems there was some change in Krypton as compared to Helix and we're back to a core Kodi issue

(2016-05-30, 10:52)BatterPudding Wrote: [ -> ]

Quote:Such UI issues are common within Kodi and Kodi add-ons, so it's nothing new here but it's not ideal.
For example, right click any addon and select 'information' and then 'configuration' has the same result as right clicking any add-on and selecting 'settings' (not good to use two terms for the same object).
Also, right click add-on and select 'settings' then 'changelog' and the popup will have the 'X' on the top left hand side rather than on the top right hand side as is the case with most other popups (and it doesn't look like other popups)

This is a good example of what is out of our control as add-on writers. You are just describing standard KODI inconsistencies.

Quote:It's as if there is no Kodi add-on design guide to provide guidance to add-on authors (if there is one, it may not be not well enough enforced).

HAHAHAHAHAHAHAHAHAHA.... sorry. That just made me laugh. You think there is a manual? There isn't exactly much out there. There are some basic guide lines in the wiki, but otherwise we are running on common sense.

Glad i made you laugh, we need more laughter in this world

(2016-05-30, 10:52)BatterPudding Wrote: [ -> ]We also have the extra fun that our add-on is doing maintenance directly to KODI's database which is not something really supported by the current add-on system. But as KODI is so flexible it has been possible for us to make this work.

Glad to see Kodi does not exert Apple style control though i'd like to see more UI consistancy across core components and the add-ons as it makes for a more polished application.

(2016-05-30, 10:52)BatterPudding Wrote: [ -> ]The settings page is a standard part of addons, so the buttons and GUI follows a standard pattern. With the summary page black_eagle has hand crafted it from bits of sticky back plastic and an old washing-up bottle. There isn't as much guidance for that kind of large pop-up window. This is also why the window is currently static and not a place you can make changes. There are thoughts of making it a bit more interactive, but there are limitations as to what can be done with our add-on. We'll see.

Thanks for you feedback. Glad to see it is mainly the cosmetics causing you problems. These are going to have more changes over time.

I think you guys have done great work in making this useful add-on.
But it should be considered a temporary fix in part as the issue of crud within the database is a core Kodi issue (which hopefully will be addressed by them in due course).

(2016-05-30, 13:25)BatterPudding Wrote: [ -> ]I am testing with Krypton now and comparing how our addon works alongside something official like Video Node Editor.

I am testing in Estuary skin as that is the new default. I assume Confluence behaves the same in Helix\Jarvis\Krypton as we work independent of the skin.

These are Krypton issues and not our add-on. I assume every other add-on is doing the same for you?

I tried Estuary but it was rather different from what i was used to with Confluence and using the mouse was a P.I.T.A hense i went back to Confluence.
And yes, other add-ons behave the same as your add-on though i've not looked at Video Node Editor which sounds interesting.

(2016-05-30, 13:25)BatterPudding Wrote: [ -> ]When testing in Krypton I am finding the version details appearing in the correct places. Though I am finding odd issues with Krypton. It has clearly been developed with an allergy to the mouse as Right Clicks are generally ignored. So one has to revert to the letter <i> to bring up information. Or <c> to bring up a context menu.

I don't want to drag our add-on off topic as these are issues that need reporting in the Krypton development threads.

Yes, there are some mouse issues with Krypton but i've only been playing with it for a few days on my PC and not in the living room.
I will need to find some time to do a shakedown and report what i find to the correct thread (though i don't think i can post on the developement forums).
As such, i'll put these core Kodi (non add-on) issues to rest and stay on topic.

(2016-05-30, 13:25)BatterPudding Wrote: [ -> ]Update: After spending a chunk of today with KODI Krusty I can see it is very Alpha. I completely agree with your comments here, but we need to move this discussion into a Krusty Korner, but I can't work out where that should be.

When looking at that page of Add-ons I'd expect to be able to Right Click and then find Run, Information and Settings on the menu. Yet bizarrely Run is left to the user to assume that is the "Click" action. I've also been tripping over all kinds of bugs with the Add-ons handling - and I am only trying to play with two addons.

Hm didn't know there was a Krusty so i'll have to look for it


So thanks again to you and black_eagle for your efforts in bringing us this great add-on which i'm sure to appreciate more as i stream youtube videos and slowly bring my database to a crawl...

Got a couple of points regarding the UI pre-clean window.

Originally I coded this with an OK/Cancel dialog box. This ensured consistency across skins as all skins supply this dialog, but due to the length of the SQL statement shown and also the fact that some users may not exactly understand what is being shown to them it was decided to re-write this to a better (?) large pop-up dialog window containing more information.

This is mainly to ensure that users are clear about what their settings are and what is going to be removed/changed.

This window is fully skinnable in the same way that Kodi's UI is skinnable but this is both an advantage and a disadvantage at the same time. The advantages are that any info can be shown in the window and it can be made to look however we want. The disadvantages are that neither of us knows anything about skinning (hence the window is pretty basic) and it is impossible to follow the style of the skin installed. It also means that the add-on itself is responsible for handling keypresses/mouse clicks rather than just getting back a result from the core, so this necessarily means more code.

Also, because of a lack of skinning knowledge, everything created in the window (apart from the top title and icon) is done programatically. On top of this there is hardly any documentation on creating and using such a window. I am pleased with what we have achieved so far, but am well aware that improvements can be made. A lack of a close 'X' is actually interesting. My dev box doesn't have mouse control turned on, so it wasn't something I was looking for. I would have thought that as those close boxes are turned on/off in core that this is a core implemented feature, but perhaps not. I will look into this to see if I can find out whether or not this is the case.

The summary window is currently undergoing some code changes in an attempt to present everything more clearly to the user.

Having discussed more interaction in this window with BP and having slept on it, I have decided that I do not think this is a good idea. Although this could be done, it is not the way add-ons are designed to work. I think that settings should remain in the settings (which after all is where they belong) and the summary screen should be just that. Users always have the option of the abort button should they wish to change anything.

Note that the current version of the addon writes a logfile (even if you abort the run) containing details of what would have happened should you have continued. I think we are supplying quite enough information for a user to make an informed decision about what will be changed.

Uninstalling the addon removes it's directory from the 'addon' directory. It does not (currently) remove the directory from 'addon-data' which is where the settings are stored. This is a core thing and not something we have control over as BatterPudding has already pointed out.

(2016-05-31, 08:10)skylarking Wrote: [ -> ]Hm didn't know there was a Krusty so i'll have to look for it

There is no Krusty... it was me taking the mickey out of the Krypton name due to the current state of the Alpha

(2016-05-31, 14:03)BatterPudding Wrote: [ -> ]

(2016-05-31, 08:10)skylarking Wrote: [ -> ]Hm didn't know there was a Krusty so i'll have to look for it

There is no Krusty... it was me taking the mickey out of the Krypton name due to the current state of the Alpha

I had a sneaky suspicion buy wasn't sure

Just don't google 'kodi krusty' as the hits were, well not what i'd expected

Finally got round to trying this out.

WOW

Over 5 minutes down to less that 5 seconds!

Gobsmacked to say the least - very impressed.

Nice one chaps!


Sent from my iPhone

(2016-05-31, 22:45)spoyser Wrote: [ -> ]Finally got round to trying this out.

WOW

Over 5 minutes down to less that 5 seconds!

Gobsmacked to say the least - very impressed.

Nice one chaps!


Sent from my iPhone

Thanks very much

I know your python skills are WAY better than ours, so we are both flattered by your praise. It also makes us more determined to make the add-on even better and we have some stuff in the pipeline at the moment. Currently bashing my head at the wall that is SQLite/SQL, but we'll get there !!