Beta - Database Pre-Wash Scrub - remove old junk links and privacy issues

  Thread Rating:
  • 5 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
MikhailCompo Offline
Junior Member
Posts: 23
Joined: May 2014
Reputation: 0
Post: #211
This does not as you say it will.

Display when I run the add on are only two sources, they are both SMB

I run the script, no errors.

I run a Clean library in Kodi.

When i close and restart Kodi to trigger a re-scan, I still have these annoying NFS: shares listed, each one takes 20 seconds to time out.

Why has your script not removed these entries?

This is Kodi 17.1 with MySql.
(This post was last modified: 2017-04-25 21:58 by MikhailCompo.)
find quote
bigdaddy1 Offline
Junior Member
Posts: 1
Joined: May 2017
Reputation: 0
Post: #212
(2016-04-30 20:47)BatterPudding Wrote:  reserved

WIll this work on Krypton?, what is the path of the zip fle
find quote
AnthonyB Offline
Senior Member
Posts: 160
Joined: Apr 2007
Reputation: 2
Location: Australia
Post: #213
(2017-04-03 09:01)AnthonyB Wrote:  Seems to be working fine for me with Kodi 17.1 and a MySQL database.

OK, so, it was working fine, but now I'm getting the same issue as others have reported.
Seems my last successful run was 8th April 2017 based on the last database-cleaner.log I have on my system.

Video Database Cleaner Error on-screen 2-3 sec after it starts.

Kodi.log contains:
Code:
0:52:21.893 T:16068   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'xml.etree.ElementTree.ParseError'>
                                            Error Contents: not well-formed (invalid token): line 26, column 6
                                            Traceback (most recent call last):
                                              File "C:\Users\Anthony\AppData\Roaming\Kodi\addons\script.database.cleaner\default.py​", line 417, in <module>
                                                advancedsettings = ET.parse(advanced_file)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 1182, in parse
                                                tree.parse(source, parser)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 656, in parse
                                                parser.feed(data)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 1642, in feed
                                                self._raiseerror(v)
                                              File "C:\Program Files (x86)\Kodi\system\python\Lib\xml\etree\ElementTree.py", line 1506, in _raiseerror
                                                raise err
                                            ParseError: not well-formed (invalid token): line 26, column 6
                                            -->End of Python script error report<--

My system is Windows 10 x64 (MSI install), no profiles, shared MySQL DB, Kodi 17.1
The Video Database Cleaner Addon is listed as version 1.0.0 and installed from Black_Eagles Repository
Happy to run further tests or provide other info for addon devs to troubleshoot if needed.

Kodi 17.3 on multiple Windows 10 x64 machines, shared MySQL 5.6.36 database, content stored on Windows Server 2012 R2 file server.
(This post was last modified: 2017-05-15 02:58 by AnthonyB.)
find quote
korcsi Offline
Junior Member
Posts: 5
Joined: May 2017
Reputation: 0
Post: #214
kodi 16.1 (openelec on rpi and windows 10) and 17.1 (openelec) + mysql 5.7

"
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: Error in db commit. Transaction rolled back
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ******************************************************************************
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ** SQL ERROR ** SQL ERROR ** SQL ERROR ** SQL ERROR ** SQL ERROR **
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ** 1054 (42S22): Unknown column 'idFile' in 'IN/ALL/ANY subquery'
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ******************************************************************************
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: Database connection closed
"
find quote
scott967 Offline
Posting Freak
Posts: 2,770
Joined: Jul 2012
Reputation: 88
Post: #215
I thought I would try this addon (ver 1.0.0 from your repo) on my Kodi 18 win32 test install. I found a few bugs.

In window skin file script.database.cleaner-summary.xml

1. For Kodi 18 (and backwards compatible), you should replace <label>-</label> with <label></label>

Also in this file line 155 there is a conditional animation, without the condition= attribute set.

3. The debug file created database-cleaner.log isn't logging the date_time correctly:

Code:
Video Database Cleaner V1.0.0 - Running at Wednesday, 2424 May 2017 1212:23:55

The problem seems to be my settings in guisettings are:

Code:
<longdateformat>DDDD, DD MMMM YYYY</longdateformat>
<timeformat>HH:mm:ss</timeformat>

Kodi python functions xbmc.getRegions('datelong') returns
Code:
%A, %d%d %B %Y
and xbmc.getRegions('time') returns
Code:
%H%H:%M:%S

though I think this is actually a bug in how getRegions works (I asked the question in addon dev forum here: http://forum.kodi.tv/showthread.php?tid=314982)

4. I tried to do a clean with retain bookmarks and got an error:
Code:
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: Executing SQL command - DELETE FROM path WHERE idPath IN ( SELECT idPath FROM path WHERE ((strPath NOT LIKE 'F:\Test MV\%' AND strPath NOT LIKE 'pvr://%' AND idFile NOT IN (SELECT idFile FROM bookmark))));
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: Error in db commit. Transaction rolled back
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: ******************************************************************************
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: **  SQL ERROR  **  SQL ERROR   **  SQL ERROR  **  SQL ERROR  **  SQL ERROR  **
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: **   no such column: idFile
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: ******************************************************************************
12:23:55.901 T:8236   DEBUG: Video Database Cleaner: Database connection closed

looking at the SQL, I don't see how it can work. I think you have to get idFile from bookmark table and use that as a key in files table to get idPath from there.

scott s.
.
find quote
scott967 Offline
Posting Freak
Posts: 2,770
Joined: Jul 2012
Reputation: 88
Post: #216
(2017-05-23 07:41)korcsi Wrote:  kodi 16.1 (openelec on rpi and windows 10) and 17.1 (openelec) + mysql 5.7

"
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: Error in db commit. Transaction rolled back
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ******************************************************************************
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ** SQL ERROR ** SQL ERROR ** SQL ERROR ** SQL ERROR ** SQL ERROR **
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ** 1054 (42S22): Unknown column 'idFile' in 'IN/ALL/ANY subquery'
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: ******************************************************************************
07:11:37.201 T:9788 DEBUG: Video Database Cleaner: Database connection closed
"

Poster I think you have a problem similar to mine, but you need to post the SQL query generated by the addon before the lines you posted.

scott s.
.
find quote
Greig Offline
Junior Member
Posts: 3
Joined: Jul 2009
Reputation: 0
Post: #217
(2017-05-25 01:30)scott967 Wrote:  Poster I think you have a problem similar to mine, but you need to post the SQL query generated by the addon before the lines you posted.

scott s.
.

If it helps, I did so in my post here (same issue - post #203) but there's been no response. I'm not sure if the devs still read this thread, or whether the addon is supported, or whether there's simply a better place to ask! Big Grin
find quote
korcsi Offline
Junior Member
Posts: 5
Joined: May 2017
Reputation: 0
Post: #218
I dont have SQL query:

"
08:15:00.609 T:1284 DEBUG: Video Database Cleaner: script version 0.5.7-1 started
08:15:00.612 T:1180 DEBUG: ------ Window Init (DialogNotification.xml) ------
08:15:00.750 T:1180 DEBUG: ------ Window Deinit (DialogAddonInfo.xml) ------
08:15:02.610 T:1284 DEBUG: Video Database Cleaner: Found advancedsettings.xml
08:15:02.610 T:1284 DEBUG: Video Database Cleaner: looking in advancedsettings for videodatabase info
08:15:02.622 T:1284 DEBUG: Video Database Cleaner: MySQL details - 192.168.2.64, xbmc, MyVideos
08:15:02.622 T:1284 DEBUG: Video Database Cleaner: Settings for file cleaning are as follows
08:15:02.622 T:1284 DEBUG: Video Database Cleaner: keeping PVR files
08:15:02.622 T:1284 DEBUG: Video Database Cleaner: Keeping bookmarks
08:15:02.622 T:1284 DEBUG: Video Database Cleaner: Prompting before deletion
08:15:02.625 T:1284 DEBUG: Video Database Cleaner: Got local sources.xml file
08:15:02.625 T:1284 DEBUG: Video Database Cleaner: Forcing video db version to MyVideos93
08:15:02.630 T:1284 DEBUG: Video Database Cleaner: Connected to forced MySQL database MyVideos93
08:15:02.631 T:1284 DEBUG: Video Database Cleaner: Contents of sources.xml file
08:15:02.632 T:1284 DEBUG: Video Database Cleaner: filmek - smb://KODI-SERVER/Server1/torrent/filmek/
08:15:02.632 T:1284 DEBUG: Video Database Cleaner: sorozat - smb://KODI-SERVER/Server1/filmek/sorozat/
08:15:02.632 T:1284 DEBUG: Video Database Cleaner: sorozat - smb://KODI-SERVER/Server1/torrent/sorozat/
08:15:02.998 T:1180 DEBUG: ------ Window Deinit (DialogNotification.xml) ------
08:15:03.132 T:1284 DEBUG: Video Database Cleaner: Writing to new log file
08:15:03.152 T:1284 DEBUG: Video Database Cleaner: Listsize is 276
08:15:03.160 T:1284 WARNING: CSkinInfo: failed to load skin settings
08:15:03.162 T:1180 DEBUG: ------ Window Init (C:\Users\korcsi\AppData\Roaming\Kodi\addons\script.database.cleaner-cosmetic-changes\resources\skins\Default\720p\script.database.cleaner-summary.xml) ------
08:15:03.162 T:1180 INFO: Loading skin file: C:\Users\korcsi\AppData\Roaming\Kodi\addons\script.database.cleaner-cosmetic-changes\resources\skins\Default\720p\script.database.cleaner-summary.xml, load type: LOAD_ON_GUI_INIT
08:15:03.168 T:1180 ERROR: Control has invalid animation type (no condition or no type)
08:15:03.179 T:1180 DEBUG: CGUIMediaWindow::GetDirectory ()
08:15:03.179 T:1180 DEBUG: ParentPath = []
08:15:03.227 T:1284 DEBUG: Video Database Cleaner: Got an action 107
08:15:05.430 T:1180 DEBUG: Previous line repeats 92 times.
08:15:05.430 T:1180 DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
08:15:05.430 T:1284 DEBUG: Video Database Cleaner: Got an action 100
08:15:05.430 T:1284 DEBUG: Video Database Cleaner: you pressed clean
08:15:05.444 T:1284 DEBUG: Video Database Cleaner: Writing to new log file
08:15:05.444 T:1284 DEBUG: Video Database Cleaner: database-cleaner.log exists - renaming to old.log
08:15:05.453 T:1284 DEBUG: Video Database Cleaner: Listsize is 276
08:15:05.455 T:1284 DEBUG: Video Database Cleaner: Error in db commit. Transaction rolled back
08:15:05.455 T:1284 DEBUG: Video Database Cleaner: ******************************************************************************
08:15:05.455 T:1284 DEBUG: Video Database Cleaner: ** SQL ERROR ** SQL ERROR ** SQL ERROR ** SQL ERROR ** SQL ERROR **
08:15:05.455 T:1284 DEBUG: Video Database Cleaner: ** 1054 (42S22): Unknown column 'idFile' in 'IN/ALL/ANY subquery'
08:15:05.455 T:1284 DEBUG: Video Database Cleaner: ******************************************************************************
08:15:05.455 T:1284 DEBUG: Video Database Cleaner: Database connection closed
08:15:05.458 T:1180 DEBUG: ------ Window Init (DialogConfirm.xml) ------
08:15:05.458 T:1180 INFO: Loading skin file: DialogConfirm.xml, load type: KEEP_IN_MEMORY
08:15:05.691 T:1180 DEBUG: ------ Window Deinit (C:\Users\korcsi\AppData\Roaming\Kodi\addons\script.database.cleaner-cosmetic-changes\resources\skins\Default\720p\script.database.cleaner-summary.xml) ------
08:15:06.572 T:1180 DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
08:15:06.671 T:1180 DEBUG: Keyboard: scancode: 0x38, sym: 0x0134, unicode: 0x0000, modifier: 0x100
08:15:06.671 T:1180 DEBUG: CInputManager::OnKey: alt-leftalt (0x4f0d4) pressed, action is
08:15:06.671 T:1180 DEBUG: ------ Window Deinit (Pointer.xml) ------
08:15:06.721 T:1180 DEBUG: ------ Window Init (Pointer.xml) ------
08:15:06.772 T:1180 DEBUG: ------ Window Deinit (DialogConfirm.xml) ------
08:15:06.778 T:1284 DEBUG: Video Database Cleaner: Script finished
08:15:06.779 T:1284 INFO: CPythonInvoker(1, C:\Users\korcsi\AppData\Roaming\Kodi\addons\script.database.cleaner-cosmetic-changes\default.py): script successfully run
08:15:06.782 T:1284 INFO: Python script stopped
"
find quote
scott967 Offline
Posting Freak
Posts: 2,770
Joined: Jul 2012
Reputation: 88
Post: #219
(2017-05-25 01:36)Greig Wrote:  
(2017-05-25 01:30)scott967 Wrote:  Poster I think you have a problem similar to mine, but you need to post the SQL query generated by the addon before the lines you posted.

scott s.
.

If it helps, I did so in my post here (same issue - post #203) but there's been no response. I'm not sure if the devs still read this thread, or whether the addon is supported, or whether there's simply a better place to ask! Big Grin

Same thing. If I get a chance I will give the python a look, but can't make any promises.

scott s.
.
find quote
Squazz Offline
Junior Member
Posts: 10
Joined: Aug 2016
Reputation: 0
Location: Denmark
Post: #220
Another "happy customer", went from 2 hours to 30 seconds... That's a difference worth mentioning Wink
find quote
skylarking Offline
Fan
Posts: 403
Joined: Jun 2013
Reputation: 13
Post: #221
Yeah, this addon can do wonnders for some peoples library Big Grin

@black_eagle last posted here 5th Jan and last visit these forums on 13th Feb so hopefully he is simply on a great holidy somewhere and all is well Smile
Wonder if he still intends to and able to support this addon Huh

I'm a XBMC novice :)
(This post was last modified: 2017-06-11 11:04 by skylarking.)
find quote
Boyzie Offline
Senior Member
Posts: 170
Joined: Feb 2017
Reputation: 0
Post: #222
Is the database cleaned (an option for it) without losing the movies / series on external drives not plugged in?

Windows 10 Home, 64bit | Newest stable Kodi in portable mode
find quote
skylarking Offline
Fan
Posts: 403
Joined: Jun 2013
Reputation: 13
Post: #223
(2017-07-04 12:45)Boyzie Wrote:  Is the database cleaned (an option for it) without losing the movies / series on external drives not plugged in?
Read the first post.

Look specifically for the following:

How Does It Work
Our addon aims to remove rtmp://, rtmpe://, http:// and plugin references from the files table. If it is run using the default settings, it builds an SQL query that excludes all the path sources defined in your sources.xml file and deletes everything else.


and:

Database Backups
Even time a scrub is performed, we make a backup of the SQLite database. These are stored in a backups folder underneath the KODI database folder. These backups can be disabled in the settings but we STRONGLY suggest you leave the backup enabled.


So by reading the first post, you should have understood that all items (movies, tv series) associated with your sources (whether connected or not) should be preserved. And in the unlikely event that they are not, you could recover by restoring the database backup.
Read the first post again as i haven't mentioned exclusions Tongue

I'm not meaning to be snarky but one should always read the first few posts of an add-on thread as a minimum before asking something that is easily found.
And the complete DB prewash scrub thread is only 15 pages long (to date) so it's not out of the question to scan the whole thread in one sitting (unlike chromebox thread at 509 pages).

I'm a XBMC novice :)
(This post was last modified: 2017-07-07 09:47 by skylarking.)
find quote
Boyzie Offline
Senior Member
Posts: 170
Joined: Feb 2017
Reputation: 0
Post: #224
Quote:So by reading the first post, you should have understood that all items (movies, tv series) associated with your sources (whether connected or not) should be preserved.
Yes, unbelevably I had read the first post (at least a part of it) before and just the lines above, but I hadn't understand that it means medias on external drives not plugged in are preserved.

Quote:I'm not meaning to be snarky but one should always read the first few posts of an add-on thread as a minimum before asking something that is easily found.
Yes, yes, of course that's excatly what I mean as well. No, no, a little bit of snarkyness at the right place cannot harm.

Quote:And the complete DB prewash scrub thread is only 15 pages long (to date) so it's not out of the question to scan the whole thread in one sitting (unlike chromebox thread at 509 pages).
Yes, indeed, absolutely. But, may by, since I hadn't understand the first lines of the first post and generally do not have any idea / understanding of anything I somehow had given up or so.

So I very much appreciate your understandable statements in this post. Thank you very much.

Windows 10 Home, 64bit | Newest stable Kodi in portable mode
find quote
skylarking Offline
Fan
Posts: 403
Joined: Jun 2013
Reputation: 13
Post: #225
Oops, just a heads up as i reasized i may have led you astray - at least a little bit Blush

If you have the default settings for Video Database Cleaner, then after running the add-on (which wont delete your movies defined within sources.xml) the add-on calls (or triggers) a database clean (which removes movies on unavailable/unconnected sources).

Hopefully you haven't run the add-on as yet so you at least have an option to go to the add-on settings and untick the "Automatically trigger clean library".

Now when you run prewash scrub, you wont wipe your library if some sources are not plugged in Tongue

If you have run the add-on, well at least you have a backup and can recover.

Sorry for the less than accurate info Blush

Just goes to show, life is simple without snarkiness Nod

I'm a XBMC novice :)
find quote
Post Reply