Beta - Database Pre-Wash Scrub - remove old junk links and privacy issues

  Thread Rating:
  • 5 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Boyzie Offline
Senior Member
Posts: 189
Joined: Feb 2017
Reputation: 0
Post: #226
Whoops-oops, well, well, a little bit of snarkyness here quite some leading one astray (or just alternative information), I appreciate any kind of your kind information very much, thank you.

No, no, no reason for sorry, now it has become cristal clear, cannot be clearer.

Yes, yes, indeed, but no, no, don't make life too simple without snarkyness.

Many thanks again

Windows 10 Home, 64bit | Newest stable Kodi in portable mode
find quote
grubia Offline
Junior Member
Posts: 15
Joined: Aug 2009
Reputation: 0
Location: Bulgaria
Post: #227
I have troubles making the script work.
I'm using Kodi 17.3 on ubuntu and library is on MySQL.

I've tried to clean heavily bloated database but the script did not remove any rows. Than I deleted all bloat rows via sql and run just one video from gag9 for easier testing.

kodi.log
Code:
22:08:15.619 T:139627369970048  NOTICE: CleanDatabase: Starting videodatabase cleanup ..
22:08:19.980 T:139627369970048 WARNING: CreateLoader - unsupported protocol(plugin) in plugin://plugin.video.bestof9gagtv/?url=L7R3gQHc25A&mode=playVideo
22:08:20.075 T:139627369970048  NOTICE: CleanDatabase: Cleaning videodatabase done. Operation took 00:04

database-cleaner.log
Code:
Video Database Cleaner V1.0.0 - Running at Monday, 17 July 2017 22:08:13

There are 3 paths in the database that meet your criteria

The following paths were removed from the database

special://skin/extras/Intro/
/
plugin://plugin.video.bestof9gagtv/

database-cleaner.old.log
Code:
Video Database Cleaner V1.0.0 - Running at Monday, 17 July 2017 22:08:05

There are 3 paths in the database that meet your criteria

The following file paths would be removed from your database

special://skin/extras/Intro/
/
plugin://plugin.video.bestof9gagtv/

database-cleaner.log claims that database is cleaned, but the corresponding rows in the database are still there.

Can you please help me solve this problem ?
(This post was last modified: 2017-07-17 21:44 by grubia.)
find quote
Karellen Online
Super Moderator
Posts: 2,534
Joined: Apr 2017
Reputation: 45
Post: #228
Please post a full debug log. Snippets out of a log are useless.

My Signature
Links to : Forum Rules | Banned Add-Ons | Free Movies/TV Shows | Debug Log |


find quote
grubia Offline
Junior Member
Posts: 15
Joined: Aug 2009
Reputation: 0
Location: Bulgaria
Post: #229
Found the problem

Code:
11:10:07.918 T:140529035298560   DEBUG: Video Database Cleaner: Got an action 7
11:10:07.919 T:140529035298560   DEBUG: Video Database Cleaner: you pressed clean
11:10:08.001 T:140529035298560   DEBUG: Video Database Cleaner: Writing to new log file
11:10:08.002 T:140529035298560   DEBUG: Video Database Cleaner: database-cleaner.log exists - renaming to old.log
11:10:08.004 T:140529035298560   DEBUG: Video Database Cleaner: Listsize is 5
11:10:08.004 T:140529035298560   DEBUG: Video Database Cleaner: Executing SQL command - DELETE FROM path WHERE idPath IN ( SELECT idPath FROM path WHERE ((strPath NOT LIKE 'pvr://recordings/active/Default/%' AND strPath NOT LIKE 'smb://NUC/Media/HD/%' AND strPath NOT LIKE 'smb://NUC/Media/SD/%' AND strPath NOT LIKE 'smb://NUC/Media/Music Videos/%' AND strPath NOT LIKE 'smb://NUC/Media/Ser/%' AND strPath NOT LIKE 'pvr://%' AND strPath NOT LIKE 'plugin://plugin.video.youtube%' AND strPath NOT LIKE 'plugin://plugin.video.vbox7%' AND strPath NOT LIKE 'plugin://plugin.video.kolibka%' AND idFile NOT IN (SELECT idFile FROM bookmark))));
11:10:08.005 T:140529035298560   DEBUG: Video Database Cleaner: Error in db commit. Transaction rolled back
11:10:08.005 T:140529035298560   DEBUG: Video Database Cleaner: ******************************************************************************
11:10:08.005 T:140529035298560   DEBUG: Video Database Cleaner: **  SQL ERROR  **  SQL ERROR   **  SQL ERROR  **  SQL ERROR  **  SQL ERROR  **
11:10:08.005 T:140529035298560   DEBUG: Video Database Cleaner: **   1054 (42S22): Unknown column 'idFile' in 'IN/ALL/ANY subquery'
11:10:08.005 T:140529035298560   DEBUG: Video Database Cleaner: ******************************************************************************
11:10:08.006 T:140529035298560   DEBUG: Video Database Cleaner: Database connection closed
11:10:08.043 T:140530132601216   DEBUG: ------ Window Init (DialogNotification.xml) ------
11:10:08.252 T:140530132601216   DEBUG: ------ Window Deinit (/home/grubia/.kodi/addons/script.database.cleaner/resources/skins/Default/720p/script.database.cleaner-summary.xml) ------
11:10:10.005 T:140529035298560   DEBUG: JSONRPC: Incoming request: { "jsonrpc": "2.0", "method": "VideoLibrary.Clean","id": 1 }

SQL statement is not correct. There is no idFile column in path table. It is in table files.

Problem only exists when "Keep any bookmarked files" is enabled in settings. Problem is in default.py here:
Code:
if bookmarks:  # have to delete from paths table rather than files as there is a conflicting trigger on the files table
        our_select = sql.replace('DELETE FROM files', 'SELECT strPath FROM path WHERE idPath in (SELECT idPath FROM files', 1)
        our_select = our_select.replace('bookmark)', 'bookmark))',1)
        sql = sql.replace('DELETE FROM files','DELETE FROM path',1)
(This post was last modified: 2017-07-18 11:05 by grubia.)
find quote
desepticon Offline
Member
Posts: 74
Joined: Jul 2013
Reputation: 1
Post: #230
When I try to run this add-on it won't start and I get the following error. Running OS X 10.11.6 Kodi 17.3:

Quote:ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'xml.etree.ElementTree.ParseError'>
Error Contents: not well-formed (invalid token): line 4, column 4
Traceback (most recent call last):
File "/Users/matt/Library/Application Support/Kodi/addons/script.database.cleaner/default.py", line 417, in <module>
advancedsettings = ET.parse(advanced_file)
File "/Users/Shared/jenkins/workspace/OSX-64/tools/depends/xbmc-depends/macosx10.10_x86_64-target/lib/python2.7/xml/etree/ElementTree.py", line 1182, in parse
File "/Users/Shared/jenkins/workspace/OSX-64/tools/depends/xbmc-depends/macosx10.10_x86_64-target/lib/python2.7/xml/etree/ElementTree.py", line 656, in parse
File "/Users/Shared/jenkins/workspace/OSX-64/tools/depends/xbmc-depends/macosx10.10_x86_64-target/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed
File "/Users/Shared/jenkins/workspace/OSX-64/tools/depends/xbmc-depends/macosx10.10_x86_64-target/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
ParseError: not well-formed (invalid token): line 4, column 4
-->End of Python script error report<--

edit: fixed it. I had commented a line out in my advancedsetting.xml file with a "!". Apparently your add-on didn't like that.
(This post was last modified: 2017-07-19 19:17 by desepticon.)
find quote
AnthonyB Offline
Senior Member
Posts: 160
Joined: Apr 2007
Reputation: 2
Location: Australia
Post: #231
(2017-07-19 18:47)desepticon Wrote:  edit: fixed it. I had commented a line out in my advancedsetting.xml file with a "!". Apparently your add-on didn't like that.

To clarify, you had to remove commented out lines from your advancedsettings.xml ?

Kodi 17.3 on multiple Windows 10 x64 machines, shared MySQL 5.6.36 database, content stored on Windows Server 2012 R2 file server.
find quote
AnthonyB Offline
Senior Member
Posts: 160
Joined: Apr 2007
Reputation: 2
Location: Australia
Post: #232
(2017-07-21 04:09)AnthonyB Wrote:  
(2017-07-19 18:47)desepticon Wrote:  edit: fixed it. I had commented a line out in my advancedsetting.xml file with a "!". Apparently your add-on didn't like that.

To clarify, you had to remove commented out lines from your advancedsettings.xml ?

Well I'll be damned, that was it.

I also removed all commented lines from my advancedsettings.xml, i.e. all the lines that started with < ! - - (double-spaced so the forums don't interpret and convert) and the plugin worked again.

Kodi 17.3 on multiple Windows 10 x64 machines, shared MySQL 5.6.36 database, content stored on Windows Server 2012 R2 file server.
find quote
desepticon Offline
Member
Posts: 74
Joined: Jul 2013
Reputation: 1
Post: #233
(2017-07-21 04:09)AnthonyB Wrote:  
(2017-07-19 18:47)desepticon Wrote:  edit: fixed it. I had commented a line out in my advancedsetting.xml file with a "!". Apparently your add-on didn't like that.

To clarify, you had to remove commented out lines from your advancedsettings.xml ?

Yup. That was the issue.
find quote
Post Reply