[kadeschs]

I thought the Clean Library function was to remove items from the library that can't be found (either renamed, deleted, or on removable storage that is unplugged).  So why is KODI still erroring on shows that I've recently removed from the library or old episode folders that I've deleted off the source hard drive after running the Clean Library function?

2022-12-23 10:26:48.186 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/Big Sky (2020)/Season 2/
2022-12-23 10:26:48.205 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/Fate The Winx Saga/Season 2/
2022-12-23 10:26:48.208 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/Ghosts (US)/Season 2/
2022-12-23 10:26:48.213 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/Judy Justice/Season 1/
2022-12-23 10:26:48.334 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/UFO Witness/Season 01/
2022-12-23 10:26:48.337 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/UFO Witness/Season 02/
2022-12-23 10:26:48.343 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/Willow/Season 1/
2022-12-23 10:26:48.345 T:11336   ERROR <general>: XFILE::CDirectory::GetDirectory - Error getting smb://192.168.1.252/TVShows/You/Season 3/

[Klojum]

Just a question up front:

How are you removing items? Are you first manually wacking them off your file server, and hope for the best that Kodi does its job during the Video Clean process?
The simplest/best way is to remove the videos from the video library instantly using the Kodi context menus, and thus also subsequently confirm for the relevant files to be deleted from the Kodi GUI.
Also confirm that these folders do not suddenly have a some type of read-only status on them.

Error getting smb://192.168.1.252/TVShows/Big Sky (2020)/Season 2/
So the folder is still in the video database, but is nowhere to be found on your network folder. Seeing some other folders, this doesn't happen with earlier seasons ?

And yes, the Clean Video Database isn't perfect yet, but removing some simple paths should not be a big challenge.

[kadeschs]

How are you removing items? Are you first manually wacking them off your file server, and hope for the best that Kodi does its job during the Video Clean process?
The simplest/best way is to remove the videos from the video library instantly using the Kodi context menus, and thus also subsequently confirm for the relevant files to be deleted from the Kodi GUI.

My typical process is to use TVRename media manager to remove a show from my NAS.  That software then asks if I want to remove files from the hard drive as well.  Once I confirm this and the show folders no longer exist on the NAS, I go to KODI, navigate to the show I just deleted from the NAS, and select "Manage... Remove from library" which generally removes it from the GUI as well. However looking at the log file, it appears that some of them are still being scanned for in the background.

Also confirm that these folders do not suddenly have a some type of read-only status on them.

They do not as these folders no longer exist on the NAS and have also been removed from the library via KODI GUI.

Error getting smb://192.168.1.252/TVShows/Big Sky (2020)/Season 2/
So the folder is still in the video database, but is nowhere to be found on your network folder. Seeing some other folders, this doesn't happen with earlier seasons ?

In this particular case, the Big Sky (2020) show folder is still on the NAS, but currently only has the Season 3 folder under it.  Season 1 and Season 2 folders were manually deleted from the NAS previously.  KODI GUI is only showing Season 3 as it should.  Why it's still scanning in the background for Season 2 folder and not Season 1 folder and why the Clean Library function doesn't remove Season 2 since this folder no longer exists is the question.  Show folders like Willow, UFO Witness, and You have been completely removed from the NAS and no longer exist at all.  Those shows don't show up in the GUI either as those have been removed from library previously.

And yes, the Clean Video Database isn't perfect yet, but removing some simple paths should not be a big challenge.

What is the recommended method of deleting the paths that no longer exist?  I thought that's what the Clean Library function did.

[Klojum]

My typical process is to use TVRename media manager to remove a show from my NAS. 

So yes, basically externally removing the video flles while Kodi is non-the-wiser AFAICT. Is the TVRename tool using its own database only? I couldn't tell that from its website.

Once I confirm this and the show folders no longer exist on the NAS, I go to KODI, navigate to the show I just deleted from the NAS, and select "Manage... Remove from library" which generally removes it from the GUI as well.

I never used that tool. Does the TVRename tool have a direct connection with the Kodi database as well? Which exact command(s) does it send to remove Kodi library items? One thing I noticed is that the TVRename's website was updated last somewhere in 2020. Is that also the case for the tool itself? Because there may be some incompatibilities creeping in between it and Kodi over time.

What is the recommended method of deleting the paths that no longer exist?  I thought that's what the Clean Library function did.

You can delete single TV episodes and full TV Shows (not separate seasons afaik, never tried that) from the Kodi database using the context menus in the Kodi GUI. That will remove the library entries, and secondly the video files when confirmed (if you have the Delete option enabled in the Kodi settings). Kodi uses a couple of database triggers to remove some data from the 'main' tables in the database, not everything. I don't know what is planned/designed for v20 in this regard, but paths should be removed.

[kadeschs]

So yes, basically externally removing the video flles while Kodi is non-the-wiser AFAICT. Is the TVRename tool using its own database only? I couldn't tell that from its website.

But I'm still going to KODI GUI and removing the show as well.  TVRename doesn't have an actual database per say.  It keeps track of shows and respective episodes through a series of confirguration files.

I never used that tool. Does the TVRename tool have a direct connection with the Kodi database as well? Which exact command(s) does it send to remove Kodi library items? One thing I noticed is that the TVRename's website was updated last somewhere in 2020. Is that also the case for the tool itself? Because there may be some incompatibilities creeping in between it and Kodi over time.

TVRename is completely independent of KODI and has no connections or commands to it. It just works with show folders and files on a hard drive. It does create nfo and image (banner, posters, etc.) files for the shows that KODI can read in.

You can delete single TV episodes and full TV Shows (not separate seasons afaik, never tried that) from the Kodi database using the context menus in the Kodi GUI. That will remove the library entries, and secondly the video files when confirmed (if you have the Delete option enabled in the Kodi settings). Kodi uses a couple of database triggers to remove some data from the 'main' tables in the database, not everything. I don't know what is planned/designed for v20 in this regard, but paths should be removed.

Okay. I was hoping that Clean library function would do this rather than me having to sift through the logs and manually take care of each one myself.  If it errors on one, I would think it could ask me if I would like to remove it from the library. Removing a show from the library through KODI GUI should provide any and all triggers to remove the show from the database and not leave residuals that it's still scanning for. The fact that this doesn't seem to always be happening is why I'm posting to this forum.

[shama84]

Hello guys,

I'm in a similar situation and may put put some highlight to both parties. Up to recent my setup was composed by a device in Kodi 18.9 accessing a NAS with SMB shares. The addon "library auto updater" was used for ages and add/remove folders and items correctly every night.
I added a new device with Kodi 19.4 to this setup ans start noticing the empty TV shows on it. I thought it was an issue with the addon and created this ticket https://github.com/robweber/xbmclibrarya.../issues/70
(Check it for the details)

@Klojum 
the behavior is so different with 2 devices with different Kodi version accessing the same SMB share.
- The 18.9 works as expected
- The 19.4 leaves a blank TV show entry and exposes in kodi.log:
WARNING <general>: Process directory 'xxxxxxxxxxxxxxxxxxxxx' does not exist - skipping scan
I need to remove the TV show manually but this is a huge step back for me. It seems using the "clean library" option inside Kodi works

@kadeschs 
For the time being instead of doing manual action or relying on "library auto updater" cleaning function (no more working at this time) I added a file "advancedsettings.xml" to the Kodi 19.4 box with the below content

<advancedsettings>
    <videolibrary>
        <cleanonupdate>true</cleanonupdate>
    </videolibrary>
    <musiclibrary>
        <cleanonupdate>true</cleanonupdate>
    </musiclibrary>
</advancedsettings>

Restart Kodi and it works

[kadeschs]

I am also currently using the Library Auto Update addon.  I had been using the Library Watchdog service for many years without issue.  The Watchdog was absolutely awesome because it would update the library in real time, and I would never need to run the Update Libray command because the library always matched what I had on the NAS.  However, the Library Watchdog started acting up recently and causing issues for no apparent reason so I had to settle for the Library Auto Update addon instead. My main issue here is that even though the Clean Library runs, it seems to be missing stuff.  I have a similar advancedsettings as you posted.

  <videolibrary>
    <importwatchedstate>true</importwatchedstate> <!-- import previously exported playdate and playcount from .nfo files. Defaults to false -->
    <importresumepoint>true</importresumepoint> <!-- import previously exported resume point from .nfo files. Defaults to false -->
    <cleanonupdate>true</cleanonupdate> <!-- clean library during library update -->
    <hideallitems>true</hideallitems> <!-- removes the "*All" items from the video library -->
    <hideemptyseries>true</hideemptyseries>  <!-- hide empty series in the video library -->
  </videolibrary>

[jbinkley60]

I've begun developing a tool called Kodi Selective Cleaner to help folks with cleaning their Kodi video libraries and doing integrity checking on data (i.e. missing / mismatched data).  The Wiki page is here and specifically the database analyzer functionality.  The support forum thread is here and here's a recent use of it.  It's designed to help with cleaning your Kodi database and isn't an auto updater technology to scan drives and add new content.  I handle that all automatically in a very different setup that automates my Kodi database and rebuilds it daily.  The Kodi selective Cleaner is a derivative of that code to help folks running stand-alone Kodi systems.  It might suit some of the needs expressed here.  It is manual and not automatic but it can analyze your entire video library very quickly. 

I've paused development on the Kodi Selective Cleaner addon until I see more folks using it and testing it.  So some functionality isn't complete yet but the database cleaner / analyzer is.  There's a lot more which can be added if there is interest.


Thanks,

Jeff

[kadeschs]

I've begun developing a tool called Kodi Selective Cleaner to help folks with cleaning their Kodi video libraries and doing integrity checking on data (i.e. missing / mismatched data).

I'll have to take a look at this.  Off the top, does it support KODI video databases using remote SQL on a NAS?  My KODI boxes don't have their own independent local databases.

[jbinkley60]

I'll have to take a look at this.  Off the top, does it support KODI video databases using remote SQL on a NAS?  My KODI boxes don't have their own independent local databases.

No, it currently doesn't support MySQL or MariaDB.  Sorry.  That might be something for the future.  I use Mezzmo for sharing and this Kodi database stuff is all automatic.  If I get some time I might setup a MySQL environment to extend development into these types of environments. 


Thanks,

Jeff

[Karellen]

I added a new device with Kodi 19.4 to this setup ans start noticing the empty TV shows on it.

That is expected as a setting that was disabled by default in v18 was changed to be enabled by default in v19 and later...
https://github.com/xbmc/xbmc/pull/19137

[shama84]

I added a new device with Kodi 19.4 to this setup ans start noticing the empty TV shows on it.

That is expected as a setting that was disabled by default in v18 was changed to be enabled by default in v19 and later...
https://github.com/xbmc/xbmc/pull/19137

Thank you pointing me this . I frozen my RPI3 version to 18.9 on purpose and missed it in the subsequent release
I'll add my lines in advancedsettings.xml file for every fresh install starting version 19.x.

[andrzejls]

Interesting add-on, "Kodi Selective Cleaner". So I got v1.0.7 (latest) and did some testing. It does clean some extraneous data in database, but.... 

My setup:
- Kodi 21.1 database MyVideos131 is in MariaDB running in Debian based Openmediavault NAS and used by 5 (five) kodi seats (3 on Nvidia Shield Pro 2019 and 2 on Firestick 4K MAX).

The scenario:
- Removing from Kodi TV Show or Movie using "Long press Enter or OK Button" --> MANAGE --> REMOVE FROM LIBRARY. I will use as an example "All.Creatures.Great.and.Small.1978" TV Show.
- SETTINGS --> MEDIA --> CLEAN LIBRARY. At this point that TV Show is cleaned in the database table "tvshow"
- Deleting this show from a drive designated and assigned to TV Shows in NAS so MariaDB does not see it when "Update Library is run in Kodi.
- Bring back this show to the drive that this show was originally on.
- When "Clean Library" was run from Kodi the Database table "path" is not cleaned (the tv show path is not removed), so when I run Update Library in Kodi that show is not recognized and is not scanned into the Database.

The workaround:
At this point I have to go into MariaDB using "dbeaver-ce" on one of me Linux boxes and remove this show entries in "path" table, so then it can be scanned back into Kodi database.
My hope was that this add-on will take of this issue but unfortunately it does not. I still think that it is a good attempt, "Kodi Selective Cleaner", but it falls short of being great, IMHO.

BTW, why Kodi SETTINGS --> MEDIA --> CLEAN LIBRARY is not cleaning library including table "path". Could this function be incorporated into future versions?

[jbinkley60]

The workaround:
At this point I have to go into MariaDB using "dbeaver-ce" on one of me Linux boxes and remove this show entries in "path" table, so then it can be scanned back into Kodi database.
My hope was that this add-on will take of this issue but unfortunately it does not. I still think that it is a good attempt, "Kodi Selective Cleaner", but it falls short of being great, IMHO.

Yes, KS Cleaner is not yet designed to delete Kodi paths, whether duplicate or not.  This topic has started to be discussed in the KSCleaner support forum here but there hasn't been enough user input for this so far.  I only add features that users request and only after I know there will be sufficient use and testing.  I have a larger number of Kodi and non-Kodi cliients in my personal setup, leveraging a more elegant sharing solution without this level of complexity, so I write and maintain KSCleaner for folks here, even though I rarely use it (mainly for testing).   

If you want to gen up interest in the KSCLeaner forum and can help define the requirements and how you would like them exposed in the interface, I'll consider adding it.


Thanks,

Jeff

[kadeschs]

Is the lack of user input on this topic have to do with users assuming the cleaner is already doing these type of functions and haven’t looked closely enough to realize that it isn’t? I also share my video database on MariaDB and made a lot of assumptions initially.