[kittmaster]

Looking forward, being on a fire tv device, flash space is premium, I think the overwrite would be a needed feature where a single 100K file is more than reasonable, stacking 100K's over time on that kind of device could be problematic without a cache clear... just thinking ahead.

I was able to add the functionality described above and have posted version1.0.6f on GitHub.  The option Retain Analysis Files needs to be disabled to delete the old files upon startup.  You'll see a message in the addon log viewer showing the number of files deleted and kept.  Nothing should be kept unless another addon writes to the output folder.  Otherwise it will be 0.


Thanks,

Jeff

F installed.... unclear on this result...thoughts?

[jbinkley60]

F installed.... unclear on this result...thoughts?

That is showing 2 files in your library that do not have a title. I am not sure how that can happen.


Thanks,

Jeff

[bossanova808]

With version F (first I have tried with the new features) it seems to be incorrectly picking up all my double episode files as duplicates....I have a bunch of these, so it's a lot of noise to wade through:



There are also a few instances where it claims to find a duplicate, but the file is the same in both listings, and there is only the one file on disk:



But then Kodi itself is also listing it twice....which is...weird.

Other than that it definitely found a few duplicate actual files on disk (leftover different formats etc, due to upgrades and what not), which was great!

[jbinkley60]

With version F (first I have tried with the new features) it seems to be incorrectly picking up all my double episode files as duplicates....I have a bunch of these, so it's a lot of noise to wade through:

I am not sure how to handle this one but I am thinking about it.  It's a bit difficult to handle all of the potential situations Kodi users leverage in their libraries.  I honestly don't know how you have / handle two library entries pointing at the same file.  For instance if you select #2 it doesn't start halfway through the file.  In Mezzmo I handle this by calling the first episode 1, changing the title to episode 1-2 or similar and then not having a second title entry for #2.
 

There are also a few instances where it claims to find a duplicate, but the file is the same in both listings, and there is only the one file on disk:

This is a similar situation except the season and episode are the same in both entries.  This is a duplicate title issue.
 

But then Kodi itself is also listing it twice....which is...weird.

Yes, that would make sense.  Both Kodi and the cleaner would be looking at the same entries.


Jeff

[bossanova808]

Re: the double episodes - no, indeed, clicking on the second just starts playing the double episode.  It's not ideal, but it works well enough.   In  your case I'd look at the duplicates you find initially, and then if you can see they have different episode numbers and titles, just don't show them, I guess?

For the other:
It's definitely odd they are in Kodi twice, I have no idea why that episode is showing up twice given there is only one file...I might try dumping the show and refreshing it from scratch.  Shouldn't be there, that's for sure.

[jbinkley60]

Re: the double episodes - no, indeed, clicking on the second just starts playing the double episode.  It's not ideal, but it works well enough.   In  your case I'd look at the duplicates you find initially, and then if you can see they have different episode numbers and titles, just don't show them, I guess?

I think the thing to do might be to ignore them when both the path and the file name are the same,  since you can't have 2 identical file names in the same path.  This would eliminate anything related to analyzing seasons, episodes and TV Shows as part of the analysis.  The goal with duplicate files is finding the same file in different paths, where you may have duplicated an episode in a different path.  I am not sure how easy this type of query would be but I can play with it.

Thoughts ?

Jeff

[jbinkley60]

Re: the double episodes - no, indeed, clicking on the second just starts playing the double episode.  It's not ideal, but it works well enough.   In  your case I'd look at the duplicates you find initially, and then if you can see they have different episode numbers and titles, just don't show them, I guess?

I tried my initial thought with leveraging the strPath variable and it got a bit complicated.  So I took your approach and modified the Enhanced Duplicate Episode Analysis option so that it adds the season and episode to the episode file check.  That seems to have worked for the combined episode issue. I've uploaded version 1.0.6g to GitHub which has this change.   I'd ask folks to run KS Cleaner with and without the enhanced setting being enabled and compare the episode analysis results.


Thanks,

Jeff

[ManilaByNight]

...... There are also a few instances where it claims to find a duplicate, but the file is the same in both listings, and there is only the one file on disk:



But then Kodi itself is also listing it twice....which is...weird.

Other than that it definitely found a few duplicate actual files on disk (leftover different formats etc, due to upgrades and what not), which was great!

Just a thought, have you tried cleaning up your library and then checking again.  I had something similar to this before but after cleaning the library, Kodi reports went back to normal.

It's under System->Media->Library->Clean Library

[bossanova808]

Thanks - yes I have indeed, but the second entry remained.  I manually removed it from the library and it seems to be gone (for now at least?), though!

[jbinkley60]

Thanks - yes I have indeed, but the second entry remained.  I manually removed it from the library and it seems to be gone (for now at least?), though!

The good news is that KS Cleaner seems to be doing its job and helping find these things.  That's the goal, to give users more visibility and control over their Kodi libraries.  In my case, leveraging my Mezzmo setup, I get a notification from the daily sync process if their ever is a duplicate in my library and it won't sync it to Kodi.  KS Cleaner allows getting close to this level of cleanliness with native Kodi libraries.


Jeff

[jbinkley60]

I have posted a new release version 1.0.7 of Kodi Selective Cleaner.  This version has a couple of key new features and a minor logging fix.   I appreciate all who provided feature suggestions and helped test this release.


Here's the complete list of highlights since 1.0.6

• Removed erroneous Kodi log message when analyzing / cleaning the seasons table
• Added Analyze All feature
• Added Duplicate Video Analysis feature
• Updated and organized addon settings tabs
• Minor error condition logging improvements

I've also added some content to the Wiki for the  Analyze / Clean All and the Duplicate Video Media Analysis features, including a table which highlights the duplicate media detection algorithms.


Thanks,

Jeff

[jbinkley60]

I have posted a new test release version 1.0.7a of Kodi Selective Cleaner.  The only change in this version is support for Kodi 22 Piers interim builds as it goes through development.  I am still gathering a list of enhancements for 1.0.8 .  I am holding on adding the individual / folder based NFO creator feature due to a lack of interest / feedback.  If that changes I can reconsider the timeline for it.

Only minimal testing has been done with 1.0.7a and Piers.  Please let me know if you find anything unusual.  I'll do more KSCleaner testing as development on Piers progresses.



Thanks,

Jeff

[rafi1]

I have a request to also find and delete duplicates with similar paths but capitalization difference. See an example in here:
https://github.com/jbinkley60/plugin.pro...r/issues/6

[jbinkley60]

I have a request to also find and delete duplicates with similar paths but capitalization difference. See an example in here:
https://github.com/jbinkley60/plugin.pro...r/issues/6

Per our discussion on Git the ask here is to add a cleaning function to the duplicate detection feature in KSCleaner.  The use cases cited are duplicates caused by path table issues:

1. Duplicate paths with different case matching when running Kodi under Windows (i.e. smb://MyPath and smb://mypath)
2. Duplicate paths due to Windows drive letter mapping in paths pointing the same place  (i.e. d:\MyVideos   and \\myserver\d\MyVideos)

KSCleaner is currently detecting duplicates across paths but the question is what is the proper way to clean these in Kodi ?  I could look to add this type of functionality in KSCleaner in a few ways:

1.  Expose the path table in the KSClleaner browsing function and let the user delete the duplicate path (and underlying media)
2.  Add selective deletion of individual items in the KSCleaner browsing function
3.  Add an option to delete duplicates from the duplicate analyzer with the user deciding which of the dupes to delete

My question to you experienced Kodi users here is, does Kodi provide an easy way to delete duplicates identified by KSCleaner or to delete a full path and the underlying media ?   Or would adding this type of selective deletion functionality be of value ? 

Sorry for asking but remember that I come from a world where the Mezzmo Kodi addon handles all Kodi database sync activities and adds/changes/deletions are all handled via Mezzmo not Kodi.so I rarely use the Kodi native tools.  I am not sure what the recommended best practices are.  My goal here wouldn't be to replicate something which already exists in Kodi natively.


Thanks,

Jeff

[rafi1]

I have a request to also find and delete duplicates with similar paths but capitalization difference. See an example in here:
https://github.com/jbinkley60/plugin.pro...r/issues/6

KSCleaner is currently detecting duplicates across paths but the question is what is the proper way to clean these in Kodi ?  I could look to add this type of functionality in KSCleaner in a few ways:

1.  Expose the path table in the KSClleaner browsing function and let the user delete the duplicate path (and underlying media)
2.  Add selective deletion of individual items in the KSCleaner browsing function
3.  Add an option to delete duplicates from the duplicate analyzer with the user deciding which of the dupes to delete

My take is to make is as simple as possible, but auto-global & smart:
1. Detect these as you do today

2a. If there is a mismatch with the defined source path - show it, and recommend to delete everything that mismatches
2b. Following this text - Request some confirmation/selection from the user (all/skip/once)

3a if there is a mismatch only in the file name(s) (so, not related to sources) - show it and recommend to remove the one that mismatches the current file/path itself.
3b. Following this - Request confirmation/selection from the user for the first one, and offer an option to do the same for all the following (or skip them all) , or proceed/skip one by one.

Maybe, start only with #2 .

AFAIK, there is no way to easily fix it in Kodi, except for removing the related source and it's entries, cleaning everything in the media->Library, re-creating the source and re-scanning everything.