[Knocks]

I recently changed video source path from smb://computername to smb://IPADDRESS (someone on OSMC forums advised that this will make network browsing faster, not sure if actually true), so now there is a duplicate of every movie in my library.  I want to remove the duplicates but keep all the movies that I have watched and deleted.  Is that possible?

[Klojum]

Option 1: Is smb://computername still active, and if so, can you rename the computer's hostname? That would make things easier.
Option 2: Are you by any chance handy enough in SQL database queries? Making a backup of the database before you started to do major changes would have been nice.

[Knocks]

Option 1: Is smb://computername still active, and if so, can you rename the computer's hostname? That would make things easier.
Option 2: Are you by any chance handy enough in SQL database queries? Making a backup of the database before you started to do major changes would have been nice.

This is an Nvidia Shield TV and I don't have a SQL backup, but how would option 1 help if you're still supposed to run "library cleanup," which removes deleted movies anyway?  I delete most of my watched movies but want to keep then in my library for recognizing actors, directors etc.

[Klojum]

and I don't have a SQL backup

Are you using the local SQLite database file, or you using a MySQL database server..?
Making a copy of a single MyVideos116d.db is so much easier. Install a decent File Manager on your Shield (e.g. X-Plore File Manager), and make a copy of the database file before you mess up more.

how would option 1 help if you're still supposed to run "library cleanup," which removes deleted movies anyway?

If you can change the hostname of your 'server', it should convince the Clean Library action, there is no such server anymore with the old name, and thus remove the old entries.
However, you say you want to keep the watched/resume statuses. Those would all be on the old movie database entries...

I delete most of my watched movies but want to keep then in my library for recognizing actors, directors etc.

Ehh.. No comprendre mucho of that. You manually delete the file from your server? The next Cleanup should wipe that database.
Normally actors, directors should be removed as well whenever a movie is removed. I guess you are still using a Kodi 'feature', that it doesn't remove all the linked info.

So, either you can rename your file server's hostname, of you'll need to do some SQL entangling.
Unplucking a SQL database is not as easy as doing a few file operations.
Also, there is no way to tell what will happen with watch statuses when you do a Library Export, since old and new files are already in there together. The new ones will probably will overwrite the old ones.

Or, you can fully start from scratch with a new/clean database...

[Knocks]

Are you using the local SQLite database file, or you using a MySQL database server..?

I am not "using" anything. This Kodi installation is the official Google Play download. It uses whatever database Kodi uses by default.

If you can change the hostname of your 'server', it should convince the Clean Library action, there is no such server anymore with the old name, and thus remove the old entries.


However, you say you want to keep the watched/resume statuses. Those would all be on the old movie database entries...

You just repeated what I wrote earlier. I realize I can press a button to clean the library, but the whole point of this exercise is not to lose unique (non-duplicate) movies.
 

you'll need to do some SQL entangling.
Unplucking a SQL database is not as easy as doing a few file operations.

Are you proposing any specific steps or just telling me how tough my life is?
 

Or, you can fully start from scratch with a new/clean database... 

lol thanks

[DarrenHill]

Basically in your usage case you cannot clean the library in the normal fashion, as it would remove the watched status too (it will remove all entries for deleted movies).

So if you want to maintain that but remove the duplicates, you will need to do it manually using an external program to access the Kodi database files. As you're not using any kind of externally hosted database (the MySQL (wiki) referenced above) it will be the default SQLlite database, whose files can be found in the userdata (wiki) folder of your install (in the Databases sub-folder, surprisingly enough).

But if doing so, ensure that Kodi isn't running and that you make backups of everything beforehand. And it will quite probably be a long and tedious task, which we may be of very minimal assistance in as we don't normally encourage people to perform brain surgery on their installs

[Knocks]

@DarrenHill

I actually kind of misspoke in my OP. I don't care so much about watched status as I simply don't want to remove deleted movies from the library. So to rephrase my original question, I want to delete all non-unique, duplicate movie entries from the library. If there is a MySQL query someone can suggest, I'll be happy to experiment and run it on the tables.

[DarrenHill]

Same answer applies - basically if you run a clean it will remove both the duplicates but also the stored data of the deleted items, as that's what it's designed to do.

So as you can't do that, you'll need to manually clean things up by removing the duplicates item by item.

The other key point is that the embedded internal database used by default with Kodi is SQLite, not MySQL (wiki). It is possible to set up Kodi to use an external MySQL (wiki) database for sharing between installations, but for the built-in one it uses SQLite which is slightly different (as it's an embedded database).

I know there are tools which can do MySQL (wiki) work, but I don't know personally if they can work on SQLite as well. As I said before it's not something we encourage people to do, for fairly obvious reasons.

What you might be able to do is export your database, set up a MySQL (wiki) external one and import your database back into that. Then you can edit it there and if you want to either keep it there or save it off again, remove your MySQL (wiki) and then reimport your saved database again. Details for that are in the wiki link. Either that or do some research into tools to edit an SQLite database.

But the key point is to ensure you back things up before you do anything in case it goes wrong. If you backup your userdata (wiki) folder (or at very least the database sub-folder of it) then you can always restore back to where you were if things do go bad.

[Knocks]

This is not an uncommon use case, so perhaps devs will consider altering the library cleanup script to add a "de-dupe only" mode.  If you do a quick Google search for 'kodi duplicate movies', there are countless pages describing this issue from lots of users and even YouTube videos about it.  This issue happens very frequently when you rename a computer, add a new device to your network, or mount a drive under a new letter.  I have had to deal with it many times over the years and have been forced to wipe my library and start over more times than I would have liked to.

[DarrenHill]

In many of those cases a simple clean will resolve most of the issues, it's just that many users don't know the functionality exists at all. Either that or their network is set up poorly (changing IP addresses or drive letters) which aren't de-duplication cases as they are now actually correctly invalid links. The only exception would be when the device has gained two access routes (multiple IP addresses or paths set up incorrectly via multiple mapped drives etc) which is a whole other scenario.

And for those when it doesn't work, exporting, clearing and reimporting the library will resolve most of the scenarios that you mention.

For example I have all my library files exported as nfo files sitting beside the media, so if I need to do a library rebuild it's a matter of 10 minutes or so, as all the data is local.

Anyway this is a niche scenario as for the vast majority of users it works fine, and whilst it certainly could be improved (both cleaning and export) at the moment there isn't the developer manpower or interest to work on something that mostly doesn't need it (or at least there are higher priority bugs to fix).

[Knocks]

@DarrenHill I understand the manpower limitations. It's always been difficult to get a feature implemented that most people don't notice or simply don't bother to report.

Having said that, unless you guys collect stats on who keeps watched movies and who deletes them, I wouldn't call this a niche case. Most users don't have unlimited storage to collect blu-ray rips/remuxes, and I would think a large portion of them just delete the files after watching, unless it's their favorite movie or something the kids will watch over and over. In fact, it could be the most common scenario. Even for someone like me, who loves hoarding hard drives, I still don't keep 90% of older movie rips. Bitrates, resolutions, and video codecs change frequently enough that it makes no sense to collect 5-year old rips.

So the issue remains. If you watch a movie and delete it, how do you preserve it in your library forever? Every time you run the cleaning script, you lose the movies you've watched and deleted. Deduping or some sort of "smart cleaning" of the library would address that, and it's really just a script modification, not a major rewrite or anything.

[DarrenHill]

I'll move this to feature requests, as that's basically what it's become.

That way if there are any of the dev team who feel inclined to look into it, at least it will be more visible.

[Karellen]

f you watch a movie and delete it, how do you preserve it in your library forever?

Replace the movie file with a stub file... https://kodi.wiki/view/Media_stubs

[Knocks]

Replace the movie file with a stub file... https://kodi.wiki/view/Media_stubs 

That's kind of neat. Is there a way to import a watched list from a third-party service like Letterboxd and create stubs automatically?

[Karellen]

You could use the Import function of Bulk Rename Utility...

With Bulk Rename Utility it is possible to rename files according to a text file. The text file should contain a list of "old name" and "new name" pairs, separated by a pipe symbol ( | ) or a comma ( , )

To use this facility, create a text file in for the correct format, for example:

Track001.mp3|Headlong.mp3
Track002.mp3|Rushes.mp3
TRACK003.mp3|AnywhereIs.mp3

or use comma separated (CSV)

Track001.mp3,Headlong.mp3
Track002.mp3,Rushes.mp3
TRACK003.mp3,AnywhereIs.mp3

One entry per line.

Next, click the Import Rename-Pairs on the Actions menu and select your file. If the file is valid, then select files to rename and you will see the "new name" column reflecting the contents of the text file.
When comparing the current filename with the text file, the application ignores the case of the filename, e.g. it will match Track001.mp3 and TRACK001.mp3. This makes it easy to rename the files.



Bulk Rename Utility will read ANSI and Unicode (UTF-16) text files.