Solved - Move library from 3 hard drives to one. How to deal with the paths?

Solved Move library from 3 hard drives to one. How to deal with the paths? - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: Windows (https://forum.kodi.tv/forumdisplay.php?fid=59)
+---- Thread: Solved Move library from 3 hard drives to one. How to deal with the paths? (/showthread.php?tid=378365)

Move library from 3 hard drives to one. How to deal with the paths? - JediK - 2024-07-29

Hello,

I'm going to move all my medias from 3 hard drives (so 6 sources - 3 movies/3series) to one bigger hard drive (2 sources - 1 movies/1 series)
I've read a lot of things, like path substitution (tried, didn't work), like creating a msql server to then edit the paths in the database, but I tried all the afternoon yesterday and I honestly suck at it. I didn't even succeeded to reimport my library after editing the advancesettings.xml. My kodi remained empty.

So I brought back everything as it was with my userdata backup.
Is there anything easy to just modify the paths. I really don't want to re-scrap and edit a lot of media. I've work during years to my library, i cherish it !

I've read in a post:

Quote:The easiest method is to export your library metadata to nfo files, and use Artwork Dump to download the artwork.
Then you just scan the nfo and artwork back into the library.

But what are the steps exactly (please correct below, it's just a hypothesis)?

Export my library (one file or separate files?)
Use Artwork Dump (does it save the exact artworks I have in my library for each media, i've never used it?)
Then set up the new paths of my new hard drive
Import my library
Bring back artworks with Artwork Dump
Start a scan/scrap to check if everything is ok.

Thank you beforehand for your help, it'll be greatly appreciated!

RE: Move library from 3 hard drives to one. How to deal with the paths? - PatK - 2024-07-29

(2024-07-29, 13:39)JediK Wrote: But what are the steps exactly (please correct below, it's just a hypothesis)?

1) Separate files will back up all your meta-data including the images to the specific movie folder. So yes this, and your hard work will be saved. (if you don't have movies in individual folders of the same name, it gets messy).

2) Artwork dump: https://forum.kodi.tv/showthread.php?tid=360078 yes it does use the same artwork and keeps the resolution, unlike the export as separate files. Use this.

3) New drive should have two main folders in the root one for TV Series and one for Movies.

4) copy using the PC copy command the Movies to the New Movie folder, the same for the TV series into the respective folders... You can have sub folders, just be sure to toggle 'scan recursively' so the scrapers will see sub folders.

5) Artwork should already be with the meta-data that was exported in #1) but check manually with file explorer in movie folder to see that is true.

6) You will need to either specify the new path in the edit source, or just create a new source...

7) Set your scraper settings and choice, then scan. Key here is that if Kodi finds meta-data in the media folder, it will use that information instead of jumping out to the internet. Making short work of the scrape. Note: if the meta-data is incomplete or not congruent with the needs of your settings, it will revert to online web information and your scan will be slow.

Hope that clears up some of the stuff.

RE: Move library from 3 hard drives to one. How to deal with the paths? - crawfish - 2024-07-29

(2024-07-29, 17:53)PatK Wrote:
(2024-07-29, 13:39)JediK Wrote: But what are the steps exactly (please correct below, it's just a hypothesis)?
1) Separate files will back up all your meta-data including the images to the specific movie folder. So yes this, and your hard work will be saved. (if you don't have movies in individual folders of the same name, it gets messy).

2) Artwork dump: https://forum.kodi.tv/showthread.php?tid=360078 yes it does use the same artwork and keeps the resolution, unlike the export as separate files. Use this.

When I moved Kodi to MariaDB, I worked out a procedure to delete the exported files while keeping my custom .nfo files. I didn't deal with artwork, so I guess Step 3 of my procedure would come after your step 2. I was leaving my files in the same place, so to clean up after moving the files as the OP wants to do, he'd need to search/replace to update the paths in file t3.txt before performing step 6 below (also t1.txt if he wants to do the verification described at the end).

   1. Use XYplorer to get a flat listing of all existing files with fully qualifed paths and save to t1.txt. These are the ones we want to keep.
   2. Export to individual files, not overwriting existing. Don't export any artwork.
   3. Use XYplorer to get a flat listing of all files with fully qualified paths and save to t2.txt; this includes the originals plus the exported clutter.
   4. Compare the files with WinMerge, Ctrl+D to "Toggle Diff Context Mode" and show just the differences, and save the differences to t3.txt. These are the newly added clutter files we want to delete.
   5. Scan to MariaDB.
   6. To delete the exported clutter, in PowerShell:

Code:
Get-Content 't3.txt' | ForEach-Object { Remove-Item -Force -LiteralPath $_.Trim() }

To verify all the original files are present, repeat step (1) but save to t4.txt. Then compare t1.txt to t4.txt.

BTW, in step 6, I ran into one of the infinite PowerShell subtleties. Using the -LiteralPath switch for Remove-Item is necessary so that PowerShell doesn't interpret [] characters as regexp character class brackets and thus fail to match some filenames, or worse, match wrong ones.

You can use any program that can output full paths, one per line, to create the listings. I have XYplorer, which is why I cited it. I just realized that I could've used it instead of PowerShell in Step 6. It has a feature called "Paper Folders", which basically makes a virtual folder out of a list of arbitrary paths contained in a text file, like my t3.txt. That is, I could've loaded t3.txt as a paper folder and deleted the files from the XYplorer window. It's a very cool feature.

RE: Move library from 3 hard drives to one. How to deal with the paths? - JediK - 2024-07-29

Thank you for your replies!
I'll read what's written carefully, then proceed. It may take a while, but I'll get back to you soon enough. Smile

RE: Move library from 3 hard drives to one. How to deal with the paths? - jbinkley60 - 2024-07-29

There's been a couple of sharing solutions mentioned here including MySQL and MariaDB. Some of the other sharing solutions make this task a matter of seconds or worst case a couple of minutes to accomplish in a simple 1 or 2 setup process. The sharing solution I use makes adding capacity a snap with virtual storage and changing drive letters and locations is handled via a provided tool. I have close to 500TB of storage with 35K items (music, pictures and videos) and while I am extremely good with SQL queries I've never once had to resort to them to manage media locations. Likewise I don't have to have all of my Kodi clients on the same version to share bookmarks, playcounts etc..

Good luck with your drive reallocations. If you need any help with SQL queries, I'd be happy to lend a hand.

My point here isn't to convince anyone to change from their favorite solution but just to raise awareness that there are other options which don't require hours and hours of storage and library management. I was there in the past and didn't like it. Posts like this may convince me to update the KSCleaner addon to assist with path changes so users don't need to write their own queries and similar.

Thanks,

Jeff

RE: Move library from 3 hard drives to one. How to deal with the paths? - scott967 - 2024-07-29

Reading back through the thread it doesn't sound like you are sharing a library with several Kodi installs. If this is just a single Kodi instance, no reason to jump through hoops to set up a separate database solution; built-in sqlite is fine.

The main problem with moving sources is that the Kodi "library" is about media metadata, that can be saved and restored via separate per-media nfo files, but some functionality is stored in the database but isn't a "library" function per se. This is primarily "file" or online "stream" info. Kodi treats a unique path/filename or network UNC as a separate item that can, but isn't required to, link to a library item. When you change a path, Kodi treats that as a new entry. You can re-link the library info to this new path/file, but still need to deal with the file info.

Some recent Kodi fixes/improvements have helped with restoring the file info without user intervention. Mostly getting resume/watched status and getting stream details. Unfortunately there is no fix for bookmarks which are lost.

scott s.
.

RE: Move library from 3 hard drives to one. How to deal with the paths? - JediK - 2024-07-30

Thank you jbinkley and scott for your additional information.

(2024-07-29, 17:53)PatK Wrote:
(2024-07-29, 13:39)JediK Wrote: But what are the steps exactly (please correct below, it's just a hypothesis)?
1) Separate files will back up all your meta-data including the images to the specific movie folder. So yes this, and your hard work will be saved. (if you don't have movies in individual folders of the same name, it gets messy).

I wanted to underline this. To be sure i'm (or not) in this case.

Below are my sources (media movies are directly in the relevant folders, and in folders Series/TV Shows, each serie has its own folder and subfolders for seasons):
E:\Elements 2To Suite\Video\Films
E:\Elements 2To Suite\Video\Series
E:\Elements 2To Suite\Video\TV Shows
H:\Movies
H:\Series
I:\Movies
I:\Series

You can see that over the years, I haven't perfectly standardized the folder names. Will I run into problems restoring the backup?

RE: Move library from 3 hard drives to one. How to deal with the paths? - PatK - 2024-07-30

(2024-07-30, 11:07)JediK Wrote: E:\Elements 2To Suite\Video\Serie

Unless all you 'series' folders are something special... (I suspect these are TV shows and scraped as TV?) you can amalgamate them.

The new drive should look something like this.

Root Drive letter:/folderMovies (all movies fit in this folder and called a source, simple)
Root Drive letter:/folderTV (all TV shows or series fit in the folder and called a source, again simple)

If your collection is large, it might be good to start making multiple folders. I use genre folders plus a few extras that don't fit for a total of 40 folders each as a source (spread over 16 small drives). Other users divide their collection with the alphabet or some numbering scheme. I have my TV collection setup with a few folders split into two segments based on the alphabet. Makes it easier to find things in windows maintenance mode. e.g.

These are folder suggestions and the word 'folder...' can be replace by any name and inside Kodi treated as a source with it's own scraper & settings.

Root Drive letter:/folderMovies Action/
Root Drive letter:/folderMovies Horror/
Root Drive letter:/folderMovies Crime/
etc.
Root Drive letter:/folderTV abcdefghijklmn/
Root Drive letter:/folderTV opqrstuvwxyz/
etc.

or
Root Drive letter:/folderMovies A/
Root Drive letter:/folderMovies B/
Root Drive letter:/folderMovies C/
etc
Root Drive letter:/folderTV good stuff/
Root Drive letter:/folderTV not so good stuff/
Root Drive letter:/folderTV personal stuff/
etc.

In this fashion you can create more folders as sources on more drives as needed. As long as your meta-data has been exported you can easily shuffle folders around.

Note: Kodi v20 users don't need set "importwatched" or "importresumepoint" to true anymore as that's the default
You need to make an advancedsettings.xml (wiki) if you want to save your resume points which will be handled at export if you have a pre Kodi V20 installation.

RE: Move library from 3 hard drives to one. How to deal with the paths? - crawfish - 2024-07-30

I organize mine into 4 TB chunks to facilitate backups:

Media-1/a/Movies
Media-1/a/TV
Media-1/b/Movies
Media-1/b/TV

Each subfolder "a", "b", etc is limited to 4 TB. I use Kodi's web interface or Windows search to locate things when doing maintenance.

RE: Move library from 3 hard drives to one. How to deal with the paths? - JediK - 2024-07-31

Copy is still on the way, pretty long ^^.
Thank you for the additionnal elements.

Quick one:

(2024-07-29, 17:53)PatK Wrote: 6) You will need to either specify the new path in the edit source, or just create a new source...

I'm afraid creating a new source will duplicate all entries (I mean the library will show movies from the old drives and the same movies from the new drive), no?

So if I edit the sources, I can take any "movies" source and point the path to the directory "movies" on the new hard drive, then take any "series" source and point to "series" on the new hard drive, and then delete the 5 other sources from old drives that will be useless, am i right?

Thank you beforehand for your reply.
I get closer to the end Smile

RE: Move library from 3 hard drives to one. How to deal with the paths? - PatK - 2024-08-01

(2024-07-31, 23:00)JediK Wrote: I'm afraid creating a new source will duplicate all entries

Do a system>settings>media>clean library and all the duplicates get vaporized. You can do this in all one go.. or just one folder/source at a time.

(2024-07-31, 23:00)JediK Wrote: edit the sources

The edit command will auto clean out the old listings, and offer a start for a new scan. I prefer to just copy & then clean one source at a time, bit safer. Keep in mind Kodi doesn't actually delete video files in this mode.. it's just the library listings, so breath easy; mistakes are just library listings.

I'm always moving sources as folders around to maximize drive space, so the motion is copy the folders to the new location, go into Kodi and change the source for the new location, scan... then clean.

As long as the meta-data is with the movie folder... the scraper will pick them up (along with any edits, new artwork etc.) I think there is an easier way of saying all this, but it's in the wiki. Updating or removing videos (wiki)

RE: Move library from 3 hard drives to one. How to deal with the paths? - JediK - 2024-08-01

So all went well:
- modifying two paths from old sources (one Series and one Movies) triggered new scraps to set up and new scans
- Deleting the old sources cleaned up what was needed.
- Then I made a global new scan of my library and a new cleaning.

Many thanks to all members for your ideas and PatK for your commitment and strong support. I'm very happy I could make it that well with your guidelines Smile

Enjoy your day!

Move library from 3 hard drives to one. How to deal with the paths? - PatK - 2024-08-01

Thread marked solved.

This forum uses Lukasz Tkacz MyBB addons.