Library Migration for Dummies
#1
Hi I am migrating a library from a RPi3 to a RPi4, and jumping from Kodi 17.x to 20.2. Running LibreElec in both cases.

I want to import my existing library from the data I already have, and then for new videos I add after that, get the data from TheMovieDB or TheTVDB. I have already set everything up including the sources same as the old Pi, all set to scrape info from either TheMovieDB or TheTVDB. The library on the new Pi is currently empty. 

Does this look right?
  1. On old Pi, Export old library to separate files
  2. On new Pi, for each source:
    1. Edit source to scrape 'Local Information only'
    2. When prompted to update for all files in the path, say YES
    3. After it's done, edit source and change back to TheMovieDB (or TheTVDB)
    4. When prompted to update for all files in the path, say NO
Is that it? Hopefully I'm right. I admit I'm a Mac user who's just used to buying a new computer and telling it to copy everything from my previous machine in a few clicks...so this is a complex migration for me Wink

I do have a couple of questions too:
  • I want to keep my 'watched status' and 'date added' intact...does the above method include that?
  • After this when I add new movies and tv shows and update the library, will it only source info for the new files from the online info providers, or is it going to try and scrape the whole library again?
Appreciate the help, want to get this right and not lose anything or do something stupid Smile
Reply
#2
(2023-08-28, 05:27)symean Wrote: Does this look right?
  1. On old Pi, Export old library to separate files
  2. On new Pi, for each source:
    1. Edit source to scrape 'Local Information only'
    2. When prompted to update for all files in the path, say YES
    3. After it's done, edit source and change back to TheMovieDB (or TheTVDB)
    4. When prompted to update for all files in the path, say NO
Those steps are correct.

(2023-08-28, 05:27)symean Wrote: I want to keep my 'watched status' and 'date added' intact...does the above method include that?
Yes. Nothing extra needs to be done.
Once you run the Export, open one of the nfo files. You will then be able to see the <playcount> and <lastplayed> tags.
You can easily change any of the info in the nfo files if you need. Then when you scan them into your new library, the updated info is also scanned in.

(2023-08-28, 05:27)symean Wrote: After this when I add new movies and tv shows and update the library, will it only source info for the new files from the online info providers,
Only new files.
Once moves and tv shows are in, then they are in! Nothing further happens with them.
The only way you can rescrape them is to delete them and rescrape, or use the "Refresh" button in the information page.

My only other question is about Artwork. Do you have local artwork? Are you fussed with artwork?
It is important because if you don't have local artwork, and your library is a few years old, chances are high that many of the artwork links scraped previously are dead which means you will end up with missing artwork in the library.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#3
(2023-08-28, 05:48)Karellen Wrote: ... Do you have local artwork? Are you fussed with artwork?
It is important because if you don't have local artwork, and your library is a few years old, chances are high that many of the artwork links scraped previously are dead which means you will end up with missing artwork in the library.

None of my sources currently have artwork in the directories, only the movie files themselves and sometimes subtitles. While browsing my library, all movies and tv shows have artwork. Occasionally I manually refresh the info for something if it doesn't have artwork or I've had it for a while and the artwork is generic or crappy, doesn't happen often though.

So if everything has artwork currently, going through this process should yield a library on the new Pi with artwork for everything as well right?

Cheers Smile
Reply
#4
(2023-08-28, 06:04)symean Wrote: So if everything has artwork currently, going through this process should yield a library on the new Pi with artwork for everything as well right?
If you also export Artwork when using Export to Separate Files, then yes artwork will be exported. Also export Actors when asked.
But...
Cached artwork (which is what you are exporting) is modified from the original. It is reduced in size. So a nice 1920x1080 fanart will be reduced to something like 1280x720 or something like that, I don't remember the size exactly.
If you want the original sized artwork to keep, then best that you first install Artwork Beef and run it so it downloads the original artwork First.
Not all artwork will be downloaded, because TMDB deletes a lot of artwork, so then you can run the Export to separate files and still export artwork and actors and it will fill in the missing artwork with cached versions as long as you select NO to "Overwrite existing files"
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#5
Awesome thanks. Does Kodi do that in general - downscale artwork when it's added to the library? Or is that only when it's exported?
Reply
#6
(2023-08-28, 06:38)symean Wrote: Awesome thanks. Does Kodi do that in general - downscale artwork when it's added to the library? Or is that only when it's exported?
It does that when added to the library.
It needs to do that to optimise the images for fast loading. Imagine Poster view with 40 or 50 movie posters that need to be fetched and displayed in less than a second.
Which is why it is a good idea to keep the original artwork also. Now that you are moving onto v20, install Artwork Dump and it will download the originals automatically when scraping new movies or shows.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#7
I started this before I saw your reply (I figured I can always start again) and yeah I noticed the artwork now is lower quality. I can switch inputs between the RPi3 and RPi4 and it's obvious.

So does this mean in my case the artwork was optimised/downscaled when it was first scraped on the RPi3 (over the last 5 years), then the export copies that faithfully, but then when scraping the local info on the RPi4 it's 'optimising' it again?

I've already exported the library on the RPi3. After Artwork Dump does it's thing, do I need to do another export?

Cheers again Smile
Reply
#8
(2023-08-28, 10:07)symean Wrote: but then when scraping the local info on the RPi4 it's 'optimising' it again?
Yes.

(2023-08-28, 10:07)symean Wrote: I've already exported the library on the RPi3.
You are supposed to run Artwork Beef first, on your v17 setup. If you have already exported artwork, then Artwork Beef sees that and does not download any artwork.
Artwork Dump is for v19 and later.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#9
Argh I’m confusing myself...got it. I guess I have to try and find a way to delete all the artwork files now!

I don’t suppose there’s a way to just update all artwork on the new Pi, leaving my watched status and date added intact?
Reply
#10
(2023-08-28, 11:42)symean Wrote: I don’t suppose there’s a way to just update all artwork on the new Pi, leaving my watched status and date added intact?
You could try the WatchedList addon... https://forum.kodi.tv/showthread.php?tid=164652
But I am hesitant to recommend that because
1. It requires a correctly scraped library with correct ID's. Yours being an older library may not have those. (I am assuming old because you are still using v17)
2. It's no longer maintained, and if you read the last couple of pages of the thread, some attempts have been made to fix it.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#11
Thanks I guess I’ll try and figure out how to delete all the images, then run Artwork Beef, then export again (do not overwrite), then scrape again on the new Pi.

Thanks again!
Reply
#12
Ok so I had issues with Artwork Beef, surprise surprise! I guess this is what I get for being lazy and waiting for three major versions to upgrade.

I ended up doing the following on the new RPi4, which had everything in the library, just with the crappy twice-optimised artwork:
  • Signed up to Trakt, installed the add-on and synced everything
  • Deleted all .nfo and artwork files
  • Set source contains to NONE
  • Removed everything in that path from the library
  • Set source contains to Movies, info provider The Movie DB
  • Updated everything in the path (>800 movies, took a good 4 hours I think)
  • Trakt auto-ran after the library update and gave me back my Watched Status on everything
Weirdly I expected I would have to wait for my Date Added to come back from Trakt as well, but that seemed correct before Trakt ran at the end. All 800 are in the right order, I know because our default sort order for years has been Date Added, Descending.

So this finally seems to be my workable process to get migrated to the new device with the least fuss and decent artwork. After I do the same above for my other sources I’ll start thinking about Artwork Dump, changing to local scraping only, and possibly moving to a NAS as well.

One final weird thing that freaked me out: after the huge library update, all individual movie posters were showing with a line of corrupt pixels down the right hand side of the poster. By corrupt I mean random colours that clearly didn’t belong there. Movie collection posters were ok though. I thought I would have to do this all over again and then I tried restarting the Pi...and they were gone. All posters appear perfect now. Good to go to bed in that’s not something I have to deal with tomorrow!
Reply
#13
(2023-08-29, 14:33)symean Wrote: So this finally seems to be my workable process to get migrated to the new device with the least fuss and decent artwork. After I do the same above for my other sources I’ll start thinking about Artwork Dump, changing to local scraping only, and possibly moving to a NAS as well.

When you start looking at NAS or other Kodi sharing solutions this process can become much easier, even across different Kodi versions with some solutions.  I will soon be upgrading 3 Vero 4K+ devices to new Vero V devices and upgrading Kodi (one Vero 4K+ is running Kodi 18.9), rebuilding the database etc.

My planned upgrade process will be:
 
  • Power up Vero V and configure it for my network
  • Install my sharing addon and restore addon configuration file (just because I am lazy)
  • Allow sharing addon to rebuild Kodi database, artwork, bookmarks/Watched statuses etc..
  • Begin watching again

It should take less than 2 hours or so to do all 3 (~20k library size) and I should be right where I was before the upgrades with all metadata, watched statuses, bookmarks, artwork etc.  

I think you'll be pleasantly surprised with most of the sharing solutions and how they can make this easier.  A NAS or sharing solutions won't solve all problems, especially if you have lots of addons but they should generally make it easier.  Congratulations on getting up and running again on a newer version of Kodi.


Thanks,

Jeff
Running with the Mezzmo Kodi addon.  The easier way to share your media with multiple Kodi clients.
Service.autostop , Kodi Selective Cleaner and Mezzmo Kodi addon author.
Reply
#14
Wow that does sound a lot easier. So an example for that kind of setup would be a Synology NAS holding my files with Emby installed, and Emby add-on installed in Kodi right?
Reply
#15
(2023-08-31, 05:38)symean Wrote: Wow that does sound a lot easier. So an example for that kind of setup would be a Synology NAS holding my files with Emby installed, and Emby add-on installed in Kodi right?

yes that's the idea, i do that with mine and it syncs across kodi installs - but with jellyfin instead of emby/other

and jellyfin installs on synology https://jellyfin.org/docs/general/installation/synology

mine runs in a docker on debian linux
Reply

Logout Mark Read Team Forum Stats Members Help
Library Migration for Dummies0