Login at Kodi Home

**Montellese** · (This post was last modified: 2021-03-15, 00:10 by Montellese.)

I've started a new github repository with a stub media importer add-on at https://github.com/Montellese/mediaimporter.stub. There's a README.md with some basic information about how to create your own media importer add-on based on mediaimporter.stub. @jbinkley60 maybe this helps you in addition to the documentation at https://github.com/Montellese/xbmc/wiki/...sion-point.

If anybody uses mediaimporter.stub please let me know if anything can be improved to make the process easier.

**Montellese** · 2021-03-15, 12:56

(2021-03-09, 17:56)Supersilver Wrote: Still can't import large TV Show libraries, everytime it gets to retrieving episodes from media provider it errors out. Probably will never work with large libraries I guess?

I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.

**Montellese** · 2021-03-15, 13:54

(2021-03-10, 22:47)Croneter Wrote:
(2021-03-10, 19:13)Montellese Wrote: I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.
I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part?

@Croneter I took a closer look at what python-plexapi does and it first retrieves a list of 100 items (specified by mediaimporter.plex) and then retrieves the metadata for every item separately. I did some manual testing and couldn't figure out how to retrieve multiple items from a library section with all necessary metadata (cast, media parts, ...). How do you do that as part of PKC?

webmaher · 2021-03-15, 18:21

(2021-03-15, 13:54)Montellese Wrote:
(2021-03-10, 22:47)Croneter Wrote:
(2021-03-10, 19:13)Montellese Wrote: I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.
I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part?

@Croneter I took a closer look at what python-plexapi does and it first retrieves a list of 100 items (specified by mediaimporter.plex) and then retrieves the metadata for every item separately. I did some manual testing and couldn't figure out how to retrieve multiple items from a library section with all necessary metadata (cast, media parts, ...). How do you do that as part of PKC?

With PKC tested last time with KOdi Matrix beta and import over 20k movies and 243k episodes in 40 minutes... but at the end no chance to import last maybe 300 movies...
Now i have on my W10 KOdi 19 Matrix with Composite. I manually add movies and series (from Plex and HBO GO) to my library with Osmosis addon which work great and for now this is the best combination with Arctic Horizon skin...

LongMan · 2021-03-15, 19:12

@webmaher, Is osmosis Matrix compatible? Is it python 3?

webmaher · 2021-03-15, 20:27

Probably yes. it works....<addon id="plugin.video.osmosis" name="OSMOSIS" version="1.3.21matrix" provider-name="Stereodruid, Maven, gismo112">

LongMan · 2021-03-15, 20:36

Thanks

Supersilver · 2021-03-16, 11:40

(2021-03-15, 12:56)Montellese Wrote:
(2021-03-09, 17:56)Supersilver Wrote: Still can't import large TV Show libraries, everytime it gets to retrieving episodes from media provider it errors out. Probably will never work with large libraries I guess?

I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.

Yeah to be honest I thought it was the add-on that was causing the problems but think it might be my broadband disconnecting sometimes because it's mobile broadband, but in saying that it takes hours (almost all day) to import all the 9K TV Shows around 276K episodes, if there was some way to save the progress along the way it might save a bit of the headache when it throws an error and having to try all over again

**Montellese** · 2021-03-16, 22:45

(2021-03-16, 11:40)Supersilver Wrote:
(2021-03-15, 12:56)Montellese Wrote: I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.
Yeah to be honest I thought it was the add-on that was causing the problems but think it might be my broadband disconnecting sometimes because it's mobile broadband, but in saying that it takes hours (almost all day) to import all the 9K TV Shows around 276K episodes, if there was some way to save the progress along the way it might save a bit of the headache when it throws an error and having to try all over again

Do you know if those are short connection losses? This could be solved by retrying a few times if it fails.

Unfortunately performing partial imports is difficult because then there's no way to know if some of the partially imported items have changed on the server so we would have to perform a full synchronization on the next run again so you wouldn't really gain anything. Or this would require an additional caching database in the media importer add-on but that is way out of scope of the proof of concept.

I'm currently working on parallelizing the work which retrieves all the metadata from the Plex server. I managed to improve the elapsed time to retrieve ~1k movies from ~180s to ~60s so it would be ~3x faster. But obviously for 276k episodes that would still take several hours.

**Montellese** · 2021-03-17, 14:29

I've released new test builds which contain

small performance improvements to the Python API which should benefit the performance of retrieving media items from Emby / Plex servers
major performance improvements (factor 4) when adding actors to the video database

Downloads: https://github.com/Montellese/xbmc/wiki/...#downloads
Changelog: https://github.com/Montellese/xbmc/wiki/...ormance-v2

LongMan · (This post was last modified: 2021-03-17, 18:58 by LongMan.)

@Montellese,
Media Sources is missing from estuary

Edit: For Others
You can add MediaImport Specific Skin Changes to Estuary

**Montellese** · 2021-03-18, 00:35

(2021-03-17, 18:18)LongMan Wrote: @Montellese,
Media Sources is missing from estuary

Edit: For Others
You can add MediaImport Specific Skin Changes to Estuary

Which platform did you check? I just downloaded the Windowx x64 installer and checked that the necessary entry is in addons/skin.estuary/xml/Settings.xml.

LongMan · 2021-03-18, 01:23

You are right. The windows build has it. However the LibreELEC build that I ran with to this commit doesn't have it. I recall @flubshi having a similar problem when he ran his build. You can ignore it. I have a modded version of Estuary that I use that has it.

Cheers
LongMan

**Montellese** · 2021-03-18, 07:48

(2021-03-18, 01:23)LongMan Wrote: You are right. The windows build has it. However the LibreELEC build that I ran with to this commit doesn't have it. I recall @flubshi having a similar problem when he ran his build. You can ignore it. I have a modded version of Estuary that I use that has it.

Cheers
LongMan

I'm assuming LibreELEC has its own patches / modifications to skin.estuary and probably also specifically to Settings.xml to integrate their own settings. Maybe there's a conflict there?

LongMan · 2021-03-18, 10:08

Quote:I'm assuming LibreELEC has its own patches / modifications to skin.estuary and probably also specifically to Settings.xml to integrate their own settings. Maybe there's a conflict there?

Maybe not a conflict but I think any changes to the build has to be done with a patch. I don't quite understand how yet.