WIP Media importing and library integration (UPnP, Emby, Plex, ...) - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32) +--- Forum: Kodi Application (https://forum.kodi.tv/forumdisplay.php?fid=93) +--- Thread: WIP Media importing and library integration (UPnP, Emby, Plex, ...) (/showthread.php?tid=224794) |
RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-15 I've started a new github repository with a stub media importer add-on at https://github.com/Montellese/mediaimporter.stub. There's a README.md with some basic information about how to create your own media importer add-on based on mediaimporter.stub . @jbinkley60 maybe this helps you in addition to the documentation at https://github.com/Montellese/xbmc/wiki/kodi.mediaimporter-extension-point.If anybody uses mediaimporter.stub please let me know if anything can be improved to make the process easier.
RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-15 (2021-03-09, 17:56)Supersilver Wrote: Still can't import large TV Show libraries, everytime it gets to retrieving episodes from media provider it errors out. Probably will never work with large libraries I guess? I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine. RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-15 (2021-03-10, 22:47)Croneter Wrote:(2021-03-10, 19:13)Montellese Wrote: I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part? @Croneter I took a closer look at what python-plexapi does and it first retrieves a list of 100 items (specified by mediaimporter.plex ) and then retrieves the metadata for every item separately. I did some manual testing and couldn't figure out how to retrieve multiple items from a library section with all necessary metadata (cast, media parts, ...). How do you do that as part of PKC?
RE: Media importing and library integration (UPnP, Emby, Plex, ...) - webmaher - 2021-03-15 (2021-03-15, 13:54)Montellese Wrote:(2021-03-10, 22:47)Croneter Wrote:(2021-03-10, 19:13)Montellese Wrote: I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part? With PKC tested last time with KOdi Matrix beta and import over 20k movies and 243k episodes in 40 minutes... but at the end no chance to import last maybe 300 movies... Now i have on my W10 KOdi 19 Matrix with Composite. I manually add movies and series (from Plex and HBO GO) to my library with Osmosis addon which work great and for now this is the best combination with Arctic Horizon skin... RE: Media importing and library integration (UPnP, Emby, Plex, ...) - LongMan - 2021-03-15 @webmaher, Is osmosis Matrix compatible? Is it python 3? RE: Media importing and library integration (UPnP, Emby, Plex, ...) - webmaher - 2021-03-15 Probably yes. it works....<addon id="plugin.video.osmosis" name="OSMOSIS" version="1.3.21matrix" provider-name="Stereodruid, Maven, gismo112"> RE: Media importing and library integration (UPnP, Emby, Plex, ...) - LongMan - 2021-03-15 Thanks RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Supersilver - 2021-03-16 (2021-03-15, 12:56)Montellese Wrote:Yeah to be honest I thought it was the add-on that was causing the problems but think it might be my broadband disconnecting sometimes because it's mobile broadband, but in saying that it takes hours (almost all day) to import all the 9K TV Shows around 276K episodes, if there was some way to save the progress along the way it might save a bit of the headache when it throws an error and having to try all over again(2021-03-09, 17:56)Supersilver Wrote: Still can't import large TV Show libraries, everytime it gets to retrieving episodes from media provider it errors out. Probably will never work with large libraries I guess? RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-16 (2021-03-16, 11:40)Supersilver Wrote:Do you know if those are short connection losses? This could be solved by retrying a few times if it fails.(2021-03-15, 12:56)Montellese Wrote: I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.Yeah to be honest I thought it was the add-on that was causing the problems but think it might be my broadband disconnecting sometimes because it's mobile broadband, but in saying that it takes hours (almost all day) to import all the 9K TV Shows around 276K episodes, if there was some way to save the progress along the way it might save a bit of the headache when it throws an error and having to try all over again Unfortunately performing partial imports is difficult because then there's no way to know if some of the partially imported items have changed on the server so we would have to perform a full synchronization on the next run again so you wouldn't really gain anything. Or this would require an additional caching database in the media importer add-on but that is way out of scope of the proof of concept. I'm currently working on parallelizing the work which retrieves all the metadata from the Plex server. I managed to improve the elapsed time to retrieve ~1k movies from ~180s to ~60s so it would be ~3x faster. But obviously for 276k episodes that would still take several hours. RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-17 I've released new test builds which contain
Downloads: https://github.com/Montellese/xbmc/wiki/Downloads#downloads Changelog: https://github.com/Montellese/xbmc/wiki/Downloads#kodi-v19-performance-v2 RE: Media importing and library integration (UPnP, Emby, Plex, ...) - LongMan - 2021-03-17 @Montellese, Media Sources is missing from estuary Edit: For Others You can add MediaImport Specific Skin Changes to Estuary RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-18 (2021-03-17, 18:18)LongMan Wrote: @Montellese,Which platform did you check? I just downloaded the Windowx x64 installer and checked that the necessary entry is in addons/skin.estuary/xml/Settings.xml .
RE: Media importing and library integration (UPnP, Emby, Plex, ...) - LongMan - 2021-03-18 You are right. The windows build has it. However the LibreELEC build that I ran with to this commit doesn't have it. I recall @flubshi having a similar problem when he ran his build. You can ignore it. I have a modded version of Estuary that I use that has it. Cheers LongMan RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2021-03-18 (2021-03-18, 01:23)LongMan Wrote: You are right. The windows build has it. However the LibreELEC build that I ran with to this commit doesn't have it. I recall @flubshi having a similar problem when he ran his build. You can ignore it. I have a modded version of Estuary that I use that has it.I'm assuming LibreELEC has its own patches / modifications to skin.estuary and probably also specifically to Settings.xml to integrate their own settings. Maybe there's a conflict there?
RE: Media importing and library integration (UPnP, Emby, Plex, ...) - LongMan - 2021-03-18 Quote:I'm assuming LibreELEC has its own patches / modifications to skin.estuary and probably also specifically to Settings.xml to integrate their own settings. Maybe there's a conflict there?Maybe not a conflict but I think any changes to the build has to be done with a patch. I don't quite understand how yet. |