•   
  • 1
  • 69
  • 70
  • 71(current)
  • 72
  • 73
  • 84
  •   
WIP Media importing and library integration (UPnP, Emby, Plex, ...)
I've started a new github repository with a stub media importer add-on at https://github.com/Montellese/mediaimporter.stub. There's a README.md with some basic information about how to create your own media importer add-on based on mediaimporter.stub. @jbinkley60 maybe this helps you in addition to the documentation at https://github.com/Montellese/xbmc/wiki/...sion-point.

If anybody uses mediaimporter.stub please let me know if anything can be improved to make the process easier.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
(2021-03-09, 17:56)Supersilver Wrote: Still can't import large TV Show libraries, everytime it gets to retrieving episodes from media provider it errors out. Probably will never work with large libraries I guess?

I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
(2021-03-10, 22:47)Croneter Wrote:
(2021-03-10, 19:13)Montellese Wrote: I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.
I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part?

@Croneter I took a closer look at what python-plexapi does and it first retrieves a list of 100 items (specified by mediaimporter.plex) and then retrieves the metadata for every item separately. I did some manual testing and couldn't figure out how to retrieve multiple items from a library section with all necessary metadata (cast, media parts, ...). How do you do that as part of PKC?
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
(2021-03-15, 13:54)Montellese Wrote:
(2021-03-10, 22:47)Croneter Wrote:
(2021-03-10, 19:13)Montellese Wrote: I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.
I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part?

@Croneter I took a closer look at what python-plexapi does and it first retrieves a list of 100 items (specified by mediaimporter.plex) and then retrieves the metadata for every item separately. I did some manual testing and couldn't figure out how to retrieve multiple items from a library section with all necessary metadata (cast, media parts, ...). How do you do that as part of PKC?

With PKC tested last time with KOdi Matrix beta and import over 20k movies and 243k episodes in 40 minutes... but at the end no chance to import last maybe 300 movies... 
Now i have on my W10 KOdi 19 Matrix with Composite. I manually add movies and series (from Plex and HBO GO) to my library with Osmosis addon which work great and for now this is the best combination with Arctic Horizon skin...

Image
HTPC W1064bit, Kodi 18
Geforce 1050Ti 4GB
Sony Android TV XE 9005
Reply
@webmaher, Is osmosis Matrix compatible? Is it python 3?
Reply
Probably yes. it works....<addon id="plugin.video.osmosis" name="OSMOSIS" version="1.3.21matrix" provider-name="Stereodruid, Maven, gismo112">
HTPC W1064bit, Kodi 18
Geforce 1050Ti 4GB
Sony Android TV XE 9005
Reply
Thanks
Reply
(2021-03-15, 12:56)Montellese Wrote:
(2021-03-09, 17:56)Supersilver Wrote: Still can't import large TV Show libraries, everytime it gets to retrieving episodes from media provider it errors out. Probably will never work with large libraries I guess?

I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.
Yeah to be honest I thought it was the add-on that was causing the problems but think it might be my broadband disconnecting sometimes because it's mobile broadband, but in saying that it takes hours (almost all day) to import all the 9K TV Shows around 276K episodes, if there was some way to save the progress along the way it might save a bit of the headache when it throws an error and having to try all over again
Reply
(2021-03-16, 11:40)Supersilver Wrote:
(2021-03-15, 12:56)Montellese Wrote: I gave importing a larger tvshow library (~29k episodes) from Plex a try and while it took several hours it worked perfectly fine.
Yeah to be honest I thought it was the add-on that was causing the problems but think it might be my broadband disconnecting sometimes because it's mobile broadband, but in saying that it takes hours (almost all day) to import all the 9K TV Shows around 276K episodes, if there was some way to save the progress along the way it might save a bit of the headache when it throws an error and having to try all over again
Do you know if those are short connection losses? This could be solved by retrying a few times if it fails.

Unfortunately performing partial imports is difficult because then there's no way to know if some of the partially imported items have changed on the server so we would have to perform a full synchronization on the next run again so you wouldn't really gain anything. Or this would require an additional caching database in the media importer add-on but that is way out of scope of the proof of concept.

I'm currently working on parallelizing the work which retrieves all the metadata from the Plex server. I managed to improve the elapsed time to retrieve ~1k movies from ~180s to ~60s so it would be ~3x faster. But obviously for 276k episodes that would still take several hours.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
I've released new test builds which contain
  • small performance improvements to the Python API which should benefit the performance of retrieving media items from Emby / Plex servers
  • major performance improvements (factor 4) when adding actors to the video database

Downloads: https://github.com/Montellese/xbmc/wiki/...#downloads
Changelog: https://github.com/Montellese/xbmc/wiki/...ormance-v2
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
@Montellese,
Media Sources is missing from estuary


Edit: For Others
You can add MediaImport Specific Skin Changes to Estuary
Reply
(2021-03-17, 18:18)LongMan Wrote: @Montellese,
Media Sources is missing from estuary


Edit: For Others
You can add MediaImport Specific Skin Changes to Estuary
Which platform did you check? I just downloaded the Windowx x64 installer and checked that the necessary entry is in addons/skin.estuary/xml/Settings.xml.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
You are right. The windows build has it. However the LibreELEC build that I ran with to this commit doesn't have it. I recall @flubshi having a similar problem when he ran his build. You can ignore it. I have a modded version of Estuary that I use that has it.

Cheers
LongMan
Reply
(2021-03-18, 01:23)LongMan Wrote: You are right. The windows build has it. However the LibreELEC build that I ran with to this commit doesn't have it. I recall @flubshi having a similar problem when he ran his build. You can ignore it. I have a modded version of Estuary that I use that has it.

Cheers
LongMan
I'm assuming LibreELEC has its own patches / modifications to skin.estuary and probably also specifically to Settings.xml to integrate their own settings. Maybe there's a conflict there?
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
Quote:I'm assuming LibreELEC has its own patches / modifications to skin.estuary and probably also specifically to Settings.xml to integrate their own settings. Maybe there's a conflict there?
Maybe not a conflict but I think any changes to the build has to be done with a patch. I don't quite understand how yet.
Reply
  •   
  • 1
  • 69
  • 70
  • 71(current)
  • 72
  • 73
  • 84
  •   



Logout Mark Read Team Forum Stats Members Help
Media importing and library integration (UPnP, Emby, Plex, ...)10
This forum uses Lukasz Tkacz MyBB addons.