Kodi Community Forum
WIP Media importing and library integration (UPnP, Emby, Plex, ...) - Printable Version

Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Kodi Application (https://forum.kodi.tv/forumdisplay.php?fid=93)
--- Thread: WIP Media importing and library integration (UPnP, Emby, Plex, ...) (/showthread.php?tid=224794)



RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Niene - 2020-01-08

(2020-01-07, 23:58)Montellese Wrote:
(2020-01-06, 22:48)Niene Wrote: Hey together,
i am searching for an addon to add upnp source to the kodi library on my firetv.

I read your post of 2015 and i am glad to read, that you try to finish it.

My dlna server is a gerbera (mediatomb fork) on a centos 8 box.

I really want to try your testbuild, but the link is dead.

My question, is this build an addon or a whole kodi version?

greetings from germany
Thanks for your interest.

In general for media import to be supported you need a test build from this thread. Then it depends on where you want to import from:
  • UPnP is supported out of the box. But only UPnP servers which support searching for specific media types (movies, tvshows, ...) are supported.
  • Emby / Jellyfin requires the mediaimporter.emby add-on
  • Plex requires the mediaimporter.plex add-on

I will post new test builds once I've fixed an annoying bug.

What platform would you require a test build for? 

Hi,
that sounds great,
my kodi runs on a firetv  stick, so i think it is armv7 32 bit. I have two, one 4k the other normal. but i think both was armv7.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Norin_Radd - 2020-01-09

I noticed one thing, i have 2 movie posters and a some series posters that i have corrected and showup on jellyfin but are not imported to kodi, even when i restart kodi and retrieves new added media it doesn´t import those posters.

I even tried to synchronize again and the posters dont show up, for example for a series it doesnt seem to get the right metadata when i press info theres no extended information about the series.

should i try to remove from library and scan again? i bet that probably will work. 

here is the log https://paste.kodi.tv/gaqegijasa.kodi

I have a feature request for you, when importing libraries from jellyfin, emby or plex, it would be nice to create a smartplaylist from those libraries, for example i have this sets of libraries films: Movies, Animation Movies, Documentaries, Short Movies, Animation Short Movies, series: Series, Animation Series, Documentaries Series.

Thats how i have it sorted and organized on jellyfin, so it would be nice on import to create that smartplaylists because they are handy to customize skins.

edit: as i suspected, after a few remove and sync again it showed properly the movie posters and info.
        about the playlists i see they are tagged, so i can create the playlists from the tags, so maybe the playlists are not needed.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-09

@Norin_Radd thanks for the feedback. Right now artwork is not properly synchronized. On initial synchronization of a media import all artwork is imported but any later synchronization does not re-synchronize artwork. The reason for this is an issue with Kodi's UPnP implementation which caused complete re-synchronizations when using media importing over UPnP. But I will remove this restriction in a future test build so that it works for Emby / Jellyfin and Plex.

For Emby / Jellyfin each imorted item is tagged with the library view from Emby / Jellyfin so you should be able to create those smartplaylists yourself.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-09

Finally there are new test builds. These test builds are based on Kodi Matrix pre-alpha with Python 3 support: Please install the media import repository and install / update the media importer add-ons from there. This way you should get all necessary dependencies.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Milhouse - 2020-01-10

I've uploaded RPi2 and Generic LibreELEC Kodi 19 master (same as #0109 test build) with the addition of https://github.com/xbmc/xbmc/compare/master...Montellese:media_import_rebase_20200101 (perma link).

I quickly tested the RPi2 build - it started Kodi but failed to upgrade the database correctly from MyVideos116:


2019-11-29 16:36:46.246 T:1287 NOTICE: MYSQL: Connected to version 5.5.21
2019-11-29 16:36:46.246 T:1287 WARNING: MYSQL: Your database server version 5.5.21 is very old and might not be supported in future Kodi versions. Please consider upgrading to MySQL 5.7 or MariaDB 10.2.
2019-11-29 16:36:46.271 T:1287 NOTICE: Running database version MyMusic73
2019-11-29 16:36:46.285 T:1287 ERROR: Unable to open database: MyVideos118 [1049](Unknown database 'MyVideos118')
2019-11-29 16:36:46.297 T:1287 ERROR: Unable to open database: MyVideos117 [1049](Unknown database 'MyVideos117')
2019-11-29 16:36:46.326 T:1287 NOTICE: Old database found - updating from version 116 to 118
2019-11-29 16:36:50.317 T:1275 DEBUG: CActiveAE - display reset event
2019-11-29 16:36:50.394 T:1276 INFO: CActiveAESink::OpenSink - initialize sink
2019-11-29 16:36:50.394 T:1276 DEBUG: CActiveAESink::OpenSink - trying to open device PI:HDMI
2019-11-29 16:36:50.395 T:1276 DEBUG: CAESinkPi:Initialize Format:15 Channels:2 Samplerate:44100 framesize:8 bufsize:8816 bytes/s=352800.00 dest=PI:HDMI
2019-11-29 16:36:50.395 T:1276 DEBUG: CAESinkPi:SetAudioProps hdmi_stream_channels 0 hdmi_channel_map 00000008
2019-11-29 16:36:50.397 T:1276 DEBUG: COMXCoreComponent::Initialize OMX.broadcom.audio_render input port 100 output port 100 m_handle 0x6ab03160
2019-11-29 16:36:50.400 T:1276 DEBUG: COMXCoreComponent::AllocInputBuffers component(OMX.broadcom.audio_render) - port(100), nBufferCountMin(1), nBufferCountActual(2), nBufferSize(8816), nBufferAlignment(16)
2019-11-29 16:36:50.401 T:1276 DEBUG: CActiveAESink::OpenSink - SinkPi Initialized:
2019-11-29 16:36:50.401 T:1276 DEBUG: Output Device : HDMI
2019-11-29 16:36:50.401 T:1276 DEBUG: Sample Rate : 44100
2019-11-29 16:36:50.401 T:1276 DEBUG: Sample Format : AE_FMT_FLOAT
2019-11-29 16:36:50.401 T:1276 DEBUG: Channel Count : 2
2019-11-29 16:36:50.401 T:1276 DEBUG: Channel Layout: FL, FR
2019-11-29 16:36:50.401 T:1276 DEBUG: Frames : 1102
2019-11-29 16:36:50.401 T:1276 DEBUG: Frame Size : 8
2019-11-29 16:36:50.404 T:1275 DEBUG: CActiveAE::ClearDiscardedBuffers - buffer pool deleted
2019-11-29 16:36:50.404 T:1276 DEBUG: CAESinkPiBig Grineinitialize
2019-11-29 16:36:50.404 T:1276 DEBUG: CAESinkPi:SetAudioProps hdmi_stream_channels 0 hdmi_channel_map 00000000
2019-11-29 16:36:50.407 T:1276 DEBUG: COMXCoreComponent:Big Grineinitialize : OMX.broadcom.audio_render handle 0x6ab03160
2019-11-29 16:37:08.897 T:1287 NOTICE: Attempting to update the database MyVideos118 from version 116 to 118
2019-11-29 16:37:08.898 T:1287 DEBUG: Mysql Start transaction
2019-11-29 16:37:13.040 T:1287 DEBUG: Mysql execute: ALTER TABLE musicvideo ADD c15 text
2019-11-29 16:37:13.041 T:1287 ERROR: SQL: [MyVideos118] Undefined MySQL error: Code (1060)
Query: ALTER TABLE musicvideo ADD c15 text
2019-11-29 16:37:13.042 T:1287 ERROR: Exception updating database MyVideos118 from version 116 to 118
2019-11-29 16:37:13.042 T:1287 ERROR: Error updating database MyVideos118 from version 116 to 118
2019-11-29 16:37:13.043 T:1287 DEBUG: Mysql rollback transaction
2019-11-29 16:37:13.050 T:1287 ERROR: Unable to open database: MyVideos115 [1049](Unknown database 'MyVideos115')
2019-11-29 16:37:13.147 T:1287 NOTICE: Old database found - updating from version 114 to 118
2019-11-29 16:37:13.151 T:1287 ERROR: SQL: Can't create database for copy: 'MyVideos114' (1007)
2019-11-29 16:37:13.151 T:1287 ERROR: Unable to copy old database MyVideos114 to new version MyVideos118
2019-11-29 16:37:13.151 T:1287 WARNING: CMediaImportManager: failed to register and initialize given import repository
2019-11-29 16:37:13.155 T:1287 NOTICE: Running database version TV36
2019-11-29 16:37:13.160 T:1287 NOTICE: Running database version Epg12
2019-11-29 16:37:13.161 T:1287 DEBUG: Initialize, updating databases... DONE

Error 1060 is a "duplicate column" error. MyVideos116.musicvideo already has a c15 column, and the upgrade to MyVideos117 (and then 118) is adding this already existing column. Maybe adding an existing column fails silently/does nothing with SQLite, and this is just a MySQL issue? Anyway, full debug log: http://ix.io/26Za

Download #0109w: RPi2, Generic

Obviously, I offer only very limited support for these upnp builds.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Norin_Radd - 2020-01-10

tried with new build and updated addon

movies play, but with this build i no longer can use embuary skin nor skinshortcuts or any subtitles addons work.

new log: https://paste.kodi.tv/gufaquxuno.kodi


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-10

@Milhouse thanks for your support and for the report. You're right that update action is not needed. I also noticed that the column already exists but forgot to remove the update action again. I've removed it and force pushed the branch.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-10

(2020-01-10, 04:37)Norin_Radd Wrote: tried with new build and updated addon

movies play, but with this build i no longer can use embuary skin nor skinshortcuts or any subtitles addons work.

new log: https://paste.kodi.tv/gufaquxuno.kodi
That's because the new test builds are based on a recent Kodi master where Kodi has migrated from Python 2 to Python 3 for add-ons (since Python 2 will be EOL soon). The problem now is that a lot of add-ons are not yet (fully) Python 3 compatible so they won't work until their authors update them.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Niene - 2020-01-10

(2020-01-10, 08:51)Montellese Wrote:
(2020-01-10, 04:37)Norin_Radd Wrote: tried with new build and updated addon

movies play, but with this build i no longer can use embuary skin nor skinshortcuts or any subtitles addons work.

new log: https://paste.kodi.tv/gufaquxuno.kodi
That's because the new test builds are based on a recent Kodi master where Kodi has migrated from Python 2 to Python 3 for add-ons (since Python 2 will be EOL soon). The problem now is that a lot of add-ons are not yet (fully) Python 3 compatible so they won't work until their authors update them. 
Hello together,
thx for the build. I want to give linux x86 or 64 a chance. So i register to github and want to download and compile it for linux x64.
I am not a github pro, i found the wiki, howto build kodi for linux, but how can i get your fork?

In the meantime i install all the dependencies to compile kodi on fedora 30


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Norin_Radd - 2020-01-10

(2020-01-10, 08:51)Montellese Wrote:
(2020-01-10, 04:37)Norin_Radd Wrote: tried with new build and updated addon

movies play, but with this build i no longer can use embuary skin nor skinshortcuts or any subtitles addons work.

new log: https://paste.kodi.tv/gufaquxuno.kodi
That's because the new test builds are based on a recent Kodi master where Kodi has migrated from Python 2 to Python 3 for add-ons (since Python 2 will be EOL soon). The problem now is that a lot of add-ons are not yet (fully) Python 3 compatible so they won't work until their authors update them.  

i thought so, i asked around in the forum already, but now iam going to ask again. Wink

and what about external subtitles from jellyfin any news? Smile


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Norin_Radd - 2020-01-11

weird

i cannot start kodi

yesterday was working fine

https://paste.kodi.tv/evojovariw.kodi

edit: uninstalled kodi and installed latest matrix version and importer repo and emby addon and it doesn´t sync. 

tested authentication ok but when i press access nothing happens.

https://paste.kodi.tv/rudogezeli.kodi


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Milhouse - 2020-01-11

(2020-01-10, 08:50)Montellese Wrote: @Milhouse thanks for your support and for the report. You're right that update action is not needed. I also noticed that the column already exists but forgot to remove the update action again. I've removed it and force pushed the branch.

@Montellese the latest version of the PR is better, but still fails during the database upgrade: http://ix.io/275k

This time it's:

2020-01-11 03:54:56.364 T:775 DEBUG: Mysql execute: CREATE INDEX ix_actor_link_1 ON actor_link (media_type(20))
2020-01-11 03:54:57.743 T:775 DEBUG: Mysql execute: CREATE UNIQUE INDEX ix_actor_link_2 ON actor_link (actor_id, media_id, media_type, role)
2020-01-11 03:54:57.744 T:775 ERROR: SQL: [MyVideos118] Undefined MySQL error: Code (1170)
Query: CREATE UNIQUE INDEX ix_actor_link_2 ON actor_link (actor_id, media_id, media_type, role)
2020-01-11 03:54:57.745 T:775 ERROR: Exception updating database MyVideos118 from version 116 to 118
2020-01-11 03:54:57.745 T:775 ERROR: Error updating database MyVideos118 from version 116 to 118
2020-01-11 03:54:57.746 T:775 DEBUG: Mysql rollback transaction

The full error for this query appears to be:

Error Code: 1170. BLOB/TEXT column 'media_type' used in key specification without a key length

Specifying a key length on media_type then results in a similar failure for role.

The following appears to work:

CREATE UNIQUE INDEX ix_actor_link_2 ON actor_link (actor_id, media_id, media_type(255), role(255))
however 255 is just a random maximum number - something lower might make more sense, or use a non-text datatype for both columns (I've no idea how these columns are used etc.).


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-11

(2020-01-10, 23:14)Norin_Radd Wrote: i thought so, i asked around in the forum already, but now iam going to ask again. Wink

and what about external subtitles from jellyfin any news? Smile 
@LongMan has created a feature request at https://github.com/Montellese/mediaimporter.emby/issues/4. But for me it's very low priority because I don't use subtitles at all and the workaround involves having to download the subtitles manually, rename them and feed them back to Kodi which sounds ugly...


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-11

(2020-01-11, 04:24)Norin_Radd Wrote: weird

i cannot start kodi

yesterday was working fine

https://paste.kodi.tv/evojovariw.kodi
It looks like there was a problem with loading the vfs.sftp add-on.

(2020-01-11, 04:24)Norin_Radd Wrote: weird
edit: uninstalled kodi and installed latest matrix version and importer repo and emby addon and it doesn´t sync. 

tested authentication ok but when i press access nothing happens.

https://paste.kodi.tv/rudogezeli.kodi
Unfortunately your logs are pretty much empty because they are not debug logs.


RE: Media importing and library integration (UPnP, Emby, Plex, ...) - Montellese - 2020-01-11

(2020-01-11, 06:03)Milhouse Wrote:
(2020-01-10, 08:50)Montellese Wrote: @Milhouse thanks for your support and for the report. You're right that update action is not needed. I also noticed that the column already exists but forgot to remove the update action again. I've removed it and force pushed the branch.

@Montellese the latest version of the PR is better, but still fails during the database upgrade: http://ix.io/275k

This time it's:

2020-01-11 03:54:56.364 T:775 DEBUG: Mysql execute: CREATE INDEX ix_actor_link_1 ON actor_link (media_type(20))
2020-01-11 03:54:57.743 T:775 DEBUG: Mysql execute: CREATE UNIQUE INDEX ix_actor_link_2 ON actor_link (actor_id, media_id, media_type, role)
2020-01-11 03:54:57.744 T:775 ERROR: SQL: [MyVideos118] Undefined MySQL error: Code (1170)
Query: CREATE UNIQUE INDEX ix_actor_link_2 ON actor_link (actor_id, media_id, media_type, role)
2020-01-11 03:54:57.745 T:775 ERROR: Exception updating database MyVideos118 from version 116 to 118
2020-01-11 03:54:57.745 T:775 ERROR: Error updating database MyVideos118 from version 116 to 118
2020-01-11 03:54:57.746 T:775 DEBUG: Mysql rollback transaction

The full error for this query appears to be:

Error Code: 1170. BLOB/TEXT column 'media_type' used in key specification without a key length

Specifying a key length on media_type then results in a similar failure for role.

The following appears to work:

CREATE UNIQUE INDEX ix_actor_link_2 ON actor_link (actor_id, media_id, media_type(255), role(255))
however 255 is just a random maximum number - something lower might make more sense, or use a non-text datatype for both columns (I've no idea how these columns are used etc.). 
Thanks for the detailed analysis. For media_type we usually use 20 but for role we should use the maximum. I'll make the necessary changes ASAP.


This forum uses Lukasz Tkacz MyBB addons.