Broken Integrate video plugin provided Movies directly into the Kodi database
#1
Pre-Alpha Testing Version 0.0.4a - 20230915

https://transfer.sh/NwCdfDr5xX/plugin.vi...0915.1.zip

Overall code restructuring (again).

Added a search results display when importing so that the correct content can be selected instead of using the first returned result.
- Search results displays only when more than 1 result returned, otherwise still uses first and only result.

Modularization of bookmarks code.

Migrate episodes' bookmarks to it's own SQLite database.

Fixed bug where tracking linked bookmarks to Kodi library entries was inoperative.
Reply
#2
Note: v20+ only

Setup

Install the addon.
Open your favorite video addon, open the context menu (or right click) and select "Import to Library" on either a Movie or a TVShow (not season/episode whole show)
This will add a bookmark to the movie or show so it can be scanned in to the database by Kodi then trigger a library update initiating the scan.

Add the sources to Kodi either by manually editiing sources.xml or under Files -> Add Videos

Note: if the second option is used then you will need to type the paths manually into Kodi because Browse will not work

Manually adding the entries to Sources.xml (with a text editor) under the section "video" -

xml:
<source>
<name>Integrate Movies</name>
<path pathversion="1">plugin://plugin.video.integrate/movies/</path>
<allowsharing>true</allowsharing>
</source>
<source>
<name>Integrate TV Shows</name>
<path pathversion="1">plugin://plugin.video.integrate//tvshows/</path>
<allowsharing>true</allowsharing>
</source>

Set the content type of these new sources.

For Movies - Local Information Only, Scan Recursively Disabled, uncheck scan for external audio tracks.
For Shows - Local Information Only, uncheck scan for external audio tracks.

How Does It Work?

It is a fancy way of keeping bookmarks to say the least.
It will add a JSON entry to bookmarks.json in it's addon_data folder.
It is set up as a "pluginsource" within Kodi serving content, similar to how Plex and Jellyfin addons are setup.
When Kodi accessing the "directory listing" the bookmarks are presented as ListItems which Kodi will then identify with the scraper and upon success add to the Kodi database.
When playing the content it will play back using the video addon that it was scanned from.

Public development terminated due to lack of user interest.

Private development only until a time I see the project is fit for mass consumption.
- After testing has been done on multiple platforms. At this time there is no interest in the project so this requirement will delay releases significantly.
- When code stand on it's own without needing to be updated for an acceptable timeframe. (Excluding adding features)
Reply
#3
(2023-08-03, 22:55)jepsizofye Wrote: Public development terminated due to lack of user interest.

Damn I was really interested in this plugin Sad
I tried to do something similar to my Discovery+ add-on but it got really complex to deal with updating new episodes of tvshow to Kodi library.
Reply
#4
(2023-08-09, 17:29)-Dis Wrote:
(2023-08-03, 22:55)jepsizofye Wrote: Public development terminated due to lack of user interest.

Damn I was really interested in this plugin Sad
I tried to do something similar to my Discovery+ add-on but it got really complex to deal with updating new episodes of tvshow to Kodi library.

this bridges functionality of a video addon which does not implement it, if you wanted to implement into your existing addon there are only a few requirements


the need for "medialibraryscanpath" in addon.xml

xml:

<extension point="xbmc.python.pluginsource" library="default.py">
    <provides>video</provides>
    <medialibraryscanpath content="tvshows">/tvshows</medialibraryscanpath>
    <medialibraryscanpath content="movies">/movies</medialibraryscanpath>
</extension>

the detection of path being accessed from your python code
only need to check sys.argv[0] for /tvshows or /movies and act accordingly, in my instance i split sys.argv[0] and check [-1], for an existing path - this would likely be similar to

default.py

python:
import sys
path = sys.argv[0].replace('plugin://','').split('/') # replace plugin:// in order to avoid extra params from the protocol being generated

if path[1]=='movies':
    listMovies()
elif path[1]=='tvshows':
    listShows(path)

def listShows(path):
    if path[-1]=='tvshows':
        #present parent listing
    else:
        #present listing of show in path[-1]

def listMovies():
    #present parent movies listing


the sources component is telling Kodi your path's content type, scraper and scraper settings
i didn't find anything officially callable so i am inserting paths directly into the database

SQL:
INSERT INTO path (strPath, strContent, strScraper, strHash, scanRecursive, useFolderNames, strSettings, noUpdate, exclude) VALUES ("plugin://plugin.video.integrate/movies/", "movies", "metadata.themoviedb.org.python", 0, 0, 0, "", 0, 0)
INSERT INTO path (strPath, strContent, strScraper, strHash, scanRecursive, useFolderNames, strSettings, noUpdate, exclude) VALUES ("plugin://plugin.video.integrate/tvshows/", "tvshows", "metadata.tvshows.themoviedb.org.python", 0, 1, 0, "", 0, 0)


on initial imports i am using json rpc to call an update, only seems to work on the root parent paths not the individual sub folders for shows

javascript:
{"jsonrpc": "2.0", "method": "VideoLibrary.Scan","params":{"directory": "plugin://plugin.video.integrate/tvshows"}, "id": 1}


the listings are generic xbmcplugin.ListItem and getVideoInfoTag

episode listings need getVideoInfoTag.setSeason(int) and getVideoInfoTag.setEpisode(int) to be picked up


subsequent library updates happen normally outside the addon, kodi/tmdb updates everything from there
Reply
#5
Thanks for examples. Seems to be very easy to integrate in to add-on but I really liked idea this being universal to all video add-ons.

Does it work if opening tvshow lists all available seasons and not all episodes?
Like this:

TVshow
  • Season 1
  • Season 2
  • ...
Reply
#6
(2023-08-09, 19:15)-Dis Wrote: Does it work if opening tvshow lists all available seasons and not all episodes?
Like this:

TVshow
  • Season 1

  • Season 2

  • ...

i havent tested that, in theory it *should* as i believe kodi would be able to scan those if they were local files

in mine i traverse an addon's listing for a show and present the episodes in a single list, same as "flatten tv shows", i feel this would be the same thing if kodi were allowed it would traverse them the same way

this way allows a single bookmark to be added to the "show" instead of multiple bookmarks for each "season" which then need to be tracked in the case that seasons within the show change, removed or added by provider f.e.
Reply
#7
(2023-08-09, 18:00)jepsizofye Wrote: the sources component is telling Kodi your path's content type, scraper and scraper settings
i didn't find anything officially callable so i am inserting paths directly into the database

You'll need to handle SQLite vs. MySQL, if you are going for supporting both.  I wrestled  with this in my Kodi Selective cleaner addon. 

I presume your target here with this was setups which use the Kodi scrapers (i.e. stand-alone, MySQL etc..) vs. sharing solutions that have their own scrapers ?  I am happy to help you write it but wouldn't have much use for it in my setup (similar to KS Cleaner). 

For folks with non-Kodi scrapers I'd think they would have the video addon write the video files to a common place and the other scrapers pick it up (automatically or manually depending upon capabilities and configuration) , insert it into the centralized sharing database and then sync from there to the Kodi video database. This is how I do it today with HDHomeRun and similar content with Mezmo.   What gets interesting is the metadata for more obscure sources which aren't going to be in the normal scrapers and how to handle that.   

One other thing to consider is handling exceptions and having the capability to exclude certain content from being written to the Kodi database vs. all content from an addon. 


Thanks,

Jeff
Running with the Mezzmo Kodi addon.  The easier way to share your media with multiple Kodi clients.
Service.autostop , Kodi Selective Cleaner and Mezzmo Kodi addon author.
Reply
#8
(2023-08-09, 20:20)jbinkley60 Wrote: You'll need to handle SQLite vs. MySQL, if you are going for supporting both.  I wrestled  with this in my Kodi Selective cleaner addon. 

i am *trying to avoid direct database access, inserting into paths wasnt my first idea but kodi does not like working with plugin sources - or i dont understand how it does yet
most of my reference material is the jellyfin addon and composite addon which use this same method, where i got the idea in the first place
neither addon is putting their path into the database but seem to directly insert content which is not what it want
i will be checking your code and unifying database calls if i cannot replace them altogether with built-in functionality

make it work then make it work better

im going to have to submit some core PRs at some point (unless i find someone else to, yea sure) to allow certain things

f.e. right now i have kodi not wanting to scan folders without being told to by jsonrpc because "they dont exist", they only dont exist because of this line
https://github.com/xbmc/xbmc/blob/master...r.cpp#L129
which checks for physical existence and has no allowance for plugin paths

just one of many things i have come across

 
(2023-08-09, 20:20)jbinkley60 Wrote: I presume your target here with this was setups which use the Kodi scrapers (i.e. stand-alone, MySQL etc..) vs. sharing solutions that have their own scrapers ?  I am happy to help you write it but wouldn't have much use for it in my setup (similar to KS Cleaner). 

For folks with non-Kodi scrapers I'd think they would have the video addon write the video files to a common place and the other scrapers pick it up (automatically or manually depending upon capabilities and configuration) , insert it into the centralized sharing database and then sync from there to the Kodi video database. This is how I do it today with HDHomeRun and similar content with Mezmo.   What gets interesting is the metadata for more obscure sources which aren't going to be in the normal scrapers and how to handle that.   


in a more advanced state, this addon would set all of the video information itself so that "metadata.local" can be used from listitem and videoinfotag instead of tmdb
back to make it work first


(2023-08-09, 20:20)jbinkley60 Wrote: One other thing to consider is handling exceptions and having the capability to exclude certain content from being written to the Kodi database vs. all content from an addon. 


so far i am, there may be a better way but for now the menu option is conditional for import


xml:
<item library="context.py">
<label>Import to Library</label>
<visible>String.IsEqual(ListItem.dbtype,movie) | String.IsEqual(ListItem.dbtype,tvshow) | !String.Contains(ListItem.Path,plugin://plugin.video.integrate)</visible>
</item>


only allowing movie and tvshow dbtypes and not import from my own addon
Reply
#9
follow up

Motivation

Time and cost of TV Series both initially and storage, if you have ever had to copy an entire series from a BluRay you know there is a lengthy process to it.
Physical copying time, naming, subtitles and if you dont want to remux you can add encoding time.

Storage at least in my collection, I tend to encode episodes to ~4GB per episode, remux can be ~5-6GB per episode.
Checking my own collection I have 1 series of 4 seasons 81 episodes which is at 430GB, another series which is remux 59 episodes for 299.1GB, this is only 2 of 57 series I am storing online and does not include offline archived series.

Alternatively, with streaming quality nearly as good as BluRay from most current services I can spend less over 2 years paying for a service to stream these same shows than I can for drives to store them and after 2 years I would likely buy a new drive anyway.

So, for me, it makes sense to still have my series organized into my library (the way Kodi does so well) where the cost and time spent is far less.


Adding movies into the equation makes sense as well but streaming services dont (yet) offer high enough quality to replace a 4K BluRay, however in some instances do offer some movie titles in better than BluRay and some titles which are not even available beyond DVD if then.
Reply
#10
(2023-08-09, 21:44)jepsizofye Wrote:
(2023-08-09, 20:20)jbinkley60 Wrote: You'll need to handle SQLite vs. MySQL, if you are going for supporting both.  I wrestled  with this in my Kodi Selective cleaner addon. 

i am *trying to avoid direct database access, inserting into paths wasnt my first idea but kodi does not like working with plugin sources - or i dont understand how it does yet
most of my reference material is the jellyfin addon and composite addon which use this same method, where i got the idea in the first place
neither addon is putting their path into the database but seem to directly insert content which is not what it want
i will be checking your code and unifying database calls if i cannot replace them altogether with built-in functionality

I am nice with the Mezzmo addon and add the proper pathing in the path table but I really don't use it.  I just do it to be compliant with other Kodi video database metadata.
 
Quote:in a more advanced state, this addon would set all of the video information itself so that "metadata.local" can be used from listitem and videoinfotag instead of tmdb
back to make it work first
 

That how I handle it, just set to metadata local.  There will still be a question of the quality of the metadata itself and where you get it from.  For me either Mezzmo scrapes it or I use the GUI metadata editor.  I don't think you want to go that route and write an editor. 


Thanks,

Jeff
Running with the Mezzmo Kodi addon.  The easier way to share your media with multiple Kodi clients.
Service.autostop , Kodi Selective Cleaner and Mezzmo Kodi addon author.
Reply
#11
(2023-08-09, 17:29)-Dis Wrote:
(2023-08-03, 22:55)jepsizofye Wrote: Public development terminated due to lack of user interest.

Damn I was really interested in this plugin Sad
I tried to do something similar to my Discovery+ add-on but it got really complex to deal with updating new episodes of tvshow to Kodi library.

@-Dis concepts like this have been around for a long time in some form or another; I've created my fair share of scripts/plugins to do exactly this;
If you need any help adding something like this to your Discovery+ add-on feel free to either DM me or start a thread under video plugin development.

Kodi already started work some time ago to add this kinda feature into core. https://forum.kodi.tv/showthread.php?tid=224794
Unfortunately, like with all past projects there isn't enough unified plugin development to incorporate features like this and most developers would rather use something they wrote then a fly-by-night project.
Image Lunatixz - Kodi / Beta repository
Image PseudoTV - Forum | Website | Youtube | Help?
Reply
#12
(2023-08-09, 22:09)jepsizofye Wrote: follow up

Motivation

Time and cost of TV Series both initially and storage, if you have ever had to copy an entire series from a BluRay you know there is a lengthy process to it.
Physical copying time, naming, subtitles and if you dont want to remux you can add encoding time.

Storage at least in my collection, I tend to encode episodes to ~4GB per episode, remux can be ~5-6GB per episode.
Checking my own collection I have 1 series of 4 seasons 81 episodes which is at 430GB, another series which is remux 59 episodes for 299.1GB, this is only 2 of 57 series I am storing online and does not include offline archived series.

Alternatively, with streaming quality nearly as good as BluRay from most current services I can spend less over 2 years paying for a service to stream these same shows than I can for drives to store them and after 2 years I would likely buy a new drive anyway.

So, for me, it makes sense to still have my series organized into my library (the way Kodi does so well) where the cost and time spent is far less.


Adding movies into the equation makes sense as well but streaming services dont (yet) offer high enough quality to replace a 4K BluRay, however in some instances do offer some movie titles in better than BluRay and some titles which are not even available beyond DVD if then.

I concur with you on some of the TV shows and quality.  The cost of storage is a different discussion since it is so cheap.  One thing I am curious is about your use of Jellyfin and scraping.  Doesn't Jellyfin scrape your TV show metadata and then make it available to your Kodi clients (either by a sync process or browsing the Jellyfin library directly )  ?    Or does Jellyfin leverage a Kodi scraper ?  I am only asking in thinking of your addon proposal. 

With Mezzmo I can scrape any type of content from anywhere and exclude what I want as long as the Mezzmo server has access to the files.  I would think Jellyfin is similar in structure and features.  Given that, I am not seeing as much benefit to you of the addon proposal for this use case vs. someone a stand-alone Kodi instance where the Kodi scraper may not easily be able to do what you want.  I am sure I am missing something.

As to your TV series organization, I concur with the approach.  I have over 16k episodes from 190+ series (a mix of DVDs, Blu-Rays, OTA/DVR and streaming) in 46TB of storage (~2.5GB per episode).  I have 25+ totes in the  basement with discs in them.  A centralized streaming server approach is definitely the way to go.  I agree that DVDs and Blu-Rays are a bit more labor intensive but I have my process down to where it doesn't take that long.  I make ISOs out of the disks as a backup to the media being damaged.  I then rip an entire season to a temporary location disc on my server, run a Pythons script to rename the files to my preferred naming structure, move the files to the permanent location on the Mezzmo server and then let Mezzmo and the Mezzmo Kodi addon do the rest automatically.    As long as the scraper sites episode numbering lines up with the episode numbering on the discs then it is smooth.  I am actually seeing that more of an issue with streaming.   I do the same for movies but then run the trailer checker to pull down local trailers.

OTA/DVR is even easier.  The files drop into their permanent folders and Mezzmo picks them up automatically and scraps the metadata.  I do pretty much nothing for them.   I record a lot of OTA/DVR stuff that I don't plan to keep for more than a year or 2 (except some football games).  It's good quality because it is all MPEG2 native with higher bitrates.


Thanks,

Jeff
Running with the Mezzmo Kodi addon.  The easier way to share your media with multiple Kodi clients.
Service.autostop , Kodi Selective Cleaner and Mezzmo Kodi addon author.
Reply
#13
(2023-08-10, 03:54)jbinkley60 Wrote: One thing I am curious is about your use of Jellyfin and scraping.  Doesn't Jellyfin scrape your TV show metadata and then make it available to your Kodi clients (either by a sync process or browsing the Jellyfin library directly )  ?    Or does Jellyfin leverage a Kodi scraper ?  I am only asking in thinking of your addon proposal. 

Existing concurrently within Kodi, neither replacing the other, I can have all my Jellyfin served series with Hulu, with Prime Video, with Paramount+, with Disney+ all in a Kodi library. Kodi being the primary media center for my office and my livingroom.

At the same time, on my mobile devices I use the individual apps for android so I still get all the content but is not as elegant.

Jellyfin does scrape it's own media and serve it to Kodi, I believe Mezzmo and Jellyfin are nearly identical in features - nearly though because I think Mezzmo might have a couple things over Jellyfin, Jellyfin just happens to run on linux and from a docker so it was my choice.


While I took this away from "public" development I have been working on it passed that.
I just can't test it on platforms I don't use in environments I don't use with the perspective of someone else (my Android and Linux versions are self compiled Alpha master as well) so I dropped the pre-alpha hoping to get some of that covered.
With no interest over 4 days I decided if it can't be tested for use on other systems best not to worry about them not to mention if there is no interest for other systems then there is no point.

If you want to browse over the code, even if just to see what someone else did, I will upload and PM you a link.
Reply
#14
@-Dis i believe i found the original PR when this was added to Kodi, they even included a nice example addon

https://github.com/xbmc/xbmc/pull/13566
Reply
#15
(2023-08-10, 04:29)jepsizofye Wrote:
(2023-08-10, 03:54)jbinkley60 Wrote: One thing I am curious is about your use of Jellyfin and scraping.  Doesn't Jellyfin scrape your TV show metadata and then make it available to your Kodi clients (either by a sync process or browsing the Jellyfin library directly )  ?    Or does Jellyfin leverage a Kodi scraper ?  I am only asking in thinking of your addon proposal. 

Existing concurrently within Kodi, neither replacing the other, I can have all my Jellyfin served series with Hulu, with Prime Video, with Paramount+, with Disney+ all in a Kodi library. Kodi being the primary media center for my office and my livingroom.

At the same time, on my mobile devices I use the individual apps for android so I still get all the content but is not as elegant.

Jellyfin does scrape it's own media and serve it to Kodi, I believe Mezzmo and Jellyfin are nearly identical in features - nearly though because I think Mezzmo might have a couple things over Jellyfin, Jellyfin just happens to run on linux and from a docker so it was my choice.


While I took this away from "public" development I have been working on it passed that.
I just can't test it on platforms I don't use in environments I don't use with the perspective of someone else (my Android and Linux versions are self compiled Alpha master as well) so I dropped the pre-alpha hoping to get some of that covered.
With no interest over 4 days I decided if it can't be tested for use on other systems best not to worry about them not to mention if there is no interest for other systems then there is no point.

If you want to browse over the code, even if just to see what someone else did, I will upload and PM you a link.

Thanks for the response.  I believe they are similar and I wasn't questioning your use of a particular sharing solution and I fully get the Mezzmo hosting platform limitations.  I live a bit of a sheltered life with regards to Kodi and other solutions so I often ask questions simply seeking to understand (often by comparison / contrast from my reference point).  I don't use or haven't used a lot of other Kodi addons but Kodi is rock solid as my main media client so I try to get the most out of it with my setup.  I sometimes question various use cases to get ideas for additional functionality for my own addons or to see if there is a process vs. technical solution to the problem.  

I agree that PR looks almost exactly like what you proposed.  Getting the addon developers to leverage it is likely part of the challenge.  Likewise the Media Importer solution that Lunatixz posted the link to is very interesting.  I've looked at it a few times and may one day write the Mezzmo importer for it.  The challenge I see for it right now is that it requires special builds.  Once it goes into the main code I suspect the usage will pick up significantly.   

With regards to addon testing, that hasn't been much of an issue for me.  I have a good testbed for various Linux and Windows versions of Kodi (hard clients and virtual machines) and I have some loyal nVidia Shield users who help on the Android side.  But having said that, this is where Kodi's Python addon approach really shines.  I can't recall ever having a platform specific issue.  I did have an issue once with a version of LibreElec which caused all Python addons to run slow due to not getting enough CPU resource allocation.  But that impacted all Python addons.  I've stayed away from binary addons, where this is a real issue to deal with.


Thanks,

Jeff
Running with the Mezzmo Kodi addon.  The easier way to share your media with multiple Kodi clients.
Service.autostop , Kodi Selective Cleaner and Mezzmo Kodi addon author.
Reply

Logout Mark Read Team Forum Stats Members Help
Integrate video plugin provided Movies directly into the Kodi database0