Kodi Community Forum

Full Version: Google Drive for KODI
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
(2020-10-16, 16:17)cguZZMan Wrote: [ -> ]So you want to pass the whole list of your files to Google Drive API so they can give you what is different. No, there is no api for that.
A full export on Kodi start up is not a good fit if you have a big list of files. You should use the watch option after a single run of full export. In the worst scenario, schedule it once in a while or so. The goal here is to have the watch function working correctly.

Thanks for the input. Yes, I am using watch and have successfully ran a first export but it looks to me that every time I turn Kodi on it runs a full export. And it takes time. Before, if I added a movie after I shut Kodi down, it would pick up just that movie next time the export ran (it could take Kodi to restart a couple of times and sometimes it took more that a day to the export to pick up the new content).

 I will run some tests here later today and send you a debug file for your analysis. 

Thank you!
It does not run a full export. Every time you use your Drive, google is saving all those files changed or touched. Then when the addon asks for latest changes, all that list will be given by the api and the addon will start analyze them. If any of those files are within your exported folders, it will export them. If no, then they will be ignored.
In any case, it has to analyze and/or process all reported files so your local copy is in sync.
I'm doing a change now to "hide" those files not really exported from the progress bar, so user does not get confused and think they are been exported. This is only a visual change, the file has to be analyzed anyway. Functionally, it changes nothing.
Hi Carlos, sorry I don't have a log but I am considering implementing the option. The 1.36 version of the add-on was working great for me but I cannot get the new version to finish a full export of my TV Shows. Granted I have a large collection and I let the export run for 36 hours. According to the progress bar it was close to finishing but the add-on crashed while Kodi was running but the add-on did restart itself. Sadly, it started exporting again from the beginning. For me the beginning is the TV Series The Mentalist. My Movies exported fine, slower than before much slower but because it's basically 1/18 the size of my TV Shows based on number of files it finished.

Honestly, I don't know what to do because I have tried this several times over the last few days including uninstalling the add-on and trying again. If I could somehow just get the add-on to scan a portion of the TV Shows and honor that as the new export starting point at a future time. As far as I can tell any failure to fully complete the export results in the process starting over. This also happened during the failed migration of my 1.36 version from cfg to db.

As per previous postings by me, disabling the add-on during export crashes kodi. Uninstalling the add-on during export crashes kodi.

I hope you know we all appreciate you hard work and dedication to the add-on and I am grateful even if I can't use it anymore.

Thanks, Droid
(2020-10-16, 16:44)cguZZMan Wrote: [ -> ]It does not run a full export. Every time you use your Drive, google is saving all those files changed or touched. Then when the addon asks for latest changes, all that list will be given by the api and the addon will start analyze them. If any of those files are within your exported folders, it will export them. If no, then they will be ignored.
In any case, it has to analyze and/or process all reported files so your local copy is in sync.
I'm doing a change now to "hide" those files not really exported from the progress bar, so user does not get confused and think they are been exported. This is only a visual change, the file has to be analyzed anyway. Functionally, it changes nothing.

What Google uses a a key to know the last time "use" my Drive? Is it when I open Kodi and export completes? I am asking because, in my case, that "changed/touched" list you mention that is provided by Google which is analysed by the addon has a lot of files/folders which have already been imported - as you noted above there can be. I wonder why this would happen since the same files were not changed nor touched since the last time the export completed. 

I am just thinking about all this extra overhead the addon need to go through of verifying everything before deciding if needs to be imported. In my case, Google is giving the add on a list WAY bigger than the actual changed or touched files (which gave me the impression of a full export on Kodi lunch), which makes that progress bar to take a looong time to finish, whether or not you hide the files not needed to be exported...

This is why I asked you before about a async type of approach. Since it seems there isn't an API that would accept a list of files, maybe do the opposite: get this changed/touched list from Google, run some kind of bulk diff with local file system (not one by one), manage (add/delete/update) the differences (one by one). May be that is what you are already doing Big Grin.
@Droidly I will review the restart function to make sure the progress is saved.

@gilsonsjc yes. that's how is already been done.
About your key question, google provides a token that have to be exchanged next time the addon ask for changes. The api will give you the list of changes and a new token. That new token is used next time and the cycle continues.
(2020-10-16, 17:42)cguZZMan Wrote: [ -> ]@Droidly I will review the restart function to make sure the progress is saved.

@gilsonsjc yes. that's how is already been done.
About your key question, google provides a token that have to be exchanged next time the addon ask for changes. The api will give you the list of changes and a new token. That new token is used next time and the cycle continues.

Thank you Carlos. I do have a personal solution which would be to temporarily move a huge portion of my TV Shows to a new location on my google drive. This would allow the add-on to finish and I could gradually move the series folders back into the TV Shows folder.

Again, thanks for your efforts and your fast reply here. I am working my way through adding the logging process now as per your link in an earlier post. I really do hope some of my questions, suggestions, crashes and failures help in some way without frustrating you.
Carlos, I took my own advice and started over.

Movies exported perfectly albeit slower than v. 1.3.6

TV Shows - here is the curious part.

1. I did an initial export with only 10 series which were all my series that started with a number in the google drive tv show folder. Perfect no problems. Closed kodi and restarted kodi to confirm it wasn't going to try and scan all of those again. It did not and functioned perfectly.

2. For my own sanity I added by alphabet the next series obviously starting with A in this case A Confession (2019). Boom perfect, kodi saw it and added it quite rapidly. Strms created immediately*****

3. Added the next A series and same result fast export and strms created immediately****.

4. I got brave (or maybe stupid) I added all of the A series which was 31 series, 31 folders. Export started immediately but the strms did not get created until all the scans were finished. Then the google add-on started scanning again and essential export those same 31 folders again without overwriting and ultimately finished successfully but it was totally different behaviors and it scanned twice and exported twice or at least went through the motions.

I do not have a log file, I'm sorry as I got sidetracked. I hope this helps somehow.
So, to be clear, you did not re-included or re-added any of the previously added folders, right?

About all your folders been created first and then your strms, that is exactly how the export works.
What the addon does first is to get the list of files and folders (of the folders been exported). If it is a folder, then a folder will be created. If it's a exportable file or movie, it will create a .strm file.
Now, if you have all your movies in separated folders, those movies files are not detected on this run. They are not part of the current folder.
Once all files and folders are created, it will go deep on each folder to repeat the same process: get all files and folders, if there is a folder, create a folder, if there is a exportable file, create a .strm file, and so on.
btw, I found a possible improvement in the export process that can avoid scaning the same folder twice when watching for changes. it can potentially speed up the watch analysis (I don't see a possible improvement in a full export though). will try to release it soon.
(2020-10-16, 20:48)cguZZMan Wrote: [ -> ]So, to be clear, you did not re-included or re-added any of the previously added folders, right?

About all your folders been created first and then your strms, that is exactly how the export works.
What the addon does first is to get the list of files and folders (of the folders been exported). If it is a folder, then a folder will be created. If it's a exportable file or movie, it will create a .strm file.
Now, if you have all your movies in separated folders, those movies files are not detected on this run. They are not part of the current folder.
Once all files and folders are created, it will go deep on each folder to repeat the same process: get all files and folders, if there is a folder, create a folder, if there is a exportable file, create a .strm file, and so on.
Okay, that would be consistent with the behaviors then. I am not touching my local db on my PC created by your add-on. I am just letting the add-on add, delete, rename etc... I learned that lesson earlier when you advised me on this.

All I am doing is manipulating the TV Show series folders on my google drive cloud storage. For the most part all is working well and accurate except it has failed on 2 series in that it created the series folder but no seasons and no strms. Pulling both from google drive the add-on updated and deleted the local folders. Adding both again resulted in one being fully successful and the other failing again.

Achieved success by manually exporting the failing series.

I will continue to work my way through all of my series from A-Z in a controlled manner so I can spot failures and fix them.
There is a new version in my repo that improves exports.
Also the restart should be working fine now. If something happens in between an export, it will automatically restart when Kodi starts, and pending files only, not the whole export again.
(2020-10-16, 23:01)cguZZMan Wrote: [ -> ]There is a new version in my repo that improves exports.
Also the restart should be working fine now. If something happens in between an export, it will automatically restart when Kodi starts, and pending files only, not the whole export again.

Did the update, google drive now displays
Google Drive Watching Changes Starting please wait...

It's been about 30 minutes of waiting so far is this normal?
it is not. it will help if you turn your debug on and try again and extract the log file.
Thanks for the update - working well on my end.
Google Drive Watching Changes Starting please wait... only waited a few seconds.
GD has 332 tv shows and <600 movies = 32TB
Using ODroiid N2 with CoreElec
(2020-10-17, 18:54)cguZZMan Wrote: [ -> ]it is not. it will help if you turn your debug on and try again and extract the log file.
All good now, I rolled back my version then updated to the new version and it worked this time as expected.

One note, and this was on the prior version 1.4.3. Renaming an episode on the google drive does not seem to get a new strm created. It happened to me last night while renaming 8 seasons.