"Update Library" issues
#16
(2020-05-13, 16:18)DanCooper Wrote: I'll check your files tonight. I still don't know why the scanning is so slow on your PC (I can scan a source with 5'000 movies and another with over 13'000 episodes in 5-10min).

Based on the time you said yours takes, then my notebook is even slow... but not as slow as the one for the desktop. I did an update library on my notebook starting around 12:52 and by 3:27 it was completed, at least that's better than a couple days that the version on my desktop takes. Another thing that's different is my notebook is connected via a LAN cable with possible transfer speeds of up to 100 Mbps (using a Intel Core i7 Q740 @ 1.73 GHz, with 6 GB RAM) and my Desktop is connected to a LAN cable with possible transfer speeds up to 1 Gbps (using a Intel Core i7 2630QM @ 2.00 GHz, with 6 GB RAM). I'd assume the desktop would be faster.

I rip the DVD's as 1 to 1 copies, to my NAS device. Doing this creates 2 folders. 1.) Audio - which is empty & 2.) Video - which contains the VOB, IFO & BUP files. I copy the contents of the Video folder to my streaming file path (which is also a file path in the source portion of Ember) in a folder named the movie title (year) [ex. Homefront (2013) naming scheme - creating a file path of Z:\Active Files\DVD Movies\Homefront (2013)]. The file path is: Z:\Active Files\DVD Movies for both my desktop and notebook. The notebook version does take longer to update the library than the 5-10 min yours does, but it's usually done within a few hours at most. The desktop version took 1 1/2 to 2 days to add the same 4 movies (3 movies & 1 TV series disc) from the same file location.

Here's a link to the screenshots showing the settings for:

Notebook: https://drive.google.com/open?id=1R3Lkjz...Znc3hi9Y_p

Desktop: https://drive.google.com/open?id=1BCvao_...g3WbJjbJX5

Link for all the uploaded logs & Screenshots (sorted by notebook & desktop): https://drive.google.com/open?id=1BOr_sc...R4-QfljmDi

I did a test with 3 DVD's that I recently added, however I made a copy of the contents of the Video folder from the DVD (# 2 above) to My Documents folder (of my desktop) in a parent folder named "Ember Test". I added the "Ember Test" folder as a source in Ember (file path: C:\Users\John Thorpe\Documents\Ember Test) and chose "Update Ember Test Only" under the "Update Library" drop down, and it added the 3 new DVD's withing 5 to 7 minutes ... course the Ember Test folder only had the 3 new DVD's in there and not my whole library, so I don't know if that is why it finished so quickly, I don't know if there is a setting in Ember that is causing the update library from a network drive device to be slow vs. the drive in my desktop to be slower... Here's a link to the log using the drive in my desktop: https://drive.google.com/open?id=1lJtjAK...AVOqc3-wdD

I don't think it's the NAS drive or any cables as the update time for my notebook, while it's not in the 5 to 10 minute range, is a lot faster than updating the desktop version to the same device and file paths.
Reply
#17
Also, on the test that I did with using the local drive in my desktop, I did have the module "Notifications" turned back on... so that doesn't seem to be the cause.
Reply
#18
(2020-05-13, 16:18)DanCooper Wrote: I'll check your files tonight. I still don't know why the scanning is so slow on your PC (I can scan a source with 5'000 movies and another with over 13'000 episodes in 5-10min).

Something else I did notice when going to the logs is that I have 2 Ember folders in the C drive

1 folder is named "Ember Media Manager" which contains: these folders: Bin, DB, Defaults, Images, Langs, Log, Modules, Profiles, Temp, Themes, x64 & x 86 along with 18 loose files like EntityFramework.dll, etc.

The other folder is named "Ember Media Manager BETA" and contains a "Profiles" & "Temp" folder. The Temp folder is empty and the Profiles folder has a file named "profiles" as an XML document and the Default folder contains 15 items:
AdvancedSettings
CertLanguages
Core.Genres
Core.ScraperLanguages
Genres
Interface.Kodi
Interface.Trakt
MovieCertLanguages
Queries
Ratings
Settings .... these are all XML documents

And MyVideos24.emm
MyVideos46.emm
MyVideos46.emm_tmp
MyVideos46.emm-journal

Looking in the C drive of the version that's on my notebook I just have the "Ember Media Manager BETA" folder, no other ones.
Reply
#19
(2020-05-13, 20:22)jthorpe Wrote: and it added the 3 new DVD's withing 5 to 7 minutes ... course the Ember Test folder only had the 3 new DVD's in there and not my whole library, so I don't know if that is why it finished so quickly
Sorry, but scanning 3 DVD on a local drive should be done in SECONDS!
 
(2020-05-13, 20:25)jthorpe Wrote: Also, on the test that I did with using the local drive in my desktop, I did have the module "Notifications" turned back on... so that doesn't seem to be the cause.
The problem with the Notification module is that every notification will be displayed as single "dialogs" and sometimes they no longer fade out because of the large number of "dialogs". In your logs a saw a hugh time gap between a notification call and the next step. So that's the reason why my first idea was to disable the module.
 
(2020-05-13, 20:35)jthorpe Wrote: file named "profiles" as an XML document
That has been added to read which profiles has to be set as "load as default profile" instead of selecting the profile every time you start Ember.
 
(2020-05-13, 20:35)jthorpe Wrote: The Temp folder is empty
Usually the temp folder is used to cache previews/thumbs and downloaded images in full quality. Image caching for movies and moviesets it's nearly useless (and disabled by default) but it's very usefull for tv shows because of the actor thumbs for episodes (usually they are the same for all or many episodes and Ember has to download each image again and again if the cache is disabled; that's the reason why image caching for tv shows is enabled by default). But that's also not the problem with your issues.
MyVideos24.emm was the old database, MyVideos46.emm is the upgraded version for the latest version of Ember. *.emm_tmp is the file that will be created while upgrading an older DB to the latest version (and should automatically been deleted after upgrade), *.emm-journal is the temporarily created "DB of changes" if several changes are in the "pipeline" (e.g. while scraping multiple movies) and not already commited by the code.

Do you run an antivirus software on your system?
Reply
#20
and another hint: Ember does NOT support DVD rips (VIDEO_TS) for tv shows atm, that works only for movies. If you want to scrape episodes as DVD rips you have to put them in an ISO (Kodi can play it also as an ISO with DVD full menu and extras).
Reply
#21
(2020-05-13, 21:26)DanCooper Wrote:
(2020-05-13, 20:22)jthorpe Wrote: and it added the 3 new DVD's withing 5 to 7 minutes ... course the Ember Test folder only had the 3 new DVD's in there and not my whole library, so I don't know if that is why it finished so quickly
Sorry, but scanning 3 DVD on a local drive should be done in SECONDS!
 
(2020-05-13, 20:25)jthorpe Wrote: Also, on the test that I did with using the local drive in my desktop, I did have the module "Notifications" turned back on... so that doesn't seem to be the cause.
The problem with the Notification module is that every notification will be displayed as single "dialogs" and sometimes they no longer fade out because of the large number of "dialogs". In your logs a saw a hugh time gap between a notification call and the next step. So that's the reason why my first idea was to disable the module.
 
(2020-05-13, 20:35)jthorpe Wrote: file named "profiles" as an XML document
That has been added to read which profiles has to be set as "load as default profile" instead of selecting the profile every time you start Ember.
 
(2020-05-13, 20:35)jthorpe Wrote: The Temp folder is empty
Usually the temp folder is used to cache previews/thumbs and downloaded images in full quality. Image caching for movies and moviesets it's nearly useless (and disabled by default) but it's very usefull for tv shows because of the actor thumbs for episodes (usually they are the same for all or many episodes and Ember has to download each image again and again if the cache is disabled; that's the reason why image caching for tv shows is enabled by default). But that's also not the problem with your issues.
MyVideos24.emm was the old database, MyVideos46.emm is the upgraded version for the latest version of Ember. *.emm_tmp is the file that will be created while upgrading an older DB to the latest version (and should automatically been deleted after upgrade), *.emm-journal is the temporarily created "DB of changes" if several changes are in the "pipeline" (e.g. while scraping multiple movies) and not already commited by the code.

Do you run an antivirus software on your system? 
Yes I do have an antivirus software on my system ... both are the same with the same settings.
Reply
#22
(2020-05-13, 21:29)DanCooper Wrote: and another hint: Ember does NOT support DVD rips (VIDEO_TS) for tv shows atm, that works only for movies. If you want to scrape episodes as DVD rips you have to put them in an ISO (Kodi can play it also as an ISO with DVD full menu and extras).

Doing the DVD rips seems to be working fine, with the exception of the Update Library taking so long.
Reply
#23
(2020-05-13, 21:26)DanCooper Wrote:
(2020-05-13, 20:22)jthorpe Wrote: and it added the 3 new DVD's withing 5 to 7 minutes ... course the Ember Test folder only had the 3 new DVD's in there and not my whole library, so I don't know if that is why it finished so quickly
Sorry, but scanning 3 DVD on a local drive should be done in SECONDS!
 
(2020-05-13, 20:25)jthorpe Wrote: Also, on the test that I did with using the local drive in my desktop, I did have the module "Notifications" turned back on... so that doesn't seem to be the cause.
The problem with the Notification module is that every notification will be displayed as single "dialogs" and sometimes they no longer fade out because of the large number of "dialogs". In your logs a saw a hugh time gap between a notification call and the next step. So that's the reason why my first idea was to disable the module.
 
(2020-05-13, 20:35)jthorpe Wrote: file named "profiles" as an XML document
That has been added to read which profiles has to be set as "load as default profile" instead of selecting the profile every time you start Ember.
 
(2020-05-13, 20:35)jthorpe Wrote: The Temp folder is empty
Usually the temp folder is used to cache previews/thumbs and downloaded images in full quality. Image caching for movies and moviesets it's nearly useless (and disabled by default) but it's very usefull for tv shows because of the actor thumbs for episodes (usually they are the same for all or many episodes and Ember has to download each image again and again if the cache is disabled; that's the reason why image caching for tv shows is enabled by default). But that's also not the problem with your issues.
MyVideos24.emm was the old database, MyVideos46.emm is the upgraded version for the latest version of Ember. *.emm_tmp is the file that will be created while upgrading an older DB to the latest version (and should automatically been deleted after upgrade), *.emm-journal is the temporarily created "DB of changes" if several changes are in the "pipeline" (e.g. while scraping multiple movies) and not already commited by the code.

Do you run an antivirus software on your system? 

What settings effect the EmberAPI.scanner? 

I did make a copy of both the Ember Media Manager folders on my C drive. I did this to try deleting the database files (the MyVideos24.emm and MyVideos46.emm, etc.) and if Ember wouldn't load, etc. I'd have the original to fall back on. After copying the Ember Media Manager and Ember Media Manager BETA folders, I went into the Profiles folder and deleted both the MyVideos24.emm and MyVideos46.emm and MyVideos46.emm-journal. I then restarted Ember Media and the file sources was empty (as well as the titles in both the Movie & TV sections). I added the file source paths back into the File Source section of the settings and it took roughly 1/2 of a day to add the 1990 Movies and 271 TV shows. I added a few titles using the local disk and it took no time at all. I then deleted the local source path and copied all of the files over to the NAS drive and added 15 new ones to the NAS drive as well. 

After adding the 15 new titles to my NAS drive, clicking on Update Library at 12:34 AM on 05-17-2020 has added 7 titles (out of the 15) so far (current time is 9:12 PM). Looking at the log while the Update Library is running, I'm noting that it scans Title A and it comes back skipped as it has already been scraped (has the extrafanart, extrathumbs, .actors folders) all these 3 have the same time... now before it starts to scan Title B there's anywhere between a 1 minute to 3 minute time lapse between scanning Title A and starting to scan Title B (for example scanning Title A shows a time of 38:35.8 but scanning Title B shows a time of 40:57.7 and it shows "EmberAPI.Scanner" in the Logger section). 

Here's the 3 most recent logs: https://drive.google.com/open?id=1Nzar0T...qECwmUPzNR

The log for 2020-05-16 is after I deleted the MyVideos files and it looks like it is around 3 seconds or so between scanning Title A and Title B, vs. the log for 2020-05-17 (which Update Library is currently running for this log) show around 1 to 3 minutes between each title.
Reply
#24
(2020-05-18, 03:28)jthorpe Wrote:
(2020-05-13, 21:26)DanCooper Wrote:
(2020-05-13, 20:22)jthorpe Wrote: and it added the 3 new DVD's withing 5 to 7 minutes ... course the Ember Test folder only had the 3 new DVD's in there and not my whole library, so I don't know if that is why it finished so quickly
Sorry, but scanning 3 DVD on a local drive should be done in SECONDS!
 
(2020-05-13, 20:25)jthorpe Wrote: Also, on the test that I did with using the local drive in my desktop, I did have the module "Notifications" turned back on... so that doesn't seem to be the cause.
The problem with the Notification module is that every notification will be displayed as single "dialogs" and sometimes they no longer fade out because of the large number of "dialogs". In your logs a saw a hugh time gap between a notification call and the next step. So that's the reason why my first idea was to disable the module.
 
(2020-05-13, 20:35)jthorpe Wrote: file named "profiles" as an XML document
That has been added to read which profiles has to be set as "load as default profile" instead of selecting the profile every time you start Ember.
 
(2020-05-13, 20:35)jthorpe Wrote: The Temp folder is empty
Usually the temp folder is used to cache previews/thumbs and downloaded images in full quality. Image caching for movies and moviesets it's nearly useless (and disabled by default) but it's very usefull for tv shows because of the actor thumbs for episodes (usually they are the same for all or many episodes and Ember has to download each image again and again if the cache is disabled; that's the reason why image caching for tv shows is enabled by default). But that's also not the problem with your issues.
MyVideos24.emm was the old database, MyVideos46.emm is the upgraded version for the latest version of Ember. *.emm_tmp is the file that will be created while upgrading an older DB to the latest version (and should automatically been deleted after upgrade), *.emm-journal is the temporarily created "DB of changes" if several changes are in the "pipeline" (e.g. while scraping multiple movies) and not already commited by the code.

Do you run an antivirus software on your system?  

What settings effect the EmberAPI.scanner? 

I did make a copy of both the Ember Media Manager folders on my C drive. I did this to try deleting the database files (the MyVideos24.emm and MyVideos46.emm, etc.) and if Ember wouldn't load, etc. I'd have the original to fall back on. After copying the Ember Media Manager and Ember Media Manager BETA folders, I went into the Profiles folder and deleted both the MyVideos24.emm and MyVideos46.emm and MyVideos46.emm-journal. I then restarted Ember Media and the file sources was empty (as well as the titles in both the Movie & TV sections). I added the file source paths back into the File Source section of the settings and it took roughly 1/2 of a day to add the 1990 Movies and 271 TV shows. I added a few titles using the local disk and it took no time at all. I then deleted the local source path and copied all of the files over to the NAS drive and added 15 new ones to the NAS drive as well. 

After adding the 15 new titles to my NAS drive, clicking on Update Library at 12:34 AM on 05-17-2020 has added 7 titles (out of the 15) so far (current time is 9:12 PM). Looking at the log while the Update Library is running, I'm noting that it scans Title A and it comes back skipped as it has already been scraped (has the extrafanart, extrathumbs, .actors folders) all these 3 have the same time... now before it starts to scan Title B there's anywhere between a 1 minute to 3 minute time lapse between scanning Title A and starting to scan Title B (for example scanning Title A shows a time of 38:35.8 but scanning Title B shows a time of 40:57.7 and it shows "EmberAPI.Scanner" in the Logger section). 

Here's the 3 most recent logs: https://drive.google.com/open?id=1Nzar0T...qECwmUPzNR

The log for 2020-05-16 is after I deleted the MyVideos files and it looks like it is around 3 seconds or so between scanning Title A and Title B, vs. the log for 2020-05-17 (which Update Library is currently running for this log) show around 1 to 3 minutes between each title. 

For more clarification, I copied the contents of both Ember Media folder that were on the C drive over to a thumb drive while Ember Media program was closed. And instead of restarting it should say starting Ember Media after deleting the MyVideo...emm files from the C drive.
Reply
 
Thread Rating:
  • 0 Vote(s) - 0 Average



Logout Mark Read Team Forum Stats Members Help
"Update Library" issues00
This forum uses Lukasz Tkacz MyBB addons.