\Tiny MediaManager\cache\image syncing very slow?
#1
The synchronizing of the files in the folder C:\Lw C\Programme\Tiny MediaManager\cache\image lasts extremely long, it takes about one second or more to copy / sync a single file. There are about 436000 files in that folder.

Is there a way to accelarate syncing / copying?
Windows 10 Home, 64bit | Newest stable Kodi in portable mode
#2
what exactly are you speaking about when saying: syncing very slow?

we do not sync anything in the cache folder?

we either
- build up the cache (rescale/create a thumb for every image found in you library) - 1 sek per file is not bad here
- read the cache if there is a cached image for the requested image from your library
tinyMediaManager - THE media manager of your choice - available for Windows, macOS and Linux
Help us translate tinyMediaManager at Weblate | Translations at 66%
Found a bug or want to submit a feature request? Contact us at GitLab
#3
Sorry for the bad explanation, I meant, synchronizing, making a back up / copy of tMM. So all of the files on my system / of tMM are copied / synchronized / backed up in a proper speed besides of the ones in that image folder of the cache folder, it is so slow (one item / half an item per second) that those files cannot be backed up. I have any idea what could cause the slowness / how I could speed up that.
Windows 10 Home, 64bit | Newest stable Kodi in portable mode
#4
- Turn off any anti virus for that time (or exclude that folder from checking)
- Use windows built-in robocopy to do the job (dont ask, google it, much faster)
- exclude image cache - no need for backup it as i can be regenerated on demand.

we might create more subfolders in v3...
tinyMediaManager - THE media manager of your choice :)
Wanna help translate TMM ?
Image
#5
Quote: - Turn off any anti virus for that time (or exclude that folder from checking)
I had tried that without success.

Quote: - Use windows built-in robocopy to do the job (dont ask, google it, much faster)
About the same speed.

Quote: - exclude image cache - no need for backup it as i can be regenerated on demand.
May be the best way although rebuilding the entire image cache lasts extremely long.

Quote: we might create more subfolders in v3...
Yes, may be there are too many files in a single folder what extremely slows down copying.
Windows 10 Home, 64bit | Newest stable Kodi in portable mode
#6
Hello there....

I use TMM very long time - and i am happy the most of the time Smile
THANK U FOR THAT !!!

Now - it is time to expand my system, because my HDD is full.
So i use a second HDD.

In the options i tell TMM for the Movie direcotiries my two HDDs.
Since i have to use 2 (two) HDDs TMM is terrible(!!) slow refreshing the list.


If i only use ONE (1) HDD and add a new move, TMM just needs a few seconds to check the HDD for new movie.

But since i added a second HDD as a source for movies, TMM takes about 30 Minutes to check for a new movie Sad


Is there anything i do wrong?
Thank u very much !!

Image
#7
the speed depends on several factors:

a) available memory (look at the lower left of the tmm window: if the free memory is constantly below 100MB you should increase the memory setting)
b) speed of the new attached hard drive
c) disc free space where tmm is installed (several SSDs can slow down if there is less than 20% of free space on them)
d) ...

try to figure out what exactly is slow and report back Wink
tinyMediaManager - THE media manager of your choice - available for Windows, macOS and Linux
Help us translate tinyMediaManager at Weblate | Translations at 66%
Found a bug or want to submit a feature request? Contact us at GitLab
#8
(2019-04-07, 19:49)mlaggner Wrote: the speed depends on several factors:

a) available memory (look at the lower left of the tmm window: if the free memory is constantly below 100MB you should increase the memory setting)
b) speed of the new attached hard drive
c) disc free space where tmm is installed (several SSDs can slow down if there is less than 20% of free space on them)
d) ...

try to figure out what exactly is slow and report back Wink
Thank you @mlaggner for your answer and sorry for my late response...

a) There is enough free memory Wink
b) Hard Drive is attached via SATA in a NAS System.
c) TMM is installed on a SSD with 90% of free space.

If i just refresh the new HDD, TMM is very (VERY) fast Smile
If i just refresh the old one (whis is FULL of movies) TMM is working normal and takes about 2 or 3 Minutes to recognise new movies.

But if i just push the refresh button for both HDDs at once... TMM get terrible slow :o

Anyway... i think it depends on the amount of movies/NFOs to search through. And if i choose to search ALL movies on ALL HDDs, TMM slows down...
Since I only use the new HDD, i just refresh the new one... so TMM is very fast and a am happy again Smile
If i have to change something in my old HDD i can choose only that movie and rescrape it (without refreshing the whole HDD).

So everything is okay for me Smile
#9
Well, TMM does not slow down - only the HDD performance/IOs is what counts.
We just walk the whole datasource to find some new files.

- new folders in datasource root are scanned first (before scanning the existing folders)
- datasources are scanned sequentially (one after another)

You could check something for us:
Perform the 3 scans (hhd1, hdd2, both)
And after each scan you should see some overview and timings in logfile like
Quote: 
 Done updating datasource Smile - took 00:00:00.293
 FilesFound 255
 moviesFound 26
 PreDir 43
 PostDir 36
 VisFile 225
 PreDirAll 41
 PostDirAll 34
 VisFileAll 20
Post it here, we might get some clue....
tinyMediaManager - THE media manager of your choice :)
Wanna help translate TMM ?
Image

Logout Mark Read Team Forum Stats Members Help
\Tiny MediaManager\cache\image syncing very slow?0