Kodi Community Forum

Full Version: Helix library scanning: skipping directories on ExFat disk
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4
Solved: mkortsiegefound the culprit in mtime not being updated for folders on (some) ExFat devices thus not working wth the new fasthash method for scanning for new files.
From 14.2 onwards (as of the beta from feb 17th) you can force the old scanning method by inserting
Code:
<advancedsettings>
  <videolibrary>
    <usefasthash>false</usefasthash>
  </videolibrary>
</advancedsettings>
into your advancedsettings.xml.

Thank you!


Hello everyone,

since beta 3 of Kodi 14 there seems to be a problem with library scanning for me. It just doesn't add any new files I placed in the directories.
When I bring up the context menu on a TV show, enter TV show information and "refresh for all items" it does find them however. It doesn't when I just "scan for new content" from the context menu of a show.
In the log it says it is skipping the directories due to no changes, even though I did change them.

I couldn't find a thread to this issue so I created one. Debug log just opening Kodi and having an unsuccessful startup scan is here.
If you think of anything I might've done wrong please tell me, but I didn't change anything from beta2 which worked for me.

Macbook Air 2012, OSX 10.10.1, Kodi 14.beta5

Thank you very much!


Edit: Sorry for misleading, it seems it was OSX's 10.10.1 update that created the bug, I changed the thread title. I really didn't think that it could affect something like the library...
Edit 2: I removed the reference to 10.10.1 again from the title. On the one hand not everyone on 10.10.1 is affected, on the other hand 10.10.2 didn't change anything for me and there is at least one user who seems to have the same issue under windows.
Edit 3: As mkortstiege found out the root of the issue seems to be that OSX's implementation of ExFat doesn't update mtime. Therefore the new fasthash scanning mechanism fails to see that the folders were modified and doesn't pick up new episodes. Workaround and further investigation is in the works. Thank you, mkortstiege!
Log looks strange. There should be no "fast12341234" like hashes anymore. Could you please scan for new changes (from context menu) on your media sources? This hopefully fixes the somehow fishy hash values once for all.
(2014-11-26, 10:41)mkortstiege Wrote: [ -> ]Log looks strange. There should be no "fast12341234" like hashes anymore. Could you please scan for new changes (from context menu) on your media sources? This hopefully fixes the somehow fishy hash values once for all.

Thanks for the prompt reply!

I scanned for new content from the context menu of the sources (in file view, that's what you meant, right?) but it didn't find the new files either.
Log shows me verifying which shows have missing episodes, then doing the mentioned scans from context menu, then going back to TV shows to see the additions still missing.
Hm. Yeah, that's exactly what I had in mind. What kind of filesystem is used on '/Volumes/media' and how are your media sources setup within the set content dialog?
FS is ExFat
(OT: I need the OS X/Win compatibility but I am seriously considering moving away from it again as OS X has some issues with it which made me disabling spotlight indexing for the drive a couple of months ago. I am open to suggestions.)

tv media: "contains TV shows", The TVDB on top, local info second row, single TV show and exclude are disabled.
movies media: "contains Movies", The Movie Database on top, local info second row, separate folders and exclude disabled, scan recursively enabled.
Log states that your movies are in separate folders that match the movie title, so you should check that option. As for the shows, I have no idea why it's missing those. Unfortunately i have no ExFat formatted drive here to test with.

Could you please try to remove your MyVideos90.db file (backup it so you can restore the current library)? On app start an empty one will be re-created and you have to re-setup the content for your media folders. I'd like to know if the db is broken or if it's more general issue.
Huh, that is strange:
I removed the MyVideos90.db file and restarted Kodi. When starting up it created a new one - but it wasn't empty. Instead it was the state of my library approximately 3-4 weeks ago, complete with movies I had deleted in the meantime, but it did add everything I added since.
Stupid me had disabled debug before so I will try to recreate with log.

Edit: It did the same thing again. Took me some time to paste the log, as pastebin.com didn't accept it for being to long and .ca timed out on me.
that's because it upgraded you old MyVideos**.db to the new version. To start fresh move out all the other MyVideos**.db files to a safe place and restart. Then it should start fresh
ah, yeah, that's kind of obvious....sorry for that, tunnel vision when following support orders...
OK, so I did a real rebuild of the DB but the issue remains. Just in case I created a new log.
I enter and confirm that the initial scan does not add the new episode. Then I choose "scan for new content" on the series which didn't work before, but now it does add the episode.

Edit: it does not seem to be FS dependent, I copied one directory to my internal hfs+ SSD, added another TV source for it and it did not add the files to the library...


(quick OT:

(2014-11-26, 11:30)mkortstiege Wrote: [ -> ]Log states that your movies are in separate folders that match the movie title, so you should check that option.

Well, not all of them are, so I figured keeping it disabled but scan recursively should give me everything, right?)
Odd. New episodes are picked up as intended here (running on Linux). Will give it a try on my mac later today.
I guess we're talking about the big bang theory located in '/Volumes/media/Video/tv/', right?. Content was set on the tv folder only or did you set content on the single parent show folders?
(2014-11-26, 17:52)mkortstiege Wrote: [ -> ]I guess we're talking about the big bang theory located in '/Volumes/media/Video/tv/', right?. Content was set on the tv folder only or did you set content on the single parent show folders?

yep, I tried with the big bang theory episode. To test I remove it, clean the library, put it back and try different kinds of scans. So far initial scan at startup AND context menu scan on the show didn't work, only "refresh" from inside show information did. But the problem is with every show, I also ran tests with 3 others after noticing the issue in the first place after I added 5 or 6 episodes for different series.

On the last test however rescan on the show worked.
Now I tried again (log) and it was back to the behavior I described: Neither the initial scan nor the scan on the library, the source or the show worked, only after refreshing from the show info window did it pick up the last episode.

Content is set for '/Volumes/media/Video/tv/' as a TV show source, inside there are folders for shows and mostly subfolders for seasons inside of them. That's how I had it for ages.

Yeah, it seems strange to me too, as I said it worked fine until beta 3. Thinking about it I think I upgraded to 10.10.1 around the same time but unfortunately don't have a bootable backup of 10.10.0 to test whether this might have changed something...
The re-scan after remove is another issue related to a missing pathhash invalidation. Sounds like the new OSX release is handling the ctime/mtime stuff differently. The VideoInfoDialog is emptying the path hash, hence it scans fine. Will investigate.
(2014-11-26, 19:02)mkortstiege Wrote: [ -> ]The re-scan after remove is another issue related to a missing pathhash invalidation. Sounds like the new OSX release is handling the ctime/mtime stuff differently. The VideoInfoDialog is emptying the path hash, hence it scans fine. Will investigate.

Hmm, ok, so to really test the issue I should create a new database for every test run, absent genuinely new episodes to scan?
It did however happen with genuinely new episodes, this is when I noticed the problem.

I am sorry to maybe have led you astray, I seriously didn't think about the OS X update until my last post as I set it to download it automatically and just confirmed it when it wanted to reboot. But it seems that it is indeed the OS, as I just tried with beta2 and the same issue arises.

So again, I am sorry, I will change the thread title accordingly...
Pages: 1 2 3 4