[giftiger_wunsch]

DEBUG LOG: https://gist.github.com/anonymous/005b500b900a9b80c0fe

Hi,

I was previously using raspbmc and have just upgraded to osmc alpha 4, and I'm trying to replicate my previous setup on osmc. I've added my TV and Movies sources and was impressed how much faster it rescraped the data, actually, but it's only scraped some TV and movie data: it's missing data for 29 movies (out of 130-odd I have available) and 61 TV Shows (out of 84 I have available).

An example TV programme it hasn't scraped is 12 Monkeys, and an example which it has scraped is American Dad. The debug log shows it skipping all the ones it's done already when it's asked to update the library, but it makes no reference to any of the others. The file list shows all the files, including missing ones, and I have confirmed that I can find it on TVDB (that's the scraper I'm using) and that it is present in the source.

For extra context, I have mounted an nfs filesystem at /mnt/media in the fstab and added /mnt/media/TV and /mnt/media/Movies as sources in the interface.

I can see in the debug log that it's running a couple of queries on the db but it seems to just be checking view counts, so I can't actually see any sign that it's recognised that it's meant to be updating the other programmes at all, or made any attempt to do so.

Thanks in advance for any advice.

[mkortstiege]

What happens if you refresh from the files node or from within the video info dialog? Permissions issues? Network issues while initial scraping?

[giftiger_wunsch]

I've just tried via the video list again (I did before but didn't collect a log). Most of the extra log info seems to just be logging me moving around the menus, but maybe there's something buried in there: https://gist.github.com/anonymous/53dcbea5d270f7f80259

EDIT: Actually what I do notice is that this time I only see a single 'Skipping %s due to no change' message, rather than a list of them, which makes me think it hasn't even registered that the others should be checked.

EDIT AGAIN: And it's the one I had highlighted when I looked for changes; looks like I've done it wrong; I think in previous versions triggering an update from the context menu on the menu triggered a full update and now doesn't. I'll try one of the unscraped shows, via the file list.

[mkortstiege]

Btw, "Update Library" from menu (not context) should work just fine. You' ve double checked the permissions? There must be reason a few were skipped on initial scan.

Could you please try to start fresh (without userdata / myvideos90.db) and post the entire log?

[giftiger_wunsch]

Okay, I've just tried updating the library in the context menu from Videos > Files > 12 Monkeys, and it still hasn't scraped.

Log: https://gist.github.com/anonymous/50adf6771404258ca901

The name of the programme I tried to update doesn't appear in the log and I don't see any obvious network issues, so I'm not sure what's going on there.

[giftiger_wunsch]

You' ve double checked the permissions? There must be reason a few were skipped on initial scan.

Yeah permissions are fine; I actually have a cron job on the server hosting these which sets all permissions of the library to the right ones. I can also run the files via the files menu, and I have all the data fine on xbmc on my old pi.

Could you please try to start fresh (without userdata / myvideos90.db) and post the entire log?

I'll give that a shot now and post the log.

[giftiger_wunsch]

So I suspect I've done something wrong; when I moved the MyVideos90.db file out I got an empty directory structure under TV Shows, and lost the context menu item to scrape. I tried triggering a scrape via jsonrpc but nothing happened. The debug log seems to be full of SQL errors where it expects the database to exist and it's, uh, gone.

I suspect this isn't quite what you wanted me to do; can you clarify?

New log: https://gist.github.com/anonymous/c56eeadda640255db154

[mkortstiege]

You've nuked the library without restarting Kodi? Stop Kodi, nuke MyVideos90 and start Kodi. This will create a clean MyVideos90.db. Now you will have to re-assing the content to your sources and let it scan. Btw, please post complete logs, not just snippets.

[giftiger_wunsch]

You've nuked the library without restarting Kodi? Stop Kodi, nuke MyVideos90 and start Kodi. This will create a clean MyVideos90.db. Now you will have to re-assing the content to your sources and let it scan. Btw, please post complete logs, not just snippets.

Yeah, I realised my error and am doing that now; it now seems to be rescraping. I'll repost the log when it's finished, which may take a while.

[mkortstiege]

Please make sure debug logging is enabled during the initial scrape session

[giftiger_wunsch]

Please make sure debug logging is enabled during the initial scrape session

Yep, debug logging is enabled I can see plenty of debug logging going on wrt scraping individual episodes at the moment, so hopefully the new log will have plenty of info to go on.

[mkortstiege]

.. or it's just working as intended ;D

[giftiger_wunsch]

.. or it's just working as intended ;D

So far, yeah. Actually since I don't see any errors and so far it's picking a seemingly random order (presumably not actually random, but definitely not alphabetical) and including several which were successfully scraped last time, I'm starting to think the initial scrape may have been interrupted and not finished, but I'll have more info when this one finishes. I can certainly see various programmes being recognised, matched by title, and episode names found based on regex (all my media is in the form Sxx Exx.xxx so that helps) thus far.

Out of interest, if the reason behind this is simply that the initial scrape is interrupted, can an individual series be forcibly rescraped without having to resort to nuking the database and rescraping everything? Or is that what should have happened when I triggered an update anyway?

[mkortstiege]

https://github.com/xbmc/xbmc/pull/6646 (already merged for 15.x) takes care of the abort initial scan issue. A manual workaround/fix would be to invalidate all path hashes before triggering "Update Library" again so it does not have to re-scrape what is already there. Will see if we can get it backported to Helix in order avoid scan issues.

[giftiger_wunsch]

Okay, the full scrape completed, and it's managed to fetch 67 episodes out of 84 this time. It's an improvement, but it's still missing several.

Here's the log, which was impressively too large for github: http://private.programmingcentre.com/dumpbox/kodi.log

EDIT: Ignore me; I've just compared this to the actual list of TV shows in the source and it seems to have gotten them all. I guess any extras I have in the old raspbmc installation are ones which have since been deleted from the source.

Looks like the rescrape fixed the problem for me. Thanks!