Slow Scraping?
#1
I've been trying to do a scrape of my movie library and even after running Media Center Manager on it to export all the info, it takes forever. It takes it 5+ minutes per movie, as opposed to about 10 seconds per movie on my windows machine and I get the SMB not found error alot which doesn't happen on the windows box and it keeps wanting a user/pass which there isn't one, they're public shares. Both the laptop and ATV2 are hard wired into the same router, connecting to the same SMB shares. The laptop has XBMC 10.0 R355647 (December 17, 2010), and the ATV2 has XBMC Pre-11.0 GIT: June 23, 2011 (I'm not sure why it says 11.0 GIT as I installed it through NitoTV and it said it downloaded the 10.0-9 DEB file.)

Any ideas how to speed it up? Or should I just scrape the library on my windows PC and export it to the ATV2.

Edit: After completely blasting away the XBMC Apps folder and userdata folders, it seems to be going faster, we'll see what happens. Big Grin
Reply
#2
Well it got about 1/4 of the way thru, and locked up and is back to not scraping right again.

Any idea/suggestions?
Reply
#3
Turn on debugging, restart XBMC, then upload the log to someplace like pastebin.

Also, did you try different scrapers? Are you downloading all the extras when you scrape (e.g. moving ratings, actor thumbs, etc.)?
Reply
#4
I'll try debugging. I already scraped it with Media Center Master, with XBMC export on so it should be able to just grab the XML and images that it already saved right? I turned off downloading trailers, all other settings are set to default. I ran the scrape on the AppleTV and my laptop (now current release) at the same time, same settings, same scraper. (My internet is 30Mbps down, 25 up so my connection wouldn't be lagged down by this at all). The laptop finished in under 30 minutes, the ATV crashed after about an hour and was only 1/4 of the way through.
Reply
#5
I just noticed a glaring bug in the log files:

22:04:21 T:111603712 M:111714304 DEBUG: OpenDir - Using authentication url smb://REMOVED:REMOVED@MATTHEW/Movies/Funny%20People

It prints the username and password in plain text! (Which it shouldn't need anyway, its a public share, theres a bug there that the windows version doesn't have).
Reply
#6
MatthewH12 Wrote:I just noticed a glaring bug in the log files:

22:04:21 T:111603712 M:111714304 DEBUG: OpenDir - Using authentication url smb://REMOVED:REMOVED@MATTHEW/Movies/Funny%20People

It prints the username and password in plain text! (Which it shouldn't need anyway, its a public share, theres a bug there that the windows version doesn't have).

that's fucked up.
Reply
#7
Ned Scott Wrote:that's fucked up.

Yeah that really needs fixing, and fast. I wonder how many people posted their logs not realizing it did that?

Edit: Latest AE build does it too.

AE Build seems to have less scraping issues, though movie scraping is still unbelievably slow, as is the update library function, TV isn't as bad. I suspect this will be improved though in future releases.
Reply
#8
Won't happen. as it is a debug log and not meant for public sharing. Removing usernames and password will make it 100% harder to debug.

FYI, LOGNOTICE and LOGINFO will not log passwords.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not PM or e-mail Team-Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#9
vdrfan Wrote:Won't happen. as it is a debug log and not meant for public sharing. Removing usernames and password will make it 100% harder to debug.

FYI, LOGNOTICE and LOGINFO will not log passwords.

Yeah, but a warning would have been nice.
Reply

Logout Mark Read Team Forum Stats Members Help
Slow Scraping?0