[Solved] OpenELEC has VERY slow library updates on my Pi 2
#1
Any REAL fix or update on this? I just purchased a Pi 2 and installed OpenELEC. I selected the autoupdate so everything should be updated to current.

I added a source to my existing SMB share. This share works perfectly fine with my Win 7 & Kodi, wicked fast. But OpenELEC does NOT like it for some reason. My structure is "smb://media/TV Shows". I've got 61 shows with numerous episodes and seasons for each. I had to just quit the library update as it was still going after about 2.5 hours.

I see my wireless bridge light hardly going crazy, so it's not the network. I got a Pi 2 so it shouldn't be the hardware, and CPU usage is ~0 most of the time spiking sometimes to ~20%.

The yellow activity light is constantly on the whole time. I'm using a class 10 microsd card. I've tested the reads around 17 and the writes around 5 or so. On another card that was tested even slower I was able to unpack Raspbian to it's ~2GB size in about 20-25 minutes. So accessing the cards seems okay.

This thread seems to blame it on the folder structure.
But I don't want to restructure my whole network share just because OpenELEC has a bug.
Anyone have some insight?
Reply
#2
what does very slow equates to in terms of real time duration? 10 mins, 10 hours?

edit: nvm...saw your reference of 2.5 hours...which scraper are you using? best just to get a log uploaded
Reply
#3
I have similar folder structure using Samba, have a lot more shows and my library updates take about a minute.

I would start with something simple. Run a long network cable to your switch and test the library update. An activity light doesn't show you the speed in which data is transmitting.
Reply
#4
(2015-05-29, 07:46)katsup Wrote: I have similar folder structure using Samba, have a lot more shows and my library updates take about a minute.

I would start with something simple. Run a long network cable to your switch and test the library update. An activity light doesn't show you the speed in which data is transmitting.

But this is the same network I used with my Win 7 box. I just pulled the network cable and HDMI cable and put those in the Pi2. An ethernet jack is going to my wireless bridge router connecting to my wireless router in the office. I know it's a good setup, as I can stream movies well with my Win 7 box. The activity lights would always be on and get about a 10-25Mbit bandwidth, unlike while using the Pi when updating. The activity lights flash for half a second, waits a second, flashes again for half a second, waits 2 seconds, etc. I know the network is good.

I tried to disable downloading FanArt and Thumbs, but still the same slowness in the library update. And as a timeframe it took probably 10 minutes to update Quantum Leap, and still didn't get all 95 files indexed until I canceled it. I also enabled NFS shares and deleted the SMB share and started over with NFS, but still the same issue. I'm also using the TVDB scraper.

The only thing I can see as an indicator is the yellow activity light is always on, like it's constantly reading/writing to the flash.

I tried to overclock the hardware, but that didn't work, it still says the CPU is running at 600MHz, even while watching a Quantum Leap episode. My config.txt settings had no affect.
Reply
#5
A default (no overclock) Pi 2 should be able to scan things in fairly reasonably. I'm not sure about TV shows, but I did a test with 1,000 movies (with NFO files and images next to the files) and I think it was about half an hour to scan them all in.

Can you get us a debug log (wiki) of when you attempt a scan? Just let it run for 10 or so minutes and cancel the scan, then upload that log.
Reply
#6
The quality of the SD card makes a massive difference with the Pi2. I originally had a class 4 card. Big mistake. Getting a class 10 card in, it felt like a completely different system.
Reply
#7
The class of sdcard isn't important, it's the 4K read/write speed.
Run crystalmark to see if your card is a dud. See here for comparisons.
Reply
#8
As per Ned Scott, upload a debug log (wiki) of your scan. Also disable cast (actor) thumb downloads before scanning.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#9
(2015-05-29, 08:47)Ned Scott Wrote: Can you get us a debug log (wiki) of when you attempt a scan? Just let it run for 10 or so minutes and cancel the scan, then upload that log.

Okay, here's my log.

Thanks.
Reply
#10
My card test. This was done on a card I pulled with OpenELEC on it. I suppose it's only testing the FAT32 partition.

Image

http://postimg.org/image/xlcfy60ff/
Reply
#11
(2015-05-30, 06:30)Skram0 Wrote: My card test. This was done on a card I pulled with OpenELEC on it. I suppose it's only testing the FAT32 partition.

Image

http://postimg.org/image/xlcfy60ff/

I suspect this is the issue. The card is pretty slow (especially the 4K writes).
Reply
#12
@Skram0: Aside from your poorly performing SD card which will slow everything down, you'd also be better off exporting your library from the Windows client so that the NFO files are written alongside each of your movies and episodes, then your Pi wouldn't have to make the slow trip out to the internet to download this same information from a web site - your TVDB queries are taking between 3 and 5 seconds per episode. Or consider using MySQL.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#13
(2015-05-30, 12:44)Milhouse Wrote: @Skram0: Aside from your poorly performing SD card which will slow everything down, you'd also be better off exporting your library from the Windows client so that the NFO files are written alongside each of your movies and episodes, then your Pi wouldn't have to make the slow trip out to the internet to download this same information from a web site - your TVDB queries are taking between 3 and 5 seconds per episode. Or consider using MySQL.
I've ordered one of these new microSD cards. I'll clone the data over and see how that goes.

Storing pre-scraped NFO files in the directories seems like a patch to the problem to me. I'd rather fix it than patch it. And having one computer rely on another for a database is just asking for headaches. I don't know what my ultimate system plan will be, but at least I know I want things working as they should.

I'll report back on my new microSD endeavors.
Reply
#14
(2015-05-31, 10:08)Skram0 Wrote: Storing pre-scraped NFO files in the directories seems like a patch to the problem to me. I'd rather fix it than patch it.

NFOs aren't a patch, they're the most efficient solution when searching for metadata. Having every client go out to the internet to find information you've already found previously is always going to be slower than working with a pre-generated NFO, so there's really nothing to "fix" (other than the performance of your SD card). Even with the best SD card, locally stored NFOs (and artwork) will always be quicker, not to mention more consistent in terms of quality - that's why it's also a good idea to generate the NFOs and artwork from third party media management tools (Ember, Sickbeard etc.), so that you only do it once not multiple times.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#15
(2015-05-31, 11:44)Milhouse Wrote: NFOs aren't a patch, they're the most efficient solution when searching for metadata. Having every client go out to the internet to find information you've already found previously is always going to be slower than working with a pre-generated NFO, so there's really nothing to "fix" (other than the performance of your SD card). Even with the best SD card, locally stored NFOs (and artwork) will always be quicker, not to mention more consistent in terms of quality - that's why it's also a good idea to generate the NFOs and artwork from third party media management tools (Ember, Sickbeard etc.), so that you only do it once not multiple times.

Confirmed. Below is my new card's benchmark. A lot better writes on the 4K.
I installed OpenELEC from scratch using Noobs on the new card and tried the library update again. It whizzed through the whole Quantum Leap series in 20-30 seconds. And that's with album art and thumbnails.
For the heck of it I formatted my old card and started from scratch again doing exactly the same process. The library update was very slow again using the old card.
Man I wish OpenELEC had a quick benchmark test during install and a warning to those people who have slow writes.

Thanks to everyone for their knowledge and help.


Image
Reply

Logout Mark Read Team Forum Stats Members Help
[Solved] OpenELEC has VERY slow library updates on my Pi 20