Unable to transfer large files to external USB drive connected to RPI2 with Kodi 17.3
#1
I’ve got Kodi running on a Raspberry Pi 2, OS is OpenElec V 8.0.4, Kodi build 17.3 June 2[sup]nd[/sup], 2017 =11ptI’m using an old 2.5” 250G laptop drive to store media, plugged into the RPI via USB.   The drive is seen properly in Kodi, which has no problem reading files from the disc.  I have not tried having Kodi write something to this disc, would not know how to get it to do that. 

I’m not  Linux guru by any means, but I know a little about it.  The RPI setup is replacing a MythTV box that I used for several years before. 

I’ve got the RPI set with  static IP on a wired network connection.  Apparently the “out of the box” configuration of Kodi is already set to allow my external drive to be shared out to the network.   If I open any Windows machine on the network I don’t automatically see the RPI share, but if I point Windows Explorer at the IP address of the RPI I can access the drive (no username or password prompt requested). 

Not having the RPI share show on the network is a small issue – I’d love to configure it so other machines on the network can see it without having to know the IP, but that’s minor.  I can map it to a drive letter on my Win machine if I need quick access to it. 

The big problem is with file transfers, which is critical.  I often create videos for the family or for a local theater company, and I need to be able to transfer those files to the Kodi box so I can preview them on our main TV. 

If I try transferring a small file across the network from Windows machine to RPI  share it works fine.  I can transfer a small text document or a photo, for example.  However, if I try to transfer a larger file, it fails – always.   

The Windows error is “An unexpected error is keeping you from copying the file.”  This is followed by error code 0x8007003B: An Unexpected Network error has occurred. 

Googling that points to discussions involving Windows Server 2012 and disabling the Windows Search service on that server – but I’m not using Windows Server. 

I haven’t quite narrowed down the break point between file sizes that will transfer and those that won’t, but it appears as I get over 20Mb the process starts failing before completion.  Way smaller than any video. 

I have a number of other windows machines on the network, most with shares, and I have no problem transferring enormous files (gigabytes) back and forth normally – whether the source and target are on wired connections or wireless – all it takes is time. 

I have another RPI on the network which I use as a dedicated music server.  That RPI is set up almost identically, with an external drive shared to the network.  It lives right next to the Kodi box in the same closet, on the same network (although it is wireless).  I have no problem transferring large files to it. 

I assume the permissions on the RPI/Kodi share are set properly since I CAN write files to the share over the network.  I know the drive is not full – it’s a 250G drive that is almost completely empty. 

Within Kodi, it has no problem reading files on that drive. 

One unusual thing… If I look at System Information it reports a partition (/dev/sdb1) with a 1TB drive on it – which doesn’t exist.  There are only two storage devices attached – the flash drive (a 16G SD) that runs OpenElec and the external USB 250G drive.  Nothing else is connected, but Kodi reports this phantom 1TB drive. 

I’ve done a small amount of work with smb.conf in past Linux setups, I’d love to look at it here but I have no idea how to get Kodi to let me see the command line, even. 

Where do I start to troubleshoot?  Is there a log file somewhere that might be useful? 

Thanks, 

Todd
Reply
#2
Upgrade to LibreELEC, as OpenELEC is no longer maintained.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#3
Seriously?  The OpenElec site says nothing about that, the latest update date on my media center is not very long ago...

Sure, I can try LibreElec, given I have nothing to lose since I've done very little custom configuration at this time.  We'll give that a try....
Reply
#4
(2018-07-08, 22:29)tmaddison Wrote: Seriously?

Based on all the evidence yes, seriously.
  • OpenELEC 8.0.4 was released in June 2017, a year ago
  • OpenELEC 8.0.4 is based on Kodi 17.3 when the last Kodi 17.x release is Kodi 17.6, so you are missing 3 Kodi bug fix releases
  • OpenELEC 8.0.4 has no Raspberry Pi3+ support - this popular hardware was released 3 months ago in March 2018
  • There have been no public commits to the OpenELEC development Github repository in 14 months (May 2017)
  • Nobody is replying to posts on the OpenELEC forum - the only posts are from spammers
  • The sole remaining OpenELEC developer is nowhere to be seen
The only conclusion that can reasonably be drawn from all of this is that OpenELEC is now abandoned.

(2018-07-08, 22:29)tmaddison Wrote: The OpenElec site says nothing about that, the latest update date on my media center is not very long ago...

Yes, if (as it appears) the developer has decided to move on and no longer intends to update OpenELEC, then posting a message to that effect would be the right thing to do.

However maybe he has plans and will reappear at some point, but who is to say he won't disappear again for an extended period leaving users without support, updates, etc. If it happens once, it's likely to happen again.

Full disclosure: I'm a member of LibreELEC, and I take no satisfaction from this post, but it needs to be said as you're clearly in the dark as in fact we all are regarding the fate of OpenELEC.

(2018-07-08, 22:29)tmaddison Wrote: Sure, I can try LibreElec, given I have nothing to lose since I've done very little custom configuration at this time.  We'll give that a try....

LibreELEC is a straight upgrade path for OpenELEC.

LibreELEC 8.2.5 is based on Kodi 17.6, fully supports RPi3+, and has an active forum and development community.

I don't know if LibreELEC will fix your issue, but at least you'll be in a better position to ask questions.

You'll also find the latest LibreELEC 9.0/Kodi 18 test builds here which might be worth trying if LibreELEC 8.2.5 doesn't solve your problem.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#5
Thanks, that all makes sense.  I'll give it a try and see if that fixes the issue, will post success/failure here....
Reply
#6
OK...  So I switched to LibreElec, which looks nice and is apparently working for all the things I've tried, but no difference in this problem.  Any file transfer across the network to an external drive plugged into the device still fails.

Since doing this I've tried the following in an attempt to get more data on the conditions...

1)  Changing the external drive.  I've tried a 256G USB flash drive (exFAT formatted) and a 1TB laptop drive in an external USB case (formatted NTFS).  No change, fails.

2)  Copying to the SD card that LibreElec is running from instead of the external USB.  No change, fails.  To get the LibreElec update file onto the machine to update it I used a small USB drive and then copied that from the USB drive to the "Updates" folder using Kodi's file manager.

3)  Tried using Kodi's file manager to "pull" the file from my Windows machine instead of "pushing" it from Windows to Kodi.  This failed because I can't get Kodi to connect to my Windows share over the network.  I get an error box that says "operation not permitted".

No idea if that's a different problem or a symptom of the same thing, but I have three Windows machines on the network and all three can access shares from all three of their drives with no problems, transferring large files back and forth.  The only machine I cannot get to access the Windows share on my main computer successfully is the LibreElec box...
 
How do you delete an unavailable share?  I accidentally clicked "yes" at the end of the process at one point after entering a typo in an IP address and now I seem to permanently have an entry in the "browse for new share" listing - nothing I do seems to get rid of it.

... and now I'm at a dead end....

I suppose I can just "sneaker-net" files to it, if I plug in a USB drive it appears to be fine transferring files between drives using Kodi's File Manager, but that seems a bit neanderthal....
Reply
#7
Todd,

   Your problem reminds me of the problems I had when they changed the default security setting for Samba in Kodi. Do you know about this?

SMB1 used to be the default on older versions of Kodi, but must now be enabled manually:

Settings > LibreELEC > Services > Samba > Minimum supported protocol > SMB1

Or, you can enable SMB2 support on your Windows system.

Regards, Clay
Reply
#8
Thanks for the direction....

So.... this link implies that Windows considers SMB1 and 2 to be security risks, implying that SMB3 is what it supports.

https://www.tenforums.com/network-sharin...3-a-2.html

I tried setting min and max SMB protocols to both SMBv3, rebooting.   No change, large file transfer fails.

Then I tried SMBv1 (reboot) and SMBv2 (reboot), both made no change.

I also tried setting min protocol to SMBv1 and max protocol SMBv3, reboot again, test again.  No change...

I have not tried explicitly checking/enabling SMBv2 on the Windows machines - that process appears to be somewhat involved in PowerShell, if Win uses SMBv3 it would be nice if I could just get Kodi to use SMBv3 (since it says it can) and have them play nicely together....


None of the changes above resulted in the Kodi box becoming visible in the Windows network as well.  It is, however, visible as a "Media Device" so if I wanted to play content from it on other devices around the house apparently I could do that.

Settings are now back to default, which is "none" for both.

Dead-ended at the moment...

Am I the only person who transfers video content across a Win network for storage on an external drive connected to a Kodi box?  Seems like that would be a pretty common use-case...
Reply
#9
(2018-07-10, 02:04)tmaddison Wrote: So.... this link implies that Windows considers SMB1 and 2 to be security risks, implying that SMB3 is what it supports.

Microsoft considers SMB1 to be a security risk, and they should know, as they invented this travesty of a networking protocol. And the experience of the NHS after WannaCry should confirm it for you.

Microsoft are now doing everything in their power to disable SMB1 in Windows, along with other security holes in their general networking policy which includes enforcing authentication when accessing SMB shares (anonymous/guest access to and from shares is being disabled).

So you may now find (if you are using Windows 10 with all recent updates) that you are not able to connect LibreELEC (the client) to a Windows share (ie. the server) without adding an account and password to the share in Windows, and then configuring the same username/password in LibreELEC (specifically in Kodi, as this is the real client from a Samba perspective).

Similarly, you may now need to enable authentication in LibreELEC Settings > Services > Samba > Use Samba Password Authentication in order to access the LE share (the server) from Windows (the client).

(2018-07-10, 02:04)tmaddison Wrote: I tried setting min and max SMB protocols to both SMBv3, rebooting.   No change, large file transfer fails.

Then I tried SMBv1 (reboot) and SMBv2 (reboot), both made no change.

I also tried setting min protocol to SMBv1 and max protocol SMBv3, reboot again, test again. No change...

There are two places you can change the min/max protocol, one affects Kodi which is the client (Services > SMB client) and the other the LibreELEC Samba Server (LE Settings > Services > Samba). If you changed the Kodi (client) settings it will have no effect on your attempts to connect to a LibreELEC server share from Windows (and vice versa).

So what I'm saying is, we don't know what setting - client or server - you are changing, or what you are using as the client (Windows? LE?) or the server (Windows? LE?) in your test. You need to be very specific, as we have had people tinkering with LE Samba Server settings and wondering why it has no effect on Kodi accessing a Windows PC share...

Also, what does "fails" mean - do you get an error message? What does it say?

(2018-07-10, 02:04)tmaddison Wrote: I have not tried explicitly checking/enabling SMBv2 on the Windows machines - that process appears to be somewhat involved in PowerShell, if Win uses SMBv3 it would be nice if I could just get Kodi to use SMBv3 (since it says it can) and have them play nicely together....

There's a good chance SMB1 is disabled (Microsoft will disable this during some updates, depending on when it was last used), but SMB2 and SMB3 will be enabled.

(2018-07-10, 02:04)tmaddison Wrote: None of the changes above resulted in the Kodi box becoming visible in the Windows network as well.
Highly unlikely that it ever will. You see, "network browsing" (which is the feature that allows a Samba client to found by a Windows PC, and for the Samba client to find other Samba/Windows shares) is based on an SMB1 related protocol, and since SMB1 is being disabled pretty much everywhere then so is "network browsing". Windows PCs continue to be able to browse the network as they use a different, proprietary protocol which is not (yet?) supported by Samba.

Until Samba implements non-SMB1 based "network browsing" support you'll need to enter the IP address of the Kodi box ie. \\192.168.0.200 if you want to browse the shares.

(2018-07-10, 02:04)tmaddison Wrote: Settings are now back to default, which is "none" for both.

Good.

For the client this means it will negotiate the best SMB connection available from the server, starting with SMB1 and negotiating up to SMB3. In future we may set SMB2 as the default minimum protocol so that an SMB1 connection is not even attempted by the client.

For the server, the default min/max protocol settings should be SMB2 and SMB3. You do not want to enable SMB1 in the LE server.

(2018-07-10, 02:04)tmaddison Wrote: Dead-ended at the moment...

Am I the only person who transfers video content across a Win network for storage on an external drive connected to a Kodi box?  Seems like that would be a pretty common use-case...

No, I do it all the time and have no issues.

We also have many thousands of users on LibreELEC 8.2.5 that are using Samba in a Windows network without issue.

It's entirely possible this is a configuration issue thanks to Microsoft constantly moving the SMB goalposts every few months, in which case just make sure you have a username/password set on your Windows share(s) (don't use your admin account, create a new unprivileged account and assign it to the share, either read-only or read/write as required). Then configure your source(s) in Kodi with the username/password of the unprivileged Windows account.

When accessing the LibreELEC shares from Windows, the default username/password is libreelec/libreelec.

The LE Samba server supports SMB1, SMB2 and SMB3, but SMB1 is disabled by default (and should remain disabled unless you really, really cannot upgrade your clients, and I don't think you are in that position).

If you continue to experience network stalls then log into your RPi with ssh and upload the contents of the system journal with journalctl -a | pastebinit and paste the link.

Weird network behaviour can be caused by networking hardware, so try a different switch, a different cable, even a different router/switch port.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#10
tThanks for all the detail!   Sorry for lack of clarity, I THOUGHT I was being clear but it’s always different on the other end of the conversation…

First – after doing all of the below, the problem appears to be fixed.  However as you can see I have no idea why anything has changed…

To clear up some lack of clarity… 
  1. The share I’m attempting to transfer files to is on the RPI, an external drive connected via USB.
  2. I’m attempting to transfer from various Windows machines (three in total) across a wired network.  That network works perfectly in all other aspects I can tell, including sharing large files between the three Windows machines with both wired and wireless connections, with no problems.
  3. “Failed” means I can access the share, view it, even delete files from it and transfer small files (under 20Meg) to it, HOWEVER if I try to transfer anything larger than that then the transfer fails - I get an error message on the windows machine I’m attempting to make the transfer from.  This error usually occurs about 10% into a transfer (using a 1G test file).
  4. The Windows error is “An unexpected error is keeping you from copying the file.”  This is followed by error code 0x8007003B: An Unexpected Network error has occurred. 
 I have attempted changing the Kodi/Services settings for SMB to all possible combinations from “None” for both to “SMBv3” for both – rebooting between each attempt - no change in failure. 

Since reading your response and understanding the difference between the Kodi and LibreElec settings, I have also changed the LibreElec settings. 

I enabled the Samba Password Authentication – first leaving username/password blank, then entering a username and leaving password blank, then entering a username and password.  Reboot between each test. 

All this did is make it so that I could not access the share at all – Windows popped up a parameter box for username/password, in all cases I entered the same combo I had just set up in LibreElec, and in all cases Windows rejected that as invalid and would not allow me to make a connection. 

Just for the heck of it, I also entered username “kodi” and blank password (rejected) and username libreelec password libreelec – rejected. 

I then was about to give up for the time being so I went back and re-set everything to default, including turning off Samba Password Authentication. 

I then tested from my Windows machine to see if that restored access to the Librelec share – it did. 

That access seemed far more “snappy” than before – directory listings coming up instantly rather than waiting as it had in the past – so just for the heck of it I tried a large file transfer (1.2G). 

It WORKED. 

I’ve now tried several transfers, from two different machines, and everything appears to be working as it should.   

So now, here I am in theory with the same configuration I had when I started this, when file transfers were reliably and repeatably failing, but now it works… 

Perhaps turning Samba Authentication on and off again “fixed” something?  I don’t know what else I really did in this round of attempts aside from try multiple username/password combinations. 

I’ll give thanks at the moment and hope it stays that way! 

Todd
Reply
#11
(2018-07-10, 14:27)tmaddison Wrote: All this did is make it so that I could not access the share at all – Windows popped up a parameter box for username/password, in all cases I entered the same combo I had just set up in LibreElec, and in all cases Windows rejected that as invalid and would not allow me to make a connection.

Odd, as LibreELEC authentication does work. However Windows does have an unfortunate tendency to "cache" information, so rebooting the Windows PC might be a good idea.

I don't know what version of Windows 10 you are using but you may find that Windows 10 requires authenticated share access when accessing LibreELEC shares in future, depending on what updates Microsoft decide to send to your machine. So if it all stops working again, first of all try enabling authentication in LibreELEC.

(2018-07-10, 14:27)tmaddison Wrote: Perhaps turning Samba Authentication on and off again “fixed” something?  I don’t know what else I really did in this round of attempts aside from try multiple username/password combinations.

Unlikely, as that's a fairly simple and reliable procedure. My guess would be that the Windows SMB and Linux Samba services were somehow out of sync with each other (see above about Windows caching information) and rebooting various clients/servers eventually got them back in sync.

SMB/Samba is an incredibly fragile environment - it's easy to make a small change only for working machines to stop working either immediately or randomly several hours later, completely out of the blue. The same of course is also true, which is why if you are having problems with SMB/Samba and your configuration changes are not working as expected (or you want to be sure your changes will continue working after a power down) then it's often best to reboot *everything*.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#12
(2018-07-10, 14:27)tmaddison Wrote: First – after doing all of the below, the problem appears to be fixed. 
Just wondering... Why are you the only one in this thread that has font tags splattered all over every post? Are you somehow using MS Word or some other office editor before copy/pasting your texts into the Kodi forum editor? I'm simply curious how this is possible.
Reply
#13
Sorry to bring this topic back to life, but I have been searching and searching for months for an answer. I am experiencing the same problem with Libreelec running on RPI3. Basically, small files seem to work fine, but If I start using heavily the hard drives connected to it, then Windows transfer fails. It is crazy how hard Windows is trying, since it slows down my WIN PC (Gaming PC) so much that everything else is freezing. Since this is happening in 2 different WIN PCs, then this has to be a problem with either Kodi or Libreelec SMB server.

My other theory is that maybe RPI cannot handle fast enough the connection and the file transfer at the same time? I am really desperate to fix this, and is really hard to explain or find the root cause. I already tried resetting Libreleec SMB server settings to default and that did not help. It is soooo annoying. I have reached the limit of frustration and I am thinking of buying another RPI3 just to install a 'NAS' OS, and have my Libreelec connect to it to reach the video files....
Reply
#14
All raspberry pi actually have a single USB port on the SoC ("cpu").
Non-A/non-Zero models have a small 5 port hub soldered which shares the bandwidth of a single USB port between 4 additional ports and the usb NIC (Network Interface Card).
Thus you actually have the bandwidth of a single (pretty weak) USB port shared between all connected USB drives and the network card.
The pi hardware simply can't keep up due to poor hardware design decision which sadly was carried over from the first model to the latest ones.
Probably your pi has a lot of iowait due to oversaturated usb bus.

Things that could help:
Make sure that your harddrives are powered via external psu, make sure that pi's psu is good, original RPF ones are recommended which are actually the best for the pi.

Don't use your pi as storage. Use it as client only, it has enough horsepower to watch any type of 1080p content via network from a proper NAS.
Mine does full blown bluray ISOs via NFS without any problem.
IMHO:
I wouldn't buy another pi to use it as NAS due to reasons outlined above.
Better get a proper NAS, I know it's much more expensive but its worth it.

Edit: I actually have 4 pies, non of them is used as storage ofcourse, a proper ubuntu server on a core-i5 fulfills my storage needs.
Reply
#15
(2019-02-04, 21:40)redespace Wrote: Sorry to bring this topic back to life, but I have been searching and searching for months for an answer. I am experiencing the same problem with Libreelec running on RPI3. Basically, small files seem to work fine, but If I start using heavily the hard drives connected to it, then Windows transfer fails. It is crazy how hard Windows is trying, since it slows down my WIN PC (Gaming PC) so much that everything else is freezing. Since this is happening in 2 different WIN PCs, then this has to be a problem with either Kodi or Libreelec SMB server.

My other theory is that maybe RPI cannot handle fast enough the connection and the file transfer at the same time? I am really desperate to fix this, and is really hard to explain or find the root cause. I already tried resetting Libreleec SMB server settings to default and that did not help. It is soooo annoying. I have reached the limit of frustration and I am thinking of buying another RPI3 just to install a 'NAS' OS, and have my Libreelec connect to it to reach the video files....
Try a different protocol, use SFTP instead...

https://winscp.net/eng/docs/guide_window...ssh_server

Just to see if it makes a difference.
HTPCs: 2 x Chromecast with Google TV
Audio: Pioneer VSX-819HK & S-HS 100 5.1 Speakers
Server: HP Compaq Pro 6300, 4GB RAM, 8.75TB, Bodhi Linux 5.x, NFS, MySQL
Reply

Logout Mark Read Team Forum Stats Members Help
Unable to transfer large files to external USB drive connected to RPI2 with Kodi 17.30