SFTP with large directory freezes kodi
#1
Hi, I have a remote SFTP server directory with 250 sub directories in it (each sub directory has 4-50 files in it).

When I point Kodi at it the application freezes, but does not crash although the entire application is unresponsive. I have to close the application and reopen it.

Kodi freezes 100% of the time when accessing this directory, although if I move one directory up (which holds 1 directory, which is the one I am trying to load) it does fine. ALTHOUGH once I try to open the directory it freezes because it is trying to load its 250 sub directories.

This freeze is also reproducible in the windows client which produces a crash, or rather a "program is not responding" error.

Likewise when I try to load a smaller directory off a local LAN server which is small (about 8 directories) it loads instantly.

My network is 105/30 so it is not likely to be a bandwidth issue. I did notice that via my bandwidth monitor that the Windows Kodi client was still using about 300kb/s of bandwidth before windows gave the "program is not responding" error all the while the UI was frozen.

Unfortiently I can't mount the remote servers directory locally on android (lack of sshfs), so I am stuck using SFTP. I haven't tried the HTTP functions yet though.

I've also tried to disable fetching of meta data which also froze when trying to access my remote server over SFTP.

My device is a Nvidia shield tablet if it matters and I can access the data fine via sftp clients and can mount the directory with sshfs.

Any tips?

Edit:: forgot to mention that when adding the SFTP source it did in fact list all the directories. The same with a HTTP source. But when trying to load them both the android and windows applications freeze.
Reply
#2
2 day bump. Still having issues.

Log files reveal nothing of interest besides the connection logs:

Code:
15:10:27 T:5000   DEBUG: ------ Window Deinit (DialogNetworkSetup.xml) ------
15:10:27 T:5048  NOTICE: Thread JobWorker start, auto delete: true
15:10:27 T:5048    INFO: SFTPSession: Creating new session on host '***.***.***.***:7070' with user 'movies'
15:10:27 T:5000   DEBUG: ------ Window Init (DialogBusy.xml) ------
15:10:30 T:5000   DEBUG: ------ Window Init (FileBrowser.xml) ------
15:10:30 T:5000   DEBUG: ------ Window Deinit (DialogBusy.xml) ------
15:10:32 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
15:10:32 T:5000   DEBUG: ------ Window Deinit (FileBrowser.xml) ------
15:10:35 T:5000   DEBUG: CInputManager::ProcessMouse: unknown mouse command 57605
15:10:35 T:5000   DEBUG: ------ Window Deinit (Pointer.xml) ------
15:10:35 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action mousedrag
15:10:35 T:5000   DEBUG: ------ Window Init (Pointer.xml) ------
15:10:35 T:5000   DEBUG: CInputManager::ProcessMouse: unknown mouse command 57606
15:10:35 T:5000   DEBUG: ------ Window Deinit (Pointer.xml) ------
15:10:35 T:5000   DEBUG: ------ Window Init (Pointer.xml) ------
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x17, sym: 0x0069, unicode: 0x0069, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: i (0xf049) pressed, trying keyboard action f169
15:10:37 T:5000   DEBUG: ------ Window Deinit (Pointer.xml) ------
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x21, sym: 0x0066, unicode: 0x0066, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: f (0xf046) pressed, trying keyboard action f166
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x13, sym: 0x0072, unicode: 0x0072, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: r (0xf052) pressed, trying keyboard action f172
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x17, sym: 0x0069, unicode: 0x0069, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: i (0xf049) pressed, trying keyboard action f169
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x14, sym: 0x0074, unicode: 0x0074, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: t (0xf054) pressed, trying keyboard action f174
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x39, sym: 0x0020, unicode: 0x0020, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: space (0xf020) pressed, trying keyboard action f120
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x32, sym: 0x006d, unicode: 0x006d, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: m (0xf04d) pressed, trying keyboard action f16d
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x18, sym: 0x006f, unicode: 0x006f, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: o (0xf04f) pressed, trying keyboard action f16f
15:10:37 T:5000   DEBUG: Keyboard: scancode: 0x2f, sym: 0x0076, unicode: 0x0076, modifier: 0x0
15:10:37 T:5000   DEBUG: CInputManager::OnKey: v (0xf056) pressed, trying keyboard action f176
15:10:38 T:4784   DEBUG: CAESinkDirectSound::Deinitialize: Cleaning up
15:10:38 T:5000   DEBUG: Keyboard: scancode: 0x17, sym: 0x0069, unicode: 0x0069, modifier: 0x0
15:10:38 T:5000   DEBUG: CInputManager::OnKey: i (0xf049) pressed, trying keyboard action f169
15:10:38 T:5000   DEBUG: Keyboard: scancode: 0x12, sym: 0x0065, unicode: 0x0065, modifier: 0x0
15:10:38 T:5000   DEBUG: CInputManager::OnKey: e (0xf045) pressed, trying keyboard action f165
15:10:38 T:5000   DEBUG: Keyboard: scancode: 0x1f, sym: 0x0073, unicode: 0x0073, modifier: 0x0
15:10:38 T:5000   DEBUG: CInputManager::OnKey: s (0xf053) pressed, trying keyboard action f173
15:10:38 T:5000   DEBUG: Keyboard: scancode: 0x1c, sym: 0x000d, unicode: 0x000d, modifier: 0x0
15:10:38 T:5000   DEBUG: CInputManager::OnKey: return (0xf00d) pressed, trying keyboard action 7
15:10:38 T:5000   DEBUG: ------ Window Init (DialogKeyboard.xml) ------
15:10:38 T:5000   DEBUG: CAnnouncementManager - Announcement: OnInputRequested from xbmc
15:10:38 T:5000   DEBUG: GOT ANNOUNCEMENT, type: 128, from xbmc, message OnInputRequested
15:10:39 T:5000   DEBUG: ------ Window Init (Pointer.xml) ------
15:10:40 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
15:10:40 T:5000   DEBUG: ------ Window Deinit (DialogKeyboard.xml) ------
15:10:40 T:5000   DEBUG: CAnnouncementManager - Announcement: OnInputFinished from xbmc
15:10:40 T:5000   DEBUG: GOT ANNOUNCEMENT, type: 128, from xbmc, message OnInputFinished
15:10:40 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
15:10:41 T:5000   DEBUG: ------ Window Init (DialogContentSettings.xml) ------
15:10:41 T:5000    INFO: Loading skin file: DialogContentSettings.xml, load type: KEEP_IN_MEMORY
15:10:41 T:5000   DEBUG: CSettingsManager: requested setting (exclude) was not found.
15:10:41 T:5000   DEBUG: ------ Window Deinit (DialogMediaSource.xml) ------
15:10:43 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
15:10:43 T:5000   DEBUG: ------ Window Deinit (DialogContentSettings.xml) ------
15:10:44 T:5000   DEBUG: CGUIMediaWindow::GetDirectory (sources://video/)
15:10:44 T:5000   DEBUG:   ParentPath = [sources://video/]
15:10:44 T:3900  NOTICE: Thread BackgroundLoader start, auto delete: false
15:10:44 T:3900   DEBUG: Thread BackgroundLoader 3900 terminating
15:10:46 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
15:10:46 T:5000   DEBUG: CGUIMediaWindow::GetDirectory (sftp://USERNAME:PASSWORD@***.***.***.***:7070/)
15:10:46 T:5000   DEBUG:   ParentPath = [sources://video/]
15:10:47 T:4576  NOTICE: Thread BackgroundLoader start, auto delete: false
15:10:47 T:4576   DEBUG: Thread BackgroundLoader 4576 terminating
15:10:49 T:5000   DEBUG: CInputManager::ProcessMouse: trying mouse action leftclick
15:10:49 T:5000   DEBUG: CGUIMediaWindow::GetDirectory (sftp://USERNAME:PASSWORD@***.***.***.***:7070/downloads/)
15:10:49 T:5000   DEBUG:   ParentPath = [sftp://USERNAME:PASSWORD@***.***.***.***:7070/]
15:10:50 T:5000   DEBUG: ------ Window Init (DialogBusy.xml) ------

***.***.***.*** is the redacted IP. USERNAMETongueASSWORD is what KODI added to the logs itself, and these are not the credentials I have typed in. Although I don't think it matters since when adding the network source it does list the directories and files, so it does indeed connect. It is only after while accessing the SFTP files from the VIDEO->Files menu that it freezes. The last line in the code block is the last line KODI wrote before it froze and did a Windows "This program is not responding" error.

As you can see the initial connection is fine in which GetDirectory returns the one sub directory "downloads", but when the GetDirectory method is called on /downloads/ KODI freezes..

File permissions on the remote server is fine since all other SFTP, SCP, and the user on the remote server itself can access the data stored in the sub directory. So it is indeed some kind of KODI issue.

When going deeper into the directory structure placing the SFTP start directory inside the directory containing the media files KODI does not freeze and actually lists the movie files and plays them.

Any help?
Reply
#3
Up. Forgot about this thread for a while, but I'm still having these troubles.

I am spinning my home server up in a bit (2x quadcore e5405, 64gb ram) and loading all my media onto its 4tb RAID0 array. I'll be done with it in the next day or so and report back if I get the same issues over a LAN with a high end server.

Although my other server in a datacenter is also decently good (quad core 2.4ghz, 4gb ram, gigabit connection fiber) and it is having troubles with SFTP as stated above when accessing across the WAN.

Although with my statements in the above posts this would seem to be a bug in KODI and not the lack of power of the servers in question.
Reply
#4
SFTP is undeniably broken in KODI 15.2 as well as 16.0-BETA4.

When trying to access my local LAN server it errors with a "unable to connect" message, even though the same PC can connect via winSCP (SFTP mode) as well as other software such as SFTP NET DRIVE (mounts a SFTP share as a local hard drive).

This happens as well with the android version of the application.

All other symptoms above still happen with the new BETA (SFTP causing freezes, hangs, crashes when trying to view a remote share with a large number of directories).

On the other hand streaming via FTP (vsftpd backend) from my LAN server works flawlessly.

I will file a ticket, but I doubt anything will get done about it since this seems to be an on going issue for a few years.
Reply

Logout Mark Read Team Forum Stats Members Help
SFTP with large directory freezes kodi0