Kodi Community Forum

Full Version: Python performance gap between OSX/Windows
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
Just curious if anyone else has noticed Python scripts/addons running much slower under Windows. Below is a video of two of the same scripts (SABnzbd/Crunchyroll) running side by side under each platform on the same machine. Directory listing is much slower under Windows. I can post some debug logs later, but for now the video explains what I'm talking about:

http://www.mediafire.com/?lxliezyfkred2oz
Yea, it's a known issue (to me anyway). From what I can tell it's due to slow filesystem access on windows.

Windows xbmc is horribly slow at loading thumbnails and such from disk aswell.

Btw.. Your link is broken.
Ah, okay, I've been working on implementing threading in the SABnzbd plugin and the speedup under OSX is phenomenal in comparison to the un-threaded version, whereas on Windows it's maybe only twice as fast. I figured I was doing something wrong. The link should be fixed now (hopefully.)
elupus Wrote:Yea, it's a known issue (to me anyway). From what I can tell it's due to slow filesystem access on windows.

Windows xbmc is horribly slow at loading thumbnails and such from disk aswell.

Btw.. Your link is broken.



Is the slow file system access in Windows considered a bug?
I guess this is a so called feature of windows ;-)
I Just did a quick timer for loading the Top Documentary add-on. I just used the simple print command and have noticed a long delay when loading the urllib2 module. It looks like that is whats causing the problem for me.

Code:
01:16:50 T:844 M:4294967295  NOTICE: Importing xbmcplugin os.path sys re
01:16:51 T:844 M:4294967295  NOTICE: Importing socket
01:16:51 T:844 M:4294967295  NOTICE: Importing urllib
01:16:51 T:844 M:4294967295  NOTICE: Importing urllib2
01:16:55 T:844 M:4294967295  NOTICE: Importing xbmcgui
01:16:55 T:844 M:4294967295  NOTICE: Importing utils
01:16:55 T:844 M:4294967295  NOTICE: Starting Work
01:16:55 T:844 M:4294967295  NOTICE: Website url ==> http://topdocumentaryfilms.com/all/page/1
01:16:57 T:844 M:4294967295  NOTICE: Finished
01:16:57 T:844 M:4294967295  NOTICE: Finished Working On Script
Nothing to do with the filesystem. The python libs and modules are loading extremely slow:

build and os:

Quote:15:16:48 T:3148 M:1025249280 NOTICE: Starting XBMC, Platform: Windows 7, 64-bit (WoW) build 7600. Built on Jul 20 2010 (SVN:31996, compiler 1500)
15:16:48 T:3148 M:1025249280 NOTICE: Intel® Atom™ CPU 330 @ 1.60GHz
15:16:48 T:3148 M:1025241088 NOTICE: Desktop Resolution: 1360x768 32Bit at 60Hz

debug snip:

- first notice the "import site" problem and the delay caused while displaying the working and progress windows (green vs red), and we are not even in my script yet!

- than the extreme load time of "import urllib" and "import urllib2" (blue vs red)

Quote:15:40:26 T:184 M:976904192 INFO: initializing python engine.
15:40:26 T:184 M:976904192 DEBUG: Win32DllLoader::Load(special://xbmcbin/system/python/python24.dll)
15:40:26 T:3148 M:976330752 DEBUG: ------ Window Init (DialogBusy.xml) ------
15:40:26 T:3148 M:976330752 DEBUG: Alloc resources: 0.02ms (0.01 ms skin load)
15:40:31 T:184 M:974041088 DEBUG: msg: 'import site' failed; use -v for traceback
15:40:31 T:184 M:974041088 DEBUG: new python thread created. id=7
15:40:31 T:184 M:974041088 DEBUG: XFILE::CPluginDirectory::WaitOnScriptResult - waiting on the Magyar TVk plugin...
15:40:31 T:3912 M:974041088 DEBUG: thread start, auto delete: 0
15:40:31 T:3912 M:974041088 DEBUG: Python thread: start processing
15:40:33 T:184 M:973869056 INFO: Loading skin file: DialogProgress.xml
15:40:33 T:184 M:973864960 DEBUG: Load DialogProgress.xml: 15.85ms
15:40:33 T:184 M:973864960 DEBUG: ------ Window Init (DialogProgress.xml) ------
15:40:33 T:184 M:973864960 DEBUG: Alloc resources: 0.92ms (0.01 ms skin load)
15:40:33 T:3148 M:973860864 DEBUG: ------ Window Deinit (DialogBusy.xml) ------
15:40:36 T:3912 M:973479936 DEBUG: msg: 'import site' failed; use -v for traceback
15:40:36 T:3912 M:973479936 NOTICE: -->Python Interpreter Initialized<--
15:40:36 T:3912 M:973479936 DEBUG: XBPyThread:Tonguerocess - The source file to load is C:\Users\mediaplayer\AppData\Roaming\XBMC\addons\Magyar TV\default.py
15:40:36 T:3912 M:973479936 DEBUG: XBPyThread:Tonguerocess - Setting the Python path to C:\Users\mediaplayer\AppData\Roaming\XBMC\addons\Magyar TV;C:\Program Files (x86)\XBMC\addons\script.module.pil\lib;C:\Program Files (x86)\XBMC\addons\script.module.pysqlite\lib;special://xbmc/system/python/Lib;special://xbmcbin/system/python/python24.zip;special://xbmc/system/python\DLLs;special://xbmc/system/python\lib;special://xbmc/system/python\lib\plat-win;special://xbmc/system/python\lib\lib-tk;C:\Program Files (x86)\XBMC
15:40:36 T:3912 M:973479936 DEBUG: XBPyThread:Tonguerocess - Entering source directory C:\Users\mediaplayer\AppData\Roaming\XBMC\addons\Magyar TV
15:40:36 T:3912 M:972951552 NOTICE: script - start
15:40:36 T:3912 M:972951552 NOTICE: imports urllib - starting
15:40:43 T:3912 M:971620352 DEBUG: LoadLibraryExA called with flags:
- LOAD_WITH_ALTERED_SEARCH_PATH
15:40:43 T:3912 M:971620352 DEBUG: Win32DllLoader::Load(special://xbmc/system/python\DLLs\_socket.pyd)
15:40:43 T:3912 M:971591680 DEBUG: LoadLibraryExA called with flags:
- LOAD_WITH_ALTERED_SEARCH_PATH
15:40:43 T:3912 M:971591680 DEBUG: Win32DllLoader::Load(special://xbmc/system/python\DLLs\_ssl.pyd)
15:40:43 T:3912 M:971431936 DEBUG: Win32DllLoader::Load(MSVCR71.dll)
15:40:43 T:3912 M:971169792 NOTICE: imports urllib - end
15:40:43 T:3912 M:971169792 NOTICE: imports urllib2 - starting
15:41:19 T:3912 M:968523776 NOTICE: imports urllib2 - end
15:41:19 T:3912 M:968523776 NOTICE: imports re - starting
15:41:19 T:3912 M:968523776 NOTICE: imports re - end
15:41:19 T:3912 M:968523776 NOTICE: imports xbmcplugin - starting
15:41:19 T:3912 M:968523776 NOTICE: imports xbmcplugin - end
15:41:19 T:3912 M:968523776 NOTICE: imports xbmcgui - starting
15:41:19 T:3912 M:968523776 NOTICE: imports xbmcgui - end
15:41:19 T:3912 M:968523776 NOTICE: imports xbmcaddon - starting
15:41:19 T:3912 M:968523776 NOTICE: imports xbmcaddon - end
15:41:19 T:3912 M:968523776 NOTICE: imports - done
15:41:19 T:3912 M:968523776 NOTICE: get params - start
15:41:19 T:3912 M:968523776 NOTICE: get params - end
15:41:19 T:3912 M:968523776 NOTICE: set params - start
15:41:19 T:3912 M:968523776 NOTICE: set params - end
15:41:19 T:3912 M:968523776 NOTICE: Mode: None
15:41:19 T:3912 M:968523776 NOTICE: URL: None
15:41:19 T:3912 M:968523776 NOTICE: Name: None
15:41:19 T:3912 M:968523776 NOTICE: VOD: None
15:41:19 T:3912 M:968523776 NOTICE: post: None
15:41:19 T:3912 M:968523776 NOTICE: select operation - start
15:41:19 T:3912 M:968523776 NOTICE: addir - start
15:41:19 T:3912 M:968511488 NOTICE: addir - end
15:41:19 T:3912 M:968511488 NOTICE: select operation - end
15:41:19 T:184 M:968511488 DEBUG: XFILE::CPluginDirectory::WaitOnScriptResult- plugin returned successfully
15:41:19 T:3912 M:968511488 NOTICE: script - end

Note: the very same script works without any problem as a plugin with the stable version.
This happens on my Windows 7 system too. The Iplayer addon is extremely slow between menus.
should we report that? open a tracker ticket or a new post in the main developer forum? or this is only nightly svn an have to wait til some beta a RC?
Try HEAD, pretty sure r32230 fixed it.
Thanks for the tip. Tried the newest nightly r32246 and it works much better. Though still has some lag in the beggining and between streems (i guess something is still loading/reloading very slowly ) but in between the menu navigation is blazing fast again.
gaborm Wrote:Thanks for the tip. Tried the newest nightly r32246 and it works much better. Though still has some lag in the beggining and between streems (i guess something is still loading/reloading very slowly ) but in between the menu navigation is blazing fast again.

Same for me, thanks for the info.
Is urllib2.urlopen slow as hell on Windows 7? The same plugin running on XP and Xbox needs less than one second to open 16 webpages stored on a localhost but with Windows 7 it needs more than 30 seconds.
A600, sure it's not the "import urllib2" that's slow as reported earlier in this thread?
elupus Wrote:A600, sure it's not the "import urllib2" that's slow as reported earlier in this thread?

No. It's urlopen (1 second each time is called)

Quote:17:53:09 T:1036 M:1069490176 NOTICE: urlopen #0
17:53:10 T:1036 M:1068228608 NOTICE: urlopen #1
17:53:11 T:1036 M:1069645824 NOTICE: urlopen #2
17:53:13 T:1036 M:1068523520 NOTICE: urlopen #3
17:53:14 T:1036 M:1068167168 NOTICE: urlopen #4
17:53:15 T:1036 M:1067646976 NOTICE: urlopen #5
17:53:16 T:1036 M:1067085824 NOTICE: urlopen #6
17:53:17 T:1036 M:1067134976 NOTICE: urlopen #7
17:53:18 T:1036 M:1066250240 NOTICE: urlopen #8
17:53:19 T:1036 M:1065508864 NOTICE: urlopen #9
17:53:20 T:1036 M:1065304064 NOTICE: urlopen #10
17:53:21 T:1036 M:1064435712 NOTICE: urlopen #11
17:53:22 T:1036 M:1064042496 NOTICE: urlopen #12
17:53:23 T:1036 M:1063305216 NOTICE: urlopen #13
17:53:24 T:1036 M:1063133184 NOTICE: urlopen #14
17:53:25 T:1036 M:1061908480 NOTICE: urlopen #15

This is the code:

Code:
i = 0
        for epg,program in match:
            req = urllib2.Request(webhost + epg)
            req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3 Gecko/2008092417 Firefox/3.0.3')
            response = urllib2.urlopen(req)
            print "urlopen #"+ str(i)
            link = response.read()
            response.close()
            desc0 = re.compile('\r\n<br>\r\n(.+?)(<BR>|<br>)', re.DOTALL).findall(link)
            desc = ''
            for desc,br in desc0:
                desc = expandEntities(desc)
            program = expandEntities(program)
            if (len (time[i]) == 4): time[i] = '0' + time[i]
            addLink(time[i]+' - '+program,streamhost + url + '.ts', webhost + 'Logos/' + logo[0] , desc)
            i += 1
            if (i == 16): break
Pages: 1 2