2017-12-19, 06:47
A few notes beforehand:
- big thanks to RomanVM for all the help with the conversion.
- the addon i converted is not very complex, f.e. it doesn't use youtube.
- the conversion was done on a windows 10 machine. For linux/Mac/etc the file paths will be different.
- i use kodi in portable mode. This means it's get started like this: Kodi -p.
- there will be some errors in the log with sha stuff. This prop has to do with python 3 kodi not being entirely correct yet with some linked binaries or something.
Step 1: in my python 2 scripts directory, install the latest version of future: C:\Python27\Scripts>pip install future
Step 2: in my python 2 kodi addon directory: futurize all the .py files with the -w1 parm: f.e. C:\Kodi_18.x_Portable\portable_data\addons\plugin.video.tweakers\resources\lib>futurize -w1 tweakers_search.py
Step 3: added this in my addon.xml: <import addon="script.module.future" version="0.0.1"/>
Step 4: check that the addon still works in python 2. It worked.
Step 5: in my python 2 kodi addon directory: futurize all the .py files with the -w2 parm: f.e. C:\Kodi_18.x_Portable\portable_data\addons\plugin.video.tweakers\resources\lib>futurize -w2 tweakers_search.py
For the brave 'n lazy it's also possible to do step 2 thu 5 a bit faster. I made a bat file that should run in the directory where the *.py files are. The content of the bat file:
for %%f in (*.py) do futurize -w0 %%f
Step 6: check that the addon still works in python 2. It didn't work: it threw a error in the logging: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128).
Step 7: refactoring the logging and the string stuff: every text i scraped from the webpage i converted with the convertToUnicodeString method:
The string stuff was this:
# Get HTML page
response = requests.get(self.video_list_page_url, headers=headers)
# response.status
html_source = response.text
--> html_source = html_source.encode('utf-8', 'ignore')
and became this:
# Get HTML page
response = requests.get(self.video_list_page_url, headers=headers)
# response.status
html_source = response.text
--> html_source = convertToUnicodeString(html_source)
if sys.version_info[0] > 2:
unicode = str
def convertToUnicodeString(s, encoding='utf-8'):
"""Safe decode byte strings to Unicode"""
if isinstance(s, bytes): # This works in Python 2.7 and 3+
s = s.decode(encoding)
return s
I also had to remove statements like this one:
html_source = ''
Logging was this:
xbmc.log("[ADDON] %s v%s (%s) debug mode, %s = %s" % (ADDON, VERSION, DATE, "title", str(title)),
xbmc.LOGDEBUG)
and became this:
log("title", title)
def log(name_object, object):
try:
xbmc.log("[ADDON] %s v%s (%s) debug mode, %s = %s" % (
ADDON, VERSION, DATE, name_object, convertToUnicodeString(object)), xbmc.LOGDEBUG)
except:
xbmc.log("[ADDON] %s v%s (%s) debug mode, %s = %s" % (
ADDON, VERSION, DATE, name_object, "Unable to log the object due to an error while converting it to an unicode string"), xbmc.LOGDEBUG)
Step 8: upgraded from beautiful soup 3 to 4. Bs4 can use a parser like html5lib.
In addon.xml:
<import addon="script.module.beautifulsoup4" version="4.5.3"/>
<import addon="script.module.html5lib" version="0.999.0"/>.
from bs4 import BeautifulSoup
def getSoup(html,default_parser="html5lib"):
try:
soup = BeautifulSoup(html, default_parser)
log("htmlparser", "html5lib")
except:
log("Unexpected error when using " + default_parser + ":", sys.exc_info()[0])
soup = BeautifulSoup(html,"html.parser")
log("htmlparser", "html.parser")
return soup
Later after a comment i realised that this would be better:
def getSoup(html,default_parser="html5lib"):
soup = BeautifulSoup(html, default_parser)
return soup
If an error would occur then it means your code and/or addon.xml isn't correct. And that should be fixed.
Also in android the usage of future's old_div caused crashing. Removing old_div and just do regular division, fixes that.
Step 9: zipped my addon and installed in my test python 2 kodo (install from zip). This way the kodi module bs4 and html5lib were downloaded.
Test 10: Tested my addon in python 2 kodi. It worked. Now it's finally time to try it out on python 3...
Step 11: Installed a path_order_patched Python 3 kodi i got from romanvm. However i have heard now that there is an official python 3 kodi that will work IF the python 3 version of future is used. See at the bottom of this post.
Step 12: zipped my addon and installed it on fresh python 3 kodi (install from zip). This way the dependencies like bs4 and html5lib get downloaded.
Step 13: Tested my addon in python 3 kodi. Starting it with the -p parameter ('Kodi -p') for portable mode. No dice: got a message that future was python 2.
Step 14: install the python 3 version of future. See the bottom of the pages for file links.
Step 15: Tested my addon in python 3 kodi. It worked. Rejoice! Time for Beer! or Wine! or some orange juice!
Step 16: Removed the py.bak files from my addon.
Helpfull files:
Official Python 3 kodi: it's in this directory http://mirrors.kodi.tv/test-builds/windows/win32/ and should be called KodiSetup-<something>-feature_python3-x86.exe
Official Python 3 future module: http://www.mediafire.com/file/f4w3ydj1kw...16.0.0.zip. This version however won't work in android. RomanVM made a new version of the future module that should work in py2/3 on Windows/linux/mac/android... This version (0.16.0.4) should be availabe soon in Gotham and higher soonish: https://github.com/xbmc/repo-scripts/pull/761
(THIS FILE IS NOT NEEDED BECAUSE THERE IS A WORKING PYTHON 3 BS4 MODULE THAT WILL BE DOWNLOADED AS A DEPENDENCY: Official Python 3 beautiful soup 4 module: http://www.mediafire.com/file/qr62a49es1...-4.6.0.zip)
My addon before conversion: http://www.mediafire.com/file/9n7kixgq1o...urized.zip
My addon after conversion: http://www.mediafire.com/file/1zdzi4ex2h...urized.zip
I hope this helps with the conversion of addons...
- big thanks to RomanVM for all the help with the conversion.
- the addon i converted is not very complex, f.e. it doesn't use youtube.
- the conversion was done on a windows 10 machine. For linux/Mac/etc the file paths will be different.
- i use kodi in portable mode. This means it's get started like this: Kodi -p.
- there will be some errors in the log with sha stuff. This prop has to do with python 3 kodi not being entirely correct yet with some linked binaries or something.
Step 1: in my python 2 scripts directory, install the latest version of future: C:\Python27\Scripts>pip install future
Step 2: in my python 2 kodi addon directory: futurize all the .py files with the -w1 parm: f.e. C:\Kodi_18.x_Portable\portable_data\addons\plugin.video.tweakers\resources\lib>futurize -w1 tweakers_search.py
Step 3: added this in my addon.xml: <import addon="script.module.future" version="0.0.1"/>
Step 4: check that the addon still works in python 2. It worked.
Step 5: in my python 2 kodi addon directory: futurize all the .py files with the -w2 parm: f.e. C:\Kodi_18.x_Portable\portable_data\addons\plugin.video.tweakers\resources\lib>futurize -w2 tweakers_search.py
For the brave 'n lazy it's also possible to do step 2 thu 5 a bit faster. I made a bat file that should run in the directory where the *.py files are. The content of the bat file:
for %%f in (*.py) do futurize -w0 %%f
Step 6: check that the addon still works in python 2. It didn't work: it threw a error in the logging: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128).
Step 7: refactoring the logging and the string stuff: every text i scraped from the webpage i converted with the convertToUnicodeString method:
The string stuff was this:
# Get HTML page
response = requests.get(self.video_list_page_url, headers=headers)
# response.status
html_source = response.text
--> html_source = html_source.encode('utf-8', 'ignore')
and became this:
# Get HTML page
response = requests.get(self.video_list_page_url, headers=headers)
# response.status
html_source = response.text
--> html_source = convertToUnicodeString(html_source)
if sys.version_info[0] > 2:
unicode = str
def convertToUnicodeString(s, encoding='utf-8'):
"""Safe decode byte strings to Unicode"""
if isinstance(s, bytes): # This works in Python 2.7 and 3+
s = s.decode(encoding)
return s
I also had to remove statements like this one:
html_source = ''
Logging was this:
xbmc.log("[ADDON] %s v%s (%s) debug mode, %s = %s" % (ADDON, VERSION, DATE, "title", str(title)),
xbmc.LOGDEBUG)
and became this:
log("title", title)
def log(name_object, object):
try:
xbmc.log("[ADDON] %s v%s (%s) debug mode, %s = %s" % (
ADDON, VERSION, DATE, name_object, convertToUnicodeString(object)), xbmc.LOGDEBUG)
except:
xbmc.log("[ADDON] %s v%s (%s) debug mode, %s = %s" % (
ADDON, VERSION, DATE, name_object, "Unable to log the object due to an error while converting it to an unicode string"), xbmc.LOGDEBUG)
Step 8: upgraded from beautiful soup 3 to 4. Bs4 can use a parser like html5lib.
In addon.xml:
<import addon="script.module.beautifulsoup4" version="4.5.3"/>
<import addon="script.module.html5lib" version="0.999.0"/>.
from bs4 import BeautifulSoup
def getSoup(html,default_parser="html5lib"):
try:
soup = BeautifulSoup(html, default_parser)
log("htmlparser", "html5lib")
except:
log("Unexpected error when using " + default_parser + ":", sys.exc_info()[0])
soup = BeautifulSoup(html,"html.parser")
log("htmlparser", "html.parser")
return soup
Later after a comment i realised that this would be better:
def getSoup(html,default_parser="html5lib"):
soup = BeautifulSoup(html, default_parser)
return soup
If an error would occur then it means your code and/or addon.xml isn't correct. And that should be fixed.
Also in android the usage of future's old_div caused crashing. Removing old_div and just do regular division, fixes that.
Step 9: zipped my addon and installed in my test python 2 kodo (install from zip). This way the kodi module bs4 and html5lib were downloaded.
Test 10: Tested my addon in python 2 kodi. It worked. Now it's finally time to try it out on python 3...
Step 11: Installed a path_order_patched Python 3 kodi i got from romanvm. However i have heard now that there is an official python 3 kodi that will work IF the python 3 version of future is used. See at the bottom of this post.
Step 12: zipped my addon and installed it on fresh python 3 kodi (install from zip). This way the dependencies like bs4 and html5lib get downloaded.
Step 13: Tested my addon in python 3 kodi. Starting it with the -p parameter ('Kodi -p') for portable mode. No dice: got a message that future was python 2.
Step 14: install the python 3 version of future. See the bottom of the pages for file links.
Step 15: Tested my addon in python 3 kodi. It worked. Rejoice! Time for Beer! or Wine! or some orange juice!
Step 16: Removed the py.bak files from my addon.
Helpfull files:
Official Python 3 kodi: it's in this directory http://mirrors.kodi.tv/test-builds/windows/win32/ and should be called KodiSetup-<something>-feature_python3-x86.exe
Official Python 3 future module: http://www.mediafire.com/file/f4w3ydj1kw...16.0.0.zip. This version however won't work in android. RomanVM made a new version of the future module that should work in py2/3 on Windows/linux/mac/android... This version (0.16.0.4) should be availabe soon in Gotham and higher soonish: https://github.com/xbmc/repo-scripts/pull/761
(THIS FILE IS NOT NEEDED BECAUSE THERE IS A WORKING PYTHON 3 BS4 MODULE THAT WILL BE DOWNLOADED AS A DEPENDENCY: Official Python 3 beautiful soup 4 module: http://www.mediafire.com/file/qr62a49es1...-4.6.0.zip)
My addon before conversion: http://www.mediafire.com/file/9n7kixgq1o...urized.zip
My addon after conversion: http://www.mediafire.com/file/1zdzi4ex2h...urized.zip
I hope this helps with the conversion of addons...
