Proposal - Add Python3 Support

  Thread Rating:
  • 3 Vote(s) - 3.67 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
arpitn30 Offline
Junior Member
Posts: 13
Joined: Mar 2017
Reputation: 0
Post: #1
I wanted to make a separate post for the proposal

ADD PYTHON3 SUPPORT

Name: Arpit Nandwani
Email: [email protected]
GitHub: https://github.com/arpitn30
LinkedIn: https://www.linkedin.com/in/arpitnandwani/

PERSONAL INFO
I’m a prefinal year undergraduate student pursuing B.Tech from Bharati Vidyapeeth’s College Of Engineering. I’m majoring in Computer Science. I reside in New Delhi, India (GMT +5:30).I started to code in C++ during my school and because of my interest in programming chose it for higher studies as well. During my sophomore year, I completed CS50 and was able to apply my knowledge of C and PHP to real world problems through its projects. I moved to Python due to its dynamic nature, easier syntax and implementation. Moreover, through use of Django and Flask, much more robust websites can be made. I’m also familiar databases like PostgreSQL and MySQL, cloud services like Azure and AWS and a bit of machine learning.

INTRODUCTION
I’ve been using Kodi Media Centre for over 4 months now for my media centre needs in Linux. I’ve contributed a bit in Open Source. I got to work on Python, Flask, PostgreSQL and Redis Server with them. Moreover, even though I knew the basics about git, I was able to learn advance git functions like rebasing and squashing. I’ve worked on projects like creating a brick breaker game or a deleted photos recovery software in C, implemented some ciphers, tries and programmed a basic server. In Python, my projects include mini scripts like a spider to crawl and search keywords on a website or a program to fetch episode titles from Wikipedia and rename them locally. I’ve also built some websites in PHP and Javascript as part of my own projects or internships. One of them is hosted on http://pocketlibrary.azurewebsites.net . Me and my team has also been selected for Smart India Hackathon organised by Govt Of India and we’ll be using React Native and Python to build an IOT embedded system. All of my work can be found on GitHub.
I usually work on Windows 10 or Linux like Ubuntu (XFCE and KDE), Antergos or Deepin depending upon the dependencies of the project. I’ve got a decent computer with a Intel Core I5, Gen 4 processor, 8GB DDR3 RAM and a dedicated 2GB NVIDIA 820m graphic card. I and am able to run multiple Virtual Machines simultaneously. For editor or compiler needs, I usually prefer Atom, Kate, Visual Studio or PyCharm.


PROJECT DETAILS
Kodi’s main code base is in C++ though it contains custom libraries for the development of add-ons in Python. Kodi currently supports Python version 2.6 and 2.7. I wish to add Python3 support to Kodi codebase so that Third Party Developers are able to code in both Python 2 and Python 3.

PHASES
• PHASE 1 – Update the C++ files with Python.h header files and the SWIG binaries to support Python 3.5
• PHASE 2 – Testing all Kodi built-in modules to check compatibility and documentation.
• PHASE 3 – Porting some of Kodi's main addons to Python 3

TIMELINE
May 05 – May 15 : Community Bonding, understanding the code base and listing files that need to be changed
May 16 – May 31 : Busy with college semester exams
June 01 – June 14 : Changing the code to add compatibility for Python 2 (2.6 and 2.7) and Python 3 (3.5 and possibly 3.6)
June 15 – June 27 : Updating the custom SWIG bindings and adding the functionality to choose version based on the individual xml files in the addons.
June 27 – June 30 : First Evaluation
July 01 – July 12 : Developing a new addon and testing the Python 3 compatibility
July 13 – July 25 : Documentation, writing tests and testing code coverage
July 26 – July 28 : Second Evaluation
July 29 – Aug 14 : Porting Kodi addons to Python 3
Aug 15 – Aug 21 : Bug fixes and code clean up
Aug 21 – Aug 29 : Submission of code and final evaluation

HOW WILL IT AFFECT KODI
This project will help the third-party developers develop their plugins and scripts in Python 3 and the programs dependent on xbmc libraries to port to python 3 as well. Moreover, I wish to continue contributing to Kodi even after the summer of code and make it a complete solution for home entertainment. I wish to work on the kodi chromium browser sometime in the future too but given the amount of ideas, I feel it’s too big of a project for the summer.

I don't have any prior engagements in summer and can dedicate my entire time to the development of this project. I will be working 7 hours a day for 6 days a week so I'll be available for communications most of the time. I've already gone through some of the code and successfully did a test build on Windows. I might need a little time to understand the working of SWIG bindings but it's nothing I can't handle.

MENTORS
(This post was last modified: 2017-03-24 23:25 by arpitn30.)
find quote
Razze Offline
Team-Kodi Member
Posts: 1,582
Joined: Jan 2015
Reputation: 50
Location: Osnabrück, Germany
Post: #2
Hey there, nice to meet you! Quiet some nice projects there and exciting proposal Smile

(2017-03-21 11:01)arpitn30 Wrote:  • PHASE 1 – Convert all the Python 2 files to support both Python 2 and Python 3 USING Python Future Modules

Can you go a bit more into details here? I'm not really getting this.

If I would be doing this, I would probably start with your PHASE 2 and maybe do a test python 3 addon and try to get that to run.

Then expand testing to all the interfaces to python kodi offers. Then if there's time left it might be nice to move some of Team Kodis addons over to python 3, but it seems more like a plus point then a PHASE 1.
(This post was last modified: 2017-03-21 23:58 by Razze.)
find quote
arpitn30 Offline
Junior Member
Posts: 13
Joined: Mar 2017
Reputation: 0
Post: #3
I was referring to the Python Libraries in the system/python/Lib directory. Some of the files support Python 3 functions but most of them are written in legacy Python 2 code.
find quote
Rechi Offline
Team-Kodi Member
Posts: 336
Joined: Feb 2015
Reputation: 4
Location: Austria
Post: #4
(2017-03-22 01:05)arpitn30 Wrote:  I was referring to the Python Libraries in the system/python/Lib directory. Some of the files support Python 3 functions but most of them are written in legacy Python 2 code.

You see those files only on Windows, if you mean the same directory I'm thinking about.
You won't need to changed them. These files are from https://github.com/python/cpython/tree/2.7/Lib and if you switch to Python3 these files should work with it as they are part of the official python code.
I'm not sure, if compiling Python3 for windows should be done by yourself, because compiling libraries for windows is a bit special.
find quote
arpitn30 Offline
Junior Member
Posts: 13
Joined: Mar 2017
Reputation: 0
Post: #5
(2017-03-21 23:56)Razze Wrote:  Then expand testing to all the interfaces to python kodi offers.
Can you please explain this a bit in detail.
find quote
Rechi Offline
Team-Kodi Member
Posts: 336
Joined: Feb 2015
Reputation: 4
Location: Austria
Post: #6
I think Razze means the interfaces listed at http://mirrors.kodi.tv/docs/python-docs/ which are used by addons to add listings and such things.
find quote
Roman_V_M Offline
Team-Kodi Member
Posts: 1,115
Joined: Jun 2011
Reputation: 55
Location: Kyiv, Ukraine
Post: #7
(2017-03-22 01:05)arpitn30 Wrote:  I was referring to the Python Libraries in the system/python/Lib directory. Some of the files support Python 3 functions but most of them are written in legacy Python 2 code.

You don't need to touch that. This is the Python Standard Library that comes with every CPython interpreter. Each CPython interpreter has its own. If you want to include CPython 3 interpreter in Kodi, you need to use the Standard Library that is bundled with sources.

I'm not a C++ developer and have a limited experience with Python/C++ interoperability but as I understand the problem consists of 2 parts:

- We need to compile CPython 3 for Kodi. I know that Python in Kodi has its own specifics, like removed stdin and stdout/stderr redirected to the Kodi log, and I'm sure there's probably more. BTW, which minor Python 3 version do you plan to use?

- We need to port wrappers for Kodi Python API (xbmc* modules) so that it correctly works in Python 3 (str/bytes differences and such).

I don't know if it's possible to have both Python 2 and 3 (at least for some time to allow smooth transition of Python addons), but if it is, then some work will be needed to implement correct support of both versions.

BTW, although automated tools for Python 2 > 3 conversion exist, it is still mainly a manual process. General purpose Python libraries converted into Kodi addons (script.module.html5lib, script.module.requests and such) are mostly written in Python 2/3 compatible code, but most addons need to be converted to Python 3 by their developers/maintainers.

Raspberry PI 2 + LibreELEC 8 (Kodi 17)
Samsung Galaxy Tab A 10.1 + Kodi 17 for Android
find quote
arpitn30 Offline
Junior Member
Posts: 13
Joined: Mar 2017
Reputation: 0
Post: #8
Proposal Updated
find quote
Razze Offline
Team-Kodi Member
Posts: 1,582
Joined: Jan 2015
Reputation: 50
Location: Osnabrück, Germany
Post: #9
Good work. Your proposal looks very promissing.
find quote
arpitn30 Offline
Junior Member
Posts: 13
Joined: Mar 2017
Reputation: 0
Post: #10
I read somewhere on this forum that there were some problems with using SWIG directly so the wrapper was instead coded with an implementation similar to SWIG. I was wondering what were the problems being faced which led to the decision against using SWIG directly
find quote
da-anda Offline
Team-Kodi Member
Posts: 5,777
Joined: Jun 2009
Reputation: 77
Location: germany
Post: #11
I think only jfcarrol (who did all the SWIG stuff) might be able to answer that. Unfortunately he hasn't be around much in the last year(s), but I try to ping him
find quote
Montellese Offline
Team Kodi Developer
Posts: 4,835
Joined: Jan 2009
Reputation: 72
Location: Switzerland
Post: #12
(2017-04-25 08:28)arpitn30 Wrote:  I read somewhere on this forum that there were some problems with using SWIG directly so the wrapper was instead coded with an implementation similar to SWIG. I was wondering what were the problems being faced which led to the decision against using SWIG directly

See https://sourceforge.net/p/swig/bugs/1114/ for the bug that Jim reported to the Python SWIG guys. Unfortunately there was nobody capable of looking into it.

Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not e-mail Team Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
find quote
arpitn30 Offline
Junior Member
Posts: 13
Joined: Mar 2017
Reputation: 0
Post: #13
Thanks a lot, I'll check it out
find quote