[mtahir]

(Alternate Google Doc Link)

Mohd Tahir 
Github:                tahirdon
Forum/e-mail:     [email protected]
Skype:                 tahirdgreat
Phone Number:  +91 7985059250
Town, country:    Roorkee, India

Project idea: Bitrate switch for inputstream.adaptive binary addon for Kodi

SUMMARY AND PROJECT FOCUS
Inputstream.adaptive is written in C++ and handles 3 bitstreams: 

• HTTP Live Streaming (HLS)
  
• Dynamic Adaptive Streaming over HTTP (DASH)
  
• Smooth Streaming (ISM)
  

This add-on provides packet to Kodi, and then Kodi renders the packets on the internal video player.
While streaming, currently there is no inbuilt feature to switch bitstream automatically when external factors like network available bandwidth, screen window/ resolution changes for a minimum time, frames are not able to be rendered properly due to computation limitation. So, my focus of work will be mostly in this area to automatically switch the streaming to higher/lower resolutions depending on external factors/parameters available. Also I will be trying  to implement playback speed/ frame dropping.  

MOTIVATION
Previously I have worked in research projects ranging from image processing, FPGA MIPI-CSI2 to HDMI protocol interfacing for video, Electrical Vehicle Control and DAQ programming, embedded system driver interface, etc. Most of them were C/ C++ based.
I have previously interned in Samsung R&D in the field of sensor fusion algorithms for navigation on Android. 
But I have never worked on open-source projects. One of my friend suggested it to begin with GSoC as his experience was very fruitful and he now a regular open-source contributor.
I regularly do C++ competitive programming and am very versed with STL and OOPs, so I thought it would be good to begin with open-source with C++. Also I had inclination towards video streaming, and thus I found inputstream.adaptive binary add-on for Kodi the best contender to begin.
Till now I got to learn a lot of stuff about adaptive streaming and the Slack community is also very responsive and amazing to begin with. Mostly I devoted time to understand the codebase of inputstream.adaptive and compiling Kodi myself and playing with Kodi tree by tweaking changes to understand the build flow. I recently had shifted to Linux, so while compiling, I also got a good understanding with Linux. I also faced CMake first time and now have a good understanding now. I would be looking to consistently work upon it.

GOALS
I have segmented the work roughly as 2 week targets.

Proposed Timeline:
01 June- 14 June

• Work on implementing a functionality to maintain buffer health in AdaptiveStream file. 
  
• A default buffer length will be set initially.
  
• A separate method function will be made in AdaptiveStream that will load the next segments from AdaptiveTree in a thread and will maintain a buffer health depending upon the constraints.
  
• Number of segments will depend on the format as single segment size depends on formats (mpd, ism, hls).
  

15 June- 2 July

• User UI input option for the buffer length duration to be made. 
  
• Implementation of best algorithm/logic to select best stream for smoothness depending on available network speed record and maintained buffer health.
  

03 July (PHASE 1 EVALUATION)

04 July- 19 July

• Work on implementation of stream selection when player window size is changed for a sufficient time. 
  
• Logic would be to periodically scan Kodi internal video player window size and then if the video resolution doesn’t match the window resolution(say for 10 seconds), it will switch.
  

20 July - 30 July

• Work on implementation of stream selection logic when computation power is getting limited.
  
• 2 parameters will be acquired from Kodi- CPU usage, memory usage. And according to it, toggling of the stream will happen.
  

31 July (PHASE 2 EVALUATION)

01 August- 14 August

• Start working on playback speed/ frame dropping.
  
• Add option to reduce playback speed say 0.97x, if network speed is low or CPU load is high.
  
• Also work out on possibility of dropping some frames.
  

17 August- 31 August

• Complete/tune all the above logic and build options so that user can configure priority of the above options.
  

24 August- 31 August (FINAL SUBMISSION)

• Polish all the code. 
  
• Complete Documentation
  

WHAT DOES IT TOUCH IN KODI
The work will be at inputstream.adaptive repository. Most of the work will be revolving around AdaptiveStream and AdaptiveTree classes, main function.  

REQUIREMENTS

• C++
  
• OOPs
  
• CMake
  
• Understanding of video streaming
  
• Familiarity with GIT
  

AVAILABILITY
In the upcoming months, I am completely free due to nationwide shutdowns. Also as this is my first org in terms of open-source contribution, I will be completely devoting my time working towards it.
I can easily give 40-50 hours a week till my college reopens. I intend to complete most of the work before my college reopens. Probably my college will reopen in August, then also I will be able to devote 30-40 hours a week. On Saturdays and Sundays there is no college and on Wednesdays I have only 2 hour session.
Also I usually devote 3 hours bi-weekly, for an NGO to distribute food for needy in the wake of Covid-19.  
Apart from above, I have no other commitments / vacations planned for the summers.
Also I do not have any internships this summer.

POSSIBLE MENTORS

• peak3d
  

[peak3d]

(...)

Reads good, just for completeness: You will also touch kodi and the inputstream API library sources to pull (or get pushed) the required constraints.
Thx much!

Edit: Regarding "Summary and Project Focus", inputstream.adaptive has an built-in video decoder for DRM protected streams.
In case libwidevine is used for decrypting DRM content, packets are first send to kodi, an then back to inputstream.adaptive through AddonVideoCodec interface.

libwidevine then decrypt and decode the packet into memory buffers provided from kodi.

[te36]

Sounds interesting, mtahir, but if you want to make readers excited that may not be familiar with the kodi internals of what does and does not work with adaptive bitstreams today, it would be great to provide some "user story" style explanation of what problems are being solved.

I don't want to badmouth the existing implementation, but from you work list i am guessing there are limitations today like:

- Can not adjust the bitrate at all during playback ?
- Chooses only initially a specific bitrate based on limited heuristics such as initial screen size, but not CPU limit or change in bandwidth / screen size ?
- Does not even manage to fill the buffer for the choosen bitrate in a way that minimized stop & go (in the backgrouns) ?
- Does not allow to set the buffer large enough for stop&go free playback if e.g.: internet rate is fluctuating too much ?
- What else.. ?

If any of these exist and you'll manage to improve on any of them, that would indeed be great.

The one thing i did not see and that i would highly suggest to include in your coding is user visible diagnostics:
When user presses "O" to get the stream diagnostics, all the important parameters of the adaptive streaming module should be shown, especially any new ones / heuristics you introduce.