[yol]

Hi,


I've seen that there is a suggested task in the Wiki list about (re)introducing Wayland support into Kodi, which I'm interested in. I've searched around a bit and found the old pull request that was rejected. But it seems that the Kodi Wayland fork of Max Kellermann is still being actively maintained and updated. So, for GSoC, what exactly should be done? Is it expected to grab Max' code and try to get it merged (with modifications if necessary)? If so, this would be very difficult to propose as a project without knowing exactly what kinds of problems the Kodi team sees with the current state of the code - which is probably quite different from back when the Wayland support was removed from the main repository. Or are there substantial problems with the current solution that require a complete rewrite from scratch?

[RockerC]

I read that Wayland was mainly removed because the developer maintaining quit maintenence on the code, but I now however suspect that it also did not fit with the work on the new agile VideoPlayer development:

http://forum.kodi.tv/showthread.php?tid=237686
http://forum.kodi.tv/forumdisplay.php?fid=240

So if you have time right now then you could just try to merge the Wayland code into FernetMenta's "kodi-agile" fork of Kodi to see if it will even fit with all the reworked code changes for Kodi already in there:

https://github.com/FernetMenta/kodi-agile/

Another tip is to checkout all the merged pull requests and the long dicussions in them about Wayland (and Mir windowing system too) in older pull requests before they was merged:

https://github.com/xbmc/xbmc/pull/10898
https://github.com/xbmc/xbmc/pull/11040
https://github.com/xbmc/xbmc/pull/2978
https://github.com/xbmc/xbmc/pull/3001
https://github.com/xbmc/xbmc/pull/8980


In addition to that I suggest that you and all students head some of the general GSoC submission advice given here as well:

http://forum.kodi.tv/showthread.php?tid=...pid2545338

other than compiling Kodi and looking at the code, you and other students should really checkout all the accepted proposals from previous years of GSoC:

http://forum.kodi.tv/forumdisplay.php?fid=220
http://forum.kodi.tv/forumdisplay.php?fid=191
http://forum.kodi.tv/forumdisplay.php?fid=161

If you read through all those accepted proposals then you should get an idea how to write your own proposal and how-to engage with the community here on the forum.

Another tip from previous years not mentioned here yet is to submit your proposal(s) early as you can always make changes to details of your proposal(s).

You can submit more than one prososal. You could post one seperate proposal for each idea, as that means less risk all of your ideas conflict with other students.

Advice from looking at previous years would be; submit early, submit many, submit revisions often, and get feedback in back-and-forth discussions with community.

[FernetMenta]

Hi,


I've seen that there is a suggested task in the Wiki list about (re)introducing Wayland support into Kodi, which I'm interested in. I've searched around a bit and found the old pull request that was rejected. But it seems that the Kodi Wayland fork of Max Kellermann is still being actively maintained and updated. So, for GSoC, what exactly should be done? Is it expected to grab Max' code and try to get it merged (with modifications if necessary)? If so, this would be very difficult to propose as a project without knowing exactly what kinds of problems the Kodi team sees with the current state of the code - which is probably quite different from back when the Wayland support was removed from the main repository. Or are there substantial problems with the current solution that require a complete rewrite from scratch?

Hi, thanks for showing interest. New Wayland support should be implemented the same way like X11 or Mir. Mir support is relatively new can can be taken as a good example of how it should be. In Kodi's context Wayland is a platform or windowing systems. Check out X11, Windows, or Mir folders under xmbc/windowing:
https://github.com/xbmc/xbmc/tree/master/xbmc/windowing

The old implementation was done in the messy structure in egl (windowing/egl). EGL itself is not a platform or windowing systems, means that basically all platforms you find under the egl folder need rework.

A requirment of the new Wyland support is a binding to OpenGL. GLES is optional and not required in the first round. The old code did only support GLES like the other plarforms that share the mess under windowing/egl.

The new implementation should consider that we finally want support for wayland protocols like "presentation time".

[yol]

Hi,
thanks for taking the time to guide me through the current state.

So if you have time right now then you could just try to merge the Wayland code into FernetMenta's "kodi-agile" fork of Kodi to see if it will even fit with all the reworked code changes for Kodi already in there:

I've taken a look at it, but it does not seem to affect the windowing code in a particularly relevant way (yet).

Hi, thanks for showing interest. New Wayland support should be implemented the same way like X11 or Mir. Mir support is relatively new can can be taken as a good example of how it should be. In Kodi's context Wayland is a platform or windowing systems. Check out X11, Windows, or Mir folders under xmbc/windowing:
https://github.com/xbmc/xbmc/tree/master/xbmc/windowing

The old implementation was done in the messy structure in egl (windowing/egl). EGL itself is not a platform or windowing systems, means that basically all platforms you find under the egl folder need rework.

Seems Max Kellermann has started dealing with this: https://github.com/MaxKellermann/xbmc/co...f140293619 - granted, it's only renaming files, but it shows that he might be considering to continue to work on this. I would want to avoid having two people work separately on this, which means some coordination/communication might be required.

So going forward, am I understanding you correctly that you want relevant portions of the EGL code (which will still be needed since Wayland requires EGL for GL contexts) moved/copied to the Wayland windowing system, like it was done with Mir? Or do you want some kind of central place for the EGL code, just not the way it is now (thinking about the other platforms based on egl that will also need to be modified some day)? I think that the CGLContextEGL required for Wayland would be mostly equal to the one Mir currently uses.

The new implementation should consider that we finally want support for wayland protocols like "presentation time".

Am I correct that you are referring to this? Should it be used for implementing CVideoSync or do you have anything else in mind? Also, it seems that sadly no compositor except weston even supports it at the moment (I've quickly checked the arguably most important ones, mutter and KWin).

[FernetMenta]

Seems Max Kellermann has started dealing with this: https://github.com/MaxKellermann/xbmc/co...f140293619 - granted, it's only renaming files, but it shows that he might be considering to continue to work on this. I would want to avoid having two people work separately on this, which means some coordination/communication might be required.

First time I see this. Max Kellermann seems not to accept any guidance so this approach may take forever.

So going forward, am I understanding you correctly that you want relevant portions of the EGL code (which will still be needed since Wayland requires EGL for GL contexts) moved/copied to the Wayland windowing system, like it was done with Mir? Or do you want some kind of central place for the EGL code, just not the way it is now (thinking about the other platforms based on egl that will also need to be modified some day)? I think that the CGLContextEGL required for Wayland would be mostly equal to the one Mir currently uses.

At this time I don't see any benefit from common code for EGL. It's only few line anyways. Have a look at the history of the egl floder. There are only platform specific changes. If it turns out that some common code could be factored out, it can be done at a later time.

Am I correct that you are referring to this? Should it be used for implementing CVideoSync or do you have anything else in mind? Also, it seems that sadly no compositor except weston even supports it at the moment (I've quickly checked the arguably most important ones, mutter and KWin).

Yes, we need a solution for CVideoSync because CVideoSyncDRM and CVideoSyncGLX are deprecated. Using wp_presenatation was actually a suggestion by Intel developers.

[yol]

First time I see this. Max Kellermann seems not to accept any guidance so this approach may take forever.

I asked him by mail directly and he said that he's working on it from time to time but he does not intend to have it merged into mainline Kodi at the moment or anytime soon, so that's that.

At this time I don't see any benefit from common code for EGL. It's only few line anyways. Have a look at the history of the egl floder. There are only platform specific changes. If it turns out that some common code could be factored out, it can be done at a later time.

Agreed.

Yes, we need a solution for CVideoSync because CVideoSyncDRM and CVideoSyncGLX are deprecated. Using wp_presenatation was actually a suggestion by Intel developers.

Yep it seems like the natural choice. Might have to try to push compositor implementors to actually support it, though.

OK, so is there anything else you would want to have in the Wayland implementation (or possibly related topics)? I'm still unsure whether I can convincingly fill a 3-month timeline with it as-is.

[FernetMenta]

OK, so is there anything else you would want to have in the Wayland implementation (or possibly related topics)? I'm still unsure whether I can convincingly fill a 3-month timeline with it as-is.

CVideoSync should work. That is for driving video player clock by vertical retrace.
In addition we need some means to tell how long it will take for a submitted frame to be on screen. Currently we don't really know. We try to guess based on double/triple bufer setting.

Proper handling of multiple monitors. Assume you have configured Kodi to use the projector. User powers up the system with projector being off. Later projector comes online. Output should switch to projector automatically.

Rework use of OS sreensaver. I think this is partly broken in current version.

[yol]

CVideoSync should work. That is for driving video player clock by vertical retrace.
In addition we need some means to tell how long it will take for a submitted frame to be on screen. Currently we don't really know. We try to guess based on double/triple bufer setting.

If presentation-time is supported, that can probably be calculated from the time difference between calling commit() on the surface and the time in the presentation event.

Rework use of OS sreensaver. I think this is partly broken in current version.

Doesn't seem to be implemented at the moment. There's an unstable wayland protocol (idle-inhibit) for it, but it's also not implemented on all compositors. Alternative would be to use the org.freedesktop.ScreenSaver interface on DBus. That's not wayland-specific, though.

I'll write up a proposal based on the discussions, shall I make a new thread or just post it as reply here?

[FernetMenta]

If presentation-time is supported, that can probably be calculated from the time difference between calling commit() on the surface and the time in the presentation event.

Don't underestimate. Main/gui thread may be blocked for what reason ever and only output every other frame or so.

I'll write up a proposal based on the discussions, shall I make a new thread or just post it as reply here?

Great. I think posting it here is fine.

[yol]

Wayland Support

Name:
Philipp Kerling

Forum:
yolk

Summary:
Wayland is a set of protocols for communication between applications and compositors that is expected to replace Xorg in the Linux graphics stack in the next years. Mainline Kodi currently lacks support for Wayland and must thus use an Xwayland-emulated X11 session for display which is not optimal in the long run. Therefore, native support for Wayland should be implemented in Kodi.

How will I achieve this:
There was a prior implementation of Wayland support that was removed: 8980 (PR). However, taking the existing code at https://github.com/MaxKellermann/xbmc and fixing the problems that lead to the removal is more efficient than re-implementing from scratch, so this is what I would do. In detail, the Mir windowing system support that is already implemented in Kodi will serve as model for the parts that integrate into Kodi while many components interacting with Wayland can be taken from the current Wayland code. Additional features (see goals) can then be added after the cleanup.

What will the project focus on:
The main focus is getting basic Wayland support ready for merging into Kodi first and then implement additional features. Compositor interoperability testing will focus on Weston (Wayland reference compositor), mutter (GNOME) and KWin (KDE).

Benefits:
Users can run Kodi natively on Wayland compositors which are the default already in some Linux distributions (e.g. Fedora) and enjoy improved performance and video playback since the Xwayland X11 emulation is no longer necessary. Going forward, Wayland support will become increasingly important as more distributions make the switch. Thus native support for Wayland is a prerequisite for Kodi being able to offer a competitive user experience on Linux in the future.

Goals:

1. Basic Wayland support
   
   1. Rebase the work of Max Kellermann on current Kodi or kodi-agile git, filtering out stuff unrelated to Wayland
      
   2. Make it independent of CWinSystemEGL and run on full OpenGL
      
   3. Fix fullscreen on HiDPI displays with scale != 1 (does not seem to work currently)
      
   4. Make sure everything else works as expected
      
2. Additional features (in order of priority)
   
   1. Support for the presentation-time protocol extension as CVideoSync implementation
      
   2. Provide ways of controlling the compositor screensaver
      
   3. Allow selecting the preferred wl_output to display on and react to hot-plug events by moving the surface to a different output if necessary
      
   4. Make sure VA-API (still) works
      
   5. Make it run on both OpenGL and OpenGLES
      
   6. Basic touch input single-tap support
      
   7. Windowed mode support (not that easy since it requires Kodi to draw client-side window decorations and support wl_shell_surface::move/resize)
      
3. Possible extensions (if desired and time allows, in no particular order)
   
   1. Make it possible to include both Wayland and X11 (as fallback) in the same build as this would be preferable for distribution packages
      
   2. Support for zero-copy video rendering of video to YUV surfaces (Wayland has native protocol support for YUV surface buffers) - questionable though if the benefits would justify the work that has to be done for this
      
   3. Support for fullscreen with the unstable fullscreen-shell protocol - would allow output mode switching (in theory)
      
   4. Full touch input support with gestures (probably needs to use CGenericTouchInputHandler for gesture detection, wl_touch interface is very similar to evdev though)
      

What does it touch in Kodi:

• Windowing subsystem: New Wayland windowing system must be added
  
• Display configuration menus: The different feature set of Wayland might make it necessary to introduce additional configuration options
  
• Video sync: Depending on the information that Wayland can deliver, the video sync interface could be augmented in order to support this additional information
  
• Video player/rendering: Using YUV surfaces would mean that a compatible renderer must be implemented that renders directly to a Wayland subsurface buffer and not GL
  

Requirements:

• GPU(s) with open-source driver(s) that natively support Wayland, VA-API (e.g. Intel)
  
• C++
  
• Knowledge of Wayland protocols and interaction with EGL/GL/GLES
  

Possible problems:

• Support for many Wayland protocols is very limited in practical compositors
  
• Information about outputs is very limited with Wayland - Monitors can only be identified by manufacturer and model, basically: this is a problem when someone has multiple monitors of the same model
  
• Video sync is very complicated to get right
  
• Screensaver is difficult to control compositor-independently: There is an idle-inhibit unstable Wayland protocol, but support varies between compositors; GNOME exhibits org.freedesktop.ScreenSaver interface on DBus but does not implement it currently - only way there would be to use org.gnome.SessionManager (which is GNOME-specific) etc.
  

Possible mentors:
tbd

[FernetMenta]

Sounds good!

Some comments:

1)
I won't use the old egl base as a starting point but that is your decision.

2)
I would stress the benefits more. Wayland support is absolutely essential for Kodi on Linux. X11 is dying.

3)
VDPAU won't work because it requires X11. Currently it looks like VDPAU is dead too.

[yol]

Thanks for your comments!

1)
I won't use the old egl base as a starting point but that is your decision.

I was thinking more along the lines of using the Wayland support code that is there (like the protocol objects etc.) and marry it with the EGL/Kodi code from Mir, so I've clarified that part. Guess another approach would be to just copy the files that would still be relevant without a complete rebase of the old code, which would save some time if you're fine with losing the history (shouldn't matter much).

2)
I would stress the benefits more. Wayland support is absolutely essential for Kodi on Linux. X11 is dying.

I've tried to make it a bit more dramatical ;-)

3)
VDPAU won't work because it requires X11. Currently it looks like VDPAU is dead too.

Ah OK - I didn't look specifically into VDPAU. But if that's clear there is no reason to include it in the proposal, so I've removed it.

[RockerC]

3)
VDPAU won't work because it requires X11. Currently it looks like VDPAU is dead too.

Ah OK - I didn't look specifically into VDPAU. But if that's clear there is no reason to include it in the proposal, so I've removed it.

Could substitute VDPAU with VAAPI (VA API)? http://en.wikipedia.org/wiki/Video_Acceleration_API

[FernetMenta]

3)
VDPAU won't work because it requires X11. Currently it looks like VDPAU is dead too.

Ah OK - I didn't look specifically into VDPAU. But if that's clear there is no reason to include it in the proposal, so I've removed it.

Could substitute VDPAU with VAAPI (VA API)? http://en.wikipedia.org/wiki/Video_Acceleration_API

AMD is working on this.

[RockerC]

Ah OK - I didn't look specifically into VDPAU. But if that's clear there is no reason to include it in the proposal, so I've removed it.

Could substitute VDPAU with VAAPI (VA API)? http://en.wikipedia.org/wiki/Video_Acceleration_API

AMD is working on this.

Nvidia is working on VAAPI support to replace VDPAU? Or is AMD working on VAAPI support to replace XvBA?