[velocity]

I'd greatly appreciate some feedback on my proposal for working on RetroPlayer:


https://docs.google.com/document/d/1aGof...sp=sharing

[garbear]

Hi Vel0cityX,

I think you have a strong proposal. I've looked through your PRs stretching back through 2016 and it shows excellent familiarity with the open source process. If you can open a PR for one of the known issues it would cement your candidacy, but that's not required.

Shader support would be a perfect feature for GSoC. It's a nice, concise feature, and carries low risk - not much can go wrong.

Multiplayer would require a new version of RetroPlayer. Netplay depends on some external code, which is too risky for a GSoC project. So the idea would be to build a new RetroPlayer behind a set of abstractions that can be scaled to handle multiplayer in the future. I actually started with the algorithm used by Google's Atari AI and worked backwards to develop a theory of emulation that could handle all the complexity we could possibly throw at it. Implementing just the current functionality behind this abstraction would be a cool summer project.

So definitely create a proposal like yours just for shader support. If you're interested, I can finish my abstractions by the April 3rd deadline so we could build a proposal around a new version of RetroPlayer.

It depends on what you want out of GSoC. Shader support is more of a frontend feature, not so applicable to core emulation but working on a project the size of Kodi would be very valuable. From my player experience (RetroPlayer, a Live TV player, and Steam Link support in VideoPlayer) I can say that player development also gives you experience for the kind of complexity and performance qualities central to emulation.

[velocity]

Hello garbear,

I will start getting into it and perhaps submit a PR as well. I've been busy with other proposals so I haven't had time to work on KODI.

>Multiplayer would require a new version of RetroPlayer.

My bad, I was under the impression you had already started work on it.

>So the idea would be to build a new RetroPlayer behind a set of abstractions that can be scaled to handle multiplayer in the future. I actually started with the algorithm used by Google's Atari AI and worked backwards to develop a theory of emulation that could handle all the complexity we could possibly throw at it.

Ah the DeepMind project? Not sure how that relates, maybe you could elaborate?

>Shader support would be a perfect feature for GSoC. It's a nice, concise feature, and carries low risk - not much can go wrong.
>So definitely create a proposal like yours just for shader support.

So, I should modify my proposal to include just shader support, got it. I suppose this involves removing the "general bugfixes" goal as well.
About the first thing, I agree and as I've expressed it's one of my stronger interests, so I'm sure you understand why I have to go with that, at least for this GSoC.
Couple that with the fact that I don't know anything about KODI's codebase, I think it would be best if I started with a more manageable project.

Speaking of not knowing anything about KODI's codebase, I think I'll go ahead and fix that right now,

Cheers!


PS: I changed my proposal as requested. I'll probably be changing it a bit until the final deadline. I also added an "Availability" section at the end.

[garbear]

Updated proposal looks good. Support for shaders from other projects would be good, how portable are most shaders?

[velocity]

Good question, I will most likely go with GLSL ES 2.0 for maximum compatibility. At least for the default shaders, that is. I'm not sure how difficult it would be to support full GLSL.

GLSL ES is a subset of GLSL that as you can see supports embedded platforms (like mobile devices and the Rapsberry Pi).

I also have a RPi 3 so I will be able to use it for testing.

Edit: I just realized I misunderstood your question, I'll still leave that there since it's still slightly relevant.

If it's a GLSL ES shader, it's probably very portable. That's not a small if however, I imagine that most relevant shaders won't be (actually a lot of emulators are also on mobile so that might not be true). Not an issue though, I have some experience with converting shaders to ES.
If it's just a GLSL shader (most common) it might need a bit of work depending on how complex it is. If it's simple enough it might be a matter of literally renaming a few variables and adding a few preprocessor definitions.

[natethomas]

Can GLSL ES shaders be used in Windows and Mac, as well as in Linux environments?

[AchimTuran]

I guess Windows uses DirectX, because Kodi does not support OpenGL under Windows. It would be HLSL.

[RockerC]

Support for shaders from other projects would be good, how portable are most shaders?

Hope that Kodi will be able to reuse shader filters from RetroArch Slang/HLSL/Cg shaders collections similar to how it uses Libretro game cores:

https://github.com/libretro/slang-shaders

https://github.com/libretro/common-shaders

"These shaders are usable by either HLSL and/or Cg runtime compilers. The cg2glsl script will translate most of these into GLSL shaders."


Previous discussion thread:

http://forum.kodi.tv/showthread.php?tid=222198


Note! Please also remember that Kodi's existing renderers for VideoPlayer already uses shaders for scaling, deinterlacing, and other effects. See example:

http://forum.kodi.tv/showthread.php?tid=170855

Therefor maybe consider making all shaders usable in all Kodi's renderers could be fun to do? Not useful for end-users maybe, but fun.

I will most likely go with GLSL ES 2.0 for maximum compatibility. At least for the default shaders, that is.

Shaders written for the GLSL ES 1.0 specifications should be compatible with all Kodi platforms but Windows, then only have to port to HLSL for Windows.


Three most common types of shaders are written for GLSL, HLSL, Cg runtime compilers, of which GLSL ES 1.0 is probably the main standard for which you want to go with.

OpenGL Shading Language (abbreviated: GLSL, GLslang, or just Slang) is for OpenGL and OpenGL ES, so it is cross-platform for platforms where Kodi runs on OpenGL/ES.

HLSL (High-Level Shading Language) is a proprietary shading language developed by Microsoft for Direct3D and it is for Windows only, so only needed in Kodi for Windows.


Benefit of using GLSL ES 1.0 is that it is very portable code and there are even tools to help convert one code format to the other and vice versa, back and forth to GLSL.

https://docs.unity3d.com/Manual/SL-Platf...ences.html

https://docs.microsoft.com/en-us/windows...-reference

https://msdn.microsoft.com/en-GB/library...66865.aspx

https://github.com/google/angle
https://github.com/Microsoft/angle
https://github.com/aras-p/hlsl2glslfork


Cg was cross-platform but it is deprecated since 2012, though many existing open source Cg shaders are already available which could be ported to GLSL and/or HLSL.

Cg is very similar to HLSL as Nvidia developed it closley with Microsoft before HLSL and then Microsoft based HLSL on Cg, which made Nvidia stop developing Cg.

Just like there are tools to help convert GLSL to HLSL and tools to help convert HLSL to GLSL, there are tools to help convert Cg formated code to GLSL or HLSL code.

https://github.com/Themaister/RetroArch/...cg2glsl.py

It is for example possible to compile Cg shaders into GLSL shaders automatically using the cg2glsl script from the RetroArch development team as linked above.

[da-anda]

I hope the shader support would not be limited to just retroplayer but could be used by skins as well (at least abstracted in a way so that they can be applied to GUI later on)

[velocity]

Very informative post, thanks RockerC.

I will look into compatibility with libretro's shader system. There is detailed information about it here if anyone is interested:
https://github.com/libretro/slang-shader...ER_SPEC.md

I haven't worked with HLSL before, this should be fun!
Good to see that there are so many conversion tools too.

Can GLSL ES shaders be used in Windows and Mac, as well as in Linux environments?

Yes.

I hope the shader support would not be limited to just retroplayer but could be used by skins as well (at least abstracted in a way so that they can be applied to GUI later on)

Shaders on GUI huh, we'll have to discuss that. This proposal is just for RetroPlayer but there may be a way to abstract shader support.

[AchimTuran]

I hope the shader support would not be limited to just retroplayer but could be used by skins as well (at least abstracted in a way so that they can be applied to GUI later on)

I guess this is to much for a gsoc project. But if they are nicely integrated behind and API, I guess we will find a dev that could make it possible to use them in the UI.

So please let the guys focus on RetroPlayer, because it is a big project. Adding stuff could be done afterwards or if some time is available.

[AchimTuran]

@Vel0cityX

Just some notes about what garbear already wrote. It's good to focus during gsoc on one task like sharers or network play. If you really like the Kodi project, there is enough time to contribute to Kodi after gsoc.
So if you wanna use/see network play you could start to develop it afterwards. Then you will have the benefit that you're already familiar with the codebase and buildprocess.
We re always searching for developers that share our love to create Kodi.
Don't feel a pressure it's only a possible way you could choose.

[sarbes]

I would LOVE to see a shader system for video too. I don't know the details of the current Retroplayer implementation, but a general (plugin based) shader pipeline should be possible.

For me, an optimal rendering pipe would be:
1. YUV manipulation (e.g. scaling lumina)
2. YUV to RGB conversation (e.g. YUV2RGB, LUT shader)
3. RGB manipulation (e.g. 2D LUTs [for GB games])
4. Scaling (e.g. Lanczos, NNEDI)
5. Post processing (e.g. dithering, line shading)

Obviously, steps 1 and 2 are not necessary for most (all?) emulators, but it would enable the processing of games, videos and images with the same infrastructure/shaders.

The shader itself should be able to:
- accept the previous image from the chain and pass the output to the next shader
- get the image size of the input
- set the resulting image size if it is a scaler
- get/set dynamic vectors
- load images specified by the plugin itself
- store and load additional textures.

Fragment shaders should be enough for most applications. Vertex shaders would be fancy, but also a bit "over the top".

[FernetMenta]

Note that RetroPlayer in current state uses private APIs of VideoPlayer which is not ok. RP is not supposed to use VP's renderining system. Bfore looking into shaders for RP, RP needs to implement its own rendering.

[FernetMenta]

Note that RetroPlayer in current state uses private APIs of VideoPlayer which is not ok. RP is not supposed to use VP's renderining system. Before looking into shaders for RP, RP needs to implement its own rendering.