2014-02-09, 21:17
I have finally figured that out. I can get the current window and current control by calling
Now the exercise is to find out how to enable notifications and look for a notification to call this on so I don't have to be crazy and poll for this kind of info. With this working I am afraid setting XBMC up might be a bit awkvard but browsing and playing the library should be possible. If you are reading this and have better experiences and / or understanding of this API can you please help me? For a skilled coder this might really be a simple thing to do I hope.
Edit: oops what's the ID in that request. I think it changes and where to get it.
Edit2: I am now also looking at python scripts / addons development. It appears it is possible to retrieve current window via python library. Also it is possible to query the parent window and left, up, right down in the navigation direction. The UI elements inherit from the Control class, and the classes representing concrete UI elements include ControlButton, ControlCheckMark, ControlEdit etc. Pure theoretically speaking if there is a way to have a plugin running in background in its own thread, responding to the input events, checking, traversing and inspecting XBMC windows using this API we might be able to write something that would resemble a screen reader for XBMC. I would really like to play with this more. Can I somehow create at least basic script / plugin which will run all the time or run at least when I press a keystroke?
Edit3: umm. It appears it is not as easy as it looked initially. I am unable to figure out if it is at all possible to hook into a global events. I know there is a way to respond to events in my own window, however for screen reader like functionality we need to be able to react when focus changes, when properties of a control change etc.
Edit4: one more issue. From python it appears it is only possible to access UI elements also created in python. It's discussed here with other issues usefull for what we are after.
Edit5: I have looked at the RPC API again as with that I had already been able to at least feel how the navigation works. I have nearly added an audio source to my running XBMC by periodically manually polling for the current control via telnet. Unfortunatelly this is not sufficient for a comfortable XBMC experience because it only communicates control labels. Also neither in python nor with this API we do have a kind of events / notifications we can process so without the help from the core team we are unlikelly to be able to move forward I am afraid.
Edit6: Okay I have created a simple python script which will periodically check the current control using xbmcjson python library and feed that to the NVDA screen reader on windows and speech-dispatcher on linux. For adding sources it's sufficient. For controlling the XBMC I will have a look at remote clients for IOS and Android. I don't think this is even close to a proper solution so I have nothing to post.
Edit7: Did a bit more reading and here are updated ideas:
- It is possible to create service type addons which will run automagically at xbmc start. I might be able to create addons in python that will provide functionality to call native tts according to the platform xbmc is running on. E.G. it will do text to speech via NVDA or via Sapi5 on Windows and there might be other addons with the same functionality which will use speech-dispatcher for TTS output on linux or even self-contained addon which will use eSpeak directly on open elec or other such enviromments. I am not targetting IOS, Android and xbox at this time.
- Since we have no events to bind to we might be able to provide alternate keyboard and remote control commands for navigating the XBMC UI. These commands will call the original navigation commands e.g. left, up, right, down, page up, page down etc and they will also retrieve current control and send that for TTS output by the service addons suggested above. I need to figure out if it is possible to include the keybindings within the addon or if it is possible to include additional key mappings. We might either be able to come up with alternate keyboard commands e.g. shift+arrow keys instead of plain arrow keys or we might be able to provide turn on / turn off screen reader functionality which will exchange the key mappings. I am afraid this leaves issues with touch support which will require implementing platform specific accessibility support or rewriting touch input support to provide seperate screen reader like experience.
Edit8: okay I have verified I can just drop in user modified keyboard layout calling parts of my addon. Also I have learned how to package python libraries. I should stop writing these reports and get to some coding.
Code:
{"id":0,"jsonrpc":"2.0","method":"Gui.GetProperties","params":{"properties":["currentcontrol","currentwindow"]}}Edit: oops what's the ID in that request. I think it changes and where to get it.
Edit2: I am now also looking at python scripts / addons development. It appears it is possible to retrieve current window via python library. Also it is possible to query the parent window and left, up, right down in the navigation direction. The UI elements inherit from the Control class, and the classes representing concrete UI elements include ControlButton, ControlCheckMark, ControlEdit etc. Pure theoretically speaking if there is a way to have a plugin running in background in its own thread, responding to the input events, checking, traversing and inspecting XBMC windows using this API we might be able to write something that would resemble a screen reader for XBMC. I would really like to play with this more. Can I somehow create at least basic script / plugin which will run all the time or run at least when I press a keystroke?
Edit3: umm. It appears it is not as easy as it looked initially. I am unable to figure out if it is at all possible to hook into a global events. I know there is a way to respond to events in my own window, however for screen reader like functionality we need to be able to react when focus changes, when properties of a control change etc.
Edit4: one more issue. From python it appears it is only possible to access UI elements also created in python. It's discussed here with other issues usefull for what we are after.
Edit5: I have looked at the RPC API again as with that I had already been able to at least feel how the navigation works. I have nearly added an audio source to my running XBMC by periodically manually polling for the current control via telnet. Unfortunatelly this is not sufficient for a comfortable XBMC experience because it only communicates control labels. Also neither in python nor with this API we do have a kind of events / notifications we can process so without the help from the core team we are unlikelly to be able to move forward I am afraid.
Edit6: Okay I have created a simple python script which will periodically check the current control using xbmcjson python library and feed that to the NVDA screen reader on windows and speech-dispatcher on linux. For adding sources it's sufficient. For controlling the XBMC I will have a look at remote clients for IOS and Android. I don't think this is even close to a proper solution so I have nothing to post.
Edit7: Did a bit more reading and here are updated ideas:
- It is possible to create service type addons which will run automagically at xbmc start. I might be able to create addons in python that will provide functionality to call native tts according to the platform xbmc is running on. E.G. it will do text to speech via NVDA or via Sapi5 on Windows and there might be other addons with the same functionality which will use speech-dispatcher for TTS output on linux or even self-contained addon which will use eSpeak directly on open elec or other such enviromments. I am not targetting IOS, Android and xbox at this time.
- Since we have no events to bind to we might be able to provide alternate keyboard and remote control commands for navigating the XBMC UI. These commands will call the original navigation commands e.g. left, up, right, down, page up, page down etc and they will also retrieve current control and send that for TTS output by the service addons suggested above. I need to figure out if it is possible to include the keybindings within the addon or if it is possible to include additional key mappings. We might either be able to come up with alternate keyboard commands e.g. shift+arrow keys instead of plain arrow keys or we might be able to provide turn on / turn off screen reader functionality which will exchange the key mappings. I am afraid this leaves issues with touch support which will require implementing platform specific accessibility support or rewriting touch input support to provide seperate screen reader like experience.
Edit8: okay I have verified I can just drop in user modified keyboard layout calling parts of my addon. Also I have learned how to package python libraries. I should stop writing these reports and get to some coding.
