Thread Rating:
  • 2 Vote(s) - 5 Average
Current state and vision about AudioDSP
#1
Hi community,

today I want to share my vision about AudioDSP. I think there is a huge gap between the community and the developers, which are involved into AudioDSP. Maybe the most of you don't know what AudioDSP will offer to us. So hopefully this post will reduce the gap, otherwise feel free to ask your questions in this post.

Introduction
ADSP or AudioDSP stands for Audio Digitial Signal Processing and is a technique to transform or modify signals with computers. These signals and the theory is presented in the digital domain. They heavily relay on mathematics and computer science. Most of you use this technique every day without knowing it. It's used in wireless communication, image processing, ... and even for audio signals. A very familiar procedure is to to boost or cut the low frequency range of your subwoofer, which is possibiliy connected to an AV-Receiver. Currently these technique is outside Kodi and it would be nice to have it all in one place with a customizionable setup process.
Furthermore it extends setups, which aren't connected to an external AV-Receiver.

If you wanna learn more about digital signal processing, a quick google search can give a lot of stuff to read, e.g. this wikipedia article gives you more or less a better introduction than I did.

For me this theory and technique is awesome, but I will not bore you with it Blush. I think the most of you want know what Kodi will offer to us with this new feature. In theory it could improve our lovely player so much with new functionalities that aren't available in commercial products. For example custom scripts to calculate your filter design equation, head phone simulations ...

Our core system implementation is cross platform and will be available on all supported Kodi platforms like Windows, Android, Linux ... (I think you know the list otherwise follow this link). The processing modes are delivered through binary add-ons, which are compiled for each platfrom and deliverd with every Kodi release or our development builds (similiar to PVR add-ons). Every add-on could offer several processing modes, which are described in the next paragraph.

The core system supports the following processing steps:
  1. input-resampling, which could resample the audio stream before entering the processing chain. This step supports mutliple add-on, but only one can be set to active or enabled status.
  2. pre-processing, is the process that is performed before the stream enters the master-processing stage. E.g. here you could amplify one specific audio channel or process a custom filter for equalizing. This category supports multiple active add-ons.
  3. master-processing can be seen as a upmix or downmix step to map the audio signal to the sinks layout. It supports multiple add-ons with one active processing mode.
  4. The post-processing step could e.g. calculate an audio equalizer. Again multiple active add-on modes.
  5. The last step is output-resampling, which could resample your audio stream to your sinks sample frequency. It has the same restrictions as input-resampling.

As you see the system is very flexibel and this feature is on going. You should see my presented processing/modes steps as examples, maybe other devs have different use cases or ideas. At the time of this writting we don't have add-ons for all steps, maybe some day Huh.

Here is an image that should give you a good overview:
Image
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#2
Status of AudioDSP
The first version of AudioDSP is fully integrated into Kodi's core codebase. Alwin and I tested it under Linux and Windows. The API of the add-on side should also be stable. It's a pure C implementation and gives you the full flexibility for add-on creation, but it requieres a lot of configuration. Because of that reason I started to develop a "small" (it grows with every new add-on) framework wich is called adsp.template. As you see it is still under heavy development and I plan to do a huge refactor, which will simplify the usage. If it is stable enough I will create a new thread in the AudioDSP development forum.

Available AudioDSP Add-Ons
adsp.basic
It started as the first add-on for AudioDSP. It's usecase is to provide basic and usefull tasks for audio processing.

Available modes:
  • Speaker Gain. Here you can set gain values for all speaker channels of your configured ActiveAE sink.
  • Speaker Delay. Here you can set delay values for all speaker channels of your configured ActiveAE sink.
  • Dolby Pro Logic II downmixes your audio stream to stereo if your sink is configured with 2.0.

Supported platforms:
  • All except ARM Android based systems. I don't know why but our jenkins build still fails. Maybe someone can help

Planned features:
  • Adapt this add-on to adsp.template, which will increase maintainability. Consequently adding new features or modes would be faster and easier.
  • Add a new mode to allow audio channel remapping. Currenlty it's a pain to setup the audio channels with the help of each platform Kodi supports.
  • Fix the ARM Android build.

adsp.freesurround
Is based on foo_dsp_fsurround, which is a open source plugin for foobar, from pro_optimizer. Its purpose is to decode surround information from your stereo music to produce 5.1 output.
It works with any stereo surround sound format like - Dolby Surround, Dolby ProLogic, Dolby ProLogic II... for the full list please read the forum thread from pro_optimizer. It will also faithfully play back your 'plain old' stereo music, with the exception that echoes are usually played over the surround speakers, and some sound sources, usually strongly distorted guitars, choirs etc., tend to surround the listener instead of being played over the front speakers alone.

Available modes:
  • Freesurround upmix is available if you configure your ActiveAE sink through more than 2.0 (stereo).

Supported platforms:
  • All except ARM Android based systems.

Planned features:
  • Adapt add-on to adsp.template, which will increase maintainability. Consequently adding porting this add-on to newer API versions will be faster.
  • Fix the ARM Android build.

adsp.template
This is a small framework for faster AudioDSP add-on development. My dream is that the developer only has to code the processing algorithms in C/C++ and the basic logic of the user interface in C/C++ and or XML. Consequently the backend system has to handle the registered settings parameters and the communication between the processing algorithms and the user interface.
Future versions of this add-on can be seen as an example how to quickly write AudioDSP add-ons.

Available features:
  • Currently the template can be configured with a few header files, which allows to define the available processing modes.
  • Furthermore the template has a settings class to write, read and restore user settings.
  • A basic message system implementation for communication between the user interface and the processing class.
  • Exception handling for error reports inside an AudioDSP add-on.
  • Basic GUI dialog class for faster user interface development.

Supported platforms:
  • Should be supported on all platforms, but I haven't tested it.

Planned features:
  • For a future version will include an tutorial on how to use it.
  • The configuration with the header files is quite good, but it doesn't feel right. So the next step for the template will be to create a system, which can handle automatic object registrations. With that system the developer only has to create a new class and derive it from a basic AudioDSP processing class. Consquently the class can connect to the internal AddonHandler, which handles object creation and tracks the status of this mode.
  • Add more generic helper classes for faster developments.


adsp.biquad.filters
This add-on is based on adsp.template and asplib, which implements a biquad filter structure for signal filtering. The filter structure is very familiar in audio processing and allows the realisation of almost all filter types like low-, high-, band-pass ...
Currently this add-on only has one design equation for an graphical equalizer with 10 frequency bands and a main gain value. It processes all audio channels with the same filter configuration except the LFE (Low Frequency Effects or sub woofer) channel. This channel should be created or processed by another pre-processing add-on when LFE is not available.

Available modes:
  • Post processing mode for all available audio sink channels except LFE.

Supported platforms:
  • Windows
  • Linux

Planned features:
  • The add-on source code is cross platfrom, but asplib is not ported to all supported platforms by Kodi. Consequently the add-on is not usable on all platforms. So the next step for this add-on is to port asplib to all platforms.
  • Create a interface that allows loading of exported filter files from e.g. Room EQ Wizard
  • Integrate scripting interface to shift filter design equations to another programming language like Python (which is already inside Kodi).
  • Direct filter assigning to a custom created post- or pre-processing mode. This will allow faster configuration changes or sharing your setups.
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#3
Vision about AudioDSP
I like to see AudioDSP enabled per default. Consequently it will process all audio signal processing tasks. Furthermore I would like to see internal modes, which are able to modify ffmpeg parameters for audio inside ActiveAE.
One of the next steps for our add-ons is to get the continous integration builds working with Travis CI, Appveyor and native installers for every platform that Kodi supports. This will give us the possibility to release add-ons without our normal merge windows and e.g. let us create something similiar to an add-on market. I guess we could use repository add-ons for that task. I'm already able to compile the add-ons for Linux, Windows and OS X, but the deployment processing is still missing. Both services are very great because you can nicely integrate them into your general workflow with Github. On every commit you create it will build the add-ons for you, which is very awesome because you can see if it works or not on other platfroms without setting up the build environment locally. If someone creates an pull request against your add-on repository it will also start a new build job and show the status on the pull request page. Very similiar to our jenkins server but without setting up an external server or deal with jenkins jobs. If you wanna use the services just copy the files appveyor.yml and .travis.yml to your project. Then login with your Github account and enable the build for your repository. How awesome is that?Big Grin
This will give 3rd party developers, which are not part of the team the possibility to work on binary add-ons and deploy them very easily on hopefully all platforms that are supported by Kodi.

For deployment I plan to use Bintray, which is also free for open source projects. With that you can manage your releases. If you don't like this service you can use Github releases.

You see I'm currently working more on backend tasks, which will help others or me to faster develop AudioDSP add-ons for many platforms without much effort. I also started to work port and extend asplib which will be finished soon. You can track the current status here.

I have many ideas for add-ons, but I'm not sure if I will find the time to implement them all. Hopefully one dayWink

This is list:
  • adsp.xconvolver, which I have already started. This add-on will include a convolution engine for FIR-filters. This filters can be used for binaural, room correction, digital crossover filters, cross-talk cancellation ...
  • adsp.dynamics that would bring a compressor and
  • adsp.remapper to map audio channels inside Kodi without the need of an OS audio mixer
  • many more, I guess I have to extend the list in the future Wink
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#4
FAQ
When will it be available?
The first step with the merge of AudioDSP inside Kodi's code base is already done. But all AudioDSP related stuff should be seen as a ongoing project. Now we can continously improve things like bug fixing, working on add-ons, port them to other platforms that are supported by Kodi and so on.

Can I use AudioDSP with passthrough?
No that is not possible, because a passtrough stream contains raw audio data and will not be encoded by ActiveAE. Consquently AudioDSP is not able to process your requested audio stream.
But ffmpeg (Kodi's basic decoder for everything) is able to decode the most audio formats, which makes it possible to use AudioDSP for almost all audio streams without passtrough. I only used passtrough before AudioDSP was available. Blush Now I use multichannel pcm and enjoy the freesurround add-on for stereo streams.

How can I contribute?
Any contribution is welcome. E.g. you could fork one of our add-ons at kodi-adsp and start working on it. Afterwards send us an PR and we can try to merge your work. Also skinning is very welcome. If you're familiar with signal processing and know how to code algorithms in C/C++ you're welcome tooBig Grin
So please let me know your interest by starting a new thread or writting me a private message. For more information on how to contribute please read our how to get involved page, coding conventions and how to submit a patch.
But please stop asking me if adsp.xconvolver is available for platform X or when will be addon Z available at X. When there is a release or something to test I'm sure that I will create a new post about it.

Where can I find documentation about AudioDSP?
Currently no where Blush until Alwinus is finished with the add-on library rework. A preview of his work is available here. We are in an early stage and more heavily involved in development for AudioDSP. But the C-API is now very stable. In future you could use my adsp.template, which will make it very easy to integrate new processing algorithms. A preview can be seen here and here.
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#5
Changes

15.03.2016
  • Removed work in progress prefix
  • Added FAQ section
  • Removed ongoing title
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#6
Looking forward for room correction. But is it necessary to decode dts/ac3 on raspberry pi 2 to get room correction working or will it manipulate the dts stream and send it to my Logitech z906?
Reply
#7
Hallo Wisler,

I installed Kodi 16 OSMC on Raspberry Pi 2. There is only available adsp.basic and adsp.freesoround. Is there a simple way to get adsp.biquad.filters installed?

Thank you

Meggs
Reply
#8
(2016-02-24, 10:43)Meggs Wrote: Hallo Wisler,

I installed Kodi 16 OSMC on Raspberry Pi 2. There is only available adsp.basic and adsp.freesoround. Is there a simple way to get adsp.biquad.filters installed?

Thank you

Meggs

Hi Meggs,

perfect timing Wink.

Since a few days it should be available with our latest master builds. Furthermore it should be available for OS XBlush. See my pull request.
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#9
how to get working DSP addons ? for hd-ma audio ect ect

kodi + nvidia shield tv <-- updated
Reply
#10
(2016-03-09, 15:37)kilzo Wrote: how to get working DSP addons ? for hd-ma audio ect ect

kodi + nvidia shield tv <-- updated

Currently they're only available in our nightly builds. They will need a backports to Android when someone tested them enough.
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#11
(2016-03-05, 23:52)wisler Wrote:
(2016-02-24, 10:43)Meggs Wrote: Hallo Wisler,

I installed Kodi 16 OSMC on Raspberry Pi 2. There is only available adsp.basic and adsp.freesoround. Is there a simple way to get adsp.biquad.filters installed?

Thank you

Meggs

Hi Meggs,

perfect timing Wink.

Since a few days it should be available with our latest master builds. Furthermore it should be available for OS XBlush. See my pull request.

Hi Wisler,

meanwhile I've got some updates, but it's still not available. Where can I get your latest master builds?
Reply
#12
(2016-03-27, 14:34)Meggs Wrote: meanwhile I've got some updates, but it's still not available. Where can I get your latest master builds?

Ah sorry I misread your post. You're using OSMC. We do not provide builds for OSMC and as far as I know we don't have nightly builds for RPi. Only for Linux here .
Consequently it is up to the maintainer of OSMC to integrate adsp add-ons.
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply
#13
Would it be possible to output sound to more than one device (optical spdif and Headphone Jack and HDMI)? With this capabitlity it would be easier to plug a headphone on (e.g. if someone is sleeping), without reconfiguring kodi.
Backend: Asrock N3150 with Ubuntu 16.04 Server with TvHeadend and many QEMU VMs
Living Room: Asrock Q1900TM-ITX with Libreelec 9 (Milhouse)
Other Kodi Clients: 4 Windows 10, 1 Android
Reply
#14
(2016-03-29, 14:38)wisler Wrote:
(2016-03-27, 14:34)Meggs Wrote: meanwhile I've got some updates, but it's still not available. Where can I get your latest master builds?

Ah sorry I misread your post. You're using OSMC. We do not provide builds for OSMC and as far as I know we don't have nightly builds for RPi. Only for Linux here .
Consequently it is up to the maintainer of OSMC to integrate adsp add-ons.

Hi Whisler,

thank you for your answer.
In OSMC its possible to install an Addon from Zip-File. So if I had a Link to download compiled ADSP.Biquad.Filters as validated Zip-File I could install it manually. It would be great, if you could give me that link.Thank you very much

Meggs
Reply
#15
Hi Meggs,

if I had such link I would gave it to you.
Latest news about AudioDSP and my libraries are available on Twitter.

Developers can follow me on Github.
Reply



Current state and vision about AudioDSP52