• 1
  • 2(current)
  • 3
  • 4
  • 5
  • 12
WIP Adding Audio DSP Plugin to AddonManager
#16
Hello thecYrus,

have looked about VST now (was new for me) and make no sense to use it.

There are a lot of things which does not match with XBMC:
- Found this interface type not under XBMC's GPL
- Found only official support for Windows and MacOS
- Every plugin I have seen has a own GUI dialogue and does not match the XBMC skin system

I know from the past the Linux LADSPA Interface and on start have thought to use this, but I come to my type. I want to have a easy from XBMC skin interface controllable processing system, which must only process from the played source to the output signal. To have a complex sound processing system like on a Studio needed is, make no sense for me. It must be easy to use.

Start of it was to have a XBMC Home Entertainment Centre for me which support also things like on a buy able AV-Receiver for user selectable sound modes.

Maybe someone can write a add-on which then handle the VST system, but have not found a free download on http://www.steinberg.net/en/company/developer.html to compare the interface to see that it is supported.
Reply
#17
I'm not very familiar with the VST plugins, but after a quick search I found this tutorial :
VST Tutorial
It describes how to create a VST plugin with Visual Studio. So it looks like you only need these two methods:
Code:
void processReplacing (float ** inputs , float ** outputs , VstInt32 sample frames) ;
Audio Effect * createEffectInstance ( audiomaster callback audiomaster );

I think it could be done with the changes from Alwinus and it should be possible to integrate a VST host in the form of an ADSP-Addon. But I don't know if it possible to integrate the GUI from a VST into xbmc.

The advantage would be that you can rely on already existing effect plugins.
The disadvantage is that you are not allowed to publish the binary files under the GPL (http://wiki.ubuntuusers.de/VST#Warum-geh...-einfacher sorry only found this in German ) and it is actually no official support under other operating systems. So the user must compile the VST plugins by himself.
That's why I also think that xbmc should have it's own addons.

So we need a simple API and easy to configure dialogs. So users whithout knowledge in digital signal processing can use the audio addons. (Best example is an AV-Receiver )
Reply
#18
thumbs up 1000 times
Reply
#19
Let's be straight and practical. Let's check if this can be compliant with a straightforward (should I say canonic and orthogonal) diyAudio usage.

Stage 1 in the audio DSP pipeline :
I'm watching a 5.1 movie.
I'm using a 8-channel USB audio card like SWEEX SC016.
Actually I'm not interested in multichannel audio.
I prefear listening to a 2.0 downmix with the left and the right speakers both conveying the sub channel.
All what matters, is to select, load and execute a VST plugin doing the stereo downmix.
Such VST plugin can easily be designed and compiled using Flowstone - but how to deal with the VST GUI defining the sub amplitude and the sub phase ? Relying on dialog boxes ? Or naming a text file ?

Stage 2 in the audio DSP pipeline :
My listening room has damping and resonances that I would like to compensate in stereo.
- eight IIR BiQuads filters per channel (16 IIR BiQuad filters in total)
- one 1024-tap FIR filter per channel (2 FIR filters in total)
- one delay (multiples of the audio sampling period) per channel (2 delays in total)
Again, this can be a VST designed and compiled by Flowstone, and this time the VST GUI should be very simple like naming the text file containing all IIR BiQuad coefficients, all FIR coefficients, and all delay values.

Stage 3 in the audio DSP pipeline :
I would like to add multiband dynamics processing, say Hans van Zutphen "Stereo Tool" VST plugin for Windows X86 32 bit hosts
All what matters, is to select, load and execute Hans van Zutphen "Stereo Tool".
Two channels in, two channels out.
And what about the GUI ?

Stage 4 in the audio DSP pipeline :
After correcting the acoustics of my listening room I would like to synthesize tailor-made room acoustics.
All what matters, is to select, load and execute .... (write here your preferred room acoustics synthesis VST)
Two channels in, two channels out.
And what about the GUI ?

Stage 5 in the audio DSP pipeline :
What to do with a 8-channel USB audio attachement like SWEEX SC016, while only processing 2 audio channels (Left and Right) ?
Answer : the aim is to implement a stereo 4-way crossover.
Thus, "left" becomes 4 channels and "right" becomes 4 channels also.
There are many different crossovers topologies.
Let's go for a straightforward one.
- eight IIR BiQuad filters per output channel (64 IIR BiQuad filters in total)
- one 1024-tap FIR filter per output channel (8 FIR filters in total)
- one delay (multiples of the audio sampling period) per output channel (8 delays in total)
The VST GUI should be very simple like naming the text file containing all IIR BiQuad coefficients, all FIR coefficients, and all delay values.
There can be more elaborated crossovers, to be selected as alternate crossovers, requiring other kind of text files for the parameters.

Stage 6 in the audio DSP pipeline :
The eight channels must be routed to the SWEEX SC016.
or Terratec Aureon 7.1 USB
or Creative Sound Blaster X-Fi Surround 5.1 Pro USB Sound Card.
or ASUS Xonar U7.
Using XBMC on a Windows PC
or using XBMC on a ARM-Android TV Box.

What's described above should also work when listening to music (WMA, MP3, FLAC, OGG, APE).

Are we heading towards this, really, and multiplatform ?

Plese read Tom Gersic about VST User Interfaces :

1. All user-interface issues are entirely separated from the audio processing issues. At its simplest there is an option where you can avoid providing a user interface at all. In this case the host requests character strings from the Plug-In representing each of the parameters. The host can use the separate ASCII strings for the value, the label, and the units of the parameters to construct its own user interface. This is how the simple code-examples, AGain & ADelay, work. This is also often a good way to develop a VST Plug-In, it offers a very short development cycle to start to test the algorithm. The proper interface can come later.

2. The next user interface level is provided when the Plug-In defines its own editor. This allows practically any user interface to be defined. A negative aspect is that then you can quickly land up in platform specifics when dealing with the nuts an bolts of the interface issues, even though the audio process, the concepts and methodology remain platform independent.

3. The final option is to use a portable framework for creating sophisticated user interfaces. This framework takes the form of the VSTGUI Library files that are available for all supported VST platforms. Apart from converting a few pictures from one resource format to another, for which there are tools available, and setting up the initial project files, writing a VST 2.3 Plug-In can really be completely cross-platform (some given examples: ControlsGUI, ADelayEditGUI, SurroundDelay). VSTGUI Library files and their usage is completely described in the HMTL files that accompany this SDK. After reading about the underlying issues in this document and associated code examples, the VSTGUI comes very highly recommended.

1-2-3 come from Tom Gersic - http://www.gersic.com/vstsdk/

Hope this "wall of text" helps.
Reply
#20
One small omission.

Stage 6 in the audio DSP pipeline :
Modern hardware now supports the HDMI 8-channel LPCM modality.
It is thus desirable to output the 8 audio channels on the HDMI connector using the HDMI 8-channel LPCM modality.
This, when watching a movie, and also when listening to music (WMA, MP3, FLAC, OGG, APE).
Reply
#21
What's about the audio sampling rate in audiophile (high-end audio) usage ?

Can we ask ffmpeg to never resample the audio, for delivering "bit exact" audio (44.1 kHz, 48 kHz, 96 kHz or 192 kHz) to Stage 1 in the audio DSP pipeline ?
Say that the whole audio DSP pipeline is made of audiophile VSTs custom-designed supporting the 44.1 kHz, 48 kHz, 96 kHz, 192 kHz audio sampling rates.
Can ffmpeg tell the original audio resolution (16-bit, 24-bit, 32-bit) to Stage 1 in the audio DSP pipeline, for Stage 1 adding the optimal dithering for achieving a 32-bit data width, without denormal issues ?
Can ffmpeg tell all VSTs the original audio sampling rate for all VSTs automatically selecting the corresonding IIR BiQuads coefficients, FIR coefficients and delays ?
Can the XBMC DSP pipeline manager measure the actual CPU usage and flag any overload for asking Stage 1 to downsample to 96 kHz when the current audio is 192 kHz, or to downsample to 48 kHz when the current audio is 96 kHz ?
Reply
#22
Hello steph_tsf,

have not so much time now to make a big answer, but one question, have you looked before to following links?

https://github.com/xbmc/xbmc/pull/4402
https://github.com/AlwinEsch/xbmc/blob/a...sp_types.h
https://github.com/AlwinEsch/xbmc/blob/a...adsp_dll.h

About the ffmpeg re sampling, if needed this becomes performed only after all dsp processes are finished. If the a higher/other sample rate is requested a re sample add-on must be present to perform it (see InputResampleProcess on xbmc_adsp_dll.h).

The DSP system becomes the nearly unmodified audio stream, only the data format becomes converted to float format if needed. Currently is from XBMC the only dsp stream process change after a called MasterProcess to fix a channel alignment if input channels are still higher (no DSP down mix) as requested output signal.

With CPU usage see the pictures on the pull request, there is currently no sample rate change after a to high cpu usage.

Best regards,

Alwin
Reply
#23
(2014-04-20, 00:33)alwinus Wrote: have not so much time now to make a big answer, but one question, have you looked before to following links? https://github.com/xbmc/xbmc/pull/4402
Yes I did. There is a problem. You don't care about the sampling frequency when entering the audio DSP pipeline. IMO, a more practical approach is to explicitely manage the sampling frequency before entering the audio DSP pipeline. Compare your proposed pipeline, with my proposed pipeline. What you propose is only good if you ask ffmpeg to do a sample rate conversion, like converting the 48 kHz audio coming from a movie, to the 44.1 kHz that Windows is using. We don't want this. We want ffmpeg to remain "bit exact" and deliver "bit exact" audio to the audio DSP pipeline. Or, if we are lazy and don't want to design IIR Biquads, FIR filters and delays for 44.1 kHz, 48 kHz, 96 kHz and 192 kHz, we must ask ffmpeg to do a sample rate conversion, possibly a high quality sample rate conversion enabling to tune the CPU load / data precision tradeoff. If ffmpeg cannot do the required sample rate conversion, the first stage (Stage 1) of the audio DSP pipeline must do it.

In first place, consider XBMC as a sophisticated GUI for ffmpeg :
- asking ffmpeg to do things
- grabbing data streams coming out from ffmpeg.

You can see the ffmpeg audio options here : https://ffmpeg.org/ffmpeg.html#Audio-Options.

That's not all.

XBMC also takes care of kindly interfacing the driver of the audio hardware attachment (Realtek HD audio, USB audio, HDMI), actually this is more complicated because you always have an Audio Server deployed on the platform. Basically this is Windows Direct Sound (DS), mixing different audio sources having different sample rates, sending the resulting 44.1 kHz audio to the hardware using the driver that's attached to the hardware. Oops, this means that for movies there is a 48 kHz -> 44.1 kHz resampling occurring within Windows Direct Sound. This is thus not "bit exact". We are kaput. What to do ?

There are more sophisticated Audio Servers like JACK, able to take a Windows Sound source (like a stereo audio stream coming from ffmpeg) and route it to ASIO. Starting from there, if you have a VST host installed, you can stack a few VST for implementing any kind of audio DSP pipeline. The last VST of the pipeline will talk to the ASIO driver of your hardware, for outputting the sound. Currently there are ASIO drivers for Realtek HD Audio on the motherboard (see ASIO4ALL), there are ASIO drivers for multichannel USB audio attachments, but I fear there is no ASIO driver for the HDMI 8-channel LPCM modality. Will you check that ?

Let's consider that there will be a software utility called ASIO4HDMI. Starting from that moment, everything will simplify after the last VST. Let me explain. If you want to output multichannel audio on the Realtek HDaudio motherboard, you will rely on the ASIO4ALL driver. If you want to output multichannel audio on high quality multichannel USB audio attachments, you will rely on the ASIO driver that's supplied with it. And, if you want to output multichannel audio on HDMI 8-channel LPCM, you will rely on the ASIO4HDMI driver.

Thus, it is better to avoid XBMC taking care of the audio that's coming out of the VST pipeline.
Thus, it is better to leave XBMC unaware of the sound that's coming out of the VST pipeline.
Well, it is a little bit more complicated than this, as when you chose the HDMI 8-channel LPCM modality, you need to deliver the 8-channel LPCM audio to XBMC, for XBMC embedding the video on HDMI.

So, in a nutshell, the ultimate, most flexible and most easy to understand audio DSP manager for XBMC would consist on :
- a selection box enabling to chose between "disable advanced audio" and "enable advanced audio"
- when "advanced audio" gets enabled, there should be a selection box about the audio peripheral hardware (typically : Realtek HDaudio motherboard, USB audio attachment, HDMI 8-channel LPCM)

Actually, even in the "advanced audio" modality, XBMC can remain in charge of almost nothing what's regarding audio
XBMC will talk to ffmpeg for :
- asking ffmpeg to do things
- grabbing the audio stream coming out from ffmpeg
XBMC would NOT route the audio CD, DVD or Bluray sound to the Windows sound mixer.

Instead, XBMC would route the audio CD, DVD or Bluray sound to the ASIO subsystem. Actually there are two sound domains :
- the XBMC sound domain (audio CD, DVD, Bluray, MP3, high quality streamed audio)
- the Windows sound domain (YouTube, Security camera, spotify, etc ...)
How to deal with those two audio domains, potentially exhibiting different sample rates ?

The "bit-perfect" strategy would avoid any sample rate conversion what's regarding the XBMC audio.
This means that the ASIO sample rate must change on-the-fly, like when you go from listening a CD (44.1 kHz) to watching a movie (48 kHz).
Don't know if this is feasible. Will you check ?
Say it is feasible.
Comes the need of mixing the XBMC sound, with the sound coming from the Windows mixer ("beeps" from Windows, sound from a security camera, sound from YouTube, ...).
You thus need to resample the Windows sound, for matching the XBMC sound sample rate.
There can be a button telling "disable Windows sound" in which case only the XBMC sound domain gets processed, hence no mixing, no resampling.
For each VST in the pipeline, you need to prepare four sets of filters coefficients and delays (44.1 kHz, 48 kHz, 96 kHz, 192 kHz). You need to select them on the fly.

The "ffmpeg 44.1 kHz resample" strategy would rely on ffmpeg for delivering an audio sample rate, that's matching the nominal Windows sample rate which is 44.1 kHz.
This means that the ASIO sample rate is always equal to the Windows sample rate.
Mixing other sounds is easy (a simple addition) like the "beeps" from Windows, or like allowing your security camera to announce a visitor while watching a movie.
There can be a button telling "disable Windows sound" in which case only the XBMC sound gets played, hence no mixing.
All VSTs in the pipeline will select the 44.1 kHz coefficients and delays.

The "ffmpeg 96 kHz resample" strategy would rely on ffmpeg for delivering an audio sample rate, that's always equal to 96 kHz.
Comes the need of mixing the XBMC sound, with the 44.1 kHz sound coming from the Windows mixer ("beeps" from Windows, sound from a security camera, sound from YouTube, ...).
You thus need to resample the Windows sound, for matching the 96 kHz XBMC sound sample rate.
There can be a button telling "disable Windows sound" in which case only the XBMC sound domain gets plrocessed.
Reply
#24
Just want to say something about the the vst gui. it's not required to display the advanced gui. every plugin is built upon parameters with defined units. All vst hosts can just display this parameter list if the user prefers this over the full blown gui. And this parameter list would be the way to go for rendering it in xbmc imho.
Reply
#25
@steph_tsf: xbmc is multiplattform and does not only run on windows! As far as I know there is no support for ASIO and JACK within xbmc.

Why are you talking about VSTs? At the moment there is no VST support from our addon system!!!!!!!!!

If you want VST addons you have to write a wrapper addon for xbmc!!!!

Why should your "wall of text" help us? The audio dsp addon handling is almost finished and it works. Just compile alwinus fork and test it.
Reply
#26
(2014-04-20, 10:03)wisler Wrote: Why are you talking about VSTs? At the moment there is no VST support from our addon system!!!!!!!!!

why create another audio plugin standard when there's already an established industry around one?
Reply
#27
Code:
why create another audio plugin standard when there's already an established industry around one?

Because VSTs cannot be used on any plattfrom which xbmc runs! Have a look at VSTs under linux. There is no official host and not every one uses windows to run xbmc.
Reply
#28
(2014-04-20, 10:03)wisler Wrote: @steph_tsf: xbmc is multiplattform and does not only run on windows! As far as I know there is no support for ASIO and JACK within xbmc. Why are you talking about VSTs? At the moment there is no VST support from our addon system!!!!!!!!! If you want VST addons you have to write a wrapper addon for xbmc!!!! Why should your "wall of text" help us? The audio dsp addon handling is almost finished and it works. Just compile alwinus fork and test it.
Hello wisler. Thanks for summarizing it this way, clearly, frankly. I commented alwinus work with "one million thumbs up". Alwinus asked if his proposed DSP pipeline was right. Unfortunately alwinus proposed DSP pipeline is wrong from a diyAudio goal targeting bit-exact audio. Unfortunately alwinus proposed DSP pipeline ignores ffmpeg downmix and resampling possibilities. Thus, generating my "wall of text" here, I was expecting that XBMC revisited AddonManager could align on some useful VST practices. We live in a free world. Do what you want and continue your effort. I'm warmly and positively hoping that you have the required skills for triggering a revolution, with XBMC embedding all bolts and nuts for designing any audio DSP pipeline running on Windows X86, ARM Android and (why not ?) Apple iOS, able to output audio on multichannel HDaudio motherboards, multichannel USB audio attachements and HDMI 8-channel LPCM. All this relying on the XBMC GUI that should be multiplatform.

Now come a few points I would like you to consider, seriously :

1. Have you understood how a given VST plugin can grab operational parameters (like BiQuad coefficients, FIR filters coefficients, delay values) from text files ?
2. Have you understood how the Audio DSP Plugin addition to XBMC AddonManager could replicate such approach for specifying all audio DSP Plugins to be pipelined, and also, for specifying the text file to be associated (as parameters list) to each Audio DSP Plugin ?
3. What software framework do you recommend for writing the Audio DSP code of each Audio DSP plugin ? - please remember the aim of 1) remaining multi platform and 2) avoiding the VST wrapper method
4. Have you understood that for remaining "bit-perfect", you need to ask ffmpeg to do no audio resample ?
5. Have you understood that for remaining "bit-perfect", all audio DSP Plugins need to support various sample rates (44.1 kHz, 48 kHz, 96 kHz, 192 kHz) - automatically selecting the right parameters sets ?
6. Have you understood that for remaining compatible with low cost hardware only capable of 44.1 kHz, you may need to ask ffmpeg to do audio resample, or you may need an Audio DSP Plugin doing that the way you want ?
7. Have you understood that there are two Audio domains, the XBMC Audio domain (possibly handled "bit perfect"), and the "Platform" Audio domain carrying sounds from Internet, from security cameras, from VOIP, etc ?
8. Have you understood that some mixing may be required, for allowing the user to listen both to the XBMC Audio, and the "Platform" Audio ? Have you understood that this may require further audio resample ?

Your goal is ambitious. As I said, if some of this is already working, that's absolutely remarkable.

Unfortunately, very few people is willing to invest time in compiling and testing, very few people is willing to spend money in hiring specialized people for compiling and testing, if you don't confirm that you bear the above 8 points in mind.
Reply
#29
no more "have you understood", you're coming off pretty badly here.
Consider this a warning. Have you understood?
Reply
#30
(2014-04-20, 20:49)wsnipex Wrote: no more "have you understood", you're coming off pretty badly here. Consider this a warning. Have you understood?
Pretty well, thanks. I clearly understand how gigahertz clock multicore CPUs needing to execute undescriptible software architectures relying on thousands of code lines, supposed to run on many different OS and hardware, render things complex. Clearly, the risk of working on something that will have little practical use becomes high, when software developers don't have the time (or the care) to interact with people coming from different horizons, like the ones accustomed to the ASIO/VST paradigms. This becomes worrying when there is a recurrent pattern consisting on adopting regressive behaviors, like complaining from agression when comes the time to confront, or when comes the time to acknowledge. Thus, in a particular context when the time allocated to software development/testing is pure generosity, one would expect something different, than what's happening now. An important question arises. Are they proud of doing complicated things on a complicated platform, or are they proud of giving more usability to an existing product, for reaching a new, fresh base of power users ? The frontier of obsolescence lies here. The beginning of the NIH syndrome also lies here. Nobody's perfect. However, everybody can try improving.
Reply
  • 1
  • 2(current)
  • 3
  • 4
  • 5
  • 12

Logout Mark Read Team Forum Stats Members Help
Adding Audio DSP Plugin to AddonManager7