[marcselman]

For anyone else who experiences the same issues, I have found the problem and have a solution.

The issue is actualy described in the Kodi wiki:
https://kodi.wiki/view/PulseAudio
Under "6.5: Known issues" you can read the the following:

When Pulseaudio is configured to be multi-channel output, e.g. 5.1 profile from within pavucontrol, 2.0 streams are automatically upmixed, no matter what setting you have chosen in Kodi. This is default pulseaudio server behaviour. You can workaround that by disabling enable-remixing in /etc/pulse/daemon.conf
enable-remixing = no

After I changed that I noticed that stereo videos were indead playing only over the front speakers. But the problem then is that the output is still multi-channel surround but with only sound for the two front channels.
This issue is also nicely described in the Kodi wiki:
User "2: PulseAudio Output Configuration" you can read the following note:

Note: Audio decoded to PCM and sent through PulseAudio the audio will not likely be "bit perfect". As Pulseaudio will always open the number of channels you have configured. You can stop it doing upmixing (see Known issues), but it will still open all configured channels which makes Dolby Pro Logic II of your AVR not usable 

On the same wiki page under "6: FAQ" you can read about the pros and cons of PulseAudio and ALSA. After reading the whole wiki page I decided that ALSA was the best choice for me.

So the final solution was disabling PulseAudio on my Linux machine so Kodi will use the underlying 'Advanced Linux Sound Architecture'. This has some downsides, but for me, since this is a Kodi only machine, it's no problem.
Here you can rad how to disable PulseAudio (and revert it if needed):
https://kodi.wiki/view/PulseAudio/HOW-TO...for_Ubuntu

Props to the Kodi team for having all the information in the wiki!