[wrxtasy]

Set PCM Settings to Stereo 2.0
Enable AC3 capable receiver
Enable AC3 Transcoding

Try this combination. Your TV should get Dolby for multi channel and should pass that.

Yes transcoding is really is the only option for Android Kodi legacy 5.1 audio Optical support.

As another note the Android TV (OS) has the worst Dolby Audio OS support vs FireTV and Apple TV’s.
Even devices like the Mi Box that supposedly have a Dolby Audio licence cannot for example downmix 5.1 DD+ > 5.1 DD and then send output over Optical.

Use Kodi instead. 

hi wrxtasy, won't we get 5.1 optical or 7.1 hdmi passthrough with kodi in android nougat amlogic tv boxes. will that be applicable to kodi krypton only or even with latest kodi versions too. regards.

Yes generally AML devices running Nougat 7.x Firmware have IEC61937 support for such optical / HDMI audio passthru for use with Kodi Krypton, Leia and Matrix.

You also have to manually select in Android Settings > Audio > Digital Sounds > HDMI or SPDIF

AML devices running Oreo 8.1 or Pie 9.x Firmware seem to have lost IEC61937 support.