[ArcticBeast]

Imagine an Kodi add-on that will seamlessly generate Subtitles based on Audio track from Movie, TV Show, documentary etc. and display it on screen.

On top of that, it could use Google translate to directly show translation as Subtitle.

Audio --> Generated Subtitle --> Translation to ALL Google Translate languages.

WHY the heck we would need such add-on?!!!

• Newest Movies don't get Subtitles that fast. INSANE amount of Documentaries NEVER get Subtitles so such add-on would be very useful for Documentaries.
  
• Newest Movies, TV Shows, Documentaries ALWAYS 1st get Subtitles on English, with such add-on you could load any Subtitle language (English recommended as it's only available at the beginning) and instantly translate it in real-time to any language during watching.
  

Automatic recognition and translating might not be very nice experience but it could be up to 60% accurate while instant translation from already created English.srt Subtitle track can be up to 99% accurate. English to any language would be best combination. Same as English Audio to English Subtitle to any language combination. For example, Russian Audio to English Subtitle could not be that good but still...

Kodi is used worldwide for long time already, it has thousands of add-on's and I am really surprised how such add-on is still not made.

As far as I know there is already Google API for Audio recognition and auto translate to any language.

It can't be perfect add-on but at least something. Right?

[El_Guero]

Nice idea ... let’s do it.

Wayne

[Zombi]

Good idea

[Klojum]

We're looking forward to your PR(s).

[ymca]

I've always searched for that kind of thing.
I have a google addon for my movies on kodi.
But some movies don't have subtitles. 
I remember that google has something like live captions.
I hope someone can integrate the translation of google live translation to that.

[User 272336]

@ymca
FYI: You are responding to a nearly 2-year-old thread.

[ymca]

Yes. Was looking for a solution. This is the only thing I found.

[JediFonger]

sorry to bump this, but i'm just following the rules of not creating duplicate posts, this is pretty much exactly what i was going to write in to suggest... and we're now almost 4yrs removed from the original post started! now we're living in the wide wide world of chatgpt & many other machine learning algorithm (i hate using the blanket term AI). i searched for when live caption was added to youtube and it was 2021: https://support.google.com/youtube/threa...tube?hl=en

why is this (still) important? i enjoy many international imported DVDs and not everything has English subs, what i've often done is leveraged youtube's capability to live caption by uploading a video of the rip into the youtube (not published publically) but just to take advantage of their live caption and live translation concurrently. 

EVEN THEN, not every single video uploaded can kick this feature open... but chrome browser itself... now has LIVE CAPTION built into the browser, so i can play the youtube video using the chrome browser... and then enable the live caption feature... that you can also LIVE translate... you just need to go the extra step of downloading an offline language pack. let's say you've got an italian film... you need to download that italian language pack intsalled locally on the chrome browser and voila!

anyway... having this as a kodi plugin would save me all these extra manual steps. i dont think this is as niche as it was even 2 or 4yrs ago now.

the only issue that i think may kill these addon subittle project is the cost of APIs, https://thesquareplanet.com/blog/ai-capt...ly%20limit).
^google's live caption API calls are extremely limited for free accounts.

caption ninja maybe?

barring a public cloud solution, i'd say maybe https://www.assemblyai.com/blog/the-top-...e-engines/
^just wholesale integrate a free/open source engine that'll perform the live caption live translation straight up.

this would really make kodi the ONLY GAME IN TOWN when it comes to live captioning media where you can enjoy content that you dont need to hunt down pre-scripted subtitles for.

[MatteN]

Every subtitle i checked that is either Google or AI (translated to swedish) sucks big time, its like its made by a 3 yo.

[JediFonger]

Every subtitle i checked that is either Google or AI (translated to swedish) sucks big time, its like its made by a 3 yo.

it'll slowly improve over time with larger sample sizes, assuming we dont start throwing up paywalls.

also, it's better than nothing.

[MatteN]

also, it's better than nothing.

No, its not.
Every decent subtitlesite removes machinetranslated subs for a reason.

[JediFonger]

also, it's better than nothing.

No, its not.
Every decent subtitlesite removes machinetranslated subs for a reason.

agree to disagree.

i'm currently watching Basileus Quartet (Quartetto Basileus), a 1982 film by Fabio Carpi. there's a poorass VHS transfer and no ENG subs anywhere that i can access.

would i LOVE to have a properly translated sub, ABSOLUTELY? is it going to stop me from watching it? no. i'd rather have seen it with anything, even if shitty to get the gist of the movie vs. not having to watch it.

[kereltje]

You might be interested in https://github.com/dimkroon/translate_subs - some experiments I did to create an add-on to translate subtitles.

Initially intended to automatically translate subs (usually for the hearing impaired) provided by video add-ons, but with the latest test build it should be possible to manually translate existing subtitles files by using the normal subtitles dialog in Kodi. Downloadable as zip file from https://dimkroon.net/kodi-addons/service...htly-1.zip.

Development is slow though, usually in short bursts, but constructive feedback is always welcome.

In my experience, the translation is generally pretty decent, but It rather depends on the content and context. Documentaries are near perfect, but things like gangster slang can be quite poor. But then again, the 3 year old's I know do not perform very well on gangster slang either.

[JediFonger]

that is a pretty nifty project… but i think that requires pre-existing srt or sub file right?

this thread is more about live caption where no files exist but machine learning has to create captions from audio only.

You might be interested in https://github.com/dimkroon/translate_subs - some experiments I did to create an add-on to translate subtitles.

Initially intended to automatically translate subs (usually for the hearing impaired) provided by video add-ons, but with the latest test build it should be possible to manually translate existing subtitles files by using the normal subtitles dialog in Kodi. Downloadable as zip file from https://dimkroon.net/kodi-addons/service...htly-1.zip.

Development is slow though, usually in short bursts, but constructive feedback is always welcome.

In my experience, the translation is generally pretty decent, but It rather depends on the content and context. Documentaries are near perfect, but things like gangster slang can be quite poor. But then again, the 3 year old's I know do not perform very well on gangster slang either.

[kereltje]

that is a pretty nifty project… but i think that requires pre-existing srt or sub file right?

this thread is more about live caption where no files exist but machine learning has to create captions from audio only.

I'm aware of the OP's question, but recent posts discussed (the quality of) translation, so I gathered this could be of interest.
But you're absolutely right, it does translation, not transcription, and depends on existing subtitles being available.