[viljoviitanen]

Hey all.

Browsing through the recent threads, it seems making things easier for translators is not going anywhere. And the state of the translation issues in the wiki ain't looking good either (a translation tool done in dotnet needing wine on on linux? c'mon.)

Especially with the different addons, there is a need for a centralized database from which common strings can be automatically fetched so tons of duplicate work and using different words in different addons for same things can be avoided.

XBMC project active members, devs, users, all.

Please.

Me, I'm not up for this task but I realise it is a _lot_ of work to set up this thing. I'm just fiddling with a single addon + few translations, being a native speaker of one satanic backwards language, Finnish.

We have "core" (xbmc itself) + tons of different utilities (addons), although we're of course not that big as the Gnome and KDE projects, but we still have the same issues those guys have, and we could also benefit a lot from taking a look how they do things.

To get things started, though, here's a tool written in python in a couple of minutes that can help checking the state of the translations of a single addon:

https://github.com/viljoviitanen/Simple-...ation-Tool

The tool parses all the files (assumed to be strings.xml files) given as arguments, loops through the files and stores all ids it finds, then loops through the ids and checks if all files have all the ids and complains if an id is missing.

Ideas for the tool are welcome, ideas with code are even more welcome!

[spiff]

nice.

for starters i would love to use that tool to generate a wiki page.
another useful functionality would be the ability to register who translates a particular addon, with autogen'd mails when/if a translation changes.

[Popeye]

What about a web service listing all the official addons and their translations. The service could offer translators the option to add a new language and after translation the xml could get downloaded for a pull request to the addon authour (or automated...)...

[viljoviitanen]

What about a web service listing all the official addons and their translations. The service could offer translators the option to add a new language and after translation the xml could get downloaded for a pull request to the addon authour (or automated...)...

Would be totally awesome, but I'm guessing quite a lot of work to implement... But many many things can be automated with github with some wizardry. Github even has a tool: http://hubot.github.com/ that's useful for this kind of things.

Still there'd be the need to check for common translations - I think all the translations in the addons have a LOT of common words.

[viljoviitanen]

nice.

for starters i would love to use that tool to generate a wiki page.
another useful functionality would be the ability to register who translates a particular addon, with autogen'd mails when/if a translation changes.

Generated wiki page would list missing translations (for addons in official repo, so you can easily run it on the checked out git repo)? I can work on that, it should be easy.

As for the second thing, I think that's totally doable, and as I said in the other post, github even has a tool for github-based automation, hubot, which probably has even ready made stuff that could be used (github mentions it's doing at least translations with it!). But that's too much work for me. I'm a total newbie with git and github.

[viljoviitanen]

I've been pointed (thanks ppic) to an awesome looking translation-making addon waiting for a dev to adopt it:

http://forum.xbmc.org/showthread.php?tid=117536

Combine this with an easy way to post modifications somewhere the addon dev(s) can pick it up.

Think about this: an user installs an addon of which translation not available in the language the user is using. Display a gentle plea that you can help translating the addon by installing the translation addon and translate a few strings... we'd have complete translations of all popular addons in no time.

[viljoviitanen]

Ok, now my tool is more suited for real work.

It reports extra strings in translation and provides templates for missing strings.

First argument is "base" (English) to which the rest of the arguments are compared to.

Like this (example from fresh checked out github repo):[noparse]
viljo@dell:~/git/xbmc/addons/skin.confluence/language$ ~/git/Simple-XBMC-Translation-Tool/translationtool.py English/strings.xml Italian/strings.xml
Using base: English/strings.xml
Italian/strings.xml
<string id="31006">View Options</string>
<string id="31008">Fullscreen</string>
<string id="31050">Sort: Ascending</string>
<string id="31051">Sort: Descending</string>
<string id="31131"></string>
<string id="31132">Lyrics Add-on</string>
<string id="31133">Subtitle Add-on</string>
<string id="31134">Home Page Videos Submenu</string>
<string id="31135">Home Page Music Submenu</string>
<string id="31136">Home Page Pictures Submenu</string>
not in base:31250
not in base:31251
not in base:31252
<string id="31331">Album Details</string>
<string id="31390">Skin default</string>
<string id="31391">Skin default with no Caps</string>
<string id="31392">Arial based</string>
<string id="31900">Weather Maps</string>
<string id="31901">36 Hour Forecast</string>
<string id="31902">Hourly Forecast</string>
<string id="31903">Weekend Forecast</string>
<string id="31904">10 Day Forecast</string>
<string id="31905">Forecast</string>
<string id="31906">Maps & Video</string>
<string id="31907">Video Forecast [color=grey2](Fullscreen Playback)[/color]</string>
<string id="31908">Chance of Precipitation</string>
<string id="31909">Fetching forecast info...</string>
[/noparse]

It's suggested to run it like this to see the status of all languages:

translationtool.py English/strings.xml */strings.xml

[mad-max]

Just for you as a hint...there was an attempt to host all translations in an svn...
See here:
http://forum.xbmc.org/showthread.php?tid=93426

The project had 3 members...only translating into German...so I would say: FAIL!

But hey...maybe someone else picks up the idea again and we get this thing rolling...

@viljoviitanen:
If you didn't check the Confluence-Thread, you got a solution in there ;-)

cheers,
Max

[Roman_V_M]

As a localization professional I would like to share some thoughts. I'm not sure that all this can be implemented but still I'd want to describe a general concept.
What we need is basically an online CAT (Computer Assisted Translation) tool. It does not have to be as advanced as commercial tools like SDL TMS or Liox TWS, but it should be able to do at least the following:

1) Process strings.xml files, allowing to edit meaningful text while preserving xml tags.
2) Display source and target (translated) strings in one view.
3) Maintain a centralized translation database. Such database is called Translation Memory ™. This would allow to re-use existing translations, thus simplifying the process for new translations, and to maintain translation consistency between various modules, skins, plugins etc.
4) Include at least a basic search feature throughout a working TM.

The workflow, as I imagine it, should be the following:
1) A translator loads a source (English) strings.xml into the tool. This strings.xml can from be any module - the main GUI, a skin, a plugin etc.
2) The tool auto-populates existing translations from a TM for their language.
3) The translator adds new translated strings and/or edits existing ones.
4) He/she commits completed translation into the TM. Or translated string are committed in the process, allowing to re-use similar strings throughout the file.
5) He/she saves the target strings.xml for their language.

Also it would be useful if this (imaginary ) tool had the following features:
1) At least basic QA features like spellchecking.
2) Similarity (concordance) analysis with suggesting possible translation. E.g. if there is a new string "Press LEFT to see a trailer" and a TM already includes translation for "Press RIGHT to see a trailer", the tool would suggest this translation with marking the difference.

AFAIK there are at least a couple free opensource CAT tools and, I don't know, maybe they can be used for XBMC too.

[viljoviitanen]

As a localization professional I would like to share some thoughts. I'm not sure that all this can be implemented but still I'd want to describe a general concept.
...

I'm quite sure it could be implemented, it just requires hours and hours of work...

Unfortunately it would seem almost nobody is interested in getting the translation situation better (for evidence, see the number of people participating in this thread). Which sort of surprises me, because the core would seem quite well translated, but even the default skin (confluence) is much worse, and other skins and addons it's just random.

I think a centralized system like above combined with an in-xbmc-translation tool (there's already work done for such an addon, see my above post) would be a winning combo, provided there would be a gentle reminder when using an addon in a language it's not been translated to so we could get the masses of the users involved. I think just a tool for "professionals" will not be enough. It would at least have to be a website, and it would also be REALLY simple to use, and not require going thru many hoops before you can start contributing (e.g. registration like this forum).

Thinking about this, maybe google translation toolkit (http://support.google.com/translate/toolkit/ , http://translate.google.com/toolkit/TOS.html) could be used. Google has some experience on making things easy to use... Of course it would need some serious intregration to the xmbc strings.xml files.

But anyway I don't think any of this will happen unless some core xbmc dev starts to at least coordinating this. And they seem to have their hands full or work already, or rather, they are more interested in other parts of xbmc.

[alanwww1]

Guys !

I am taking over the handling of translations for xbmc. Tickets, pull requests, github stuff basically, but I am planning to write a completely new QT based multiplatform translator tool which can handle also context explanations for each strings.

With first step I start to make context comments for most of the strings in the English xml file (It will be a PITA :-( ). These comments will be handled of course with the new QT based translator tool.

At the end and if I have some help I plan to build a GIT client into the Utility so that it can handle things directly using GIT.

I'll keep you updated with further plans builds etc.

Cheers, Attila

[viljoviitanen]

Guys !

I am taking over the handling of translations for xbmc. Tickets, pull requests, github stuff basically, but I am planning to write a completely new QT based multiplatform translator tool which can handle also context explanations for each strings.

With first step I start to make context comments for most of the strings in the English xml file (It will be a PITA :-( ). These comments will be handled of course with the new QT based translator tool.

At the end and if I have some help I plan to build a GIT client into the Utility so that it can handle things directly using GIT.

I'll keep you updated with further plans builds etc.

Cheers, Attila

Excellent news, thanks!

For sure the tool will handle addon translations as well as core xbmc?

But are you sure it's worth it to create a new tool, as opposed to using existing tools? Google translation tool looks pretty powerful to me... And if existing tools are for sure ruled out, if it was up to me, I'd probably try to make a web translator thing, not a qt one - modern javascript ui libraries makes creating even complex ui's pretty easy (I'm thinking http://jqueryui.com/). I'd prefer the web thing because it'll lower the barrier for contributing, and it would probably make git integration easier too, as a translator would not need to set up/understand git at all, it would be handled server side. As a bonus it would be possible to integrate the currently orphan xbmc translation addon to the central system quite easily - you could even allow completely anonymous "suggestions", and then have a logged in "moderator" accept them. (I still think it would be best if creating/correcting translations would require as little effort from the potential translator as possible...)

And if you were to do it the web way, you'd have at least one developer willing to help :-) I do python, php and even perl (I'm so old I still think perl is cool, like it was 10 years ago :-)). Java, C++/C and ruby I don't do. Javascript/node.js... I'm not sure.

OR you could have the qt client contact (optionally?) some kind of cental web system so fetching and updating translations would be super easy. Hmm... I think this is necessary, because forcing a translator to use git is not cool. Git is powerful, github even more so, but grokking all that is not easy, even if you have some version control system experience. (I started using cvs in 1995, and lately subversion, but still...)

Anyway the context idea is very good, and obviously necessary to create quality translations easily. Otherwise you'll need to be very familiar with the whole thing that's being translated. Maybe if you do some "context" strings at first and commit them in git, then other people can help when they have seen what it's about, so it'll be a smaller PITA to deal with.

[alanwww1]

Hi viljoviitanen !

It is really good to know that I can expect a help on this. The ideas and points are really good what you wrote. The QT way was just because I know C++ but absolutely no experience with web programming languages. I will reconsider the best way to go for. I'll leave this discussion open so that we can really find the best possible solution.

I am really count on the help you offered and really appreciate it.

About the context commenting. I am sure I'll need all the help I can get. From team members and as well as users, translators.

Thanks, again, we'll keep in touch !

[viljoviitanen]

Ok, I think I have a winning idea here (Charlie Sheen type winning, dunno )

I'm still liking the javascript, 'cos with modern html5 thingies you can probably work also with local files. So we can (at first) work with a website, but later also add local support. (I'm not an html5 expert, but you can do wonders with modern browsers.) For the "local" use, we could probably just target chrome/chromium, it's popular and cross platform enough.

Then, the app. My great idea is to generate a really really big "translation memory" from freely available gnome, kde, firefox.. translations in addition to existing xbmc translations. Store the "where" part in the database, as to provide reference. Set up the search so that xbmc specific strings are displayed first.

Use json as internal format, only convert to xbmc xml when "exporting" the translated string.

And since we're talking about xbmc, changes in the core translations could even probably make an automated pull request, http://develop.github.com/p/pulls.html .

Make all this configurable so that other projects than xbmc can use the tool as well, and join developing it.

Winning!

Or maybe there is such a tool already, but at least the gnome guys, who have "some" experience in translating, do not seem to have such a tool: http://live.gnome.org/TranslationProject...ranslating

---

Also, alanwww1, I suggest to look if the "context" part in xbmc could be an attribute, like so:

Code:

<string id="0" contexthint="main menu">Programs</string>

then parsing the xml to the tool can be done more easily than if it's a comment. I just hope xbmc accepts such xml.

Alternative would be e.g.

Code:

<contexthint id="0">main menu</contexthint>

or if even that's not possible, then maybe just store the context hints in a separate file, using the same format. Anyway xml comments probably is not a good idea because of the parsing problem.

[alanwww1]

Thanks. The idea is great. I found extjs pretty impressive. TVHeadend uses it and it is great. http://www.sencha.com/products/extjs/examples/

What do you think ?

Except I would keep the xml format for the internal format.

To have the context hints as attributes was also in my mind. I will talk to the team members about this.

Thanks