Question: How do you make sure the translation is reliable and do you ever take users' advice?

Sheila Castilho answered on 3 Nov 2018:

Great question so long answer! Brace yourself!
We have different levels of quality for translations. Some people need a quick translation to understand the main meaning of a newspaper headline for example or the content of an email, so for those ‘good enough’ translation is different from people that need it to read a legal letter they got from the lawyer, for example.
We use both automatic metrics and human evaluation to see if the machine translation system is providing that type of quality level people are expecting. The automatic metric will look at the system’s translation and compare it to a human translation and will give the system’s translation a score of how similar it is to the human translation.
Human evaluation is basically translators looking at some examples of the system’s translation and giving a score of how good they are.
And yes, I do take the users’ advice into account. So if the users say to me “I need this translation just to understand the topic of this text” then I know that they don’t need a very high quality level translation. Sometimes I provide the translation to the users and ask them to perform the task they need with the translation. If they manage, it means that the translation was ‘good enough’
=)