Question: How many translators do ye have employed to help you. Do they give a. Straight answer or does it take longer based on how long the text is

Sheila Castilho answered on 5 Nov 2018:

That’s a great question! The university doesn’t employ any translators so we need to hire them from translation companies.

Generally for my experiments I need at least 2 translators because when I ask them to evaluate how good a translated text is I need to make sure that they agree on it. So I have to calculate what we call inter-annotator agreement (IAA), and for that I need at least 2 people. If the IAA is low, it means the translators do not agree if the translation is good or not. In that case I have to go back and try to solve the sentences they disagree on and then send them more sentences. If the IAA is high it means that I can be more confident on the final results of the evaluation – if they are good or bad, the translators agree.

We also do one evaluation where translators need to fix the machine translation output and then we see how much modifications they need to make. The more modification means the machine translated text was poor, the fewer modification means the machine translated text was good! We can use this corrections and re-train the system.

So depending on the type and size of text it can take a few weeks even months!