Best Practices for Test and Assessment Translation

The fact is that not everyone who takes a test or assessment speaks English. And not everyone who speaks English speaks it well enough to answer questions accurately. Making people take a test in a language they don’t understand well actually trashes the results.

Accuracy demands that assessments be given in the language of understanding. This is especially important for any organization working internationally. To get the best from a respondent, you owe it to him to communicate in his best language. This means translation. And assessment translation is hard.

In tests and assessments, questions are kept short, and so are short on context. Without context word choice gets tricky, as any test preparer knows. But assessment translation is rarely word for word. Nuance in expression and cultural context call meanings into question. In some cases, test respondents may not understand poorly-translated questions or may be unable to give accurate answers.

These problems undermine a test’s effectiveness. The effort you put into creating a useful assessment in the original language may go to waste if you can’t get reliable responses from different language communities.

Even worse, some questions simply don’t translate. They have to be adapted, changed to fit a given cultural context or modified because of the peculiarities of a particular grammar.

And with each change in each language, it becomes more and more difficult to achieve ideal testing outcomes among different groups of language speakers.

Harmonization becomes the ideal, the level playing field where all test takers or respondents get the same chance at an accurate response, regardless of language.

One answer is team assessment translation. This ensures good quality, easily understood assessments in the target languages. Let’s take a look at an ideal team translation process below, but clients should take their own customized approach, selecting the tools that work best for them.

Team Translation

First things first: What is team translation? Simply put, team assessment translation refers to a group of people with specific skills and roles who are brought together to complete a defined process. We’ll show you what we mean in a moment.

You need to start off with the right translators though. Translators working with assessments and tests need to be able to recognize the design features and various components in order to handle them appropriately. Assessment questions have special vocabulary and syntax that is sometimes at odds with normal written language; instruments have sections addressed to different audiences (interviewer, respondent, programmer, etc.) and questions and answer scales reflect measurement goals that are opaque to an untrained reader.

Draft

Two heads are better than one. To begin, we arrange for a draft translation of the survey using a team of two or more translators. This way, we benefit from the assessment translators’ combined expertise and different points of view.

If the survey is long and the budget is tight, we can have each translator translate different sections. The goal is to bring more than one point of view across without doubling costs for the review conference to follow.

But whichever approach our clients decide to take, we ask the translators to keep careful notes as they proceed. These notes include questions about word choice and points on cultural nuance.

Review

This is where things start getting good. Once the translation team has finished the first draft, we arrange for the translators to meet with a reviewer in a virtual conference. The reviewer should not have been part of the original translation team.

The goal is to reach agreement on what translation is best. The reviewer brings a fresh approach and works alongside the translators to create a second draft.

Harmonization

Does everything play well together? With the second draft of the assessment translation completed, a second meeting takes place to make sure the translation is balanced with the assessment in other languages. This time the translators, the reviewer and the language manager all attend.

The language manager may be a testing expert in the subject matter who will work with the translators to decide if the translated assessment is ready for pretesting. The language manager works with the rest of the team to make sure the final assessment is harmonized across all languages, so that the small adaptations required by each language are compared across languages to ensure that the result is balanced as much as possible.

At the meeting, the team looks over the two drafts. Discussions are likely to focus on three areas: 1. Regional and multilingual variations in meaning, 2. Individual opinions on the use of words and phrases and 3. Issues that one or more of the translators may have missed in previous iterations.

To encourage teamwork, we try to ensure that everyone understands there is not a “correct” translation as such. Instead, the team’s goal is an assessment translation that meets the organization’s and the respondents’ needs.

Pretesting

Things are moving along. The meeting with the language manager may lead to a third draft. Either way, once the team is satisfied with the assessment translation, it goes for pretesting with a small group of respondents.

Any pretesting results then go to the translation team. If necessary, the team makes further changes to the translation. When they’ve reached agreement, the language manager and team all sign off on the document. This becomes the version for use in the field. It’s game time!

Documentation

There’s a certain beauty to the process. As with any ISO 9001-certified workflow, each step of the team translation process is documented. Any process errors are corrected for continuous quality improvement.

Also, experience breeds wisdom. Once the assessment translation is approved, each question and response becomes a valuable asset for future projects. Using translation memory, we maintain a database of all previously-used translations, so that if a similar phrase occurs in the future, we already have it mostly covered. Glossaries and style guides are updated at the end of each project for the same reason.

Other Approaches

When it comes to assessments, alternatives to team translation do exist, such as machine translation, back translation, ad hoc translation and oral translation. But these all have major drawbacks.

Machine Translation

Software translation is quick and cheap. But machine translation of test questions spells trouble. The result is a series of poorly worded questions that at their best make no sense, and at their worst confuse and mislead.

Ad Hoc Translation

Ad hoc translators tend to be bilingual staffers at the office “volunteered” for translation tasks. Asking someone who is bilingual to do an ad hoc translation of an assessment sometimes leads to amateur results. Translation is a profession for a reason: pros are worth the extra money that amateurs cost in bad translation.

Oral Translation

On-the-spot oral translation of assessments also produces inaccuracies. Actually, this “short form” is a license to kill in many settings. For example, healthcare interpreters are rarely trained for precise oral translation of questions. It is always a better option is to use a translation team to produce a written version of the survey. You can monitor a written translation and ensure its quality. You can’t do that for an oral translation.

Back Translation

With back translation, you arrange for the translation of a test from the original language into the target language. You then ask another translator to translate the target language survey into the original language. The goal is to identify translation problems by comparing the two original versions.

This method is useful for a sentence-by-sentence content check, but for matters of style and understanding, it can be very misleading. Back translations are useful only to help reviewers work in languages they do not know. Additional edits and review are always a better value.

Accuracy and Quality

In the end, what do you expect from your assessment? Being aware of other translation approaches allows you to make an informed decision about assessment translation. But what you want above all are accurate responses from your tests. Without these, you can’t investigate the experience and opinions of a target group of people relevant to your organization’s plans.

It is team translation that ensures precise assessment responses, not to mention a defensible process in the event of litigation. Team translation also helps you meet the standards of quality assurance your organization demands and the test takers themselves deserve.

For more best practices for assessment translation, contact us at 1-800-872-6752 (+1-212-818-1102 outside the US) or [email protected].