Let’s say you are taking a survey. Question #1: Are you in favor of military rules? As in, the military being governed by rules? That sounds reasonable, doesn’t it?
Or wait, does the question ask about military rule? Are you in favor of military rule? As in, the military being in charge of the government? Wait, what?
This is just one real example of the dangers in survey translation! A subtle change in meaning can make a huge difference and can quite possibly render the findings of a cross-national survey meaningless.
Case in point is the World Values Survey. Started in 1981, this project is an admirable pursuit by an international team of social scientists (with headquarters in Sweden) to survey and document the changing social and political values found around the world. The World Values Survey polls people in 100 countries in an attempt to understand “changes in the beliefs, values and motivations of people throughout the world.” As such, the project’s data depends on a base level of standardization.
Charles Kurzman, Professor of Sociology at the University of North Carolina at Chapel Hill, recently wrote about the World Values Survey’s translation problem and its results for The Washington Post, citing the seemingly almost overnight difference in attitudes towards authoritarianism among the people in Iran, Vietnam and Albania. There wasn’t a sudden wave of changing attitudes, but a problem with the translation.
The rewording of one of the questions in Vietnamese, for example, meant that 99 percent of Vietnamese people surveyed favorably supported authoritarian leadership, while five years later, only 9 percent did. What a difference the way a question is translated can make!
There is no underestimating the importance of good survey translation, ensuring equivalence among all language versions, and translation validation. Once a bad translation gets to the field, the data is unfortunately already corrupted. I feel sad for the World Values Survey. Their premise is such a good one.