At 1-800-Translate, we are only as good as our last phone call. Sure, the linguistic competence of our telephone interpreters is job number one, but that’s not the only factor that makes for effective telephone communication.

Naomi Harte of Dublin’s Trinity College recently gave a presentation on all the different ways communication can go bad. Telephone conversation is problematic from the get-go. “Anything that disrupts our ordinary speech rhythms, as well as the way we process tone of voice, facial expression and other physiological cues, can affect interpretation of the speech act and transform meaning,” says Harte.

Harte, along with other researchers, is working on ways to improve “audio-visual speech recognition.” This means using visual data such as tracking lip movements to improve speech recognition and audio signal, and by helping machines sense lip and eye movements to discern meaning from gesture and intonation.

Emotion is really the most important aspect of human speech. It is human nature to feel first and then use language to rationalize and communicate for consensus. Humans are able to readily discern emotional content in conversation, but machines do not yet have the same skill set. The field of “affective computing” (that is, computing that manages emotional content) is still in its infancy. That’s one of the reasons why automated systems struggle to communicate effectively with telephone callers.

Surprising as it may seem, visual information can even define the sounds we hear. Check out the “McGurk Effect” in action from the BBC.

And timing is everything. While our back office switched to Skype years ago, for our telephone interpreters landlines are de rigueur. The brief lag familiar to voice over IP callers is a problem for telephone interpretation. We keep our interpreters on landlines to avoid that phenomenon.  That lag can break up speech rhythms which can really slow down the interpreting process and add to caller fatigue.

Speech rhythms already differ from language to language, and response patterns vary widely, too. This is also a factor in listener and speaker fatigue in bilingual conversations. Poor call quality or lags caused by voice over IP or excessive looping of a landline call can really decrease understanding and increase frustration during phone interpretation.

More on this from Philippa Nicole Barr in Does Skype help or hinder communication?