Through a Glass, Darkly

by Translation Guy on January 19, 2011
41 comments

Word Lens enables iPhone users to instantly translate Spanish into English. Heard about it? A Google search reveals about 1.4 million pages, so market awareness is pretty good. The video is great, so great that it made me want to pick up the phone and order some ceramic steak knives, but since that required a couple of swipes and 10 key strokes for the toll free for the steak knives, I decided instead to download the translation app onto my new iPhone and test it out.

I test these language tools all the time. You don’t read about it too often because most of the tests are perfect fizzles. It goes something like this: I load the app. If I can figure it out, I use it until I get bored with it―10 minutes tops―then spend another half-hour pulling it off of my PC. So now, to reduce fizzling, I ignore them all. But that video on YouTube was so cool, even though as I look at it I can see how jammed it is.

Word Lens allows you to hold your iPhone up to a sign in Spanish, which it then translates into English on your iPhone screen. In the demo, someone holds up various signs, which the app then translates into a pitch for the app itself. The idea is that the app will allow you to read signs and menus when traveling. That’s if you are travelling in a Spanish-speaking country, of course, since English and Spanish are the only languages the tool can read. This type of interface is called “augmented reality,” a live direct or indirect view of a physical real-world environment whose elements are augmented by virtual computer-generated sensory input such as sound or graphics.

In the demo, all the signs are printed using highly readable fonts against clear backgrounds, face-on and held at waist height about 6 feet from the lens. This is what I call augmented demo reality, as in you set up a “real world” demo that augments reality so that the device can actually work in a setting made completely and deceptively artificial.

So this app would be great for reading Burma Shave signs in Mexico, with little road signs rolling straight past you (and with your passenger holding the smartphone)… please. Otherwise, what have you got besides roadkill? Optical character recognition from hell, where every misread letter is a monkey wrench tossed into the gearbox of the translation engine, so that each misread character blows up a sentence worth of translation.

These kinds of apps have been around for years and, well, fizzle is as fizzle does.

But who cares, really? If you are planning on using a smartphone to find your way out of the museum or to tell your Goya from your Dalí, I’m not so sure that Word Lens will give you the language leg up you are looking for. If you reach a point where you need a camera to order dinner, best take your nose out of the small screen and keep your eyes on the road.

And now on to the final part of the demo, where this fizzle of a review will encourage the 50 cent party members of Word Lens to comment on the multiple benefits of what they are paid to perceive as a useful, even life-saving, tool. I prepare to stand corrected.

This is because all language technology demos must be massaged into mimicking functionality for a few brief moments during the presentation. Otherwise, the bilingual experience will be alienating and frustrating, just like it’s supposed to be. (Isn’t that what different languages are for, after all, to keep us all from speaking to strangers?) So I don’t test. But that video was so cool.

41 Comments

  1. Tammy Lang says:

    This is highly nifty.

  2. This is some whacky Star Trek stuff here, holy smokes

  3. Jon Baker says:

    This product already exists with all the langauges you talk about, and on android and blackberry. It is called TGPhoto and was released by LinguaSys two weeks ago.

  4. The Clown says:

    Nice post Ken. There’s a funny independent test of the app using colorful Spanish expressions at drupt, I think you’ll get a kick out of it :-)

  5. A novel idea that I bet these guys will make plenty of money on.

  6. Joel Wiley says:

    I came up with this idea a couple of years ago, and said it to my brother (who writes phone apps), but he thought it was impractical… curse that fool..

  7. Sort of like a talking dog. It’s not that it talks poorly that’s of note; just that it talks at all. That said, I don’t see how it’s going to handle more than a few fonts in a couple of orientations on anything but plain one-colour backgrounds. On the other hand, people might start actually making legible signs so that they can be read by apps like this (people never move on ergonomic problems until they inconvenience a machine) and that would be a good thing.

    • Ken says:

      Wow. Surely you are correct. So real reality will be altered to suit the augmented version. And augmented reality will be altered based on what Google thinks will please or alarm us. Oh, brave new world!

      “And that after this is accomplished, and the brave new world begins
      “When all men are paid for existing and no man must pay for his sins…
      -R. Kipling

  8. Pearl says:

    Now why would I use this app? I can just type in the words into Google Translate and get it translated into any host of languages for FREE. For $4.99 this is price that is way too much to pay for it.
    Couple of apps that I thought have truly done some innovation:
    1) Spotify – First app that caches your music for offline listening from within the app (no iTunes needed)
    2) Azul Media Player – First app that downloads videos for offline viewing from with app (no iTunes needed)

  9. Kat says:

    I thought you might like this Ken. So two German professors, having newly learned English, arrive at JFK International Airport in New York. Proud of their newfound ability to speak to the natives, they immediately head for the airport bar, where the bartender asks them what they’d like.

    “We,” one enunciates carefully, “would like two martinis.”

    The bartender nods and moves off to his drinks. Then he turns and says, “Dry?”

    The frustrated German shouts, “Nein! Nein, nur zwei!”

    • Ken says:

      “No! No, only two.” Love it, Kat. Puns are always funnier when bilingual. Thanks.

  10. Best app of the year – hands down.

  11. Ducky says:

    Does this app actually manage on-the-fly, augmented reality translation?

  12. Nokia has had something similar to this for years! http://store.ovi.com/content/22499

  13. Harry Bush says:

    Looks a great idea. I wonder if it would work “internally” on the mobile? e.g like on http://www.mycall.mobi (local search) to show whatever in the language of your choice?

  14. Sweet Potato says:

    In terms of language translation, Word Lens does a respectable job; in terms of convenience and ease of use, Word Lens will make you want to chuck your language guide.

  15. PickleHead says:

    It’s really amazing. I face a lot of problems to read Spanish and understand it but it would be really helpful. What is the accuracy ratio of this app?

  16. Emily Davies says:

    It does seem like science fiction for what it does, regardless of how good the translation is. It has the same limitations of all machine translation that uses statistical analysis. That is, with limited words, the words tend to be literal and potentially incorrect; it appears that with more words, it does a better job, which means they’ve got it wired correctly.

  17. Dewey says:

    if the translations come out anything like these blahblahfish I’m happy. best app ever.

  18. Mad Dog says:

    The first thing I thought when I saw this app on Gizmodo was “Holy shit! This is just what the TARDIS does!”

    Yes. The TARDIS is an augmented reality engine.

  19. Google goggles anyone? Take a picture of text and translate it in 20+ different languages. Free on android

  20. Pookie says:

    In addition the verbiage are all anchors and can be looked up for their meanings. Now that’s added value.

  21. Big Daddy says:

    A transformative application. Best new thing I have seen in 2010.
    A brilliant integration of many technologies into a mobile computing app.
    Hard to believe some say $5.00 is “expensive”.
    Hard to understand complaints about performance.
    Just imagine this in a few years. Can’ wait for Chinese.

  22. Kitty says:

    As a bilingual person, I have to say that most automated translators suck. However, that being said, they can also be immensely helpful. Many times you can get the gist, which is better than not knowing what the text means at all. Also, often the results are quite hilarious.

  23. This app is facing several of the hardest problems in CS: image/glyph extraction, OCR, and translation. An error in any of these processes results in substantially greater error as the processing progresses.

    The fact that it works at all on a piece of hardware like an iPhone is amazing, and a promise of really cool shit yet to come!

  24. Ronald Miles says:

    So Wittgenstein meets his friend on a train to Vienna exclaiming that he has finally converted language into a mathematical process, solving all translation issues and settling numerous historically troublesome debates simultaneously.
    His friend shows him an image of the ORLY Owl and says, “What’s the mathematical symbol for that?”
    Ach Ja… sighed Wittgenstein and went back to the drawing board.

  25. Papito says:

    If it works without a network connection I’d consider it, not really much point otherwise unless roaming charges come down drastically.

  26. Helen Bolton says:

    I reviewed this as well and I can say that The translations are of the pidgin English that we’ve come to expect from computerised services such as Babelfish and Google Translate, enough to get the gist, if far from grammatically perfect.

  27. Tessie says:

    I tried this out on the cover of Linux Journal #175 that I had on my desk at work and it didn’t do too poorly. I used the reverse text plugin supplied for free and it worked on pretty much the whole cover. It didn’t recognize some words like GPU (not a statistically likely combination?) or Cory Doctorow (it didn’t like the other red letters on the page, either), but otherwise it did a pretty good job. There were some strange results when pointing it at handwriting, especially parts that I had scratched out. :-) I am impressed with the recognition capability, but I would assume san serif typefaces on large signs would provide the best results. Entertaining at the very least.

  28. Shnookums says:

    I’ve used other augmented reality apps for the iPhone, and there is a sense of the bizarre. Hold up Urbanspoon or Yelp in their augmented reality mode, and you see overlays of restaurants names and ratings hanging there in space.

    It’s not hard to project that into rechargeable eyeglass displays (which exist today) using Bluetooth communication from a smartphone.

    Next up, Terminator style social responses in our field of vision.

    • Ken says:

      That’s what I want.

  29. Pinky says:

    Early voice recognition and text-to-speech software were klunky too. And remember the early days of Babelfish, or the even crappier translation programs that preceded it? All grew more refined as new iterations come to pass. This is magic. And magic needs time to evolve.

  30. Nina Nguyen says:

    Worthy of noting is that while this app might be free, if you want the translation feature you have to pay $2.99 for the language pack.

  31. Ozzie says:

    I’ve tried it and it’s not perfect but it’s amazing! Just do an image search for “spanish signs” and point the iPhone at your computer screen to try it out. Every once in a while you come across something that reminds one that we are, in fact, living in the 21st century and Word Lens is one of those things.

  32. Kent Hartman says:

    Our Israeli friend does make a somewhat valid point (while missing the point of the augmented reality magic going on here), but I’ve long wanted something like this that works with Kanji. Pleco is doing something similar already (http://www.engadget.com/2010/11/24/pleco-chinese-dictionary-iphone-app-now-handling-real-time-image/), but for Chinese rather than Japanese Kanji (and my iPod doesn’t have a camera anyway).

  33. I really like your transitions and clarity. I have been producing for Ghost Writers for a while now, and they pay me good to write blog posts like this, or content articles. I clear $100-$200 on a awful evening.
    Judging by your skill with the written word, you may enjoy doing the same.
    It wouldnt hurt to check them out. [REDACTED]

    • Ken says:

      You flatter me. While I’ve done a transition or two in my time, I could never match your insouciance when it comes to English grammar. Tip: gin-soaked lunches at the Algonquin Club will generally attract a better set of better ghost writers than your mindless spam.

  34. I was wondering why there is only Spanish into English –what about the Spanish Translation of a road sign –just in case you feel like travelling through South America and still believe that it’s not necessary to at least flick through the Lonely Planet Phrase book before going. As long as people only use those apps for fun rather than really thinking this will allow them to fully understand written texts in other languages like the idea and can see the challenge in developing and improving this software. As it is often the case with mankind, it is mostly about seeing how far you can get.

    • Ken says:

      Two good points, Ben. I spoke to the creator of the app this weekend and I’ll be sharing his thoughts on both these topics in an upcoming post.

LiveZilla Live Chat Software