As an inherently immersive experience, holographic line-of-sight interaction provides an interesting opportunity for the language-learning experience.
Working in the graduate studio at the University of Washington, we had a unique opportunity to think about a truly emergent technology — line-of-sight interaction, as seen in the Microsoft HoloLens (augmented reality) and the Oculus Rift (virtual reality), and to investigate how design could take a leading role in anticipating applications for an upcoming interaction paradigm. The immersive quality seemed to be a natural fit for language acquisition, so we set out to envision what the process of learning a language (in this case, Dutch) might look like with a line-of-sight interactive design.
Holographic, mobile technology provides an experience that is highly contextual. In our case, the most obvious manifestation of this idea was to superimpose translated words onto objects — het boek over a book, and so forth. But nouns are only one small part of acquiring a new vocabulary. We conceptualized a system that would respond to actions and relationships to introduce verbs, adjectives, and prepositions into the experience, and which would also utilize repeated actions to reinforce learning.
For example, not only is the book het boek, but one might openen (open) or sluiten (close) it, or move it links and rechts (right and left). As shown in the concept video, openen and sluiten can then be applied toward other objects, like a bottle or a window.
Our team consisted of just two designers. Scott Tsukamaki illustrated our storyboards, created and animated 3d models in Rhino, and demonstrated the physical actions in our video. I filmed and edited the video, animated the interactive effects, and composited in Adobe After Effects.