Leaping into Gesture-Based Computing

Gesture-Based Computing

Gesture-based technology, the ability to control devices through gestures or body movements rather than mouse clicks and keystrokes, has appeared in pop culture and sci-fi for many years (the go-to example is the 2002 movie Minority Report in which Tom Cruise’s gesture-controlled computer helps him identify the location of future crimes). However, gesture control technology has been slower to take hold in higher education, with the first appearance of gesture-based computing in the 2010 Horizon Report as an emerging technology trend with a potential to impact teaching and learning in colleges and universities.

Until recently, gesture-based technology was prohibitively expensive for most consumers, including libraries and librarians. With the release of gaming systems like the Nintendo Wii in 2006 and Xbox Kinect in 2010, gesture-based computing is making its way into many consumer households. Additionally, most modern mobile devices (including smartphones and tablets) emphasize interaction and navigation through gesture control.  

Leap Motion

Although gesture-based computing has established itself in the consumer market, many of these devices require a steeper price point than

he Leap Motion Controller, as compared to the size of an iPhone 5

The Leap Motion Controller, as compared to the size of an iPhone 5

libraries may have to devote to experimental technology. Due to it’s $79.99 list price (+shipping), Leap Motion  has become an affordable entry into the gesture-based computing market for interested educators and librarians. Released in July 2013, the Mac, Linux, and Windows-compatible Leap Motion system comprises of two parts – the Leap Motion Controller and apps available for download from the Airspace app store.

The Leap Motion controller is a small (about 3in long x 1in wide) device that plugs into the computer via USB. Installing the Leap Motion controller is as easy as plugging the controller into a USB port using one of the two provided cables, downloading the necessary software from the Leap Motion set-up site, and setting up a free Airspace account. A small selection of apps is already included in your Airspace account area, including the Orientation app, which demonstrates what the device “sees.” The device detects the space directly above it, with an approximately 2ft by 2ft by 2ft detection area.

When it was initially launched, the Airspace store included about 70 paid and free apps; today, there are nearly 150 apps. Games represent the largest portion of apps by far, but other categories include apps for education, entertainment, and productivity (including computer control).

Cyber Science 3D Motion App

Using the Cyber Science 3D Motion App with Leap Motion

Although the novelty of using gestures to play games and control the computer definitely make this an engaging device, for most librarians it’s not yet going to be an everyday use item. Due to the small detection area, you really need to hold your hands right above the device for it to function properly. Continue to interact this way for an hour or two, and your arms start to get tired. The device seems to be very accurate in detecting motions, though perhaps a little too sensitive since it tends to interpret jewelry (like a ring or a watch) as an additional finger or hand.

And the user experience really varies greatly by app. In many of the apps, the gesture sensing does not seem as sensitive as the initial Leap Motion Orientation demonstrates and there can be a lot of flailing about to figure out exactly what the app wants you to do. It is particularly difficult to understand where the invisible “screen” plane is when using Touchless for Windows to control your computer, which means getting real productivity out of the device is difficult (there are good examples of frustrations using Touchless included in this post from Scott Hanselman).

That being said, as app developers continue to improve and add to their offerings, the Leap Motion experience will likely improve. Right now, it’s limited to being useful for introducing library staff and users to the concept of gesture-based computing, perhaps by setting up a game demonstration for staff enjoyment or as an outreach program for students.

Future of Gesture-Based Computing in Libraries

Although inexpensive, practical gesture-based computing is still very much in development, there are already some excellent examples of how the technology trend may impact academic libraries and higher education institutions. Examples include:

  • The work done at the Bavarian State Library, which allows for interacting with 3D digital objects purely through gestures. Imagine digitally “flipping” the pages of a rare book, increasing the reach of archives and special collections materials.
  • The entrance to the recently constructed Johns Hopkins University Brody Learning Commons features the 12ft by 7ft Balaur Wall, which uses the Microsoft Kinect to allow users to interact with different types of media through gestures.
  • Researchers at the University of Washington have developed a gesture-based system using Wi-fi signals to detect body movements and control household electrics and appliances. Using Wi-fi signals to detect gestures may be useful because it allows for applying gesture-based technology to situations where it is not possible (or wise) to detect motion with the cameras and infrared light required by many gesture-based devices, including the Leap Motion controller and the Xbox Kinect.

What do you think the future of gesture-control technology will hold for libraries?