Using Radio Waves to See

By Mark Roberti

The HearMeFeelMe project is developing NFC applications to assist visually impaired seniors with daily tasks.

  • TAGS

One of the virtually inevitable results of growing old is impaired eyesight. While technology can't prevent visual impairment, it can help support those who must deal with it, says Minna Isomursu, coordinator of the HearMeFeelMe project, which is using Near- Field Communication (NFC), a short-range form of RFID, to develop applications to help seniors identify everyday items and manage their medications.

The idea behind the HearMeFeelMe project, says Isomursu—who is also a research professor at the VTT Technical Research Centre of Finland, a prestigious applied-research institute—came about during work on a larger NFC initiative called SmartTouch, funded by the European Union (see Finnish Parking Pilot Taps RFID Tags and NFC Phones). VTT worked with Top Tunniste (an NFC systems integrator in Finland), Tecnalia (a private research company in Spain) and the National Centre for Scientific Research (a research organization in Greece that is also known as Demokritos) on the SmartTouch project.


Minna Isomursu

"We've been exploring the technical challenges and application potential of NFC at VTT for at least 10 years now," Isomursu says. "Before the HearMeFeelMe project, we had a big European collaboration project called SmartTouch, and the idea for using NFC for helping visually impaired people emerged in this project."

During work on the SmartTouch project, the researchers found they could use NFC technology to build intuitive, easy-to-use interfaces for digital services for older adults. "NFC-based interfaces can be designed so you do not need to read the screen or press the buttons of a mobile phone—both skills that become more difficult with aging," Isomursu says. "We built several service interfaces for older users, and found the technology was robust, easy to learn and use for older adults with physical or cognitive challenges."

In 2007, the collaborators won the annual NFC Global Competition, sponsored by the NFC Forum, for their "Seeing Eye Phone" idea. "We got lots of feedback and interest in the concept," Isomursu says, "and this made us think it would be worth investigating the idea further in a project specifically addressing the needs of visually impaired."

The HearMeFeelMe project was funded by the Ambient Assisted Living Joint Programme, a European organization dedicated to enhancing the quality of life of older people; the organization requires at least three countries to participate in a project. "Also, partners had complementing technical competences," Isomursu says. "Tecnalia has been doing NFC-related technical development for a long time. At VTT, we had expertise on service design and user studies, and Demokritos had competence in indoor location technologies."


Photo: VTT Technical Research Centre of Finland | Illustration: iStockphoto

The Touch 'n' Tag application lets visually impaired people identify everyday items, such as clothing and food, with the help of voice memos. Using an NFC-enabled phone, the user records an audio file—"This is a green sock" or "This is a red T-shirt," for example—that is stored in the phone and associated with a specific tag. When the user touches the NFC label affixed to the item, the tag is read, triggering a playback of the recording. Elderly volunteers who tested the applications developed by the HearMeFeelMe researchers said this was their favorite, Isomursu says. They used it most often to mark food packaging, and said it was useful in recognizing items and recalling product information. They liked that they could record information useful to them, she adds.

The researchers also developed five NFC applications for acquiring medical information. By touching an NFC tag on a pill bottle with a mobile phone, for example, the user can download product and dosage information, which can be played back on a phone or computer.

Another application that has not been tested yet by volunteers is an "almanac" demo designed to ensure users take medication at the proper times and tend to other important activities. The application makes use of an elderly person's social network. Nurses and family members can send reminder messages to the patient via cell phone about scheduled medicine doses or meetings, and the user can reply with an NFC acknowledgement. For example, the user can touch the pill dispenser with the phone to inform the caretaker he or she has taken the medicine.

To make the solutions attractive from a business point of view, the researchers needed to develop solutions that could be used across European markets, if not global markets. "Medication-management practices, such as procedures followed at pharmacy and doctor's office, and cultural and social aspects related to medication management, are very different in different regions," Isomursu says.

The researchers considered using 2-D bar codes for the applications, but chose NFC because it is easier for the visually impaired to use. "Through user studies, we have found out that users find touch-based interaction more intuitive and natural compared with using cameras and bar codes," Isomursu says. "The users describe the experience as 'magical.'"

The Touch 'n' Tag application is ready for commercialization. The other applications created would need further development before they were ready for consumer use. Now that the project is complete, Isomursu says, VTT and the other institutions involved have no plans to develop additional applications. Some folks who are visually impaired might feel that's a bit shortsighted.