iCare-Reader - A Truly Portable Reading Device for the Blind

Terri Hedgpeth, Mike Rush PE, Vivek Iyer,

John Black, Mehmet Donderler, Sethuraman Panchanathan

Arizona State University

Tempe, AZ - 85287. USA.

email: terrih@asu.edu, mike.rush@asu.edu, vivek.iyer@asu.edu,

john.black@asu.edu, mehmet.donderler@asu.edu, panch@asu.edu

Abstract

According to the world health report of 1997, there are approximately ten million blind and visually-impaired persons in the United States [1], and forty five million blind individuals world-wide [2]. Though there are many existing solutions to the problem of assisting individuals who are blind to read, however none of them provide a reading experience that in any way parallels that of the sighted population. In particular, there is a need for a portable text reader that is affordable and readily available to the blind community. A truly portable text reader would allow the user to take the reader to the print, rather than bringing the print to the reader. This paper describes our efforts to design such a device – the iCare-Reader.

1. Introduction

While there are many existing solutions that assist individuals who are blind with accessing print such as Braille books, refreshable Braille devices, audio recordings, screen readers, and text scanners; none of these provide a reading experience that parallels the ease with which sighted persons access print. For example, these currently available technologies don’t allow the user to take a book from the shelf, peruse the table of contents, and then flip through the pages of a book to find the desired page. Aside from Braille and DAISY books, the user even finds it awkward to re-read difficult passages in a book, as is often necessary when studying a textbook or a reference book. The goal of the iCare-Reader project is to allow the readers who are blind or visually impaired to do all of these things.

2. Motivation

Though we rarely stop to think about it, sighted individuals are continually bombarded every day by the printed word. Some of the sources of this abundance of print media in our environment include transportation, advertising, news and commercial signs. Sighted individuals are continually exposed to information from these and other sources, without even being consciously aware of this fact. However, this is a phenomenon that people who are blind currently do not experience.

The workplace is another area in which the ability to read text is simply taken for granted. Staff meetings and task groups often revolve around print and graphic based presentations from which an employee who is blind is simply excluded from. Even the process of eating out is complicated by the fact that few restaurants have menus in Braille.

All of these facts underscore the necessity for a portable text reader that is affordable and readily available to the blind community. Such a device would go a long way towards filling this socially vital need to freely access printed material. While several assistive devices have been developed in the past to assist with the reading of printed text, they have all fallen short of the user expectations. In particular, most have been too cumbersome and/or not readily available to be practical and truly portable. What is needed is a portable reading device that would allow the user to take the reader to the print, instead of having to bring the print to the reader. This is one of the important long-term goals of the iCare-Reader project.

3. The Current Technologies for Book Reading

Before discussing our iCare-Reader, it is appropriate to review the existing methods that are being used by people who are blind for reading. These include Braille books, audio recorded books and Optical Character Recognition (OCR) based systems. Each of which has its advantages and disadvantages.

3.1. Braille Books

The primary advantage of Braille is that it allows users to read in their preferred manner such as skimming the text, search for bold or highlighted text, headings, subheadings, paragraphs and check the spelling of any name or word. However, their sheer bulk makes Braille books to cumbersome to store. For example, the Webster’s Collegiate Dictionary in print is about 3 pounds, 3 inches thick and is only one volume; but, the Braille copy of that same book is 75+ volumes each about 3-4 inches thick and measures 12 x 12 inches square and it costs over 5 thousand dollars. So, although a large percentage of sighted persons have a dictionary in their household, you can understand why it isn’t practical for a blind individual to own their own copy. Another problem is the limited number of books available in Braille. Due to the cost and large amount of room necessary for storage, there aren’t all that many Braille books produced. Also, most of the written material encountered in day-to-day life, other than books, is seldom converted into Braille. In the academic setting, this may include class handouts, homework assignments, supplementary readings and exams.

3.2. Books in Audio Formats

Cassette tapes provide a means for storing and accessing audio recordings of information. However, there are several disadvantages to the use of tape for reading books. First, tapes provide access to the user in a strictly linear manner. If the user wants to start with the third chapter of a book, it is tedious and time consuming to fast forward to the appropriate place on the tape. Second, if a particular passage of the text is not fully understood on the first reading, the tape can be rewound. But the exact location where the tape stops is not precisely controlled, so it’s difficult to backtrack a sentence or a phrase at a time. Third, the reader can’t check spelling, as is possible when reading Braille. Fourth, the user has virtually no access to the critical formatting elements such as bolding and italics, paragraph beginning and ending or any other techniques that publishers use to emphasize specific terms or passages.

Then there’s mp3 recordings in the DAISY format, which allows the reader to navigate through a text by word, line, sentence, paragraph, page and chapters, with the DAISY Consortium special Extensive Markup Language. However, the user still cannot check spelling and there are still a very limited number of text offerings when compared to the sheer volume of text available to sighted readers. Just consider that a typical library is filled with thousands of books that have never been converted into any of these highly specialized formats. Another major drawback is that Braille and audio recordings are conversion processes that require the involvement of a sighted reader and special facilities to do the conversions.

3.3. Flat Bed Scanners and OCR software

One solution for accessing print is to use commercial off-the-shelf flat-bed scanners and OCR software. A book is opened and placed face down on the scanner, allowing a page to be scanned. Then the OCR software processes the image, producing a text file that can be then heard through the use of text-to-speech technology.

While this is a practical solution for scanning and reading fairly simple and small volumes of printed material, it is inadequate for scanning math, science text and tables both simple and complex. Also the user cannot easily read and/or reference two or more books simultaneously, which sighted students do when they are studying or doing research. For example, it’s not hard to imagine the difficulties a student encounters when told by his/her instructor to “read chapter 4” or “read pages 47 through 58” of the textbook before the next class. Short of asking for assistance, the only option open to him/her is to repeatedly leaf forward and backward through the book, scanning pages in search for the appropriate page number or section.

3.4. Reading Common print

All of the solutions described above are intended to assist with the reading of hardcopy text in a controlled environment. However, these solutions do nothing to provide access to much of the common print that we all encounter in day-to-day situations, such as restaurant menus or vending machine labels. In order to access this kind of text, it would be very helpful to have a wearable device that could capture images of the environment, detect the presence of text, and then apply OCR technology to the extracted text in real time.

4. The iCare-Reader

Ideally, a text reader should be wearable, so that the user can use it in any day-to-day situation. The text reader hardware should also be unobtrusive, so that the user does not “look like a Martian” when wearing it. It must not be power hungry, since it will be battery powered, and it must be adaptable to environmental variations, such as lighting.

Such a reader is not hard to imagine. A tiny video camera could be mounted in a pair of stylish glasses, and connected wirelessly to a Personal Data Assistant (PDA) sized device. This device processes the video stream with OCR and text-to-speech software, and then synthesizes a voice that can be heard through tiny sound emitters fixed to the ends of the leads on the glasses where they would fit neatly and comfortably behind the ears, thus not impeding the users’ ability to hear. Unfortunately, today’s miniature video camera technology is not yet able to deliver accurately auto-focused image streams that are tolerant of lighting variations, and have enough spatial resolution to satisfy the needs of OCR software. Rather than waiting for miniature camera technology to become available, we have chosen to take a 3-phase approach to the development of our iCare-Reader.

The iCare-Reader is a device that can be used by people who are blind or visually-impaired to read printed material, such as newspapers, magazines and books, in real time. It employs a collection of commercial off-the-shelf hardware, including a video camera, OCR software, text-to-speech and voice synthesis software, which is augmented and integrated with custom software to compensate for image distortions and lighting variations.

4.1. Phase 1 - The Table-Based Book Reader

The first phase of the iCare-Reader project involves the development of a table-based book reader prototype that is suitable for deployment in a library. Figure 1 shows our current prototype. The figure shows a user seated at a desk-sized workstation, with a book open in front of him. A digital video camera with a computer-controlled tilt and swivel mechanism is mounted above the desk. Buttons are provided on the tabletop for controlling the book reader. A desktop computer controls the camera, and responds to the user input.

Figure 1: The table-based book reader

When the user pushes a button instructing the book reader to read the open page in the book, the camera, under the control of software running in a desktop computer, takes a snapshot of the table. Image processing software then determines the exact position of the book on the table. Using the resulting coordinates, the camera is zoomed in on the page to be read.

Several reading modes are provided. A “Table of Contents” mode can be used to read the titles of the chapters, and the page numbers where each chapter begins. A similar “index mode” can be used to read the index at the back of a book. When the user determines the desired page number, a “page number” mode recites the page numbers as the user turns the pages of the book. When the desired page is reached, the “Text” mode is used to read the text on that page. A special “finger tracking” mode allows the user to guide the reader, backtracking when a difficult passage must be re-read.

The table-top book reader is intended for deployment in libraries, and additional prototypes are being constructed for deployment in the various on-campus libraries at Arizona State University. Hopefully other libraries will also be equipped with these book readers in the future. However, people who are blind will not always be able to travel to a library to read books. For this reason, Phase 2 involves the design of a portable book reader.

4.2. Phase 2 - The Portable Book Reader

It is anticipated that in the near future, greatly advanced and highly sophisticated smaller cameras and computer technologies will allow the development of a portable book reader prototype, which can be carried in a briefcase. The user would then be able to place this briefcase on a table at a location of his/her choice and read a book. It is anticipated that the portable book reader will utilize a similar tilt-and-swivel camera as in the table-top-book reader, though the portable system will be much smaller and lightweight for true portability. The advancement into Phase 2 is primarily dependent upon the development of the necessary smaller camera and computer hardware.

4.3. Phase 3 - The Wearable Text Reader

Phase 3 involves the development of a lightweight wearable text reader, which can be used for reading common print, such as signage, vending machines and hardcopy text. This text reader would consist of a miniature video camera in a pair of glasses and a belt-mounted PDA-sized computer, as describe previously. As in the case of the portable book reader, this prototype must await advances in technology.

However, there is one significant research problem that will need to be overcome in order to make this approach practical. Since the camera will be aimed by head motions, instead of a computer-controlled tilt-and-swivel mechanism, there will be a need for an interaction protocol between the computer and the user, in order to aim the camera at the text to be read.

(a) (b)

Figure 2: Developing a Human Computer Interface

In order to begin research in this area, we have constructed some head-mounted camera prototypes, using commercial Firewire digital video cameras. Figure 2(a) shows one of these prototypes in use. Here a sighted user wearing the head-mounted camera has been blindfolded to simulate blindness, but is able to communicate verbally with a sighted person at a remote location through a headset. Figure 2(b) shows that remotely located person seated in front of a video monitor, which displays the video stream that is being captured by the head-mounted camera.

These two people then collaborate verbally, via their 2-way voice link, to aim the camera at the book and to read the contents of the page. As they collaborate to achieve this task, the video and the audio streams for the entire session are recorded and archived for later study. The goal of this experiment is to learn what types of interactions are used between pairs of humans to aim and focus the camera at the book and to navigate through the book to find the desired content. These interactions can then be formalized into protocols, and the human in front of the video monitor can be replaced with a computer system that implements these same protocols with voice synthesis and voice recognition devices.

5. Conclusion

The iCare-Reader will ultimately provide people who are blind with a level of access to print that information that will be more like that of sighted people than other assistive devices have been able to achieve. While the user interaction protocols are still being developed, using feedback from users who are blind, it is already clear that this approach has significant advantages over the current flatbed scanner approach to book reading. While the transition to the smaller portable “briefcase” hardware awaits the advancement of technology, there isn’t any other obvious impediment to the development of phase 2. However, the transition to a wearable device will present significant new challenges. Specifically, it will require a much more collaborative interface between the user and the reader, to allow the reader to “coach” the user in aiming the camera at the reading material by head movements. Experiments that are currently underway are being used to develop the interaction protocols that are necessary for accomplishing this complex task.

6. References

[1] http://www.the-fbc.org , Foundation for Blind Children.

[2] http://www.who.int/whr2001/2001/archives/1997/exsum97e.htm World Health Report 1997 Executive Summary

[3] http://www.w3c.org World Wide Web Consortium w3c.

[4] http://www.DAISYconsortium.org Digital Access to Information System.

[5] http://www.afb.org American Foundation for the Blind.