Spring 3-20-2006 Improved Tool for Fingerspelling Recognition

Our new fingerspelling practice software displays realistic animations of fingerspelling, including naturalistic transitions between letters that were previously impossible. Introduction Fingerspelling is an important component of American Sign Language (ASL) and is a necessary skill for complete communication in sign [BATTISON 78]. Padden found that fingerspelling makes up 7 to 10 percent of the signs produced in everyday ASL conversations [Mulrooney 02]. Fingerspelling is useful for spelling proper nouns, technical terms, acronyms, initialized signs, loan signs and words from foreign languages. When fingerspelling, people use their dominant hand to create a series of manual symbols, one corresponding to each letter of the word. A person fluent in ASL can produce fingerspelled words at a rate of four characters per second. In contrast, the fingerspelling recognition rate for people learning sign language is far lower. Acquisition of fingerspelling recognition skills typically lags far behind other sign language skills [GRUSHKIN 98] [SCHLEPER 03]. Wilcox surveyed sign language students, who reported that recognizing fingerspelled words was the toughest part of learning ASL [WILCOX 92]. “Fingerspelling is the first skill learned and the last skill mastered” [LAKE 04]. Patrie found that experienced interpreters were significantly more likely than novices to correctly identify fingerspelled pseudowords words [PATRIE 92]. All 31 Baccalaureate degree programs in ASL/English Interpreter Training devote significant portions of several classes to fingerspelling and over half offer an entire course in developing fingerspelling skills [REGISTRY 04]. Achieving fingerspelling fluency requires the visual comprehension of the manual representation of letters. One reason students experience difficulty in fingerspelling recognition is its high rate of symbol presentation. Most signs in ASL use no more than two hand symbols [BATTISON 78], but fingerspelling uses as many symbols as there are letters in a word. An additional barrier to improving fingerspelling comprehension is the lack of resources for self-study. Several interactive programs do exist that are capable of showing a series of static images of the manual alphabet [VISION 93] [ASL 05] [GAY 01] [INSTITUTE 03], but none show the motion between letters. These motionss are essential in fingerspelling recognition. When interviewed, fluent signers mention that they look not for individual letters, but for the “shape” of a word [GROODE 92]. Watching a series of static images in sequence will not help. To solve the problem of realistically displaying fingerspelling, we have developed an approach that displays the transitions between letters. We have developed new software which implements this technique as part of an interactive learning tool for practicing fingerspelling recognition. As Figure 1 demonstrates, this new technology displays the transitions that naturally arise when a person produces each letter in succession.