International Symposium: Programs: Summary: Session M1D

There have been some breakthroughs in sign to text (audio) to date, yet they remain impractical. The approach is to employ advanced imaging techniques incorporated with novel feature extraction technology in order to provide data for processing in real-time. Research steps include the determination of the figures of merit and statistical degrees of freedom involved in the extraction of physical data from signing, the development of appropriate and forward-thinking image capture techniques, development and implementation of real-time feature extraction algorithms for image data, the development of pseudo-real-time correlation algorithms for extracted data with statistical databases, the translation of positional data to ASL phrases, and the transliteration of ASL phrases to American English. We are building collaborative relationships with leading people in all of the key areas to accomplish this task.
The research and development goals of the Sign2 project are threefold. First, we intend to build into the body of scientific knowledge regarding sign-language to digital conversion. This goal is began by a formulating a foundational understanding of the necessary physical components of signing, its nuances, and the possible variation range within one classification of signing (e.g. ASL). Only with this understanding can optimal algorithms for the translation of static and dynamic physical position data be developed. This understanding will lead us to establish standardized digital points of articulation (dPOA) from the minimal distinction points that have been well established by the ASL community. Our direct association with researchers in the National Technical Institute for the Deaf (NTID) affords us access to the support we need at various levels in this area.
Our second goal is to engineer a fully functional prototype device that converts all of the dPOAs associated with a broad and critical range of ASL phrases to a form of Standard English, either text or audio, or both. The components of the Sign2 System are (a) the Physical Capture System, (b) the Data2Language Processing Algorithm, and (c) the Conversion Output Device. Each of these components has their unique challenges, yet we have assembled a team of researchers whose expertise makes them uniquely suited to successfully achieve this goal.
Our third goal is to develop a set of standards in the area of collecting and processing sign-language conversion data, and to build an ongoing statistical database useful to researchers around the world who are interested in furthering this work. Our primary mission is to improve the situation for the deaf and hearing, and to close the communication gap that exists between these communities through the judicious application of technology.