Key Speak and Assistive Technology
A keyboard recognition application for Assistive Technology
Captain Luis Avila, U. S. Army at Walter Reed, was severely injured by an improvised explosive device in Afghanistan at the end of December 2011. On his fifth wartime deployment at the time he was wounded, Captain Avila lost a leg and sustained a severe brain injury. His road to recovery was complicated by two strokes and a heart attack suffered early in his course of treatment. From Afghanistan, Captain Avila was transferred first to Landhaus, Germany and then to Brooke Army Medical Center in San Antonio. In July 2012, he was admitted to Walter Reed National Military Medical Center in Bethesda, MD.
President Obama recently shared part of the Avila's story in a White House "Salute to the Troops" introduction.
The ProjectAs a result of his injuries, Captain Avila’s speech, vision, and dexterity are seriously impaired. He is knowledgeable about, and well versed in the use of, computers. He has access to a computer that has been adapted to meet some of his needs, but his ability to use the keyboard is limited inasmuch as he is able to use just one finger to type. At that, he frequently misses the key he intended to strike, often missing his target by more than one key.
The Assistive Technology Program has been an integral part of the team caring for Captain Avila for most of his stay at Walter Reed. The circumstances described above have made it necessary to rule out use of a variety of assistive technology strategies that might otherwise support Captain Avila’s use of the computer.
The device sought for use here would facilitate Captain Avila’s use of the computer by ensuring that his finger is on the intended key before the keystroke is sent to the computer. The device in question would establish a USB interface between the keyboard and the computer. When Captain Avila strikes a key on the keyboard the first time the interface box would interrupt the signal before it reaches the computer and speak the name of the chosen key. If the chosen key is the one he intended to hit, Captain Avila would strike the same key a second time and that strike would the pass the signal on to the computer. As a result, each key strike will effectively require two strikes to enter the desired information.
Key Speak Utility
Using our PS/2 Host Emulator to capture scan codes from a PS/2 keyboard, we wrote an application to use Windows Text to Speech feature independent from the accessibility settings. We dubbed the application Key Speak (free download here).
The first press of any key triggers the name of the key. A second press of the same key sends the USB key code to the OS with a confirming ping. If, instead of a second press on the same key, the next press is a different key, the name of the new key is spoken.
Shift, Alt, and Ctrl also require two presses and are held down (like Sticky Keys) until another key is pressed twice. A different confirming tone is used to indicate the state of the modifier. They work cumulatively such that a “Ctrl+Alt+Shift+X” macro is possible.
Upper Case and Shifted Keys
An alternate vocabulary is referenced for Shift modified keys on the keyboard. With the Shift modifier on, pressing the “A” key results in “Cap A” being spoken, and pressing the “4” key results in “Dollar Sign” being spoken. Thus the user gets a more accurate vocalization of the key being sent.
Caps Lock, Num Lock, and Scroll Lock
Key Speak does not support the Caps Lock, Num Lock, or Scroll Lock states. However, it does recognize the key presses. For our test case, we programmed the commonly used macros Ctrl+C, Ctrl+V, and Alt+F4 on these keys respectively.
Key Speak works right out of the box, but it also uses a simple text file for configuration which is human readable and easily edited in Wordpad or any text editor. Macros may be created for any key on the keyboard. The vocabulary for the Text to Speech feature may also be edited. Thus our macro keys may be identified as “Copy” and “Paste” or as “Control C” and “Control V” at the user’s preference.
- Key Speak Utility (free download, 45KB)
- PS/2 Host Emulator from P. I. Engineering
- PS/2 keyboard (commonly available)
- Windows 7 or newer
- Speakers or headset
How it Works
The Host Emulator essentially turns the PS/2 keyboard into an X-keys device. PS/2 scan codes sent from the keyboard are captured before they get to the operating system and converted to our X-keys USB HID messages.
Key Speak receives our HID message and triggers Windows Text to Speech on the first press of any key. Two sequential presses of any key will send a USB keyboard message to the operating system.
Limitations and More Solutions
The method used for sending USB Keyboard to the operating system does not support boot level keyboard messages. Thus the “Ctrl+Alt+Delete” combination will go to the OS, but not at the level required to trigger a Windows reset. The same is true for the Windows key. We solved this for the client by adding an X-keys Stick programmed at the hardware level to send “Ctrl+Alt+Delete”. One of our X-keys USB Switch Interface devices could also have been employed.