A group of academic researchers have figured out how to use off-the-shelf computer equipment and a standard Wi-Fi connection to sniff out keystrokes coming from someone typing on a keyboard nearby. The keystroke recognition technology, called WiKey, isn’t perfect, but is impressive with a reported 97.5 percent accuracy under a controlled environment.
WiKey is similar to other types of motion and gesture detection technologies such as Intel’s RealSense. But what makes WiKey unique is that instead of recognizing hand gestures and body movement, it can pick up micro-movements as small as keystrokes.
The research, conducted by Michigan State University and China’s Nanjing University, relies 100 percent on the 802.11n/ac Wi-Fi protocol and uses a TP-Link WR1043ND WiFi router ($43) and a Lenovo X200 laptop ($200).
Using the above equipment, researchers were able to use the Wi-Fi signal’s Channel State Information values to detect movements within a given environment. Channel State Information (CSI) in the past has been used to detect macro movements such as the presence of someone in a room, or hand or arm movements. A variation of this technology called WiHear was even developed to detect movements of a mouth with the ability to detect nearly a dozen different syllables spoken by a test subject.
But WiKey takes WiHear lip reading to an entirely new level by detecting finger, hand, and keyboard key movements. The researchers see the WiKey technology as a theoretic attack vector, but they also see WiKey with applications that go beyond attacks.
“The techniques proposed in this paper can be used for several HCI (human computer interaction) applications. Examples include zoom-in, zoom-out, scrolling, sliding, and rotating gestures for operating personal computers, gesture recognition for gaming consoles, in-home gesture recognition for operating various household devices, and applications such as writing and drawing in the air,” wrote co-authors of the scientific research (PDF) Kamran Ali, Alex X. Liu, Wei Wang and Muhammad Shahzad.
To capture keystrokes, or micro-movements, isn’t easy. Under a controlled environment, which doesn’t include a lot of movement such as people walking around or multiple people sitting close to one another using a laptop, researchers are able to detect even the slightest variations in wireless channel activities. Along with that data researchers also factor in wealth of information including signal strength, where the keyboard is located and what, where and why is interference occurring.
In order to collect micro-movement data using Wi-Fi, researchers use the router’s MIMO channels. MIMO is a wireless term used to refer to a router’s ability to use multiple antennas between a sender (router) and receiver (WNIC) that pass more than one data signal simultaneously of the same radio channel. The researchers explain:
“Each MIMO channel between each transmit-receive antenna pair of a transmitter and receiver comprises of multiple subcarriers. These WiFi devices continuously monitor the state of the wireless channel to effectively perform transmit power allocations and rate adaptations for each individual MIMO stream such that the available capacity of the wireless channel is maximally utilized. These devices quantify the state of the channel in terms of CSI values. The CSI values essentially characterize the Channel Frequency Response for each subcarrier between each transmit-receive antenna pair.”
If that didn’t sound challenging enough, next researchers have to filter out radio noise (frequency changes) and environmental movements not related to typing. Then, even after noise is removed, there are other considerations researchers needed to factor such as the time it takes to press a key.
By associating values based on the above culling of data researchers assigned number values to each keystroke (as seen below) based on individual typists.
Average values of features extracted from keystrokes of keys a-z collected from users.
Under the most ideal controlled circumstances where test subjects were limited to type only one a half-dozen different sentences and typing one key every one second the researchers achieved 97.5 percent accuracy. That controlled environment also didn’t include real-world scenarios such as people walking around in the same room and typing on additional laptops.
In what researchers call a real-world scenario WiKey drops to an average keystroke recognition accuracy of 77.5 percent.
“WiKey requires many samples per key from each user which may be difficult to obtain in real life attack scenarios. Still, there exist ways through which an attacker can obtain the training data. For example, an attacker can start an online chat session with a person sitting near him and record CSI values while chatting with him,” researchers wrote.
Researchers point out that this level of accuracy might be all that’s needed sniff out a password typed into a laptop. Other than being used in a potential attack, researchers hope WiKey can have a variety of non-attack applications such as gesture recognition.
“We have shown that our technique works in controlled environments (using commodity hardware), and in future we plan to address the problem of mitigating the effects of more harsh wireless environments by building on our micro-gesture extraction and recognition techniques proposed in this paper,” the researchers wrote.
