kbd-audio

This is a collection of command-line and GUI tools for capturing and analyzing audio data.

Keytap

The most interesting tool is called keytap - it can guess pressed keyboard keys only by analyzing the audio captured from the computer's microphone.

Check this blog post for more details:

Keytap: description and some random thoughts

Video: short demo of Keytap in action

Keytap2

The keytap2 tool is another interesting tool for recovering text from audio. It does not require training data - instead it uses statistical information about the frequencies of the letters and n-grams in the English language. A more detailed description of the tool is available here: Keytap2 discussion

Online demo using WASM

Video: short demo of Keytap2 in action

CTF: can you guess the text being typed?

What people say about Keytap

"This works incredibly well.\ I hope you realize what you've created (and made available to every person in the world)." -- ffpip

"I just tried it and it works incredibly well. It kind of makes me want to stop using a mechanical keyboard." -- Karawebnetwork

"This attack and Van Eck phreaking are why Edward Snowden, while typing passwords and other sensitive information, would pull a blanket over himself and his laptop." -- aarchi

"This is what mechanical keyboard users deserve" -- super guy

"fuck.." -- Lluis Franco

Build instructions

Dependencies:

• SDL2 - used to capture audio and to open GUI windows libsdl

  [Ubuntu] $ sudo apt install libsdl2-dev

  [Mac OS with brew] $ brew install sdl2

• FFTW3 (optional) - some of the helper tools perform Fourier transformations fftw

Linux, FreeBSD and Mac OS

git clone https://github.com/ggerganov/kbd-audio
cd kbd-audio
git submodule update --init
mkdir build && cd build
cmake ..
make

Windows

(todo, PRs welcome)

Tools

Short summary of the available tools. If the status of the tool is not stable, expect problems and non-optimal results.

| Name | Type | Status | | --- | --- | --- | | record | text | stable | | record-full | text | stable | | play | text | stable | | play-full | text | stable | | view-gui | gui | stable | | view-full-gui | gui | stable | | keytap | text | stable | | keytap-gui | gui | stable | | keytap2 | text | development | | keytap2-gui | gui | stable | | - | extra | - | | guess_qp | text | experiment | | guess_qp2 | text | experiment | | key_detector | text | experiment | | scale | text | experiment | | subreak | text | experiment | | keyaveragegui | gui | experiment |

Tool details

• record-full

Record audio to a raw binary file on disk

  ./record-full output.kbd [-cN]

---

• play-full

Playback a recording captured via the record-full tool

  ./play-full input.kbd [-pN]

---

• record

Record audio only while typing. Useful for collecting training data for keytap

  ./record output.kbd [-cN] [-CN]

---

• play

Playback a recording created via the record tool

  ./play input.kbd [-pN]

---

• keytap

Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool.

  ./keytap input0.kbd [input1.kbd] [input2.kbd] ... [-cN] [-CN] [-pF] [-tF]

---

• keytap-gui

Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool. GUI version.

  ./keytap-gui input0.kbd [input1.kbd] [input2.kbd] ... [-cN] [-CN]

*Live demo *(WebAssembly threads required) **

---

• keytap2-gui record.kbd n-gram-dir [-pN] [-cN] [-CN]

Detect pressed keys via microphone audio capture. Uses statistical information (n-gram frequencies) about the language. No training data is required. The 'record.kbd' input file has to be generated via the record-full tool and contains the audio data that will be analyzed. The 'n-gram-dir' folder file has to contain n-gram probability files for the corresponding language.

  ./keytap2-gui record.kbd ../data

---

• view-full-gui

Visualize waveforms recorded with the record-full tool. Can also playback the audio data.

  ./view-full-gui input.kbd [-pN]

---

• view-gui

Visualize training data recorded with the record tool. Can also playback the audio data.

  ./view-gui input.kbd [-pN]

---

Feedback

Any feedback about the performance of the tools is highly appreciated. Please drop a comment here.