From cloning actors’ voices to detecting missiles: Ukraine’s AI scene contributes to air defense

“Zvook” (звук, Ukrainian for “sound”) is a project bringing together software and hardware components to identify cruise missiles, helicopters, drones, and jet fighters at low to medium heights. There are currently 40 instances of the system deployed throughout Ukraine. While operational details are scarce, the project’s team says that it helps the Ukrainian military locate and destroy Russian rockets targeting civilian infrastructure.

The system uses Machine Learning. Zvook was trained on a sound database containing the sound of cars, trucks, flies, cows, human voices, and Russian missiles that account for about 0.1% of the data. The sound data originate mainly from the internet, data on enemy aircraft noise (mostly of close rocket fly-bys) were recorded with the help of the military. The Zvook algorithm identifies the aircraft within the mass of non-threatening sound and notifies the military in such a case. Engineers constantly train the algorithm and add more sound data to the database.

Acoustic location

Hardware-wise, Zvook sensors consist of acoustic mirrors and a microphone. The mirrors focus the signal for the microphone to better discern the sounds. By comparing data from different sensors, Zvook can identify the type, as well as estimate a missile's speed and direction. The sensors are decentralized and operate as a network which accounts for a resistant effect and therefore make it harder to destroy. 

Zvook builds on acoustic detection systems from the 20th century. Acoustic location is as old as airborne warfare, starting in the 1920s. In an interview for Ekonomichna Pravda, a Ukrainian news outlet, the system's creators say that Zvook is a digital, automated version of those early acoustic locators.

They also say that they faced a “domain shift” issue. The sounds in their database weren't always the ones they needed and data scraped from the internet weren't always adequate. For instance, the sound of a missile flying far away and the sound of the same missile flying up close are completely different. Zvook asked the military for help in order to record the close rocket fly-bys and to check the sound quality.

The team had to work on the system’s accuracy. As any detection system, Zvook can fail in two ways. “False positives” happen when the system falsely identifies a sound as hostile, for instance a cow’s mooing as the sound of a missile’s engine (the team spent a month to get rid of this so-called “cow problem”). “False negatives” happen when the system identifies the sound of the real rocket as a non-threat. During initial tests, Zvook was correct in 50% of the cases, a coin’s toss. As training went on and the data quality improved, the results became good enough to assist in the country’s air defense. Zvook’s creators stated that they managed to bring false negatives to zero.

Respeecher’s shadow

The team behind the project is built around people from technology, telecommunication, and the military. Its founders are Dmytro Believtsov, co-founder of Respeecher, a world-(in)famous Ukrainian company that uses Artificial Intelligence to mimic celebrity voices, Pavlo Tsiupka, CEO of i3 Engineering, a Ukrainian company providing Smart Home solutions, and Marian Sulym, a serviceman from the 125th brigade of the territorial defense forces. (Territorial defense units consist of both reservists and civilians). Engineers from Respeecher also contribute to Zvook, as well as staff from Label Your Data, a Ukrainian data annotation company that usually works for global corporations.

The creators of Zvook may have their roots in business enterprises. But right now, their main focus is on protecting Ukraine.