In 2019 alone, the ÖAMTC (Austrian Automobile Association) dispatched helicopters to fly over 2000 alpine search and rescue missions. The search and rescue of missing or injured persons often takes place in rough terrain and with air search. Whereas thermal cameras can detect differences between body heat and ambient temperature, using these types of cameras in forested areas can be difficult as trees and ground vegetation may be too dense. Manned search and rescue flights not only become challenging, but costly and time-consuming as well. In addition, these missions may often be unsuccessful or, in the worst-case scenario, succeed too late.

As technology becomes an increasingly important part of search and rescue, the first responder community will be involving more autonomous drones as these drones can aid in covering more ground as well as shortening search times. These drones, however, must be reliable and independently capable of finding individuals before a search team can be alerted to undertake a rescue mission. Whereas technological advances in the field of Artificial Intelligence are fundamental milestones, they do not solely solve the problem. 

Prof. Oliver Bimber and his team at the Institute for Computer Graphics at the JKU have introduced a globally unique drone prototype that is up to the task. Subjects in individual thermal images often appear completely or partially hidden; the drone instead combines several individual images into one integral image that can be used for classification and to better detect people. As the integral images significantly reduce concealment (see image), modern deep-learning methods can aid in correctly detecting individuals up to a probability of well over 90%. When using conventional individual images, the same procedure attains a recognition rate of less than 25%. As there is little to no data available to support this kind of classification, over the past few months, the team had to create its own training database. This database is now publicly accessible. Many JKU employees and students served as test subjects.

Initial field study findings and pivotal early insight indicate that when it comes to concealed objects, combined images significantly support classification purposes better than individual images. These findings have now been published in the renowned scientific journal Nature Machine Intelligence and will not only aid in search and rescue missions, but will also be used effectively in other areas, such as police and military surveillance, developing autonomous vehicles, and in wildlife observation, particularly in  support of conservation efforts. The team is currently working on advanced methods aimed at improving search speed and radius.

This basic research project is funded by the Austrian Science Fund (FWF) and the Upper Austrian government in collaboration with the Austrian Ministry of Education, Science and Research (BMBWF) via the Linz Institute of Technology (LIT) at the Johannes Kepler University Linz.