From 28th to 30th of September 2022 the Small Scale Test #14 (SST#14) which took place at ERTZAINTZA’s training facilities in Brigada movil, Bilbao, Spain, focused on testing and validation of the INGENIOUS Smart Devices in the air and on the ground.

During SST#14, our partners in FOI demonstrated the Multi-purpose Autonomous eXploring drone’s – MAX (FOI) ability to autonomously explore and map out a building, while continuously sending data (e.g. visual and thermal images) to the Ground Control Station – GCS (ITC) for analysis and visualisation. Improvements to MAX since the last field trial include the ability to fly autonomously through openings, fusion of two independent positioning systems for improved robustness in different environments, adjusted control parameters for improved in-flight stability, and a function that re-plans a route around obstacles suddenly appearing in the planned flight path.

Our partners in ITC tested their Ground Control Station (GCS), which was designed to integrate data among the different drones, and process image data to provide First Responders (FR) with enhanced situational awareness. ITC’s researchers used AI algorithms to generate semantic maps of a scene. Objects or building structures of interest to the FR were marked in different colors. GCS also ran a detection algorithm on RGB images to help FR find victims.

In addition to image analysis functionalities, GCS also tested the data communication with Fusion Engine – FE (EXUS) and Common Operational Picture – COP (STWS). Each component had its own unique ID, and was pre-registered in COP. They were co-registered using the global coordinate system built by the Modular Aerial Camera System – Search and Rescue – MACS-SaR (DLR), and their geo-referenced data, such as positions, were sent to FE through GCS, and visualised on COP. Owning to GCS’s device management function, COP could know from which component the received data came. Besides, GCS was able to transmit complete point clouds file generated by MAX and Integrated Positioning System (IPS) to COP by SFTP (SSH File Transfer Protocol).

The INGENIOUS Micro Indoor drones’ – MINS (SINTEF) functionalities were also demonstrated during SST#14. While our partners in SINTEF could show rudimental deployment and localisation service using Ultra-Wideband technology of our MIN drones in SST#5 in Bilbao they used the time since then to improve the stability of both localisation and the flight of the drones during deployment. During LIT#23 in Berlin they were able to demonstrate these improvements.

As technical challenges to the core system seemed to be overcome, they focused on improving the usability of the overall system prior to SST#14. While in previous tests the base-station setup with references for their initial positioning was cumbersome by having to manually measure distances and taping drones as anchors to the table, a procedure that easily took half an hour, all the components are now within a ruggedized box where anchors can be unfolded into known positions within a minute or two.

SINTEF developed a small user interface that allows to see the status of all components, relay them out to be displayed on the COP and allows to configure drones and FR tags automatically as soon as they start communicating with the swarm. Previously, manual commands had to be entered into a terminal prior to taking off for each drone. A cradle to hold the MINs before their startup to charge them and switch them on has been prototypically worked on as well. SINTEF also wanted to show their automatic waypoint generation. It takes a map of the environment provided by MAX and calculates where to best place the next MIN drone considering area coverage and communication-range limitations.

Unfortunately, these improvements were difficult to show off, as SINTEF’s base functionality (localisation) showed unexpected problems which made flying and autonomous deployment difficult. The remaining time for the 2^nd Full Scale eXercise (2^nd FSX) was therefore used to make the localization functionality more robust.

During SST#14 the Integrated Positioning System – IPS of DLR was also tested to transmit the carrier’s position and user-triggered images to the COP in realtime. Moreover, immediately after returning IPS was expected to transmit a point cloud of the seen objects to the COP.

First, IPS was referenced with the global coordinates of the AprilTags provided by MACS-SaR in front of the main hall. The coordinates were automatically extracted from the Aerial images and IPS simply targets the AprilTags (see Image below) to coregister its coordinates, as already shown in previous tests.

After coregistering, the test person walked through the main hall and the southern exercise room and took some took photos. For the first time not only the IPS’ position but also the user-triggered images appeared in real-time in the COP (see Figure 10: The trajectory is the coloured curvy line and the user-triggered images are shown as photo-symbols at the corresponding locations. The user can click on one of these photos to view its content). In addition, the point cloud recorded by IPS was displayed in the COP for the first time (coloured point cloud in Figure 11).

An additional extensive walk around the main hall and the southern exercise room was performed to demonstrate the long-term stability of IPS’s accuracy without intermediate GPS reception or other external means of localization. The resulting trajectory (red) and point cloud (white) are shown in Figure Z in a top-view.

Field Communications

Field Communication, a component provided by ICCS, builds a stable field communication system to support onsite data transmission for the INGENIOUS Next Generation Integrated Toolkit (NGIT).  The system provided wireless connection to the tools of the kit. Furthermore, wireless repeaters were introduced in the system and were successfully tested for the first time. Two User Interfaces were presented to the end users. The first presented a list of the components of the system along with their status and relevant information and the second was traffic graphs for selected interfaces. The system generated alert messages for device connection/disconnection or when devices had low RSSI. In addition, a message with a list of the connected devices and their RSSIs was produced as well.

End Users experience

Our partners in HRTA were the responsible End Users to test and validate the INGENIOUS technologies for SST#14 in real conditions. HRTA was briefed extensively on the setup and current functional level of drones (MAX, MIN, MACS-SaR), Indoor Positioning System (IPS) and Field Communications package. A set of tests were conducted with the MAX drone in fully automatic navigation mode, demonstrating its capability to autonomously scan, analyse, plan and progress through an indoor environment for dynamic mapping and victim detection.

The impression the FRs gained throughout these field tests is that the overall development of these technologies is on track, although it needs further development and some minor changes in order to be operationally usable in real-life SAR deployments. The next field activity of the project was the second Full Scale eXercise (2^nd FSX), in November 2022.