From 11 to 15 October 2021, our SBFF partners hosted three Small Scale Tests (SSTs), SST#1, SST#3 and SST#9 of INGENIOUS, in collaboration with the Swedish Civil Contingencies Agency (MSB) at their main training field in the south of Sweden.

INGENIOUS is developing a broad spectrum of tools and the partners had the opportunity to demonstrate six of them with seasoned first responders as users at relevant operation environment. The tools tested are:

• Real time translation app
• Augmented reality platform
• Common operational picture platform
• Pressure sensitive insole for boots
• Helmet
• Personal area network hub in uniform

Overall set up

The significance of the SSTs for the project was that this was one of the first times due to Covid-19 that end users and technical partners could physically meet, discuss and evaluate the current progress of the tools and also test them in a real scenario environment.

Using MSBs real life testbed concept with access to all needed facilities at site was a winning concept. The training field and rooms could be adjusted to the specific needs with both indoor and outdoor testing areas at our disposal.

To isolate use cases and functionalities, not all tools were tested at the same time, however, most of the partners did stay for the whole week to discuss and examine future steps for the complete integration of the INGENIOUS system. This also gave a good opportunity to get to know one another over social activities such as working out in the gym, getting to try smoke diving or simply over a beer in the bar watching football qualifications, where Sweden and Greece happen to play.

• 

• 

• 

• 

The focus was given on interaction even during actual work and testing, with a clear and adjusted daily schedule with dedicated end users for each tool. The tests prioritized flexibility and adaptability in order to allow the INGENIOUS partners to dig deeper into critical aspects of use case scenarios and how their tools could improve the work and operations of the First Responders. This was very appreciated as it made it easier to actually meet and discuss all the details needed from the end user perspective both in terms of understanding the tool and then testing it.

As one technical partners said: “One of the best things during the week was that we got to be in your shoes and try to do things with your uniform on. Now we understand that it might be hard to work with a smartphone app or small buttons with gloves on, etc.”

SST#1 Helmet & ARRP

During SST#1 two of the INGENIOUS tools were demonstrated, the Smart Helmet and the Augmented Reality Response Platform (ARRP). The INGENIOUS Helmet is being developed by our partners from KIRO to facilitate the collection of field information to assist First Responders (FRs) in emergency situations. Two INGENIOUS helmets were used during SST#1 for a demonstration based on the scenario of a human life search using a helmet in a heavy smoke environment and in a dark environment. In the first case, the surrounding situation was identified using the IR camera image of the helmet and the person lost in the smoke was found relatively easily. In the second test, the video data generated from the helmets were sent to the Command Center and voice communication among First Responders and commanders was achieved. As in the first case, the human search was successful. The results of the tests have shown that both helmets were functional under these situations and the tests were completed successfully. However, the need to improve some of the helmet parts in order to make it fully functional for use in the field was highlighted.

The INGENIOUS ARRP is being developed by our CS-GROUP partner to enhance situational awareness and provide operational assistance to field units, as it offers new possibilities in terms of information display and contextual communication. One of the main achievements of SST#1 was the provision of a functional prototype of the ARRP platform connected with the Fusion Engine (FE) and the Common Operational Picture Platform (COP), which was tested indoor and outdoor by the end users.

FRs considered it easy to use after a first stage of training (to get used to the display and interaction proposed by the AR devices). They also found the functionalities proposed by the AR application very useful, such as seeing the teammates’ position or having the environment annotated (moreover in a big building or wide area). Important and useful feedback for improvements was also received such as; holograms need to be smaller and more transparent. There were also discussions about useful functionalities, such as the capability to create an hologram path to show the right way to the next group, or being able to position annotations more precisely (on the ground, on the ceiling…) or on an area, not just as a point.

In addition to the progress made in the development of the ARRP, a technological barrier was identified during the SST and is linked to the AR devices themselves, as they are not ready for operational use and are sensitive to environmental conditions. Though it was already known, it was a good opportunity for the end users to experience the limitations during the SST. What is challenging for the INGENIOUS partners is to provide end users with a functional, versatile component, to demonstrate the AR features in scenarios close enough to their use cases.

SST#3 Uniform & Boot Insoles

SST#3 was dedicated to the testing of the INGENIOUS Uniform and Smart Boot Insoles developed by our TEKNIKER and CyRIC partners respectively.

These two tools provide a range of benefits and capabilities for the FRs that have been tested and evaluated for their technical performance and effectiveness during the SST:

1. Individual FR’s health status remote monitoring: Thanks to miniaturized electronics units integrated within the boots’ insoles, raw data coming from distributed force sensors and inertial movements are wirelessly collected and locally processed at the uniform. Specific activities are automatically identified, and the corresponding event is signaled to the central coordination center where an alert can be triggered considering also aggregated inputs.
2. Team leader notification management: Team leader can receive visual and sound notification if one of the team members could be in danger. This early alert enables fast response time.  Once received an acknowledgment is sent back to the central coordination to be fully synchronized.
3. End user real hands on experience: Close interaction with FR in a real scenario has invaluable in assessing the current prototype solutions against user requirements. During the SST multiple sessions were flexibly arranged among FR and tech partners covering initial explanation, training and real on field experience including tool cycle: device registration, deployment, status monitoring, after mission procedures, etc. Received have been analyzed and included in the next iteration cycle.
4. Testing under near real conditions: The Smart Boot Insole was tested during the SST under hot and cold conditions and had an excellent performance. The detection of FR hanging, immobilized and carrying heavy load events was possible.

• 

• 

• 

• 

SST#9 COP Platform & C3 System & Multilanguage Operations App

The SST#9 included validation and evaluation of the Common Operational Picture and Command, Control and Coordination System (COP Platform & C3 System) developed by SATWAYS (STWS) and the Multilanguage Operations App developed by the Natural Language Processing (TALN) Group of the Pompeu Fabra University (UPF). Testing of the COP & C3 System gave significant insights into the standard operation procedures of different agencies. During the SST#9, SATWAYS had the opportunity to learn more about Standard Operating Procedures (SOPs) which vary among public safety organizations of different or even the same discipline (e.g., the agencies of the attended end-users SBFF, ERTZ, ISAR).  The functionality of the COP system was limited due to the fact that it is dependent on data received from the rest of the components within the INGENIOUS NGIT (Next Generation Integrated Toolkit), many of which were not present during these tests. INGENIOUS development and next steps will further enhance the COP as the NGIT becomes fully integrated. However, during the further development of the INGENIOUS Toolkit new unseen issues may arise, thus, testing of the component as part of the fully integrated system and in more realistic full-scale scenarios is still needed to ensure that it fulfills the expected objectives in the future tests.

The next steps towards the development of the next version of the COP Platform & C3 System will include:  a) Analysis and incorporation of the feedback received during the SST, b) Finalization of the Multi-Agency collaboration capabilities of the system and c)  integration with more NGIT components to enhance the common operational picture depiction with receipt and visualization of i)Status and Position of Resources  ii)  Situational Reports  iii) Live Video Streaming  and iv) Drone Data (Images, Point-Clouds, LSOPs etc.).

• 

• 

• 

The Multilingual Operations App (MOA) aims to facilitate the communication between international First Responder teams (FRs) and the Command Center through the COP/C3 platform. The MOA does speech-to-speech translation, enabling the FRs to send voice messages in their own language that will be transcribed, translated to English and received in the COP/C3 as text and/or voice messages. SST#9 was the first opportunity to work in person with end users to test the functionality of the MOA. In this phase of the project, the bot was connected with the COP through the Kafka bus used throughout the INGENIOUS project for communication between components. This allows operators at the COP to have access to a log of conversations on the field, along with the corresponding transcripts and translations. Management of users and teams is handled by the COP. German and Swedish were added as input languages. This version was tested during the SST #9 in Sweden, with partners from different countries representing all four input languages participating in the test. The interaction with end users from different backgrounds was very helpful in getting much more immediate feedback on the potential as well as the current limitations of the Multilingual Operations App.

Conclusions

An important part of the project is the dissemination and communication of the activities, progress and achievements of the consortium. In line with this, one of the days was dedicated for executives from project partners and external stakeholders to be at site. During the day they had presentations and discussions on the need for innovation projects for First Responders and also an opportunity to get a firsthand experience of the tools. Since the tests were held in Sweden most of the stakeholders invited were from Sweden, with both regional and national representatives. The spin off effect from this event has paid off with new and intensified discussions on H2020 projects and other innovation projects, especially with the newly started national development center for rescue services in Sweden.

From the test week a lot of experience was gained, not only for the specific tools and continuous development but also on how to organize future tests as INGENIOUS has set up an ambitious plan with many iterations of testing. With two more rounds of small Scale Tests and Laboratory Integration Tests and also two Full Scale Excercises to be conducted before the project is over in 2023.

Most importantly, during the SSTs the technical partners received relevant and adequate feedback from the end users, which they can use collaboratively in the next round of prototypes development.

If you want to know more about what the intention of each tool is, please read a more in depth description from the INGENIOUS Leaflet for Stakeholders. Click also here to learn more about MSBs real life testbed.

 Watch the video from the SSTs #1, #3 and #9 to find out more: